Featured

My favorite posts.

Robust Lane Tracking

This summer I’ve been taking some time off and working on a few side projects. The most interesting stuff has come from Udacity’s Self-Driving Car class, which has been a wonderful way to build on my experience and exercise what I already learned from my robotics experience at Duke (I just graduated some months ago!).

Today I’m going to talk about the “advanced” lane following project. The goals Udacity set up were direct, but certainly open-ended.

Given a video:

  • Highlight the current lane the car is in,
  • Determine the position of the car in the lane,
  • Determine the lane’s radius of curvature.

These goals emulate some of the core information you would need for Level 2 automation, such as lane centering, drift alerts, and highway lane following, all of which we can see in existing production cars. Of course, tracking lanes is also used in higher levels of autonomy, but is paired with a ton more information to make it robust to more situations. For this project, there is no LIDAR, no high-resolution maps with known lane information, no GPS, no inertial data. All we are using is video.

If you’re looking for the code, head on over to the GitHub repository.

Continue reading “Robust Lane Tracking” →

CSI Cameras on the TX2 (The Easy Way)

I love Nvidia’s new embedded computers. The Nvidia Jetson embedded computing product line, including the TK1, TX1, and TX2, are a series of small computers made to smoothly run software for computer vision, neural networks, and artificial intelligence without using tons of energy. Better yet, their developer kits can be used as excellent single board computers, so if you’ve ever wished for a beefed up Raspberry Pi, this is what you are looking for. I personally use the Jetson TX2, which is the most powerful module available and is widely used.

One of the big fallbacks with Jetson devices is that the documentation does not (and cannot) cover all use cases. The community has yet to mature to the point where you can find some random blog’s guide on any random thing you need to do (à la Raspberry Pi and Arduino), so you’ll often have to figure out things for yourself.

But, I am here to dispell the mystery around at least one thing — using CSI cameras on your TX2. These methods should work on other Jetson devices too!

We’re going to look at utilizing the Jetson’s image processing powers and capturing video from the TX2’s own special CSI camera port. Specifically, I’ll show you:

  • Why you’d even want a CSI camera.
  • Where to get a good CSI camera.
  • How to get high resolution, high framerate video off your CSI cameras using gstreamer and the Nvidia multimedia pipeline.
  • How to use that video in OpenCV and ROS.

Continue reading “CSI Cameras on the TX2 (The Easy Way)” →