Advanced Lane Finding

Camera Calibration

The camera calibration was made using the chessboard images along with the opencv functions presented in the course material.

cv2.findChessboardCorners() failed to find corners in the following images:

Unable to find corners in: ../camera_cal/calibration1.jpg
Unable to find corners in: ../camera_cal/calibration4.jpg
Unable to find corners in: ../camera_cal/calibration5.jpg

calibration4.jpg has one missing corner, the remaining ones probably have the chessboard overly cropped.

Undistortion examples:

Pipeline (images)

Filter to binary

filter.py is the module used to filter the images in order to identify lane-lines.

The filter used is a combination of gradient (using sobel) and color (using saturation channel).

Sobel filter: (X & Y) | (DIR & MAG) Pretty much what was covered on the course. The thresholds are based on the mean values of each sobel component.

Saturation filter: s_channel > low_threshold The threshold is simply the mean of the saturation values of the half bottom of the image (where the road is usually located) multiplied by constant. This helps to adapt to different frames with different exposure levels.

- Sobel filter:

- Saturation filter:

- Combined filter:

---

Perspective Transform

Class Warper is used to perform the perspective transform, the image with straight lines was used to define a source polygon (green in the image below) that would match a straight line from a bird's-view perspective.

Note: The polygon is hardcoded, so it doesn't adapt to road slope, that's very noticeable on the challenge/harder videos and a limitation.

Finding Lane-lines

line_fit.pyis used to identify the pixels belonging to the left and right lane, if there's no previous detection a sliding windows approach is used with an histogram to idenfity the first windows.

If there's previous detections the area around the last lines are used to search for the pixels.

Curvature and position

measure_curvature_real() is used to calculate the curvature of each lane-line, based on the calculated polynomials.

ego_distance_from_center() calculates the distance from the center of the image to the center of the lane (at the very bottom of the image).

The polynomials are adjusted to meters from pixels based on the ratios provided in the course.

The values displayed on the video are averaged out accross several frames to avoid sudden jumps.

Lane identification

Final result with the lane space fitted in the original image:

Pipeline (video)

Aditional logic to validate the detection of lines was added, based on the normalized residuals of the polynomial fit.

If the algorithm fails to detect lines for several frames the sliding_window approach is used instead. If there's a successfull detection then the area around the last detection is used.

Links to the result videos:

project_video_output.mp4

project_video_output_debug.m4v (2x2 panel with the different stages of the pipeline)

Discussion (video)

Shortcomings:

• The warped process is hardcoded, sometimes when the car hits a bump or the road slope increases/descreases the lines don't appear straight anymore. Also, sometimes the lines of interest are out of the warped region.

• Detection of the lines could be done in separate for the left and right, currently if the fitting fails for one it counts as an invalid detection.

• A better technique for evaluating the detection of lines could be used, instead of the residuals of the polyfit. Similarity with the previous detected lines could be an alternative.

• Dashed lines are sometimes a problem because the fitting is strongly affected by single pixels on the extremes of the image when there's no lane markings on it.

• Averaging of the detected lines could be done to smooth some jumps in the video, however, averaging 2nd degree polynomials is not trivial.

• The algorithm struggles when there's multiple lines close to the center, for example road boundaries, separation walls, and so on. More robust logic should be used to identify the car's lane.