Deep Learning for Autonomous Driving
As part of a Special Topics class during my master's degree, I had the opportunity to take part in a group project under the supervision of Cédric Pradalier from The Dream Lab at Georgia Tech Europe. Our task was to create a deep learning-based algorithm for an RC car, to make it drive autonomously on the track around the nearby lake.
The RC car was controlled using ROS (Robot Operating System) and was equipped with a front camera. The proposed solution was to use image segmentation to differentiate the road from the rest of the image, run an object detection model, and combine this information as input to a final model that controlled the car's throttle and steering.
I was responsible for working on the image segmentation part. We chose the MobileNetV2 model, as it offered a good trade-off between performance and accuracy. This model was designed specifically for mobile devices by using techniques like depthwise separable convolutions. Instead of performing regular convolutions, it first applies a depthwise convolution on each input channel individually, followed by a 1x1 pointwise accross all channels (building features through linear combinations of the input channels). This dramatically reduces computational cost while maintaining competitive performance.
To build our testing dataset, we drove the RC car for a lap around the lake, recording both the controller inputs and images from the onboard camera.
For the initial training phase, we used the Cityscapes dataset, which provides images captured from the hood of real cars along with their semantic segmentation annotations.
I first tested the pre-trained model directly on our collected images, but as the Cityscapes dataset differed significantly from our environment, the results were not optimal. To improve performance, we retrained the model on Cityscapes while keeping only two labels: road and other. By simplifying the task, the model became more focused on identifying the track and was less distracted by irrelevant objects. This approach led to slightly improved results on our testing images.
Unfortunately, due to COVID-19 lockdowns, we were not able to test the complete pipeline on the actual RC car. Still, this project allowed me to explore some deep learning applications in computer vision and get my first hands-on experience with TensorFlow.