Behavioral Cloning (Code)
The goal of this project is to use deep neural networks and convolutional neural networks to clone driving behavior.
The steps followed are:
- Use the simulator to collect data of good driving behavior
- Build a convolution neural network in Keras that predicts steering angles from images
- Train and validate the model with a training and validation set
- Test that the model successfully drives around track one without leaving the road
The project is developed using Python and Keras. You can download the full code from GitHub.
Model Architecture and Training Strategy
My model consists of a convolution neural network inspired by the architecture published by the Nvidia team:
model = Sequential()
model.add(Lambda(lambda x: x / 127.5 - 1., input_shape=(160,320,3)))
model.add(Cropping2D(cropping=((70,25),(0,0))))
model.add(Conv2D(24, (5, 5), activation="relu", strides=(2, 2)))
model.add(Conv2D(36, (5,5), activation='relu', strides=(2, 2)))
model.add(Conv2D(48, (5,5), activation='relu', strides=(2, 2)))
model.add(Conv2D(64, (3,3),activation='relu'))
model.add(Conv2D(64, (3,3),activation='relu'))
model.add(Flatten())
model.add(Dense(100))
model.add(Dropout(0.3))
model.add(Dense(50))
model.add(Dropout(0.3))
model.add(Dense(10))
model.add(Dropout(0.3))
model.add(Dense(1)) # I only want to predict steering angle
The model includes RELU layers throughout the convolutional layers to introduce nonlinearity, and the data is normalized in the model using a Keras lambda layer.
Here is a visualization of the architecture of the Nvidia Team:
Attempts to reduce overfitting in the model
The model contains dropout layers between the final three fully connected layers to combat overfitting:
model.add(Dropout(0.3))
The model was trained and validated on different data sets to ensure that it was not overfitting and it was tested by running it through the simulator and ensuring that the vehicle could stay on the track.
Model parameter tuning
The model used an Adam Optimizer, so the learning rate was not tuned manually. Since this is a regression network instead of a classification network I used mean square error as loss function:
model.compile(loss='mse', optimizer='adam')
Creation of the Training Set & Training Process
To capture good driving behavior, I first recorded two laps on track one using center lane driving. Here is an example image of center lane driving:
I then recorded the vehicle recovering from the left side and right side of the road back to center so that the vehicle would learn to steer properly and get back on track. A little trick here is to delete the data entries where the steering value is zero, because it contains the action of driving the car along the sideline.
These images show what a recovery looks like:
To help the model generalize better I recorded the vehicle driving counter-clockwise
Data augmentation can help generate more points in the “feature” space and make the trained model more robust. To augment the data set, I also flipped the images horizontally and inverted the angles: this would teach the car to steer clockwise and counter-clockwise.
I used the side car images for training too. This carries two benefits:
- it’s three times as much data
- using these images will teach the vehicle to steer towards the center if it starts drifting off towards the sides
Since the steering angle associated to the side cameras is the same as the one in the center even though the image appears off to one side of the road, I applied a correction factor of +- 0.2 to the relative steering angle.
After the collection process, I had 30492 data points. I then preprocessed this data by removing the top 70 pixels that contains the landscape and the bottom 25 pixels that contains the car hood.
Possible improvements
- Run the network for a higher number of epochs (if you have a big enough GPU)