Genetic algorithm and Backpropagation in autonomous cars
Autonomous cars need learning in order to recognize their environment and behave consistently. Neural networks give us the opportunity to develop complex behaviors and tasks like driving.
I have implemented two algorithms that are widely used in machine learning: backpropagation (supervised learning) and genetic algorithm (reinforcement learning).
Supervised Learning (Backpropagation)
This is the most used algorithm and compared to the rest, this is the most powerful and needs fewer resources to be performed. However, there is a disadvantage, we need training data for the learning process and thus is really restrictive.
Reinforcement Learning (Genetic Algorithm)
It is an algorithm that was inspired by the theory of evolution by Charles Darwin.
The implementation simulates the process of natural selection where the fittest individuals have higher probabilities to transfer their genes to the next generation. It is usually divided into 4 parts.
Genes frequently use Boolean representation (0 or 1). However, here we use a real number in between -1 and 1 to depict the gene value. We now need to design the neural net topology, for example: if I use 5 sticks to measure the distance to the nearest object and we want to return acceleration, right and left turn, this represents 5 inputs and 3 outputs, and you can change the hidden layers topology. We have used 4 layers in our experiments. Then, to initialize the weight values one can select a random number, in the interval [-1..1].
This process orders individuals from each generation, according to their fitness. The
fitness function returns a probability to transmit ”genes” to the next generation.
There are different variants of this step. I have designed a function that uses 2 variables: accuracy and diversity. Genetic algorithm, like others, have a paramount problem while you are trying to reach the global minimum, it usually comes to a standstill in a local minimum. To solve this problem, I have used a measure of the distance from one individual to the rest of the individuals.
The accuracy can be measured using the positioning system that can tell us the percentage of the circuit traveled, by the individual. Once we have the accuracy and the diversity of each individual, we can transform absolute data to relative. The result will be given with a real number between 0 and 1.
The elements we have chosen will experiment with the crossover operation. It is the process of exchanging genes among parents to reach a new individual made by progenitors. This part can be modified as you may require. It will be possible to design it with some hyper-parameters: crossover rate, number of progenitors.
We change some of the genes with a random number generator. By the fact that if decimal numbers, we need some hyper-parameters: mutation rate, mutation change.