Category: Artificial Intelligence

AI + GPU

Deep Neural Network in GPU

I have been developing a neural network API for GPUs. It is based on CUDA technology and I tried to exploit the full potential of the computer.

Most of us, do not know the real potential of GPUs in AI. This kind of hardware is specialized in logical operations and it was used by the industry of engines and videogames. Some years ago, we have experienced a ‘boom’ in graphics cards, the improvement in speed, price and the ease of programming had triggered a new mother-lode.

GPUs provide a bunch of parallelism algorithms to improve performance.

As we know, neural networks use matrix-to-matrix and matrix-to-vector operations. We can take advance in the optimization of them. Unfortunately, feed-forward and backpropagation are linear functions, they can neither be optimized nor parallelized.

    \[Output=\sigma(W*\sigma(...\sigma(W*Input+Bias)...)+Bias)\]

Continue reading “AI + GPU”

RNN, LSTM & GRU

Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM) & Gated Recurrent Unit (GRU)

Is a type of artificial neural network where connections between nodes form a sequence. This allows temporal dynamic behavior for time sequence.
There are 3 types of vanilla recurrent neural network: the simple (RNN), gated recurrent unit (GRU) and long short term memory unit (LSTM).

Continue reading “RNN, LSTM & GRU”

Autonomous Car – Learning Algorithms

Genetic algorithm and Backpropagation in autonomous cars

Autonomous cars need learning in order to recognize their environment and behave consistently. Neural networks give us the opportunity to develop complex behaviors and tasks like driving.

I have implemented two algorithms that are widely used in machine learning: backpropagation (supervised learning) and genetic algorithm (reinforcement learning).

Continue reading “Autonomous Car – Learning Algorithms”

LiDAR: Autonomous Car

LiDAR implementation in self-driving cars

LiDAR (Light Imaging, Direction And Ranging) is a method that measures the distance to a target by illuminating the target with pulsed laser light and measuring the reflected pulses with a sensor. Autonomous cars take advantage of this new technology.

Continue reading “LiDAR: Autonomous Car”

Convolutional Neural Network (CNN)

Convolutional neural network introduction and tutorial

Introduction

Convolutional neural network (CNN or ConvNet) is a type of neural network used in artificial intelligence that is commonly applied to analyzing images.

They can be considered a pre-processing compared to image classification algorithms. They have applications in image and video recognition, recommender systems, image classification, natural language processing, etc.

Notation

f^l : filter size.
p^l : padding.
s^l : stride.
n^{l-1}_H \times n^{l-1}_W \times n^{l-1}_C: input.
n^{l}_H \times n^{l}_W \times n^{l}_C: output.
n^l = \floor{\frac{n^{l-1}+2p^l-f^l}{s^l}+1}.
f^l \times f^l \times n_c^l : filter.
a^l : activation function.
n_c^l : bias.

Continue reading “Convolutional Neural Network (CNN)”