RNN, LSTM & GRU

Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM) & Gated Recurrent Unit (GRU)

Is a type of artificial neural network where connections between nodes form a sequence. This allows temporal dynamic behavior for time sequence.
There are 3 types of vanilla recurrent neural network: the simple (RNN), gated recurrent unit (GRU) and long short term memory unit (LSTM).

Continue reading “RNN, LSTM & GRU”

Convolutional Neural Network (CNN)

Convolutional neural network introduction and tutorial

Introduction

Convolutional neural network (CNN or ConvNet) is a type of neural network used in artificial intelligence that is commonly applied to analyzing images.

They can be considered a pre-processing compared to image classification algorithms. They have applications in image and video recognition, recommender systems, image classification, natural language processing, etc.

Notation

f^l : filter size.
p^l : padding.
s^l : stride.
n^{l-1}_H \times n^{l-1}_W \times n^{l-1}_C: input.
n^{l}_H \times n^{l}_W \times n^{l}_C: output.
n^l = \floor{\frac{n^{l-1}+2p^l-f^l}{s^l}+1}.
f^l \times f^l \times n_c^l : filter.
a^l : activation function.
n_c^l : bias.

Continue reading “Convolutional Neural Network (CNN)”

Self Driving Car in Unreal Engine 4

Autonomous car: learning and testing process.

Description

Pages: 25

Author: Daniel López Montero

Abstract

AI is a new field that appeared a few decades ago and has seen unprecedented growth. The AI approach has been used in numerous fields, such as finance, medicine, music, customer service, and transportation. Some of the big challenges we have to cope with are the computing power and lack of documentation. Big companies such as Google and Apple do not share advances in this field, the secretism characterizes this scientific field and thus the information we can get is very restrictive. In this work, we introduce a model of an autonomous car and then examine different algorithms capable of driving the car in a simulated. We also discuss the difficulties we need to deal with such as local minimums, losing diversity, fitness functions, localization, performance, and future applications.

Keywords

Artificial intelligence, neural network, genetic algorithm, backpropagation, self-driving car.

Continue reading “Self Driving Car in Unreal Engine 4”

Gradient Descent Algorithm

Mini-batch, Stochastic & Batch Gradient Descent

There are 3 gradient descent algorithms used for backpropagation in neural networks: stochastic, batch and mini-batch.

    1. L_i: loss function (calculate error)
    1. y_i: neural network output (predicted data)
    1. x_i: real output (train label)

Continue reading “Gradient Descent Algorithm”

3D Neural Network Simulator

Neural Network implementation in UE4

Our neural network implementation in Unreal Engine 4 can have different purposes, educational or as a tool for data scientist developers.

This tool is really easy to be used: you can change the number of layers and the neurons per layer. In the example showed I have used the MNIST dataset for number recognition, furthermore, I can use different datasets.

If you want to use this tool for educational purposes, you can debug and see how the neural networks work. It is really practical and I recommend whoever wants to start AI. This application is progress, although you would be able to see the advances.

Continue reading “3D Neural Network Simulator”