Convolutional Neural Network (CNN)

Convolutional neural network introduction and tutorial

Introduction

Convolutional neural network (CNN or ConvNet) is a type of neural network used in artificial intelligence that is commonly applied to analyzing images.

They can be considered a pre-processing compared to image classification algorithms. They have applications in image and video recognition, recommender systems, image classification, natural language processing, etc.

Notation

f^l : filter size.
p^l : padding.
s^l : stride.
n^{l-1}_H \times n^{l-1}_W \times n^{l-1}_C: input.
n^{l}_H \times n^{l}_W \times n^{l}_C: output.
n^l = \floor{\frac{n^{l-1}+2p^l-f^l}{s^l}+1}.
f^l \times f^l \times n_c^l : filter.
a^l : activation function.
n_c^l : bias.

Layers


Convolutional

The convolution operation is the most representative. The convolutional operation is the Hadamard/Element-wise product.

    \[ A^{'}=A\circ B \]

We can rewrite our operation using our notation:

    \[ A_{i j}^l=\sum_{i=1}^{f^l}{\sum_{j=1}^{f^l}{{A_{i j}^{l-1}B_{i j}^l}}} \]

where B is the matrix output and A is a subset of the input matrix which first element is i j and the same size of B, the filter.

Pooling

The pooling layer combines the output of neuron clusters at one single neuron, it is used to reduce the size of the image.
Pooling layer needs two hyper-parameter: stride and filter size
There are two kinds of pooling layers: \textit{max pooling} and \textit{average pooling}.

  • Max pooling
    Outputs the highest value of the cluster.
  • Average pooling
    Outputs the average of the cluster.

ReLU

ReLU is the abbreviation of rectified linear unit. f(x)=max(0,x).
Consequently, it removes the negative values.

Residual

This neural network is based on the cerebral cortex. Its characteristic feature is that it jumps over some layers.

Fully connected

Fully connected layers connect every neuron in one layer to every neuron in another layer. It is in principle the same as the traditional multilayer perceptron neural network (MLP).

Multilayer Perceptron Tutorial


Link

Convolutional Neural Network paper

14 thoughts on “Convolutional Neural Network (CNN)”

  1. Wow! This blog looks exactly like my old one! It’s on a completely different topic but it has pretty much the same layout and design. Superb choice of colors!

  2. Hello There. I found your blog using msn. This is a really well written article. I will make sure to bookmark it and return to read more of your useful info. Thanks for the post. I will definitely comeback.

  3. Way cool! Some extremely valid points! I appreciate you writing
    this article plus the rest of the website is very good.

  4. Hello, outstanding website! Does running a blog like this require a massive amount work? I have absolutely no understanding of programming
    however, I was hoping to start my own blog in the near future.
    Anyway, if you have any ideas or tips for new blog owners please share.
    I know this is off topic nevertheless, I just needed to ask.
    Thank you!

  5. I wanted to thank you for this excellent read!! I absolutely loved every bit of it. I have got you bookmarked to look at new stuff you post…

  6. I need to thank you for this excellent read!! I certainly loved every little bit of it. I have you book-marked to check out new things you post…

Leave a Reply

Your email address will not be published. Required fields are marked *