There are 3 gradient descent algorithms used for backpropagation in neural networks: **stochastic**, **batch** and **mini-batch.**

- : loss function (calculate error)

- : neural network output (predicted data)

- : real output (train label)

- Stochastic gradient descent: descent turn out to be faster than batch, so is widely used. However, it has low accuracy and you need more iterations to get to the minimum.

Loss function:The derivative of Loss function:

- Batch gradient descent: As we need to calculate the gradient on the whole dataset to perform just one update, batch gradient descent can be very slow and is intractable for datasets that don’t fit in memory.

Loss function:Derivative of loss function

- Mini-batch gradient descent: is the most favorable and widely used algorithm that makes precise and faster results using a batch of ‘ m ‘ training examples. In mini batch algorithm rather than using the complete data set, in every iteration we use a set of ‘m’ training examples called batch to compute the gradient of the cost function.

Loss function:Derivative of loss function

Really cool!

You’re actually compelling. May I ask a few questions? Because..I can think of additional points that synergize with this.

Yes, of course. Tell me what are your questions

I don’t know about the listicles.. seems somewhat overdone.

I’d perpetually want to be update on new articles on this website , saved to favorites! .

Very nice post. I just stumbled upon your weblog and wanted to say that I’ve really enjoyed surfing around your blog posts. In any case I’ll be subscribing to your rss feed and I hope you write again soon!

With thanks! Valuable information!

Excellent blog here!