Batch_Gradient_descent_vs_Mini_Gradient_Descent
Gradient descent is a first-order iterative optimization algorithm for finding the minimum of a function.In Batch Gradient Descent, all the training data is taken into consideration to take a single step. We take the average of the gradients of all the training examples and then use that mean gradient to update our parameters. So that’s just one step of gradient descent in one epoch.In Batch Gradient Descent we were considering all the examples for every step of Gradient Descent. But what if our dataset is very huge. Deep learning models crave for data. The more the data the more chances of a model to be good.SGD can be used for larger datasets. It converges faster when the dataset is large as it causes updates to the parameters more frequently.the average cost over the epochs in mini-batch gradient descent fluctuates because we are averaging a small number of examples at a time.
