Epoch Questions

Deep Learning, Week 4: every "how many weight updates" question from the PYQs (Quiz-1 Sep'24 to Jan'26) and the Practice Assignment, solved step by step.

The only formula you need

$N$ = number of training samples, $B$ = batch size, $E$ = number of epochs. One epoch is one full pass over the data, and one step is one parameter update.

AlgorithmUpdates per epochTotal updates
Vanilla / Batch GD$1$$E$
Stochastic GD (SGD)$N$$N \times E$
Mini-batch GD$N / B$$\dfrac{N}{B} \times E$
Batch GD + line search$1$$E$

Questions

Q1Quiz-1 · Sep 2024MCQ · 3 marksMini-batch

A team has a data set that contains 20000 samples for training a feed-forward neural network. Suppose they used a mini-batch gradient descent algorithm with a batch size of 500 to update the weights. How many times do the weights get updated after training the network for 10 epochs?

  1. Given: $N = 20000,\; B = 500,\; E = 10$.
  2. Updates per epoch $= \dfrac{N}{B} = \dfrac{20000}{500} = 40$.
  3. Total updates $= 40 \times 10 = 400$.
Answer: (a) 400
Trap: option (d) 40 is the count for one epoch only.
Q2Quiz-1 · Jan 2025Numeric · 3 marksMini-batch

Consider a network trained on a dataset of 5000 data points using mini-batch gradient descent with a batch size of 50. If the network is trained for 100 epochs, how many times does each parameter in the network get updated?

  1. Given: $N = 5000,\; B = 50,\; E = 100$.
  2. Updates per epoch $= \dfrac{5000}{50} = 100$.
  3. Total updates $= 100 \times 100 = 10000$.
Answer: 10000
Q3Quiz-1 · 2025 Term 2Numeric · 2 marksMini-batch

To learn the parameters of a neural network for a classification problem, mini-batch gradient descent is run on a dataset of size 1000 with a batch size of 25 for 10 epochs. Find the number of times each parameter is updated.

  1. Given: $N = 1000,\; B = 25,\; E = 10$.
  2. Updates per epoch $= \dfrac{1000}{25} = 40$.
  3. Total updates $= 40 \times 10 = 400$.
Answer: 400
Q4Practice Assignment · Week 4MCQSGD

A team has a data set that contains 10000 samples for training a feed-forward neural network. Suppose they decided to use the stochastic gradient descent algorithm to update the weights. How many times do the weights get updated after training the network for 10 epochs?

  1. SGD updates the weights after every single sample.
  2. Updates per epoch $= N = 10000$.
  3. Total updates $= 10000 \times 10 = 100000$.
Answer: (c) 100000
Trap: option (b) 10000 is the count for one epoch only.
Q5Practice Assignment · Week 4MCQBatch GD + line search

A team has a data set that contains 10000 samples for training a feed-forward neural network. Suppose they decided to use the gradient descent algorithm to update the weights. Suppose further that they use a line search algorithm for the learning rate as follows: $\eta = [0.01, 0.1, 1, 2, 10]$. How many times do the weights get updated after training the network for 10 epochs? (Note: for each weight update, the absolute difference between the current loss and the loss at the previous epoch should be high.)

  1. Plain (batch) gradient descent computes the gradient over the whole dataset, so it makes 1 update per epoch.
  2. Line search tries all 5 values of $\eta$ but keeps only the best one. Trying a value only evaluates the loss. It is not an update.
  3. Updates per epoch $= 1$, so total updates $= 1 \times 10 = 10$.
Answer: (d) 10
Trap: option (a) 50 $= 5 \times 10$ counts every trial learning rate as an update.
Q6Quiz-1 · Jan 2026Numeric · 3 marksVanilla GDTricky

Consider a training set consisting of 12 samples. The vanilla gradient descent algorithm is used to update the model parameters over 4 epochs. The learning rate follows an exponential decay scheme given by $$\eta_t = 0.8\,\exp(-0.2t),$$ where the iteration index $t$ starts from zero and increments only when the parameters are updated. What is the value of the learning rate $\eta_t$ at the end of training? (Enter your answer up to 3 decimal places. For example, if your answer is 0.12345, enter 0.123.)

  1. Count the updates. Vanilla (batch) GD makes 1 update per epoch, so 4 epochs give 4 updates. The 12 samples do not matter here.
  2. Find the last index. $t$ starts at 0, so the 4 updates use $t = 0, 1, 2, 3$. The last update uses $t = 3$.
  3. Compute the learning rate. $$\eta_3 = 0.8\,e^{-0.2\times 3} = 0.8\,e^{-0.6} = 0.8 \times 0.5488 = 0.439$$
Answer: 0.439 (accepted range 0.436 – 0.442)
Trap: taking $12 \times 4 = 48$ updates gives $0.8\,e^{-9.6} \approx 0.00005$, which is wrong. One update per sample is SGD, not vanilla GD. The handwritten solution in the PYQ PDF makes this mistake.

Exam checklist

  1. First find the algorithm's name in the question: vanilla/batch means 1 update per epoch, stochastic means $N$, and mini-batch means $N/B$.
  2. Multiply by the number of epochs. Several options are usually the one-epoch count, so watch out for those.
  3. Line search tries many learning rates but still makes one update per step.
  4. If an index $t$ starts at 0, the last update has index (total updates − 1).