Georgia Tech CS 7643 Deep
Learning Quiz 2 Complete Practice
& Review Bundle | 56 Questions,
Answers & Detailed Rationales |
Fall 2026 Pass Guaranteed Graded
A+
Section 1: Optimization & Training Issues
Question 1
Which of the following are common issues while optimizing
the weights of a deep neural network? (Select all that apply)
A. Existence of local minima
B. Ill-conditioned loss surface
C. Noisy gradient estimates
D. Saddle points
Answer: B, C, D
Rationale: While local minima are a concern in shallow networks,
in deep learning, saddle points and ill-conditioned curvature are
more significant obstacles. The stochastic nature of mini-batch
,gradient descent introduces noise in gradient estimates. Modern
optimizers are often designed specifically to address these
challenges.
Question 2
Why does stochastic gradient descent (SGD) often converge
to better solutions than full batch gradient descent?
A. It guarantees global minima
B. It uses second-order derivatives
C. The noise helps escape saddle points and local minima
D. It requires fewer epochs
Answer: C
Rationale: The inherent noise in SGD's gradient estimates helps
the optimizer escape saddle points and shallow local minima,
often leading to better generalization compared to full batch
gradient descent.
Question 3
What is the role of momentum in gradient descent?
A. It increases the learning rate
B. It accelerates convergence by accumulating past gradient
information
,C. It reduces the batch size
D. It prevents overfitting
Rationale: Momentum helps accelerate SGD in relevant directions
and dampens oscillations by accumulating a moving average of
past gradients. This is particularly useful when the loss surface has
ill-conditioned curvature.
Question 4
The vanishing gradient problem occurs primarily with:
A. ReLU activations
B. Tanh and Sigmoid activations
C. Linear transformations
D. Max pooling layers
Answer: B
Rationale: Vanishing gradients occur when gradients are
repeatedly multiplied by values in (0,1) as they propagate
backward through many layers. Sigmoid and tanh activations have
derivatives that are ≤ 1, causing gradients to become
exponentially small in deep networks.
Section 2: Convolutional Neural Networks & Parameter
Counting
, Question 5
Suppose you have an input volume of dimension 64x64x16.
How many parameters would a single 1x1 convolutional filter
have, including the bias?
Answer: 17
Rationale: A 1x1 convolution with depth 16 has 1×1×16 = 16
weights + 1 bias = 17 parameters.
Question 6
Suppose your input is a 300×300 color (RGB) image, and you
use a convolutional layer with 100 filters that are each 5×5.
How many parameters does this layer have, including the bias
parameters?
Answer: 7,600
Rationale: Parameter count = (kernel_height × kernel_width ×
input_depth + 1) × number_of_filters = (5×5×3 + 1) × 100 = 76 ×
100 = 7,600.
Question 7
You have an input volume that is 63×63×16 and convolve it
with 32 filters that are each 7×7, with stride of 1. You want to
use a "same" convolution. What is the padding required?
Learning Quiz 2 Complete Practice
& Review Bundle | 56 Questions,
Answers & Detailed Rationales |
Fall 2026 Pass Guaranteed Graded
A+
Section 1: Optimization & Training Issues
Question 1
Which of the following are common issues while optimizing
the weights of a deep neural network? (Select all that apply)
A. Existence of local minima
B. Ill-conditioned loss surface
C. Noisy gradient estimates
D. Saddle points
Answer: B, C, D
Rationale: While local minima are a concern in shallow networks,
in deep learning, saddle points and ill-conditioned curvature are
more significant obstacles. The stochastic nature of mini-batch
,gradient descent introduces noise in gradient estimates. Modern
optimizers are often designed specifically to address these
challenges.
Question 2
Why does stochastic gradient descent (SGD) often converge
to better solutions than full batch gradient descent?
A. It guarantees global minima
B. It uses second-order derivatives
C. The noise helps escape saddle points and local minima
D. It requires fewer epochs
Answer: C
Rationale: The inherent noise in SGD's gradient estimates helps
the optimizer escape saddle points and shallow local minima,
often leading to better generalization compared to full batch
gradient descent.
Question 3
What is the role of momentum in gradient descent?
A. It increases the learning rate
B. It accelerates convergence by accumulating past gradient
information
,C. It reduces the batch size
D. It prevents overfitting
Rationale: Momentum helps accelerate SGD in relevant directions
and dampens oscillations by accumulating a moving average of
past gradients. This is particularly useful when the loss surface has
ill-conditioned curvature.
Question 4
The vanishing gradient problem occurs primarily with:
A. ReLU activations
B. Tanh and Sigmoid activations
C. Linear transformations
D. Max pooling layers
Answer: B
Rationale: Vanishing gradients occur when gradients are
repeatedly multiplied by values in (0,1) as they propagate
backward through many layers. Sigmoid and tanh activations have
derivatives that are ≤ 1, causing gradients to become
exponentially small in deep networks.
Section 2: Convolutional Neural Networks & Parameter
Counting
, Question 5
Suppose you have an input volume of dimension 64x64x16.
How many parameters would a single 1x1 convolutional filter
have, including the bias?
Answer: 17
Rationale: A 1x1 convolution with depth 16 has 1×1×16 = 16
weights + 1 bias = 17 parameters.
Question 6
Suppose your input is a 300×300 color (RGB) image, and you
use a convolutional layer with 100 filters that are each 5×5.
How many parameters does this layer have, including the bias
parameters?
Answer: 7,600
Rationale: Parameter count = (kernel_height × kernel_width ×
input_depth + 1) × number_of_filters = (5×5×3 + 1) × 100 = 76 ×
100 = 7,600.
Question 7
You have an input volume that is 63×63×16 and convolve it
with 32 filters that are each 7×7, with stride of 1. You want to
use a "same" convolution. What is the padding required?
