CS7643 DEEP LEARNING TEST WITH VERIFIED ANSWERS | 100% CORRECT| LATEST
2025/2026 UPDATE - GEORGIA INSTITUTE OF TECHNOLOGY.
Core Domains
Machine Learning Fundamentals and Linear Classifiers
Neural Network Fundamentals and Backpropagation
Optimization Algorithms and Training Dynamics
Loss Functions and Regularization
Convolutional Neural Networks and Architectures
Sequence Modeling, RNNs, and Transformers
Generative Models and Advanced Topics
Model Evaluation, Generalization, and Error Analysis
Introduction
The CS7643 Deep Learning Quiz 3 Examination evaluates the graduate student's mastery of
advanced neural network training, optimization theory, and modern deep learning architectures.
The assessment measures conceptual understanding of optimization algorithms including SGD,
Momentum, RMSProp, and Adam, alongside critical analysis of loss functions, weight
regularization techniques, and advanced architectural components. Content emphasizes the
mathematical foundations of backpropagation, hyperparameter tuning strategies, and practical
implementation challenges in frameworks such as TensorFlow and PyTorch. This comprehensive
test bank challenges the candidate to analyze training dynamics, diagnose optimization failures,
compare architectural design choices, and apply deep learning principles to real-world computer
vision and natural language processing scenarios.
SECTION ONE: QUESTIONS 1–100
1. Which of the following best describes the primary difference between batch gradient
descent and stochastic gradient descent (SGD)?
A. Batch gradient descent uses a subset of data for each update, while SGD uses the full
dataset
B. Batch gradient descent computes the gradient using the entire training set for each
parameter update, while SGD uses a single example or mini-batch
,C. Batch gradient descent is always faster than SGD in terms of wall-clock time per epoch
D. SGD is guaranteed to converge to the global minimum for non-convex loss functions
🟢 B. Batch gradient descent computes the gradient using the entire training set for each
parameter update, while SGD uses a single example or mini-batch
🔴 RATIONALE: Batch gradient descent computes the gradient over the entire training set
before performing a single parameter update, providing a stable but computationally
expensive estimate of the true gradient. SGD approximates the gradient using a single example
or mini-batch, introducing noise that can help escape saddle points but causes oscillations
during convergence. Neither method guarantees global convergence for non-convex
problems.
2. In the context of deep learning optimization, what is the primary purpose of using
momentum in SGD?
A. To increase the learning rate dynamically during training
B. To accumulate a velocity vector that smooths parameter updates and accelerates
convergence in consistent gradient directions
C. To reduce the memory footprint of the optimizer
D. To guarantee convergence to the global minimum
🟢 B. To accumulate a velocity vector that smooths parameter updates and accelerates
convergence in consistent gradient directions
🔴 RATIONALE: Momentum maintains a running average of past gradients, creating a velocity
vector that dampens oscillations in directions of high curvature and accelerates movement in
consistent directions. This helps the optimizer navigate ravines and escape shallow local
minima more effectively than vanilla SGD.
3. Which optimizer combines the ideas of momentum and adaptive learning rates by
maintaining estimates of both first and second moments of the gradients?
A. SGD with Nesterov momentum
B. RMSProp
C. Adam
D. Adagrad
,🟢 C. Adam
🔴 RATIONALE: Adam (Adaptive Moment Estimation) maintains exponential moving averages
of both the first moment (mean) and second moment (uncentered variance) of the gradients.
This combination allows it to adapt the learning rate per parameter while also benefiting from
momentum-like smoothing.
4. What is the primary computational advantage of using the ReLU activation function
compared to sigmoid or tanh?
A. ReLU has a bounded output range, preventing exploding activations
B. ReLU is zero-centered, which improves optimization dynamics
C. ReLU involves only a comparison and no expensive exponential operations, making it
computationally cheaper
D. ReLU is differentiable at all points, simplifying backpropagation
🟢 C. ReLU involves only a comparison and no expensive exponential operations, making it
computationally cheaper
🔴 RATIONALE: ReLU(x) = max(0, x) requires only a comparison operation, whereas sigmoid
and tanh require computing exponentials. This makes ReLU significantly faster to compute in
both forward and backward passes. ReLU is not zero-centered, is not differentiable at zero, and
has an unbounded positive range.
5. What is the "dying ReLU" problem?
A. ReLU activations cause the loss function to become non-convex
B. Neurons with ReLU activation can become permanently inactive if they consistently receive
negative inputs, producing zero gradients
C. ReLU causes gradients to explode during backpropagation
D. ReLU cannot be used in convolutional neural networks
🟢 B. Neurons with ReLU activation can become permanently inactive if they consistently
receive negative inputs, producing zero gradients
🔴 RATIONALE: When a ReLU neuron receives negative input, its output is zero and the
gradient is zero. If this persists, the neuron's weights may never update, effectively "killing" the
, neuron. Leaky ReLU and ELU were introduced to address this issue by allowing small non-zero
gradients for negative inputs.
6. Which of the following is NOT a benefit of using batch normalization in deep neural
networks?
A. It reduces internal covariate shift by normalizing layer inputs
B. It allows for higher learning rates, accelerating training
C. It acts as a regularizer, reducing the need for dropout
D. It guarantees that the network will converge to the global minimum
🟢 D. It guarantees that the network will converge to the global minimum
🔴 RATIONALE: Batch normalization provides several benefits including reducing internal
covariate shift, enabling higher learning rates, and acting as a mild regularizer. However, it does
not guarantee global convergence — deep neural networks remain non-convex optimization
problems with no such guarantees.
7. In batch normalization, what is the purpose of the learnable parameters γ (gamma) and β
(beta)?
A. They control the learning rate for each layer
B. They allow the network to scale and shift the normalized activations, preserving
representational capacity
C. They determine which neurons to drop during training
D. They set the momentum for the running statistics
🟢 B. They allow the network to scale and shift the normalized activations, preserving
representational capacity
🔴 RATIONALE: After normalizing activations to zero mean and unit variance, batch
normalization applies γ (scale) and β (shift) to allow the network to learn the optimal
distribution of activations. Without these parameters, normalization would restrict the layer's
representational power.
8. What is the primary difference between L1 and L2 regularization?
A. L1 regularization adds the sum of squared weights to the loss, while L2 adds the sum of
absolute weights
2025/2026 UPDATE - GEORGIA INSTITUTE OF TECHNOLOGY.
Core Domains
Machine Learning Fundamentals and Linear Classifiers
Neural Network Fundamentals and Backpropagation
Optimization Algorithms and Training Dynamics
Loss Functions and Regularization
Convolutional Neural Networks and Architectures
Sequence Modeling, RNNs, and Transformers
Generative Models and Advanced Topics
Model Evaluation, Generalization, and Error Analysis
Introduction
The CS7643 Deep Learning Quiz 3 Examination evaluates the graduate student's mastery of
advanced neural network training, optimization theory, and modern deep learning architectures.
The assessment measures conceptual understanding of optimization algorithms including SGD,
Momentum, RMSProp, and Adam, alongside critical analysis of loss functions, weight
regularization techniques, and advanced architectural components. Content emphasizes the
mathematical foundations of backpropagation, hyperparameter tuning strategies, and practical
implementation challenges in frameworks such as TensorFlow and PyTorch. This comprehensive
test bank challenges the candidate to analyze training dynamics, diagnose optimization failures,
compare architectural design choices, and apply deep learning principles to real-world computer
vision and natural language processing scenarios.
SECTION ONE: QUESTIONS 1–100
1. Which of the following best describes the primary difference between batch gradient
descent and stochastic gradient descent (SGD)?
A. Batch gradient descent uses a subset of data for each update, while SGD uses the full
dataset
B. Batch gradient descent computes the gradient using the entire training set for each
parameter update, while SGD uses a single example or mini-batch
,C. Batch gradient descent is always faster than SGD in terms of wall-clock time per epoch
D. SGD is guaranteed to converge to the global minimum for non-convex loss functions
🟢 B. Batch gradient descent computes the gradient using the entire training set for each
parameter update, while SGD uses a single example or mini-batch
🔴 RATIONALE: Batch gradient descent computes the gradient over the entire training set
before performing a single parameter update, providing a stable but computationally
expensive estimate of the true gradient. SGD approximates the gradient using a single example
or mini-batch, introducing noise that can help escape saddle points but causes oscillations
during convergence. Neither method guarantees global convergence for non-convex
problems.
2. In the context of deep learning optimization, what is the primary purpose of using
momentum in SGD?
A. To increase the learning rate dynamically during training
B. To accumulate a velocity vector that smooths parameter updates and accelerates
convergence in consistent gradient directions
C. To reduce the memory footprint of the optimizer
D. To guarantee convergence to the global minimum
🟢 B. To accumulate a velocity vector that smooths parameter updates and accelerates
convergence in consistent gradient directions
🔴 RATIONALE: Momentum maintains a running average of past gradients, creating a velocity
vector that dampens oscillations in directions of high curvature and accelerates movement in
consistent directions. This helps the optimizer navigate ravines and escape shallow local
minima more effectively than vanilla SGD.
3. Which optimizer combines the ideas of momentum and adaptive learning rates by
maintaining estimates of both first and second moments of the gradients?
A. SGD with Nesterov momentum
B. RMSProp
C. Adam
D. Adagrad
,🟢 C. Adam
🔴 RATIONALE: Adam (Adaptive Moment Estimation) maintains exponential moving averages
of both the first moment (mean) and second moment (uncentered variance) of the gradients.
This combination allows it to adapt the learning rate per parameter while also benefiting from
momentum-like smoothing.
4. What is the primary computational advantage of using the ReLU activation function
compared to sigmoid or tanh?
A. ReLU has a bounded output range, preventing exploding activations
B. ReLU is zero-centered, which improves optimization dynamics
C. ReLU involves only a comparison and no expensive exponential operations, making it
computationally cheaper
D. ReLU is differentiable at all points, simplifying backpropagation
🟢 C. ReLU involves only a comparison and no expensive exponential operations, making it
computationally cheaper
🔴 RATIONALE: ReLU(x) = max(0, x) requires only a comparison operation, whereas sigmoid
and tanh require computing exponentials. This makes ReLU significantly faster to compute in
both forward and backward passes. ReLU is not zero-centered, is not differentiable at zero, and
has an unbounded positive range.
5. What is the "dying ReLU" problem?
A. ReLU activations cause the loss function to become non-convex
B. Neurons with ReLU activation can become permanently inactive if they consistently receive
negative inputs, producing zero gradients
C. ReLU causes gradients to explode during backpropagation
D. ReLU cannot be used in convolutional neural networks
🟢 B. Neurons with ReLU activation can become permanently inactive if they consistently
receive negative inputs, producing zero gradients
🔴 RATIONALE: When a ReLU neuron receives negative input, its output is zero and the
gradient is zero. If this persists, the neuron's weights may never update, effectively "killing" the
, neuron. Leaky ReLU and ELU were introduced to address this issue by allowing small non-zero
gradients for negative inputs.
6. Which of the following is NOT a benefit of using batch normalization in deep neural
networks?
A. It reduces internal covariate shift by normalizing layer inputs
B. It allows for higher learning rates, accelerating training
C. It acts as a regularizer, reducing the need for dropout
D. It guarantees that the network will converge to the global minimum
🟢 D. It guarantees that the network will converge to the global minimum
🔴 RATIONALE: Batch normalization provides several benefits including reducing internal
covariate shift, enabling higher learning rates, and acting as a mild regularizer. However, it does
not guarantee global convergence — deep neural networks remain non-convex optimization
problems with no such guarantees.
7. In batch normalization, what is the purpose of the learnable parameters γ (gamma) and β
(beta)?
A. They control the learning rate for each layer
B. They allow the network to scale and shift the normalized activations, preserving
representational capacity
C. They determine which neurons to drop during training
D. They set the momentum for the running statistics
🟢 B. They allow the network to scale and shift the normalized activations, preserving
representational capacity
🔴 RATIONALE: After normalizing activations to zero mean and unit variance, batch
normalization applies γ (scale) and β (shift) to allow the network to learn the optimal
distribution of activations. Without these parameters, normalization would restrict the layer's
representational power.
8. What is the primary difference between L1 and L2 regularization?
A. L1 regularization adds the sum of squared weights to the loss, while L2 adds the sum of
absolute weights
