CS 7643 Deep Learning Quiz 4 Exam Prep Document |
2026/2027 Edition | 250 Verified Questions
CS 7643 Deep Learning Quiz 4 2026-2027 QUESTIONS AND ANSWERS ALREADY GRADED A+. 100% Verified
Solutions | Updated Per Latest Guidelines | Graded A+
This comprehensive exam preparation document contains 250 verified questions for CS 7643 Deep
Learning Quiz 4, covering all key topics from the 2026/2027 curriculum. Each question includes
detailed rationales to reinforce understanding and ensure mastery of deep learning concepts. Designed
to guarantee a passing grade, this resource is meticulously updated to reflect the latest course
guidelines and exam patterns.
Abstract:
This document serves as a definitive study resource for CS 7643 Deep Learning Quiz 4, containing 250
meticulously verified questions with detailed rationales. The content spans fundamental to advanced topics,
including convolutional and recurrent architectures, attention mechanisms, generative models, and optimization
strategies. Each question is accompanied by a thorough explanation of the correct answer and analysis of common
distractors, enabling students to deepen their conceptual understanding and exam readiness. Updated for the
2026/2027 academic year, this guide reflects the latest course developments and emphasizes critical thinking over
rote memorization. By systematically working through these questions, learners can identify knowledge gaps,
reinforce key principles, and build confidence for achieving a top grade. The structured format and comprehensive
coverage make it an indispensable tool for both initial learning and final review.
Content Area Overview:
Content Area Questions Key Topics Weight
Convolutional Neural Networks 1-50 Conv layers, pooling, architectures 20%
(AlexNet, VGG, ResNet), transfer learning
Recurrent Neural Networks 51-100 RNN, LSTM, GRU, vanishing gradients, 20%
sequence-to-sequence models
Attention and Transformers 101-150 Self-attention, multi-head attention, 20%
Transformer architecture, BERT, GPT
Generative Models 151-200 GANs, VAEs, autoregressive models, 20%
diffusion models
Optimization and Regularization 201-250 SGD, Adam, batch normalization, dropout, 20%
weight decay, learning rate schedules
Page 1
,Q1. In the context of the Transformer architecture, which of the following best explains why
multi-head attention is preferred over single-head attention with the same total dimensionality?
A. It reduces the total number of parameters by factor of the number of heads.
B. It allows the model to attend to information from different representation subspaces at different
positions.
C. It eliminates the need for positional encodings by providing multiple views of the input.
D. It enables parallel computation across heads, reducing training time linearly with the number of
heads.
Correct Answer: B. It allows the model to attend to information from different representation
subspaces at different positions.
Rationale: Multi-head attention projects the queries, keys, and values multiple times with different
learned linear projections, allowing the model to jointly attend to information from different
representation subspaces. This enhances the model's ability to capture diverse patterns, which
single-head attention with the same total dimensionality cannot achieve due to averaging across all
dimensions.
Why Wrong:
A - Multi-head attention does not reduce parameters; it typically increases them slightly due to the
output projection.
C - Positional encodings are still required; multi-head attention does not replace them.
D - Parallel computation is a practical benefit but not the primary theoretical reason for improved
performance.
Reference: Vaswani et al. (2017). Attention Is All You Need.
Q2. Consider a Variational Autoencoder (VAE) trained with a standard Gaussian prior. During
training, the KL divergence term in the ELBO becomes very small. Which of the following is the
most likely cause?
A. The decoder is too powerful and ignores the latent code, leading to posterior collapse.
B. The encoder is learning a posterior that exactly matches the prior for all inputs.
C. The reconstruction loss is dominating the ELBO, causing the encoder to output zero variance.
D. The latent dimensionality is too high, causing the KL divergence to vanish.
Correct Answer: A. The decoder is too powerful and ignores the latent code, leading to posterior
collapse.
Rationale: Posterior collapse occurs when the decoder becomes powerful enough to reconstruct the input
without relying on the latent variable, causing the encoder to learn a posterior that matches the prior (KL
near zero). This is a known issue in VAEs, especially with strong decoders like autoregressive models.
Why Wrong:
B - While this results in small KL, it is a symptom of posterior collapse rather than the root cause;
the encoder is forced to match the prior because the decoder ignores the latent code.
C - If the encoder outputs zero variance, the KL divergence would actually increase because the
posterior would be a delta function, not match the prior.
D - Higher latent dimensionality typically increases KL divergence, not decreases it.
Reference: Kingma & Welling (2014). Auto-Encoding Variational Bayes.
Page 2
,Q3. In policy gradient methods for reinforcement learning, the REINFORCE algorithm is known to
have high variance. Which modification directly addresses this by subtracting a baseline that is a
function of the state?
A. Using a critic network to estimate the state-value function V(s) as baseline.
B. Using the return G_t directly without any baseline.
C. Using the action-value function Q(s,a) as baseline.
D. Normalizing the rewards to have zero mean and unit variance.
Correct Answer: A. Using a critic network to estimate the state-value function V(s) as baseline.
Rationale: Subtracting a state-dependent baseline, such as the state-value function V(s), reduces variance
without introducing bias because the baseline is independent of the action. This is a standard variance
reduction technique in policy gradient methods, leading to algorithms like Advantage Actor-Critic (A2C).
Why Wrong:
B - Using raw returns does not reduce variance; it is the standard REINFORCE with high variance.
C - Using Q(s,a) as baseline would introduce bias because Q depends on the action, and the
expectation of the gradient would be affected.
D - Normalizing rewards can help but is not a state-dependent baseline and does not directly address
the variance issue in the same principled way.
Reference: Sutton & Barto (2018). Reinforcement Learning: An Introduction, 2nd Ed., Ch. 13.
Q4. In the context of training Generative Adversarial Networks (GANs), which of the following is a
direct consequence of using the original minimax loss (non-saturating) for the generator?
A. The generator loss is equivalent to minimizing the Jensen-Shannon divergence between the data
and model distributions.
B. The generator loss is equivalent to minimizing the reverse KL divergence.
C. The generator loss saturates when the discriminator is optimal, providing strong gradients early in
training.
D. The generator loss is equivalent to minimizing the forward KL divergence.
Correct Answer: A. The generator loss is equivalent to minimizing the Jensen-Shannon divergence
between the data and model distributions.
Rationale: In the original GAN formulation, the minimax game leads to the generator minimizing the
Jensen-Shannon divergence (JSD) between the data distribution and the model distribution when the
discriminator is optimal. This is a theoretical result from Goodfellow et al. (2014).
Why Wrong:
B - Reverse KL divergence is minimized in some other generative models like VAEs, not in standard
GANs.
C - The original loss saturates (gradients vanish) when the discriminator is optimal, which is why the
non-saturating loss was introduced.
D - Forward KL divergence is not directly minimized by the standard GAN objective.
Reference: Goodfellow et al. (2014). Generative Adversarial Nets.
Page 3
, Q5. A researcher trains a deep neural network for image classification using stochastic gradient
descent (SGD) with a learning rate schedule. The training loss decreases rapidly initially but then
plateaus at a high value. Which of the following is the most likely cause?
A. The learning rate is too high, causing the loss to oscillate around a high value.
B. The learning rate is too low, preventing the optimizer from escaping a sharp local minimum.
C. The model has overfitted to the training data, causing the loss to plateau.
D. The optimizer is stuck in a saddle point due to insufficient momentum.
Correct Answer: B. The learning rate is too low, preventing the optimizer from escaping a sharp
local minimum.
Rationale: A learning rate that is too low can cause the optimizer to converge slowly and get stuck in a
poor local minimum or plateau, especially if the loss landscape has flat regions. The initial rapid
decrease followed by a plateau suggests the learning rate is too small to make further progress.
Why Wrong:
A - A high learning rate would cause the loss to diverge or oscillate, not plateau at a high value.
C - Overfitting would cause training loss to continue decreasing while validation loss increases, not
plateau.
D - Saddle points can be an issue, but the plateau is more commonly due to low learning rate, and
momentum helps escape saddle points.
Reference: Goodfellow, Bengio, Courville (2016). Deep Learning, Ch. 8.
Q6. Which of the following best describes the role of the temperature parameter in the softmax
function used in knowledge distillation?
A. It controls the sharpness of the output distribution; higher produces softer probabilities.
B. It scales the gradients during backpropagation to prevent vanishing gradients.
C. It adjusts the learning rate for the student network relative to the teacher.
D. It determines the weight of the distillation loss versus the student's cross-entropy loss.
Correct Answer: A. It controls the sharpness of the output distribution; higher produces softer
probabilities.
Rationale: The temperature parameter Ä in softmax (softmax(x/Ä)) controls the entropy of the output
distribution. A higher temperature produces a softer probability distribution over classes, which reveals
more information about the relative similarities between classes, aiding knowledge transfer.
Why Wrong:
B - Temperature does not directly scale gradients; it affects the softmax output, which in turn affects
gradients.
C - The temperature is not a learning rate; it is a hyperparameter of the softmax function.
D - The weight of the distillation loss is a separate hyperparameter (often denoted ).
Reference: Hinton, Vinyals, Dean (2015). Distilling the Knowledge in a Neural Network.
Page 4
2026/2027 Edition | 250 Verified Questions
CS 7643 Deep Learning Quiz 4 2026-2027 QUESTIONS AND ANSWERS ALREADY GRADED A+. 100% Verified
Solutions | Updated Per Latest Guidelines | Graded A+
This comprehensive exam preparation document contains 250 verified questions for CS 7643 Deep
Learning Quiz 4, covering all key topics from the 2026/2027 curriculum. Each question includes
detailed rationales to reinforce understanding and ensure mastery of deep learning concepts. Designed
to guarantee a passing grade, this resource is meticulously updated to reflect the latest course
guidelines and exam patterns.
Abstract:
This document serves as a definitive study resource for CS 7643 Deep Learning Quiz 4, containing 250
meticulously verified questions with detailed rationales. The content spans fundamental to advanced topics,
including convolutional and recurrent architectures, attention mechanisms, generative models, and optimization
strategies. Each question is accompanied by a thorough explanation of the correct answer and analysis of common
distractors, enabling students to deepen their conceptual understanding and exam readiness. Updated for the
2026/2027 academic year, this guide reflects the latest course developments and emphasizes critical thinking over
rote memorization. By systematically working through these questions, learners can identify knowledge gaps,
reinforce key principles, and build confidence for achieving a top grade. The structured format and comprehensive
coverage make it an indispensable tool for both initial learning and final review.
Content Area Overview:
Content Area Questions Key Topics Weight
Convolutional Neural Networks 1-50 Conv layers, pooling, architectures 20%
(AlexNet, VGG, ResNet), transfer learning
Recurrent Neural Networks 51-100 RNN, LSTM, GRU, vanishing gradients, 20%
sequence-to-sequence models
Attention and Transformers 101-150 Self-attention, multi-head attention, 20%
Transformer architecture, BERT, GPT
Generative Models 151-200 GANs, VAEs, autoregressive models, 20%
diffusion models
Optimization and Regularization 201-250 SGD, Adam, batch normalization, dropout, 20%
weight decay, learning rate schedules
Page 1
,Q1. In the context of the Transformer architecture, which of the following best explains why
multi-head attention is preferred over single-head attention with the same total dimensionality?
A. It reduces the total number of parameters by factor of the number of heads.
B. It allows the model to attend to information from different representation subspaces at different
positions.
C. It eliminates the need for positional encodings by providing multiple views of the input.
D. It enables parallel computation across heads, reducing training time linearly with the number of
heads.
Correct Answer: B. It allows the model to attend to information from different representation
subspaces at different positions.
Rationale: Multi-head attention projects the queries, keys, and values multiple times with different
learned linear projections, allowing the model to jointly attend to information from different
representation subspaces. This enhances the model's ability to capture diverse patterns, which
single-head attention with the same total dimensionality cannot achieve due to averaging across all
dimensions.
Why Wrong:
A - Multi-head attention does not reduce parameters; it typically increases them slightly due to the
output projection.
C - Positional encodings are still required; multi-head attention does not replace them.
D - Parallel computation is a practical benefit but not the primary theoretical reason for improved
performance.
Reference: Vaswani et al. (2017). Attention Is All You Need.
Q2. Consider a Variational Autoencoder (VAE) trained with a standard Gaussian prior. During
training, the KL divergence term in the ELBO becomes very small. Which of the following is the
most likely cause?
A. The decoder is too powerful and ignores the latent code, leading to posterior collapse.
B. The encoder is learning a posterior that exactly matches the prior for all inputs.
C. The reconstruction loss is dominating the ELBO, causing the encoder to output zero variance.
D. The latent dimensionality is too high, causing the KL divergence to vanish.
Correct Answer: A. The decoder is too powerful and ignores the latent code, leading to posterior
collapse.
Rationale: Posterior collapse occurs when the decoder becomes powerful enough to reconstruct the input
without relying on the latent variable, causing the encoder to learn a posterior that matches the prior (KL
near zero). This is a known issue in VAEs, especially with strong decoders like autoregressive models.
Why Wrong:
B - While this results in small KL, it is a symptom of posterior collapse rather than the root cause;
the encoder is forced to match the prior because the decoder ignores the latent code.
C - If the encoder outputs zero variance, the KL divergence would actually increase because the
posterior would be a delta function, not match the prior.
D - Higher latent dimensionality typically increases KL divergence, not decreases it.
Reference: Kingma & Welling (2014). Auto-Encoding Variational Bayes.
Page 2
,Q3. In policy gradient methods for reinforcement learning, the REINFORCE algorithm is known to
have high variance. Which modification directly addresses this by subtracting a baseline that is a
function of the state?
A. Using a critic network to estimate the state-value function V(s) as baseline.
B. Using the return G_t directly without any baseline.
C. Using the action-value function Q(s,a) as baseline.
D. Normalizing the rewards to have zero mean and unit variance.
Correct Answer: A. Using a critic network to estimate the state-value function V(s) as baseline.
Rationale: Subtracting a state-dependent baseline, such as the state-value function V(s), reduces variance
without introducing bias because the baseline is independent of the action. This is a standard variance
reduction technique in policy gradient methods, leading to algorithms like Advantage Actor-Critic (A2C).
Why Wrong:
B - Using raw returns does not reduce variance; it is the standard REINFORCE with high variance.
C - Using Q(s,a) as baseline would introduce bias because Q depends on the action, and the
expectation of the gradient would be affected.
D - Normalizing rewards can help but is not a state-dependent baseline and does not directly address
the variance issue in the same principled way.
Reference: Sutton & Barto (2018). Reinforcement Learning: An Introduction, 2nd Ed., Ch. 13.
Q4. In the context of training Generative Adversarial Networks (GANs), which of the following is a
direct consequence of using the original minimax loss (non-saturating) for the generator?
A. The generator loss is equivalent to minimizing the Jensen-Shannon divergence between the data
and model distributions.
B. The generator loss is equivalent to minimizing the reverse KL divergence.
C. The generator loss saturates when the discriminator is optimal, providing strong gradients early in
training.
D. The generator loss is equivalent to minimizing the forward KL divergence.
Correct Answer: A. The generator loss is equivalent to minimizing the Jensen-Shannon divergence
between the data and model distributions.
Rationale: In the original GAN formulation, the minimax game leads to the generator minimizing the
Jensen-Shannon divergence (JSD) between the data distribution and the model distribution when the
discriminator is optimal. This is a theoretical result from Goodfellow et al. (2014).
Why Wrong:
B - Reverse KL divergence is minimized in some other generative models like VAEs, not in standard
GANs.
C - The original loss saturates (gradients vanish) when the discriminator is optimal, which is why the
non-saturating loss was introduced.
D - Forward KL divergence is not directly minimized by the standard GAN objective.
Reference: Goodfellow et al. (2014). Generative Adversarial Nets.
Page 3
, Q5. A researcher trains a deep neural network for image classification using stochastic gradient
descent (SGD) with a learning rate schedule. The training loss decreases rapidly initially but then
plateaus at a high value. Which of the following is the most likely cause?
A. The learning rate is too high, causing the loss to oscillate around a high value.
B. The learning rate is too low, preventing the optimizer from escaping a sharp local minimum.
C. The model has overfitted to the training data, causing the loss to plateau.
D. The optimizer is stuck in a saddle point due to insufficient momentum.
Correct Answer: B. The learning rate is too low, preventing the optimizer from escaping a sharp
local minimum.
Rationale: A learning rate that is too low can cause the optimizer to converge slowly and get stuck in a
poor local minimum or plateau, especially if the loss landscape has flat regions. The initial rapid
decrease followed by a plateau suggests the learning rate is too small to make further progress.
Why Wrong:
A - A high learning rate would cause the loss to diverge or oscillate, not plateau at a high value.
C - Overfitting would cause training loss to continue decreasing while validation loss increases, not
plateau.
D - Saddle points can be an issue, but the plateau is more commonly due to low learning rate, and
momentum helps escape saddle points.
Reference: Goodfellow, Bengio, Courville (2016). Deep Learning, Ch. 8.
Q6. Which of the following best describes the role of the temperature parameter in the softmax
function used in knowledge distillation?
A. It controls the sharpness of the output distribution; higher produces softer probabilities.
B. It scales the gradients during backpropagation to prevent vanishing gradients.
C. It adjusts the learning rate for the student network relative to the teacher.
D. It determines the weight of the distillation loss versus the student's cross-entropy loss.
Correct Answer: A. It controls the sharpness of the output distribution; higher produces softer
probabilities.
Rationale: The temperature parameter Ä in softmax (softmax(x/Ä)) controls the entropy of the output
distribution. A higher temperature produces a softer probability distribution over classes, which reveals
more information about the relative similarities between classes, aiding knowledge transfer.
Why Wrong:
B - Temperature does not directly scale gradients; it affects the softmax output, which in turn affects
gradients.
C - The temperature is not a learning rate; it is a hyperparameter of the softmax function.
D - The weight of the distillation loss is a separate hyperparameter (often denoted ).
Reference: Hinton, Vinyals, Dean (2015). Distilling the Knowledge in a Neural Network.
Page 4
