BYU CS 474 Deep Learning Midterm Study Guide
EXPECTED QUESTIONS WITH VERIFIED ANSWERS
Question 1.
1. If W has dimensions 6 x 10 and x has dimensions 10 x 1, how many dot products are involved in
computing Wx?
ANSWER
6
Question 2.
1. Increasing regularization _____ bias and ____ variance
ANSWER
a. Increases bias, reduces variance.
Question 3.
1. Dropout
ANSWER
L2 (Ridge) regularization
(set certain activations to zero/squared error penalty)
Question 4.
1. What is the memory requirement of SGD
ANSWER
2*m
Question 5.
What is the memory requirement of Momentum
ANSWER
3*m
1/7
, Question 6.
What is the memory requirement of AdaGrad
ANSWER
3*m
Question 7.
What is the memory requirement of Adam
ANSWER
4*m
Question 8.
Shakeout
ANSWER
L1 + L2 (Elastic) regularization
(assign random labels to some of your nodes)
Question 9.
regularization techniques that are "sparsity inducing"
ANSWER
L1 (Lasso) regularization (penalty on absolute value), L1 + L2 (Elastic) regularization
Question 10.
architectures that are naturally suited to variable-length data
ANSWER
RNN, Transformer
Question 11.
1. In the skip-gram negative sampling algorithm, what would happen if you omitted negative
sampling?
ANSWER
All word embeddings would be learned to be the same.
2/7
EXPECTED QUESTIONS WITH VERIFIED ANSWERS
Question 1.
1. If W has dimensions 6 x 10 and x has dimensions 10 x 1, how many dot products are involved in
computing Wx?
ANSWER
6
Question 2.
1. Increasing regularization _____ bias and ____ variance
ANSWER
a. Increases bias, reduces variance.
Question 3.
1. Dropout
ANSWER
L2 (Ridge) regularization
(set certain activations to zero/squared error penalty)
Question 4.
1. What is the memory requirement of SGD
ANSWER
2*m
Question 5.
What is the memory requirement of Momentum
ANSWER
3*m
1/7
, Question 6.
What is the memory requirement of AdaGrad
ANSWER
3*m
Question 7.
What is the memory requirement of Adam
ANSWER
4*m
Question 8.
Shakeout
ANSWER
L1 + L2 (Elastic) regularization
(assign random labels to some of your nodes)
Question 9.
regularization techniques that are "sparsity inducing"
ANSWER
L1 (Lasso) regularization (penalty on absolute value), L1 + L2 (Elastic) regularization
Question 10.
architectures that are naturally suited to variable-length data
ANSWER
RNN, Transformer
Question 11.
1. In the skip-gram negative sampling algorithm, what would happen if you omitted negative
sampling?
ANSWER
All word embeddings would be learned to be the same.
2/7
