CS7643 – Deep Learning Quiz 4 Exam (2026) GRADED
A+ Study Guide
Description:
Comprehensive A+ study guide for CS7643 Quiz 4 Exam (2026). Covers advanced deep
learning concepts such as recurrent neural networks (RNNs), LSTMs, GRUs, sequence
modeling, gradient clipping, and optimization techniques. Includes clear explanations, key
equations, and worked examples to help you fully grasp the material. Perfect for efficient
revision and achieving top results.
Tags: CS7643, Deep Learning, Quiz 4, 2026, GRADED A+, Machine Learning, Neural
Networks, RNN, LSTM, Study Guide, Exam Prep
CS7643 Quiz 4
Embedding - CORRECT ANSWER ✔✔✔✔✔ A learned map from entities to
vectors that encodes similarity
Graph Embedding - CORRECT ANSWER ✔✔✔✔✔ Optimize the objective that
connected nodes have more similar embeddings than unconnected nodes.
Task: convert nodes to vectors
- effectively unsupervised learning where nearest neighbors are similar
- these learned vectors are useful for downstream tasks
Multi-layer Perceptron (MLP) pain points for NLP - CORRECT ANSWER
✔✔✔✔✔ - Cannot easily support variable-sized sequences as inputs or outputs
- No inherent temporal structure
- No practical way of holding state
- The size of the network grows with the maximum allowed size of the input or
output sequences
Truncated Backpropagation through time - CORRECT ANSWER ✔✔✔✔✔ -
Only backpropagate a RNN through T time steps
, Recurrent Neural Networks (RNN) - CORRECT ANSWER ✔✔✔✔✔ h(t) =
activation(U*input + V*h(t-1) + bias)
y(t) = activation(W*h(t) + bias)
- activation is typically the logistic function or tanh
- outputs can also simply be h(t)
- family of NN architectures for modeling sequences
Training Vanilla RNN's difficulties - CORRECT ANSWER ✔✔✔✔✔ - Vanishing
gradients
- Since dx(t)/dx(t-1) = w^t
- if w > 1: exploding gradients
- if w < 1: vanishing gradients
Long Short-Term Memory Network Gates and States - CORRECT ANSWER
✔✔✔✔✔ - f(t) = forget gate
- i(t) = input gate
- u(t) = candidate update gate
- o(t) = output gate
- c(t) = cell state
- c(t) = f(t) * c(t - 1) + i(t) * u(t)
- h(t) = hidden state
- h(t) = o(t) * tanh(c(t))
Perplexity(s) - CORRECT ANSWER ✔✔✔✔✔ = product( 1 / P(w(i) | w(i-1), ...) )
^ (1 / N)
= b ^ (-1/N sum( log(b) (P(w(i) | w(i-1), ...) ) )
- note exponent of b is per word CE loss
- perplexity of a discrete uniform distribution over k events is k
Language Model Goal - CORRECT ANSWER ✔✔✔✔✔ - estimate the
probability of sequences of words
- p(s) = p(w1, w2, ..., wn)
Masked Language Modeling - CORRECT ANSWER ✔✔✔✔✔ - pre-training task
- an auxiliary task different from the final task we're really interested in, but which
can help us achieve better performance finding good initial parameters for the
model
- By pre-training on masked language modeling before training on our final task,
it is usually possible to obtain higher performance than by simply training on the
final task
A+ Study Guide
Description:
Comprehensive A+ study guide for CS7643 Quiz 4 Exam (2026). Covers advanced deep
learning concepts such as recurrent neural networks (RNNs), LSTMs, GRUs, sequence
modeling, gradient clipping, and optimization techniques. Includes clear explanations, key
equations, and worked examples to help you fully grasp the material. Perfect for efficient
revision and achieving top results.
Tags: CS7643, Deep Learning, Quiz 4, 2026, GRADED A+, Machine Learning, Neural
Networks, RNN, LSTM, Study Guide, Exam Prep
CS7643 Quiz 4
Embedding - CORRECT ANSWER ✔✔✔✔✔ A learned map from entities to
vectors that encodes similarity
Graph Embedding - CORRECT ANSWER ✔✔✔✔✔ Optimize the objective that
connected nodes have more similar embeddings than unconnected nodes.
Task: convert nodes to vectors
- effectively unsupervised learning where nearest neighbors are similar
- these learned vectors are useful for downstream tasks
Multi-layer Perceptron (MLP) pain points for NLP - CORRECT ANSWER
✔✔✔✔✔ - Cannot easily support variable-sized sequences as inputs or outputs
- No inherent temporal structure
- No practical way of holding state
- The size of the network grows with the maximum allowed size of the input or
output sequences
Truncated Backpropagation through time - CORRECT ANSWER ✔✔✔✔✔ -
Only backpropagate a RNN through T time steps
, Recurrent Neural Networks (RNN) - CORRECT ANSWER ✔✔✔✔✔ h(t) =
activation(U*input + V*h(t-1) + bias)
y(t) = activation(W*h(t) + bias)
- activation is typically the logistic function or tanh
- outputs can also simply be h(t)
- family of NN architectures for modeling sequences
Training Vanilla RNN's difficulties - CORRECT ANSWER ✔✔✔✔✔ - Vanishing
gradients
- Since dx(t)/dx(t-1) = w^t
- if w > 1: exploding gradients
- if w < 1: vanishing gradients
Long Short-Term Memory Network Gates and States - CORRECT ANSWER
✔✔✔✔✔ - f(t) = forget gate
- i(t) = input gate
- u(t) = candidate update gate
- o(t) = output gate
- c(t) = cell state
- c(t) = f(t) * c(t - 1) + i(t) * u(t)
- h(t) = hidden state
- h(t) = o(t) * tanh(c(t))
Perplexity(s) - CORRECT ANSWER ✔✔✔✔✔ = product( 1 / P(w(i) | w(i-1), ...) )
^ (1 / N)
= b ^ (-1/N sum( log(b) (P(w(i) | w(i-1), ...) ) )
- note exponent of b is per word CE loss
- perplexity of a discrete uniform distribution over k events is k
Language Model Goal - CORRECT ANSWER ✔✔✔✔✔ - estimate the
probability of sequences of words
- p(s) = p(w1, w2, ..., wn)
Masked Language Modeling - CORRECT ANSWER ✔✔✔✔✔ - pre-training task
- an auxiliary task different from the final task we're really interested in, but which
can help us achieve better performance finding good initial parameters for the
model
- By pre-training on masked language modeling before training on our final task,
it is usually possible to obtain higher performance than by simply training on the
final task
