CS 7641 RL UNIT QUIZ PROBLEMS
AND SOLUTIONS (VERIFIED
ANSWERS) PLUS RATIONALES 2026
Q&A | INSTANT DOWNLOAD PDF.
Core Domains
• Markov Decision Processes (MDPs)
• Bellman Equations and Dynamic Programming
• Value Iteration and Policy Iteration
• Q-Learning and Temporal Difference Learning
• Exploration vs. Exploitation Strategies
• SARSA and On-Policy vs. Off-Policy Methods
• Model-Based vs. Model-Free Reinforcement Learning
• Game Theory and Multi-Agent Reinforcement Learning
Introduction
The CS 7641 Reinforcement Learning Unit Quiz assesses graduate-
level understanding of core reinforcement learning concepts within
the Machine Learning curriculum at Georgia Institute of Technology.
This comprehensive assessment evaluates knowledge of Markov
decision processes, dynamic programming algorithms, temporal
difference learning methods, and game-theoretic foundations. The
examination employs multiple-choice and scenario-based questions
requiring applied decision-making skills. Emphasis is placed on
,understanding algorithmic convergence properties, policy evaluation
techniques, and the mathematical foundations underlying modern
reinforcement learning systems. Successful completion demonstrates
readiness to implement and analyze reinforcement learning solutions
in complex real-world environments.
SECTION ONE: QUESTIONS 1–100
Question 1
In a Markov Decision Process (MDP), which of the following
statements about the Markov property is correct?
A. The next state depends only on the current state and action, not
on the history of previous states and actions
B. The next state depends on the entire history of states and actions
C. The reward function must be deterministic
D. The state space must be finite
A. The next state depends only on the current state and action,
not on the history of previous states and actions
RATIONALE: The Markov property states that the future is
independent of the past given the present. The next state and
reward depend only on the current state and action, not on the
sequence of events that preceded it.
Question 2
Which component of an MDP defines the probability of
transitioning from one state to another given an action?
A. Reward function
B. Policy
,C. Transition function
D. Value function
C. Transition function
RATIONALE: The transition function P(s'|s,a) specifies the
probability of moving to state s' when action a is taken in state s.
The reward function defines expected immediate rewards, the
policy maps states to actions, and the value function estimates long-
term return.
Question 3
What is the primary objective of reinforcement learning?
A. To minimize the number of actions taken
B. To maximize the cumulative discounted reward over time
C. To classify states into categories
D. To predict the next state given the current state
B. To maximize the cumulative discounted reward over time
RATIONALE: The reinforcement learning agent's goal is to learn
a policy that maximizes the expected sum of discounted rewards
received over the long run.
Question 4
In a discounted MDP, what is the role of the discount factor γ?
A. It determines the learning rate
B. It controls the trade-off between immediate and future rewards
C. It defines the probability of exploration
D. It measures the accuracy of the value function
, B. It controls the trade-off between immediate and future
rewards
RATIONALE: The discount factor γ ∈ [0,1) determines how
much the agent values future rewards relative to immediate
rewards. A lower γ makes the agent more short-sighted.
Question 5
What does the Bellman Expectation Equation express?
A. The optimal value of a state under any policy
B. The relationship between the value of a state and the values of its
successor states under a given policy
C. The convergence rate of value iteration
D. The probability of reaching a terminal state
B. The relationship between the value of a state and the values of
its successor states under a given policy
RATIONALE: The Bellman Expectation Equation decomposes
the value of a state into the immediate reward plus the discounted
expected value of the next states, following the current policy.
Question 6
Which algorithm is guaranteed to converge to the optimal value
function for a finite MDP?
A. Monte Carlo policy evaluation only
B. Value iteration
C. SARSA with constant learning rate
D. Q-learning with a constant learning rate
B. Value iteration
AND SOLUTIONS (VERIFIED
ANSWERS) PLUS RATIONALES 2026
Q&A | INSTANT DOWNLOAD PDF.
Core Domains
• Markov Decision Processes (MDPs)
• Bellman Equations and Dynamic Programming
• Value Iteration and Policy Iteration
• Q-Learning and Temporal Difference Learning
• Exploration vs. Exploitation Strategies
• SARSA and On-Policy vs. Off-Policy Methods
• Model-Based vs. Model-Free Reinforcement Learning
• Game Theory and Multi-Agent Reinforcement Learning
Introduction
The CS 7641 Reinforcement Learning Unit Quiz assesses graduate-
level understanding of core reinforcement learning concepts within
the Machine Learning curriculum at Georgia Institute of Technology.
This comprehensive assessment evaluates knowledge of Markov
decision processes, dynamic programming algorithms, temporal
difference learning methods, and game-theoretic foundations. The
examination employs multiple-choice and scenario-based questions
requiring applied decision-making skills. Emphasis is placed on
,understanding algorithmic convergence properties, policy evaluation
techniques, and the mathematical foundations underlying modern
reinforcement learning systems. Successful completion demonstrates
readiness to implement and analyze reinforcement learning solutions
in complex real-world environments.
SECTION ONE: QUESTIONS 1–100
Question 1
In a Markov Decision Process (MDP), which of the following
statements about the Markov property is correct?
A. The next state depends only on the current state and action, not
on the history of previous states and actions
B. The next state depends on the entire history of states and actions
C. The reward function must be deterministic
D. The state space must be finite
A. The next state depends only on the current state and action,
not on the history of previous states and actions
RATIONALE: The Markov property states that the future is
independent of the past given the present. The next state and
reward depend only on the current state and action, not on the
sequence of events that preceded it.
Question 2
Which component of an MDP defines the probability of
transitioning from one state to another given an action?
A. Reward function
B. Policy
,C. Transition function
D. Value function
C. Transition function
RATIONALE: The transition function P(s'|s,a) specifies the
probability of moving to state s' when action a is taken in state s.
The reward function defines expected immediate rewards, the
policy maps states to actions, and the value function estimates long-
term return.
Question 3
What is the primary objective of reinforcement learning?
A. To minimize the number of actions taken
B. To maximize the cumulative discounted reward over time
C. To classify states into categories
D. To predict the next state given the current state
B. To maximize the cumulative discounted reward over time
RATIONALE: The reinforcement learning agent's goal is to learn
a policy that maximizes the expected sum of discounted rewards
received over the long run.
Question 4
In a discounted MDP, what is the role of the discount factor γ?
A. It determines the learning rate
B. It controls the trade-off between immediate and future rewards
C. It defines the probability of exploration
D. It measures the accuracy of the value function
, B. It controls the trade-off between immediate and future
rewards
RATIONALE: The discount factor γ ∈ [0,1) determines how
much the agent values future rewards relative to immediate
rewards. A lower γ makes the agent more short-sighted.
Question 5
What does the Bellman Expectation Equation express?
A. The optimal value of a state under any policy
B. The relationship between the value of a state and the values of its
successor states under a given policy
C. The convergence rate of value iteration
D. The probability of reaching a terminal state
B. The relationship between the value of a state and the values of
its successor states under a given policy
RATIONALE: The Bellman Expectation Equation decomposes
the value of a state into the immediate reward plus the discounted
expected value of the next states, following the current policy.
Question 6
Which algorithm is guaranteed to converge to the optimal value
function for a finite MDP?
A. Monte Carlo policy evaluation only
B. Value iteration
C. SARSA with constant learning rate
D. Q-learning with a constant learning rate
B. Value iteration
