RL Unit Quiz Practice CS 7641 Questions With Answers Graded A+ Assured Success

Part 1. Policies in Reinforcement Learning (MCMA) You are training an agent to navigate a maze with multiple goal states. The agent uses a stochastic policy that maps states to probability distributions over actions. Which of the following statements about policies are true? Select all that apply. A. A deterministic policy always selects the same action for a given state. B. A stochastic policy can assign non-zero probability to multiple actions in the same state. C. The optimal policy is always unique for any MDP. D. A policy can be evaluated by computing its expected return starting from each possible state. E. The policy improvement theorem guarantees that a better policy can always be found after each iteration. F. A random policy that chooses all actions uniformly is generally optimal for large state spaces. Part 2. Rewards in Reinforcement Learning (MCMA) An agent is learning to maximize total reward in an environment with delayed and sparse rewards. Which of the following statements about rewards are true? Select all that apply. A. Shaping rewards can help the agent learn faster by providing intermediate signals. B. The reward signal directly defines what the agent should optimize for. C. Adding random noise to the reward always improves exploration. D. Sparse rewards can make it more difficult for the agent to learn an optimal policy. 1 E. Discounting future rewards too heavily can cause the agent to ignore long-term consequences. F. The total return is defined as the sum of immediate rewards only, without any consideration of future rewards. 2 Part 3. Bellman Update A robot explorer travels along a straight path with 4 states in sequence: S1 → S2 → S3 → S4 (terminal) The robot has only one action: “Go Forward”. The rewards for each step are: • S1 → S2: +1 • S2 → S3: +2 • S3 → S4: +4 The robot follows a fixed policy: always “Go Forward”. The discount factor is γ = 0.8. Initial value estimates are: Questions V (S1) = V (S2) = V (S3) = V (S4) = 0. a) Write the general Bellman Expectation Equation for this policy. b) Compute the updated value for V (S3). c) Compute the updated value for V (S2). d) Compute the updated value for V (S1). e) If the robot starts in S1, what is its expected total return after this one update step? (Give only your final numerical answer, rounded to two decimal places.) 3 2 Question 2 - Reinforcement Learning Part 1. Q-Learning (MCMA) You are training an agent using Q-Learning in a grid world with discrete states and actions. The agent uses the standard Q-Learning update rule: h Q(s, a) ← Q(s, a) + α r + γ max Q(s′, a′) − Q(s, a) a′ i Which of the following statements about Q-Learning are true? Select all that apply. A. Q-Learning is an off-policy method because it learns the optimal policy independently of the agent’s behavior policy. B. Q-Learning updates require a model of the environment to simulate next states and rewards. C. A high learning rate (α) can make Q-values converge faster but may increase instability. D. If the agent always picks the greedy action, it may fail to visit all states sufficiently often. E. Q-Learning always converges to the optimal Q-values, even with a non-decaying learning rate and purely greedy actions. F. The Q-values represent the expected discounted return when following the behavior policy used during training.