CS7641 FINAL EXAM
75+ (Latest 2025-2026 Edition) 100% Verified Q&A + Answer Key Solutions
Question 1
Four optimization approaches
Correct!
1) Generate and test
2) Calculus
3) Newton's Method
4) Randomized Optimization
Question 2
Hill Climbing Algorithm
Correct!
Guess x∈X
Repeat the following:
Let n*=argmax_n∈N(x) f(n)
If f(n)>f(x): x=n
Else: stop
Disadvantage:
- Get stuck in local optima
Question 3
Randomized Restart Hill Climbing
Correct!
Same as Hill Climbing but once local optimum reached, restart again with a different starting x
Advantage:
- Won't get stuck in local optimum
- Not much more expensive than HC (constant factor)
Disadvantage:
- May not do better than enumeration (depends on size of attraction basin around global optimum)
, Question 4
Entropy
Correct!
-∑p(s)log₂p(s)
Number of bits per symbol (probability of symbol X # of bits to describe that symbol)
Question 5
Joint Entropy
Correct!
H(x,y)=-∑p(x,y)log₂p(x,y)
Randomness contained in two variables together
Question 6
Conditional Entropy
Correct!
H(y|x)=-∑p(x,y)log₂p(y|x)
Randomness of one variable given the other variable
Question 7
Entropy if x and y are independent
Correct!
H(Y|X)=H(Y) Y doesn't get any info from x
H(X,Y)=H(X)+H(Y) Joint entropy is sum
, Question 8
Mutual Information
Correct!
I(x,y)=H(y)-H(y|x)=I(y,x)
Measure of reduction of randomness of variable given some knowledge of another variable.
Specific case of KL Divergence
Question 9
Kullback-Leibler Divergence
Correct!
Always non-negative
Zero when P is equal to Q
Measures distance between any two distributions
Question 10
Supervised Learning
Correct!
Use labeled training data to generalize labels to new instances (function approximation)
Question 11
Unsupervised Learning
Correct!
Make sense out of unlabeled data (data description)
75+ (Latest 2025-2026 Edition) 100% Verified Q&A + Answer Key Solutions
Question 1
Four optimization approaches
Correct!
1) Generate and test
2) Calculus
3) Newton's Method
4) Randomized Optimization
Question 2
Hill Climbing Algorithm
Correct!
Guess x∈X
Repeat the following:
Let n*=argmax_n∈N(x) f(n)
If f(n)>f(x): x=n
Else: stop
Disadvantage:
- Get stuck in local optima
Question 3
Randomized Restart Hill Climbing
Correct!
Same as Hill Climbing but once local optimum reached, restart again with a different starting x
Advantage:
- Won't get stuck in local optimum
- Not much more expensive than HC (constant factor)
Disadvantage:
- May not do better than enumeration (depends on size of attraction basin around global optimum)
, Question 4
Entropy
Correct!
-∑p(s)log₂p(s)
Number of bits per symbol (probability of symbol X # of bits to describe that symbol)
Question 5
Joint Entropy
Correct!
H(x,y)=-∑p(x,y)log₂p(x,y)
Randomness contained in two variables together
Question 6
Conditional Entropy
Correct!
H(y|x)=-∑p(x,y)log₂p(y|x)
Randomness of one variable given the other variable
Question 7
Entropy if x and y are independent
Correct!
H(Y|X)=H(Y) Y doesn't get any info from x
H(X,Y)=H(X)+H(Y) Joint entropy is sum
, Question 8
Mutual Information
Correct!
I(x,y)=H(y)-H(y|x)=I(y,x)
Measure of reduction of randomness of variable given some knowledge of another variable.
Specific case of KL Divergence
Question 9
Kullback-Leibler Divergence
Correct!
Always non-negative
Zero when P is equal to Q
Measures distance between any two distributions
Question 10
Supervised Learning
Correct!
Use labeled training data to generalize labels to new instances (function approximation)
Question 11
Unsupervised Learning
Correct!
Make sense out of unlabeled data (data description)
