• Wrong document? Swap it for free
  • Written by students who passed
  • Immediately available after payment
  • Read online or as PDF
Sell
Where do you study
Your language
Document preview thumbnail
Preview 2 out of 7 pages
Other

Cs7641 Final Exam Verified Study Guide

Document preview thumbnail
Preview 2 out of 7 pages

CS7641 FINAL EXAM VERIFIED STUDY GUIDE

Content preview

CS7641 FINAL EXAM | VERIFIED STUDY GUIDE

Four optimization approaches - Answers - 1) Generate and test
2) Calculus
3) Newton's Method
4) Randomized Optimization

Hill Climbing Algorithm - Answers - Guess x∈X
Repeat the following:
Let n*=argmax_n∈N(x) f(n)
If f(n)>f(x): x=n
Else: stop

Disadvantage:
- Get stuck in local optima

Randomized Restart Hill Climbing - Answers - Same as Hill Climbing but once local
optimum reached, restart again with a different starting x

Advantage:
- Won't get stuck in local optimum
- Not much more expensive than HC (constant factor)
Disadvantage:
- May not do better than enumeration (depends on size of attraction basin around
global optimum)

Entropy - Answers - -∑p(s)log₂p(s)

Number of bits per symbol (probability of symbol X # of bits to describe that symbol)

Joint Entropy - Answers - H(x,y)=-∑p(x,y)log₂p(x,y)

Randomness contained in two variables together

Conditional Entropy - Answers - H(y|x)=-∑p(x,y)log₂p(y|x)

Randomness of one variable given the other variable

Entropy if x and y are independent - Answers - H(Y|X)=H(Y) Y doesn't get any info from
x
H(X,Y)=H(X)+H(Y) Joint entropy is sum

Mutual Information - Answers - I(x,y)=H(y)-H(y|x)=I(y,x)

, Measure of reduction of randomness of variable given some knowledge of another
variable.

Specific case of KL Divergence

Kullback-Leibler Divergence - Answers - Always non-negative
Zero when P is equal to Q
Measures distance between any two distributions

Supervised Learning - Answers - Use labeled training data to generalize labels to new
instances (function approximation)

Unsupervised Learning - Answers - Make sense out of unlabeled data (data description)

Single Linkage Clustering (SLC) Algorithm - Answers - - Consider each object a cluster
(n objects)
- Define intercluster distance as distance between closest two points in the two clusters
- Merge 2 closest clusters
- Repeat n-k times to make k clusters

Notes:
- Running time O(n²)x(~n)

k-Means Clustering Algorithm - Answers - - Pick k centers at random
- Each center claims closest points
- Recompute the centers by averaging the clustered points
- Repeat until convergence

Notes:
- Can get stuck

Expectation Maximization - Answers - Expectation - soft clustering:
E[Zᵢⱼ]=(P(x=xᵢ|µ=µⱼ)/(∑ᵏP(x=xᵢ|µ=µⱼ)
"Likelihood data element i comes from cluster j"

Maximization - calculating meas of clusters:
µⱼ=(∑E[Zᵢⱼ]xᵢ)/(∑E[Zᵢⱼ])

P(x=xᵢ|µ=µⱼ)=exp[-½σ²(xᵢ-µⱼ)²]

Properties:
- Monotonically non-decreasing likelihood
- Does not converge
- Will not diverge
- Can get stuck -> overcame with randomization
- Works with any distribution

Document information

Uploaded on
April 15, 2026
Number of pages
7
Written in
2025/2026
Type
Other
Person
Unknown
$13.99

Wrong document? Swap it for free Within 14 days of purchase and before downloading, you can choose a different document. You can simply spend the amount again.
Written by students who passed
Immediately available after payment
Read online or as PDF

Seller avatar
Reputation scores are based on the amount of documents a seller has sold for a fee and the reviews they have received for those documents. There are three levels: Bronze, Silver and Gold. The better the reputation, the more your can rely on the quality of the sellers work.
GEEKA
3.8
(379)
Sold
2239
Followers
1451
Items
63563
Last sold
20 hours ago




Why students choose Stuvia

Created by fellow students, verified by reviews

Quality you can trust: written by students who passed their tests and reviewed by others who've used these notes.

Didn't get what you expected? Choose another document

No worries! You can instantly pick a different document that better fits what you're looking for.

Pay as you like, start learning right away

No subscription, no commitments. Pay the way you're used to via credit card and download your PDF document instantly.

Student with book image

“Bought, downloaded, and aced it. It really can be that simple.”

Alisha Student

Working on your references?

Create accurate citations in APA, MLA and Harvard with our free citation generator.

Working on your references?

Frequently asked questions