Data Science Exam Questions with
100% Verified Answers
EM determines in an iterative process the parameters of a model, so that
they explain the observed data optimally.
For this purpose, the algorithm presumes the presence of unobserved categories.
Alternately it estimates (1) the belonging to one of these categories and (2) the
parameters which determine this cateogry - ✔ ✔ EM (wiki)
› Model-based procedures
• 'Imputation' of missing data + estimation of model in one step
Limitations: model must fit the type of analysis that you plan to execute - ✔ ✔
Full information maximum likelihood (FIML)
Outliers are extreme values, unlike the rest of the sample and with no special
treatment could lead to over-estimates. - ✔ ✔ What is an outlier?
- Use descriptives/ graphical summaries
- Critical value of boxplots:
x is less than Q1 -1.5 x IQR
,x is greater than Q3 + 1.5 x IQR
-> this however is a random definition, not all outliers are included there and not
all values included are outliers - ✔ ✔ How to detect an outlier?
Area between Q1 & Q3 comprises where 50% of the data lies
Q2 is the median of the data, i.e. 50% of values above and below this point
length of the whiskers is max 1.5 IQR, but can be shorter if data doesn't go that
far. If it goes further, data points are shown as points --> "outliers" - ✔ ✔
Boxplot - How to read it
Potential solution: trimming
-> cutting of x% of values(e.g. 5%) on both sides of the distribuition
--> This way getting rid of outliers which exert extreme influence (outlier values
weigh in quadratically) on e.g. the mean
BUT: check if outliers make sense first, maybe there is a reason they are there
(e.g. high sales bcs special promotion or competition ran out of stock) - ✔ ✔
How to deal with outliers?
ML:
More heuristic ("shortcut to find satisfactory, but not necessarily optimal
solution")
Focus on improving performance of a learning agent
+ Predictive analysis
,Supervised and/ or unsupervised learning techniques
Also looks at real time learning (DM not!)
DM:
Integrates theory and heuristics
Focus on entire process of Knowledge Discovery, including data cleaning,
learning, and integration, as well as visualization of results
+ focus on discovery of new patterns (exploratory data analysis)
Unsupervised learning technique
--> Many algorithms can be used for both purposes - ✔ ✔ Distinction ML and
Data Mining:
Exploration & analysis of big data in order to discover meaningful patterns
-> Find patterns which are implicit, novel, valid and potentially useful
• Data Mining is one step in Knowledge Discovery in Databases (KDD) - ✔ ✔
What is Data Mining?
• implicit (by contrast to explicit)
• valid (patterns should be valid on new data)
• novel (novelty can be measured by comparing to expected values)
• potentially useful (should lead to useful actions)
• understandable (to humans) patterns in data - ✔ ✔ Knowledge Discovery in
Databases (KDD) is the non-trivial process of identifying patterns in data, which
are ...
, "Machine Learning is concerned with computer programs that
automatically improve their performance through experience. " - ✔ ✔
What is Machine Learning?
• Develop systems that can automatically adapt and customize themselves to
individual users.
o Personalized news or mail filter
• Discover new knowledge from large databases (data mining).
o Market basket analysis (e.g. diapers and beer)
• Ability to mimic human and replace certain monotonous tasks - which
require some intelligence.
o like recognizing handwritten characters
• Develop systems that are too difficult/expensive to construct manually because
they require specific detailed skills or knowledge tuned to a specific task
(knowledge engineering bottleneck) - ✔ ✔ Why Machine Learning?
Linear SVM:
Maximize the width of a linear classifier (margin) between two support vectors
(extreme data points of different classes, which are closest to each other)
--> looks only at those extreme data points, doesn't really take into account
the other points that lie behind them
Requirement:
Training data necessary, incl. data for which the classes are known to which they
belong to
100% Verified Answers
EM determines in an iterative process the parameters of a model, so that
they explain the observed data optimally.
For this purpose, the algorithm presumes the presence of unobserved categories.
Alternately it estimates (1) the belonging to one of these categories and (2) the
parameters which determine this cateogry - ✔ ✔ EM (wiki)
› Model-based procedures
• 'Imputation' of missing data + estimation of model in one step
Limitations: model must fit the type of analysis that you plan to execute - ✔ ✔
Full information maximum likelihood (FIML)
Outliers are extreme values, unlike the rest of the sample and with no special
treatment could lead to over-estimates. - ✔ ✔ What is an outlier?
- Use descriptives/ graphical summaries
- Critical value of boxplots:
x is less than Q1 -1.5 x IQR
,x is greater than Q3 + 1.5 x IQR
-> this however is a random definition, not all outliers are included there and not
all values included are outliers - ✔ ✔ How to detect an outlier?
Area between Q1 & Q3 comprises where 50% of the data lies
Q2 is the median of the data, i.e. 50% of values above and below this point
length of the whiskers is max 1.5 IQR, but can be shorter if data doesn't go that
far. If it goes further, data points are shown as points --> "outliers" - ✔ ✔
Boxplot - How to read it
Potential solution: trimming
-> cutting of x% of values(e.g. 5%) on both sides of the distribuition
--> This way getting rid of outliers which exert extreme influence (outlier values
weigh in quadratically) on e.g. the mean
BUT: check if outliers make sense first, maybe there is a reason they are there
(e.g. high sales bcs special promotion or competition ran out of stock) - ✔ ✔
How to deal with outliers?
ML:
More heuristic ("shortcut to find satisfactory, but not necessarily optimal
solution")
Focus on improving performance of a learning agent
+ Predictive analysis
,Supervised and/ or unsupervised learning techniques
Also looks at real time learning (DM not!)
DM:
Integrates theory and heuristics
Focus on entire process of Knowledge Discovery, including data cleaning,
learning, and integration, as well as visualization of results
+ focus on discovery of new patterns (exploratory data analysis)
Unsupervised learning technique
--> Many algorithms can be used for both purposes - ✔ ✔ Distinction ML and
Data Mining:
Exploration & analysis of big data in order to discover meaningful patterns
-> Find patterns which are implicit, novel, valid and potentially useful
• Data Mining is one step in Knowledge Discovery in Databases (KDD) - ✔ ✔
What is Data Mining?
• implicit (by contrast to explicit)
• valid (patterns should be valid on new data)
• novel (novelty can be measured by comparing to expected values)
• potentially useful (should lead to useful actions)
• understandable (to humans) patterns in data - ✔ ✔ Knowledge Discovery in
Databases (KDD) is the non-trivial process of identifying patterns in data, which
are ...
, "Machine Learning is concerned with computer programs that
automatically improve their performance through experience. " - ✔ ✔
What is Machine Learning?
• Develop systems that can automatically adapt and customize themselves to
individual users.
o Personalized news or mail filter
• Discover new knowledge from large databases (data mining).
o Market basket analysis (e.g. diapers and beer)
• Ability to mimic human and replace certain monotonous tasks - which
require some intelligence.
o like recognizing handwritten characters
• Develop systems that are too difficult/expensive to construct manually because
they require specific detailed skills or knowledge tuned to a specific task
(knowledge engineering bottleneck) - ✔ ✔ Why Machine Learning?
Linear SVM:
Maximize the width of a linear classifier (margin) between two support vectors
(extreme data points of different classes, which are closest to each other)
--> looks only at those extreme data points, doesn't really take into account
the other points that lie behind them
Requirement:
Training data necessary, incl. data for which the classes are known to which they
belong to
