Data Mining Concepts Questions and
All Correct Answers.
What is the meaning of Business Analytics? - Answer - Practice/Art of bringing quantitative
data to bear on decision making.
- Includes BI (Business Intelligence) AND other sophisticated models in order to quantify/explain
relationships between measurements and predicting new records.
How is data mining related to terms such as predictive analytics, predictive modeling, and
machine learning related? - Answer - Predictive Analytics: Form of advanced analytics. Uses
both new/historical data to forecast activity or behavior.
- Predictive Modeling: Places numerical value ~or score~ on the likelihood of a particular event
occurring.
- Machine Learning: Algorithms that learn directly from data, especially local patterns, often in
layered or iterative fashion. Focus is on predicting individual records. (Ex: predicted demand for
per x given a $1 price increase is 1 box, while for person j it is 3 boxes.)
- Data Mining: Stands at the confluence of the fields of statistics and machine
learning..."statistics at a scale and speed"
What does big data mean/include? - Answer - Data today is big when referencing the past
Challenges in Big Data characterized by 4 V's:
- Volume: refers to the amount of data
- Velocity: refers to the flow rate. The speed at which it is being generated and changed
- Variety: refers to the different types of data being generated (currency, dates, numbers, texts)
- Veracity: refers to the fact that data is being generated by organic distributed processes and
not subject to the controls or quality checks that apply to data collected for a study
Why in data mining are there different methods to evaluate a model of interest? - Answer -
Each method has pros/cons. Usefulness of model can depend on factors such as the size of the
dataset, the types of patterns that exist in the data, whether the data meet some underlying
assumptions of the method, how noisy the data are, etc.
- Different methods can lead to different results and their performance can vary.
What are the differences between classification and prediction in Data Mining? - Answer -
Classification = most basic form of data analysis. Typical classification task is to examine data
where the classification is unknown and trying to predict what the classification will end up
being (ex: An applicant for a loan can repay on time, repay late, or declare bankruptcy)
- Prediction = trying to predict the actual value of a # (i.e. amount of purchase rather than
whether there was a purchase or not)
, When are association (affinity analysis) rules useful and how are they performed? - Answer -
Can find general association patterns between items in a large data set
- Examples: Grocery stores can use such information for product placement. Can use the rules
for weekly promotional offers or for bundling products
What is the difference between supervised and unsupervised learning? - Answer -
Supervised = learning algorithms, must have data available in which the value of outcome of
interest is known. Such data are called "labeled data". Includes training, validation, and test.
- Unsupervised = no outcome variable to predict or classify, association rules/dimension
reduction methods, and clustering techniques are all unsupervised learning methods.
Explain the steps in data mining, along with an example at each step. - Answer 1) Develop an
understanding of purpose of the data mining project. (ex: How will the stakeholder use the
results? Who will be affected by the results? Will the analysis be a one-shot effort or an ongoing
procedure?)
2) Obtain the dataset to be used in the analysis. Includes sampling from a large database to
capture records to be used in analysis. (internal ex: past purchases made by customers, external
ex: credit ratings)
3) Explore, clean, and preprocess the data. This step involves verifying that the data are in
reasonable condition. Are there obvious outliers? (ex: "duration" can be computed from start
and end dates).
4) Reduce the data dimension, if necessary. (ex: eliminating unneeded variables, transforming
variables, etc.
5) Determine the data mining task. Translate general question/problem of Step 1 into a more
specific data mining question (ex: classification, prediction, clustering, etc.)
6) Partition the data, if the task is supervised (classification/prediction), randomly partition the
data set into training, validation, and test datasets.
7) Choose the data mining techniques to be used
8) Use algorithms to perform the task. (ex: Iterative process -- trying multiple variants, and often
using multiple variants of the same algorithm. Use feedback from algorithm's performance on
validation data and refine settings)
9) Interpret the results of the algorithms. (ex: Chose best algorithm to deploy, and where
possible, testing the final choice on the test data to get an idea as to how well it will perform.)
10) Deploy the model (ex: model might be applied to a purchased list of possible customers,
and the action might be "include in the mailing if the predicted amount of purchase is > $10.
Key step here is "scoring the new records, or using the chosen model to predict the outcome
value ("score") for each new record
Why is the concept of oversampling a rare event important? - Answer - When an event is
rare (customer purchasing a product in response to a mailing/fraudulent credit card
transactions), we would have a lot of data on non-purchase/non-fraudulent, but not enough
data to base a model that distinguishes purchases from non-purchasers or fraudulent from non-
fraudulent.
All Correct Answers.
What is the meaning of Business Analytics? - Answer - Practice/Art of bringing quantitative
data to bear on decision making.
- Includes BI (Business Intelligence) AND other sophisticated models in order to quantify/explain
relationships between measurements and predicting new records.
How is data mining related to terms such as predictive analytics, predictive modeling, and
machine learning related? - Answer - Predictive Analytics: Form of advanced analytics. Uses
both new/historical data to forecast activity or behavior.
- Predictive Modeling: Places numerical value ~or score~ on the likelihood of a particular event
occurring.
- Machine Learning: Algorithms that learn directly from data, especially local patterns, often in
layered or iterative fashion. Focus is on predicting individual records. (Ex: predicted demand for
per x given a $1 price increase is 1 box, while for person j it is 3 boxes.)
- Data Mining: Stands at the confluence of the fields of statistics and machine
learning..."statistics at a scale and speed"
What does big data mean/include? - Answer - Data today is big when referencing the past
Challenges in Big Data characterized by 4 V's:
- Volume: refers to the amount of data
- Velocity: refers to the flow rate. The speed at which it is being generated and changed
- Variety: refers to the different types of data being generated (currency, dates, numbers, texts)
- Veracity: refers to the fact that data is being generated by organic distributed processes and
not subject to the controls or quality checks that apply to data collected for a study
Why in data mining are there different methods to evaluate a model of interest? - Answer -
Each method has pros/cons. Usefulness of model can depend on factors such as the size of the
dataset, the types of patterns that exist in the data, whether the data meet some underlying
assumptions of the method, how noisy the data are, etc.
- Different methods can lead to different results and their performance can vary.
What are the differences between classification and prediction in Data Mining? - Answer -
Classification = most basic form of data analysis. Typical classification task is to examine data
where the classification is unknown and trying to predict what the classification will end up
being (ex: An applicant for a loan can repay on time, repay late, or declare bankruptcy)
- Prediction = trying to predict the actual value of a # (i.e. amount of purchase rather than
whether there was a purchase or not)
, When are association (affinity analysis) rules useful and how are they performed? - Answer -
Can find general association patterns between items in a large data set
- Examples: Grocery stores can use such information for product placement. Can use the rules
for weekly promotional offers or for bundling products
What is the difference between supervised and unsupervised learning? - Answer -
Supervised = learning algorithms, must have data available in which the value of outcome of
interest is known. Such data are called "labeled data". Includes training, validation, and test.
- Unsupervised = no outcome variable to predict or classify, association rules/dimension
reduction methods, and clustering techniques are all unsupervised learning methods.
Explain the steps in data mining, along with an example at each step. - Answer 1) Develop an
understanding of purpose of the data mining project. (ex: How will the stakeholder use the
results? Who will be affected by the results? Will the analysis be a one-shot effort or an ongoing
procedure?)
2) Obtain the dataset to be used in the analysis. Includes sampling from a large database to
capture records to be used in analysis. (internal ex: past purchases made by customers, external
ex: credit ratings)
3) Explore, clean, and preprocess the data. This step involves verifying that the data are in
reasonable condition. Are there obvious outliers? (ex: "duration" can be computed from start
and end dates).
4) Reduce the data dimension, if necessary. (ex: eliminating unneeded variables, transforming
variables, etc.
5) Determine the data mining task. Translate general question/problem of Step 1 into a more
specific data mining question (ex: classification, prediction, clustering, etc.)
6) Partition the data, if the task is supervised (classification/prediction), randomly partition the
data set into training, validation, and test datasets.
7) Choose the data mining techniques to be used
8) Use algorithms to perform the task. (ex: Iterative process -- trying multiple variants, and often
using multiple variants of the same algorithm. Use feedback from algorithm's performance on
validation data and refine settings)
9) Interpret the results of the algorithms. (ex: Chose best algorithm to deploy, and where
possible, testing the final choice on the test data to get an idea as to how well it will perform.)
10) Deploy the model (ex: model might be applied to a purchased list of possible customers,
and the action might be "include in the mailing if the predicted amount of purchase is > $10.
Key step here is "scoring the new records, or using the chosen model to predict the outcome
value ("score") for each new record
Why is the concept of oversampling a rare event important? - Answer - When an event is
rare (customer purchasing a product in response to a mailing/fraudulent credit card
transactions), we would have a lot of data on non-purchase/non-fraudulent, but not enough
data to base a model that distinguishes purchases from non-purchasers or fraudulent from non-
fraudulent.
