1BVK00: Business Analytics & Decision Support
Lecture 1: Data analytical thinking (Chapter 1&2)
Types of decisions
• Strategical: unstructured, one-time employee levels, industry trends, rebranding
• Tactical: semi-structured, reporting forecasts, pricing, profitability
• Operational: structured, recurrent scheduling, order processing
Data science: interdisciplinary fields using variety of techniques to create value based on
extracting knowledge
• Extracting useful/valuable knowledge to solve business problems in a systematic way of
well-defined stages
- requires good understanding of application domain
- considers ethics, business models, human behaviour
• CRISP-DM methodology: Cross Industry Standard Process for Data Mining
- Dependency to context
- Finding informative (statistical) attributes
- Generalizing beyond the available data
1. Business Understanding: 4. Modeling:
- Business objectives - Select modeling techniques
- Success criteria (KPI) - Build/train model
- Project plan - Prediction
- Deliverables
2. Data Understanding 5. Evaluation:
- Model validation
- Initial data collection
- Data description - Performance metrics
- Data Exploration - Visualization
3. Data preparation - Review results
6. Deployment:
- Data cleaning
- Model in production
- Sampling
- Normalization
- Feature Selection
,Data mining tasks
• Classification: Determine which discrete category the example is
• Regression: attempts to estimate or predict, for each individual, the numerical value of
some variable for that individual.
• Clustering: attempts to group individuals in a population together by their similarity, but
not driven by any specific purpose.
• Similarity matching: attempts to identify similar individuals based on data known about
them.
• Co-occurrence grouping: attempts to find associations between entities based on
transactions involving them.
• Profiling: attempts to characterize the typical behavior of an individual, group, or
population.
• Link prediction: attempts to predict connections between data items, usually by
suggesting that a link should exist, and possibly also estimating the strength of the link.
Lecture 2: Business Problems & Data Science Solutions (Chapter 2&3&4)
• Unsupervised learning: there is no specific target
• Supervised learning: there exists a specific target
,Target attribute (feature) types:
• Classification: Categorical
• Regression: Numerical (continuous/discrete)
Examples:
“Will this customer purchase service S1 if given incentive I?”
Target: ‘’Will purchase’’, ‘’Will not purchase’’.
→ Classification
“Which service package (S1, S2, or none) will a customer purchase if given incentive I?” Target set: {S1, S2, or
none}
→ (Multi-class) classification
“How much will this customer use the service?”
Target: Service use, e.g., in GB
→ Regression
“What is the probability that the customer will continue?”
Target: Likelihood value of continuation of the customer
→ Regression
• Data mining:
1. Mining the data to find patterns and build models
2. Using the results of data mining
Detailed roadmap for
building ML models:
, Issues affecting data quality:
- Missing values - Attribute dependencies
- Invalid values - Formats
- Misfielded values - Uniqueness
- Misspellings
How to detect these issues?
- Visualization: Visualizing all the values of each feature or taking a random sample to
see if it’s right.
- Outlier analysis: Analyzing if data can be a human error. E.g. a 300 year old person
in the “age” feature.
- Validation code: It’s possible to create a code that checks if the data is right. E.g., in
uniqueness, checking if the length of the data is the same as the length of the vector
of unique values.
Major tasks for preparing good dataset:
• Dealing with missing data
1. Ignore records (use only cases with all values)
- Not effective when the percentage of missing values per attribute varies
considerably as it can lead to insufficient and/or biased sample sizes
2. Ignore attributes with missing values
- Use only features (attributes) with all values (may leave out important features)
3. Use a global constant to fill in the missing value
- e.g., “unknown”. (May create a new class!) Building good datasets
4. Use the attribute mean to fill in the missing value
5. Use the attribute median or mode to fill in the missing value
6. Many other techniques.
• Handling categorical data
- Represented as strings or categories and are in finite numbers
- Ordinal Data: The categories have an inherent order
- Nominal Data: The categories do not have an inherent order
• Building features onto the same scale
- Feature scaling is a crucial step in data preprocessing
- Feature scaling is useful when features values highly vary in magnitudes, units and
range such as age, salary, weight, etc.
- Gradient decent and distance based methods behave much better if features are on
the same scale
- Tree based methods (e.g., Decision tree, Random forest) are invariant to feature
scaling
Lecture 1: Data analytical thinking (Chapter 1&2)
Types of decisions
• Strategical: unstructured, one-time employee levels, industry trends, rebranding
• Tactical: semi-structured, reporting forecasts, pricing, profitability
• Operational: structured, recurrent scheduling, order processing
Data science: interdisciplinary fields using variety of techniques to create value based on
extracting knowledge
• Extracting useful/valuable knowledge to solve business problems in a systematic way of
well-defined stages
- requires good understanding of application domain
- considers ethics, business models, human behaviour
• CRISP-DM methodology: Cross Industry Standard Process for Data Mining
- Dependency to context
- Finding informative (statistical) attributes
- Generalizing beyond the available data
1. Business Understanding: 4. Modeling:
- Business objectives - Select modeling techniques
- Success criteria (KPI) - Build/train model
- Project plan - Prediction
- Deliverables
2. Data Understanding 5. Evaluation:
- Model validation
- Initial data collection
- Data description - Performance metrics
- Data Exploration - Visualization
3. Data preparation - Review results
6. Deployment:
- Data cleaning
- Model in production
- Sampling
- Normalization
- Feature Selection
,Data mining tasks
• Classification: Determine which discrete category the example is
• Regression: attempts to estimate or predict, for each individual, the numerical value of
some variable for that individual.
• Clustering: attempts to group individuals in a population together by their similarity, but
not driven by any specific purpose.
• Similarity matching: attempts to identify similar individuals based on data known about
them.
• Co-occurrence grouping: attempts to find associations between entities based on
transactions involving them.
• Profiling: attempts to characterize the typical behavior of an individual, group, or
population.
• Link prediction: attempts to predict connections between data items, usually by
suggesting that a link should exist, and possibly also estimating the strength of the link.
Lecture 2: Business Problems & Data Science Solutions (Chapter 2&3&4)
• Unsupervised learning: there is no specific target
• Supervised learning: there exists a specific target
,Target attribute (feature) types:
• Classification: Categorical
• Regression: Numerical (continuous/discrete)
Examples:
“Will this customer purchase service S1 if given incentive I?”
Target: ‘’Will purchase’’, ‘’Will not purchase’’.
→ Classification
“Which service package (S1, S2, or none) will a customer purchase if given incentive I?” Target set: {S1, S2, or
none}
→ (Multi-class) classification
“How much will this customer use the service?”
Target: Service use, e.g., in GB
→ Regression
“What is the probability that the customer will continue?”
Target: Likelihood value of continuation of the customer
→ Regression
• Data mining:
1. Mining the data to find patterns and build models
2. Using the results of data mining
Detailed roadmap for
building ML models:
, Issues affecting data quality:
- Missing values - Attribute dependencies
- Invalid values - Formats
- Misfielded values - Uniqueness
- Misspellings
How to detect these issues?
- Visualization: Visualizing all the values of each feature or taking a random sample to
see if it’s right.
- Outlier analysis: Analyzing if data can be a human error. E.g. a 300 year old person
in the “age” feature.
- Validation code: It’s possible to create a code that checks if the data is right. E.g., in
uniqueness, checking if the length of the data is the same as the length of the vector
of unique values.
Major tasks for preparing good dataset:
• Dealing with missing data
1. Ignore records (use only cases with all values)
- Not effective when the percentage of missing values per attribute varies
considerably as it can lead to insufficient and/or biased sample sizes
2. Ignore attributes with missing values
- Use only features (attributes) with all values (may leave out important features)
3. Use a global constant to fill in the missing value
- e.g., “unknown”. (May create a new class!) Building good datasets
4. Use the attribute mean to fill in the missing value
5. Use the attribute median or mode to fill in the missing value
6. Many other techniques.
• Handling categorical data
- Represented as strings or categories and are in finite numbers
- Ordinal Data: The categories have an inherent order
- Nominal Data: The categories do not have an inherent order
• Building features onto the same scale
- Feature scaling is a crucial step in data preprocessing
- Feature scaling is useful when features values highly vary in magnitudes, units and
range such as age, salary, weight, etc.
- Gradient decent and distance based methods behave much better if features are on
the same scale
- Tree based methods (e.g., Decision tree, Random forest) are invariant to feature
scaling
