Lecture 1 26-10-2021
5V’s of Big Data ➔ Volume, Variety, Velocity, Plus Value, Veracity
● Impose technical and managerial challenges
● Also represent opportunities and threats for the business
Business value
In a recent report, Nucleus Research found that for every dollar a company spends on
analytics, it gets back $10.66
Tom Davenport
● Peter Drucker, who noted almost 20 years ago that one of the risks of corporate IT
lay in the fact that because it relies only on internal data, it encourages a
degenerative tendency … to focus inward on costs and efforts, rather than outward
on opportunities, changes, and threats. One of the great advances of big data is that
so much of the data now comes from external sources
● Of all you can do with big data, developing new products and services is the most
valuable
Caveat 1: Big Data and Thick Data
“There are many reasons for Nokia’s downfall, but one of the biggest reasons that I
witnessed in person was that the company over-relied on numbers. They put a higher value
on quantitative data, they didn’t know how to handle data that wasn’t easily measurable, and
that didn’t show up in existing reports.” - Tricia Wang 2016
Message:
● Beware of quant bias or quant addiction
● Big data in many cases needs to be supported with thick data
● Thick data (emotion, context, meaning,...)
○ Ethnography - people’s way of living, culture
● Improve the use of big data or analytics by seeing the whole picture of decision
making
Caveat 2: Relate to business challenges
Data monetization strategy
● Asset sale
● Business process improvement
● Product / service optimization
● Data insights sale
● Contextualization
● Individualization
● Build and strengthen customer relationship
● Strategically opening data
● Data enrichment
● Data bartering
● Data privacy and control guarantee
1
,Why big data now?
● Availability of massive amount of digital data
● Combination of technical developments and societal needs
● A philosophical view
○ Rationalism vs empiricism
● The discovery of the power of data
Why big data now?: technological developments
Radical changes in:
● The way elementary data are captured
○ Sensors (automated) vs keyboard (human)
● The way data is stored
○ Main memory and cloud vs hard disk
● The way data is analyzed
○ Data-driven methods vs sampling
● The way data is provided to users
○ Data logistics vs data integration
● The way data is presented
○ Graphical interactive visualizations vs management reports
● The way knowledge (business rules, models) is created
○ Learning / mining vs (labor-intensive) knowledge acquisition
Why big data now: the researcher’s view
More data is more important than better algorithms (Michele Banko & Eric Brill, Microsoft)
From Big Data to AI
Data became the problem, the problem empowered the solution.
The new AI technologies are driven by data and analytics.
Machine Learning is at the forefront of most of them.
2
,Three types of learning
But we don’t solve problems with Machine Learning.
We solve problems with the rules and knowledge that ML builds.
Each step supports the next
Moving AI tools to the use context (Lebovitz et al, 2021)
● We conducted an 11-month qualitative study investigating how managers formed
evaluations or AI tools across multiple sections of a department of diagnostic
radiology.
● As each study concluded, managers were quite surprised by the internal results, as
many results conflicted with the accuracy measures reported by tool developers.
● Managers began scrutinizing how ground truth labels were defined and generated,
extending their earlier focus on assessing who generated the labels.
● The readers were not looking at prior images. It was done intentionally, to keep it,
you know, apples-to-apples, a controlled way to do the study. But it’s not even close
to real practice.
Data & Information
3
, Business analytics is about data-driven decision making and data science.
Definitions (Provost)
● Data science is a set of fundamental principles that support and guide the principled
extraction of information and knowledge from data
○ Sometimes referred to as Applied AI
● Data mining is the actual extraction of knowledge from data, via technologies that
incorporate these principles
● Data-driven Decision Making (DDD) is the practice of basing decisions on the
analysis of data, rather than purely on intuition
Data science principles
● Entities that are similar with respect to known
features or attributes often are similar with
respect to unknown features or attributes
● Deal with missing information as far as it goes
○ Cf. old closed-world view in traditional
database: not in DB, then false
● Extracting useful knowledge from data to solve
business problems can be treated
systematically by following a process with
reasonable well-defined stages. The
Cross-Industry Standard Process for Data
Mining (CRISP-DM7) is one codification of this
process
● If you look too hard at a set of data, you will find something - but it might not
generalize beyond the data you’re looking at (problem of overfitting)
● To draw causal conclusions, one must pay very close
attention to the presence of confounding factors, possibly
unseen ones (observation vs intervention)
● When using AI heuristics to find some optimum you may
end up in a local maximum
● The relationship between the business problem and the
analytics solution often can be decomposed into tractable subproblems via the
framework of analyzing expected value (ex. Machine maintenance)
Business analytics not only about numbers
● For business, often the purpose of data analysis is to improve business processes
● Business process data consists of events
○ Process mining
● Many systems can be represented by graphs
○ Social network mining
○ Knowledge graphs
4
5V’s of Big Data ➔ Volume, Variety, Velocity, Plus Value, Veracity
● Impose technical and managerial challenges
● Also represent opportunities and threats for the business
Business value
In a recent report, Nucleus Research found that for every dollar a company spends on
analytics, it gets back $10.66
Tom Davenport
● Peter Drucker, who noted almost 20 years ago that one of the risks of corporate IT
lay in the fact that because it relies only on internal data, it encourages a
degenerative tendency … to focus inward on costs and efforts, rather than outward
on opportunities, changes, and threats. One of the great advances of big data is that
so much of the data now comes from external sources
● Of all you can do with big data, developing new products and services is the most
valuable
Caveat 1: Big Data and Thick Data
“There are many reasons for Nokia’s downfall, but one of the biggest reasons that I
witnessed in person was that the company over-relied on numbers. They put a higher value
on quantitative data, they didn’t know how to handle data that wasn’t easily measurable, and
that didn’t show up in existing reports.” - Tricia Wang 2016
Message:
● Beware of quant bias or quant addiction
● Big data in many cases needs to be supported with thick data
● Thick data (emotion, context, meaning,...)
○ Ethnography - people’s way of living, culture
● Improve the use of big data or analytics by seeing the whole picture of decision
making
Caveat 2: Relate to business challenges
Data monetization strategy
● Asset sale
● Business process improvement
● Product / service optimization
● Data insights sale
● Contextualization
● Individualization
● Build and strengthen customer relationship
● Strategically opening data
● Data enrichment
● Data bartering
● Data privacy and control guarantee
1
,Why big data now?
● Availability of massive amount of digital data
● Combination of technical developments and societal needs
● A philosophical view
○ Rationalism vs empiricism
● The discovery of the power of data
Why big data now?: technological developments
Radical changes in:
● The way elementary data are captured
○ Sensors (automated) vs keyboard (human)
● The way data is stored
○ Main memory and cloud vs hard disk
● The way data is analyzed
○ Data-driven methods vs sampling
● The way data is provided to users
○ Data logistics vs data integration
● The way data is presented
○ Graphical interactive visualizations vs management reports
● The way knowledge (business rules, models) is created
○ Learning / mining vs (labor-intensive) knowledge acquisition
Why big data now: the researcher’s view
More data is more important than better algorithms (Michele Banko & Eric Brill, Microsoft)
From Big Data to AI
Data became the problem, the problem empowered the solution.
The new AI technologies are driven by data and analytics.
Machine Learning is at the forefront of most of them.
2
,Three types of learning
But we don’t solve problems with Machine Learning.
We solve problems with the rules and knowledge that ML builds.
Each step supports the next
Moving AI tools to the use context (Lebovitz et al, 2021)
● We conducted an 11-month qualitative study investigating how managers formed
evaluations or AI tools across multiple sections of a department of diagnostic
radiology.
● As each study concluded, managers were quite surprised by the internal results, as
many results conflicted with the accuracy measures reported by tool developers.
● Managers began scrutinizing how ground truth labels were defined and generated,
extending their earlier focus on assessing who generated the labels.
● The readers were not looking at prior images. It was done intentionally, to keep it,
you know, apples-to-apples, a controlled way to do the study. But it’s not even close
to real practice.
Data & Information
3
, Business analytics is about data-driven decision making and data science.
Definitions (Provost)
● Data science is a set of fundamental principles that support and guide the principled
extraction of information and knowledge from data
○ Sometimes referred to as Applied AI
● Data mining is the actual extraction of knowledge from data, via technologies that
incorporate these principles
● Data-driven Decision Making (DDD) is the practice of basing decisions on the
analysis of data, rather than purely on intuition
Data science principles
● Entities that are similar with respect to known
features or attributes often are similar with
respect to unknown features or attributes
● Deal with missing information as far as it goes
○ Cf. old closed-world view in traditional
database: not in DB, then false
● Extracting useful knowledge from data to solve
business problems can be treated
systematically by following a process with
reasonable well-defined stages. The
Cross-Industry Standard Process for Data
Mining (CRISP-DM7) is one codification of this
process
● If you look too hard at a set of data, you will find something - but it might not
generalize beyond the data you’re looking at (problem of overfitting)
● To draw causal conclusions, one must pay very close
attention to the presence of confounding factors, possibly
unseen ones (observation vs intervention)
● When using AI heuristics to find some optimum you may
end up in a local maximum
● The relationship between the business problem and the
analytics solution often can be decomposed into tractable subproblems via the
framework of analyzing expected value (ex. Machine maintenance)
Business analytics not only about numbers
● For business, often the purpose of data analysis is to improve business processes
● Business process data consists of events
○ Process mining
● Many systems can be represented by graphs
○ Social network mining
○ Knowledge graphs
4
