H1: Inleiding
Machine learning
- Supervised Learning
o Algorithms = step by step procedures to find good “model”
o Needs lots of examples described by measurable characteristics
o Model optimally consistent
o Model can make predictions for new samples
- Reinforcement Learning: Intern model adaptation
Vooruitgang
- New powerful hardware
- GPU = Graphics Processing Unit
o Initially: graphics of games
o Useful: Bitcoin mining, training neural networks
Neural networks
-
- Training Neural Network expensive
o Huge data
o Enormous computations
o Alignment by reinforcement learning with human feedback
- Using Neural Network
o Cheap
o Requires orders of magnitude fewer computations
- Open source models can be fine-tuned
- Closed source model used as service
1
,Terms
- Closed-source
o Model only accessible through API
o Send request answer computed on serves of model owner answer send back
o Potential privacy problems
- Open-source
o Weights (trained model) is published
o Everyone can run model themselves
o No need to send data, no privacy issues
o Significant compute infrastructure required
- Program to train models is usually known/obvious
Legislation EU
- European commission adopted “AI Act”
- 3 levels of AI applications
o Unacceptable risk forbidden
e.g. social scoring, large-scale biometrical identification
o High risk regulated
e.g. employment, education, law enforcement
o Limited/minimal risk transparency/self-regulated
e.g. chatbot/games/spam filter
2
,H2: Fairness and bias in AI
Promise of Artificial Intelligence
- Super-human performance
- Not hindered by cultural stereotypes
- However
o Tay the Chatbot started out fine with innocent statements quickly got out of
control
o AI may unintentionally pick up & amplify bias from observations leading to
unintended effects
o Particularly problematic as models are very complex & predictions cannot always be
explained
- But humans discriminate to?!
o Scale totally different
o Use of automation in decision procedure offers opportunities
Sources of bias
Historical discrimination: stereotypes
- Amazon (2015): algorithm for hiring biased against women
o Based on # resumes submitted over past 10y
o Most applicants were men trained to favour men > women
- Most language processing transforms data first into numbers: transformation methods
widely available
- AI will learn model to predict word
o Structure of model is fixed
o Becomes optimization problem
o Easy to generate test data
o Only interested in part of model that captures semantic info in vector
- Embeddings capture semantic information but also capture cultural biases
- Hard to fix
Label bias, measurement bias, selection bias
- 2019: predict which patients would likely need extra medical care, identify which patients
will benefit from “high-risk care management” programs: access to specially trained nurses,
heavily favoured white patients over black patients (race wasn’t variable)
- Race wasn’t variable but healthcare cost history was: black patients incurred lower health-
care costs than white patients with same conditions
Aggregation bias, under- and overfitting
- Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) tool to
predict risk of recidivism
o Label: was there new arrest within 2 years
o Data: pending charges, prior arrest history, previous pretrial failure, residential
stability, substance abuse …
o 44.9 of African Americans that do not re-offend are misclassified as high-risk
3
, How to measure bias?
- Statistical parity: positive label equally distributed across groups
- Equal odds: compare errors between ground: TPR (true pos rate) & FPR (false pos rate)
- Calibration: interpretation of label does not depend on group
- 2 different things are measured
o No ethnic group should be disproportionally affected by errors in system
o Interpretation of label should be independent of ethnicity of person
- Only way to satisfy both conditions: perfect predictability, equal base rates
- In reasonable circumstances it is impossible to satisfy both conditions simultaneously
How to avoid bias?
- Decision should not be based directly on sensitive attributes does not work: we can still
build models based on correlated attributes = redlining
- Fairness interventions denote algorithmic solutions that ensure fairness by design: models
are constrained by fairness measure (sacrifice accuracy for fairness)
- Provable optimal classifier makes mistakes on purpose
- Not straightforward to measure
o Obtain high-quality data, gain control of data gathering process, counter biases
o Measure: reinforcement of stereotypes / unfairness should not be hidden deeply in
model
o Understand: what is source of disparate impact? Can it be explained? Is explanation
acceptable
o Involve right stakeholders: data scientists should not be the ones deciding which
error level is acceptable
Fairness audits
- Study differences in tax audit rates by Internal Revenue Service (IRS) between black & white
tax payers
- Some issues
o Incomplete data (only selected samples were audited)
o No information on ethnicity stored
- Predict ethnicity based on name & postal code
- Black community audited at much higher rate than general population:
o Audit rates for blacks: 0.84 – 1.34 percentage points higher than for non-blacks
o Base audit rate is only 0.54%
o blacks 2.9 – 4.7 x more likely to be audited
- Black taxpayers claiming EITC (Earned income tax credit) are between 2.9 – 4.4 x more likely
to be audited than non-black taxpayers
- Objective that is optimized influences disparity
o Maximize number of fraud cases detected
o Maximize dollars retrieved (would give opposite result)
o Focus on underreported income
o Focus on unjustified refunds
- Bias measure:
o Measure disparity of errors between groups
o Monitor differences in acceptance rates between groups
- Use bias measures as KPIs: follow up on these KPIs when model deployed
- Understand sources bias: make policy decisions at right level
4
Machine learning
- Supervised Learning
o Algorithms = step by step procedures to find good “model”
o Needs lots of examples described by measurable characteristics
o Model optimally consistent
o Model can make predictions for new samples
- Reinforcement Learning: Intern model adaptation
Vooruitgang
- New powerful hardware
- GPU = Graphics Processing Unit
o Initially: graphics of games
o Useful: Bitcoin mining, training neural networks
Neural networks
-
- Training Neural Network expensive
o Huge data
o Enormous computations
o Alignment by reinforcement learning with human feedback
- Using Neural Network
o Cheap
o Requires orders of magnitude fewer computations
- Open source models can be fine-tuned
- Closed source model used as service
1
,Terms
- Closed-source
o Model only accessible through API
o Send request answer computed on serves of model owner answer send back
o Potential privacy problems
- Open-source
o Weights (trained model) is published
o Everyone can run model themselves
o No need to send data, no privacy issues
o Significant compute infrastructure required
- Program to train models is usually known/obvious
Legislation EU
- European commission adopted “AI Act”
- 3 levels of AI applications
o Unacceptable risk forbidden
e.g. social scoring, large-scale biometrical identification
o High risk regulated
e.g. employment, education, law enforcement
o Limited/minimal risk transparency/self-regulated
e.g. chatbot/games/spam filter
2
,H2: Fairness and bias in AI
Promise of Artificial Intelligence
- Super-human performance
- Not hindered by cultural stereotypes
- However
o Tay the Chatbot started out fine with innocent statements quickly got out of
control
o AI may unintentionally pick up & amplify bias from observations leading to
unintended effects
o Particularly problematic as models are very complex & predictions cannot always be
explained
- But humans discriminate to?!
o Scale totally different
o Use of automation in decision procedure offers opportunities
Sources of bias
Historical discrimination: stereotypes
- Amazon (2015): algorithm for hiring biased against women
o Based on # resumes submitted over past 10y
o Most applicants were men trained to favour men > women
- Most language processing transforms data first into numbers: transformation methods
widely available
- AI will learn model to predict word
o Structure of model is fixed
o Becomes optimization problem
o Easy to generate test data
o Only interested in part of model that captures semantic info in vector
- Embeddings capture semantic information but also capture cultural biases
- Hard to fix
Label bias, measurement bias, selection bias
- 2019: predict which patients would likely need extra medical care, identify which patients
will benefit from “high-risk care management” programs: access to specially trained nurses,
heavily favoured white patients over black patients (race wasn’t variable)
- Race wasn’t variable but healthcare cost history was: black patients incurred lower health-
care costs than white patients with same conditions
Aggregation bias, under- and overfitting
- Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) tool to
predict risk of recidivism
o Label: was there new arrest within 2 years
o Data: pending charges, prior arrest history, previous pretrial failure, residential
stability, substance abuse …
o 44.9 of African Americans that do not re-offend are misclassified as high-risk
3
, How to measure bias?
- Statistical parity: positive label equally distributed across groups
- Equal odds: compare errors between ground: TPR (true pos rate) & FPR (false pos rate)
- Calibration: interpretation of label does not depend on group
- 2 different things are measured
o No ethnic group should be disproportionally affected by errors in system
o Interpretation of label should be independent of ethnicity of person
- Only way to satisfy both conditions: perfect predictability, equal base rates
- In reasonable circumstances it is impossible to satisfy both conditions simultaneously
How to avoid bias?
- Decision should not be based directly on sensitive attributes does not work: we can still
build models based on correlated attributes = redlining
- Fairness interventions denote algorithmic solutions that ensure fairness by design: models
are constrained by fairness measure (sacrifice accuracy for fairness)
- Provable optimal classifier makes mistakes on purpose
- Not straightforward to measure
o Obtain high-quality data, gain control of data gathering process, counter biases
o Measure: reinforcement of stereotypes / unfairness should not be hidden deeply in
model
o Understand: what is source of disparate impact? Can it be explained? Is explanation
acceptable
o Involve right stakeholders: data scientists should not be the ones deciding which
error level is acceptable
Fairness audits
- Study differences in tax audit rates by Internal Revenue Service (IRS) between black & white
tax payers
- Some issues
o Incomplete data (only selected samples were audited)
o No information on ethnicity stored
- Predict ethnicity based on name & postal code
- Black community audited at much higher rate than general population:
o Audit rates for blacks: 0.84 – 1.34 percentage points higher than for non-blacks
o Base audit rate is only 0.54%
o blacks 2.9 – 4.7 x more likely to be audited
- Black taxpayers claiming EITC (Earned income tax credit) are between 2.9 – 4.4 x more likely
to be audited than non-black taxpayers
- Objective that is optimized influences disparity
o Maximize number of fraud cases detected
o Maximize dollars retrieved (would give opposite result)
o Focus on underreported income
o Focus on unjustified refunds
- Bias measure:
o Measure disparity of errors between groups
o Monitor differences in acceptance rates between groups
- Use bias measures as KPIs: follow up on these KPIs when model deployed
- Understand sources bias: make policy decisions at right level
4
