Lecture 11: Value-based Choice
Epistemological detour
• David Marr - computational scientist, Vision
• Interested in how to solve complex problems like how the brain works
• Marr proposed the well-known 3-level model, by which you can adapt your understanding of
how a complex system like the brain works: computational level deals with why problem,
algorithmic levels deals with how the why problem is processed in a set of rule and finally
how you physically implement this algorithm
o 3-level model (computation [why problem], algorithm, [what rule], implementation
[how physical])
§ E.g. Flight in a bird: flight (why), flapping (rules), feathers (how) Commented [IM225]: The computational problem you want to
attack
§ This does not always have to be a top-down hierarchy e.g. can be made
Commented [IM226]: Through which algorithm can you solve
aware of physical limitations of the system, which can inform you of the this ‘why’ problem - flight is only one of the possible algorithms for
feasibility of certain algorithms flight e.g. also plane wings do not flap, and uses a different
algorithm to solve flying problem
This is a good guide to follow when understanding the brain system and how it works, but not the
Commented [IM227]: How would you implement this
only one algorithm in the physical world
Neuroeconomics
Value-based Choice
Value-based decision involves choosing between two perceptually different ‘objects’ e.g. apples
and oranges, but also more abstract decisions such a monetary savings versus spending, as well as
decisions with larger scale consequences concerning multiple people, such as the BREXIT
referendum
• All of these are value comparisons – comparing between two items that are different from
each other
People often conflate two very different types of decisions i.e. value-based choices versus
perceptual choices
• In perceptual decisions you are comparing things that have to be in the same domain (e.g.
which of these two lines is longer). In order to make this categorisation/choice, the
dimension (length) has to be the same
This is definitely not the case for value-based decisions – the whole point of value-based decisions
are to compare things that are very different at the perceptual level
Perceptual choice
Sensory inputs (in the same domain) —> sensory categorisation —> action selection
Value-based choice
Sensory inputs (very different) à value representation (abstract scale) à action selection
• The two inputs that are very different on a sensory level, are brought together on a value Commented [IM228]: Very different inputs brought together at
scale, allowing for comparison and consequent action selection the intermediate step of the same scale/dimension (value scale)
This is an abstract scale, removed from the sensory input, that
enables comparison
, • However, these types of decisions (value-based) have inputs that go beyond the simple
sensory input
o Value representation is affected by many other factors, such as internal states/ goals
(e.g. hunger, time frame) and memories (e.g. previous good/bad taste) Commented [IM229]: Memories have a strong impact on
value-based choice, arguably moreso than current visual information
Value-based decisions are strongly idiosyncratic - there is not an objective dimension in value; the
subjective dimension is completely overwhelming
Value in the decision-making process
Value-based decision is a huge field, but it can be subdivided into smaller subproblems (e.g.
representation, valuation, action selection, outcome evaluation and learning/updating)
We are going to concentrate on the value process within decision-making
There are different value systems in the brain, sometimes in conflict; it is not a monolytic system
1. Pavlovian - value hard-wired in brain; evolution; essential for survival; quick response, cam
even go beyond conscious awareness; hard-wired à but, as a consequence of being hard-
wired, it is very inflexible, thus can interfere with other valuation systems Commented [IM230]: Important for behaviours/associations
that you do not have time to learn e.g. drinking breast milk is good
2. Habitual - assign value to something that doesn’t necessarily have a value (e.g. light). for you, do not jump from a great height
Devaluation of an item can occur (more flexible). Still rather inflexible once created, habit is
hard to change. A relatively simple system. Commented [IM231]: Slightly more sophisticated than the
Pavlovian system.
3. Goal-directed - most complex and flexible, also system least understood; allows abstract
comparisons between very different items/events; systems has huge flexibility; many factors Transfer of value of food to light in associational learning
can influence value; computationally very demanding; requires a much more sophisticated This value can also be changed e.g. if you stop associating food with
hardware; goal-directed choice not found in simple animals – this value system is quite light, the animal will devalue the item
common in humans This value system is much more flexible than the Pavlovian, but a
habit once created is still difficult to change
This is a quite simple, with a good degree of flexibility, but not as
*** much as in goal-directed value system
Goal-directed system (abstract comparisons between two very different items, then putting them
in the same scale (valuation))
This lecture concerns features of the goal-directed system.
Expected Utility Theory
Economic theory
Bernoulli & Bernoulli (1713-1738 developed the EUT, which was then formalised by Von Neumann &
Morgenstern (1944)
• Simply, to make this comparison one needs to assign utility to different items
• They discovered some properties to this utility (‘value’) function
• An important feature of this utility is that it is a part of a common scale, so one can assign
utility to everything – allows us to compare very different items via this ‘common currency’
Epistemological detour
• David Marr - computational scientist, Vision
• Interested in how to solve complex problems like how the brain works
• Marr proposed the well-known 3-level model, by which you can adapt your understanding of
how a complex system like the brain works: computational level deals with why problem,
algorithmic levels deals with how the why problem is processed in a set of rule and finally
how you physically implement this algorithm
o 3-level model (computation [why problem], algorithm, [what rule], implementation
[how physical])
§ E.g. Flight in a bird: flight (why), flapping (rules), feathers (how) Commented [IM225]: The computational problem you want to
attack
§ This does not always have to be a top-down hierarchy e.g. can be made
Commented [IM226]: Through which algorithm can you solve
aware of physical limitations of the system, which can inform you of the this ‘why’ problem - flight is only one of the possible algorithms for
feasibility of certain algorithms flight e.g. also plane wings do not flap, and uses a different
algorithm to solve flying problem
This is a good guide to follow when understanding the brain system and how it works, but not the
Commented [IM227]: How would you implement this
only one algorithm in the physical world
Neuroeconomics
Value-based Choice
Value-based decision involves choosing between two perceptually different ‘objects’ e.g. apples
and oranges, but also more abstract decisions such a monetary savings versus spending, as well as
decisions with larger scale consequences concerning multiple people, such as the BREXIT
referendum
• All of these are value comparisons – comparing between two items that are different from
each other
People often conflate two very different types of decisions i.e. value-based choices versus
perceptual choices
• In perceptual decisions you are comparing things that have to be in the same domain (e.g.
which of these two lines is longer). In order to make this categorisation/choice, the
dimension (length) has to be the same
This is definitely not the case for value-based decisions – the whole point of value-based decisions
are to compare things that are very different at the perceptual level
Perceptual choice
Sensory inputs (in the same domain) —> sensory categorisation —> action selection
Value-based choice
Sensory inputs (very different) à value representation (abstract scale) à action selection
• The two inputs that are very different on a sensory level, are brought together on a value Commented [IM228]: Very different inputs brought together at
scale, allowing for comparison and consequent action selection the intermediate step of the same scale/dimension (value scale)
This is an abstract scale, removed from the sensory input, that
enables comparison
, • However, these types of decisions (value-based) have inputs that go beyond the simple
sensory input
o Value representation is affected by many other factors, such as internal states/ goals
(e.g. hunger, time frame) and memories (e.g. previous good/bad taste) Commented [IM229]: Memories have a strong impact on
value-based choice, arguably moreso than current visual information
Value-based decisions are strongly idiosyncratic - there is not an objective dimension in value; the
subjective dimension is completely overwhelming
Value in the decision-making process
Value-based decision is a huge field, but it can be subdivided into smaller subproblems (e.g.
representation, valuation, action selection, outcome evaluation and learning/updating)
We are going to concentrate on the value process within decision-making
There are different value systems in the brain, sometimes in conflict; it is not a monolytic system
1. Pavlovian - value hard-wired in brain; evolution; essential for survival; quick response, cam
even go beyond conscious awareness; hard-wired à but, as a consequence of being hard-
wired, it is very inflexible, thus can interfere with other valuation systems Commented [IM230]: Important for behaviours/associations
that you do not have time to learn e.g. drinking breast milk is good
2. Habitual - assign value to something that doesn’t necessarily have a value (e.g. light). for you, do not jump from a great height
Devaluation of an item can occur (more flexible). Still rather inflexible once created, habit is
hard to change. A relatively simple system. Commented [IM231]: Slightly more sophisticated than the
Pavlovian system.
3. Goal-directed - most complex and flexible, also system least understood; allows abstract
comparisons between very different items/events; systems has huge flexibility; many factors Transfer of value of food to light in associational learning
can influence value; computationally very demanding; requires a much more sophisticated This value can also be changed e.g. if you stop associating food with
hardware; goal-directed choice not found in simple animals – this value system is quite light, the animal will devalue the item
common in humans This value system is much more flexible than the Pavlovian, but a
habit once created is still difficult to change
This is a quite simple, with a good degree of flexibility, but not as
*** much as in goal-directed value system
Goal-directed system (abstract comparisons between two very different items, then putting them
in the same scale (valuation))
This lecture concerns features of the goal-directed system.
Expected Utility Theory
Economic theory
Bernoulli & Bernoulli (1713-1738 developed the EUT, which was then formalised by Von Neumann &
Morgenstern (1944)
• Simply, to make this comparison one needs to assign utility to different items
• They discovered some properties to this utility (‘value’) function
• An important feature of this utility is that it is a part of a common scale, so one can assign
utility to everything – allows us to compare very different items via this ‘common currency’
