Keyboard Shortcuts?f

×
  • →Next step
  • ←Previous step
  • ↓Skip this slide
  • ↑Previous slide
  • mShow slide thumbnails
  • nShow notes
  • hShow handout latex source
  • NShow talk notes latex source

Click here and press the right key for the next slide.

(This may not work on mobile or ipad. You can try using chrome or firefox, but even that may fail. Sorry.)

also ...

Press the left key to go backwards (or swipe right)

Press n to toggle whether notes are shown (or add '?notes' to the url before the #)

Press m or double tap to slide thumbnails (menu)

Press ? at any time to show the keyboard shortcuts

 

Preference vs Aversion: A Dissociation

What kinds of processes in

individual animals

guide actions?

1. two kinds of process -- habitual vs instrumental

2. two kinds of motivational state -- primary vs preferences

done before

‘I eated you dinner because I was hungry.’

Seems like a very direct form of explanation. (Not going to say that your hunger was irrelevant, just that the explanation is less immediate than it seems.)

‘primary motivational states, such as hunger, do not determine the value of an instrumental goal directly; rather, animals have to learn about the value of a commodity in a particular motivational state through direct experience with it in that state’ (Dickinson & Balleine, 1994, p. 7)

‘primary motivational states have no direct impact on the current value that an agent assigns to a past outcome of an instrumental action; rather, it appears that agents have to learn about the value of an outcome through direct experience with it’ (Dickinson & Balleine, 1994, p. 8)

two motivational systems

motivational states

primary motivational states

largely unchanging, not learned

  • hunger
  • thirst
  • satiety
  • disgust
  • ...

preferences

changing, influenced by learning (and fashion, ...)

  • chocolate over rhubarb
  • lime over lemon
  • red over blue
  • ...

Q1: Can your primary motivational states diverge from your preferences?

For example, can sugar solution rank highly among your preferences even after you have become averse to it?

In thinking about this question, it is useful to recall some ideas from the previous lecture ...

 

You see a rat and a lever. The rat presses the lever occasionally. Now you start rewarding the rat: when it presses the lever it is rewarded with a particular kind of food. As a consequence, the rat presses the lever more often.

Devaluation - standard procedure:

Training: Rat is put in chamber with Lever; pressing Lever dispenses sucrose (novel food).

Devaluation: Rat is taken into another chamber, poisoned, and then exposed to sucrose.

Extinction Test: Rat returns to chamber with Lever; pressing Lever does nothing.

Dickinson, 1985 figure 3; Balleine & Dickinson, 1991 figure 1 (part)

‘Mean lever-press rates during the extinction (left-handpanel) and reacquisitiontests(right-handpanel) followingthe devaluation of either the contingent (group D-N) or non-contingentfood (group N-D).’
‘Experiment I: Mean number of lever presses ... during the extinction test session ... The various groups received either immediate (Groups IMM/SUC and IMM/ H20) or delayed (Groups DELjSUC and DEL/H2O) toxicosis [delayed did not cause aversion] and were re-exposed either to the sucrose solution (Groups IMM/SUC and DEL/SUC) or to water (Groups IMM/H2O and DEL/H20).’
[skip this in this sectino; may add a section specifically covering it]
Pavlovain conditioning, primary motivational states can have a direct effect on actions.
Now consider a question.

recap

What causes devaluation?

Devaluation is defined as a reversal in _preference_.

In the standard procedure, the subjects are poisoned then re-exposed to the food.

Poisoning changes primary motivational state. Does it also change preferences?

Q2: What happens if we poison the subjects but do not re-expose them to the food?

Is just poisoning enough?

Hypothesis 1: Poisoning does directly influence preferences

Prediction: lever-pressing should reduce (as in the standard procedure)

Hypothesis 2: Poisoning does not directly influence preferences

Prediction: lever-pressing should not reduce (unlike in the standard procedure)

You already saw this

Balleine & Dickinson, 1991 figure 1 (part)

‘Experiment I: Mean number of lever presses ... during the extinction test session ... The various groups received either immediate (Groups IMM/SUC and IMM/ H20) or delayed (Groups DELjSUC and DEL/H2O) toxicosis [delayed did not cause aversion] and were re-exposed either to the sucrose solution (Groups IMM/SUC and DEL/SUC) or to water (Groups IMM/H2O and DEL/H20).’
‘lever presses during the re-acquisition session in 5-min blocks (right-hand panel)’ re-acquisition : i.e. you now have exposure to the sucrose solution

Aversion does not directly influence preferences.

‘The pattern of results accords [...] with a role for an incentive learning process in the reinforcer devaluation effect;

not only must consumption of the reinforcer be paired with toxicosis,

the animals must also have an opportunity to contact the reinforcer after aversion conditioning if there is to be a change in instrumental performance’

(Balleine & Dickinson, 1991, p. 293)

Balleine & Dickinson, 1991 p. 293

[skip in this presentation]
[To introduce the term ‘incentive learning’]

Q1: Can your primary motivational states diverge from your preferences?

For example, can sugar solution rank highly among your preferences even after you have become averse to it?

motivational states

primary motivational states

largely unchanging, not learned

  • hunger
  • thirst
  • satiety
  • disgust
  • ...

preferences

changing, influenced by learning (and fashion, ...)

  • chocolate over rhubarb
  • lime over lemon
  • red over blue
  • ...

An interface problem ...

‘we should search in vain among the literature for a consensus about the psychological processes by which primary motivational states, such as hunger and thirst, regulate simple goal-directed [i.e. instrumental] acts’

(Dickinson & Balleine, 1994, p. 1)

Dickinson & Balleine , 1994 p. 1

This is a very basic question. Why do you go to the kitchen and press the lever to get some water when you are thirsty?

two motivational systems

... an interface problem where there are two sets of preferences.

An Interface Problem:

How are non-accidental matches possible?

Primary motivational states guide some actions.

Preferences guide some actions.

Pursuing a single goal can involve both kinds of state.

As in the case of lever pressing then magazine entry to get the sugar solution. (Not demonstrated in this talk.)

Primary motivational states can differ from preferences.

Two motivational states match in a particular context just if, in that context, the actions one would cause and the actions the other would cause are not too different.

What kinds of processes in

individual animals

guide actions?

1. two kinds of process -- habitual vs instrumental

2. two kinds of motivational state -- primary vs preferences

conclusion

In conclusion, ...

Philosophical theories of action and joint action
assume a single system in which
belief, desire, intention and the rest
are normatively
and inferentially
integrated.

inferential integration

One instrumental action can involve multiple, dissociable
motivational states
and multiple, dissociable
goal-selection processes.

This gives rise to interface problems.