Goal-Directed and Habitual: Some Evidence

Click here and press the right key for the next slide.

(This may not work on mobile or ipad. You can try using chrome or firefox, but even that may fail. Sorry.)

also ...

Press the left key to go backwards (or swipe right)

Press n to toggle whether notes are shown (or add '?notes' to the url before the #)

Press m or double tap to slide thumbnails (menu)

Press ? at any time to show the keyboard shortcuts

Goal-Directed and Habitual: Some Evidence

‘instrumental behavior itself involves two systems, the goal-directed and the habitual’

(Dickinson & Pérez, 2018, p. 12)

prediction?

We are intervening on the cognitive load (stress), so we would expect ...

Lower influence from goal-directed processes that without the intevention

But how does this lead to the prediction that preferences will matter less?

habitual process

Action occurs in the presence of Stimulus.

Outcome follows action

Agent is rewarded

Stimulus-Action Link is strengthened due to reward

Given Stimulus, will Action occur? It depends on the strength of the Stimulus-Action Link.

‘goal-directed’ process

Action leads to Outcome.

Belief in Action-Outcome link is strengthened.

Agent has a Desire for the Outcome

Will Action occur? It depends on the Belief in the Action-Outcome Link and Agent’s Desire.

what we are seeing is that the influence of preferences wanes, indicating that habitual processes

‘instrumental behavior itself involves two systems, the goal-directed and the habitual’

(Dickinson & Pérez, 2018, p. 12)

prediction?

prediction: increasing stress will reduce the influence of your preferences

Evidence: Schwabe & Wolf, 2010

Schwabe and Wolf, 2010 figure 1

‘Figure 1. (A) Time line of the experiment. Participants were first trained in the instrumental task. After the selective outcome devaluation (satiation with oranges or chocolate pudding) but before the extinction test, subjects were exposed to stress (socially evaluated cold pressor test) or a control condition. (B) The instrumental task (reproduced with permission from the Society for Neuroscience). Participants completed three trial types (chocolate, orange, and neutral). In each trial type, there was one action that led with a high probability to a food outcome and one action that led with a low probability to a food outcome. Depending on the trial type, the high probability action yielded chocolate milk or orange juice with a probability of p = 0.5, a common outcome (peppermint tea) with a probability of p = 0.2, or nothing. The low probability action led to the common liquid with a probability of p = 0.2. After an action was chosen, the referring symbol was highlighted for 3 s before the food was delivered. During the extinction test, chocolate milk and orange juice were no longer presented.’

Schwabe and Wolf, 2010 figure 6

‘Figure 6. Percent high probability actions of controls and stressed participants in the last 15-trial block of training and the first 15-trial block of extinction testing. After selective outcome devaluation, controls showed a decrease in the choice of the high probability action associated with the food eaten to satiety (* p < .01) whereas the choice behavior of stressed participants was insensitive to the changes in outcome value. Data represent M ± SEM.’

When stressed,

your preferences matter less:

habits dominate.

How is this evidence for the dual-process theory of instrumental action?

We are intervening on the cognitive load (stress), so we would expect ...

Lower influence from goal-directed processes that without the intevention

But how does this lead to the prediction that preferences will matter less?

How exactly do we derive that prediction?

I.e. the prediction that cognitive load will selectively hinder the goal-directed process.

Having two processes
allows you to make complementary
speed–accuracy trade-offs:
habitual processes are fast but limited, whereas goal-directed processes are more flexible but slower

Any broadly computational system will face ...

We are intervening on the cognitive load (stress), so we would expect ...

Lower influence from goal-directed processes that without the intevention

But how does this lead to the prediction that preferences will matter less?

more evidence

training effects

background: goal-directed processes in young children

Klossek & Dickinson, 2012 figure 1a

This is from a different study than the one I will emphasise: in this study they only demonstrate instrumental behaviour in young children.

Klossek & Dickinson, 2012 figure 2

Even young children can perform instrumental actions, but perhaps not very young children.

Why is this evidence of goal-directed processes in the older age groups?

If we intervene on preferences, we should hardly influence habitual processes but should strongly influence goal-directed processes. The fact that the intervention on preferences modulates a certain instrumental action therefore indicates that the goal-directed processes are influencing that instrumental action.

habitual process

Action occurs in the presence of Stimulus.

Outcome follows action

Agent is rewarded

Stimulus-Action Link is strengthened due to reward

Given Stimulus, will Action occur? It depends on the strength of the Stimulus-Action Link.

‘goal-directed’ process

Action leads to Outcome.

Belief in Action-Outcome link is strengthened.

Agent has a Desire for the Outcome

Will Action occur? It depends on the Belief in the Action-Outcome Link and Agent’s Desire.

Another way of looking at it ...

what we are seeing is that the influence of preferences wanes, indicating that habitual processes

Where is the evidence for the dual-process theory of instrumental action?

What happens if we train people with just one action possibility (rather than two)?

premise:

‘Once responding at a high and constant rate in the single-action condition after extended training, agents no longer experience [...] episodes in which they do not respond and do not receive the outcome. As a result, the action-outcome causal representation necessary for goal-directed action is not maintained.’

(Dickinson, 2016, p. 181)

prediction: devaluation will have less effect in one- than in two-action-training

(because habitual processes will be more influential in one- than in two-action-training )

Training Effects (Klossek, Yu & Dickinson, 2011)

Source: (Dickinson, 2016, p. 180) Which is about Klossek, U. M. H., Yu, S., & Dickinson, A. (2011). Choice and goal-directed behavior in preschool children. Learning and Behavior, 39, 350-357.

Subjects: 3-4 year olds

Training:

Choice Group : perform Action1 to see Clip1 or Action2 to see Clip2

Single-Action Group : only one action is available at once

(But both actions are trained, just sequentially)

(Frequency of Action1 and Action2 is matched across groups!)

Devalue Clip1 (expose to satiety)

Test: both actions available. What do Ss select?

test is done in EXTINCTION

Results:

Choice group selects Action2

Single-Action Group selects Action1 and Action2 equally

As predicted if Instrumental

As predicted if Habitual

‘We argued that the variation in the development of behavioral autonomy arose from the different contingency experienced of the two groups. Once responding at a high and constant rate in the single-action condition after extended training, agents no longer experience the full causal contingency, speci cally episodes in which they do not respond and do not receive the outcome. As a result, the action-outcome causal representation necessary for goal-directed action is not maintained.’ (Dickinson, 2016, p. 181)

Klossek et al, 2011 figure 1

‘Mean response rates per second during training for the choice and single-action groups. Error bars represent the standard errors of the means’

Klossek et al, 2011 figure 2

‘Mean percentage response rates for the choice and single- action groups during the postdevaluation extinction test. Error bars represent the standard errors of the means’

Whether you learn about the effects of an action

can influence

whether that action becomes dominated by instrumental or habitual processes.

Why is this evidence for the dual-process theory of instrumental action?

Only the dual-process theory can predict that devaluation will have less effect in the one- than in the two-action condition

What happens if we train people with just one action possibility (rather than two)?

premise:

(Dickinson, 2016, p. 181)

prediction: devaluation will have less effect in one- than in two-action-training

(because habitual processes will be more influential in one- than in two-action-training )

more evidence

neurophysiology

‘[instumental] and habitual control have been doubly dissociated in two brain regions.

In the PFC, lesions of the prelimbic and infralimbic areas disrupt goal-directed and habitual behavior, respectively ...

prelimbic cortex is part of dmPFC (dorsomedial prefrontal cortex)

infralimbic cortex is part of vmPFC (ventromedial prefrontal cortex)

These dissociations suggest that different neural circuits mediate the two forms of control’

(Dickinson, 2016, p. 184)

conclusion - three bits of evidence

cognitive load (via stress) - Schwabe & Wolf (2010)

representation of contingency - Klossek, Yu, & Dickinson (2011)

neurophysiology - Dickinson (2016)

Check the objection is now sorted.

zahidnasser

Regarding Question 1: 'What is the relation between an instrumental action and the outcome or outcomes to which it is directed?'.

It is said that we can coherently answer this question via habitual processes without invoking intention at all,

but I struggle to see how this is possible.

Isn't intention always present in an instrumental action, even if it is just performed ‘out of habit’?

^ there may not be explicit conscious intention, but there will always be some form of underlying intention

Good objection. What to do?

Reply to zahidnasser

1. intentions are caused by beliefs and desires [premise]

2. intentions only explain actions that are appropriately related to them

3. habitual processes sometimes cause actions contrary to our current desires [tbs]

∴

3. such actions are not explained by intentions

Before I said just need to establish the second premise.

Keyboard Shortcuts?f

Goal-Directed and Habitual: Some Evidence

Keyboard Shortcuts`?f`