The Minor Puzzle about Habitual Action

Click here and press the right key for the next slide.

(This may not work on mobile or ipad. You can try using chrome or firefox, but even that may fail. Sorry.)

also ...

Press the left key to go backwards (or swipe right)

Press n to toggle whether notes are shown (or add '?notes' to the url before the #)

Press m or double tap to slide thumbnails (menu)

Press ? at any time to show the keyboard shortcuts

The Minor Puzzle about Habitual Action

What kinds of processes in individual animals guide actions?

individual agents

habitual vs goal-directed processes

In this section I want to introduce a minor puzzle that follows from distinguishing habitual vs goal-directed processes.

It is a minor puzzle in the sense that we can solve it quite easily. But it is important because the solution reveals something interesting about the dual-process theory of action.

Which of these are habitual actions:

doing your teeth;

smoking tobacco;

watching TV;

cycling the same route to work every day?

habitual vs habitual

‘A habitual action, state, or way of behaving is one that someone usually does or has, especially one that is considered to be typical or characteristic of them.’

--- collinsdictionary.com

habitual process

Action occurs in the presence of Stimulus.

Agent is rewarded [/punished]

Stimulus-Action Link is strengthened [/weakened] due to reward [/punishment]

Given Stimulus, will Action occur? It depends on the strength of the Stimulus-Action Link.

‘goal-directed’ process

Action leads to Outcome.

Belief in Action-Outcome link is strengthened.

Agent has a Desire for the Outcome

Will Action occur? It depends on the Belief in the Action-Outcome Link and Agent’s Desire.

Which of these are habitual actions:

doing your teeth;

smoking tobacco;

watching TV;

cycling the same route to work every day?

Too hard to do a controlled experiment. Let’s consider an easier case.

You see a rat and a lever. The rat presses the lever occasionally. Now you start rewarding the rat: when it presses the lever it is rewarded with a particular kind of food. As a consequence, the rat presses the lever more often.

Is this lever pressing a habitual action?

It’s a trick question -- we can’t tell

habitual process

Stimulus is the layout of this room.

Rat (=Agent) is rewarded with food

Room-LeverPress (=Stimulus-Action) Link is strengthened due to reward

Thf LeverPress (=Action) will occur in this room (=Stimulus).

Problem: different hypotheses, same prediction

What if we devalue the food?

Explain devaluation (poison, or satiation)

habitual process

Stimulus is the layout of this room.

Rat (=Agent) is rewarded with food

Room-LeverPress (=Stimulus-Action) Link is strengthened due to reward

Thf LeverPress (=Action) will occur in this room (=Stimulus).

‘goal-directed’ process

Lever pressing (=Action) leads to food (=Outcome).

Thf LeverPress-Food (=Action-Outcome) Link is strong.

Rat (=Agent) has strong positive Preference for food.

Thf LeverPress (=Action) will occur.

Devaluation affects Preference, so changes what the instrumental hypothesis predicts.

Devaluation does not affect the Simulus-Action link. (It’s the fact that food was preferred in the past that matters: because of this, getting food was rewarding and so strengthened the Simulus-Action link.)

What if we devalue the food?

‘Goal-directed’ process : it will reduce lever pressing (to none)

Habitual process : it will have no effect on lever pressing

‘Mean lever-press rates during the extinction (left-handpanel) and reacquisitiontests(right-handpanel) followingthe devaluation of either the contingent (group D-N) or non-contingentfood (group N-D).’

Dickinson, 1985 figure 3

What if we devalue the food?

‘Goal-directed’ process : it will reduce lever pressing (to none)

Habitual process : it will have no effect on lever pressing

(a) Rat’s behaviour is dominated by a ‘goal-directed’ process (explained by their Preferences). (b) Hypotheses about processes underpinning decisions are scientifically testable.

‘the laboratory rat fits the teleological [goal-directed] model; performance of this particular instrumental behaviour really does seem to be controlled byknowledge about the relation between the action and the goal’

(Dickinson, 1985, p. 72)

Dickinson, 1985 p. 72

But there is a complication ...

It is not none!!!

Dickinson, 1985 figure 3

puzzle

If the action is not habitual but controlled by goal-directed processes, why does it still occur after devaluation?

Solution is to stop thinking that actions can be just one or the other. _The instrumental/habitual distinction concerns proceses, not actions!_

What if we devalue the food?

‘Goal-directed’ process : it will reduce lever pressing (to none)

Habitual process : it will have no effect on lever pressing

(Dickinson, 1985, p. 72)

Dickinson, 1985 p. 72

‘we did not conclude that all such responding was of this form.

Indeed, we observed some residual responding during the post-re-valuation test that appeared to be impervious to outcome devaluation and therefore autonomous of the current incentive value,

and we speculated that this responding was habitual’

and established by a process akin to the stimulus-response (S-R)/reinforcement mechanism embodied in Thorndike’s classic Law of Effect (Thorndike, 1911).

(Dickinson, 2016, p. 179)

Dickinson, 2016 p. 179

Dual-Process Theory of Action

some instrumental actions are ‘controlled by two dissociable processes: a goal-directed and an habitual process’

\citep{Dickinson:1985qp,dickinson:2016_instrumental}

Dickinson, 2016 p. 179

one action, two processes

(Strictly speaking, we might think that some actions were habitual and others goal-directed and that only the former remain.)

Earlier I asked, You see a rat and a lever. The rat presses the lever occasionally. Now you start rewarding the rat: when it presses the lever it is rewarded with a particular kind of food. As a consequence, the rat presses the lever more often.

Is this lever pressing a habitual aciton?

Now we can say the question is confused.

Is this action a consequence of habitual or ‘goal-directed’ processes? Probably both!

Which of these are habitual actions:

doing your teeth;

smoking tobacco;

watching TV;

cycling the same route to work every day?

We find out in the evidence section how plausible it is that habitual processes play a significant role in your life.

Keyboard Shortcuts?f

The Minor Puzzle about Habitual Action

Keyboard Shortcuts`?f`