19@222 Contingency and contiguity

Contingency vs Contiguity

Introducing Robert Rescorla

Contingency: Predictive relationship between CS and UCS; CS1 and CS2.

Contingency means dependence and correlation. Clouds predict rain

1) for many yeary it was assumed that the critical determinant of Pavlovian conditioning essential,

was the pairing of the CS and UCS in close temporal contiguity. (trace, celayed, backward ISI effects)

2) Recently, Robert Rescorla put this long held notion to task by suggesting that contingency not continuity was the determinant factor for learning.

Conditioning depends on the CS allowing the subject to predict the occurrence

of the UCS.

Following example: have contiguity in all of following groups

1) p CS/UCS > p no CsS/UCS =+1.0 acquisition

2) p CS/UCS < p no CS/UCS =-1.0 extinction

3) p CS/UCS = p no CS/UCS = .5

notice that number 3 has been brought up as a possible control group for Pavlovian conditioning. Random presentations.

Rescorla Predictability and number of pairings

Three groups of dogs Sidman avoidance traained. so it starts off with an instrumental task where escape conditioning is presented. Shock, hklurdle, escape. And back again.

Now to a Pavlovian fear conditining situaition.

Group 1 CS UCS information > .5 acqisition P or Paired

Group 2 CS not paired <.5 U or unpaired

Group 3 Css and UCSs occured random R or random

random and independent and non-contingent

Results

Avoidance: P > U , produced increases and decreases in avoidance byt R group produced no effect on learning or avoidance.

Inhibition and excitation.

Ramification: leaned helplessness

Rescorla suggest that this is the best control for sensitization aearnd pseudoconditioning

Seligman a fellow U Penn mafia does chronic fear produced by unpredictable

All groups bar press for food

P shock gropup

U paired shock

3. Basics of the R-W model

Associative strength:

How strongly two events are connected/associated

Change in associative strength resulting from a single experience/trial

Salience/associability of an event

Rate of learning

Asymptote for associative strength

now to skinner box

P group

BAR PRESS learned

and received CSs either paired or unpaired with UCS/shock

R group stop bar pressing

develop ulcers.

Chronic fear produced by unpredictable shock (1968)

The dysfunctional .5

Ramifications: conditioned neurosis

physical and psychological control is lost

child abuse

bonaventure

welfare

soviet union

The Rescorla-Wagner Model of Associative Learning

The study of learning is frustrating for many "hard-science" people, because, if nothing else,

Psychology is a science that yields few quantitative results. How is it possible to assign a value to

learning? How can a scientist say, "Well, Fluffy the incontinent chihuaha has only exhibited 4 units of

associative learning while Socks the Cat has shown 5.3." Learning, in general, cannot be quantified.

It can be observed and the strength of associations measured, but assigning a numeric value to

describe "learning" seems impossible.

But, of course, two geeky little scientists, who are still probably being picked last in every game of

pick-up basketball that's ever played, have cleverly devised a model to do exactly what I just went

to great pains saying couldn't be done.

Robert Rescorla and Allan Wagner were two scientists who couldn't stand studying learning

behavior without a sophisticated mathematical equation to quantify their experimental results, so, like

any good scientists would do, they simply created their own. Thus did the Rescorla-Wagner model

come into existance. And, as if forcing poor college students to do math in a Psych class weren't bad

enough, Rescorla and Wagner decided, most prudently, that they should make their equation as

cryptic and incomprehensible as humanly possible to absolutely everyone in the universe -- that is,

except for the countless many who sit around writing poems to their cats in Ancient Greek and

spend the rest of their time playing with equations that quantitatively represent how clever they are

(namely Rescorla and Wagner).

Here is the equation the crazy scientists came up with:

Note: need help with ASCII characters -- Equation to be added later.

As confusing as it looks, the explanation of the equation is slightly simpler: The amount of association

that occurs on any trial is determined by the maximum learning possible, minus how much has

already been learned, taking into consideration the nature of the CS's and US's that are being

associated.

The "nature of the CS's and US's" refers to the SALIENCE of the stimuli. Salience basically means

the strength, or power of a stimulus. A brighter light is more noticeable than a dim one, as a sharp

pain is more SALIENT than a weak pain.

That, in a nutshell -- and an ugly, deformed, pathetic nutshell, at that -- is what the Rescorla-Wagner

Model is all about. It makes me feel, if even just a tiny bit, that Eric P. Wiertelak is the only person

on the planet who will find it necessary to read the preceding bit of mindless drivel.

Now on to the Miller Comparator Hypothesis (The disclaimer, found at the top of this page, applies

to this section as well).

Contingency or Contiguity?

Instrumental learning normally clearly depends on a contingency between response and reinforcement,

but must this always be the case? Normally, if a contingency is not present - if responding has no

effect on whether reinforcement is obtained, then no learning occurs. There is, however, the possibility

that a contingency is perceived where, in fact, there is none. To truly assess the contingency between

response and reinforcement we need to know both the chances of obtaining a reinforcer if we respond

and the chances of obtaining a reinforcer if we don't respond. If we never evaluate the latter

probability because we are responding all the time then we may attribute a contingency to responding

where there is none. The opposite can also occur. An extreme example of this is 'learned

helplessness'. In the first part of a learned helplessness experiment an animal is subject to unavoidable

shocks - there may be a potential path to escape, for example a wall to jump over, but escape is

impossible, for example because the wall is too high. Soon the animal learns that escape is impossible

and ceases attempting it. If the animal is now moved to a different situation in which escape is possible

it will, nevertheless, fail to learn. Because it never performs escape behavior it does cannot discover

that the chances of being shocked when it makes an escape attempt now are different from those it

experience when not behaving. The lack of contingency perceived between behavior and shock is

illusory. In these circumstances then conditioning is really being controlled by the contiguity of response

and reinforcer not their contingency. It should, however, be emphasised that, in general, the

effectiveness of instrumental learning depends on contingency.

Rescorla and Wagner's model of classical

conditioning.

According to Rescorla and Kamin, associations are only learned when a surprising event accompanies

a CS. In a normal simple conditioning experiment the US is surprising the first few times it is

experienced so it is associated with salient stimuli which immediately precede it. In a blocking

experiment once the association between the CS (CS1) presented in the first phase of the procedure

and the US has been made the US is no longer surprising (since it is predicted by CS1). In the second

phase, where both CS1 and CS2 are experienced, as the US is no longer surprising it does not induce

any further learning and so no association is made between the US and CS2. This explanation was

presented by Rescorla and Wagner (1972) as a formal model of conditioning which expresses the

capacity a CS has to become associated with a US at any given time. This associative strength of the

US to the CS is referred to by the letter V and the change in this strength which occurs on each trial of

conditioning is called dV. The more a CS is associated with a US the less additional association the US

can induce. This informal explanation of the role of US surprise and of CS (and US) salience in the

process of conditioning can be stated as follows:

dV = ab(L - V)

where a is the salience of the US, b is the salience of the CS and L is the amount of processing given

to a completely unpredicted US. In words: when the US is first encountered the CS has no association

to it so V is zero. On the first trial the CS gains a strength of abL in its association with the US which

is proportional to the saliences of the CS and the US and to the initial amount of processing given to the

US. As we start trial two the associative strength is V is abL so the change in strength that occurs

with the second pairing of the CS and US is ab(L - abL). It is smaller than the amount learned on the

first trial and this reduction in amount that is learned reflects the fact that the CS now has some

association with the US, so the US is less surprising. As more trials ensue, the equation predicts a

gradually decreasing rate of learning which reaches an asymptote at L. However, the diagramm below

shows: this is not what is seen when the development CS-US associations is measured over time.

Instead the learning curve is sigmoidal. Rescorla has argued that the equation is consistent with

observed behavior if one assumes that very small changes in associative strength are undetectable and

that there is a limit to the amount of effect that very large changes can have on behavior.

There are other respects, however, where the model performs better in predicting experimental

outcomes. It can also be applied to a number of CSs each of which contributes to an overall

associative strength V of the US in the right hand side of the equation. It is reasonably clear that the

presence of the CS salience term b in the equation lets it account for overshadowing. The meaning of

the equation is clearest if the specific dVs on the left hand side are seen as referring to the increments

in association between specific CSs while V on the right hand side is referring to the predictability of

the US and so is the sum of all the different CS-US associations. If the conditioning strength accrued

to CS1 is denoted by dV1 and that to CS2 by dV2 then our equations are

dV1 = ab1(L - V)

dV2 = ab2(L - V)

and both dV1 and dV2 accrue to V on each trial. The amount of association directed to each CS is

proportional to their salience.

The equation also models blocking well. During the initial phase of a blocking experiment the

associative strength of the US is increased so later, when a second CS is presented the amount of

associative strength it can gain has been reduced.

The critical question is, however, does the model predict experimental outcomes it was not explicitly

divised for, i.e. can it be generalized? In one example the model predicts the effects of pairing two

previously learned CSs on learning about a third new stimulus. If on separate occasions (not as

compound stimuli) two CSs of equal salience have both been completely associated with a US then

V=L for both stimuli and dV on subsequent trials is zero for both. Now a third CS in conjunction with

the original pair is presented so three CSs are presented together whereas only two of them were

presented singly in the past. The overall associative strength of the US is now 2L, a contribution of L

from both of the original CSs. The equation predicts that there will be a negative change in associative

strength on this trial proportional to the salience of the CSs:

dV = ab(L - 2L)

dV = -abL

Conducting the experiment shows: the third stimulus becomes a conditioned inhibitor of the US - it

provokes a CR of the opposite quality to that produced by the other two CSs.

Rescorla's explanation of the "blocking and predictability" experiment is more debatable. During phase

1 of the experiment the 'No US' are undergoing habituation. Rescorla argues that the 'No US' group

learn in the first phase of the experiment that CS1 is a predictor of 'no US' and hence that, when it is

followed by a US in phase 2 this US is even more surprising than it would have been normally, hence it

provokes especially strong learning. His own model, however, predicts that there should be no change

in the associative strength associated with the stimulus when there is no US. First, is is not very logical

to assign an amount of processing devoted to a non-event if that non-event is unpredicted. Second,

Rescorla's model revolves around the surprisingness of specific USs - and 'no US' must be a different

US from 'US' so prior exposure to a good predictor of 'no US' should not effect the amount of

processing devoted to a different US 'US'. For these, and other reasons a series of more sophisticated

models have subsequently been developed in which the rate of learning is not driven by the

'surprisingness' of the US (as in the L-V term of the Rescorla-Wagner model) but by terms which

represent the predictive power of individual CSs independently (for example Mackintosh's 1975

model). In this sort of model a CS which had been experienced many times unpaired with a significant

US would be evaluated as having less than average predictive power. If, however, the CS had been

paired with a different US during phase 1 of Rescorla's (1971) experiment, then it should be evaluated

as having predictive power and hence still be associable with a different US during phase 2, reducing

the 'superconditioning' to the other CS previously found. Dickinson (1976) has reported such an effect.

Rescorla-Wagner model and contingency

In a zero contingency you may think that the rat is comparing two probabilities P(USICS) P(USINo

CS)

The Rescorla-Wagner model argues that this is not the case and that the reason the CS does not

acquire any associative strength is because of context blocking.

Note that there are trials in which the US is presented in the context alone and the CS + context - US

trials: the context acquires the associative strength and blocks ant conditioning to the tone.

Durlach (1983)

Test to see if context blocking may be an explanation for why the CS fails to elicit a CR in a

zero-contingency.

Group 2 also was given a tone just prior to the delivery of all grain in the context alone.

The associative strength of the tone was previously conditioned so that its value was near asymptote

and should thus block the conditioning of the context-US association.

If the pigeons are sensitive to contingency because they calculate them then the addition of a tone

should have no effect on conditioning to the key light; if the addition of the tone blocks conditioning to

the context; then the context should not be able to block conditioning to the key light and conditioning

should occur.

Results: Group 2 showed significant increase in pecking to the light than Group 1.

Interpretation: Lack of responding to CS in a zero contingency is due to the context blocking

conditioning to the CS.

I. Conditioning and Changes in CS Effectiveness

In R&W model salience remains constant throughout experiment.

Is salience fixed in an experiment?

Latent inhibition

explain procedure again

R&W model does not account for effect

is the tone an inhibitor? No!

interpretation CS pre-expressed reduces salience; may ignore CS

Learned irrelevance Baker & Mackintosh (1977) - if CS & US are unpaired conditioning

acquisition is slowed when you try and turn CS into excitor.

CS does not predict anything and is irrelevant

Second interpretation : organisms associate US with context; when CS-US the context blocks

conditioning to tone.

How can this be explicitly tested?