Get 20M+ Full-Text Papers For Less Than $1.50/day. Subscribe now for You or Your Team.

Learn More →

Context and Behavioral Processes in Extinction

Context and Behavioral Processes in Extinction Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Review Mark E. Bouton Department of Psychology, University of Vermont, Burlington, Vermont 05405, USA This article provides a selective review and integration of the behavioral literature on Pavlovian extinction. The first part reviews evidence that extinction does not destroy the original learning, but instead generates new learning that is especially context-dependent. The second part examines insights provided by research on several related behavioral phenomena (the interference paradigms, conditioned inhibition, and inhibition despite reinforcement). The final part examines four potential causes of extinction: the discrimination of a new reinforcement rate, generalization decrement, response inhibition, and violation of a reinforcer expectation. The data are consistent with behavioral models that emphasize the role of generalization decrement and expectation violation, but would be more so if those models were expanded to better accommodate the finding that extinction involves a context-modulated form of inhibitory learning. Extinction is a well-known and important behavioral phenom- tinction is not the same as unlearning, and because all of them enon that allows the organism to adapt its behavior to a chang- can be seen as context effects (see Bouton 1993, 2002), they also ing environment. Nonetheless, it is difficult to find a clear answer support the idea that performance after extinction is context- to the question of why repeated presentation of the conditional dependent. Extinction involves new learning, and it therefore stimulus (CS) alone after conditioning (CS–unconditional stimu- leaves the CS with two available “meanings” or associations with lus [US] pairings) causes behavior to return to essentially zero. the US. As is true for an ambiguous word, the context is crucial in One idea, surprisingly common in models of learning and selecting between them. memory (see Rescorla and Wagner 1972; McClelland and Rumel- hart 1985; McCloskey and Cohen 1989), is that extinction in- The Renewal Effect volves the destruction of what was originally learned. However, Perhaps the most fundamental of these effects is the renewal there is ample evidence that this is not true; much of the original effect (see Bouton and Bolles 1979a; Bouton and King 1983). In learning survives extinction (see Rescorla 2001; Bouton 2002; this phenomenon, a change of context after extinction can cause Myers and Davis 2002; Delamater 2004). In this article, I will a robust return of conditioned responding. Several versions of selectively review results and theory from the behavioral litera- the renewal effect have been studied. In the most common one, ture in an effort to understand what is learned in extinction, and “ABA renewal,” conditioning is conducted in one context (con- what causes the organism to learn it. text A) and extinction is then conducted in a second one (con- The first part of the article introduces several extinction phe- text B). (The contexts are typically separate and counterbalanced nomena that any adequate theory of extinction will need to ex- apparatuses housed in different rooms of the laboratory that dif- plain and accommodate. They suggest that extinction does not fer in their tactile, olfactory, and visual respects.) When the CS is destroy the original learning but instead involves new learning returned to the original conditioning context (context A), re- that is at least partly modulated by the context. In the second sponding to the CS returns (see Bouton and Bolles 1979a; Bouton part of the article, I will discuss a set of behavioral phenomena and King 1983; Bouton and Peck 1989). In a second version, that are theoretically linked to extinction, and ask what they “ABC renewal,” conditioning is conducted in context A, extinc- further reveal about it. Extinction is just one example of a retro- tion is conducted in context B, and then testing is conducted in active inhibition phenomenon in which new learning inhibits a third, “neutral” context—context C. Here again, a renewal of old, and the principles that explain it may have wide applicabil- responding is observed (see Bouton and Bolles 1979a; Bouton ity. In the final part of the article, I will consider the question “If and Brooks 1993; Harris et al. 2000). In a final version, condi- extinction is an example of new learning, what events ‘reinforce’ tioning and extinction are both conducted in the same context or cause it?” Several possibilities will be considered. In the end, (context A) and then the CS is tested in a second context (context the results may favor the view that extinction occurs because the B). Here again, conditioned responding returns (see Bouton and omission of the US (1) causes generalization decrement and (2) Ricker 1994; Tamai and Nakajima 2000), although there is cur- violates the organism’s expectation of the US and therefore ini- rently less evidence of this “AAB renewal” effect in operant con- tiates new learning. These ideas have a long history in behavioral ditioning than in Pavlovian conditioning (see Nakajima et al. theories of extinction. Their fuller integration with a contextual 2000; Crombag and Shaham 2002). analysis (part 1) may provide a more comprehensive behavioral Several facts about the renewal effect are worth noting. First, account of extinction. it has been observed in virtually every conditioning preparation in which it has been investigated (for a review, see Bouton 2002). Extinction Learning Is Especially Context-Dependent Second, it can occur after very extensive extinction training. In For many years, my colleagues and I have studied a number of fear conditioning (conditioned suppression) in rats, Bouton and experimental manipulations that can be conducted after extinc- Swartzentruber (1989) observed it when 84 extinction trials fol- tion has taken place. In each of them, the extinguished response lowed eight conditioning trials. Other evidence suggests that it returns to performance. All of them therefore indicate that ex- can occur after as many as 160 extinction trials (Gunther et al. 1998; Rauhut et al. 2001; Denniston et al. 2003), although a E-MAIL Mark.Bouton@uvm.edu; FAX (802) 656-8783. recent report suggests that it might not survive an especially Article and publication are at http://www.learnmem.org/cgi/doi/10.1101/ lm.78804. “massive” extinction treatment (800 extinction trials after eight 11:485–494 ©2004 by Cold Spring Harbor Laboratory Press ISSN 1072-0502/04; www.learnmem.org Learning & Memory 485 www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Bouton conditioning trials; Denniston et al., 2003). Third, the role of the 1993; Devenport et al. 1997), and it seems likely to be multiply context is different from the one anticipated by standard models determined. However, we have argued (see Bouton 1988, 1991, of classical conditioning (see Rescorla and Wagner 1972; Pearce 1993) that just as extinction is relatively specific to its physical and Hall 1980; Wagner 1981; Wagner and Brandon 1989, 2001). context, so it may be specific to its “temporal context.” Sponta- Those models accept the view that the context is merely another neous recovery can be seen as the renewal effect that occurs CS that is presented in compound with the target CS during when the CS is tested outside its temporal context. Both are due reinforcement or nonreinforcement. It therefore enters into to a failure to retrieve memories of extinction outside the extinc- simple excitatory or inhibitory associations with the US. In the tion context. Consistent with this perspective, a cue that is pre- ABA renewal effect (for example), context A might acquire excit- sented intermittently during the extinction session can attenuate atory associations with the US, and context B might acquire in- either spontaneous recovery or renewal if it is presented just be- hibitory associations. Either kind of association would summate fore the final test (Brooks and Bouton 1993, 1994; Brooks 2000). with the CS to produce the renewal effect (inhibition in B would The parallel results suggest that the two effects might be con- reduce responding to the CS, whereas excitation in A would en- trolled by a common mechanism: a failure to retrieve a memory hance it). However, a number of experiments have shown that of extinction outside the extinction context. Interestingly, the renewal effect can occur in the absence of demonstrable ex- changing the physical context and temporal context together citation in context A or inhibition in context B (see Bouton and can have a bigger effect than changing either context alone, as if King 1983; Bouton and Swartzentruber 1986, 1989). These find- their combination creates an even larger context change (Rosas ings, coupled with others showing that strong excitation in a and Bouton 1997, 1998). context does not influence performance to a CS unless the CS is Rapid Reacquisition under the influence of extinction (described below; Bouton 1984; Bouton and King 1986), suggest that direct associations in a con- A third effect further indicates that conditioning is not destroyed text are neither necessary nor sufficient for a context to influence in extinction. In rapid reacquisition, when new CS–US pairings responding to a CS. The implication (see Bouton and Swartzen- are introduced after extinction, the reacquisition of responding truber 1986; Bouton 1991) is that the contexts modulate or “set can be more rapid than is acquisition with a novel CS, indicating the occasion” for the current CS–US or CS–no US association (see that the original learning has been “saved” through extinction Holland 1992; Swartzentruber 1995; Schmajuk and Holland (see Napier et al. 1992; Ricker and Bouton 1996; Weidemann and 1998). Put another way, they activate or retrieve the current re- Kehoe 2003). Unfortunately, the early literature on rapid reac- lation of CS with the US. quisition was often difficult to interpret because many early de- A further important characteristic of the renewal effect is signs were not equipped to rule out less interesting explanations that it implies that extinction learning is more context-specific (for a review, see Bouton 1986). To add to the complexity, studies than is original conditioning. Notice that this must be true if one of fear conditioning (conditioned suppression; Bouton 1986; observes ABC and AAB renewal; in either case, conditioning Bouton and Swartzentruber 1989) and flavor aversion learning transfers better to the final test context than extinction. But our (Danguir and Nicolaidis 1977; Hart et al 1995), have shown that experiments on renewal have often involved comparisons of reacquisition can be slower than acquisition with a new CS. (It is groups that received extinction training in the context in which more rapid than initial acquisition with a CS that has received conditioning had occurred or in a discriminably different con- the same number of nonreinforced trials without conditioning text. Strikingly, there was no measurable effect of switching the [Bouton and Swartzentruber 1989].) In fear conditioning, slow context after conditioning on responding to the CS (see Bouton reacquisition requires extensive extinction training; more lim- and King 1983; Bouton and Peck 1989). In contrast, extinction ited extinction training yields reacquisition that is neither fast itself was relatively context-specific, as the renewal effect itself nor slow (Bouton 1986). At least part of the reason these prepa- suggests. Recent research suggests that both conditioning and rations support slow reacquisition is that both typically involve extinction become somewhat context-specific after extinction very few initial conditioning trials. In contrast, procedures in has occurred (Harris et al. 2000). But there is little question that which rapid reacquisition has been shown (rabbit nictitating extinction is still more context-dependent than is the original membrane response (NMR) conditioning and rat appetitive con- conditioning. We have therefore emphasized the fact that extinc- ditioning) have usually involved a relatively large number of ini- tion learning is especially context-dependent. tial conditioning trials. Consistent with a role for number of A final fact about the renewal effect is that it appears to be trials, Ricker and Bouton (1996) demonstrated that slow reacqui- supported by many kinds of contexts. For example, when fear sition occurred in an appetitive conditioning preparation when extinction was conducted in the interoceptive context provided the procedure used the number of conditioning and extinction by benzodiazepine tranquilizers chlordiazepoxide and diazepam, trials that had been used in previous fear conditioning experi- renewed fear was observed when the rat was tested in the original ments. In rabbit NMR and heart rate conditioning, extensive ex- nondrug state (Bouton et al. 1990). Cunningham (1979) had re- tinction training has abolished rapid reacquisition, although ported compatible evidence with alcohol, and we have recently slow reacquisition has yet to be observed (Weidemann and Ke- collected similar observations with the benzodiazepine mid- hoe 2003). azolam (L. Pain, P. Oberling, and M.E. Bouton; unpubl.). State- Ricker and Bouton (1996) suggested that rapid reacquisition dependent learning or retention can be conceptualized as the may partly be an ABA renewal effect that occurs when the animal drug playing the role of context (see Overton 1985). has learned that previous USs or conditioning trials are part of the original “context” of conditioning. That is, the animal might Spontaneous Recovery learn that recent CS–US pairings are part of the context of con- The passage of time might also bring about changes in internal ditioning, whereas recent CS-only presentations are part of the and external stimulation that provide a gradually-changing con- context of extinction. When CS–US pairings are resumed after text. Pavlov (1927) first observed another well-known extinction extinction, they would thus return the animal to the original effect. In spontaneous recovery, if time is allowed to pass follow- conditioning context. The hypothesis is compatible with Capal- ing extinction, the extinguished response can recover. There are di’s (1967, 1994) sequential analysis of extinction, which has several available explanations of spontaneous recovery (for a dis- made excellent use of the idea that responding on a particular cussion of alternatives, see Robbins 1990; Brooks and Bouton trial is determined by how the animal has learned to respond in 486 Learning & Memory www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Context, Inhibition, and Extinction the presence of similar memories of previous trials (see below). may cause a return of responding after extinction because of Presumably, conditioning preparations that use a relatively large another ABA renewal effect (Bouton et al. 1993). number of conditioning trials allow ample opportunity for the In summary, a variety of research indicates that responding animal to learn that previous reinforced trials are part of the to an extinguished CS is susceptible to any of a number of recov- context of conditioning. Ricker and Bouton (1996) also reported ery effects, suggesting that extinction is not unlearning. Indeed, evidence that high responding during the reacquisition phase based on the results of a number of tests that allow a specific was more likely after a reinforced than a nonreinforced trial, which comparison of the strength of the CS–US association before and presumably signaled conditioning and extinction, respectively. after extinction (see Delamater 1996; Rescorla 1996), Rescorla In more recent experiments, Bouton et al. (2004) reasoned (2001) has suggested that extinction involves no unlearning that if rapid reacquisition is caused by recent reinforced trials whatsoever; the original CS–US association seems to survive es- generating ABA renewal, then an extinction procedure that in- sentially intact. Extinction must thus depend on other mecha- cludes occasional reinforced trials among many nonreinforced nisms. The renewal effect, and the fact that extinction leaves the trials should slow down rapid reacquisition by making recent CS so especially sensitive to manipulations of context, is consis- reinforced trials part of the context of both conditioning and tent with the idea that extinction involves new learning that is extinction. Consistent with this hypothesis, a very sparse partial especially context-dependent. We have therefore suggested that reinforcement procedure in extinction slowed reacquisition in a extinction leaves the CS under a contextually modulated form of final phase compared with a group that had received simple ex- inhibition (see Bouton 1993): The presence of the extinction con- tinction. Such a result is consistent with the idea that rapid re- text retrieves or sets the occasion for a CS–no US association. acquisition is at least partly an ABA renewal effect. Because the partial reinforcement treatment involved many more CS–US Other Phenomena With Theoretical Links to Extinction pairings than did simple extinction, it is difficult to reconcile Several behavioral phenomena have been linked theoretically with the view that rapid reacquisition is a simple function of the with extinction, and it is worth considering them to see what strength of an association that remains after extinction (see Ke- insights they provide. hoe 1988; Kehoe and Macrae 1997). Counterconditioning and Other Interference Paradigms Reinstatement In counterconditioning, a CS that has been associated with one A fourth context-dependent extinction phenomenon is rein- US is associated with a second US, often incompatible with the statement. In this effect, the extinguished response returns after first, in a second phase. Not surprisingly, performance corre- extinction if the animal is merely reexposed to the US alone (see sponding to the second association replaces performance corre- Pavlov 1927; Rescorla and Heth 1975; Bouton and Bolles 1979b). sponding to the first. Counterconditioning is thus a paradigm If testing of the CS is contemporaneous with US delivery, then that, similar to extinction, involves a form of retroactive inter- the USs may cause a return of responding because they were ference. encoded as part of the conditioning context (as above; see Reid The literature on counterconditioning is not as large as the 1958; Baker et al. 1991; Bouton et al. 1993). On the other hand, literature on extinction. But there is evidence that similar prin- in many studies of reinstatement, testing is conducted at an in- ciples may apply. For example, experiments in my own labora- terval of at least 24 h after US re-exposure; here one still observes tory have demonstrated a renewal effect (Peck and Bouton 1990): reinstatement compared with controls that were not re-exposed If rats receive CS–shock pairings in one context and then CS– to the US (see Rescorla and Heth 1975; Bouton and Bolles 1979b). food pairings in another, the original fear performance returns In this case, evidence strongly suggests that the effect is due to (and replaces food performance) when the animals are returned conditioning of the context. When the US is presented after ex- to the original context. Complementary results were obtained tinction, the organism associates it with the context; this con- when CS–food preceded CS–shock. Other experiments have dem- textual conditioning then creates reinstatement. For example, if onstrated spontaneous recovery (Bouton and Peck 1992): In this the reinstating USs are presented in an irrelevant context, there is case, after CS–shock and then CS–food, animals tested at a 1-d no reinstatement when the CS is tested again (see Bouton and retention interval showed primarily appetitive performance, Bolles 1979b; Bouton and King 1983; Bouton 1984; Baker et al. whereas animals tested 28 d later showed a recovery of fear per- 1991; Wilson et al. 1995; Frohardt et al. 2000). Independent mea- formance (and a suppression of appetitive). A complementary sures of contextual conditioning also correlate with the strength pattern was observed when CS–food preceded CS–shock. Finally, of reinstatement (Bouton and King 1983; Bouton 1984). And if we have observed reinstatement (Brooks et al. 1995): When CS– the animal receives extensive extinction exposure to the context food follows CS–shock, a number of noncontingent shocks de- after the reinstatement shocks are presented, reinstatement is not livered in the same context (but not in a different context) can observed (Bouton and Bolles 1979b; Baker et al. 1991). These reinstate the original fear performance. Counterconditioning results indicate that mere re-exposure to the US is not sufficient thus supports at least three of the effects suggesting that extinc- to generate reinstatement. It is necessary to test the CS in the tion involves context-dependent new learning. context in which the US has been re-exposed. Bouton (1993) reviewed the behavioral literature on a num- This effect of context conditioning is especially potent with ber of “interference paradigms” in conditioning. In these para- an extinguished CS. For example, Bouton (1984) compared the digms, a CS is associated with different outcomes in successive effects of US exposure in the same or a different context on fear phases of the experiment. Importantly, performance in all such of a partially extinguished CS or another CS that had reached the paradigms is sensitive to manipulations of context and time. For same low level of fear through simple CS–US pairings (and no example, in addition to counterconditioning, the list includes extinction). Although contextual conditioning enhanced fear of discrimination reversal learning, in which two CSs (X and Y) are the extinguished CS, it had no impact on the nonextinguished reinforced and nonreinforced (e.g., X+/Y) before the relation- CS (see also Bouton and King 1986). This result is consistent with ship is reversed in a second phase (X/Y+). The list also includes the effects of context switches mentioned above: An extin- latent inhibition, in which a single CS is nonreinforced on a guished CS is especially sensitive to manipulations of the con- number of trials before it is paired with the US in a second phase. text. One reason is that contextual conditioning may be another In both paradigms, if the first and second phase are conducted in feature of the conditioning context; its presence during a test separate contexts, a return to the phase 1 context can cause a Learning & Memory 487 www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Bouton renewal of phase 1 performance (discrimination reversal: Spear et and Y. Because Y has no other association, it will become a pure al. 1980; Thomas et al. 1981; Bouton and Brooks 1993; latent conditioned inhibitor. For X, however, the inhibition is overlaid inhibition: Bouton and Swartzentruber,1989; Maren and Holt on its existing (and unchanged) excitatory association. The same 2000; Westbrook et al. 2000). Effects of time related to sponta- thing occurs in simple extinction, because the CS also activates neous recovery have also been observed (discrimination reversal: the US into A2 at that time. Because these processes occur in real Gordon and Spear 1973; Thomas et al. 1984; Bouton and Brooks time, during any nonreinforced trial, inhibition will accrue to the 1993; latent inhibition: Kraemer et al. 1991; Aguado et al. 1994; CS from the point in time at which the US node is first activated cf. De la Casa and Lubow 2000, 2002; Lubow and De la Casa to A2 until the CS leaves the A1 state, which may not occur until 2002). Bouton (1993) argued that a retrieval account that accepts the CS is turned off at the end of the trial (for further discussion, that both phases are learned and available, and that performance see the section Violation of Reinforcer Expectation below). is therefore determined by which is retrieved, can go some dis- Because of these connections between inhibition and ex- tance in explaining all examples of interference (see also Spear tinction, and the evidence that extinction is more context- 1981). At a broad level of analysis, then, extinction is just one specific than is simple conditioning, Bouton and Nelson (1994) example from a set of interference phenomena that all depend and Nelson and Bouton (1997) asked whether pure inhibition on context and time. acquired in the feature-negative paradigm was also context- specific. The designs of the experiments are sketched in Table 1. Conditioned Inhibition Rats were given different feature-negative discriminations in a Several theories of conditioning specifically attribute extinction series of intermixed sessions in two contexts (A and B), as shown to a build-up of inhibition (see Konorski 1948, 1967; Pearce and at left. The design in the upper half of Table 1 asks whether Hall 1980; Wagner 1981; Pearce 1994; see also Pavlov 1927). Al- inhibition conditioned to the feature in context A (CS Y) trans- though the Rescorla-Wagner model (1972) attributed extinction fers to the other context (context B) in the final test; an inhibitor to unlearning rather than inhibitory learning, it provided a con- (Y) was tested in its original context and/or in another one. Sur- ceptualization of inhibition that has been extremely influential. prisingly, its inhibition transferred without measurable disrup- Specifically, inhibition was viewed as a negative form of learning tion to the new context. However, we also performed experi- that develops when the summed strengths of all the CSs present ments along the lines shown in the lower portion of Table 1. on a conditioning trial “overpredict” the magnitude of the US These asked whether any inhibition acquired by stimulus X, that actually occurs on the trial. Most models now use a version rather than the pure inhibitory Y, is lost with a context switch. of its error-correction learning mechanism, which essentially Here there was clear evidence of a context effect: When switched brings the overall expectation of the US (the summed values of to the alternate context, responding to X became more difficult the CSs present) into line with reality (the value of the US that is to inhibit. Stimulus X, with its mixed history of both reinforce- actually present); this means decrementing associative strengths ment and nonreinforcement, is similar to an extinguished CS: Its until what is predicted equals what actually occurs. inhibitory association is context-specific. But Y’s pure inhibition According to this view, the main method for generating is not. pure conditioned inhibition is the conditioned inhibition or fea- Because inhibition was not generally context-specific, the ture-negative paradigm, in which the experimenter intermixes results implied that extinction is not context-specific merely be- trials on which one CS (X) is paired with the US and other trials cause it is a form of inhibition. A second reason why extinction on which X is combined with another CS (Y) and presented with- might be context-specific is that it is the second thing the organ- out a US (X+, XY). (X is sometimes called the “target” stimulus ism has learned about the CS. Notice that the same is true in and Y the “feature” stimulus.) After early X+ trials, X begins to other interference paradigms, such as counterconditioning: It is predict that the US will occur; when no US occurs on XY trials, the second-learned association that seems dependent on the con- the error-correction mechanism therefore decrements both CSs text and on time. And it is presumably also true of the target to bring their strengths in line with no US. In the Rescorla- stimulus (X) in the feature-negative design; there must be exci- Wagner model, X will lose some excitatory strength on the nega- tation to X before the XY trials can generate inhibition. Nelson tive trials, but it will not go below zero. But because Y starts with (2002) confirmed the second-association hypothesis in a series of zero value, its associative strength becomes negative and thus experiments. As usual, excitatory conditioning (tone-food con- becomes a pure conditioned inhibitor. ditioning) transferred undisturbed across contexts, unless the Other models actually predict inhibitory learning to both Y tone had first been trained as a conditioned inhibitor (stimulus Y and X in this paradigm (see Pearce and Hall 1980; Wagner 1981; in the feature-negative paradigm). Conversely, inhibition to a Wagner and Brandon 1989, 2001). For example, Wagner (1981) conditioned inhibitor also transferred across contexts unless the and Wagner and Brandon (1989, 2001) have presented a com- prehensive real-time model that expands enormously on the Res- corla-Wagner model and is worth describing here. According to Table 1. Designs Used by Bouton and Nelson (1994) and the model, known as SOP (for Sometimes-Opponent-Process), CS Nelson and Bouton (1997) and US are represented as memory nodes that can become asso- ciated during conditioning. For the association between them to Training Testing be strengthened, both nodes must be activated from inactivity to Is inhibition of the feature (Y) context-specific? an active state, A1, at the same time. Once the association has A: X+. XY A: X, XY been formed, the presentation of the CS activates the US node to B: X+, XZ B: X, XY a secondarily-active state, A2. This in turn generates the condi- Is inhibition of the target (X) context-specific? tioned response. An inhibitory connection is formed between a A: X+, XY A: X, XY CS and a US when the CS is activated to the A1 state and the US B: Z+, ZY B: X, XY is activated to A2 rather than A1. These conditions are met in the A and B are contexts; X, Y, and Z are CSs. + indicates reinforced; , feature-negative paradigm (X+, XY). After some initial condi- nonreinforced. During training, all rats received intermixed sessions in tioning, X is able to activate the US node to A2. On XY trials, both contexts. Testing was then conducted in both contexts with the US node is therefore put into the A2 state at the same time either between-subject or within-subject methods. that X and Y are in A1. This will create inhibition to both X 488 Learning & Memory www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Context, Inhibition, and Extinction CS had first been trained as a conditioned excitor (through initial CS in most experiments), and therefore undergoes extinction to tone–food pairings). Thus, regardless of whether the association early parts of the CS (see Rosas and Alonso 1996). Interestingly, was excitatory or inhibitory, the second thing learned was more Rosas and Alonso (1997) have shown that inhibition of delay, context-specific than the first. Compatible data had been shown developing over trials this way, is in fact attenuated with a con- by Swartzentruber and Bouton (1992), who found that excitatory text switch. However, in my laboratory the nonmonotonicity is conditioning was relatively context-specific if it had been pre- still observed when shock occurs at unpredictable times in the ceded by nonreinforced pre-exposure to the CS. CS, and there is again an increase in fear when the context is Thus, the evidence suggests that the learning and memory changed under these conditions. The result is also not due to an system treats the first association as context-free, but the second opioid process that helps the animal adapt to the US (see Vigorito association as a kind of context-specific exception to the rule. and Ayres 1987); indeed, because our context switch experiments There may be functional reasons for this (Bouton 1994). A con- test the rat’s reaction to the CS, not the US, the results indicate ditioning trial provides a sample from which an animal may some sort of adaptation to the CS or to conditioned fear itself. make inferences about the state of the world (see Staddon 1988). Another possibility is suggested by SOP (see Wagner 1981): Re- Statistically, if the world is composed of two types of trials (CS– peated exposure to the CS would allow the animal to associate US and CS–no US), then the probability of sampling a particular the CS with the context, which would permit the context to type of trial will reflect its true prevalence in the world. There- associatively activate the CS into the secondarily active state fore, an early run of conditioning trials would reflect its high (A2). Putting the CS into that state might reduce its ability to incidence in the population; a subsequent trial of another type evoke a response (see Hall and Honey 1990; Honey et al. 1993; might reflect an exception to the rule. Learning and memory Hall and Mondragón 1998). But perhaps contrary to this view, may thus be designed to treat second-learned information as a there is little correlation between the number of CSs presented in conditional and context-specific. A mechanistic account has not the conditioning procedure and the degree of nonmonotonicity. been fully tested. One possibility is that when extinction begins, In addition, we have yet to observe nonmonotonic learning the omission of the US is surprising and boosts attention to the curves in appetitive conditioning, in which the animal also re- CS and context, as suggested by the attention rule of the Pearce- ceives many opportunities to associate the context and CS. Al- Hall model (Pearce and Hall 1980; see also Kaye and Pearce 1984). though we do not yet understand the inhibition-despite- Interestingly, there is evidence that conditioned excitation may reinforcement phenomenon, there is a suggestion that, similar to be context specific after one conditioning trial—when the US is extinction, it involves a context-specific inhibitory process that also new and surprising (Hall and Honey 1990). might be the second thing learned about the CS. Inhibition Despite Continued Reinforcement What Causes Extinction? Other research has identified another inhibitory effect that de- If we return specifically to extinction, another question is what serves mention alongside extinction. Ayres et al. (1979; see also event or behavioral process actually causes the loss of respond- Vigorito and Ayres 1987) have shown in the conditioned sup- ing? Several ideas have been examined in recent experiments. pression preparation that repeated conditioning trials may yield nonmonotonic learning curves in which conditioned fear Discrimination of Reinforcement Rate reaches a peak and then declines despite continued pairings of One possibility is that the animal eventually learns that the rate the CS and shock. The process is poorly understood, although of reinforcement in the CS is lower in extinction than it was similar effects have been observed in several conditioning prepa- during conditioning. Gallistel and Gibbon (2000) have argued rations (Kimmel and Burns 1975). Pavlov himself saw it often that the animal continually decides whether or not to respond in (1927), and attributed it to inhibition developing despite rein- extinction by comparing the current rate of reinforcement in the forcement. CS with its memory of the rate that prevailed in conditioning. Experiments in my laboratory have further documented the Because rate is the reciprocal of time, the animal computes a ratio effect in the conditioned suppression preparation (M.E. Bouton, between the amount of time accumulated in the CS during ex- R.J. Frohardt, C. Sunsay, and J. Waddell, in prep.). We have also tinction and the amount of time accumulated in the CS between discovered a role for context. Rats received intermixed sessions in USs during conditioning. When the ratio exceeds a threshold, two contexts; in one context, one CS was repeatedly paired with the animal stops responding. footshock, and in the other, a second CS was similarly paired This approach has been tested in several recent experiments. with the same shock. Nonmonotonic learning curves developed. Haselgrove and Pearce (2003) examined the impact of varying However, when we then tested the CS in the alternate context, the duration of the CS during extinction; when longer CSs are we saw a significant increase in fear of the CS. Over experiments, used in extinction, time in the CS accumulates more quickly, and there was a strong correlation between the size of this increase the animal should stop responding after fewer trials. In some and the degree of nonmonotonicity shown in the conditioning experiments, rats were given appetitive conditioning with a 10- curve. The context switch thus attenuated the inhibitory process sec CS and then given extinction exposures to a series of 10-sec or that led to the nonmonotonic learning curve. 270-sec presentations of the CS. When responding was examined We do not have a good understanding of this inhibitory at the start of each CS, there was an occasionally significant, but process at the present point in time. In our experiments, the surprisingly small, effect of increasing the duration of the CS decline in fear over training is not a result of inhibition of delay, during extinction. For instance, by the 12th two-trial block, the in which the animal learns the timing of the US (at the end of the 10-sec and 270-sec CS groups had similar nonzero levels of re- sponding, even though they had accumulated a total of 4 and 108 min of exposure in the CS, respectively. On the other hand, The main exception to the second-association rule is latent inhibition, in responding did decline as a function of time within a single pre- which the first phase can be shown to exert a context-dependent influence on the second phase (see Hall and Channell 1985) despite the fact that it is sentation of the 270-sec CS, perhaps reflecting generalization arguably the first thing learned. Latent inhibition is unique, however, in that decrement resulting from the increasing difference between the the CS is not paired with anything significant in the first phase. One possibility, current CS and the 10-sec CS employed in conditioning. Consis- therefore, is that it is in part encoded as a feature of the context, making it tent with that view, when conditioning first occurred with a difficult to extract it from that context when it is paired with the US in phase 2 (cf. Gluck and Myers 1993). 60-sec CS, extinction of responding occurred more rapidly with a Learning & Memory 489 www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Bouton 10-s CS than with a 60-sec CS. Thus, either an increase or a quisition as an ABA renewal effect (Ricker and Bouton 1996; Bou- decrease in the duration of the CS relative to conditioning accel- ton et al. 2004). Interestingly, the recent finding that occasional erated the loss of responding. This effect of time was not antici- reinforced trials in extinction (partial reinforcement) can slow pated by the rate-discrimination view (Gallistel and Gibbon down the rate of reacquisition (Bouton et al. 2004) is really just 2000). the inverse of the PRE: In the PRE, nonreinforced trials in con- Drew et al. (2004) reported compatible results in experi- ditioning allow more generalization from conditioning to extinc- ments on autoshaping in ring doves. Doubling or halving the tion, whereas Bouton et al.’s finding suggests that reinforced tri- duration of the CS from the 8-sec value used in conditioning did als in extinction allowed for more generalization of extinction to not affect the number of trials required to stop responding. The reconditioning. Either finding suggests the importance of con- fact that extinction was thus largely controlled by the number of sidering recent trials as part of the context that controls perfor- CS presentations is consistent with experiments that have exam- mance in extinction. ined the effects of the number and duration of nonreinforced In summary, there is little support for the idea that respond- trials added to conditioning schedules (Bouton and Sunsay ing extinguishes when the US is omitted because the organism 2003). On the other hand, Drew et al. (2004) found that a more detects a lower rate of reinforcement in the CS. The number of extreme increase in CS duration (from 8 to 32 sec) increased the extinction trials, rather than merely the accumulating time in rate of extinction. This was attributed to the animal learning to the CS across trials, appears to be important to the extinction discriminate the longer nonreinforced CS presentations from the process. Time in the CS can have an effect: It appears to be an- shorter reinforced CS presentations: When 8-sec CSs were pre- other dimension over which animals generalize and discriminate sented again after extinction, birds extinguished with 4-sec and (Haselgrove and Pearce 2003; Drew et al., 2004). But explanation 32-sec CSs responded again. Animals are sensitive to time in the of the PRE appears to be most consistent with a view that animals CS, but the number of extinction trials appears to be an impor- use their memories of the outcomes of preceding trials as a di- tant factor. mension over which they generalize and respond (for a more As noted by Gallistel and Gibbon (2000), the rate discrimi- extended review, see also Mackintosh 1974). nation theory seems especially consistent with a well-known ex- Generalization Decrement tinction phenomenon, the partial reinforcement effect (PRE; for a review, see Mackintosh, 1974). In this phenomenon, condi- It is thus possible to claim that the animal stops responding in tioning with partial reinforcement schedules (in which nonrein- extinction at the point at which it stops generalizing between the forced trials are intermixed with reinforced trials) creates a slower stimuli that prevailed in conditioning than those that prevail in loss of responding in extinction than does conditioning with a extinction (see Capaldi 1967, 1994). This idea has had a long and continuous reinforcement schedule (in which every trial is rein- influential history in research on extinction, especially in re- forced). According to a rate-discrimination hypothesis (Gallistel search on the PRE. It is interesting to note that a generalization and Gibbon 2000), the partially reinforced subjects have learned decrement theory of extinction does not imply destruction of the to expect the US after more accumulated time in the CS, and it original learning in extinction, or indeed any new learning at all. thus takes more CS time in extinction to exceed the threshold of However, there is still good reason to think that extinction also accumulated extinction time/expected time to each US. The involves new learning. For instance, nonreinforcement of a food more traditional approach, in contrast, has been to think that CS elicits measurable frustration, and this can be associated with partially reinforced subjects have learned to expect the US after stimuli present in the environment (Daly 1974). Nonreinforce- more trials than continuously reinforced subjects have. It there- ment of the CS in a feature-negative paradigm also generates fore takes more trials to stop generalizing from conditioning to measurable new learning in the form of conditioned inhibition extinction (see Mowrer and Jones 1945; Capaldi 1967, 1994). (see above). And there is also evidence for new learning in the Contrary to the rate discrimination hypothesis, Haselgrove renewal effect. For example, either ABC renewal or AAB renewal et al. (2004) and M.E. Bouton and A.M. Woods (in prep.) have (see above) imply that context B (the extinction context) ac- shown that a PRE still occurs when partially and continuously quires an ability to modulate (suppress) performance to the CS. reinforced subjects expect the reinforcer after the same amount Such observations suggest that the animal has not merely of CS time. For example, both sets of investigators showed that a stopped responding in extinction because of a failure to general- group that received a 10-sec CS reinforced on half its presenta- ize. Instead, it appears to have learned that the CS means no US tions (accumulated CS time of 20 sec) extinguished more slowly in the extinction context (see above). than did a continuously reinforced group that received every Role of Inhibition of the Response 20-sec CS presentation reinforced. M.E. Bouton and A.M. Woods (in prep.) further distinguished the “time-discrimination” ac- Rescorla (2001) has recently suggested that extinction might in- count from the traditional “trial-discrimination” account (see volve learning to inhibit the conditioned response. For example, Mowrer and Jones 1945; Capaldi 1967, 1994). Rats that had every he has summarized evidence from instrumental (operant) condi- fourth 10-sec CS reinforced extinguished more slowly over a se- tioning experiments indicating that the effects of extinction can ries of alternating 10-sec and 30-sec extinction trials than rats be specific to the response that undergoes extinction. For ex- that had received every 10-sec CS reinforced. This PRE was still ample, Rescorla (1993) reinforced two operant behaviors (lever observed when extinction responding was plotted as a function pressing and chain pulling) with food pellets and then extin- of time units over which the US should have been expected (ev- guished each response in combination with a new stimulus (a ery 40 sec for the PRF group but every 10 sec for the CRF group). light or a noise). Subsequent tests of the two responses with both In contrast, the PRE disappeared when extinction responding light and noise indicated that each response was more depressed was plotted as a function of the trials over which the US should when it was tested in combination with the cue in which it had have been expected (every fourth trial for the PRF group and been extinguished (see also Rescorla 1997). There is thus good every trial for the CRF group). Ultimately, the PRE is better cap- reason to think that the animal learns something specific about tured by trial-based theories (e.g., Capaldi 1967, 1994). the response itself during operant extinction: It learns not to We have already seen that responding on a particular trial perform a particular response in a particular stimulus. One pos- occurs in the context of memories of the outcomes of previous sibility is that the animal learns a simple inhibitory S-R associa- trials—that was the explanation provided earlier of rapid reac- tion (Colwill 1991). Another possibility, perhaps more consistent 490 Learning & Memory www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Context, Inhibition, and Extinction with the context-modulation account of extinction emphasized and M.E. Bouton, in prep.) suggest a similar conclusion even above, is that the animal learns that S sets the occasion for a though the results were different. Spaced extinction trials again response–no reinforcer relationship. Rescorla (1993, p. 335; 1997, yielded more responding in extinction than did massed trials, p. 249) has observed that the experiments do not separate the but the treatments caused indistinguishable amounts of extinc- two possibilities. To my knowledge, no analogous experiments tion learning as assessed in spontaneous recovery and reinstate- have been performed in the Pavlovian conditioning situation. ment tests. In related conditioned suppression experiments, M.E. Instead, the main implication examined in Pavlovian con- Bouton, A. García-Gutiérrez, J. Zilski, and E.W. Moody (in prep.) ditioning is that extinction procedures should be especially suc- compared the effects of extinction in multiple contexts on the cessful at causing inhibitory S-R learning if they generate high strength of the ABA and ABC renewal effects. Rats received fear levels of responding in extinction. This prediction may provide a conditioning with a tone CS in context A, and then extinction of reasonable rule of thumb (Rescorla 2001). For example, when a the tone for three sessions in context B, or a session in B, then C, CS is compounded with another excitatory CS and the com- and then D, before final renewal tests in the original context pound is extinguished, there is especially strong responding in (context A) or a neutral fifth context (context E). Although the extinction (due to summation between the CSs), and especially successive context switches in the BCD group caused more fear effective extinction as evidenced when the CS is tested alone responding during extinction (due to renewal effects), the groups (Wagner 1969; Rescorla 2000; Thomas and Ayres 2004). Con- showed strikingly similar renewal in either context A or context versely, when the target CS is compounded with an inhibitory E. Gunther et al. (1998) and Chelonis et al. (1999) have shown CS, there is relatively little responding to the compound (excita- more favorable effects of extinction in multiple contexts on re- tion and inhibition negatively summate), and there is also less newal. But higher responding in extinction does not guarantee evidence of extinction when the target is tested alone (Soltysik et better extinction learning. The results seem inconsistent with a al. 1983; Rescorla 2003; Thomas and Ayres 2004). However, al- response-inhibition hypothesis. Their impact on the expectancy though these findings are consistent with the hypothesis that the violation hypothesis is perhaps less clear. effectiveness of extinction correlates with the degree of respond- In summary, although animals that receive extinction after ing, they can also be interpreted in a different way. Either treat- operant conditioning may in fact learn to refrain from perform- ment also affects the degree to which the animal’s expectation of ing a particular response in a particular context (see Rescorla the reinforcer is violated: The stimulus compound influences the 1993, 1997), the importance of response inhibition in Pavlovian size of the error term in the Rescorla-Wagner model and, in more extinction is not unequivocally supported at the present time. cognitive terms, the extent to which the expectation of the US High responding in extinction does not guarantee more effective created by the compound is violated when the US does not occur. extinction learning. To date, the hypothesis has been difficult to The results do not separate the response-inhibition hypothesis distinguish from the violation-of-expectation hypothesis that is from an expectancy-violation hypothesis, which will be covered built into many mainstream models of classical conditioning (see in the next section. Rescorla and Wagner 1972; Pearce and Hall 1980; Wagner 1981; A recent eyeblink experiment by Krupa and Thompson Wagner and Brandon 1989, 2001) and is considered next. (2003) manipulated the level of responding another way. During Violation of Reinforcer Expectation extinction, rabbits were given microinjections of the GABA ago- nist muscimol adjacent to the motor nuclei that control the con- It is commonly thought that each CS presentation arouses a sort ditioned response (the facial nucleus and the accessory abdu- of expectation of the US that is disconfirmed on each extinction cens). The injection therefore eliminated the CR during extinc- trial. For example, in the error-correction rule provided by Res- tion. However, when the subjects were then tested without corla and Wagner (1972), the degree of unlearning (which we muscimol, the CS evoked considerable responding, suggesting have seen can create inhibition) is provided by the difference in that evocation of the CR was necessary for extinction learning. the overall associative strength present on a trial and the actual Unfortunately, the muscimol microinjections also had robust US that occurs on the trial. In the Pearce-Hall model (Pearce and stimulus effects. They caused complete inactivation of the ipsi- Hall 1980), the discrepancy was conceptualized as an event that lateral facial musculature: “the external eyelids were flaccid, the reinforced new inhibitory learning that is overlaid on the origi- left ear hung down unsupported, and no vibrissae movements nal excitatory learning (see also Daly and Daly 1982). Wagner’s were observed on the side of the infusion” (p. 10579). In effect, SOP model (1981) accepts a similar idea. One piece of evidence the rabbits received extinction in a context that was different that seems especially consistent with the expectation-violation from the one in which conditioning and testing occurred (the view is the “overexpectation experiment,” in which two CSs are ordinary state without partial facial paralysis). There are thus separately associated with the US and then presented together in strong grounds for expecting a renewal effect. The hypothesis a compound that is then paired with the US. Despite the fact that that elicitation of the CR is necessary for extinction must await the compound is paired with a US that can clearly generate ex- further tests. citatory learning, the two CSs undergo some extinction (see Kre- There are also data suggesting that the number of responses mer 1978; Lattal and Nakajima 1998). The idea is that summa- or level of responding in extinction does not correlate with ef- tion of the strengths of the two CSs causes a discrepancy between fective extinction learning. For example, Drew et al. (2004) noted what the animal expects and what actually occurs, and some that although animals given long CSs in extinction responded extinction is therefore observed. As mentioned above, the expec- many more times in extinction than did animals given shorter tation-violation view is also consistent with the effects of com- CSs, extinction was mainly a function of the number of extinc- pounding excitors and inhibitors with the target CS during no- tion trials. In fear conditioning experiments with mice, Cain et US (extinction) trials (Wagner 1969; Soltysik et al. 1983; Rescorla al. (2003) reported that extinction trials that were spaced in time 2000, 2003; Thomas and Ayres 2004). produced a slower loss of freezing than did extinction trials that One theoretical challenge has been to capture the expect- were massed in time. Nevertheless, there was less spontaneous ancy violation in real time. Gallistel and Gibbon (2000) have recovery after the massed treatment, suggesting that extinction emphasized the fact that traditional trial-based models such as was more effective when the treatment involved less overall re- the Rescorla-Wagner model have been vague about the precise sponding. Experiments in my own laboratory with different ap- point in time in a trial when the violation of expectation actually petitive conditioning methods in rats (E.W. Moody, C. Sunsay, occurs. The issue is especially clear when trial-based models ex- Learning & Memory 491 www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Bouton plain the extinction that occurs with a single extended presen- current performance depends on which of two associations is tation of the CS, as is the case for the context or background in retrieved. Consistent with this idea, another fact that emerges conditioning protocols with very widely spaced conditioning tri- from behavioral research on extinction is that it is relatively con- als. (Spaced trials are held to facilitate conditioning of the CS text-dependent. I have therefore suggested that the second (in- because long intertrial intervals allow more context extinction hibitory) association of CS is especially dependent on the context and thus less blocking by context.) There is good evidence that for its activation or retrieval. The role of the context is modula- widely spaced trials do create less contextual conditioning than tory; its activates or retrieves the CS’s own second (inhibitory) massed trials (see Barela 1999). To account for contextual extinc- association, much as a negative occasion setter might (see Hol- tion over long intertrial intervals, many trial-based models arbi- land, 1992). This hypothesis begins to integrate several facts trarily assume that the single long context exposure is carved about extinction and brings relapse effects such as the renewal into many imaginary trials, and that more imaginary trials occur effect, spontaneous recovery, rapid reacquisition, and reinstate- and create more extinction in longer context exposures. ment to center stage. Extinction is not the same as unlearning, It is worth noting, however, that Wagner’s SOP model (see and the context-dependence of extinction performance is a cen- Wagner 1981; Wagner and Brandon 1989, 2001) is relatively spe- tral part of that insight. cific about when in time the process that generates extinction occurs. As already mentioned, extinction occurs because an in- ACKNOWLEDGMENTS hibitory CS–US association develops when the CS is in the A1 I thank John Pearce for many useful discussions. Preparation of state and the US is in the A2 state. After conditioning has oc- this article was supported by Grant RO1 MH64847 from the Na- curred, the CS will continuously activate elements of the US node tional Institute of Mental Health. It was written while the author to the A2 state as long as it is present; therefore, extinction learn- was a visiting fellow at Cardiff University under the support of ing will occur continuously as long as the CS is on and no US the Cardiff Visiting Fellows Scheme. occurs. A limiting factor, however, is the extent to which the CS itself is in the A2 state: The longer it remains on, the more likely REFERENCES the elements in the CS node will be in A2 rather than A1, making Aguado, L., Symonds, M., and Hall, G. 1994. Interval between new learning about the CS more difficult. Nonetheless, exten- preexposure and test determines the magnitude of latent inhibition: sions of the CS in extinction will have an effect, because elements Implications for an interference account. Anim. Learn. Behav. 22: 188–194. in A2 eventually return to the inactive state, from which they Ayres, J.J.B., Berger-Gross, P., Kohler, E.A., Mahoney, W.J., and Stone, S. will return to A1 because of the continued presence of the CS. 1979. Some orderly nonmonotonicities in the trial-by-trial SOP thus accounts for extinction in extended CSs without re- acquisition of conditioned suppression: Inhibition with course to imaginary trials, and a recent extension of the model reinforcement? Anim. Learn. Behav. 7: 174–180. Baker, A.G., Steinwald, H., and Bouton, M.E. 1991. Contextual (Vogel et al. 2003) may also account for generalization decre- conditioning and reinstatement of extinguished instrumental ment as a function of CS time (Haselgrove and Pearce 2003; Drew responding. Q. J. Exp. Psychol. B 43: 199–218. et al. 2004). Although a truly rigorous analysis of SOP requires Barela, P.B. 1999. Theoretical mechanisms underlying the trial spacing computer simulations that are beyond the scope of the present effect in Pavlovian conditioning. J. Exp. Psychol. Anim. Behav. Processes 25: 177–193. article, the principles contained in the model are consistent with Bouton, M.E. 1984. Differential control by context in the inflation and many of the facts of extinction reviewed here. From the current reinstatement paradigms. J. Exp. Psychol. Anim. Behav. Processes point of view, its most significant problem is that it underesti- 10: 56–74. mates the role of context in extinction and might not account for . 1986. Slow reacquisition following the extinction of conditioned suppression. Learn. Motivat. 17: 1–15. the negative occasion-setting function of context (see Bouton . 1988. Context and ambiguity in the extinction of emotional and King 1983; Bouton and Swartzentruber 1986, 1989; Bouton learning: Implications for exposure therapy. Behav. Res. Ther. and Nelson 1998) that arguably provides the key to understand- 26: 137–149. ing renewal, spontaneous recovery, rapid reacquisition, and re- . 1991. Context and retrieval in extinction and in other examples of interference in simple associative learning. In: Current topics in instatement phenomena (for a start at addressing occasion set- animal learning: Brain, emotion, and cognition (eds. L. Dachowski and ting phenomena in terms of SOP, see Brandon and Wagner 1998; C.F. Flaherty), pp. 25–53. Lawrence Erlbaum, Hillsdale, NJ. Wagner and Brandon 2001). . 1993. Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychol. Bull. 114: 80–99. . 1994. Conditioning, remembering, and forgetting. J. Exp. Psychol. Anim. Behav. Processes 20: 219–231. Conclusions . 2002. Context, ambiguity, and unlearning: Sources of relapse Extinction is a highly complex phenomenon, even when ana- after behavioral extinction. Biol. Psychiatry 52: 976–986. lyzed at a purely behavioral level. It is worth noting that it is Bouton, M.E. and Bolles, R.C. 1979a. Contextual control of the extinction of conditioned fear. Learn. Motivat. 10: 445–466. probably multiply determined. But according to the results re- . 1979b. Role of conditioned contextual stimuli in reinstatement of viewed here, it does not involve destruction of the original learn- extinguished fear. J. Exp. Psychol. Anim. Behav. Processes 5: 368–378. ing. Instead, the main behavioral factors that cause the loss of Bouton, M.E., and Brooks, D.C. 1993. Time and context effects on responding appear to be generalization decrement, and new performance in a Pavlovian discrimination reversal. J. Exp. Psychol. Anim. Behav. Processes 19: 165–179. learning that may be initiated by the violation of an expectation Bouton, M.E., and King, D.A. 1983. Contextual control of the extinction of the US. In SOP, perhaps the most powerful and comprehensive of conditioned fear: Tests for the associative value of the context. J. model of associative learning that is currently available, that ex- Exp. Psychol. Anim. Behav. Processes 9: 248–265. pectation violation takes the form of the CS activating the US . 1986. Efect of context on performance to conditioned stimuli with mixed histories of reinforcement and nonreinforcement. J. Exp. node into a secondarily-active (A2) state that potentially enables Psychol. Anim. Behav. Processes 12: 4–15. new inhibitory learning as long as the CS remains on and no US Bouton, M.E. and Nelson, J.B. 1994. Context-specificity of target versus is presented. Importantly, this new inhibitory learning leaves the feature inhibition in a feature-negative discrimination. J. Exp. original CS–US association intact. Psychol. Anim. Behav. Processes 20: 51–65. Bouton, M.E. and Nelson, J.B. 1998. Mechanisms of feature-positive and I have argued that the fact that extinction might leave the feature-negative discrimination learning in an appetitive original learning intact means that the CS emerges from extinc- conditioning paradigm. In Occasion setting: Associative learning and tion with two available associations with the US. It therefore has cognition in animals (eds. N. Schmajuk and P.C. Holland), pp. properties analogous to those of an ambiguous word, and the 69–112. American Psychological Association, Washington, DC. 492 Learning & Memory www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Context, Inhibition, and Extinction Bouton, M.E. and Peck, C.A. 1989. Context effects on conditioning, Behav. 24: 437–449. extinction, and reinstatement in an appetitive conditioning . 2004. Experimental extinction in Pavlovian conditioning: preparation. Anim. Learn. Behav. 17: 188–198. Behavioural and neuroscience perspectives. Q. J. Exp. Psychol. Bouton, M.E. and Peck, C.A. 1992. Spontaneous recovery in 57B: 97–132. cross-motivational transfer (counter-conditioning). Anim. Learn Denniston, J.C., Chang, R.C., and Miller, R.R. 2003. Massive extinction Behav. 20: 313–321. treatment attenuates the renewal effect. Learn. Motivat. 34: 68–86. Bouton, M.E. and Ricker, S.T. 1994. Renewal of extinguished responding Devenport, L., Hill, T., Wilson, M., and Ogden, E. 1997. Tracking and in a second context. Anim. Learn. Behav. 22: 317–324. averaging in variable environments: A transition rule. J. Exp. Psychol. Bouton, M.E. and Sunsay, C. 2003. Importance of trials versus Anim. Behav. Processes 23: 450–460. accumulating time across trials in partially-reinforced appetitive Drew, M.R., Yang, C., Ohyama, T., and Balsam, P.D. 2004. Temporal conditioning. J. Exp. Psychol. Anim. Behav. Processes 29: 62–77. specificity of extinction in autoshaping. J. Exp. Psychol. Anim. Behav. Bouton, M.E. and Swartzentruber, D. 1986. Analysis of the associative Processes 30: 163–176. and occasion-setting properties of contexts participating in a Frohardt, R.J., Guarraci, F.A., and Bouton, M.E. 2000. The effects of Pavlovian discrimination. J. Exp. Psychol. Anim. Behav. Processes neurotoxic hippocampal lesions on two effects of context after fear 12: 333–350. extinction. Behav. Neurosci. 114: 227–240. Bouton, M.E. and Swartzentruber, D. 1989. Slow reacquisition following Gallistel, C.R. and Gibbon, J. 2000. Time, rate, and conditioning. extinction: Context, encoding, and retrieval mechanisms. J. Exp. Psychol. Rev. 107: 289–344. Psychol. Anim. Behav. Processes 15: 43–53. Gluck, M. and Myers, C.E. 1993. Hippocampal mediation of stimulus . 1992. Spontaneous recovery in cross-motivational transfer representation: A computational theory. Hippocampus 3: 492–516. (counterconditioning). Anim. Learn. Behav. 20: 313–321. Gordon, W.C. and Spear, N.E. 1973. Effect of reactivation of a Bouton, M.E., Kenney, F.A., and Rosengard, C. 1990. State-dependent previously acquired memory on the interaction between memories fear extinction with two benzodiazepine tranquilizers. Behav. in the rat. J. Exp. Psychol. 99: 349–355. Neurosci. 104: 44–55. Gunther, L.M., Denniston, J.C., and Miller, R.R. 1998. Conducting Bouton, M.E., Rosengard, C., Achenbach, G.G., Peck, C.A., and Brooks, exposure treatment in multiple contexts can prevent relapse. Behav. D.C. 1993. Effects of contextual conditioning and unconditional Res. Ther. 36: 75–91. stimulus presentation on performance in appetitive conditioning. Q. Hall, G. and Channell, S. 1985. Differential effects of contextual change J. Exp. Psychol. 46B: 63–95. on latent inhibition and on the habituation of an orienting Bouton, M.E., Woods, A.M., and Pineño, O. 2004. Occasional reinforced response. J. Exp. Psychol. Anim. Behav. Processes 11: 470–481. trials during extinction can slow the rate of rapid reacquisition. Hall, G. and Honey, R.C. 1990. Context-specific conditioning in the Learn. Motivat. (in press). conditioned-emotional-response procedure. J. Exp. Psychol. Anim. Brandon, S.E. and Wagner, A.R. 1998. Occasion setting: Influences of Behav. Processes 16: 271–278. conditioned emotional responses and configural cues. In Occasion Hall, G. and Mondragón, E. 1998. Contextual control as occasion setting: Associative learning and cognition in animals (eds. N.A. setting. In Occasion setting: Associative learning and cognition in Schmajuk and P.C. Holland), pp. 343–382. American Psychological animals (eds. N. Schmajuk and P.C. Holland), pp. 199–222). Association, Washington, DC. American Psychological Association, Washington, DC. Brooks, D.C. 2000. Recent and remote extinction cues reduce Harris, J.A., Jones, M.L., Bailey, G.K., and Westbrook, R.F. 2000. spontaneous recovery. Q. J. Exp. Psychol. 53B: 25–58. Contextual control over conditioned responding in an extinction Brooks, D.C. and Bouton, M.E. 1993. A retrieval cue for extinction paradigm. J. Exp. Psychol. Anim. Behav. Processes 26: 174–185. attenuates spontaneous recovery. J. Exp. Psychol. Anim. Behav. Hart, J.A., Bourne, M.J., and Schachtman, T.R. 1995. Slow reacquisition Processes 19: 77–89. of a conditioned taste aversion. Anim. Learn. Behav. 23: 297–303. . 1994. A retrieval cue for extinction attenuates response recovery Haselgrove, M. and Pearce, J.M. 2003. Facilitation of extinction by an (renewal) caused by a return to the conditioning context. J. Exp. increase or a decrease in trial duration. J. Exp. Psychol. Anim. Behav. Psychol. Anim. Behav. Processes 20: 366–379. Processes 29: 153–166. Brooks, D.C., Hale, B., Nelson, J.B., and Bouton, M.E. 1995. Haselgrove, M., Aydin, A., and Pearce, J.M. 2004. A partial Reinstatement after counterconditioning. Anim. Learn. Behav. reinforcement extinction effect despite equal rates of reinforcement 23: 383–390. during Pavlovian conditioning. J. Exp. Psychol. Anim. Behav. Processes Cain, C.K, Blouin, A.M., and Barad, M. 2003. Temporally massed CS 30: 240–250. presentations generate more fear extinction than spaced Holland, P.C. 1992. Occasion setting in Pavlovian conditioning. In The presentations. J. Exp. Psychol. Anim. Behav. Processes 29: 323–333. psychology of learning and motivation, Vol. 28 (ed. D.L. Medin), pp. Capaldi, E.J. 1967. A sequential hypothesis of instrumental learning. In 69–125. Academic Press, San Diego, CA. Psychology of learning and motivation, Vol. 1 (eds. K.W. Spence and Honey, R.C., Hall, G., and Bonardi, C. 1993. Negative priming in J.T. Spence), pp. 67–156. Academic Press, New York. associative learning: Evidence from a serial-conditioning procedure. . 1994. The sequential view: From rapidly fading stimulus traces J. Exp. Psychol. Anim. Behav. Processes 19: 90–97. to the organization of memory and the abstract concept of number. Kaye, H. and Pearce, J.M. 1984. The strength of the orienting response Psychonomic Bull. Rev. 1: 156–181. during Pavlovian conditioning. J. Exp. Psychol. Anim. Behav. Processes Chelonis, J.J., Calton, J.L., Hart, J.A., and Schachtman, T.R. 1999. 10: 90–109. Attenuation of the renewal effect by extinction in multiple contexts. Kehoe, E.J. 1988. A layered network model of associative learning: Learn. Motivat. 30: 1–14. Learning to learn and configuration. Psychol. Rev. 95: 411–433. Colwill, R.W. 1991. Negative discriminative stimuli provide information Kehoe, E.J. and Macrae, M. 1997. Savings in animal learning: about the identity of omitted response-contingent outcomes. Anim. Implications for relapse and maintenance after therapy. Behav. Ther. Learn. Behav. 19: 326–336. 28: 141–155. Crombag, H.S. and Shaham, Y. 2002. Renewal of drug seeking by Kimmel, H.D. and Burns, R.A. 1975. Adaptational aspects of contextual cues after prolonged extinction in rats. Behav. Neurosci. conditioning. In Handbook of learning and cognitive processes: 116: 169–173. Conditioning and behavior theory, Vol. 2 (ed. W.K. Estes), pp. 99–142. Cunningham, C.L. 1979. Alcohol as a cue for extinction: State Erlbaum, Hillsdale, NJ. dependency produced by conditioned inhibition. Anim. Learn. Konorski, J. 1948. Conditioned reflexes and neuron organization. Cambridge Behav. 7: 45–52. University Press, Cambridge, UK. Daly, H.B. 1974. Reinforcing properties of escape from frustration . 1967. Integrative activity of the brain. University of Chicago Press, aroused in various learning situations. In The psychology of learning Chicago. and motivation, Vol. 8 (ed. G.H. Bower), pp. 187–232. Academic Kraemer, P.J., Randall, C.K., and Carbary, T.J. 1991. Release from latent Press, New York. inhibition with delayed testing. Anim. Learn. Behav. 19: 139–145. Daly, H.B. and Daly, J.T. 1982. A mathematical model of reward and Kremer, E.F. 1978. The Rescorla-Wagner model: Losses in associative aversive nonreward: Its application in over 30 appetitive learning strength in compound conditioned stimuli. J. Exp. Psychol. Anim. situations. J. Exp. Psychol. Gen. 111: 441–480. Behav. Processes 4: 22–36. Danguir, J. and Nicolaidis, S. 1977. Lack of reacquisition in learned taste Krupa, D.J. and Thompson, R.F. 2003. Inhibiting the expression of a aversions. Anim. Learn. Behav. 5: 395–397. classically conditioned behavior prevents its extinction. J. Neurosci. De la Casa, L.G. and Lubow, R.E. 2000. Super-latent inhibition with 23: 10577–10584. conditioned taste aversion testing. Anim. Learn. Behav. 28: 389–399. Lattal, K.M. and Nakajima, S. 1998. Overexpectation in appetitive . 2002. An empirical analysis of the super-latent inhibition effect. Pavlovian and instrumental conditioning. Anim. Learn. Behav. Anim. Learn. Behav. 30: 112–120. 26: 351–360. Delamater, A.R. 1996. Effects of several extinction treatments upon the Lubow, R.E. and De la Casa, L.G. 2002. Superlatent inhibition and integrity of Pavlovian stimulus-outcome associations. Anim. Learn. spontaneous recovery: Differential effects of pre- and Learning & Memory 493 www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Bouton postconditioning CS-alone presentations after long delays in 27: 43–57. different contexts. Anim. Learn. Behav. 30: 376–386. . 1997. The effect of context change upon long-term memory of Mackintosh, N.J. 1974. The psychology of animal learning. Academic Press, CS duration. Behav. Processes 39: 69–76. New York. Rosas, J.M. and Bouton, M.E. 1997. Additivity of the effects of retention Maren, S. and Holt, W. 2000. The hippocampus and contextual memory interval and context change on latent inhibition: Toward resolution retrieval in Pavlovian conditioning. Behav. Brain Res. 110: 97–108. of the context forgetting paradox. J. Exp. Psychol. Anim. Behav. McClelland, J.L. and Rumelhart, D.E. 1985. Distributed memory and the Processes 23: 283–294. representation of general and specific information. J. Exp. Psychol. . 1998. Context change and retention interval can have additive, Gen. 114: 159–188. rather than interactive, effects after taste aversion extinction. McCloskey, M. and Cohen, N.J. 1989. Catastrophic interference in Psychonomic Bull. Rev. 5: 79–83. connectionist networks: The sequential learning problem. In The Schmajuk, N.A. and Holland, P.C., eds. 1998. Occasion-setting: Associative psychology of learning and motivation, Vol. 24 (ed. G.H. Bower), pp. learning and cognition in animals. American Psychological Association, 109–165. Academic Press, San Diego, CA. Washington, DC. Mowrer, O.H. and Jones, H.M. 1945. Habit strength as a function of the Soltysik, S.S., Wolfe, G.E., Nicholas, T., Wilson, J., and Garcia-Sanchez, pattern of reinforcement. J. Exp. Psychol. 35: 293–311. J.L. 1983. Blocking of inhibitory conditioning within a serial Myers, K.M. and Davis, M. 2002. Behavioral and neural analysis of conditioned stimulus-conditioned inhibitor compound: extinction. Neuron 36: 567–584. Maintenance of acquired behavior without an unconditioned Nakajima, S., Tanaka, S., Urshihara, K., and Imada, H. 2000. Renewal of stimulus. Learn. Motivat. 14: 1–29. extinguished lever-press responses upon return to the training Spear, N.E. 1981. Extending the domain of memory retrieval. In context. Learn. Motivat. 31: 416–431. Information processing in animals: Memory mechanisms (eds. N.E. Spear Napier, R.M., Macrae, M., and Kehoe, E.J. 1992. Rapid reacquisition in and R.R. Miller), pp. 341–378. Erlbaum, Hillsdale, NJ. conditioning of the rabbit’s nictitating membrane response. J. Exp. Spear, N.E., Smith, G.J., Bryan, R., Gordon, W., Timmons, R., and Psychol. Anim. Behav. Processes 18: 182–192. Chiszar, D. 1980. Contextual influences on the interaction between Nelson, J.B. 2002. Context specificity of excitation and inhibition in conflicting memories in the rat. Anim. Learn. Behav. 8: 273–281. ambiguous stimuli. Learn. Motivat. 33: 284–310. Staddon, J E.R. 1988. Learning as inference. In Evolution and learning Nelson, J.B. and Bouton, M.E. 1997. The effects of a context switch (eds. R.C. Bolles and M.D. Beecher), pp. 59–78. Erbaum, Hillsdale, following serial and simultaneous feature-negative discriminations. NJ. Learn. Motivat. 28: 56–84. Swartzentruber, D. 1995. Modulatory mechanisms in Pavlovian Overton, D.A. 1985. Contextual stimulus effects of drugs and internal conditioning. Anim. Learn. Behav. 23: 123–143. states. In Context and learning (eds. P.D. Balsam and A. Tomie), pp. Swartzentruber, D. and Bouton, M.E. 1992. Context sensitivity of 357–384. Erlbaum, Hillsdale, NJ. conditioned suppression following preexposure to the conditioned Pavlov, I.P. 1927. Conditioned reflexes. Oxford University Press, Oxford, stimulus. Anim. Learn. Behav. 20: 97–103. UK. Tamai, N. and Nakajima, S. 2000. Renewal of formerly conditioned fear Pearce, J.M. 1994. Similarity and discrimination: A selective review and in rats after extensive extinction training. Int. J. Comp. Psychol. a connectionist model. Psychol. Rev. 101: 587–607. 13: 137–147. Pearce, J.M. and Hall, G. 1980. A model for Pavlovian conditioning: Thomas, B.L. and Ayres J.J.B. 2004. Use of the ABA fear renewal Variations in the effectiveness of conditioned but not unconditioned paradigm to assess the effects of extinction with co-present fear stimuli. Psychol. Rev. 87: 332–352. inhibitors or excitors: Implications for theories of extintion and for Peck, C.A. and Bouton, M.E. 1990. Context and performance in treating human fears and phobias. Learn. Motivat. 35: 22–51. aversive-to-appetitive and appetitive-to-aversive transfer. Learn. Thomas, D.R., McKelvie, A.R., Ranney, M., and Moye, T.B. 1981. Motivat. 21: 1–31. Interference in pigeons’ long-term memory viewed as a retrieval Rauhut, A.S., Thomas, B.L., and Ayres, J.J.B. 2001. Treatments that problem. Anim. Learn. Behav. 9: 581–586. weaken Pavlovian conditioned fear and thwart its renewal in rats: Thomas, D.R., Moye, T.B., and Kmose, E. 1984. The recency effect in Implications for treating human phobias. J. Exp. Psychol. Anim. pigeons’ long-term memory. Anim. Learn. Behav. 12: 21–28. Behav. Processes 27: 99–114. Vigorito, M. and Ayres, J.J.B. 1987. Effect of naloxone on conditioned Reid, R.L. 1958. The role of the reinforcer as a stimulus. B. J. Psychol. suppression in rats. Behav. Neurosci. 101: 576–586. 49: 202–209. Vogel, E.H., Brandon, S.E., and Wagner, A.R. 2003. Stimulus Rescorla, R.A. 1993. Inhibitory associations between S and R in representation in SOP, II: An application to inhibition of delay. extinction. Anim. Learn. Behav. 21: 327–336. Behav. Processes 62: 27–48. . 1996. Preservation of Pavlovian associations through extinction. Wagner, A.R. 1969. Stimulus selection and a “modified continuity Q. J. Exp. Psychol. 49B: 245–258. theory.” In The psychology of learning and motivation, Vol. 3 (eds. G.H. . 1997. Response inhibition in extinction. Q. J. Exp. Psychol. Bower and J.T. Spence), pp. 1–41. Academic Press, New York. 50B: 238–252. . 1981. SOP: A model of automatic memory processing in animal . 2000. Extinction can be enhanced by a concurrent exciter. J. behavior. In Information processing in animals: Memory mechanisms Exp. Psychol. Anim. Behav. Processes 26: 251–260. (eds. N.E. Spears and R.R. Miller) pp. 5–47. Erlbaum, Hillsdale, NJ. . 2001. Experimental extinction. In Handbook of contemporary Wagner, A.R. and Brandon, S.E. 1989. Evolution of a structured learning theories (eds. R.R. Mowrer and S.B. Klein), pp. 119–154. connectionist model of Pavlovian conditioning (AESOP). In Erlbaum, Mahwah, NJ. Contemporary learning theories: Pavlovian conditioning and the status of . 2003. Protection from extinction. Learn. Behav. 31: 124–132. traditional learning theory (eds. S.B. Klein and R.R. Mowrer), pp. Rescorla, R.A. and Wagner, A.R. 1972. A theory of Pavlovian 149–189. Lawrence Erlbaum Associates, Hillsdale, NJ. conditioning: Variations in the effectiveness of reinforcement and . 2001. A componential theory of Pavlovian conditioning. In nonreinforcement. In Classical conditioning II: Current research and Handbook of contemporary learning theories (eds. R.R. Mowrer and S.B. theory (eds. A.H. Black and W.K. Prokasy), pp. 64–99. Klein), pp. 23–64. Erlbaum, Mahwah, NJ. Appleton-Century-Crofts, New York. Weidemann, G. and Kehoe, E.J. 2003. Savings in classical conditioning Rescorla, R.A. and Heth, C.D. 1975. Reinstatement of fear to an in the rabbit as a function of extended extinction. Learn. and Behav. extinguished conditioned stimulus. J. Exp. Psychol. Anim. Behav. 31: 49–68. Processes 1: 88–96. Westbrook, R.F., Jones, M.L., Bailey, G.K., and Harris, J.A. 2000. Ricker, S.T., and Bouton, M.E. 1996. Reacquisition following extinction Contextual control over conditioned responding in a latent in appetitive conditioning. Anim. Learn. Behav. 24: 423–436. inhibition paradigm. J. Exp. Psychol. Anim. Behav. Processes Robbins, S.J. 1990. Mechanisms underlying spontaneous recovery in 26: 157–173. autoshaping. J. Exp. Psychol. Anim. Behav. Processes 16: 235–249. Wilson, A., Brooks, D.C., and Bouton, M.E. 1995. The role of the rat Rosas, J.M. and Alonso, G. 1996. Temporal discrimination and hippocampal system in several effects of context in extinction. forgetting of CS duration in conditioned suppression. Learn. Motivat. Behav. Neurosci. 109: 828–836. 494 Learning & Memory www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Mark E. Bouton Learn. Mem. 2004, 11: Access the most recent version at doi:10.1101/lm.78804 This article cites 89 articles, 1 of which can be accessed free at: References http://learnmem.cshlp.org/content/11/5/485.full.html#ref-list-1 License Receive free email alerts when new articles cite this article - sign up in the box at the Email Alerting top right corner of the article or click here. Service Cold Spring Harbor Laboratory Press http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Learning & Memory Unpaywall

Context and Behavioral Processes in Extinction

Learning & MemorySep 1, 2004

Loading next page...
 
/lp/unpaywall/context-and-behavioral-processes-in-extinction-f80mItLr1b

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Unpaywall
ISSN
1072-0502
DOI
10.1101/lm.78804
Publisher site
See Article on Publisher Site

Abstract

Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Review Mark E. Bouton Department of Psychology, University of Vermont, Burlington, Vermont 05405, USA This article provides a selective review and integration of the behavioral literature on Pavlovian extinction. The first part reviews evidence that extinction does not destroy the original learning, but instead generates new learning that is especially context-dependent. The second part examines insights provided by research on several related behavioral phenomena (the interference paradigms, conditioned inhibition, and inhibition despite reinforcement). The final part examines four potential causes of extinction: the discrimination of a new reinforcement rate, generalization decrement, response inhibition, and violation of a reinforcer expectation. The data are consistent with behavioral models that emphasize the role of generalization decrement and expectation violation, but would be more so if those models were expanded to better accommodate the finding that extinction involves a context-modulated form of inhibitory learning. Extinction is a well-known and important behavioral phenom- tinction is not the same as unlearning, and because all of them enon that allows the organism to adapt its behavior to a chang- can be seen as context effects (see Bouton 1993, 2002), they also ing environment. Nonetheless, it is difficult to find a clear answer support the idea that performance after extinction is context- to the question of why repeated presentation of the conditional dependent. Extinction involves new learning, and it therefore stimulus (CS) alone after conditioning (CS–unconditional stimu- leaves the CS with two available “meanings” or associations with lus [US] pairings) causes behavior to return to essentially zero. the US. As is true for an ambiguous word, the context is crucial in One idea, surprisingly common in models of learning and selecting between them. memory (see Rescorla and Wagner 1972; McClelland and Rumel- hart 1985; McCloskey and Cohen 1989), is that extinction in- The Renewal Effect volves the destruction of what was originally learned. However, Perhaps the most fundamental of these effects is the renewal there is ample evidence that this is not true; much of the original effect (see Bouton and Bolles 1979a; Bouton and King 1983). In learning survives extinction (see Rescorla 2001; Bouton 2002; this phenomenon, a change of context after extinction can cause Myers and Davis 2002; Delamater 2004). In this article, I will a robust return of conditioned responding. Several versions of selectively review results and theory from the behavioral litera- the renewal effect have been studied. In the most common one, ture in an effort to understand what is learned in extinction, and “ABA renewal,” conditioning is conducted in one context (con- what causes the organism to learn it. text A) and extinction is then conducted in a second one (con- The first part of the article introduces several extinction phe- text B). (The contexts are typically separate and counterbalanced nomena that any adequate theory of extinction will need to ex- apparatuses housed in different rooms of the laboratory that dif- plain and accommodate. They suggest that extinction does not fer in their tactile, olfactory, and visual respects.) When the CS is destroy the original learning but instead involves new learning returned to the original conditioning context (context A), re- that is at least partly modulated by the context. In the second sponding to the CS returns (see Bouton and Bolles 1979a; Bouton part of the article, I will discuss a set of behavioral phenomena and King 1983; Bouton and Peck 1989). In a second version, that are theoretically linked to extinction, and ask what they “ABC renewal,” conditioning is conducted in context A, extinc- further reveal about it. Extinction is just one example of a retro- tion is conducted in context B, and then testing is conducted in active inhibition phenomenon in which new learning inhibits a third, “neutral” context—context C. Here again, a renewal of old, and the principles that explain it may have wide applicabil- responding is observed (see Bouton and Bolles 1979a; Bouton ity. In the final part of the article, I will consider the question “If and Brooks 1993; Harris et al. 2000). In a final version, condi- extinction is an example of new learning, what events ‘reinforce’ tioning and extinction are both conducted in the same context or cause it?” Several possibilities will be considered. In the end, (context A) and then the CS is tested in a second context (context the results may favor the view that extinction occurs because the B). Here again, conditioned responding returns (see Bouton and omission of the US (1) causes generalization decrement and (2) Ricker 1994; Tamai and Nakajima 2000), although there is cur- violates the organism’s expectation of the US and therefore ini- rently less evidence of this “AAB renewal” effect in operant con- tiates new learning. These ideas have a long history in behavioral ditioning than in Pavlovian conditioning (see Nakajima et al. theories of extinction. Their fuller integration with a contextual 2000; Crombag and Shaham 2002). analysis (part 1) may provide a more comprehensive behavioral Several facts about the renewal effect are worth noting. First, account of extinction. it has been observed in virtually every conditioning preparation in which it has been investigated (for a review, see Bouton 2002). Extinction Learning Is Especially Context-Dependent Second, it can occur after very extensive extinction training. In For many years, my colleagues and I have studied a number of fear conditioning (conditioned suppression) in rats, Bouton and experimental manipulations that can be conducted after extinc- Swartzentruber (1989) observed it when 84 extinction trials fol- tion has taken place. In each of them, the extinguished response lowed eight conditioning trials. Other evidence suggests that it returns to performance. All of them therefore indicate that ex- can occur after as many as 160 extinction trials (Gunther et al. 1998; Rauhut et al. 2001; Denniston et al. 2003), although a E-MAIL Mark.Bouton@uvm.edu; FAX (802) 656-8783. recent report suggests that it might not survive an especially Article and publication are at http://www.learnmem.org/cgi/doi/10.1101/ lm.78804. “massive” extinction treatment (800 extinction trials after eight 11:485–494 ©2004 by Cold Spring Harbor Laboratory Press ISSN 1072-0502/04; www.learnmem.org Learning & Memory 485 www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Bouton conditioning trials; Denniston et al., 2003). Third, the role of the 1993; Devenport et al. 1997), and it seems likely to be multiply context is different from the one anticipated by standard models determined. However, we have argued (see Bouton 1988, 1991, of classical conditioning (see Rescorla and Wagner 1972; Pearce 1993) that just as extinction is relatively specific to its physical and Hall 1980; Wagner 1981; Wagner and Brandon 1989, 2001). context, so it may be specific to its “temporal context.” Sponta- Those models accept the view that the context is merely another neous recovery can be seen as the renewal effect that occurs CS that is presented in compound with the target CS during when the CS is tested outside its temporal context. Both are due reinforcement or nonreinforcement. It therefore enters into to a failure to retrieve memories of extinction outside the extinc- simple excitatory or inhibitory associations with the US. In the tion context. Consistent with this perspective, a cue that is pre- ABA renewal effect (for example), context A might acquire excit- sented intermittently during the extinction session can attenuate atory associations with the US, and context B might acquire in- either spontaneous recovery or renewal if it is presented just be- hibitory associations. Either kind of association would summate fore the final test (Brooks and Bouton 1993, 1994; Brooks 2000). with the CS to produce the renewal effect (inhibition in B would The parallel results suggest that the two effects might be con- reduce responding to the CS, whereas excitation in A would en- trolled by a common mechanism: a failure to retrieve a memory hance it). However, a number of experiments have shown that of extinction outside the extinction context. Interestingly, the renewal effect can occur in the absence of demonstrable ex- changing the physical context and temporal context together citation in context A or inhibition in context B (see Bouton and can have a bigger effect than changing either context alone, as if King 1983; Bouton and Swartzentruber 1986, 1989). These find- their combination creates an even larger context change (Rosas ings, coupled with others showing that strong excitation in a and Bouton 1997, 1998). context does not influence performance to a CS unless the CS is Rapid Reacquisition under the influence of extinction (described below; Bouton 1984; Bouton and King 1986), suggest that direct associations in a con- A third effect further indicates that conditioning is not destroyed text are neither necessary nor sufficient for a context to influence in extinction. In rapid reacquisition, when new CS–US pairings responding to a CS. The implication (see Bouton and Swartzen- are introduced after extinction, the reacquisition of responding truber 1986; Bouton 1991) is that the contexts modulate or “set can be more rapid than is acquisition with a novel CS, indicating the occasion” for the current CS–US or CS–no US association (see that the original learning has been “saved” through extinction Holland 1992; Swartzentruber 1995; Schmajuk and Holland (see Napier et al. 1992; Ricker and Bouton 1996; Weidemann and 1998). Put another way, they activate or retrieve the current re- Kehoe 2003). Unfortunately, the early literature on rapid reac- lation of CS with the US. quisition was often difficult to interpret because many early de- A further important characteristic of the renewal effect is signs were not equipped to rule out less interesting explanations that it implies that extinction learning is more context-specific (for a review, see Bouton 1986). To add to the complexity, studies than is original conditioning. Notice that this must be true if one of fear conditioning (conditioned suppression; Bouton 1986; observes ABC and AAB renewal; in either case, conditioning Bouton and Swartzentruber 1989) and flavor aversion learning transfers better to the final test context than extinction. But our (Danguir and Nicolaidis 1977; Hart et al 1995), have shown that experiments on renewal have often involved comparisons of reacquisition can be slower than acquisition with a new CS. (It is groups that received extinction training in the context in which more rapid than initial acquisition with a CS that has received conditioning had occurred or in a discriminably different con- the same number of nonreinforced trials without conditioning text. Strikingly, there was no measurable effect of switching the [Bouton and Swartzentruber 1989].) In fear conditioning, slow context after conditioning on responding to the CS (see Bouton reacquisition requires extensive extinction training; more lim- and King 1983; Bouton and Peck 1989). In contrast, extinction ited extinction training yields reacquisition that is neither fast itself was relatively context-specific, as the renewal effect itself nor slow (Bouton 1986). At least part of the reason these prepa- suggests. Recent research suggests that both conditioning and rations support slow reacquisition is that both typically involve extinction become somewhat context-specific after extinction very few initial conditioning trials. In contrast, procedures in has occurred (Harris et al. 2000). But there is little question that which rapid reacquisition has been shown (rabbit nictitating extinction is still more context-dependent than is the original membrane response (NMR) conditioning and rat appetitive con- conditioning. We have therefore emphasized the fact that extinc- ditioning) have usually involved a relatively large number of ini- tion learning is especially context-dependent. tial conditioning trials. Consistent with a role for number of A final fact about the renewal effect is that it appears to be trials, Ricker and Bouton (1996) demonstrated that slow reacqui- supported by many kinds of contexts. For example, when fear sition occurred in an appetitive conditioning preparation when extinction was conducted in the interoceptive context provided the procedure used the number of conditioning and extinction by benzodiazepine tranquilizers chlordiazepoxide and diazepam, trials that had been used in previous fear conditioning experi- renewed fear was observed when the rat was tested in the original ments. In rabbit NMR and heart rate conditioning, extensive ex- nondrug state (Bouton et al. 1990). Cunningham (1979) had re- tinction training has abolished rapid reacquisition, although ported compatible evidence with alcohol, and we have recently slow reacquisition has yet to be observed (Weidemann and Ke- collected similar observations with the benzodiazepine mid- hoe 2003). azolam (L. Pain, P. Oberling, and M.E. Bouton; unpubl.). State- Ricker and Bouton (1996) suggested that rapid reacquisition dependent learning or retention can be conceptualized as the may partly be an ABA renewal effect that occurs when the animal drug playing the role of context (see Overton 1985). has learned that previous USs or conditioning trials are part of the original “context” of conditioning. That is, the animal might Spontaneous Recovery learn that recent CS–US pairings are part of the context of con- The passage of time might also bring about changes in internal ditioning, whereas recent CS-only presentations are part of the and external stimulation that provide a gradually-changing con- context of extinction. When CS–US pairings are resumed after text. Pavlov (1927) first observed another well-known extinction extinction, they would thus return the animal to the original effect. In spontaneous recovery, if time is allowed to pass follow- conditioning context. The hypothesis is compatible with Capal- ing extinction, the extinguished response can recover. There are di’s (1967, 1994) sequential analysis of extinction, which has several available explanations of spontaneous recovery (for a dis- made excellent use of the idea that responding on a particular cussion of alternatives, see Robbins 1990; Brooks and Bouton trial is determined by how the animal has learned to respond in 486 Learning & Memory www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Context, Inhibition, and Extinction the presence of similar memories of previous trials (see below). may cause a return of responding after extinction because of Presumably, conditioning preparations that use a relatively large another ABA renewal effect (Bouton et al. 1993). number of conditioning trials allow ample opportunity for the In summary, a variety of research indicates that responding animal to learn that previous reinforced trials are part of the to an extinguished CS is susceptible to any of a number of recov- context of conditioning. Ricker and Bouton (1996) also reported ery effects, suggesting that extinction is not unlearning. Indeed, evidence that high responding during the reacquisition phase based on the results of a number of tests that allow a specific was more likely after a reinforced than a nonreinforced trial, which comparison of the strength of the CS–US association before and presumably signaled conditioning and extinction, respectively. after extinction (see Delamater 1996; Rescorla 1996), Rescorla In more recent experiments, Bouton et al. (2004) reasoned (2001) has suggested that extinction involves no unlearning that if rapid reacquisition is caused by recent reinforced trials whatsoever; the original CS–US association seems to survive es- generating ABA renewal, then an extinction procedure that in- sentially intact. Extinction must thus depend on other mecha- cludes occasional reinforced trials among many nonreinforced nisms. The renewal effect, and the fact that extinction leaves the trials should slow down rapid reacquisition by making recent CS so especially sensitive to manipulations of context, is consis- reinforced trials part of the context of both conditioning and tent with the idea that extinction involves new learning that is extinction. Consistent with this hypothesis, a very sparse partial especially context-dependent. We have therefore suggested that reinforcement procedure in extinction slowed reacquisition in a extinction leaves the CS under a contextually modulated form of final phase compared with a group that had received simple ex- inhibition (see Bouton 1993): The presence of the extinction con- tinction. Such a result is consistent with the idea that rapid re- text retrieves or sets the occasion for a CS–no US association. acquisition is at least partly an ABA renewal effect. Because the partial reinforcement treatment involved many more CS–US Other Phenomena With Theoretical Links to Extinction pairings than did simple extinction, it is difficult to reconcile Several behavioral phenomena have been linked theoretically with the view that rapid reacquisition is a simple function of the with extinction, and it is worth considering them to see what strength of an association that remains after extinction (see Ke- insights they provide. hoe 1988; Kehoe and Macrae 1997). Counterconditioning and Other Interference Paradigms Reinstatement In counterconditioning, a CS that has been associated with one A fourth context-dependent extinction phenomenon is rein- US is associated with a second US, often incompatible with the statement. In this effect, the extinguished response returns after first, in a second phase. Not surprisingly, performance corre- extinction if the animal is merely reexposed to the US alone (see sponding to the second association replaces performance corre- Pavlov 1927; Rescorla and Heth 1975; Bouton and Bolles 1979b). sponding to the first. Counterconditioning is thus a paradigm If testing of the CS is contemporaneous with US delivery, then that, similar to extinction, involves a form of retroactive inter- the USs may cause a return of responding because they were ference. encoded as part of the conditioning context (as above; see Reid The literature on counterconditioning is not as large as the 1958; Baker et al. 1991; Bouton et al. 1993). On the other hand, literature on extinction. But there is evidence that similar prin- in many studies of reinstatement, testing is conducted at an in- ciples may apply. For example, experiments in my own labora- terval of at least 24 h after US re-exposure; here one still observes tory have demonstrated a renewal effect (Peck and Bouton 1990): reinstatement compared with controls that were not re-exposed If rats receive CS–shock pairings in one context and then CS– to the US (see Rescorla and Heth 1975; Bouton and Bolles 1979b). food pairings in another, the original fear performance returns In this case, evidence strongly suggests that the effect is due to (and replaces food performance) when the animals are returned conditioning of the context. When the US is presented after ex- to the original context. Complementary results were obtained tinction, the organism associates it with the context; this con- when CS–food preceded CS–shock. Other experiments have dem- textual conditioning then creates reinstatement. For example, if onstrated spontaneous recovery (Bouton and Peck 1992): In this the reinstating USs are presented in an irrelevant context, there is case, after CS–shock and then CS–food, animals tested at a 1-d no reinstatement when the CS is tested again (see Bouton and retention interval showed primarily appetitive performance, Bolles 1979b; Bouton and King 1983; Bouton 1984; Baker et al. whereas animals tested 28 d later showed a recovery of fear per- 1991; Wilson et al. 1995; Frohardt et al. 2000). Independent mea- formance (and a suppression of appetitive). A complementary sures of contextual conditioning also correlate with the strength pattern was observed when CS–food preceded CS–shock. Finally, of reinstatement (Bouton and King 1983; Bouton 1984). And if we have observed reinstatement (Brooks et al. 1995): When CS– the animal receives extensive extinction exposure to the context food follows CS–shock, a number of noncontingent shocks de- after the reinstatement shocks are presented, reinstatement is not livered in the same context (but not in a different context) can observed (Bouton and Bolles 1979b; Baker et al. 1991). These reinstate the original fear performance. Counterconditioning results indicate that mere re-exposure to the US is not sufficient thus supports at least three of the effects suggesting that extinc- to generate reinstatement. It is necessary to test the CS in the tion involves context-dependent new learning. context in which the US has been re-exposed. Bouton (1993) reviewed the behavioral literature on a num- This effect of context conditioning is especially potent with ber of “interference paradigms” in conditioning. In these para- an extinguished CS. For example, Bouton (1984) compared the digms, a CS is associated with different outcomes in successive effects of US exposure in the same or a different context on fear phases of the experiment. Importantly, performance in all such of a partially extinguished CS or another CS that had reached the paradigms is sensitive to manipulations of context and time. For same low level of fear through simple CS–US pairings (and no example, in addition to counterconditioning, the list includes extinction). Although contextual conditioning enhanced fear of discrimination reversal learning, in which two CSs (X and Y) are the extinguished CS, it had no impact on the nonextinguished reinforced and nonreinforced (e.g., X+/Y) before the relation- CS (see also Bouton and King 1986). This result is consistent with ship is reversed in a second phase (X/Y+). The list also includes the effects of context switches mentioned above: An extin- latent inhibition, in which a single CS is nonreinforced on a guished CS is especially sensitive to manipulations of the con- number of trials before it is paired with the US in a second phase. text. One reason is that contextual conditioning may be another In both paradigms, if the first and second phase are conducted in feature of the conditioning context; its presence during a test separate contexts, a return to the phase 1 context can cause a Learning & Memory 487 www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Bouton renewal of phase 1 performance (discrimination reversal: Spear et and Y. Because Y has no other association, it will become a pure al. 1980; Thomas et al. 1981; Bouton and Brooks 1993; latent conditioned inhibitor. For X, however, the inhibition is overlaid inhibition: Bouton and Swartzentruber,1989; Maren and Holt on its existing (and unchanged) excitatory association. The same 2000; Westbrook et al. 2000). Effects of time related to sponta- thing occurs in simple extinction, because the CS also activates neous recovery have also been observed (discrimination reversal: the US into A2 at that time. Because these processes occur in real Gordon and Spear 1973; Thomas et al. 1984; Bouton and Brooks time, during any nonreinforced trial, inhibition will accrue to the 1993; latent inhibition: Kraemer et al. 1991; Aguado et al. 1994; CS from the point in time at which the US node is first activated cf. De la Casa and Lubow 2000, 2002; Lubow and De la Casa to A2 until the CS leaves the A1 state, which may not occur until 2002). Bouton (1993) argued that a retrieval account that accepts the CS is turned off at the end of the trial (for further discussion, that both phases are learned and available, and that performance see the section Violation of Reinforcer Expectation below). is therefore determined by which is retrieved, can go some dis- Because of these connections between inhibition and ex- tance in explaining all examples of interference (see also Spear tinction, and the evidence that extinction is more context- 1981). At a broad level of analysis, then, extinction is just one specific than is simple conditioning, Bouton and Nelson (1994) example from a set of interference phenomena that all depend and Nelson and Bouton (1997) asked whether pure inhibition on context and time. acquired in the feature-negative paradigm was also context- specific. The designs of the experiments are sketched in Table 1. Conditioned Inhibition Rats were given different feature-negative discriminations in a Several theories of conditioning specifically attribute extinction series of intermixed sessions in two contexts (A and B), as shown to a build-up of inhibition (see Konorski 1948, 1967; Pearce and at left. The design in the upper half of Table 1 asks whether Hall 1980; Wagner 1981; Pearce 1994; see also Pavlov 1927). Al- inhibition conditioned to the feature in context A (CS Y) trans- though the Rescorla-Wagner model (1972) attributed extinction fers to the other context (context B) in the final test; an inhibitor to unlearning rather than inhibitory learning, it provided a con- (Y) was tested in its original context and/or in another one. Sur- ceptualization of inhibition that has been extremely influential. prisingly, its inhibition transferred without measurable disrup- Specifically, inhibition was viewed as a negative form of learning tion to the new context. However, we also performed experi- that develops when the summed strengths of all the CSs present ments along the lines shown in the lower portion of Table 1. on a conditioning trial “overpredict” the magnitude of the US These asked whether any inhibition acquired by stimulus X, that actually occurs on the trial. Most models now use a version rather than the pure inhibitory Y, is lost with a context switch. of its error-correction learning mechanism, which essentially Here there was clear evidence of a context effect: When switched brings the overall expectation of the US (the summed values of to the alternate context, responding to X became more difficult the CSs present) into line with reality (the value of the US that is to inhibit. Stimulus X, with its mixed history of both reinforce- actually present); this means decrementing associative strengths ment and nonreinforcement, is similar to an extinguished CS: Its until what is predicted equals what actually occurs. inhibitory association is context-specific. But Y’s pure inhibition According to this view, the main method for generating is not. pure conditioned inhibition is the conditioned inhibition or fea- Because inhibition was not generally context-specific, the ture-negative paradigm, in which the experimenter intermixes results implied that extinction is not context-specific merely be- trials on which one CS (X) is paired with the US and other trials cause it is a form of inhibition. A second reason why extinction on which X is combined with another CS (Y) and presented with- might be context-specific is that it is the second thing the organ- out a US (X+, XY). (X is sometimes called the “target” stimulus ism has learned about the CS. Notice that the same is true in and Y the “feature” stimulus.) After early X+ trials, X begins to other interference paradigms, such as counterconditioning: It is predict that the US will occur; when no US occurs on XY trials, the second-learned association that seems dependent on the con- the error-correction mechanism therefore decrements both CSs text and on time. And it is presumably also true of the target to bring their strengths in line with no US. In the Rescorla- stimulus (X) in the feature-negative design; there must be exci- Wagner model, X will lose some excitatory strength on the nega- tation to X before the XY trials can generate inhibition. Nelson tive trials, but it will not go below zero. But because Y starts with (2002) confirmed the second-association hypothesis in a series of zero value, its associative strength becomes negative and thus experiments. As usual, excitatory conditioning (tone-food con- becomes a pure conditioned inhibitor. ditioning) transferred undisturbed across contexts, unless the Other models actually predict inhibitory learning to both Y tone had first been trained as a conditioned inhibitor (stimulus Y and X in this paradigm (see Pearce and Hall 1980; Wagner 1981; in the feature-negative paradigm). Conversely, inhibition to a Wagner and Brandon 1989, 2001). For example, Wagner (1981) conditioned inhibitor also transferred across contexts unless the and Wagner and Brandon (1989, 2001) have presented a com- prehensive real-time model that expands enormously on the Res- corla-Wagner model and is worth describing here. According to Table 1. Designs Used by Bouton and Nelson (1994) and the model, known as SOP (for Sometimes-Opponent-Process), CS Nelson and Bouton (1997) and US are represented as memory nodes that can become asso- ciated during conditioning. For the association between them to Training Testing be strengthened, both nodes must be activated from inactivity to Is inhibition of the feature (Y) context-specific? an active state, A1, at the same time. Once the association has A: X+. XY A: X, XY been formed, the presentation of the CS activates the US node to B: X+, XZ B: X, XY a secondarily-active state, A2. This in turn generates the condi- Is inhibition of the target (X) context-specific? tioned response. An inhibitory connection is formed between a A: X+, XY A: X, XY CS and a US when the CS is activated to the A1 state and the US B: Z+, ZY B: X, XY is activated to A2 rather than A1. These conditions are met in the A and B are contexts; X, Y, and Z are CSs. + indicates reinforced; , feature-negative paradigm (X+, XY). After some initial condi- nonreinforced. During training, all rats received intermixed sessions in tioning, X is able to activate the US node to A2. On XY trials, both contexts. Testing was then conducted in both contexts with the US node is therefore put into the A2 state at the same time either between-subject or within-subject methods. that X and Y are in A1. This will create inhibition to both X 488 Learning & Memory www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Context, Inhibition, and Extinction CS had first been trained as a conditioned excitor (through initial CS in most experiments), and therefore undergoes extinction to tone–food pairings). Thus, regardless of whether the association early parts of the CS (see Rosas and Alonso 1996). Interestingly, was excitatory or inhibitory, the second thing learned was more Rosas and Alonso (1997) have shown that inhibition of delay, context-specific than the first. Compatible data had been shown developing over trials this way, is in fact attenuated with a con- by Swartzentruber and Bouton (1992), who found that excitatory text switch. However, in my laboratory the nonmonotonicity is conditioning was relatively context-specific if it had been pre- still observed when shock occurs at unpredictable times in the ceded by nonreinforced pre-exposure to the CS. CS, and there is again an increase in fear when the context is Thus, the evidence suggests that the learning and memory changed under these conditions. The result is also not due to an system treats the first association as context-free, but the second opioid process that helps the animal adapt to the US (see Vigorito association as a kind of context-specific exception to the rule. and Ayres 1987); indeed, because our context switch experiments There may be functional reasons for this (Bouton 1994). A con- test the rat’s reaction to the CS, not the US, the results indicate ditioning trial provides a sample from which an animal may some sort of adaptation to the CS or to conditioned fear itself. make inferences about the state of the world (see Staddon 1988). Another possibility is suggested by SOP (see Wagner 1981): Re- Statistically, if the world is composed of two types of trials (CS– peated exposure to the CS would allow the animal to associate US and CS–no US), then the probability of sampling a particular the CS with the context, which would permit the context to type of trial will reflect its true prevalence in the world. There- associatively activate the CS into the secondarily active state fore, an early run of conditioning trials would reflect its high (A2). Putting the CS into that state might reduce its ability to incidence in the population; a subsequent trial of another type evoke a response (see Hall and Honey 1990; Honey et al. 1993; might reflect an exception to the rule. Learning and memory Hall and Mondragón 1998). But perhaps contrary to this view, may thus be designed to treat second-learned information as a there is little correlation between the number of CSs presented in conditional and context-specific. A mechanistic account has not the conditioning procedure and the degree of nonmonotonicity. been fully tested. One possibility is that when extinction begins, In addition, we have yet to observe nonmonotonic learning the omission of the US is surprising and boosts attention to the curves in appetitive conditioning, in which the animal also re- CS and context, as suggested by the attention rule of the Pearce- ceives many opportunities to associate the context and CS. Al- Hall model (Pearce and Hall 1980; see also Kaye and Pearce 1984). though we do not yet understand the inhibition-despite- Interestingly, there is evidence that conditioned excitation may reinforcement phenomenon, there is a suggestion that, similar to be context specific after one conditioning trial—when the US is extinction, it involves a context-specific inhibitory process that also new and surprising (Hall and Honey 1990). might be the second thing learned about the CS. Inhibition Despite Continued Reinforcement What Causes Extinction? Other research has identified another inhibitory effect that de- If we return specifically to extinction, another question is what serves mention alongside extinction. Ayres et al. (1979; see also event or behavioral process actually causes the loss of respond- Vigorito and Ayres 1987) have shown in the conditioned sup- ing? Several ideas have been examined in recent experiments. pression preparation that repeated conditioning trials may yield nonmonotonic learning curves in which conditioned fear Discrimination of Reinforcement Rate reaches a peak and then declines despite continued pairings of One possibility is that the animal eventually learns that the rate the CS and shock. The process is poorly understood, although of reinforcement in the CS is lower in extinction than it was similar effects have been observed in several conditioning prepa- during conditioning. Gallistel and Gibbon (2000) have argued rations (Kimmel and Burns 1975). Pavlov himself saw it often that the animal continually decides whether or not to respond in (1927), and attributed it to inhibition developing despite rein- extinction by comparing the current rate of reinforcement in the forcement. CS with its memory of the rate that prevailed in conditioning. Experiments in my laboratory have further documented the Because rate is the reciprocal of time, the animal computes a ratio effect in the conditioned suppression preparation (M.E. Bouton, between the amount of time accumulated in the CS during ex- R.J. Frohardt, C. Sunsay, and J. Waddell, in prep.). We have also tinction and the amount of time accumulated in the CS between discovered a role for context. Rats received intermixed sessions in USs during conditioning. When the ratio exceeds a threshold, two contexts; in one context, one CS was repeatedly paired with the animal stops responding. footshock, and in the other, a second CS was similarly paired This approach has been tested in several recent experiments. with the same shock. Nonmonotonic learning curves developed. Haselgrove and Pearce (2003) examined the impact of varying However, when we then tested the CS in the alternate context, the duration of the CS during extinction; when longer CSs are we saw a significant increase in fear of the CS. Over experiments, used in extinction, time in the CS accumulates more quickly, and there was a strong correlation between the size of this increase the animal should stop responding after fewer trials. In some and the degree of nonmonotonicity shown in the conditioning experiments, rats were given appetitive conditioning with a 10- curve. The context switch thus attenuated the inhibitory process sec CS and then given extinction exposures to a series of 10-sec or that led to the nonmonotonic learning curve. 270-sec presentations of the CS. When responding was examined We do not have a good understanding of this inhibitory at the start of each CS, there was an occasionally significant, but process at the present point in time. In our experiments, the surprisingly small, effect of increasing the duration of the CS decline in fear over training is not a result of inhibition of delay, during extinction. For instance, by the 12th two-trial block, the in which the animal learns the timing of the US (at the end of the 10-sec and 270-sec CS groups had similar nonzero levels of re- sponding, even though they had accumulated a total of 4 and 108 min of exposure in the CS, respectively. On the other hand, The main exception to the second-association rule is latent inhibition, in responding did decline as a function of time within a single pre- which the first phase can be shown to exert a context-dependent influence on the second phase (see Hall and Channell 1985) despite the fact that it is sentation of the 270-sec CS, perhaps reflecting generalization arguably the first thing learned. Latent inhibition is unique, however, in that decrement resulting from the increasing difference between the the CS is not paired with anything significant in the first phase. One possibility, current CS and the 10-sec CS employed in conditioning. Consis- therefore, is that it is in part encoded as a feature of the context, making it tent with that view, when conditioning first occurred with a difficult to extract it from that context when it is paired with the US in phase 2 (cf. Gluck and Myers 1993). 60-sec CS, extinction of responding occurred more rapidly with a Learning & Memory 489 www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Bouton 10-s CS than with a 60-sec CS. Thus, either an increase or a quisition as an ABA renewal effect (Ricker and Bouton 1996; Bou- decrease in the duration of the CS relative to conditioning accel- ton et al. 2004). Interestingly, the recent finding that occasional erated the loss of responding. This effect of time was not antici- reinforced trials in extinction (partial reinforcement) can slow pated by the rate-discrimination view (Gallistel and Gibbon down the rate of reacquisition (Bouton et al. 2004) is really just 2000). the inverse of the PRE: In the PRE, nonreinforced trials in con- Drew et al. (2004) reported compatible results in experi- ditioning allow more generalization from conditioning to extinc- ments on autoshaping in ring doves. Doubling or halving the tion, whereas Bouton et al.’s finding suggests that reinforced tri- duration of the CS from the 8-sec value used in conditioning did als in extinction allowed for more generalization of extinction to not affect the number of trials required to stop responding. The reconditioning. Either finding suggests the importance of con- fact that extinction was thus largely controlled by the number of sidering recent trials as part of the context that controls perfor- CS presentations is consistent with experiments that have exam- mance in extinction. ined the effects of the number and duration of nonreinforced In summary, there is little support for the idea that respond- trials added to conditioning schedules (Bouton and Sunsay ing extinguishes when the US is omitted because the organism 2003). On the other hand, Drew et al. (2004) found that a more detects a lower rate of reinforcement in the CS. The number of extreme increase in CS duration (from 8 to 32 sec) increased the extinction trials, rather than merely the accumulating time in rate of extinction. This was attributed to the animal learning to the CS across trials, appears to be important to the extinction discriminate the longer nonreinforced CS presentations from the process. Time in the CS can have an effect: It appears to be an- shorter reinforced CS presentations: When 8-sec CSs were pre- other dimension over which animals generalize and discriminate sented again after extinction, birds extinguished with 4-sec and (Haselgrove and Pearce 2003; Drew et al., 2004). But explanation 32-sec CSs responded again. Animals are sensitive to time in the of the PRE appears to be most consistent with a view that animals CS, but the number of extinction trials appears to be an impor- use their memories of the outcomes of preceding trials as a di- tant factor. mension over which they generalize and respond (for a more As noted by Gallistel and Gibbon (2000), the rate discrimi- extended review, see also Mackintosh 1974). nation theory seems especially consistent with a well-known ex- Generalization Decrement tinction phenomenon, the partial reinforcement effect (PRE; for a review, see Mackintosh, 1974). In this phenomenon, condi- It is thus possible to claim that the animal stops responding in tioning with partial reinforcement schedules (in which nonrein- extinction at the point at which it stops generalizing between the forced trials are intermixed with reinforced trials) creates a slower stimuli that prevailed in conditioning than those that prevail in loss of responding in extinction than does conditioning with a extinction (see Capaldi 1967, 1994). This idea has had a long and continuous reinforcement schedule (in which every trial is rein- influential history in research on extinction, especially in re- forced). According to a rate-discrimination hypothesis (Gallistel search on the PRE. It is interesting to note that a generalization and Gibbon 2000), the partially reinforced subjects have learned decrement theory of extinction does not imply destruction of the to expect the US after more accumulated time in the CS, and it original learning in extinction, or indeed any new learning at all. thus takes more CS time in extinction to exceed the threshold of However, there is still good reason to think that extinction also accumulated extinction time/expected time to each US. The involves new learning. For instance, nonreinforcement of a food more traditional approach, in contrast, has been to think that CS elicits measurable frustration, and this can be associated with partially reinforced subjects have learned to expect the US after stimuli present in the environment (Daly 1974). Nonreinforce- more trials than continuously reinforced subjects have. It there- ment of the CS in a feature-negative paradigm also generates fore takes more trials to stop generalizing from conditioning to measurable new learning in the form of conditioned inhibition extinction (see Mowrer and Jones 1945; Capaldi 1967, 1994). (see above). And there is also evidence for new learning in the Contrary to the rate discrimination hypothesis, Haselgrove renewal effect. For example, either ABC renewal or AAB renewal et al. (2004) and M.E. Bouton and A.M. Woods (in prep.) have (see above) imply that context B (the extinction context) ac- shown that a PRE still occurs when partially and continuously quires an ability to modulate (suppress) performance to the CS. reinforced subjects expect the reinforcer after the same amount Such observations suggest that the animal has not merely of CS time. For example, both sets of investigators showed that a stopped responding in extinction because of a failure to general- group that received a 10-sec CS reinforced on half its presenta- ize. Instead, it appears to have learned that the CS means no US tions (accumulated CS time of 20 sec) extinguished more slowly in the extinction context (see above). than did a continuously reinforced group that received every Role of Inhibition of the Response 20-sec CS presentation reinforced. M.E. Bouton and A.M. Woods (in prep.) further distinguished the “time-discrimination” ac- Rescorla (2001) has recently suggested that extinction might in- count from the traditional “trial-discrimination” account (see volve learning to inhibit the conditioned response. For example, Mowrer and Jones 1945; Capaldi 1967, 1994). Rats that had every he has summarized evidence from instrumental (operant) condi- fourth 10-sec CS reinforced extinguished more slowly over a se- tioning experiments indicating that the effects of extinction can ries of alternating 10-sec and 30-sec extinction trials than rats be specific to the response that undergoes extinction. For ex- that had received every 10-sec CS reinforced. This PRE was still ample, Rescorla (1993) reinforced two operant behaviors (lever observed when extinction responding was plotted as a function pressing and chain pulling) with food pellets and then extin- of time units over which the US should have been expected (ev- guished each response in combination with a new stimulus (a ery 40 sec for the PRF group but every 10 sec for the CRF group). light or a noise). Subsequent tests of the two responses with both In contrast, the PRE disappeared when extinction responding light and noise indicated that each response was more depressed was plotted as a function of the trials over which the US should when it was tested in combination with the cue in which it had have been expected (every fourth trial for the PRF group and been extinguished (see also Rescorla 1997). There is thus good every trial for the CRF group). Ultimately, the PRE is better cap- reason to think that the animal learns something specific about tured by trial-based theories (e.g., Capaldi 1967, 1994). the response itself during operant extinction: It learns not to We have already seen that responding on a particular trial perform a particular response in a particular stimulus. One pos- occurs in the context of memories of the outcomes of previous sibility is that the animal learns a simple inhibitory S-R associa- trials—that was the explanation provided earlier of rapid reac- tion (Colwill 1991). Another possibility, perhaps more consistent 490 Learning & Memory www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Context, Inhibition, and Extinction with the context-modulation account of extinction emphasized and M.E. Bouton, in prep.) suggest a similar conclusion even above, is that the animal learns that S sets the occasion for a though the results were different. Spaced extinction trials again response–no reinforcer relationship. Rescorla (1993, p. 335; 1997, yielded more responding in extinction than did massed trials, p. 249) has observed that the experiments do not separate the but the treatments caused indistinguishable amounts of extinc- two possibilities. To my knowledge, no analogous experiments tion learning as assessed in spontaneous recovery and reinstate- have been performed in the Pavlovian conditioning situation. ment tests. In related conditioned suppression experiments, M.E. Instead, the main implication examined in Pavlovian con- Bouton, A. García-Gutiérrez, J. Zilski, and E.W. Moody (in prep.) ditioning is that extinction procedures should be especially suc- compared the effects of extinction in multiple contexts on the cessful at causing inhibitory S-R learning if they generate high strength of the ABA and ABC renewal effects. Rats received fear levels of responding in extinction. This prediction may provide a conditioning with a tone CS in context A, and then extinction of reasonable rule of thumb (Rescorla 2001). For example, when a the tone for three sessions in context B, or a session in B, then C, CS is compounded with another excitatory CS and the com- and then D, before final renewal tests in the original context pound is extinguished, there is especially strong responding in (context A) or a neutral fifth context (context E). Although the extinction (due to summation between the CSs), and especially successive context switches in the BCD group caused more fear effective extinction as evidenced when the CS is tested alone responding during extinction (due to renewal effects), the groups (Wagner 1969; Rescorla 2000; Thomas and Ayres 2004). Con- showed strikingly similar renewal in either context A or context versely, when the target CS is compounded with an inhibitory E. Gunther et al. (1998) and Chelonis et al. (1999) have shown CS, there is relatively little responding to the compound (excita- more favorable effects of extinction in multiple contexts on re- tion and inhibition negatively summate), and there is also less newal. But higher responding in extinction does not guarantee evidence of extinction when the target is tested alone (Soltysik et better extinction learning. The results seem inconsistent with a al. 1983; Rescorla 2003; Thomas and Ayres 2004). However, al- response-inhibition hypothesis. Their impact on the expectancy though these findings are consistent with the hypothesis that the violation hypothesis is perhaps less clear. effectiveness of extinction correlates with the degree of respond- In summary, although animals that receive extinction after ing, they can also be interpreted in a different way. Either treat- operant conditioning may in fact learn to refrain from perform- ment also affects the degree to which the animal’s expectation of ing a particular response in a particular context (see Rescorla the reinforcer is violated: The stimulus compound influences the 1993, 1997), the importance of response inhibition in Pavlovian size of the error term in the Rescorla-Wagner model and, in more extinction is not unequivocally supported at the present time. cognitive terms, the extent to which the expectation of the US High responding in extinction does not guarantee more effective created by the compound is violated when the US does not occur. extinction learning. To date, the hypothesis has been difficult to The results do not separate the response-inhibition hypothesis distinguish from the violation-of-expectation hypothesis that is from an expectancy-violation hypothesis, which will be covered built into many mainstream models of classical conditioning (see in the next section. Rescorla and Wagner 1972; Pearce and Hall 1980; Wagner 1981; A recent eyeblink experiment by Krupa and Thompson Wagner and Brandon 1989, 2001) and is considered next. (2003) manipulated the level of responding another way. During Violation of Reinforcer Expectation extinction, rabbits were given microinjections of the GABA ago- nist muscimol adjacent to the motor nuclei that control the con- It is commonly thought that each CS presentation arouses a sort ditioned response (the facial nucleus and the accessory abdu- of expectation of the US that is disconfirmed on each extinction cens). The injection therefore eliminated the CR during extinc- trial. For example, in the error-correction rule provided by Res- tion. However, when the subjects were then tested without corla and Wagner (1972), the degree of unlearning (which we muscimol, the CS evoked considerable responding, suggesting have seen can create inhibition) is provided by the difference in that evocation of the CR was necessary for extinction learning. the overall associative strength present on a trial and the actual Unfortunately, the muscimol microinjections also had robust US that occurs on the trial. In the Pearce-Hall model (Pearce and stimulus effects. They caused complete inactivation of the ipsi- Hall 1980), the discrepancy was conceptualized as an event that lateral facial musculature: “the external eyelids were flaccid, the reinforced new inhibitory learning that is overlaid on the origi- left ear hung down unsupported, and no vibrissae movements nal excitatory learning (see also Daly and Daly 1982). Wagner’s were observed on the side of the infusion” (p. 10579). In effect, SOP model (1981) accepts a similar idea. One piece of evidence the rabbits received extinction in a context that was different that seems especially consistent with the expectation-violation from the one in which conditioning and testing occurred (the view is the “overexpectation experiment,” in which two CSs are ordinary state without partial facial paralysis). There are thus separately associated with the US and then presented together in strong grounds for expecting a renewal effect. The hypothesis a compound that is then paired with the US. Despite the fact that that elicitation of the CR is necessary for extinction must await the compound is paired with a US that can clearly generate ex- further tests. citatory learning, the two CSs undergo some extinction (see Kre- There are also data suggesting that the number of responses mer 1978; Lattal and Nakajima 1998). The idea is that summa- or level of responding in extinction does not correlate with ef- tion of the strengths of the two CSs causes a discrepancy between fective extinction learning. For example, Drew et al. (2004) noted what the animal expects and what actually occurs, and some that although animals given long CSs in extinction responded extinction is therefore observed. As mentioned above, the expec- many more times in extinction than did animals given shorter tation-violation view is also consistent with the effects of com- CSs, extinction was mainly a function of the number of extinc- pounding excitors and inhibitors with the target CS during no- tion trials. In fear conditioning experiments with mice, Cain et US (extinction) trials (Wagner 1969; Soltysik et al. 1983; Rescorla al. (2003) reported that extinction trials that were spaced in time 2000, 2003; Thomas and Ayres 2004). produced a slower loss of freezing than did extinction trials that One theoretical challenge has been to capture the expect- were massed in time. Nevertheless, there was less spontaneous ancy violation in real time. Gallistel and Gibbon (2000) have recovery after the massed treatment, suggesting that extinction emphasized the fact that traditional trial-based models such as was more effective when the treatment involved less overall re- the Rescorla-Wagner model have been vague about the precise sponding. Experiments in my own laboratory with different ap- point in time in a trial when the violation of expectation actually petitive conditioning methods in rats (E.W. Moody, C. Sunsay, occurs. The issue is especially clear when trial-based models ex- Learning & Memory 491 www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Bouton plain the extinction that occurs with a single extended presen- current performance depends on which of two associations is tation of the CS, as is the case for the context or background in retrieved. Consistent with this idea, another fact that emerges conditioning protocols with very widely spaced conditioning tri- from behavioral research on extinction is that it is relatively con- als. (Spaced trials are held to facilitate conditioning of the CS text-dependent. I have therefore suggested that the second (in- because long intertrial intervals allow more context extinction hibitory) association of CS is especially dependent on the context and thus less blocking by context.) There is good evidence that for its activation or retrieval. The role of the context is modula- widely spaced trials do create less contextual conditioning than tory; its activates or retrieves the CS’s own second (inhibitory) massed trials (see Barela 1999). To account for contextual extinc- association, much as a negative occasion setter might (see Hol- tion over long intertrial intervals, many trial-based models arbi- land, 1992). This hypothesis begins to integrate several facts trarily assume that the single long context exposure is carved about extinction and brings relapse effects such as the renewal into many imaginary trials, and that more imaginary trials occur effect, spontaneous recovery, rapid reacquisition, and reinstate- and create more extinction in longer context exposures. ment to center stage. Extinction is not the same as unlearning, It is worth noting, however, that Wagner’s SOP model (see and the context-dependence of extinction performance is a cen- Wagner 1981; Wagner and Brandon 1989, 2001) is relatively spe- tral part of that insight. cific about when in time the process that generates extinction occurs. As already mentioned, extinction occurs because an in- ACKNOWLEDGMENTS hibitory CS–US association develops when the CS is in the A1 I thank John Pearce for many useful discussions. Preparation of state and the US is in the A2 state. After conditioning has oc- this article was supported by Grant RO1 MH64847 from the Na- curred, the CS will continuously activate elements of the US node tional Institute of Mental Health. It was written while the author to the A2 state as long as it is present; therefore, extinction learn- was a visiting fellow at Cardiff University under the support of ing will occur continuously as long as the CS is on and no US the Cardiff Visiting Fellows Scheme. occurs. A limiting factor, however, is the extent to which the CS itself is in the A2 state: The longer it remains on, the more likely REFERENCES the elements in the CS node will be in A2 rather than A1, making Aguado, L., Symonds, M., and Hall, G. 1994. Interval between new learning about the CS more difficult. Nonetheless, exten- preexposure and test determines the magnitude of latent inhibition: sions of the CS in extinction will have an effect, because elements Implications for an interference account. Anim. Learn. Behav. 22: 188–194. in A2 eventually return to the inactive state, from which they Ayres, J.J.B., Berger-Gross, P., Kohler, E.A., Mahoney, W.J., and Stone, S. will return to A1 because of the continued presence of the CS. 1979. Some orderly nonmonotonicities in the trial-by-trial SOP thus accounts for extinction in extended CSs without re- acquisition of conditioned suppression: Inhibition with course to imaginary trials, and a recent extension of the model reinforcement? Anim. Learn. Behav. 7: 174–180. Baker, A.G., Steinwald, H., and Bouton, M.E. 1991. Contextual (Vogel et al. 2003) may also account for generalization decre- conditioning and reinstatement of extinguished instrumental ment as a function of CS time (Haselgrove and Pearce 2003; Drew responding. Q. J. Exp. Psychol. B 43: 199–218. et al. 2004). Although a truly rigorous analysis of SOP requires Barela, P.B. 1999. Theoretical mechanisms underlying the trial spacing computer simulations that are beyond the scope of the present effect in Pavlovian conditioning. J. Exp. Psychol. Anim. Behav. Processes 25: 177–193. article, the principles contained in the model are consistent with Bouton, M.E. 1984. Differential control by context in the inflation and many of the facts of extinction reviewed here. From the current reinstatement paradigms. J. Exp. Psychol. Anim. Behav. Processes point of view, its most significant problem is that it underesti- 10: 56–74. mates the role of context in extinction and might not account for . 1986. Slow reacquisition following the extinction of conditioned suppression. Learn. Motivat. 17: 1–15. the negative occasion-setting function of context (see Bouton . 1988. Context and ambiguity in the extinction of emotional and King 1983; Bouton and Swartzentruber 1986, 1989; Bouton learning: Implications for exposure therapy. Behav. Res. Ther. and Nelson 1998) that arguably provides the key to understand- 26: 137–149. ing renewal, spontaneous recovery, rapid reacquisition, and re- . 1991. Context and retrieval in extinction and in other examples of interference in simple associative learning. In: Current topics in instatement phenomena (for a start at addressing occasion set- animal learning: Brain, emotion, and cognition (eds. L. Dachowski and ting phenomena in terms of SOP, see Brandon and Wagner 1998; C.F. Flaherty), pp. 25–53. Lawrence Erlbaum, Hillsdale, NJ. Wagner and Brandon 2001). . 1993. Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychol. Bull. 114: 80–99. . 1994. Conditioning, remembering, and forgetting. J. Exp. Psychol. Anim. Behav. Processes 20: 219–231. Conclusions . 2002. Context, ambiguity, and unlearning: Sources of relapse Extinction is a highly complex phenomenon, even when ana- after behavioral extinction. Biol. Psychiatry 52: 976–986. lyzed at a purely behavioral level. It is worth noting that it is Bouton, M.E. and Bolles, R.C. 1979a. Contextual control of the extinction of conditioned fear. Learn. Motivat. 10: 445–466. probably multiply determined. But according to the results re- . 1979b. Role of conditioned contextual stimuli in reinstatement of viewed here, it does not involve destruction of the original learn- extinguished fear. J. Exp. Psychol. Anim. Behav. Processes 5: 368–378. ing. Instead, the main behavioral factors that cause the loss of Bouton, M.E., and Brooks, D.C. 1993. Time and context effects on responding appear to be generalization decrement, and new performance in a Pavlovian discrimination reversal. J. Exp. Psychol. Anim. Behav. Processes 19: 165–179. learning that may be initiated by the violation of an expectation Bouton, M.E., and King, D.A. 1983. Contextual control of the extinction of the US. In SOP, perhaps the most powerful and comprehensive of conditioned fear: Tests for the associative value of the context. J. model of associative learning that is currently available, that ex- Exp. Psychol. Anim. Behav. Processes 9: 248–265. pectation violation takes the form of the CS activating the US . 1986. Efect of context on performance to conditioned stimuli with mixed histories of reinforcement and nonreinforcement. J. Exp. node into a secondarily-active (A2) state that potentially enables Psychol. Anim. Behav. Processes 12: 4–15. new inhibitory learning as long as the CS remains on and no US Bouton, M.E. and Nelson, J.B. 1994. Context-specificity of target versus is presented. Importantly, this new inhibitory learning leaves the feature inhibition in a feature-negative discrimination. J. Exp. original CS–US association intact. Psychol. Anim. Behav. Processes 20: 51–65. Bouton, M.E. and Nelson, J.B. 1998. Mechanisms of feature-positive and I have argued that the fact that extinction might leave the feature-negative discrimination learning in an appetitive original learning intact means that the CS emerges from extinc- conditioning paradigm. In Occasion setting: Associative learning and tion with two available associations with the US. It therefore has cognition in animals (eds. N. Schmajuk and P.C. Holland), pp. properties analogous to those of an ambiguous word, and the 69–112. American Psychological Association, Washington, DC. 492 Learning & Memory www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Context, Inhibition, and Extinction Bouton, M.E. and Peck, C.A. 1989. Context effects on conditioning, Behav. 24: 437–449. extinction, and reinstatement in an appetitive conditioning . 2004. Experimental extinction in Pavlovian conditioning: preparation. Anim. Learn. Behav. 17: 188–198. Behavioural and neuroscience perspectives. Q. J. Exp. Psychol. Bouton, M.E. and Peck, C.A. 1992. Spontaneous recovery in 57B: 97–132. cross-motivational transfer (counter-conditioning). Anim. Learn Denniston, J.C., Chang, R.C., and Miller, R.R. 2003. Massive extinction Behav. 20: 313–321. treatment attenuates the renewal effect. Learn. Motivat. 34: 68–86. Bouton, M.E. and Ricker, S.T. 1994. Renewal of extinguished responding Devenport, L., Hill, T., Wilson, M., and Ogden, E. 1997. Tracking and in a second context. Anim. Learn. Behav. 22: 317–324. averaging in variable environments: A transition rule. J. Exp. Psychol. Bouton, M.E. and Sunsay, C. 2003. Importance of trials versus Anim. Behav. Processes 23: 450–460. accumulating time across trials in partially-reinforced appetitive Drew, M.R., Yang, C., Ohyama, T., and Balsam, P.D. 2004. Temporal conditioning. J. Exp. Psychol. Anim. Behav. Processes 29: 62–77. specificity of extinction in autoshaping. J. Exp. Psychol. Anim. Behav. Bouton, M.E. and Swartzentruber, D. 1986. Analysis of the associative Processes 30: 163–176. and occasion-setting properties of contexts participating in a Frohardt, R.J., Guarraci, F.A., and Bouton, M.E. 2000. The effects of Pavlovian discrimination. J. Exp. Psychol. Anim. Behav. Processes neurotoxic hippocampal lesions on two effects of context after fear 12: 333–350. extinction. Behav. Neurosci. 114: 227–240. Bouton, M.E. and Swartzentruber, D. 1989. Slow reacquisition following Gallistel, C.R. and Gibbon, J. 2000. Time, rate, and conditioning. extinction: Context, encoding, and retrieval mechanisms. J. Exp. Psychol. Rev. 107: 289–344. Psychol. Anim. Behav. Processes 15: 43–53. Gluck, M. and Myers, C.E. 1993. Hippocampal mediation of stimulus . 1992. Spontaneous recovery in cross-motivational transfer representation: A computational theory. Hippocampus 3: 492–516. (counterconditioning). Anim. Learn. Behav. 20: 313–321. Gordon, W.C. and Spear, N.E. 1973. Effect of reactivation of a Bouton, M.E., Kenney, F.A., and Rosengard, C. 1990. State-dependent previously acquired memory on the interaction between memories fear extinction with two benzodiazepine tranquilizers. Behav. in the rat. J. Exp. Psychol. 99: 349–355. Neurosci. 104: 44–55. Gunther, L.M., Denniston, J.C., and Miller, R.R. 1998. Conducting Bouton, M.E., Rosengard, C., Achenbach, G.G., Peck, C.A., and Brooks, exposure treatment in multiple contexts can prevent relapse. Behav. D.C. 1993. Effects of contextual conditioning and unconditional Res. Ther. 36: 75–91. stimulus presentation on performance in appetitive conditioning. Q. Hall, G. and Channell, S. 1985. Differential effects of contextual change J. Exp. Psychol. 46B: 63–95. on latent inhibition and on the habituation of an orienting Bouton, M.E., Woods, A.M., and Pineño, O. 2004. Occasional reinforced response. J. Exp. Psychol. Anim. Behav. Processes 11: 470–481. trials during extinction can slow the rate of rapid reacquisition. Hall, G. and Honey, R.C. 1990. Context-specific conditioning in the Learn. Motivat. (in press). conditioned-emotional-response procedure. J. Exp. Psychol. Anim. Brandon, S.E. and Wagner, A.R. 1998. Occasion setting: Influences of Behav. Processes 16: 271–278. conditioned emotional responses and configural cues. In Occasion Hall, G. and Mondragón, E. 1998. Contextual control as occasion setting: Associative learning and cognition in animals (eds. N.A. setting. In Occasion setting: Associative learning and cognition in Schmajuk and P.C. Holland), pp. 343–382. American Psychological animals (eds. N. Schmajuk and P.C. Holland), pp. 199–222). Association, Washington, DC. American Psychological Association, Washington, DC. Brooks, D.C. 2000. Recent and remote extinction cues reduce Harris, J.A., Jones, M.L., Bailey, G.K., and Westbrook, R.F. 2000. spontaneous recovery. Q. J. Exp. Psychol. 53B: 25–58. Contextual control over conditioned responding in an extinction Brooks, D.C. and Bouton, M.E. 1993. A retrieval cue for extinction paradigm. J. Exp. Psychol. Anim. Behav. Processes 26: 174–185. attenuates spontaneous recovery. J. Exp. Psychol. Anim. Behav. Hart, J.A., Bourne, M.J., and Schachtman, T.R. 1995. Slow reacquisition Processes 19: 77–89. of a conditioned taste aversion. Anim. Learn. Behav. 23: 297–303. . 1994. A retrieval cue for extinction attenuates response recovery Haselgrove, M. and Pearce, J.M. 2003. Facilitation of extinction by an (renewal) caused by a return to the conditioning context. J. Exp. increase or a decrease in trial duration. J. Exp. Psychol. Anim. Behav. Psychol. Anim. Behav. Processes 20: 366–379. Processes 29: 153–166. Brooks, D.C., Hale, B., Nelson, J.B., and Bouton, M.E. 1995. Haselgrove, M., Aydin, A., and Pearce, J.M. 2004. A partial Reinstatement after counterconditioning. Anim. Learn. Behav. reinforcement extinction effect despite equal rates of reinforcement 23: 383–390. during Pavlovian conditioning. J. Exp. Psychol. Anim. Behav. Processes Cain, C.K, Blouin, A.M., and Barad, M. 2003. Temporally massed CS 30: 240–250. presentations generate more fear extinction than spaced Holland, P.C. 1992. Occasion setting in Pavlovian conditioning. In The presentations. J. Exp. Psychol. Anim. Behav. Processes 29: 323–333. psychology of learning and motivation, Vol. 28 (ed. D.L. Medin), pp. Capaldi, E.J. 1967. A sequential hypothesis of instrumental learning. In 69–125. Academic Press, San Diego, CA. Psychology of learning and motivation, Vol. 1 (eds. K.W. Spence and Honey, R.C., Hall, G., and Bonardi, C. 1993. Negative priming in J.T. Spence), pp. 67–156. Academic Press, New York. associative learning: Evidence from a serial-conditioning procedure. . 1994. The sequential view: From rapidly fading stimulus traces J. Exp. Psychol. Anim. Behav. Processes 19: 90–97. to the organization of memory and the abstract concept of number. Kaye, H. and Pearce, J.M. 1984. The strength of the orienting response Psychonomic Bull. Rev. 1: 156–181. during Pavlovian conditioning. J. Exp. Psychol. Anim. Behav. Processes Chelonis, J.J., Calton, J.L., Hart, J.A., and Schachtman, T.R. 1999. 10: 90–109. Attenuation of the renewal effect by extinction in multiple contexts. Kehoe, E.J. 1988. A layered network model of associative learning: Learn. Motivat. 30: 1–14. Learning to learn and configuration. Psychol. Rev. 95: 411–433. Colwill, R.W. 1991. Negative discriminative stimuli provide information Kehoe, E.J. and Macrae, M. 1997. Savings in animal learning: about the identity of omitted response-contingent outcomes. Anim. Implications for relapse and maintenance after therapy. Behav. Ther. Learn. Behav. 19: 326–336. 28: 141–155. Crombag, H.S. and Shaham, Y. 2002. Renewal of drug seeking by Kimmel, H.D. and Burns, R.A. 1975. Adaptational aspects of contextual cues after prolonged extinction in rats. Behav. Neurosci. conditioning. In Handbook of learning and cognitive processes: 116: 169–173. Conditioning and behavior theory, Vol. 2 (ed. W.K. Estes), pp. 99–142. Cunningham, C.L. 1979. Alcohol as a cue for extinction: State Erlbaum, Hillsdale, NJ. dependency produced by conditioned inhibition. Anim. Learn. Konorski, J. 1948. Conditioned reflexes and neuron organization. Cambridge Behav. 7: 45–52. University Press, Cambridge, UK. Daly, H.B. 1974. Reinforcing properties of escape from frustration . 1967. Integrative activity of the brain. University of Chicago Press, aroused in various learning situations. In The psychology of learning Chicago. and motivation, Vol. 8 (ed. G.H. Bower), pp. 187–232. Academic Kraemer, P.J., Randall, C.K., and Carbary, T.J. 1991. Release from latent Press, New York. inhibition with delayed testing. Anim. Learn. Behav. 19: 139–145. Daly, H.B. and Daly, J.T. 1982. A mathematical model of reward and Kremer, E.F. 1978. The Rescorla-Wagner model: Losses in associative aversive nonreward: Its application in over 30 appetitive learning strength in compound conditioned stimuli. J. Exp. Psychol. Anim. situations. J. Exp. Psychol. Gen. 111: 441–480. Behav. Processes 4: 22–36. Danguir, J. and Nicolaidis, S. 1977. Lack of reacquisition in learned taste Krupa, D.J. and Thompson, R.F. 2003. Inhibiting the expression of a aversions. Anim. Learn. Behav. 5: 395–397. classically conditioned behavior prevents its extinction. J. Neurosci. De la Casa, L.G. and Lubow, R.E. 2000. Super-latent inhibition with 23: 10577–10584. conditioned taste aversion testing. Anim. Learn. Behav. 28: 389–399. Lattal, K.M. and Nakajima, S. 1998. Overexpectation in appetitive . 2002. An empirical analysis of the super-latent inhibition effect. Pavlovian and instrumental conditioning. Anim. Learn. Behav. Anim. Learn. Behav. 30: 112–120. 26: 351–360. Delamater, A.R. 1996. Effects of several extinction treatments upon the Lubow, R.E. and De la Casa, L.G. 2002. Superlatent inhibition and integrity of Pavlovian stimulus-outcome associations. Anim. Learn. spontaneous recovery: Differential effects of pre- and Learning & Memory 493 www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Bouton postconditioning CS-alone presentations after long delays in 27: 43–57. different contexts. Anim. Learn. Behav. 30: 376–386. . 1997. The effect of context change upon long-term memory of Mackintosh, N.J. 1974. The psychology of animal learning. Academic Press, CS duration. Behav. Processes 39: 69–76. New York. Rosas, J.M. and Bouton, M.E. 1997. Additivity of the effects of retention Maren, S. and Holt, W. 2000. The hippocampus and contextual memory interval and context change on latent inhibition: Toward resolution retrieval in Pavlovian conditioning. Behav. Brain Res. 110: 97–108. of the context forgetting paradox. J. Exp. Psychol. Anim. Behav. McClelland, J.L. and Rumelhart, D.E. 1985. Distributed memory and the Processes 23: 283–294. representation of general and specific information. J. Exp. Psychol. . 1998. Context change and retention interval can have additive, Gen. 114: 159–188. rather than interactive, effects after taste aversion extinction. McCloskey, M. and Cohen, N.J. 1989. Catastrophic interference in Psychonomic Bull. Rev. 5: 79–83. connectionist networks: The sequential learning problem. In The Schmajuk, N.A. and Holland, P.C., eds. 1998. Occasion-setting: Associative psychology of learning and motivation, Vol. 24 (ed. G.H. Bower), pp. learning and cognition in animals. American Psychological Association, 109–165. Academic Press, San Diego, CA. Washington, DC. Mowrer, O.H. and Jones, H.M. 1945. Habit strength as a function of the Soltysik, S.S., Wolfe, G.E., Nicholas, T., Wilson, J., and Garcia-Sanchez, pattern of reinforcement. J. Exp. Psychol. 35: 293–311. J.L. 1983. Blocking of inhibitory conditioning within a serial Myers, K.M. and Davis, M. 2002. Behavioral and neural analysis of conditioned stimulus-conditioned inhibitor compound: extinction. Neuron 36: 567–584. Maintenance of acquired behavior without an unconditioned Nakajima, S., Tanaka, S., Urshihara, K., and Imada, H. 2000. Renewal of stimulus. Learn. Motivat. 14: 1–29. extinguished lever-press responses upon return to the training Spear, N.E. 1981. Extending the domain of memory retrieval. In context. Learn. Motivat. 31: 416–431. Information processing in animals: Memory mechanisms (eds. N.E. Spear Napier, R.M., Macrae, M., and Kehoe, E.J. 1992. Rapid reacquisition in and R.R. Miller), pp. 341–378. Erlbaum, Hillsdale, NJ. conditioning of the rabbit’s nictitating membrane response. J. Exp. Spear, N.E., Smith, G.J., Bryan, R., Gordon, W., Timmons, R., and Psychol. Anim. Behav. Processes 18: 182–192. Chiszar, D. 1980. Contextual influences on the interaction between Nelson, J.B. 2002. Context specificity of excitation and inhibition in conflicting memories in the rat. Anim. Learn. Behav. 8: 273–281. ambiguous stimuli. Learn. Motivat. 33: 284–310. Staddon, J E.R. 1988. Learning as inference. In Evolution and learning Nelson, J.B. and Bouton, M.E. 1997. The effects of a context switch (eds. R.C. Bolles and M.D. Beecher), pp. 59–78. Erbaum, Hillsdale, following serial and simultaneous feature-negative discriminations. NJ. Learn. Motivat. 28: 56–84. Swartzentruber, D. 1995. Modulatory mechanisms in Pavlovian Overton, D.A. 1985. Contextual stimulus effects of drugs and internal conditioning. Anim. Learn. Behav. 23: 123–143. states. In Context and learning (eds. P.D. Balsam and A. Tomie), pp. Swartzentruber, D. and Bouton, M.E. 1992. Context sensitivity of 357–384. Erlbaum, Hillsdale, NJ. conditioned suppression following preexposure to the conditioned Pavlov, I.P. 1927. Conditioned reflexes. Oxford University Press, Oxford, stimulus. Anim. Learn. Behav. 20: 97–103. UK. Tamai, N. and Nakajima, S. 2000. Renewal of formerly conditioned fear Pearce, J.M. 1994. Similarity and discrimination: A selective review and in rats after extensive extinction training. Int. J. Comp. Psychol. a connectionist model. Psychol. Rev. 101: 587–607. 13: 137–147. Pearce, J.M. and Hall, G. 1980. A model for Pavlovian conditioning: Thomas, B.L. and Ayres J.J.B. 2004. Use of the ABA fear renewal Variations in the effectiveness of conditioned but not unconditioned paradigm to assess the effects of extinction with co-present fear stimuli. Psychol. Rev. 87: 332–352. inhibitors or excitors: Implications for theories of extintion and for Peck, C.A. and Bouton, M.E. 1990. Context and performance in treating human fears and phobias. Learn. Motivat. 35: 22–51. aversive-to-appetitive and appetitive-to-aversive transfer. Learn. Thomas, D.R., McKelvie, A.R., Ranney, M., and Moye, T.B. 1981. Motivat. 21: 1–31. Interference in pigeons’ long-term memory viewed as a retrieval Rauhut, A.S., Thomas, B.L., and Ayres, J.J.B. 2001. Treatments that problem. Anim. Learn. Behav. 9: 581–586. weaken Pavlovian conditioned fear and thwart its renewal in rats: Thomas, D.R., Moye, T.B., and Kmose, E. 1984. The recency effect in Implications for treating human phobias. J. Exp. Psychol. Anim. pigeons’ long-term memory. Anim. Learn. Behav. 12: 21–28. Behav. Processes 27: 99–114. Vigorito, M. and Ayres, J.J.B. 1987. Effect of naloxone on conditioned Reid, R.L. 1958. The role of the reinforcer as a stimulus. B. J. Psychol. suppression in rats. Behav. Neurosci. 101: 576–586. 49: 202–209. Vogel, E.H., Brandon, S.E., and Wagner, A.R. 2003. Stimulus Rescorla, R.A. 1993. Inhibitory associations between S and R in representation in SOP, II: An application to inhibition of delay. extinction. Anim. Learn. Behav. 21: 327–336. Behav. Processes 62: 27–48. . 1996. Preservation of Pavlovian associations through extinction. Wagner, A.R. 1969. Stimulus selection and a “modified continuity Q. J. Exp. Psychol. 49B: 245–258. theory.” In The psychology of learning and motivation, Vol. 3 (eds. G.H. . 1997. Response inhibition in extinction. Q. J. Exp. Psychol. Bower and J.T. Spence), pp. 1–41. Academic Press, New York. 50B: 238–252. . 1981. SOP: A model of automatic memory processing in animal . 2000. Extinction can be enhanced by a concurrent exciter. J. behavior. In Information processing in animals: Memory mechanisms Exp. Psychol. Anim. Behav. Processes 26: 251–260. (eds. N.E. Spears and R.R. Miller) pp. 5–47. Erlbaum, Hillsdale, NJ. . 2001. Experimental extinction. In Handbook of contemporary Wagner, A.R. and Brandon, S.E. 1989. Evolution of a structured learning theories (eds. R.R. Mowrer and S.B. Klein), pp. 119–154. connectionist model of Pavlovian conditioning (AESOP). In Erlbaum, Mahwah, NJ. Contemporary learning theories: Pavlovian conditioning and the status of . 2003. Protection from extinction. Learn. Behav. 31: 124–132. traditional learning theory (eds. S.B. Klein and R.R. Mowrer), pp. Rescorla, R.A. and Wagner, A.R. 1972. A theory of Pavlovian 149–189. Lawrence Erlbaum Associates, Hillsdale, NJ. conditioning: Variations in the effectiveness of reinforcement and . 2001. A componential theory of Pavlovian conditioning. In nonreinforcement. In Classical conditioning II: Current research and Handbook of contemporary learning theories (eds. R.R. Mowrer and S.B. theory (eds. A.H. Black and W.K. Prokasy), pp. 64–99. Klein), pp. 23–64. Erlbaum, Mahwah, NJ. Appleton-Century-Crofts, New York. Weidemann, G. and Kehoe, E.J. 2003. Savings in classical conditioning Rescorla, R.A. and Heth, C.D. 1975. Reinstatement of fear to an in the rabbit as a function of extended extinction. Learn. and Behav. extinguished conditioned stimulus. J. Exp. Psychol. Anim. Behav. 31: 49–68. Processes 1: 88–96. Westbrook, R.F., Jones, M.L., Bailey, G.K., and Harris, J.A. 2000. Ricker, S.T., and Bouton, M.E. 1996. Reacquisition following extinction Contextual control over conditioned responding in a latent in appetitive conditioning. Anim. Learn. Behav. 24: 423–436. inhibition paradigm. J. Exp. Psychol. Anim. Behav. Processes Robbins, S.J. 1990. Mechanisms underlying spontaneous recovery in 26: 157–173. autoshaping. J. Exp. Psychol. Anim. Behav. Processes 16: 235–249. Wilson, A., Brooks, D.C., and Bouton, M.E. 1995. The role of the rat Rosas, J.M. and Alonso, G. 1996. Temporal discrimination and hippocampal system in several effects of context in extinction. forgetting of CS duration in conditioned suppression. Learn. Motivat. Behav. Neurosci. 109: 828–836. 494 Learning & Memory www.learnmem.org Downloaded from learnmem.cshlp.org on November 1, 2021 - Published by Cold Spring Harbor Laboratory Press Mark E. Bouton Learn. Mem. 2004, 11: Access the most recent version at doi:10.1101/lm.78804 This article cites 89 articles, 1 of which can be accessed free at: References http://learnmem.cshlp.org/content/11/5/485.full.html#ref-list-1 License Receive free email alerts when new articles cite this article - sign up in the box at the Email Alerting top right corner of the article or click here. Service Cold Spring Harbor Laboratory Press

Journal

Learning & MemoryUnpaywall

Published: Sep 1, 2004

There are no references for this article.