Nothing Special   »   [go: up one dir, main page]
Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2011 Aug 17;31(33):11786–11794. doi: 10.1523/JNEUROSCI.2711-11.2011

The General and Outcome-Specific Forms of Pavlovian-Instrumental Transfer Are Differentially Mediated by the Nucleus Accumbens Core and Shell

Laura H Corbit 1,✉, Bernard W Balleine 2
PMCID: PMC3208020  NIHMSID: NIHMS318930  PMID: 21849539

Abstract

Tests of Pavlovian–instrumental transfer (PIT) demonstrate that reward-predictive stimuli can exert a powerful motivational influence on the performance of instrumental actions. Recent evidence suggests that predictive stimuli produce this effect through either the general arousal (general PIT) or the specific predictions (outcome-specific PIT) produced by their association with reward. In two experiments, we examined the effects of pretraining lesions (Experiment 1) or muscimol-induced inactivation (Experiment 2) of either the core or shell regions of the nucleus accumbens (NAC) on these forms of PIT. Rats received Pavlovian training in which three auditory stimuli each predicted the delivery of a distinct food outcome. Separately, the rats were trained to perform two instrumental actions, each of which earned one of the outcomes used in Pavlovian conditioning. Finally, the effects of the three stimuli on performance of the two actions were assessed in extinction. Here we report evidence of a double dissociation between general and outcome-specific PIT at the level of the accumbens. Shell lesions eliminated outcome-specific PIT but spared general PIT, whereas lesions of the core abolished general PIT but spared outcome-specific PIT. Importantly, the infusion of muscimol into core or shell made immediately before the PIT tests produced a similar pattern of results. These results suggest that whereas the NAC core mediates the general excitatory effects of reward-related cues, the NAC shell mediates the effect of outcome-specific reward predictions on instrumental performance, and thereby serve to clarify reported discrepancies regarding the role of the NAC core and shell in PIT.

Introduction

It is well established that, in instrumental conditioning, animals encode the relationship between their actions and the consequences or outcome of those actions. Nevertheless, it has become clear that multiple incentive processes contribute to instrumental performance. In addition to an instrumental incentive process, through which animals evaluate the current value of the instrumental outcome or goal (Dickinson and Balleine, 2002; Corbit and Balleine, 2003), Pavlovian incentive learning, which mediates the excitatory effects of stimuli associated with the outcome, can also contribute importantly to various aspects of instrumental performance (Estes, 1943; Lovibond, 1983). The interaction of these incentive processes can be readily examined using the Pavlovian–instrumental transfer (PIT) paradigm and experiments using this task have shown that Pavlovian stimuli can influence instrumental performance in at least two ways. A stimulus may produce an expectancy of a particular reward and so selectively enhance the performance of responses associated with that unique reward. Additionally reward related stimuli can enhance responding due to arousal elicited by association with reward generally (Rescorla and Solomon, 1967; Trapold and Overmier, 1972; Overmier and Lawry, 1979; Corbit and Balleine, 2005; Corbit et al., 2007).

The nucleus accumbens (NAC) appears to play a central role in the way reward-related cues influence instrumental performance. However, the heterogeneous structure of the NAC itself and differences in the learning paradigms used in previous studies have made it difficult to precisely specify this role. With regard to PIT, Hall et al. (2001) (see also Holland and Gallagher, 2003) reported that lesions of the NAC core but not shell abolished PIT, whereas in apparent opposition, Corbit et al. (2001) reported that lesions of the NAC shell but not core eliminate PIT. However, differences in the training procedures, and thus potentially in the forms of PIT, in these experiments complicate interpretation of the lesion data. Hall et al. (2001) assessed the influence of a single excitatory cue on performance on a single lever, a procedure likely to maximize the influence of the general activational effects of the cue on performance. In contrast, Corbit et al. (2001) assessed the effects of two excitatory conditional stimuli (CSs) on the performance of two different actions each rewarded with one of the unique outcomes predicted by the CSs, a procedure likely to maximize the influence of the outcome-specific predictions on performance.

To assess this account of these discrepant findings, two experiments examined the role of the NAC core and shell in PIT using a procedure that allows both the general and specific forms of transfer to be assessed within the same subject (see Table 1) (Corbit and Balleine, 2005; Corbit et al., 2007). In Experiment 1, we examined the effects of pretraining lesions, and in Experiment 2 the effect of temporary inactivation of the core or shell immediately before the PIT test. If it is the form of PIT that is responsible for the differences in previous results, lesion or inactivation of the shell should eliminate outcome-specific PIT, but spare general PIT. In contrast, lesion or inactivation of the core should eliminate general PIT while sparing the outcome-specific effects.

Table 1.

Design of Experiments 1 and 2

Conditioning Instrumental training Transfer test
S1–O1 S1 graphic file with name permzz1000bct030.jpg
S2–O2 R1→O1; R2→O2 S2 R1 vs R2
S3–O3 S3
Open in a new tab

During initial conditioning rats received pairing of three auditory cues, S1, S2, and S3 (noise, tone, and clicker, counterbalanced), with three separate outcomes, O1, O2, and O3 (pellet, sucrose, or polycose, counterbalanced). They then received instrumental training on two separate lever press actions, R1 and R2, each delivering one of the outcomes used in initial conditioning, O1 or O2. Finally a test was conducted on the levers in extinction in the presence of each of the three stimuli. The tests of S1 and S2 provide an assessment of the effects of the outcome-specific predictions of the cues. The test of S3, predicting an outcome not earned on the levers, provides a test of the general excitatory effects of reward prediction.

Materials and Methods

Experiment 1: pre-training lesions

Subjects and apparatus

Thirty-two experimentally naive male Long–Evans rats (Harlan) served as subjects. All animals were housed in pairs and handled daily for 1 week before surgery or training. Training and testing took place in 24 Med Associates operant chambers housed within sound- and light-resistant shells. Each chamber was equipped with two pumps, each of which was fitted with a syringe that delivered 0.1 ml of fluid into a recessed magazine in the chamber. The two fluids were a 20% solution of sucrose or a 20% solution of polycose with 0.9% sodium chloride. The chambers were also equipped with a pellet dispenser that delivered one 45 mg pellet when activated (Bio-Serve). The chambers contained two retractable levers that could be inserted to the left and the right of the magazine. The chambers contained a white noise generator, a Sonalert that delivered a 3 kHz tone, and a solenoid that, when activated, delivered a 5 Hz clicker stimulus. All stimuli were adjusted to 80 dB in the presence of background noise of 60 dB provided by a ventilation fan. A 3 W, 24 V houselight mounted on the wall across from the levers and magazine illuminated the chamber. Microcomputers equipped with MED-PC software (Med Associates) controlled the equipment and recorded responses.

Surgery

Rats received cell-body lesions of the NAC core or the NAC shell or sham surgery. Rats were anesthetized using sodium pentobarbital (Nembutal; 50 mg/kg; Abbott Laboratories) and treated with atropine (0.1 mg) and were then placed in a stereotaxic frame (Stoelting) with the incisor bar set at −3.3 mm. The scalp was retracted and small burr holes were drilled above the target regions. For lesions of the NAC core, animals received bilateral injections of 0.5 μl of 0.12 m NMDA into two sites (one per side) using a 1.0 μl Hamilton syringe (coordinates relative to bregma; anteroposterior, +1.2; mediolateral, ±2.1; dorsoventral (from dura), −7.0). For lesions of the NAC shell, animals received 0.2 μl injections of 0.015 m AMPA hydrobromide at four sites (two per hemisphere, anteroposterior, +1.6; mediolateral, ±0.8; dorsoventral (from dura), −6.0, −6.8). Injections were made over 2 min, and a minimum of an additional 2 min was allowed for diffusion before any movement of the needle. Animals in the surgical control group underwent similar treatment except that no neurotoxin was injected.

Histology

At the end of the experiment, animals were killed with a barbiturate overdose and perfused transcardially with 0.9% saline followed by 10% formaldehyde solution. The brains were stored in 10% formalin solution and transferred to a 25% sucrose solution 48 h before slicing. Fifty microgram coronal sections were cut through the region of the NAC. Tissue was stained using thionin and slides were examined for placement and extent of the lesions, with the latter assessed by microscopically examining regions of marked cell loss and gliosis as well as shrinkage of a region relative to sham controls.

Behavioral procedures

General.

See Table 1 for the design of the behavioral components of these experiments. Animals were given 2 weeks to recover from surgery before being placed on a food deprivation schedule. Feeding was such that rats received 1 h access to their maintenance diet daily to maintain them at ∼90% of their free-feeding weight. The animals were fed after the training sessions of the day and had ad libitum access to tap water while in the home cage. Each session began with the illumination of the houselight and insertion of levers where appropriate and ended with the retraction of the levers and turning off of the houselight. Sessions were 30 min in duration unless otherwise stated.

Pavlovian training.

The rats received 10 sessions of Pavlovian conditioning. Three auditory stimuli (tone, white noise, and clicker) served as CSs and were paired with either pellet, 20% sucrose, or 20% polycose (plus 0.9% sodium chloride) delivery (counterbalanced). Four presentations of each stimulus were given in each session in random order interspersed with periods in which no stimuli were presented. The length of these intervals varied, but on average, were 4 min. The stimuli presentations were 2 min long, during which the appropriate outcome was delivered on a random time-30 s schedule of reinforcement. The number of magazine entries during the stimuli as well as in a prestimulus interval of equal length (2 min) was measured. Additionally, the final training session contained probe trials (one of each stimulus type) in which the stimulus was presented but its associated reinforcer was omitted to measure conditioned responding in the absence of the unconditional stimulus. Sessions were ∼75 min in duration.

Instrumental training.

The rats were trained to perform two lever-press responses (left and right) each reinforced on random ratio (RR) schedules. Each lever was trained separately and earned one of three possible outcomes: pellets, sucrose, or polycose. Any given animal earned two of these outcomes, one by performance of the left lever response and one by performance of the right lever response. These assignments were counterbalanced across animals and within each lesion condition. The animals first received 1 d of continuous reinforcement and were then shifted to an RR-5 schedule (i.e., each action delivered an outcome with a probability of 0.2). After 3 d on this schedule, animals were shifted to an RR-10 schedule (i.e., each action delivered an outcome with a probability of 0.1) for an additional 3 d. The animals received two training sessions each day, one with each action–outcome pair with a break of at least 1 h between sessions. The order of sessions was alternated each day.

Pavlovian–instrumental transfer tests.

The animals received two extinction tests, one on each lever, conducted on separate days. During each test, one of the levers was available, and the three stimuli were each presented three times interspersed with intervals of no stimulus. Each test was 40 min in duration. In the first 4 min, the levers were available, but no stimuli were presented. This period was followed by eighteen 2 min bins that contained a total of nine stimulus trials (three trials for each stimulus) starting with and intermixed with no stimulus intervals. The order of these trials was randomly determined by the computer program with the constraint that there could be only three of each trial type.

Experiment 2: reversible inactivation

Subjects and apparatus

Sixteen experimentally naive male Long–Evans rats (Monash) served as subjects. All animals were housed in pairs and handled daily for 1 week before surgery or training. Training and testing took place in 16 Med Associates operant chambers outfitted in a fashion identical to those described above.

Surgery

Stereotaxic surgery was conducted under isoflurane anesthesia (5% induction; 1–2% maintenance) to implant 26 gauge guide cannulae (Plastics One) targeted at either the core or shell (N = 8 per group). The general procedures were as described above. The same coordinates were used as for the lesions above except that the tips of the guide cannulae were positioned 3 mm dorsal to the intended infusion site and anchored with machine screws and dental acrylic.

Training

The Pavlovian and instrumental training sessions were identical to those described above.

Pavlovian–instrumental transfer tests

Each animal underwent a total of four tests such that they were tested in both the inactivation and control conditions for both the left and right levers (order counterbalanced for each placement group). The tests themselves were identical to those described above. Following the first two tests, the rats received two instrumental sessions for each lever and one Pavlovian session to help ameliorate the effects of repeated extinction tests.

Microinfusions

Inactivation was achieved with the GABAA receptor agonist muscimol (MUS; 0.1 mm, Sigma) delivered via infusion cannulae (33 gauge; Plastics One) extending 3 mm below the guide cannula tip. A volume of 0.3 μl was delivered per hemisphere at a rate of 0.3 μl per minute by a syringe pump (Harvard Apparatus, PHD 22/2000) 10 min before the test. Saline vehicle was administered as the control condition. Infusions took place over 1 min and the cannulae were left in place for an additional 2 min to allow for diffusion.

Histology

At the end of the experiment, tissue was prepared and examined as described above to determine the placement of the cannula tips.

Results

Experiment 1: pretraining lesions

Histology

No recovery problems or weight loss were observed after surgery. Figure 1 displays the maximum and minimum damage resulting from the lesions for the rats included in the behavioral analysis. Core lesions resulted in substantial loss of neurons in the region of the core bilaterally and typically extended in the anteroposterior plane from 0.7 to 1.7 mm anterior to bregma. Generally the lesion did not extend ventrally into the ventral pallidum or dorsally into the caudate. Any animals without bilateral core lesions or with marked damage to the shell region were excluded from the behavioral analyses. Shell lesions destroyed neurons in the mediodorsal shell bilaterally and damage typically extended in the rostral–caudal plane from 1.0 to 1.7 mm anterior to bregma. The ventral and ventrolateral portions of the shell appeared unaffected by the lesions. Any animals without bilateral shell lesions or with substantial damage to the core were excluded. One shell and two core animals were excluded due to unilateral damage. One sham animal was excluded due to a failure to acquire one of the instrumental actions leaving the following final group sizes: shell n = 9, core n = 8, sham n = 11.

Figure 1.

Figure 1.

Open in a new tab

Schematic representation of excitotoxic lesions of the NAC core (A) and shell (B) for Experiment 1. Shaded areas represent the maximum (dark gray) and minimum (light gray) extent of the lesions for the animals included in the behavioral analyses. Coronal sections are taken from the following points in the anteroposterior plane beginning at top left: +1.7, +1.2, +1.0, and +0.7 mm anterior to bregma (Paxinos and Watson, 1998). C, Schematic representation of the placement of cannula tips for the NAC core (left) and shell (right) groups for Experiment 2.

Pavlovian training

To assess whether the animals learned the relationship between the stimuli and food deliveries, the number of magazine entries during the stimuli (CS) was compared to the entries during a prestimulus interval of equal length. These training data are illustrated in Figure 2A, which indicates that Pavlovian conditioning was similar in the three groups; all of the animals made more entries during the stimulus presentations than during the pre-CS intervals. Preliminary analyses revealed no effect of stimulus type (F < 1), and so the data are presented collapsed across stimulus. ANOVA revealed no effect of group (F(2,25) = 0.37, p > 0.05), but a main effect of training day (F(9,225) = 11.88, p < 0.01), suggesting that more magazine entries were made as training progressed, and of interval (CS vs pre-CS), which, as indicated by Figure 2A, demonstrates that more entries were made during the CS than during the pre-CS intervals (F(1,25) = 304.7, p < 0.01). There was also an interaction between day and interval, suggesting that the difference in magazine entries between the CS and pre-CS intervals increased with acquisition across days of training (F(9,225) = 19.33, p < 0.01). No other interactions, and particularly no interactions involving group, were significant.

Figure 2.

Figure 2.

Open in a new tab

A, Pavlovian conditioning. Mean number of magazine entries during the CS presentations and during the pre-CS intervals (±1 SEM) across days of Pavlovian training for the three lesioned groups. Rats in all groups learned to enter the magazine selectively during the stimulus periods. B, Instrumental conditioning. Mean lever-press responses for the three groups across days of training (±1 SEM). For the first day, each response was reinforced. Thereafter, responding was reinforced on an RR-5 schedule of reinforcement for days 2–4, and on an RR-10 schedule of reinforcement for days 5–7. Rats in all groups acquired the instrumental responses and increased response rates across days.

Instrumental training

The rats acquired the lever-press responses similarly regardless of the outcomes used to reward performance (outcome effect, F values <1). The mean number of lever presses across days of training is presented in Figure 2B and illustrates that all groups acquired the instrumental responses. This description is supported by the ANOVA, which revealed no effect of group (F(2,25) = 0.19, p > 0.05); a significant effect of training day, suggesting that responding increased across days (F(6,150) = 73.85, p < 0.01); and no interaction between day and group (F(12,150) = 0.32, p > 0.05).

Effect of NAC core and shell lesions on Pavlovian–instrumental transfer

The objective of this test was to assess the impact of Pavlovian predictors of reward on instrumental performance. For each lever, one of the stimuli was paired with the same outcome as had been earned by that lever in training (thus, designated the “same” stimulus), whereas another stimulus was paired with an outcome earned by a different response (thus, designated the “different” stimulus), i.e., stimuli S1 and S2 in Table 1. The remaining stimulus was paired with reward but not with any outcome specifically associated with either lever (designated the “general” stimulus), i.e., stimulus S3 in Table 1. Preliminary analyses indicated no effect of lever (left vs right), and so for the purposes of presentation and subsequent analyses, the data are collapsed across this variable and thus, the two tests. The data are presented in Figure 3 in two ways: In A, we present, more completely, the effects of the stimuli on performance, separated by lesion group, showing lever press performance during the stimuli and during the baseline period of the tests. In B and C, the data are separated by specific and general PIT and the baseline response rates were subtracted to facilitate between-groups comparisons of the stimulus effects. It is clear from these panels that whereas all groups showed some impact of stimulus presentation on performance, the pattern of effects differed across groups. Consistent with our previous findings, lesions of the shell reduced the specific excitatory effect of the stimulus paired with the same outcome as the available lever (specific transfer) but left the excitatory effect of the stimulus paired with reward not earned by a specific instrumental response (general transfer) intact (Fig. 3A, middle). The opposite pattern was observed for rats with lesions of the core (Fig. 3A, right). In these animals, there was a clear specific transfer effect; however, the ability of the stimulus paired generally with reward to elevate responding was eliminated. Performance of sham animals showed robust evidence of both specific and general PIT and responding was elevated in the presence of both the same and general stimuli (Fig. 3A, left). This description was supported by the statistical analysis, which revealed no overall effect of group (F(2,25) < 1) but a significant effect of stimulus (baseline, same, different, general; F(3,75) = 6.34, p < 0.01) and an interaction between these factors (F(6,75) = 2.3, p < 0.05). Simple effects analyses for the effect of stimulus presentation were conducted for each group to identify the source of the interaction. In sham animals, there was a significant effect of stimulus (F(3,30) = 10.76, p < 0.01) and post hoc analyses demonstrated that, compared to the baseline intertrial intervals, performance was elevated by both the same (t(10) = 5.4, p < 0.01) and general (t(10) = 3.4, p < 0.01) stimuli but not the different stimulus (t(10) = 1.66, p > 0.05). Importantly, responding during the same stimulus was significantly greater than during the different stimulus (t(10) = 3.49, p < 0.01). In shell animals, there was also an overall effect of stimulus (F(3,24) = 3.85, p < 0.05) and, in contrast to sham animals, post hoc analyses demonstrated that, compared to the baseline, only the general stimulus significantly elevated responding (t(8) = 3.19, p < 0.01; same; t(8) = 1.2, p > 0.05; different; t(8) = 1.5, p > 0.05) and importantly, there was no difference in responding between the same and different stimuli (t(8) = 0.3, p > 0.05), thus indicating no evidence of a specific transfer effect. Finally, in core rats there was also an overall effect of stimulus (F(3,21) = 5.58, p < 0.01). Post hoc analyses indicated that presentation of the same stimulus elevated responding relative to baseline (t(7) = 4.5, p < 0.01) and the different stimulus (t(7) = 3.16, p < 0.01) but the different (t(7) = 0.93, p > 0.05) and general (t(7) = 0.62, p > 0.05) stimuli were without effect.

Figure 3.

Figure 3.

Open in a new tab

Effects of core and shell lesions on the outcome-specific and general forms of Pavlovian–instrumental transfer. A, Mean number of lever presses during the Pavlovian–instrumental transfer tests (+SEM). The number of responses during the baseline period in which no stimuli were present is displayed in each case. “Same” refers to the stimulus paired in training with the same outcome as was earned by the available lever. “Different” refers to the stimulus paired with the outcome earned by the other response. “General” refers to the third stimulus paired with the reward not earned by either response. Sham rats (left) demonstrated both outcome-specific and general transfer effects. The performance of shell rats (middle) was increased by stimulus presentation, but they showed no evidence of an outcome-specific PIT effect. In contrast, core rats (right) showed evidence of outcome-specific PIT but a reduced general PIT effect. *Responding during the stimulus was significantly different from responding during the baseline intervals (p < 0.01). ⋀Responding during the “same” stimulus was greater than during the “different” stimulus (p < 0.01). B, Effects of shell and core lesions on outcome-specific PIT. Mean lever presses during each of the same and different stimuli with baseline response rates for each group subtracted. *Responding was greater during the same compared to the different stimulus; **significant difference compared to the sham group. C, Effects of shell and core lesions on general PIT. Mean lever presses during the general stimulus for each group with their respective baseline response rates subtracted. **Significant difference compared to the sham group.

In Figure 3, B and C, we present a summary of the lesion effects on the transfer tests now organized by specific (Fig. 3B) and general (Fig. 3C) PIT, and presented as an elevation in instrumental performance over baseline (i.e., CS − pre-CS) induced by the various Pavlovian cues to facilitate between group comparisons. Thus, in Figure 3B, we present the effects of the lesions on specific transfer induced by the presentation of stimuli S1 and S2, whereas in Figure 3C, we present the effects of the general stimulus S3 (refer to Table 1) on performance over baseline. This summary helps to make clear that, in sham animals, there is evidence of outcome-specific transfer; presentation of the stimulus paired with the same outcome as the available lever enhanced responding, whereas presentation of a stimulus paired with a different outcome, earned by another currently unavailable response, was without effect. This pattern was intact in animals with core lesions but abolished in those with shell lesions. Again, this is supported by the statistical analysis, which demonstrates an effect of stimulus (same vs different; F(1,25) = 12.0, p < 0.01), no main effect of group (F(2,25) = 1.9, p > 0.05), and an interaction between these factors (F(2,25) = 4.8, p < 0.05). An effect of lesion was supported by an effect of group for the same (F(2,25) = 5.4, p < 0.01) but not different (F(2,25) < 1) stimulus. That is, relative to the sham group, the effect of the same stimulus was reduced by lesions of the shell (t(19) = 3.02, p < 0.01), but not core (t(18) < 1).

In addition, there was evidence in sham animals of general transfer; presentation of a stimulus paired with food reward that was not earned by either lever response elevated performance on both levers. For rats with core lesions, presentation of the third stimulus had no effect on performance, indicating that the general transfer process was impaired in these animals. In contrast, animals with shell lesions, while there was no evidence of outcome-specific transfer, the general excitatory effect of a reward-related stimulus remained. Again, this was supported by the statistical analyses, which indicated a main effect of group (F(2,25) = 4.08, p < 0.05), and post hoc analyses confirm that the effect of the general stimulus relative to shams was reduced for rats in the core (t(18) = 2.5, p < 0.05) but not shell (t(19) = 1.04, p > 0.05) groups.

Experiment 2: reversible inactivation

It is unclear from the data above whether the impairments observed in PIT are the result of a failure to integrate knowledge of the specific Pavlovian and instrumental associations established in training to inform behavioral choice and vigor, or whether the acquisition of these associations was in some way altered by the pretraining lesions. Thus, we were interested in whether the same pattern of results would be found if manipulations of the core and shell came only at the time of testing, thus providing an opportunity for animals to form the same Pavlovian and instrumental associations as sham animals during training. To examine this question, we performed a similar study but used reversible inactivation to inactivate either the core or shell just before PIT testing.

Histology

No recovery problems or weight loss were observed after surgery. Figure 1C displays the location of the cannula tips.

Pavlovian training

To assess whether the animals learned the relationship between the stimuli and food deliveries, the number of magazine entries during the stimuli (CS) was compared to the entries during a prestimulus interval of equal length. These training data are illustrated in Figure 4A, which illustrates that Pavlovian conditioning was similar for the two groups. Rats made more entries during the stimulus presentations than during the pre-CS intervals. Preliminary analyses revealed no effect of stimulus type (F < 1), so the data are presented collapsed across stimulus. ANOVA revealed a main effect of training day (F(9,126) = 24.0, p < 0.01), suggesting that more magazine entries were made as training progressed, and of interval (CS vs pre-CS), which, as shown in the figure, demonstrates that more entries were made during the CS than during the pre-CS intervals (F(1,14) = 145.4, p < 0.01). There was also an interaction between day and interval, suggesting that the difference in magazine entries between the CS and pre-CS intervals increased with acquisition across days of training (F(9,126) = 13.7, p < 0.01). There was no effect of group (F(1,14) < 1), and no interactions involving group were significant.

Figure 4.

Figure 4.

Open in a new tab

A, Pavlovian conditioning. Mean number of magazine entries during the CS presentations and during the pre-CS intervals (±1 SEM) across days of Pavlovian training for the core and shell groups. Both groups learned to enter the magazine selectively during the stimulus periods. B, Instrumental conditioning. Mean lever-press responses for the core and shell groups across days of training (±1 SEM). For the first day, each response was reinforced. Thereafter, responding was reinforced on an RR-5 schedule of reinforcement for days 2–4, and on an RR-10 schedule of reinforcement for days 5–7. Rats in both groups acquired the instrumental responses and increased response rates across days.

Instrumental training

The rats acquired the lever-press responses similarly regardless of the outcomes used to reward performance (outcome effect, F values <1). The mean number of lever presses across days of training is presented in Figure 4B and illustrates that both groups acquired the instrumental responses. This description is supported by the ANOVA, which revealed no effect of group (F(1,14) < 1); a significant effect of training day, suggesting that responding increased across days (F(1,14) = 159.4, p < 0.01); and no interaction between day and group (F(1,14) < 1).

Effect of NAC core and shell inactivation on Pavlovian–instrumental transfer

Following inactivation of either the core or shell, the pattern of PIT effects was largely similar to those found with pretraining lesions. Following saline infusions, rats in both groups showed elevated responding following presentation of both the same and general stimuli. Following inactivation of the core, while the same stimulus still elevated responding relative to each baseline and the different stimulus, the effect of the general stimulus was diminished (Fig. 5B). The opposite was seen following inactivation of the shell (Fig. 5A); while the impact of the general stimulus remained intact, the specific excitatory effect of the same stimulus was reduced. This description is confirmed by the statistical analysis, which indicates an effect of inactivation (F(1,14) = 13.91, p < 0.01), an effect of stimulus (F(1,14) = 29.76, p < 0.01), and an interaction between these factors, suggesting that inactivation altered the stimulus effects (F(3,42) = 9.29, p < 0.01). While there was no overall effect of group (F(1,14) = 3.6, p > 0.05) and no two-way interactions with this factor (group × inactivation; F(1,14) < 1; group × stimulus; F(3,42) = 2.3, p > 0.05), there was a three-way interaction (group × inactivation × stimulus; F(3,42) = 3.44, p < 0.05), indicating that the effect of inactivation on the forms of PIT differed between groups. To explore the source of the interaction, we examined the effects of inactivation on PIT for each group. Analysis of the shell group indicated an effect of inactivation (F(1,7) = 6.8, p < 0.05), stimulus (F(3,21) = 16.37, p < 0.01), and an interaction between these factors (F(3,21) = 10.75, p < 0.01). Based on our previous results, we predicted that inactivation of the shell would reduce specific transfer while leaving general transfer intact. Post hoc analyses indicate that following saline infusion, each of the same (t(7) = 6.98, p < 0.001) and general (t(7) = 6.87, p < 0.001) stimuli elevated responding above baseline, and while different stimulus also increased responding somewhat (t(7) = 3.35, p < 0.05), the increase in response to the same stimulus was greater than that to the different stimulus (t(7) = 4.93, p < 0.01). The impact of the same and general stimuli was reduced by the inactivation treatment (same; t(7) = 4.8, p < 0.01; general; t(7) = 2.98, p < 0.05), and the pattern of stimulus effects was also changed. Following inactivation, all stimuli still elevated performance from the baseline intertrial interval (same; t(7) = 2.62, p < 0.05; different; t(7) = 2.5, p < 0.05; general; t(7) = 4.3, p < 0.01); however, the selectivity of the PIT effect was lost. That is, the same stimulus no longer had a greater effect than the different stimulus (t(7) = 1.0, p > 0.05). Of note, if we apply a more stringent criterion (Bonferroni correction) while under saline, each of the same and general stimuli still significantly increased responding from baseline; under inactivation, only the general stimulus effect was significant. Together, these data indicate that while an excitatory effect of reward-predictive stimuli remains following shell inactivation, an intact shell is required for the expression of outcome-specific stimulus effects.

Figure 5.

Figure 5.

Open in a new tab

Effects of reversible inactivation of the core or shell on the outcome-specific and general forms of Pavlovian–instrumental transfer. A, Mean lever presses (+SEM) during each stimulus and the baseline intervals in the PIT tests following saline and muscimol infusions for rats in the shell group. Following saline, there was evidence of both specific and general PIT. Following inactivation, while there was still an excitatory influence of the stimuli, the outcome-specific PIT effect was lost. B, Mean lever presses (+SEM) in the PIT tests following saline and muscimol infusions for rats in the core group. Following saline, there was evidence of both specific and general PIT. Following inactivation, while the same stimulus still elevated performance relative to both baseline and the different stimulus, providing evidence of outcome-specific PIT, the general excitatory influence of the stimuli was reduced. **Significant increase from baseline (p < 0.0063); *p < 0.05; ⋀responding during the “same” stimulus was greater than during the “different” stimulus (p < 0.05). C, Effects of shell and core inactivation on outcome-specific PIT. Baseline responses were subtracted from lever presses during the same and different stimuli to further evaluate the effects of inactivation on any excitatory effects. *Greater responding during the same compared to the different stimulus; **reduction from saline treatment and a significant group difference. D, Effects of shell and core inactivation on general PIT. Mean lever presses during the general stimulus with the baseline response rate subtracted. **Reduction from saline treatment and a significant group difference.

Analysis of the core group also revealed an effect of inactivation (F(1,7) = 7.14, p < 0.05), stimulus (F(3,21) = 15.7, p < 0.01), and an interaction between these factors (F(3,21) = 3.48, p < 0.05). Post hoc analyses show that following saline infusion, both the same (t(7) = 4.2, p < 0.01) and general (t(7) = 4.56, p < 0.01) but not different (t(7) = 2.2, p > 0.05) stimuli elevated responding from baseline. Evidence for a specific transfer effect was found by the greater responding to the same than different stimulus (t(7) = 3.1, p < 0.05). Following inactivation, while the same stimulus still elevated responding from baseline (t(7) = 2.7, p < 0.01) and was significantly higher than to the different stimulus (t(7) = 2.78, p < 0.05), the excitatory effect of the general stimulus was lost (t(7) = 0.13, p > 0.05).

Again, to facilitate between-group comparisons, we subtracted the baseline response rate for each group and evaluated the effects of inactivation on each outcome-specific (Fig. 5C) and general (Fig. 5D) PIT. While there were no differences between core and shell groups under saline conditions (F values <1), inactivation altered the effects of each stimulus. Following inactivation, the excitatory effect of the same stimulus was greater for the core than shell group (F(1,14) = 6.54, p < 0.05), whereas the effect of the different stimulus was further reduced by core inactivation (F(1,14) = 6.7, p < 0.05), confirming that the outcome-specific effects of the stimuli remained intact in core animals but were abolished by shell inactivation (Fig. 5C). In contrast, the effect of the third stimulus was greater in shell than in core rats following inactivation, suggesting that the core, not shell, is critical for this effect (F(1,14) = 21.9, p < 0.01).

Discussion

These results demonstrate that whereas the NAC plays an important role in the way reward-related cues influence instrumental performance, the core and shell regions serve distinct motivational functions in this regard. In sham rats, the stimuli that were predictive of outcomes that the rats also had earned through performance of an instrumental response produced outcome-specific PIT, selectively elevating responses paired with that same reward while leaving actions that earned a different reward unaffected. In addition, a third stimulus that was paired with a rewarding outcome that was not earned by either instrumental action had a general excitatory effect on performance, increasing pressing on both levers equally. Importantly, lesions or inactivation of the shell eliminated the outcome-specific PIT effect, consistent with our previous findings (Corbit et al., 2001). However, the general PIT effect produced by presentation of the third excitatory stimulus remained intact. In contrast, lesions or inactivation of the core had no effect on the outcome-specific PIT effect; i.e., stimuli paired with a reward that was also earned by the available instrumental action elevated performance of that response, again consistent with our previous finding (Corbit et al., 2001). However, both core lesions and inactivation eliminated the excitatory effect of the third, general stimulus, suggesting that the generally arousing impact of that stimulus was attenuated consistent with findings using a single-stimulus version of PIT task (e.g., Hall et al., 2001; Holland and Gallagher, 2003).

The double dissociation observed in the data above provides further empirical support for the suggestion that Pavlovian stimuli can influence instrumental behavior in at least two ways (cf. Dickinson and Balleine, 2002; Corbit and Balleine, 2005; Corbit et al., 2007). One of these involves the ability of Pavlovian cues to influence (in this case invigorate) instrumental performance indirectly by altering levels of arousal or behavioral activation. The other is more selective and involves the ability of CSs to activate the memory of the unique sensory properties of different outcomes or rewards used in instrumental conditioning and thus affect performance of responses also associated with delivery of that particular outcome, perhaps by producing the expectancy of the outcome (Trapold and Overmier, 1972). The unique effects of NAC core or shell lesions on these processes further suggest that these transfer mechanisms are mediated by different neural systems (cf. Corbit and Balleine, 2005).

These distinctions between general and outcome-specific transfer mechanisms and the neural systems that support them are also important because they help reconcile some conflicting reports in the literature regarding the role of the nucleus accumbens core and shell in PIT effects.

The general excitatory effect of the third stimulus in our current study may parallel the effects produced when a single excitatory stimulus is trained as in various other studies examining PIT (Hall et al., 2001; Holland and Gallagher, 2003), and the failure of the shell lesions to alter this effect as well as the deficit seen in animals with lesions of the core in the current study are both consistent with previous studies using that design (Hall et al., 2001; Holland and Gallagher, 2003) or in a related discriminative stimulus task (Ambroggi et al., 2011). The deficit seen in animals with core lesions reported by Hall et al. (2001) and lack of excitatory effect of the third, general stimulus in the current study together suggest that when a single excitatory stimulus is trained, the excitatory effects of this stimulus on instrumental performance may be quite general, although this issue has not been explicitly examined in the current study.

The current results compliment previous studies examining the role of the amygdala in controlling PIT effects where specific transfer has been argued to be mediated by the basolateral amygdala (BLA) (Blundell et al., 2001), and general transfer appears to be mediated by the amygdala central nucleus (CeN) (Hall et al., 2001; Holland and Gallagher, 2003). Indeed, using the three stimulus procedures described here, we have previously doubly dissociated the involvement of the BLA and CeN in these aspects of transfer, finding that BLA lesions abolished outcome-specific but spared general transfer, whereas lesions of the CeN abolished general transfer but spared the outcome-specific transfer effect (Corbit and Balleine, 2005). Together, these data suggest that distinct amygdalostriatal circuits mediate the action selection and general arousing influences of predictive stimuli on instrumental performance. However, it is important to note that, despite similar effects on specific PIT, the influences of BLA and NAC shell lesions on incentive motivation are not entirely parallel. Whereas lesions of the BLA disrupt both specific PIT and sensitivity to outcome devaluation (Corbit and Balleine, 2005), we have previously shown that lesions of the NAC shell have no effect on sensitivity to outcome devaluation (Corbit et al., 2001). Importantly, this distinction lends further support to the idea that the roles of predicted and experienced reward on responding are also dissociable.

While it is possible that the BLA serves a function common to both processes, thus accounting for the observed deficits in both PIT and devaluation, as the BLA projects to both the core and shell (McDonald, 1991; Brog et al., 1993), it is also possible that distinct outputs to these regions mediate these different motivational influences on action selection and performance. Recently, this has been confirmed using asymmetrical lesions of the BLA and accumbens core or shell (Shiflett and Balleine, 2010). This study found that whereas a unilateral BLA lesion combined with a lesion of contralateral shell abolished outcome-specific PIT, it did not influence outcome devaluation. Conversely, a unilateral BLA lesion combined with a lesion of contralateral core abolished outcome devaluation whist sparing outcome-specific PIT. Furthermore, electrophysiological recording during an instrumental discriminative stimulus paradigm has demonstrated that cue-related activity in the BLA precedes that seen in the NAC and that inactivation of the BLA reduces cue-evoked responding in the accumbens as well as behavioral responses elicited by the stimulus (Ambroggi et al., 2008; Jones et al., 2010), providing evidence for a functional role for the BLA–NAC projections. While these latter studies have not looked at the selectivity of these effects with regard to multiple outcomes or action choice, together with the results of Shiflett and Balleine (2010), they support the suggestion that a functional circuit exists between the amygdala and NAC involved in the online control of instrumental responding in the face of reward-predictive stimuli. Nevertheless, the role of such a circuit or circuits in the selective cuing versus the generally arousing influence of reward-predictive stimuli awaits further study.

In light of the excitatory effect of the general stimulus, it is interesting to consider why the different stimulus fails to excite responding despite having been paired previously with reward. Given that the general stimulus is able to enhance responding, failure of the different stimulus to have a similar effect must have to do with the fact that it predicts an outcome that is common to another behavioral response that is currently unavailable. Choice between actions depends not only on selecting the appropriate response but also on the ability to inhibit competing or inappropriate responses. The demonstration that lesions of the shell, not core, eliminate the specific PIT effect suggests that the shell (and potentially inputs from the amygdala) is important for this selective inhibitory process within instrumental conditioning. One hypothesis regarding the basal ganglia in general is that it produces tonic suppression of motor programs and that a process of disinhibition is required for action selection (Nicola, 2007). An example of neural inhibition in behavioral control comes from the observation that pharmacological inactivation of the shell can produce robust feeding, a result that has been taken as evidence that under normal circumstances, one function of the shell is to inhibit this behavior and that activity within the shell must be inhibited to allow feeding to proceed (Stratford and Kelley, 1997). In line with this idea, inhibition of neural activity in a population of NAC neurons has recently been reported to accompany sucrose consumption (Taha and Fields, 2005). Further, stimulation of the NAC produces a pause in licking behavior, again suggesting that activity within the NAC inhibits some behaviors and a pause in this activity is needed to allow consumption to proceed (Taha and Fields, 2005, 2006; Krause et al., 2010). It is possible that the NAC shell also contributes to inhibition of alternative responses, allowing Pavlovian or instrumental discrimination learning and that lesions or inactivation of this region interfere with this process and thus result in a loss of specific PIT effects. Indeed, a recent study using a discriminative stimulus task has found that inactivation of the core reduced responding to the rewarded cue, whereas inactivation of the shell reduced response selectivity by increasing responding to the nonrewarded cue and on the inactive lever (Ambroggi et al., 2011). Further study of the role of the NAC shell in inhibiting competing responses, such as to the different stimulus, is needed to elucidate the role of inhibition in producing specific PIT effects. Nonetheless, reports of loss of selective responding following inactivation of the shell are consistent with this interpretation.

The heterogeneous nature of the NAC has been well documented (see Zahm, 2000 for a review), and the core and shell regions have been dissociated in a variety of behavioral tasks. Utilization of stimulus-related information is one way in which specific actions can be selected from among competing alternatives. Stimuli carry information not only about the valence of a predicted outcome but also about the unique features of that outcome, and it is clear that the NAC contributes importantly to the way that both of these processes influence performance with the core controlling the general motivational effects of such stimuli and the shell directing choice based on the specific features of unique outcomes.

Footnotes

The research reported in this paper was supported by grants from the National Health and Medical Research Council (NHMRC #633267) and the National Institute of Mental Health (NIMH #56446) to B.W.B.

The authors declare no competing financial interests.

References

  1. Ambroggi F, Ishikawa A, Fields HL, Nicola SM. Basolateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons. Neuron. 2008;59:648–661. doi: 10.1016/j.neuron.2008.07.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ambroggi F, Ghazizadeh A, Nicola SM, Fields HL. Roles of nucleus accumbens core and shell in incentive-cue responding and behavioural inhibition. J Neurosci. 2011;31:6820–6830. doi: 10.1523/JNEUROSCI.6491-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Blundell P, Hall G, Killcross S. Lesions of the basolateral amygdala disrupt selective aspects of reinforcer representation in rats. J Neurosci. 2001;21:9018–9026. doi: 10.1523/JNEUROSCI.21-22-09018.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Brog JS, Salyapongse A, Deutch AY, Zahm DS. The patterns of afferent innervation of the core and shell in the “accumbens” part of the rat ventral striatum: immunohistochemical detection of retrogradely transported fluoro-gold. J Comp Neurol. 1993;338:255–278. doi: 10.1002/cne.903380209. [DOI] [PubMed] [Google Scholar]
  5. Corbit LH, Balleine BW. Instrumental and Pavlovian incentive processes have dissociable effects on components of a heterogeneous instrumental chain. J Exp Psychol Anim Behav Process. 2003;29:99–106. doi: 10.1037/0097-7403.29.2.99. [DOI] [PubMed] [Google Scholar]
  6. Corbit LH, Balleine BW. Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian-instrumental transfer. J Neurosci. 2005;25:962–970. doi: 10.1523/JNEUROSCI.4507-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Corbit LH, Muir JL, Balleine BW. The role of the nucleus accumbens in instrumental conditioning: evidence for a functional dissociation between core and shell. J Neurosci. 2001;21:3251–3260. doi: 10.1523/JNEUROSCI.21-09-03251.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Corbit LH, Janak PH, Balleine BW. General and outcome-specific forms of Pavlovian-instrumental transfer: the effect of shifts in motivational state and inactivation of the ventral tegmental area. Eur J Neurosci. 2007;26:3141–3149. doi: 10.1111/j.1460-9568.2007.05934.x. [DOI] [PubMed] [Google Scholar]
  9. Dickinson A, Balleine BW. The role of learning in motivation. In: Gallistel CR, editor. Steven's handbook of experimental psychology, Vol 3, Learning, motivation, and emotion. Ed 3. New York: Wiley; 2002. pp. 497–533. [Google Scholar]
  10. Estes WK. Discriminative conditioning. I. A discriminative property of conditioned anticipation. J Exp Psychol. 1943;32:150–155. [Google Scholar]
  11. Hall J, Parkinson JA, Connor TM, Dickinson A, Everitt BJ. Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behavior. Eur J Neurosci. 2001;13:1984–1992. doi: 10.1046/j.0953-816x.2001.01577.x. [DOI] [PubMed] [Google Scholar]
  12. Holland PC, Gallagher M. Double dissociation of the effects of lesion of basolateral and central amygdala on conditioned stimulus-potentiated feeding and Pavlovian-instrumental transfer. Eur J Neurosci. 2003;17:1680–1694. doi: 10.1046/j.1460-9568.2003.02585.x. [DOI] [PubMed] [Google Scholar]
  13. Jones JL, Day JJ, Wheeler RA, Carelli RM. The basolateral amygdala differentially regulates conditioned neural responses within the nucleus accumbens core and shell. Neuroscience. 2010;169:1186–1198. doi: 10.1016/j.neuroscience.2010.05.073. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Krause M, German PW, Taha SA, Fields HL. A pause in nucleus accumbens neuron firing is required to initiate and maintain feeding. J Neurosci. 2010;30:4746–4756. doi: 10.1523/JNEUROSCI.0197-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Lovibond PF. Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. J Exp Psychol Anim Behav Process. 1983;9:225–247. [PubMed] [Google Scholar]
  16. McDonald AJ. Topographical organization of amygdaloid projections to the caudatoputamen, nucleus accumbens, and related striatal-like areas of the rat brain. Neuroscience. 1991;44:15–33. doi: 10.1016/0306-4522(91)90248-m. [DOI] [PubMed] [Google Scholar]
  17. Nicola SM. The nucleus accumbens as part of a basal ganglia action selection circuit. Psychopharmacology (Berl) 2007;191:521–550. doi: 10.1007/s00213-006-0510-4. [DOI] [PubMed] [Google Scholar]
  18. Overmier JB, Lawry JA. Pavlovian conditioning and the mediation of behavior. In: Bower GH, editor. The psychology of learning and motivation. Vol 13. New York: Academic; 1979. pp. 1–55. [Google Scholar]
  19. Paxinos G, Watson C. The rat brain in stereotaxic coordinates. San Diego: Academic; 1998. [DOI] [PubMed] [Google Scholar]
  20. Rescorla RA, Solomon RL. Two-process learning theory: relationship between Pavlovian conditioning and instrumental learning. Psychol Rev. 1967;74:151–182. doi: 10.1037/h0024475. [DOI] [PubMed] [Google Scholar]
  21. Shiflett MW, Balleine BW. At the limbic-motor interface: disconnection of basolateral amygdala from nucleus accumbens core and shell reveals dissociable components of incentive motivation. Eur J Neurosci. 2010;32:1735–1743. doi: 10.1111/j.1460-9568.2010.07439.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Stratford TR, Kelley AE. GABA in the nucleus accumbens shell participates in the central regulation of feeding behavior. J Neurosci. 1997;17:4434–4440. doi: 10.1523/JNEUROSCI.17-11-04434.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Taha SA, Fields HL. Encoding of palatability and appetitive behaviors by distinct neuronal populations in the nucleus accumbens. J Neurosci. 2005;25:1193–1202. doi: 10.1523/JNEUROSCI.3975-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Taha SA, Fields HL. Inhibitions of nucleus accumbens neurons encode a gating signal for reward-directed behavior. J Neurosci. 2006;26:217–222. doi: 10.1523/JNEUROSCI.3227-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Trapold MA, Overmier JB. The second learning process in instrumental learning. In: Black AA, Prokasy WF, editors. Classical conditioning II: current research and theory. New York: Appleton-Century-Crofts; 1972. pp. 427–452. [Google Scholar]
  26. Zahm DS. An integrative neuroanatomical perspective on some subcortical substrates of adaptive responding with emphasis on the nucleus accumbens. Neurosci Biobehav Rev. 2000;24:85–105. doi: 10.1016/s0149-7634(99)00065-2. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES