The Role of the Nucleus Accumbens in Instrumental Conditioning: Evidence of a Functional Dissociation between Accumbens Core and Shell

Subjects and apparatus

The subjects were 24 experimentally naïve female Long–Evans rats. The rats were housed singly and were handled daily for 1 week before surgery. Training and testing took place in 16 Med Associates (East Fairfield, VT) operant chambers housed within sound- and light-resistant shells. Each chamber was equipped with a pump fitted with a syringe that delivered 0.1 ml of a 20% sucrose solution into a recessed magazine in the chamber. Each chamber was also equipped with a pellet dispenser that delivered one 45 mg Noyes pellet (formula A/I) when activated. The chambers contained two retractable levers that could be inserted to the left and right of the magazine. A 3 W, 24 V house light mounted on the top-center of the wall opposite the magazine provided illumination. Microcomputers equipped with the MED-PC program (Med Associates) controlled the equipment and recorded the lever presses.

Surgery

At the time of surgery, animals weighed between 266 and 338 gm. There were three surgical groups; rats received cell body lesions of either the core or shell region of the nucleus accumbens or sham surgery. Rats were anesthetized using sodium pentobarbital (Nembutal, 50 mg/kg), treated with atropine (0.1 mg), and then placed in a stereotaxic frame (Stoelting Company, Wood Dale, IL) with the incisor bar set at −3.3 mm. The scalp was retracted to expose the skull, and small burr holes were drilled above the target regions. For lesions of the core (n = 7), animals received bilateral injections of 0.5 μl of 0.12 m NMDA in two sites (one per side) using a 1 μl Hamilton syringe (all coordinates in millimeters relative to bregma; anteroposterior, +1.2; mediolateral, ±2.1; and dorsoventral, −7.0). Each injection was made over 2 min and allowed to diffuse for an additional 2 min before removal of the needle. For lesions of the shell region (n = 9), animals received 0.2 μl injections of 0.015m AMPA hydrobromide at four sites (two per side, anteroposterior, +1.6; mediolateral, ±0.8; and dorsoventral, −6.8, −6.0). Injections were again made over 2 min with an additional 2 min allowed before any movement of the needle. Animals in the surgical control group (n = 8) underwent similar treatment except that no neurotoxin was injected.

Histology

At the end of the experiment, the animals were killed using a lethal barbiturate overdose and perfused transcardially with 0.9% saline followed by 10% formaldehyde solution. The brains were stored in 10% formalin solution for at least 48 hr and then transferred to a 25% sucrose–formalin solution before 40 μm coronal sections were cut throughout the region of the nucleus accumbens. Alternate slices were stained using thionin. Slides were examined for placement and extent of the lesion; the latter was assessed by microscopically examining sections for areas of marked cell loss as well as general shrinkage of a region relative to sham controls.

Procedure

Training. After recovery from surgery, subjects were placed on a food deprivation schedule such that they received 15 gm of their maintenance diet daily to maintain them at ∼85% of their free-feeding weight. The animals were fed each day after the training sessions. Animals were provided tap water ad libitumwhile in the home cage. Each session started with the illumination of the house light and insertion of the levers where appropriate and ended with the retraction of the levers and turning off of the house light. All sessions were 30 min in duration unless otherwise stated.

Magazine training. Initially, all subjects received two sessions of magazine training in which the pellet and sucrose outcomes were delivered on independent random time (RT) 60 sec schedules with the levers withdrawn.

Lever training. The animals were next trained on random ratio (RR) schedules of reinforcement. Each lever was trained separately, and for half of the animals in each group, the left lever earned pellets and the right lever earned the sucrose solution. The remaining animals received the opposite action–outcome pairings. The animals first received 1 d of continuous reinforcement and were then shifted to an RR-5 schedule (i.e., each action delivered an outcome with a probability of 0.2). After 3 d of training this was changed to an RR-10 schedule (or a probability of 0.1) for 3 d and then to an RR-20 schedule (or a probability of 0.05) for an additional 3 d of training. The animals received two training sessions each day, one with each action–outcome pair. The animals had a break of at least 1 hr between sessions, and the order was alternated each day.

Devaluation extinction tests. After the final day of RR-20 training, all of the rats were given access ad libitum to one of the two outcomes for 1 hr in the home cage. Half of the animals in each action–outcome pair assignment received pellets (50 gm placed in a bowl in the home cage), and the remaining animals received sucrose (50 ml in a drinking bottle fixed to the front of the home cage). Immediately after the prefeeding, the animals were placed in the operant chambers. A 10 min choice extinction test was then conducted in which both levers were extended and the number of presses was counted on each lever. No outcomes were delivered during the test. After the first devaluation test, the animals received 1 d of retraining (RR-20; one session for each action–outcome pair) and were then given a second devaluation test on the following day. The second test was identical to the first except that those animals that had had pellets devalued previously now had sucrose devalued and those that had had sucrose devalued were now prefed pellets.

Reward test. After the second devaluation test conducted in extinction, the animals were retrained (RR-20; one 30 min session for each action–outcome pair) and on the following day tested for their sensitivity to the devaluation manipulation when performance was rewarded. This test was conducted in the same manner as the extinction test except that the outcomes were delivered as a consequence of instrumental performance. In this 20 min session, the two outcomes were delivered according to independent ratio schedules (RR-20).

Subjects and apparatus

The subjects and general apparatus are as described for experiment 1.

Procedure

Contingency degradation training. After the devaluation tests, the rats received 2 d of retraining on RR-20 schedules before the selective degradation of one of the instrumental contingencies. At the end of training, each lever press earned a unique outcome (pellets or a 20% sucrose solution) with a fixed probability [p(outcome/action) = 0.05]. The rats continued to be trained on the two actions with the appropriate paired outcomes, but in addition to being earned by one of the actions, one of the outcomes was now also delivered noncontingently with the same probability [p(outcome/no action) = 0.05] in each secondwithout a response. For one lever, the noncontingent or free reinforcer was the same as that earned by a response on that lever. Thus, the experienced probability of the delivery of that particular outcome was the same whether or not the animals performed that action, a procedure that should act to degrade that specific action–outcome contingency. For the other lever, the free reinforcer was different from the earned reinforcer, and so this contingency was not degraded. For half of the animals the lever press–pellet contingency was degraded, whereas for the remainder the lever press–sucrose contingency was degraded. The rats were given two 20 min training sessions each day, one on each lever with a break of ∼2 hr between sessions. The order of the sessions was alternated each day and training continued for 8 d.

Contingency degradation extinction test. On the day after the final day of contingency training, rats in both groups received a choice extinction test. The test began with the insertion of both levers and the onset of the house light and ended 10 min later with the retraction of the levers and the offset of the house light. No outcomes were presented during this session.

Subjects and apparatus

The subjects and apparatus are as described for experiment 1.

Procedure

Pavlovian training. Initially the animals were retrained for 3 d on RR-20 schedules after the contingency extinction test. Next the animals received eight sessions of Pavlovian conditioning. Two 80 dB auditory stimuli (tone and white noise) served as conditioned stimuli (CSs) and were paired with either pellet or sucrose delivery. For half of each lesion condition, the tone was paired with pellet delivery, and the noise was paired with sucrose delivery. The remaining half received the reverse pairings. Four presentations of each stimulus were given in each session in random order interspersed with periods in which no stimuli were present. The length of the intertrial intervals varied, but on average these intervals were 5 min. The stimuli presentations were 2 min long during which the appropriate outcome was delivered on an RT 30 sec schedule. The number of magazine entries during the stimuli as well as in a prestimulus interval of equal length (2 min) was measured. After 8 d of stimulus training, the animals were retrained with the levers for 1 d on the RR-20 schedule before testing.

Pavlovian-instrumental transfer test. The animals received two extinction tests (one on each lever) 1 d apart. During each test one of the levers was available, and each stimulus was presented four times interspersed with intervals of no stimulus (Ø). Each test was 32 min in duration. In the first 8 min the levers were available, but no stimuli were presented. This period was followed by 12 bins of 2 min each and contained a total of eight stimulus trials [four tone trials (T) and four noise trials (N) intermixed with four Ø trials in the following order: Ø, T, N, Ø, N, T, Ø, N, T, Ø, T, and N].

Histology

No recovery problem or weight loss was observed after surgery. Figure 1 displays the maximum and minimum damage resulting from the lesions for the animals included in the behavioral analyses of the core (A) and shell (B) based on the stereotaxic atlas of the rat brain by Paxinos and Watson (1998). Any animals with unilateral damage or damage outside the target region were excluded from the behavioral analyses. Figure 2 displays photomicrographs taken of representative lesions of the sham (right images), core (top images), and shell (bottom images) groups. The photographs on the right in Figure 2 show magnifications of the lesions shown in the images on the left. Magnifications of the sham brain in the region of the core and shell are shown for comparison in the top right and bottom right images, respectively, of Figure 2. Core lesions (Fig. 2, top images) resulted in substantial neuronal loss in the core region bilaterally and typically extended in the anteroposterior direction from 0.7 to 1.7 mm anterior to bregma. Generally the lesion did not extend ventrally to the ventral pallidum or dorsally to the caudate-putamen. Any animals with marked damage to the NAC shell were excluded from the behavioral analysis. NAC shell lesions (Fig. 2,bottom images) destroyed neurons in the mediodorsal shell bilaterally and typically extended in the anteroposterior direction from 1.0 to 1.7 mm anterior to bregma. The ventral and ventrolateral portions of the NAC shell appeared unaffected by the lesions. Any animals with substantial damage of the NAC core or any other surrounding structures outside the shell were excluded.

Training

As shown in Figure 3, animals from each group acquired the lever-press response for the two outcomes, and their response rates increased as the ratio schedule parameter increased across days. Nevertheless, the groups clearly diverged in their performance during training; the core-lesioned rats performed at a generally lower rate and the shell-lesioned rats performed at a slightly higher rate than did the sham-lesioned rats. Initial statistical analysis revealed that there was no effect of outcome type (F < 1), and so the data were collapsed across outcome for presentation and subsequent analysis. ANOVA revealed a significant effect of training day [F_(8,16) = 13.205; p< 0.01]. Also, as indicated by Figure 3, a significant effect of group confirms that the groups responded at different rates [F_(2,21) = 6.907; p< 0.01]. Additionally there was a marginal day × group interaction suggesting that, at least numerically, the increase in responding across days differed between the three groups [F_(16,168) = 1.655; p= 0.06]. Tukey pairwise comparisons conducted on the mean rate of performance during training revealed that the core-lesioned group responded at a reliably lower rate than did both the sham- and the shell-lesioned groups (p < 0.01). The numerical difference observed between the shell- and sham-lesioned groups failed to achieve significance (p > 0.05).

Devaluation extinction tests

Mean lever presses per minute during the choice extinction test are presented in Figure 4 collapsed across the two tests. Inspection of Figure 4 reveals a clear devaluation effect in both the sham- and shell-lesioned rats. However, the core animals show a very different pattern of responding, decreasing responding on both levers. A two-way ANOVA conducted using factors of group, separating the three lesion conditions, and of devaluation, separating performance on the lever that, in training, delivered the now-devalued outcome from responding on the other lever, revealed a significant main effect of group [F_(2,21) = 6.223; p < 0.01], an effect of devaluation [F_(1,21) = 26.696; p< 0.01], and more important, a significant group × devaluation interaction [F_(1,21) = 4.821; p < 0.05]. Simple effects analyses revealed that, although a significant effect of devaluation emerged in both the sham-lesioned [F_(1,21) = 24.581;p < 0.01] and shell-lesioned [F_(1,21) = 13.244; p< 0.01] rats, core-lesioned rats did not respond differently on the two levers; thus whereas the devaluation manipulation clearly affects their lever-press performance, the core-lesioned animals fail to show a selective devaluation effect [F_(1,21) < 1].

Reward test

The failure of core-lesioned rats to show a selective devaluation effect could be explained in several ways. It could be that the core region of the accumbens is critical for encoding the value of different outcomes, or alternatively, it could be that the memory of the two outcomes is impaired. If animals are unable to differentiate the two outcomes in extinction, then devaluation of one outcome may generalize to the other resulting in a decrease in performance of both responses. To assess whether the deficit observed in the extinction test was caused by a failure to discriminate the outcomes in extinction rather than in devaluation per se, the animals were retrained and tested again for their sensitivity to outcome devaluation but this time with the two outcomes being delivered. Because the two outcomes are delivered in this test, any deficit observed in the core group cannot be attributed to failure to remember and distinguish the two outcomes but, rather, would suggest a deficit in the instrumental incentive processes that mediate outcome devaluation. The results of this test are presented in the right panels of Figure 4. Generally, the pattern of results in the reward test was similar to those results observed in the extinction test; sham- and shell-lesioned rats showed strong selective devaluation effects, whereas core-lesioned rats failed to show any selective effect of the devaluation treatment. The statistical analysis revealed significant main effects of devaluation [F_(1,21) = 9.752;p < 0.01] and group [F_(2,21) = 5.205; p< 0.05], and again, simple effects analyses found that, although the devaluation effect was reliable in the sham-lesioned [F_(1,21) = 9.447; p< 0.01] and shell-lesioned [F_(1,21)= 6.171; p < 0.01] rats, even when the outcomes were delivered, the core-lesioned rats failed to show a selective devaluation effect [F_(1,21) < 1] and performed very few responses on either lever. This suggests that the deficit in the core animals, observed both when tested in extinction and when performance is rewarded, is caused by a failure of the current incentive values of the instrumental outcomes to control performance selectively. Clearly the incentive value of the nondevalued outcome was unable to maintain instrumental responding in the core-lesioned group.

In addition to leaning the value of the instrumental outcome, considerable evidence suggests that, in instrumental conditioning, rats can encode the specific action–outcome relations to which they are exposed during training. Furthermore, numerous findings suggest that, in intact animals, instrumental performance is sensitive to the contingency or causal relationship between performance of an action and delivery of its specific outcome (Balleine and Dickinson, 1998a; Corbit and Balleine, 2000).

With this in mind, one possible explanation of the low responding observed in core-lesioned animals both in training and particularly after outcome devaluation in experiment 1 is that these lesions render rats unable to encode specific action–outcome associations, and as such, changes in the value of one outcome produced a general rather than a selective reduction in lever-press performance on the test. This possibility is examined in experiment 2.

Contingency degradation training

The left panels of Figure5 depict the effects of contingency degradation training across days. ANOVA revealed a significant main effect of group [F_(2,21) = 6.107;p < 0.01], suggesting that, as in experiment 1, rates of responding differed between the three groups and suggesting an effect of day [F_(7,147) = 2.996;p < 0.01] and, importantly, an effect of contingency [F_(1,21) = 9.117; p< 0.01] indicating that, overall, rats reduced responding on the lever for which the contingency had been degraded. Furthermore, although a reliable day × contingency interaction emerged [F_(7,147) = 3.434; p< 0.01], indicating that the contingency effect increased across days of training, no other interactions were significant (F< 1), confirming that a selective effect of contingency degradation emerged in each of the lesion groups. Importantly, although performance in the core-lesioned rats was lower than that in the sham- and shell-lesioned groups, they showed an effect of contingency degradation across days, an effect that persisted in the final extinction test (Fig. 5, right panels). In general, therefore, it appears that all three groups were sensitive to degradation of the action–outcome contingency and selectively reduced performance accordingly.

Contingency extinction test

As indicated by the right panels of Figure 5, the effect of contingency degradation found in training persisted in the test conducted in extinction; i.e., the rats in each group performed fewer responses on the lever for which the instrumental contingency had been degraded. ANOVA found a significant effect of group [F_(2,27) = 3.664; p< 0.05] and of contingency [F_(1,21)= 20.661; p < 0.01], but, again, the group × contingency interaction was not reliable [F_(2,21) = 1.123; p> 0.05]. This finding adds additional weight to the argument that, although the overall rate of responding differed between the lesioned groups, degradation of one instrumental contingency was effective in each of the groups. As such, it is clear that neither core nor shell lesions affected the ability of the rats to encode the specific action–outcome associations to which they were exposed during training. These data suggest that the effects of core lesions on the devaluation tests in experiment 1 are unlikely to have been secondary to a deficit in instrumental learning produced by the lesion.

Pavlovian training

To assess whether animals learned about the relationship between the stimuli and food presentations, the number of magazine entries during the stimuli was compared with the number of entries in the prestimulus interval. The training data are displayed in Figure6, inspection of which suggests that Pavlovian training was similar in the three groups. Preliminary analysis suggested that there was no effect of stimulus type (noise vs tone; F < 1), and so the data are presented collapsed across stimulus. ANOVA revealed a significant effect of interval (stimulus vs prestimulus) [F_(1,20) = 127.3; p < 0.01], confirming that the rats made more magazine entries during the stimuli than in the prestimulus intervals, and a significant effect of training day [F_(7,140) = 8.765; p< 0.01], suggesting that as training proceeded the animals entered the magazine more during the stimuli. There was also a significant interval × day interaction [F_(7,140) = 4.690; p< 0.01], indicating that the difference between stimulus and prestimulus intervals increased over days. There was, however, no effect of group, and none of the interactions involving groups approached significance (all F values < 1). There is no evidence in this experiment, therefore, that the acquisition of Pavlovian conditioning was affected by lesions of either the NAC core or shell.

Pavlovian-instrumental transfer tests

The objective of this experiment was to assess the impact of Pavlovian cues for reward on instrumental performance, and so for the purpose of analysis, the number of lever presses during the baseline or no-stimulus (Ø) period was subtracted from the number of lever presses performed during each of the stimuli. Positive transfer is indicated, using this measure, by positive numbers, whereas reduced transfer should be reflected as numbers close to zero. The data from the transfer tests are displayed in Figure7.

It is clear from Figure 7 that a strong and selective positive transfer effect emerged in both the sham- and core-lesioned groups with responding on the lever that, in training, delivered the same outcome signaled by the stimulus being elevated more than responding on the other lever. In contrast, no positive transfer emerged in the shell-lesioned group. The statistical analysis confirmed this description; analysis of the test data revealed no effect of group [F_(2,20) = 2.568; p> 0.05] but did find a significant effect of stimulus (i.e., same vs different) [F_(1,2) = 13.379;p < 0.01] and, critically, a significant stimulus × group interaction [F_(2,20) = 6.257; p< 0.01]. Simple effects analyses revealed that, whereas there was a significant effect of stimulus in both the sham-lesioned [F_(1,20) = 22.154; p< 0.01] and core-lesioned [F_(1,20)= 3.288; p < 0.05] groups, there was no stimulus effect in the shell-lesioned animals [F_(1,20) < 1].

These data strongly oppose the suggestion that outcome devaluation effects can be explained in terms of the excitatory impact of Pavlovian signals for reward on performance and suggest that the effects of core lesions induced in experiment 1 were not produced by any deficit in either Pavlovian conditioning or the excitatory impact of Pavlovian cues on instrumental performance. This conclusion is further supported by the striking deficit in transfer induced by shell lesions, a lesion that was found in experiment 1 to have no impact whatever on the sensitivity of the instrumental performance of the rat to the effects of outcome devaluation.