Clues on the coding of reward cues by the nucleus accumbens

The study by Calipari et al. in PNAS provides a glimpse into the in vivo pattern of neuronal activation in the mouse nucleus accumbens during the acquisition and expression of learned reward–context associations (1). The nucleus accumbens sits in the ventral part of the mouse striatum, and is required for reward-motivated learning. The activity of neurons in this region is thought to code information on the reward value of cues and contexts. Two distinct subpopulations of striatal neurons can be identified based on their anatomical projections and the degree of expression of dopamine D1 and D2 receptors. Striatal neurons expressing D1 receptors (D1-expressing neurons) have been shown to enhance reward learning, whereas striatal neurons expressing D2 receptors (D2-expressing neurons) have been shown to reduce reward learning when activated via optogenetic stimulation (2, 3). These manipulations, together with decades of experimentation in the field, have led to consensus around a general model in which the outputs of D1-expressing neurons and D2-expressing neurons exert opposing actions on coding of reward value and where the balance of activity between these outputs determines the final state (Fig. 1A). Based on this conceptual framework, predictions can be made about the activity of D1- and D2-expressing neurons during reward learning. However, recording the activity of the striatal neurons in a cell-specific manner has been very difficult. To this end, Calipari et al. (1) introduce the genetically coded calcium indicator GCaMP6 selectively in D1- or D2-expressing neurons of the nucleus accumbens, and use fiber photometry to monitor the calcium signals generated in this cell-specific fashion during the acquisition of condition place preference for cocaine. It is important to stress that although each detected calcium signal is generated by the firing of one or more action potentials in the neuron, not every action potential appears to lead to a detectable calcium signal. The evidence for this arises from the fact that the frequency of recorded calcium signals (5–25 events per minute, see also ref. 4) is at least an order-of-magnitude lower than the frequency of action potentials recorded with implanted electrodes in the nucleus accumbens (5).

That said, Calipari et al. (1) make several significant observations that directly test core predictions from the general model. When animals are placed in a context with little or no rewarding value, the frequency of calcium signals is larger in D2-expressing neurons than D1-expressing neurons. Noncontingent administration of cocaine produces opposite changes on the frequency of the calcium events in the two subpopulations of striatal neurons. During these conditioning sessions, cocaine decreases the frequency of calcium signals in D2-expressing neurons while increasing the frequency in D1-expressing neurons, likely through the well-described enhancement of dopamine extracellular concentration by cocaine. Interestingly, the study also shows that only a decrease in the frequency of events is detected under conditions in which the calcium indicator is expressed in a noncell-specific manner and signals are measured indiscriminately from both subpopulations of striatal neurons. This is probably because of the low frequency of transients in D1-expressing neurons relative to D2-expressing neurons at baseline, which would minimize the overall impact of the cocaine-induced increase in frequency of D1-expressing neurons that is masked by the cocaine-dependent reduction in frequency in the dominant D2-expressing neurons. However, independent of the reasons, these parallel experiments are highly valuable because once again they highlight the critical importance of cell-specific measurements and manipulations in getting a more complete understanding of the coding of reward information by the mammalian brain.

During the choice test in the Calipari et al. (1) experiment, animals received no drug and they were given access to the compartment that was previously paired with saline and the compartment previously paired with cocaine. Animals freely roamed between the compartments and the relative time spent in the cocaine context was used as a measure of the acquired preference for cocaine. Despite the fact that there was no drug on board, when mice were in the cocaine-paired context, the pattern of calcium signals in D1- and D2-expressing neurons resembled the pattern observed when they were under the influence of cocaine: lower frequencies in D2-expressing neurons and higher frequencies in D1-expressing neurons compared with the saline-paired environment. This is an interesting observation because it suggests that the reward–context association is coded, in part, in the nucleus accumbens by assigning the reward representation to the context. In other words, after learning the association between cocaine and the environment, it is as if mice experience a cocaine-like state when in the environment previously paired with the drug, even though at the time they have not received the drug. This finding might be analogous to the transference that occurs from the unconditional to the conditional stimuli observed in the firing of dopamine neurons and dopamine release following Pavlovian conditioning (6, 7).

The temporal pattern of calcium signals during the choice test is also different in D1- and D2-expressing neurons. D1-expressing neurons show a large transient as mice approach the cocaine context, approximately 1 s before they cross into the context, whereas the reduction in calcium signaling in D2-expressing neurons only develops once the animal enters the context. These results provide evidence that D1- and D2-expressing neurons code different aspects of the reward/value representation.

The timing of this large signal from D1-expressing neurons that precedes the entry in the cocaine-paired context is quite reproducible and, as a consequence, is preserved when multiple entries are aligned and averaged. Importantly, the amplitude of the preceding calcium signal in D1-expressing neurons is correlated with the overall time spent in the cocaine context, and as such correlated with the degree of preference for cocaine that each animal developed. No such correlation was found between the preference and the magnitude of the reduction of signals in D2-expressing neurons. Calipari et al. (1) wisely interpret these results as evidence that high activation of D1-expressing neurons in the nucleus accumbens is largely responsible for coding the cocaine–context association. The study goes on to show that a history of cocaine exposure impairs the extinction of the preference, and it also enhances the magnitude of the preceding calcium signals in D1-expressing neurons. More interesting findings follow from experiments using chemogenetic manipulation of D1 neuron calcium signals.

There are a few technical considerations to illuminate the discussion of these novel findings. In fiber photometry methods, the fiber optic is implanted in the brain and used to both transmit the excitation light and collect the fluorescence emission from the calcium indicator. The fluorescence emitted at any given time from a single neuron or multiple neurons is combined into a single fiber output and collectively form the integrated fluorescent signal (no image of the neurons is formed) (Fig. 1C). This method has multiple advantages over fluorescence imaging techniques that require the implantation of a much larger probe in the brain, which leads to more tissue damage and could, as a result, affect the behavior. In addition, the signal integration of the photometry approach provides significant amplification and benefits fluorescence detection. As depicted in Fig. 1C, synchronized signals become amplified by the integration process compared with unsynchronized signals from the same number of neurons. The effect becomes even more pronounced under conditions of high fluctuations in the baseline fluorescence, in which case the unsynchronized signals can become undetectable from the “noise.” As a consequence, this technique, like many other techniques, has a detection bias for synchronized signals, which are able to summate and be amplified (Fig. 1C).