Multiple studies have documented an inverse relationship between the number of to-be-attended or remembered items in a display ("set size") and task performance. The neural source of this decline in cognitive performance is currently under debate. Here, we used a combination of fMRI and a forward encoding model of orientation selectivity to generate population tuning functions for each of two stimuli while human observers attended either one or both items. We observed (1) clear population tuning functions for the attended item(s) that peaked at the stimulus orientation and decreased monotonically as the angular distance from this orientation increased, (2) a set-size-dependent decline in the relative precision of orientation-specific population responses, such that attending two items yielded a decline in selectivity of the population tuning function for each item, and (3) that the magnitude of the loss of precision in population tuning functions predicted individual differences in the behavioral cost of attending an additional item. These findings demonstrate that attending multiple items degrades the precision of perceptual representations for the target items and provides a straightforward account for the associated impairments in visually guided behavior.