Chimeric antigen receptors (CARs) targeting CD19 have mediated dramatic antitumor responses in hematologic malignancies, but tumor regression has rarely occurred using CARs targeting other antigens. It remains unknown whether the impressive effects of CD19 CARs relate to greater susceptibility of hematologic malignancies to CAR therapies, or superior functionality of the CD19 CAR itself. We show that tonic CAR CD3-ζ phosphorylation, triggered by antigen-independent clustering of CAR single-chain variable fragments, can induce early exhaustion of CAR T cells that limits antitumor efficacy. Such activation is present to varying degrees in all CARs studied, except the highly effective CD19 CAR. We further determine that CD28 costimulation augments, whereas 4-1BB costimulation reduces, exhaustion induced by persistent CAR signaling. Our results provide biological explanations for the antitumor effects of CD19 CARs and for the observations that CD19 CAR T cells incorporating the 4-1BB costimulatory domain are more persistent than those incorporating CD28 in clinical trials.