Sequence polymorphisms affect gene expression by perturbing the complex network of regulatory interactions. We propose a probabilistic method, called Geronemo, which directly aims to identify the mechanism by which genetic changes perturb the regulatory network. Geronemo automatically constructs a set of coregulated genes (modules), whose regulation can involve both sequence variations and expression of regulators. By exploiting the modularity of genetic regulatory systems, Geronemo reveals regulatory relationships that are indiscernible when genes are considered in isolation, allowing the recovery of intricate combinatorial regulation. By incorporating both expression and genotype of regulators, Geronemo captures cases where the effect of sequence variation on its targets is indirect. We applied Geronemo to a data set from the progeny generated by a cross between laboratory BY4716 (BY) and wild RM11-1a (RM) isolates of Saccharomyces cerevisiae. Geronemo produced previously undescribed hypotheses regarding genetic perturbations in the yeast regulatory network, including transcriptional regulation, signal transduction, and chromatin modification. In particular, we find a large number of modules that have both chromosomal characteristics and are regulated by chromatin modification proteins. Indeed, a large fraction of the variance in the expression can be explained by a small number of markers associated with chromatin modifiers. Additional analysis reveals positive selection for sequence evolution of elements in the Swi/Snf chromatin remodeling complex. Overall, our results suggest that a significant part of individual expression variation in yeast arises from evolution of a small number of chromatin structure modifiers.