In networks of excitatory and inhibitory neurons with mutual synaptic coupling, specific drive to sub-ensembles of cells often

leads to gamma-frequency (25–100 Hz) oscillations. When the number of driven cells is too small, however, the synaptic

interactions may not be strong or homogeneous enough to support the mechanism underlying the rhythm. Using a combination of

computational simulation and mathematical analysis, we study the breakdown of gamma rhythms as the driven ensembles become too

small, or the synaptic interactions become too weak and heterogeneous. Heterogeneities in drives or synaptic strengths play an

important role in the breakdown of the rhythms; nonetheless, we find that the analysis of homogeneous networks yields insight

into the breakdown of rhythms in heterogeneous networks. In particular, if parameter values are such that in a homogeneous

network, it takes several gamma cycles to converge to synchrony, then in a similar, but realistically heterogeneous network,

synchrony breaks down altogether. This leads to the surprising conclusion that in a network with realistic heterogeneity, gamma

rhythms based on the interaction of excitatory and inhibitory cell populations must arise either rapidly, or not at all. For

given synaptic strengths and heterogeneities, there is a (soft) lower bound on the possible number of cells in an ensemble

oscillating at gamma frequency, based simply on the requirement that synaptic interactions between the two cell populations be

strong enough. This observation suggests explanations for recent experimental results concerning the modulation of gamma

oscillations in macaque primary visual cortex by varying spatial stimulus size or attention level, and for our own experimental

results, reported here, concerning the optogenetic modulation of gamma oscillations in kainate-activated hippocampal slices. We

make specific predictions about the behavior of pyramidal cells and fast-spiking interneurons in these experiments.