A linear dynamics scheme has been used to quantify the impact of viscoelasticity of the suspending fluid on the collective structure of active particles, including rotational diffusivity. The linear stability examines the response near an isotropic state using a mean-field theory including far-field hydrodynamic interactions of the swimmers. The kinetic model uses three possible constitutive models, the Oldroyd-B, Maxwell, and generalized linear viscoelastic models inspired by fluids like saliva, mucus, and biological gels. The perturbation growth rate has been quantified in terms of wavenumber, translational diffusivity, rotational diffusivity, and material properties of the fluids. A key dimensionless group is the Deborah number, which compares the relaxation time of the fluid with the characteristic timescale of the instability. An advantage of the model formalism is the ability to calculate some properties analytically and others efficiently numerically in the presence of rotational diffusion. The different constitutive equations examined help illustrate when and why the dispersion relation can have a peak at a particular wavenumber. The fluid properties can also change the role of rotational diffusion; diffusion always stabilizes a system in a Newtonian fluid but can destabilize a system in a Maxwell fluid. (C) 2013 The Society of Rheology. [http://dx.doi.org/10.1122/1.4778578]