Four types of carbon black (CBN330, CBN550, CBN660 and CBN990) with increased particle size were chosen as the reinforcing filler for magnetorheological elastomers (MREs) preparation. The size effect of CB on the structural and mechanical property of MREs was investigated experimentally. Fourier transform infrared spectroscopy was adopted to characterize the structure of CB and MREs. Mooney viscosity, vulcanization curve, thermogravimetric analysis and mechanical property test were conducted to obtain the processing and vulcanization characteristics, thermal stability and mechanical parameters of MREs. It showed that MRE-CBN660 possessed a relatively better overall performance. Furthermore, the mechanical model of MREs and adsorption property of CB on carbonyl iron were studied theoretically. The theoretical results were coincidence with the experimental results, which interpreted the size effect of CB essentially.