The microstructure of solid source molecular beam epitaxy (MBE) lattice-matched GaInP-GaAs heterostructures has been studied by transmission electron microscopy (TEM). It is shown that atomic-scale roughening occurs in the first several (similar to five) interfaces, and beyond which roughening is developed into micrometer-scale. The {131} faceted roughening occurs in the GaInP-on-GaAs interfaces, leading to the formation of V-shaped grooves. Dislocation dipoles are observed in the top, facet surface and bottom regions of the grooves, and there is no obvious difference. For GaInP layers exhibiting atomic-scale roughening, a small number of dislocations, mostly 60 degrees -type and in a dipole configuration, are confined in a zone of approximately 5.0 nm in width along the interface, and dislocations are rare deep inside the layers. For GaInP layers, in contrast, there is a high density of dislocations in both the regions, and it has remarkable interfacial roughening. Based on their distribution and configurations, the dislocations are believed to result from the compositional modulation occurred in the GaInP layers. The interplay of roughening, dislocations and compositional modulation has been discussed in the light of the morphological features.