Self-assembled InAs quantum dots (QDs) in an InAlGaAs matrix, lattice-matched to InP substrate, have been grown by molecular beam epitaxy (MBE). Transmission electron microscopy (TEM), double-crystal X-ray diffraction (DCXRD) and photoluminescence (PL) are used to study their structural and optical properties. In InAs/InAlGaAs/ InP system, we propose that when the thickness of InAs layer deposited is small, the random strain distribution of the matrix layer results in the formation of tadpole-shaped QDs with tails towards random directions, while the QDs begin to turn into dome-shaped and then coalesce to form islands with larger size and lower density to release the increasing misfit strain with the continuous deposition of InAs. XRD rocking curves showing the reduced strain with increasing thickness of InAs layer may also support our notion. The results of PL measurements are in well agreement with that of TEM images. (C) 2004 Elsevier B.V. All rights reserved.