We present two-dimensional Fourier transform optical spectroscopy measurements of two types of molecular J-aggregate thin films and show that temperature-dependent dynamical effects govern exciton delocalization at all temperatures, even in the presence of significant inhomogeneity. Our results indicate that in the tested molecular aggregates, even when the static structure disorder dominates exciton dephasing dynamics, the extent of exciton delocalization may be limited by dynamical fluctuations, mainly exciton-phonon coupling. Thus inhomogeneous dephasing may mediate the exciton coherence time whereas dynamical fluctuations mediate the exciton coherence length.