The assembly mechanism of Metal-Organic Frameworks (MOFs) is controlled by the choice of solvent and the presence of spectator ions. In this paper, we apply enhanced sampling molecular dynamics methods to investigate the role of solvent and ions in the early stages of the synthesis of MIL-101 (Cr). Microsecond-long well-tempered metadynamics simulations uncover a rich structural free energy landscape, with secondary building units (SBUs) adopting distinct crystal and noncrystal like configurations. In the presence of ions (Na+, F-), we observe a complex effect on the crystallinity of SBUs. By modulating the interactions between terephthalate linkers and Cr atoms, ions affect the abundance of crystal-like SBUs, consequently controlling the percentage of defects. Solvent effects are assessed by comparing water with N,N-dimethylformamide, in which SBU adducts are appreciably more stable and compact. These results shed light on how solvent and ionic strength impact the free energy of the assembly phenomena that ultimately control material synthesis.