The microstructure of fully dense hot-pressed ultra-high-temperature ceramics (UHTCs), namely ZrB2 and HfB2 containing 15 vol% of TaSi2, was characterized by X-ray diffraction, scanning electron microscopy (SEM), and transmission electron microscopy (TEM). ZrB2 and HfB2 grains displayed a core-shell structure: the core was constituted by the original MB2 grain and the shell by a (M, Ta)B-2 solid solution, which grew epitaxially on the core. The compositional misfit between core-shell was accommodated by low-angle grain boundaries and dislocations pile-up, especially pronounced in the ZrB2-based composite, where a higher amount of Ta entered the boride lattice. Ta5Si3, Ta4.8Si3C0.3, and Ta5SiB2, with Zr or Hf impurities, were detected at the triple points and wetting of the grain boundaries by a Ta-Si-B-C-O amorphous phase was observed. Based on the new microstructural features detected by TEM, thermodynamic calculations and the available phase diagrams, a densification mechanism for ZrB2 and HfB2 with addition of TaSi2 is proposed. The microstructures of the UHTC composites presented here are compared with composites sintered with the addition of MoSi2 in the same amount.