The origin of size effects in the thermal conductivity and diffusivity of nanostructural semiconductors was investigated through the establishment of a unified nanothermodynamic model. The contributions of size-dependent heat capacity and cohesive energy as well as the interface scattering effects were considered during the modeling. The results indicate the following: (1) both the thermal conductivity and diffusivity decrease with decreasing nanocrystal sizes (x) of Si and Si/SiGe nanowires, Si thin films and Si/Ge(SiGe) superlattices, and GaAs/AlAs superlattices when x > 20 nm; (2) the heat transport in semiconductor nanocrystals is determined largely by the increase of the surface (interface)/volume ratio; (3) the interface scattering effect predominates in the reduction of thermal conductivity and diffusivity while the intrinsic size effects on average phonon velocity and phonon mean free path are also critical; (4) the quantum size effect plays a crucial role in the enhancement of the thermal conductivity with a decreasing x (<20 nm). These findings provide new insights into the fundamental understanding of high-performance nanostructural semiconductors toward application in optoelectronic and thermoelectric devices.