The natural contradiction in enhancing electrical conductivity and thermopower in thermoelectric oxides makes it hard to improve the performance of a single thermoelectric oxide material. We report a facile method to construct a unique architecture of thermoelectric oxides that is promising to realize a simultaneous improvement of overall electrical conductivity and thermopower. Here, a series of two-phase nanocomposites comprising of Ca3Co4O9 (CCO) and La0.8Sr0.2CoO3 (LSCO) has been synthesized through ball milling followed by spark plasma sintering (SPS) method. The electron microscope images reveal that the two constituents form the unique composites while retaining their individual crystalline and morphological identities. Owing to the hierarchical mesoscopic structure with nanoscale particles and submicrometer scale grain boundaries, an external strain is induced into the CCO grains by the LSCO nanoparticles to enhance the thermopower. The mesoscopic structure is also favorable for improving the electrical conductivity. Moreover, the long-wavelength phonons can be scattered effectively from LSCO nanoparticles and the thermal conductivity is further suppressed. With compromises between power factor and thermal conductivity, the largest ZT achieved is up to 0.41 at 1000K for the composites with 25wt% of LSCO.