A thermoelectric heat recovery unit could actively use thermoelectric refrigeration process to recover waste heat of the air exhausted from buildings. Theoretical analysis of a thermoelectric heat recovery unit is firstly conducted in the present work, and a corresponding mathematical model is proposed to incorporate with the thermoelectric cooling effect and the method of effectiveness-number of transfer units. Subsequently, the cost performance model considering the relationship between system performances and operating cost is set up, aiming to critically evaluate the global performance of thermoelectric heat recovery system. Overall thermal conductance is determined analytically from steady-state model, where the optimal operating parameters, including the quantity, filling factor, length of P-N legs and couple numbers of thermoelectric coolers are identified. Finally, this prediction method is used to analyze system performance and operating cost in a practical thermoelectric heat recovery system operated in a representative residential building area. The simulation results demonstrated that the lower operating cost of a thermoelectric heat recovery system could be lower than 0.02 $/kWh under the optimal parametric conditions. In addition, present demo investigations indicated that the use of multiple thermoelectric cooling units in the heat recovery ventilation system could further reduce operating cost and promote the cooling capacity simultaneously. Present work illuminated that the thermoelectric heat recovery unit has a great potential for being applied in future energy-efficient buildings.