The entropy generation minimization and the entransy theory are widely used in the optimization of heat transfer. In this paper, the two theories are applied to coupled steady heat transfer systems. The two different definitions of radiative entransy flux are discussed. It is found that the extremum principle of entransy dissipation for coupled heat transfer systems can be obtained only when temperature is treated as the driving force of radiative heat transfer and the definition of entransy flux for radiative heat transfer is the same as that in conductive and convective heat transfer. Taking temperature as the driving force of radiative heat transfer, we have analyzed three coupled heat transfer examples, which are the onedimensional coupled conductive and radiative heat transfer, the coupled convective and radiative heat transfer and the coupled conductive, convective and radiative heat transfer. The results show that the extremum principle of entransy dissipation always leads to the best system performance, while the entropy generation minimization does not always. The definition of radiative entransy flux in which the blackbody emissive power is treated as the driving force is also used for the three examples. However, the results show that neither the extremum radiative entransy dissipation rate nor the extremum conductive convective entransy dissipation rate results in the best system performance. Therefore, this definition is not suitable for coupled heat transfer systems. (C) 2018 Elsevier Ltd. All rights reserved.