The standalone photovoltaic power system is one of the promising solutions in rural electrification which has been widely implemented to supply electricity for basic household needs. Standalone photovoltaic power systems normally integrate energy storage devices, mainly Lead-acid battery, to compensate the supply demand mismatch due to the nature of solar energy. However, the short cycle life of Lead-acid battery increases the operating cost of photovoltaic power systems. Supercapacitor-battery hybrid energy storage system has been proposed by researchers to extend the cycle life of battery bank by mitigating the charge discharge stress due to the fluctuating power exchange. The existing hybrid energy storage systems and their corresponding energy management strategies vary in terms of topology, complexity and control algorithm which are often application oriented. This paper presents a comprehensive review of the state of the art for HESS and discusses potential topologies that are suitable for improving the service life of Lead-acid battery in standalone photovoltaic power systems. Theoretical analysis and numerical simulation in Matlab Simulink for different hybrid energy storage system topologies in rural residential energy system applications have been carried out and their effectiveness in mitigating battery stress are investigated and compared. A battery health cost function is proposed in this paper to quantify the impact of many damaging factors on battery, thus the effectiveness of different hybrid energy storage systems in mitigating battery stress and the associated financial analysis can be quantitatively compared. Finally, a scaled-down hybrid energy storage system prototype has been developed and its performances in standalone photovoltaic system are emulated to validate the simulation analysis.