The energy management strategy (EMS) is a key to reduce the equivalent hydrogen consumption and slow down fuel cell performance degradation of the plug-in fuel cell hybrid electric vehicles. Global optimal EMS based on the whole trip information can achieve the minimum hydrogen consumption, but it is difficult to apply in real driving. This paper tries to solve this problem with a novel hierarchical EMS proposed to realize the real-time application and approximate global optimization. The long-term average speed in each future trip segment is predicted by KNN, and the short-term speed series is predicted by a new model averaging method. The approximate global optimization is realized by introducing hierarchical reinforcement learning (HRL), and the strategy within the speed forecast window is optimized by introducing upper confidence tree search (UCTS). The vehicle speed prediction and the proposed EMS have been verified using the collected real driving cycles. The results show that the proposed strategy can adapt to driving style changes through self-learning. Compared with the widely used rule-based strategy, it can evidently reduce hydrogen consumption by 6.14% and fuel cell start-stop times by 21.7% on average to suppress the aging of fuel cell. Moreover, its computation time is less than 0.447 sat each step, and combined with rolling optimization, it can be used for real-time application. (C) 2018 Hydrogen Energy Publications LLC. Published by Elsevier Ltd. All rights reserved.