Accurate estimation of wind speed frequency distribution will be beneficial to predict accurately wind energy potential and select optimal wind energy conversion systems. Due to the complexity of wind regime, it has been reported that mixture models, constituted by multiple distributions (components), perform better than single models, which are commonly used to model wind speed frequency distribution. The underlying assumption in the existing mixture models is that each component is a unimodal distribution. In this work, we propose a more general model, named hierarchical mixture of multiple distributions. To deal with the bimodal or multimodal case in components, we assume that each component may also be a mixture model. In order to estimate the parameters in the proposed model, a nested expectation maximization algorithm is introduced. The experimental results indicate that, the proposed model outperforms single and mixture models. Moreover, due to the complex expression of the probability density function derived by the proposed model, we also develop a technique to estimate wind energy potential. Accordingly, the performances of different wind energy conversion systems are evaluated via four indicators, which provide comprehensive basis for selecting optimal wind energy conversion system. (C) 2016 Elsevier Ltd. All rights reserved.