Unlike the traditional policy, Generation Expansion Planning (GEP) problem in competitive framework is complicated. In the new policy, each GENeration COmpany (GENCO) decides to invest in such a way that obtains as much profit as possible. This paper presents a new hybrid algorithm to determine GEP in a Pool market. The proposed algorithm is divided in two programming levels: master and slave. In the master level a modified game theory (MGT) is proposed to evaluate the contrast of GENCOs by the Independent System Operator (ISO). In the slave level, a particle swarm optimization (PSO) method is used to find the best solution of each GENCO for decision-making of investment. The validity of the proposed method is examined in the case study including three GENCOs with multi-types of power plants. The results show that the presented method is both satisfactory and consistent with expectation. (C) 2010 Elsevier Ltd. All rights reserved.