In this paper, mixed integer nonlinear programming (MINLP) is optimized by PSO_GA-SQP, the mixed coding of a particle swarm optimization (PSO), and a hybrid genetic algorithm and sequential quadratic programming (GA-SQP). The population is separated into two groups: discrete and continuous variables. The discrete variables are optimized by the adapted PSO, while the continuous variables are optimized by the GA-SQP using the discrete variable information from the adapted PSO. Therefore, the population can be set to a smaller size than usual to obtain a global solution. The proposed PSO_GA-SQP algorithm is verified using various MINLP problems including the designing of retrofit heat exchanger networks. The fitness values of the tested problems are able to reach the global optimum.