In the present work, a mathematical and statistical approach was adopted to study the formation and stability of oil-in-water emulsion with an integrated hybrid genetic algorithm (GA) coupled with feed-forward back-propagation artificial neural network (BPANN) and response surface methodology (RSM) based on Box-Behnken design (BBD). The input parameters were oil concentration (10-50%, v/v), surfactant concentration (0.1-2%, w/v), stirring speed (2000-6000 rpm) and stirring time (5-20 min). The output parameter was relative emulsion volume expressed as emulsion stability index (ESI24). In the proposed hybrid GA model, outputs of BP-ANN model were used as initial population settings and RSM-BBD generated model equation was used as a fitness function. Error analysis was performed on the model fit to the experimental data using sum of square error (SSE), mean square error (MSE) and relative percent error (RPD). The optimum condition predicted by the hybrid GA was 0.913 of ESI24, with 4.70% error under 50% (v/v) oil concentrations, 2% (w/v) surfactant concentrations, 5691 rpm and 5 min stirring time. The proposed hybrid GA model was found to be useful for the optimization of process parameters for emulsion formation and stability analysis. (C) 2015 The Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved.