Higher order statistics (HOS) have been widely adopted to diagnose the poor control loop performance in recent years. The existing HOS tools, including bispectrum, bicoherence and bicepstrum, can easily detect severe nonlinearity, but it is still an open problem to ensure the detecting performance when short time series and non-significant nonlinearity are taken into account. In this paper, a new cepstral definition of bicoherence is proposed, namely, bihocerence, which normalizes the traditional bicepstrum. Consequently, a novel statistical index for nonlinearity characterization is defined based on bihocerence. Determination of the correct confidence limit is accomplished by utilizing surrogate data with de-trending and re-trending procedures. Compared with the existing HOS methods, the bihocerence test provides more reliable nonlinearity detection results when dealing with small nonstationary time series and weak nonlinearity. This allows the online application of bihocerence approach to detect process non linearity at its early stage. The validity of the raised approach is demonstrated on a series of simulations as well as industrial cases. (C) 2018 Elsevier Ltd. All rights reserved.