Online detection of coal and gangue is the key to intelligent separation of coal and gangue which is significant for improving coal mining efficiency and deducing environmental pollution. To improve the detection of coal and gangue under complex conditions, we proposed a cascade network that consists of detector and discriminator. Employing the idea of combining traditional computer vision with deep learning, the multi-channel feature fusion layer was designed in the discriminator. The convolutional neural network (CNN) in the discriminator was optimized from the perspective of loss function and classifier, and a decision function was designed to unify the results of the detector and the discriminator. Finally, we used the visualization to analyze the classification basis during the production by CNN. Our results showed that the cascade network proposed in this study took the characteristics of material truncation as the classification basis, and the detection accuracy reached up to 91.375%. (C) 2020 Elsevier B.V. All rights reserved.