Busbar pull tests are the generally accepted method of measuring metal-silicon adhesion for silicon solar cells. However, this method cannot be used to measure metal finger adhesion and consequently cannot be used for cells being interconnected with new technologies that require no busbars to be formed on the cell or for when finger and busbar adhesion can differ. To address this need, a stylus-based adhesion tester has been developed that enables direct finger adhesion measurements. However, the operation of this tester results in different finger impact failure modes depending on the relative cohesive and adhesive properties of the metal fingers, and these different finger impact modes need to be differentiated before the measured forces can be used for comparative purposes. In this paper, we report on the implementation of an automatic classifier of failure modes for stylus based adhesion testing. The classifier analyses video frames corresponding to when the stylus laterally impacts fingers on the solar cell surface and classifies them using a support vector machine classifier trained on features extracted from a set of example images. A classification accuracy of 94.4% +/- 0.4%(abs) was achieved on a test set of images comprising finger impacts on both screen-printed and plated silicon solar cells when a "Histogram of Oriented Gradients" (HOG) algorithm was used to generate the image features. Using a Matlab implementation, it took 3.2 s to classify 500 test images using the HOG features. This suggests that only 7-8 s would be required to analyse the finger impacts for a 156 mm industrial silicon solar cell, with even further reduced processing times made possible by software optimisation and reducing the number of scans across the solar cell.