Low ice adhesion surfaces are a promising anti-icing strategy. However, experimentally obtained ice adhesion strengths are so far not transferable or comparable, and they are very sensitive to the experimental set-up and type of accreted ice. This critical review describes both the widely used ice adhesion measurement techniques and ice generation methods, before discussing why the results cannot be directly compared. The ice adhesion measurement techniques included are horizontal and vertical shear tests, centrifugal ice adhesion tests and tensile tests. The ice generation methods included in the review are icing by freezing drizzle, wind tunnel icing and bulk water icing, and definitions of other ice types are included. Special challenges for low ice adhesion surfaces have been discussed, as well as the effect of ice sample size and the impact from gravity. A future standard within ice adhesion research should be able to directly compare different experimental results, and should, therefore, include all relevant parameters and currently available methods. Due to the large variation of available ice adhesion set-ups, it is recommended that a reference test and ice type is established for enhanced comparability. The reference suggested is a horizontal shear test with bulk water ice, and defined experimental parameters. A continuing focus on the fundamental mechanisms of ice adhesion is needed to identify the important contributions to the large variations in reported ice adhesion strengths.