Yielding behavior is well known in attractive colloidal suspensions. Adhesive non-Brownian suspensions, in which the interparticle bonds are due to finite-size contacts, also show yielding behavior. We use a combination of steady-state, oscillatory, and shear reversal rheology to probe the physical origins of yielding in the latter class of materials and find that yielding is not simply a matter of breaking adhesive bonds but involves unjamming from a shear-jammed state in which the microstructure has adapted to the direction of the applied load. Comparison with a recent constraint-based rheology model shows the importance of friction in determining the yield stress, suggesting novel ways to tune the flow of such suspensions.