Developing reversible lithium metal anodes with high rate capability is one of the central aims of current battery research. Lithium metal anodes are not only required for the development of innovative cell concepts such as lithium-air or lithium-sulfur batteries, they can also increase the energy density of batteries with intercalation-type cathodes. The use of solid electrolyte separators is especially promising to develop well-performing lithium metal anodes, because they can act as a mechanical barrier to avoid unwanted dendritic growth of lithium through the cell. However, inhomogeneous electrodeposition and contact loss often hinder the application of a lithium metal anode in solid-state batteries. In this review, we assess the physicochemical concepts that describe the fundamental mechanisms governing lithium metal anode performance in combination with inorganic solid electrolytes. In particular, our discussion of kinetic rate limitations and morphological stability intends to stimulate further progress in the field of lithium metal anodes.