Despite the fact that five kinds of uranium nitrides, i.e., uranium mononitrides (UN, R3m and Fm3n), a uranium dinitride (UN2, Fm3m), and uranium sesquinitrides (alpha-U2N3, Ia3; beta-U2N3,P3m1), have been confirmed, until now the phase relations are not well understood because of the puzzling nonstoichiometric issue. This work reinvestigated the crystallographic structures of these phases using cluster formula theory. The principal clusters (cuboctahedron with six squares and eight triangles) in these phases were determined. N atoms can occupy either six octahedral sites (square face centers) or eight tetrahedral sites (formed by a center atom and a triangle) in the principal cluster of 13 U atoms, resulting in these diversified phases and the nonstoichiometric issue. Also, phase transformations at certain temperatures and pressures (from CaF2-type UN2 to Mn2O3-type U2N3, from Mn2O3-type U2N3 to NaCl-type UN, and from NaCl-type UN to HgIn-type UN) were deduced by tracking the bond and angle changes of a simplified cluster [U-U6N6]. This investigation provides an in-depth understanding of the phase relations in a U-N system.