We report a general methodology to the rational design of thermally stimulated short-wave infrared (SWIR) luminescence between similar to 900 and 1700 nm by a new combination of using efficient energy transfer from Bi3+ to Nd3+ and an adjustable hole trap depth via valence band engineering. Predictions from a vacuum referred binding energy (VRBE) diagram are combined with the data from optical spectroscopy and thermoluminescence to show the design concept by using bismuth and lanthanide doped rare earth ortho-phosphates as model examples. Nd3+ with its characteristic F-4(3/2)-> I-4(j) (j = 9/2, 11/2, 13/2) emission in the SWIR range is first selected as the emitting centre. The energy transfer (ET) processes from Bi3+ or Tb3+ recombination centres to Nd3+ are then discussed. Photoluminescence results show that the energy transfer efficiency of Bi3+ -> Nd3+ appears to be much higher than of Tb3+ -> Nd3+. To exploit this ET, thermally stimulated Bi3+ A-band emission can then be designed by using Bi3+ as a similar to 2.7 eV deep electron trap in YPO4. By combining Bi3+ with Tb3+, Pr3+, or Bi3+ itself, the holes trapped at Tb4+, Pr4+, or Bi4+ will release earlier than the electrons captured at Bi2+. On recombination with Bi2+, Bi3+ in its excited state is formed generating Bi3+ A-band emission. Due to the ET of Bi3+ -> Nd3+ 1.06 mu m Nd3+ emission appears in YPO4. Herein, the thermally stimulated Nd3+ SWIR emission is achieved by hole release rather than the more commonly reported electron release. The temperature when thermally stimulated Nd3+ SWIR emission appears can further be engineered by changing the Tb3+ or Pr3+ hole trap depth in Y1-xLuxPO4 by adjusting x. Such valence band engineering approach can also be applied to other compounds like La1-xGdxPO4 and Gd1-xLaxAlO3 solid solutions. Our work opens the avenue to motivate scientists to explore novel SWIR afterglow phosphors in a design way instead of by trial and error approach.