In this article, we proposed a new method based on natural neighbor interpolation to recover the spectral reflectance of objects from an image captured by a traditional Red-Green-Blue (RGB) digital camera. The concept of model-based metameric spectra of eight extreme points in the standard RGB (sRGB) color gamut was further introduced to ensure that almost all test samples in the entire gamut can be simply and properly recovered without needing the extrapolation or any other auxiliary techniques. The quasi-Newton method was used to estimate iteratively the optimal parameters of these metameric spectra, satisfying the constraints of the gamut extreme points. Several experiments were performed. The effectiveness of the method with and without the metameric spectra was evaluated, including some performance comparisons with the principal component analysis (PCA) method of transformational type (classic PCA and weighted PCA) and that of interpolation type. The results showed that the proposed method greatly enlarged the accurately applicable domain of the interpolation strategy and offered spectra with feasibility and naturalness much superior to the PCA-based methods. The proposed method was obviously better than the conventional interpolation ones, and had a similar performance with the weighted PCA method in terms of color difference.