This article considers the problem of selecting sensors in a large-scale system to minimize the error in estimating its states, more specifically, the state estimation mean-square error (MSE) and worst-case error for Kalman filtering and smoothing. Such selection problems are in general NP-hard, i.e., their solution can only be approximated in practice even for moderately large problems. Due to its low complexity and iterative nature, greedy algorithms are often used to obtain these approximations by selecting one sensor at a time choosing at each step the one thatminimizes the estimation performance metric. When this metric is supermodular, this solution is guaranteed to be (1 - 1/e)-optimal. This is, however, not the case for the MSE or the worst-case error. This issue is often circumvented by using supermodular surrogates, such as the log det, despite the fact thatminimizing the log det is not equivalent to minimizing the MSE. Here, this issue is addressed by leveraging the concept of approximate supermodularity to derive near-optimality certificates for greedily minimizing the estimation mean-square and worst-case error. In typical application scenarios, these certificates approach the (1 - 1/e) guarantee obtained for supermodular functions, thus demonstrating that no change to the original problem is needed to obtain guaranteed good performance.