In this paper we discuss smooth and sensitive norms for prediction error system identification when the disturbances are magnitude bounded. Formal conditions for sensitive norms, which give an order of magnitude faster convergence of the parameter estimate variance, are developed. However, it also is shown that the parameter estimate variance convergence rate of sensitive norms is arbitrarily bad for certain distributions. A necessary condition for a norm to be statistically robust with respect to the family F(C) of distributions with support [-C,C] for some arbitrary C > 0 is that its second derivative does not vanish on the support. A direct consequence of this observation is that the quadratic norm is statistically robust among all l(p)-norms, p less than or equal to 2 < infinity for F(C).