In this paper we present time-dependent density functional calculations on frequency-dependent first (beta) and second (gamma) hyperpolarizabilities for the set of small molecules, N-2, CO2, CS2, C2H4, NH3, CO, HF, H2O, and CH4, and compare them to Hartree-Fock and correlated ab initio calculations, as well as to experimental results. Both the static hyperpolarizabilities and the frequency dispersion are studied. Three approximations to the exchange-correlation (xc) potential are used: the widely used Local Density Approximation (LDA), the Becke-Lee-Yang-Parr (BLYP) Generalized Gradient Approximation (GGA), as well as the asymptotically correct Van Leeuwen-Baerends (LB94) potential. For the functional derivatives of the re potential the Adiabatic Local Density Approximation (ALDA) is used. We have attempted to estimate the intrinsic quality of these methods by using large basis sets, augmented with several diffuse functions, yielding good agreement with recent numerical static LDA results. Contrary to claims which have appeared in the literature on the basis of smaller studies involving basis sets of lesser quality, we find that the static LDA results for beta and gamma are severely overestimated, and do not improve upon the (underestimated) Hartree-Fock results. No improvement is provided by the BLYP potential which suffers from the same incorrect asymptotic behavior as the LDA potential. The results are however clearly improved upon by the LB94 potential, which leads to underestimated results, slightly improving the Hartree-Fock results. The LDA and BLYP potentials overestimate the frequency dependence as well, which is once again improved by the LB94 potential. Future improvements are expected to come from improved models for asymptotically correct exchange-correlation potentials. Apart from the LB94 potential used in this work, several other asymptotically correct potentials have recently been suggested in the literature and can also be expected to improve considerably upon the relatively poor LDA and GGA results, for both the static properties and their frequency dependence.