A reaction field exists even for a nonpolar solute embedded in a spherical cavity within a surrounding homogeneous dielectric continuum. This arises from the tail of the electronic wave function that penetrates beyond the cavity boundary into the dielectric region. This effect, which is neglected or treated only in cursory fashion in most reaction field implementations, is examined in detail for the simple case of a ground state hydrogen atom, where very accurate solutions of the relevant equations can be obtained. Properties considered include the penetration of the electron outside the cavity, the electronic density at the nucleus, the electron binding energy, the electrostatic free energy of solvation, the polarizability, and the vertical 1s --> 2p excitation energy. Also, the effect of the common approximation of neglecting the volume polarization and treating only the surface polarization contribution to the reaction field is critically evaluated.