The concentration of electrically active impurities in in situ boron (B)-doped silicon (Si) nanowires (SiNWs) synthesized by gold (Au)-catalyzed chemical vapor deposition (CVD) is studied by Raman spectroscopy. B-doped SiNWs exhibit an asymmetric Raman spectrum due to Fano resonance between discrete phonon Raman scattering and continuous electric Raman scattering caused by the excitation of holes in the valence band. To quantitatively evaluate the concentration of active B atoms from the asymmetric spectral shape, the spectra are fitted by a Fano resonance formula and asymmetry parameters are extracted. From the comparison of the asymmetry parameter with those obtained for reference samples, the concentration of active B atoms in SiNWs is estimated. The effects of thermal annealing in nitrogen and oxygen gases on the active B concentration are also studied. The annealing in nitrogen ambience for a short period significantly increases the concentration of active B atoms especially when the doping level is high, while longer period annealing decreases the active B concentration. The reduction of active B concentration is more significant when SiNWs are annealed in oxygen gas ambience. By combining Raman results with high-resolution transmission electron microscope observations, the growth mechanism of B-doped SiNWs is discussed. We show that the number of B atoms doped into SiNWs via Au catalysts is very limited and high B concentration layers are grown by conformal growth on the sidewall of SiNWs.