Correlation consistent basis sets for accurately describing core-core and core-valence correlation effects in atoms and molecules have been developed for the second row atoms Al-Ar. Two different optimization strategies were investigated, which led to two families of core-valence basis sets when the optimized functions were added to the standard correlation consistent basis sets (cc-pVnZ). In the first case, the exponents of the augmenting primitive Gaussian functions were optimized with respect to the difference between all-electron and valence-electron correlated calculations, i.e., for the core-core plus core-valence correlation energy. This yielded the cc-pCVnZ family of basis sets, which are analogous to the sets developed previously for the first row atoms [D. E. Woon and T. H. Dunning, Jr., J. Chem. Phys. 103, 4572 (1995)]. Although the cc-pCVnZ sets exhibit systematic convergence to the all-electron correlation energy at the complete basis set limit, the intershell (core-valence) correlation energy converges more slowly than the intrashell (core-core) correlation energy. Since the effect of including the core electrons on the calculation of molecular properties tends to be dominated by core-valence correlation effects, a second scheme for determining the augmenting functions was investigated. In this approach, the exponents of the functions to be added to the cc-pVnZ sets were optimized with respect to just the core-valence (intershell) correlation energy, except that a small amount of core-core correlation energy was included in order to ensure systematic convergence to the complete basis set limit. These new sets, denoted weighted core-valence basis sets (cc-pwCVnZ), significantly improve the convergence of many molecular properties with n. Optimum cc-pwCVnZ sets for the first-row atoms were also developed and show similar advantages. Both the cc-pCVnZ and cc-pwCVnZ basis sets were benchmarked in coupled cluster [CCSD(T)] calculations on a series of second row homonuclear diatomic molecules (Al-2, Si-2, P-2, S-2, and Cl-2), as well as on selected diatomic molecules involving first row atoms (CO, SiO, PN, and BCl). For the calculation of core correlation effects on energetic and spectroscopic properties, the cc-pwCVnZ basis sets are recommended over the cc-pCVnZ ones.