Single-walled carbon nanotubes (SWCNTs) are utilized in many areas, accompanied with the ever rising safety concerns. Coating the SWCNTs by serum albumin has shown promises in reduction of their cytotoxicity. The cause of toxicity reduction can be due to the blockage of cellular protein adsorption by bovine serum albumin (BSA). Here, the mechanism of toxicity reduction is explored from the point of view of protein adsorption. Different loadings of BSA lead to varied surface coverage of the SWCNTs, which is positively related to the level of cytotoxicity. In addition, the BSA-coated SWCNTs are tested for their surface morphology change, cellular uptake, and adsorption of cellular proteins. BSA can be competed off the SWCNT surface by the cytosol proteins, and thus a higher BSA loading is needed to provide better protection to the cells. Cellular uptake is also reduced with a higher BSA loading. Moreover, the BSA coating changes the surface property of SWCNTs, and as a consequence, alters the types of proteins adsorbed by the SWCNTs. Our results support that adsorption of BSA reduces cellular uptake of SWCNTs as well as adsorption of cellular proteins on SWCNTs, both contributing to the much lower cytotoxicity observed for the BSA-coated SWCNTs.