Conductive polymer nanofiber composites (CPNCs) have promising applications in many fields. However, preparation of the CPNC with excellent mechanical and electrical properties is still challenging. Here, reduced graphene oxide (RGO) was assembled onto the nanofiber surface assisted by ultrasonication, obtaining a nanofiber composite with a polymer core-RGO shell structure. The interfacial collision between polymer and RGO would occur under ultrasonication, during which the flexible RGO was easily deformed or bent and finally formed the core-shell structure. The polymer nanofiber core endowed the composite flexibility, while the RGO shell formed a continuous network, decreasing significantly the electrical resistivity. The electrical conductivity of the RGO-based nanofiber composite was closely related to the solvent used to disperse the RGO and the ultrasonication time. RGO introduction enhanced the tensile strength, Young's modulus, and elongation at break. The RGO tubes were harvested by thermal degradation of the polymer nanofiber core.