Growing interests have been devoted to the design of polymer acceptors as potential replacement for fullerene derivatives for high-performance all polymer solar cells (all-PSCs). One key factor that is limiting the efficiency of all-PSCs is the low fill factor (FF) (normally <0.65), which is strongly correlated with the mobility and film morphology of polymer:polymer blends. In this work, we find a facile method to modulate the crystallinity of the well-known naphthalene diimide (NDI) based polymer N2200, by replacing a certain amount of bithiophene (2T) units in the N2200 backbone by single thiophene (T) units and synthesizing a series of random polymers PNDI-Tx, where x is the percentage of the single T. The acceptor PNDI-T10 is properly miscible with the low band gap donor polymer PTB7-Th, and the nanostructured blend promotes efficient exciton dissociation and charge transport. Solvent annealing (SA) enables higher hole and electron mobilities, and further suppresses the bimolecular recombination. As expected, the PTB7-Th:PNDI-T10 solar cells attain a high PCE of 7.6%, which is a 2-fold increase compared to that of PTB7-Th:N2200 solar cells. The FF of 0.71 reaches the highest value among all-PSCs to date. Our work demonstrates a rational design for fine-tuned crystallinity of polymer acceptors, and reveals the high potential of all-PSCs through structure and morphology engineering of semicrystalline polymer:polymer blends.