Thermoplastic polyurethane (TPU) with more rigid hard segments and more flexible soft segments exhibits stronger microphase separation and stronger microphase separation leads to the better mechanical properties in general. In this study, to obtain TPU with low hardness without sacrificing abrasion resistance and tensile properties, TPU was blended with ethylene-propylene-diene monomer rubber (EPDM) or polybutadiene rubber (BR), which have more flexibility than the soft segments in TPU. Also, more rigid nitrile butadiene rubber (NBR) than the soft segments in TPU was blended with TPU for comparison. Even though NBS abrasion resistance, tensile strength and tensile modulus of TPU are much higher than those of EPDM and BR, TPU/EPDM (90/10) and TPU/BR (90/10) blends exhibit the higher NBS abrasion resistance, tensile strength and tensile modulus, than does of TPU. For TPU/NBR blends, the mechanical properties of the blends continuously decrease with increasing content of NBR. Since Tg of EPDM (or BR) is lower than that of the soft segments in TPU used in this study, TPU/EPDM (90/10) and TPU/BR (90/10) blends may result in the increase of flexibility of the soft segments in TPU, and show the stronger microphase separation, which is confirmed by the dynamic mechanical analyzer (DMA) results.