Recently, it was found that element semiconductor tellurium (Te) has a high thermoelectric performance. However, it needs to be doped with high-toxic arsenic (As) to tune the carrier concentration of Te. In this paper, low-toxic bismuth (Bi) was employed as dopant to optimize the thermoelectric performance of Te combining with a high-pressure synthesis method. The effect of substituting Bi on the electrical transport and thermal transport properties of Te has been investigated. The results show that the solubility limit of Bi in Te is about 0.1 mol%. However, the trace amounts of Bi doping can tune the carrier concentration of Te effectively and thereby optimize its power factor. The thermal conductivity of non-doped Te prepared by high pressure is much lower than that of the sample prepared at ambient pressure. And Bi doping can further decrease the value due to the phonons scattered by the heavy impurity atom. An enhanced figure of merit ZT similar to 0.72 was obtained at 517 K, which is about four times that of non-doped Te and is comparable to the state-of-the-art thermoelectric alloys with more complex composition such as Bi2Te3 and PbTe.