Although the niche differentiation of anammox bacteria has been extensively observed in a lab-scale reactor, the inherent mechanism for this ecological phenomenon is still elusive. Here, we combined the long-term reactor operation, genome-centered metagenome, and metatranscriptome analyses to gain insight into the substrate competition and niche differentiation of Candidatus Jettenia and Candidatus Brocadia. After 146 days of operation, we found the anammox bacterial population shifted from Ca. Jettenia to Ca. Brocadia in the immobilization-anaerobic baffled reactor (I-ABR) with the ammonium and nitrite concentrations of 30 mg/L. Importantly, the genome and transcript comparisons of Ca. Jettenia and Ca. Brocadia showed that Ca. Brocadia harbored more complete function in cell chemotaxis, flagellar assembly, and two-component system and more redundant function in nitrite reduction, in which the genes were also highly expressed. Ca. Brocadia out-competed Ca. Jettenia at the mainstream condition. Meanwhile, though the highest biomass concentration led to the highest nitrogen removal rate (NRR) in the first compartment (C1), the competition of Ca. Jettenia and Ca. Brocadia could also affect the NRR of different compartments through affecting the bacterial activity. Substrate competition of anammox bacteria led to higher transcript activity of Ca. Jettenia and Ca. Brocadia in the second (C2) and fourth (C4) compartments, respectively. Further, high transcript activity of Ca. Brocadia led to the higher NRR in C4. A comparison of metabolic potential based on the metagenome-assembled genome adds a different dimension for understanding the discrepantly physiological characteristics and competition of anammox bacteria for wastewater treatment.