Aims This study was aimed at revealing the composition of microbiota in leaves, roots and rhizosphere soil of wheat plants that are resistant or susceptible to stripe rust, one of the most widely destructive leaf diseases in wheat production. Methods and Results A total of 36 wheat plants that showed resistant or susceptible reactions to stripe rust were sampled. Three compartments of each plant including leaves, roots and rhizosphere soil were used for whole-genomic DNA extraction and the DNA samples were subjected to high-throughput 16S rRNA gene sequencing. A total of 2885 operational taxonomic units (OTUs) were revealed from the sequencing, and they mainly distributed in the phylum of Proteobacteria. Twenty-nine OTUs formed the core microbiota of wheat plants. The differences between above- and below-ground environments could explain most of the dissimilarity of wheat-associated microbial communities. Therefore, those microbes that were able to adapt to the above-ground (leaf) environment might be more important resources for biocontrol agents against stripe rust, and they could be from genera Hymenobacter, Flavobacterium, Chitinophage, Flavisolibacter, Niastella, Mucilaginibacter, Pedobacter, Aquincola, Massilia, Citrobacter, Cronobacter, Ewingella, Acinetobacter and Pseudomonas. No matter the microbial taxa were significantly selected by the resistant or susceptible wheat plants, they contained the members with plant growth promoting (PGP) features and could be used as potential biocontrol agents to reduce stripe rust damage. Conclusions The core microbiota associated with wheat plants and microbial taxa that were significantly correlated with reactions to stripe rust were identified in this study. Significance and Impact of the Study Few studies had been done on the microbiota associated with wheat so far. Our study will not only provide fundamental knowledge about the composition of wheat-associated microbiota but also reveal the microbial taxa that have the potential to be integrated into the strategy of stripe rust management.