Using carbon dioxide as a displacing fluid to enhance shale gas recovery is a promising technique given its potential for significant contributions to both unconventional resource development and CO2 geological sequestration. The adsorption capacity of CO2 in nanoscale shale organic pores is the key issue to evaluate the feasibility of CO2-enhanced shale gas recovery technology. However, as a result of the complex organic component of the solid surface, the fluid solid interaction between the confined fluid and the solid surface, and the intermolecular interaction between the confined fluids, the adsorption behavior of CO2 in the shale is not clear. In this work, shale organic nanopores with different geometries (slit pore and cylindrical pore) and different sizes (1, 2, and 4 nm) are constructed using molecular dynamics and Monte Carlo methods. Isothermal adsorption of CO2 and methane as single components and competitive adsorption of a CO2 methane binary mixture are simulated in a nanoscale methane/CO2/organic matter system. The density profile and distribution contour indicate that CO2 adsorption in shale organic mesopores does not occur via monolayer adsorption. Considering the inadaptability of the Langmuir model to analyze the CO2 adsorption curve, a modified Brunauer Emmett Teller (BET) model is applied to describe and fit the data for the CO2 and methane adsorption amount, with the parameters in the modified BET model used to characterize the adsorption capacity and affinity of the fluid. The maximum adsorption amount, characteristic pressure, and selectivity parameter of CO2, methane, and a binary mixture indicate that the adsorption capacity and affinity of CO2 are stronger than those of methane under reservoir pressure, which provides useful support for enhancing shale gas recovery by injecting CO2.