The pressure transient technique, such as pulse decay method (PDM), has been considered an advanced approach to measure the permeability of the coalbed methane(CBM) and shale gas formations as such approach is more efficient, accurate and effective compared to that of the conventional flowrate measurement method. A constant isothermal gas compressibility factor is assumed over the testing pressure range in the experimental studies when using the PDM, which could lead to an inaccurate measured permeability. In this study, a nonlinear parabolic partial differential equation (PDE) model, taking the isothermal gas compressibility factor (gas compressibility) as a pressure dependent variable, was established to examine how pressure responses are affected by the gas compressibility variation. A novel and practical finite difference scheme was then proposed to efficiently and accurately solve the nonlinear PDE model, providing a powerful tool to numerically investigate the gas flow behavior in the CBM and shale rocks. Analytical solution for the PDE model was derived rigorously to investigate the influence of gas compressibility on the calculated permeability. Laboratory experiments were also conducted on coal permeability measurement and characteristics of the pressure pulse decay plots verified the effect of the gas compressibility on the pressure responses, especially under low pressures. Both numerical and experimental results demonstrated that the gas compressibility is a crucial parameter that should be considered for the permeability calculations in CBM and shale reservoirs. It is found that the difference in the pressure response that is predicted between the nonlinear model and the linear model with a constant gas compressibility can reach up to 41.1% at the testing pressure of 0.7 MPa. In addition, the difference in the measured permeability obtained from the two models increases as gas depletion proceeds. And also, it is suggested that both pressure changes in the upstream and downstream reservoirs should be used for permeability calculation to minimize the influence of gas compressibility on calculated permeability. This study extends the applicability of the pressure transient technique and provides an effective way to accurately measure permeability of unconventional tight gas reservoirs.