The exit tunnel of the ribosome is commonly considered to be sufficiently narrow that co-translational folding can begin only when specific segments of nascent chains are fully extruded from the tunnel. Here we show, on the basis of molecular simulations and comparison with experiment, that the long-range contacts essential for initiating protein folding can form within a nascent chain when it reaches the last 20 angstrom of the exit tunnel. We further show that, in this "exit port", a significant proportion of native and non-native tertiary structure can form without steric overlap with the ribosome itself, and provide a library of structural elements that our simulations predict can form in the exit tunnel and is amenable to experimental testing. Our results show that these elements of folded tertiary structure form only transiently and are at their midpoints of stability at the boundary region between the inside and the outside of the tunnel. These findings provide a framework for interpreting a range of recent experimental studies of ribosome nascent chain complexes and for understanding key aspects of the nature of co-translational folding.