Inspired by nanoscience and nanoengineering, numerous nanostructured materials developed by multidisciplinary approaches exhibit excellent photoelectronic properties ranging from ultraviolet to terahertz frequencies. As a new class of building block, nanoscale elements in terms of quantum dots, nanowires, and nanolayers can be used for fabricating photodetectors with high performance. Moreover, in conjunction with traditional photodetectors, they exhibit appealing performance for practical applications including high density of integration, high sensitivity, fast response, and multifunction. Therefore, with the perspective of photodetectors constructed by diverse low-dimensional nanostructured materials, recent advances in nanoscale photodetectors are discussed here; meanwhile, challenges and promising future directions in this research field are proposed.