Graphene has some outstanding properties in the photoelectric field, especially ultra fast response and broad spectrum absorption, which have led to strong attraction on graphene photodetector. However, the photoresponsivity of pure graphene device is below 10 mA W-1. It would limit the application of graphene photodetector. Here, we design a metamaterial structure with a broadband high absorption spectrum. And by integrating it into the graphene photodetector, the high photoresponsivity of a pure graphene photodetector in a broadband range could be acquired. The photoresponsivity of photodetector can achieve nearly 200 mA W(-1)which is two orders higher than the one of pure monolayer graphene photodetector. Besides, the photodetector also keeps the broadband photoresponse. The hot carriers can be generated by coupling of multiple resonances and effectively transported to graphene with the advantage of ultrathin metal film. Based on this work, more potential applications of graphene photodetector could be developed in some related fields.