In this paper, a distributed trilayer multi-agent framework is proposed for optimal electric vehicle charging scheduling (EVCS). The framework reduces the negative effects of electric vehicle charging demand on the electrical grids. To solve the scheduling problem, a novel hierarchical distributed EV charging scheduling (HDEVCS) is developed as the exchange problem, where the agents are clustered based on their coupling constraints. According to the separability of the agents' objectives and the clusters' coupled constraints, HDEVCS is solved efficiently in a distributed manner by the alternating direction method of multipliers (ADMM). Comparing to the exiting trilayer methods, HDEVCS reduces the convergence time and the iteration numbers since its structure allows the agents to update their primal optimization variable simultaneously. The performance of HDEVCS is evaluated by numerical simulation of two small- and large-scale case studies consisting of 306 and 9051 agents, respectively. The results verify the scalability and efficiency of the proposed method, as it reduces the convergence time and iteration numbers by 60% compared to the state-of-the-art methods, flattens the load profile and decreases the charging cost considerably without violating the grid feeders' capacity. The significant outcome of our method is the accommodation of a large EV population without investment in grid expansion.