In this paper, the alternating direction method is proposed to perform heliostat field layout optimization, which includes two new strategies: (1) pattern-free layout optimization using the proposed physics-based gradient of optical efficiencies, and (2) new initial designs obtained by changing a scaling factor (k). To obtain the physics-based gradient, insolation weighted optical efficiency is divided into two terms - shading/blocking efficiency and non-shading/blocking efficiency, and for more efficient and accurate layout optimization, the proposed alternating direction method performs layout optimization along alternating directions obtained from the physics-based gradient. The proposed alternating direction method is validated using two field layouts - PS10 and a field layout obtained by SolarPILOT - with two objective functions, optical efficiency and levelized cost of electricity (LCOE). The validation results show that both the insolation weighted optical efficiency and LCOE are much improved by the proposed optimization method.