Brownian dynamics simulations under large amplitude oscillatory shear flow at an intermediate volume fraction in both hard and soft sphere systems have been carried out. Elastic and viscous stresses for the two systems are calculated by using the stress decomposition method. Careful investigation of "double peaks," which are experimentally observed only in the elastic stress of hard sphere systems, has been conducted. When comparing hard and soft sphere systems in simulation, double peaks are observed only in hard sphere systems, within a specific strain amplitude range. The structures of hard sphere systems at different strain amplitudes, where double peaks appear and not, are compared. Excess entropy concept is adopted to evaluate the extent of particle alignment during the cycle. According to the structural analyses, double peaks are created when the structural difference between maximum-ordered and minimum-ordered states is large during 1 cycle.