This paper considers games of incomplete information and studies the evolution of the (not necessarily consistent) perceptions of the players using the framework of hypergames. The focus is on developing methods to modify the players' perception about other players' preferences by incorporating the lessons learned from observing their actions. If players are rational, our first update mechanism, called swap learning, is guaranteed to decrease the mismatch between a player's perception and the true payoff structure of other players. However, this method can lead to inconsistencies in the stability properties of the resulting perception. This motivates the introduction of a second update mechanism, called modified swap learning, that is guaranteed to produce a consistent perception. We also identify a class of hypergames for which modified swap is also guaranteed to decrease the mismatch in a player's perception. We introduce the novel notion of H-digraph as a useful tool to encode the information in a hypergame, and fully characterize how this digraph is affected by changes in the players' beliefs.