This technical note presents a convergence criterion for infinite products of stochastic matrices which is based on graphical decomposition of the associated graphs. We show that if the associated graphs of a set of stochastic matrices share a common graphical decomposition and the corresponding reduced graphs are rooted, then any infinite products of the given set of stochastic matrices is convergent. Specifically, we propose a numerical algorithm for finding the common graphical decomposition of the associated graphs, which has been proved to be polynomial-time fast. The proposed criterion can be applied directly to a series of classical results in distributed coordination algorithm.