Markov chains (MC) are black-box models ideal for describing non-linear stochastic digitised systems. Although the identification of their parameters can be a relatively easy task to perform, the dimensionality involved can become undesirably large. This significant drawback can be overcome by exploiting the smoothness of the underlying physical system. This paper presents the realisation of this strategy by using a novel, hybrid, off-line algorithm to locate areas which merit detailed model description. Such an algorithm comprises of Bayesian parameter estimation and the Mean-Tracking clustering algorithm, the results of which will provide all the necessary information for the construction of on-line adaptive predictors/controllers based on MC.