Hidden Markov models provide a natural way of integrating
audio and visual information with either an early or a late
strategy. Early HMM integration methods are characterized,
in the decoding process, by state-by-state estimation of observation
probabilities based on audio and visual evidence. That is, in traversing the Viterbi lattice, both modalities are
considered in determining the most likely branch leading to
each node.