We propose methods for online analysis of visual structured processes. These methods can be combined with Hidden Markov Models or Echo State Networks (ESN) to capture prior information. With the proposed method we mitigate the effective Markovian Behavior of the ESN and the HMM. We are able to keep a set of hypotheses about the entire history of behaviors and to evaluate them online based on new observations.
The performance is evaluated under several complex visual behavior understanding scenarios using public datasets: a visual process for a kitchen table preparation and a real life manufacturing process.
Related paper:
A similar approach has been followed using instead of an ESN a Hidden Markov Model to generate the observation likelihoods. In addition, we introduced a novel readjustment framework of behavior recognition and classification by incorporating the user’s feedback into the learning process. The proposed approach aims at dynamically correcting erroneous classification results to enhance the behavior modeling and therefore the overall classification rates.
Related paper:
We have also proposed methods for assignment of labels after the workflow is over and which assumes automated segmentation of the tasks that compose the workflow. The assignment of tasks is based on an objective function, which is optimized by using a genetic algorithm.
Related paper:
Alternatively to the previous approach we used a Hopfield network for the optimization. We also extracted keyframes for efficient creation of summaries and easier browsing based on the video content.
Related paper:
