A Method for Online Analysis of Structured Processes Using Bayesian Filters and Echo State Networks

  • Dimitrios I. Kosmopoulos
  • Fillia Makedon
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7585)


We propose a Bayesian filtering framework for online analysis of visual structured processes, which can be combined with the Echo State Network (ESN) to capture prior information. With the proposed method we mitigate the effective Markovian Behavior of the ESN. We are able to keep a set of hypotheses about the entire history of behaviors and to evaluate them online based on new observations. The performance is evaluated under two complex visual behavior understanding scenarios using public datasets: a visual process for a kitchen table preparation and a real life manufacturing process.


Hide Markov Model Gaussian Mixture Model Zernike Moment Online Analysis Task Duration 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Kosmopoulos, D., Chatzis, S.: Robust visual behavior recognition. IEEE Signal Processing Magazine 27, 34–45 (2010)CrossRefGoogle Scholar
  2. 2.
    Veres, G., Grabner, H., Middleton, L., Van Gool, L.: Automatic Workflow Monitoring in Industrial Environments. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 200–213. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  3. 3.
    Padoy, N., Mateus, D., Weinland, D., Berger, M.O., Navab, N.: Workflow Monitoring based on 3D Motion Features. In: Workshop on Video-Oriented Object and Event Classification in Conj. with ICCV 2009, Kyoto, pp. 585–592. IEEE (2009)Google Scholar
  4. 4.
    Behera, A., Cohn, A.G., Hogg, D.C.: Workflow Activity Monitoring Using Dynamics of Pair-Wise Qualitative Spatial Relations. In: Schoeffmann, K., Merialdo, B., Hauptmann, A.G., Ngo, C.-W., Andreopoulos, Y., Breiteneder, C. (eds.) MMM 2012. LNCS, vol. 7131, pp. 196–209. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  5. 5.
    Lalos, C., Voulodimos, A., Doulamis, A., Varvarigou, T.: Efficient tracking using a robust motion estimation technique. Mult. Tools and Applications, 1–16 (2012)Google Scholar
  6. 6.
    Jaeger, H.: The echo state approach to analysing and training recurrent neural networks. Technical Report GMD 148, German National Research Center for Information Technology (2001)Google Scholar
  7. 7.
    Gallicchio, C., Micheli, A.: Architectural and markovian factors of echo state networks. Neural Networks 24, 440–456 (2011)CrossRefGoogle Scholar
  8. 8.
    Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77, 257–286 (1989)CrossRefGoogle Scholar
  9. 9.
    Eickeler, S., Kosmala, A., Rigoll, G.: Hidden markov model based continuous online gesture recognition. In: ICPR, pp. 1206–1208 (1998)Google Scholar
  10. 10.
    Lv, F., Nevatia, R.: Recognition and Segmentation of 3-D Human Action Using HMM and Multi-class AdaBoost. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part IV. LNCS, vol. 3954, pp. 359–372. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Hoai, M., Lan, Z.Z., De la Torre, F.: Joint segmentation and classification of human actions in video. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2011)Google Scholar
  12. 12.
    Fine, S., Singer, Y., Tishby, N.: The hierarchical hidden markov model: Analysis and applications. Machine Learning 32, 41–62 (1998)zbMATHCrossRefGoogle Scholar
  13. 13.
    Oliver, N., Garg, A., Horvitz, E.: Layered representations for learning and inferring office activity from multiple sensory channels. Comput. Vis. Image Underst. 96, 163–180 (2004)CrossRefGoogle Scholar
  14. 14.
    Xiaoling, X., Layuan, L.: Real time analysis of situation events for intelligent surveillance. In: International Symposium on Computational Intelligence and Design, ISCID 2008, vol. 2, pp. 122–125 (2008)Google Scholar
  15. 15.
    Kosmopoulos, D.I., Doulamis, N.D., Voulodimos, A.S.: Bayesian filter based behavior recognition in workflows allowing for user feedback. Computer Vision and Image Understanding 3, 422–434 (2012)CrossRefGoogle Scholar
  16. 16.
    Zhang, D., Ning, X., Liu, X.: Smc method for online prediction in hidden markov models. Kybernetes 38, 1819–1827 (2009)CrossRefGoogle Scholar
  17. 17.
    Fei, H.: A hybrid HMM/particle filter framework for non-rigid hand motion recognition. In: ICASSP, vol. 5, pp. V-889–V-892 (2004)Google Scholar
  18. 18.
    Jaeger, H., Maass, W., Principe, J.: Special issue on echo state networks and liquid state machines. Neural Networks 20, 287–289 (2007)CrossRefGoogle Scholar
  19. 19.
    Skowronski, M.D., Harris, J.G.: Automatic speech recognition using a predictive echo state network classifier. Neural Networks 20, 414–423 (2007)zbMATHCrossRefGoogle Scholar
  20. 20.
    Venayagamoorthy, G.K.: Online design of an echo state network based wide area monitor for a multimachine power system. Neural Networks 20, 404–413 (2007); Echo State Networks and Liquid State MachinesGoogle Scholar
  21. 21.
    Voulodimos, A., Kosmopoulos, D., Veres, G., Grabner, H., Van Gool, L., Varvarigou, T.: 2011 special issue: Online classification of visual tasks for industrial workflow monitoring. Neural Netw. 24, 852–860 (2011)CrossRefGoogle Scholar
  22. 22.
    Arnaud, D., Simon, G., Christophe, A.: On sequential monte carlo sampling methods for bayesian filtering. Statistics and Computing 10, 197–208 (2000)CrossRefGoogle Scholar
  23. 23.
    Voulodimos, A., et al.: A dataset for workflow recognition in industrial scenes. In: IEEE Int. Conference on Image Processing, pp. 3310–3313 (2011)Google Scholar
  24. 24.
    Tenorth, M., Bandouch, J., Beetz, M.: The TUM Kitchen Data Set of Everyday Manipulation Activities for Motion Tracking and Action Recognition. In: THEMIS Workshop, In conj. with ICCV (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dimitrios I. Kosmopoulos
    • 1
  • Fillia Makedon
    • 1
  1. 1.Computer Science and EngineeringUniversity of Texas at ArlingtonUSA

Personalised recommendations