Skip to main content
SpringerLink
Log in

The Role of Dynamics in Extracting Information Sparsely Encoded in High Dimensional Data Streams

  • Chapter
  • First Online:
Dynamics of Information Systems

Part of the book series: Springer Optimization and Its Applications ((SOIA,volume 40))

Summary

A major roadblock in taking full advantage of the recent exponential growth in data collection and actuation capabilities stems from the curse of dimensionality. Simply put, existing techniques are ill-equipped to deal with the resulting overwhelming volume of data. The goal of this chapter is to show how the use of simple dynamical systems concepts can lead to tractable, computationally efficient algorithms for extracting information sparsely encoded in multimodal, extremely large data sets. In addition, as shown here, this approach leads to nonentropic information measures, better suited than the classical, entropy-based information theoretic measure, to problems where the information is by nature dynamic and changes as it propagates through a network where the nodes themselves are dynamical systems.

This work was supported in part by NSF grants IIS–0713003 and ECCS–0901433, AFOSR grant FA9550–09–1–0253, and the Alert DHS Center of Excellence.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Zaslaver, A., Bren, A., Ronen, M., Itzkovitz, S., Kikoin, I., Shavit, S., Liebermeister, W., Surette, M.G., Alon, U.: A comprehensive library of fluorescent transcriptional reporters for Escherichia coli. Nat. Methods 3, 623–628 (2006)

    Article  Google Scholar 

  2. Lee, J.A., Verleysen, M.: Nonlinear Dimensionality Reduction. Springer, Berlin (2007)

    Book  MATH  Google Scholar 

  3. Costeira, J., Kanade, T.: A multibody factorization method for independently moving objects. Int. J. Comput. Vis. 29, 159–179 (1998)

    Article  Google Scholar 

  4. Vidal, R., Ma, Y., Soatto, S., Sastry, S.: Two–view multibody structure from motion. Int. J. Comput. Vis. 68, 7–25 (2006)

    Article  Google Scholar 

  5. Zelnik-Manor, L., Irani, M.: Degeneracies, dependencies and their implications in multi-body and multi-sequence factorization. In: Proc. of the 2003 IEEE Conf. on Computer Vision and Pattern Recognition, pp. 287–293 (2003)

    Google Scholar 

  6. Ma, Y., Vidal, R.: A closed form solution to the identification of hybrid arx models via the identification of algebraic varieties. In: Hybrid Systems Computation and Control, pp. 449–465 (2005)

    Google Scholar 

  7. Bemporad, A., Garulli, A., Paoletti, S., Vicino, A.: A bounded-error approach to piecewise affine system identification. IEEE Trans. Autom. Control 50, 1567–1580 (2005)

    Article  MathSciNet  Google Scholar 

  8. Saul, L.K., Roweis, S.T.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. J. Mach. Learning Res. 4, 119–155 (2003)

    Article  MathSciNet  Google Scholar 

  9. Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)

    Article  Google Scholar 

  10. Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 1373–1396 (2003)

    Article  MATH  Google Scholar 

  11. Donoho, D.L., Grimes, C.E.: Hessian eigenmaps: locally linear embedding techniques for high-dimensional data. In: Proceedings of the National Academy of Arts and Sciences, vol. 100, pp. 5591–5596 (2003)

    Google Scholar 

  12. Weinberger, K.Q., Sha, F., Saul, L.K.: Learning a kernel matrix for nonlinear dimensionality reduction. In: Proc. of the 2004 International Conference on Machine Learning. ACM, New York (2004)

    Google Scholar 

  13. Weinberger, K.Q., Saul, L.K.: Unsupervised learning of image manifolds by semidefinite programming. In: Proc. of the 2004 IEEE Conf. on Computer Vision and Pattern Recognition, pp. 988–995 (2004)

    Google Scholar 

  14. Ozay, N., Sznaier, M., Lagoa, C., Camps, O.: A sparsification approach to set membership identification of a class of affine hybrid system. In: Proc. 47th IEEE Conf. Dec. Control (CDC), pp. 123–130 (2008)

    Google Scholar 

  15. Sznaier, M., Lagoa, C., Mazzaro, M.C.: An algorithm for sampling balls in ℋ∞ with applications to risk–adjusted performance analysis and model (in)validation. IEEE Trans. Autom. Control 50, 410–416 (2005)

    Article  MathSciNet  Google Scholar 

  16. Sznaier, M., Lagoa, C., Ma, W.: Risk adjusted identification of a class of nonlinear systems. In: Proc. 46 IEEE Conf. Dec. Control., pp. 5117–5122 (2007)

    Google Scholar 

  17. Sánchez Peña, R., Sznaier, M.: Robust Systems Theory and Applications. Wiley, New York (1998)

    Google Scholar 

  18. Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9, 137–154 (1992)

    Article  Google Scholar 

  19. Xiao, J., Chai, J., Kanade, T.: A closed-form solution to non-rigid shape and motion recovery. In: Proc. of the 8th European Conference on Computer Vision (ECCV 2004) (2004)

    Google Scholar 

  20. Vidal, R., Hartley, R.: Motion segmentation with missing data using powerfactorization and gpca. In: Proc. of the 2004 IEEE Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 310–316 (2004)

    Google Scholar 

  21. Fazel, M., Hindi, H., Boyd, S.P.: A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the 2001 American Control Conf. (2001), vol. 6, pp. 4734–4739. AACC, Washington (2001)

    Google Scholar 

  22. Lobo, M., Fazel, M., Boyd, S.: Portfolio optimization with linear and fixed transaction costs. Ann. Oper. Res. 152, 376–394 (2007)

    Article  MathSciNet  Google Scholar 

  23. Gargi, U., Kasturi, R., Strayer, S.H.: Performance characterization of video-shot-change detection methods. IEEE Trans. Circuits Syst. Video Technol. 10, 1–13 (2000)

    Article  Google Scholar 

  24. Yuan, J., Wang, H., Xiao, L., Zheng, W., Li, J., Lin, F., Zhang, B.: A formal study of shot boundary detection. IEEE Trans. Circuits Syst. Video Technol. 17, 168–186 (2007)

    Article  Google Scholar 

  25. Osian, M., Van Gool, L.: Video shot characterization. Mach. Vis. Appl. 15, 172–177 (2004)

    Article  Google Scholar 

  26. Petrovic, N., Ivanovic, A., Jojic, N.: Recursive estimation of generative models of video. In: Proc. of the 2006 IEEE Conf. on Computer Vision and Pattern Recognition, pp. 79–86 (2006)

    Google Scholar 

  27. Lu, L., Vidal, R.: Combined central and subspace clustering for computer vision applications. In: Proc. of the 2006 International Conference on Machine Learning, pp. 593–600 (2006)

    Google Scholar 

  28. Rand, W.: Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66, 846–850 (1971)

    Article  Google Scholar 

  29. Doretto, G., Chiuso, A., Wu, Y., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51, 91–109 (2003)

    Article  MATH  Google Scholar 

  30. Chan, A.B., Vasconcelos, N.: Mixtures of dynamic textures. In: Proc. of the 2005 International Conference on Computer Vision (ICCV), vol. 1, pp. 641–647 (2005)

    Google Scholar 

  31. Cooper, L., Liu, J., Huang, K.: Spatial segmentation of temporal texture using mixture linear models. In: Proceedings of the Dynamical Vision Workshop in the International Conference of Computer Vision, pp. 142–150 (2005)

    Google Scholar 

  32. Ghoreyshi, A., Vidal, R.: Segmenting dynamic textures with using descriptors, arx models and level sets. In: Proceedings of the Dynamical Vision Workshop in the European Conference of Computer Vision, pp. 127–141 (2006)

    Google Scholar 

  33. Vandenberghe, L., Boyd, S.P.: Semidefinite programming. SIAM Rev. 38(1), 49–95 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  34. Fazel, M., Hindi, H., Boyd, S.P.: Log-det heuristic for matrix rank minimization with applications to Hankel and euclidean distance matrices. In: Proceedings of the 2003 American Control Conf., vol. 3, pp. 2156–2162. AACC, Washington (2003)

    Chapter  Google Scholar 

  35. Sznaier, M., Mazzaro, C., Camps, O.I.: Open-loop worst-case identification of nonschur plants. Automatica 39, 1019–1025 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  36. Chen, J., Gu, G.: Control Oriented System Identification, an ℋ∞ Approach. Wiley, New York (2000)

    Google Scholar 

  37. Donoho, D.: High-dimensional data analysis: The curses and blessings of dimensionality. In: AMS Conf. on Math Challenges of the 21st Century (2000). Plenary presentation

    Google Scholar 

  38. Candes, E., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59, 1207–1223 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  39. Donoho, D.: Compressed sensing. IEEE Trans. Information Theory 52, 1289–13006 (2006)

    Article  MathSciNet  Google Scholar 

  40. Carin, L., Liu, D., Guo, B.: In situ compressive sensing. Inverse Probl. 24, 015023 (2008)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ECE Department, Northeastern University, Boston, MA, 02115, USA

    Mario Sznaier, Octavia Camps, Necmiye Ozay, Gilead Tadmor & Dana Brooks

  2. Department of Electrical Engineering, Penn State University, University Park, PA, 16802, USA

    Tao Ding

Authors
  1. Mario Sznaier
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Octavia Camps
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Necmiye Ozay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gilead Tadmor
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dana Brooks
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mario Sznaier .

Editor information

Editors and Affiliations

  1. Raytheon, Inc., Pelham Road 3323, Orlando, 32803, U.S.A.

    Michael J. Hirsch

  2. Dept. Industrial & Systems, University of Florida, Weil Hall 303, Gainesville, 32611-6595, USA

    Panos M. Pardalos

  3. Munitions Directorate, GNC Branch, W. Eglin Blvd 101, Eglin AFB, 32542, U.S.A.

    Robert Murphey

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Sznaier, M., Camps, O., Ozay, N., Ding, T., Tadmor, G., Brooks, D. (2010). The Role of Dynamics in Extracting Information Sparsely Encoded in High Dimensional Data Streams. In: Hirsch, M., Pardalos, P., Murphey, R. (eds) Dynamics of Information Systems. Springer Optimization and Its Applications, vol 40. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-5689-7_1

Download citation

Publish with us

Policies and ethics

Search

Navigation