Skip to main content
SpringerLink
Log in

Human Behavior Analysis from Smartphone Data Streams

  • Conference paper
  • First Online:
Human Behavior Understanding (HBU 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9997))

Included in the following conference series:

Abstract

In the past decade multimedia systems have started including diverse modes of data to understand complex situations and build more sophisticated models. Some increasingly common modes in multimedia are intertwined data streams from sensor modalities such as wearable/mobile, environmental, and biosensors. These data streams offer new information sources to model and predict complex world situations as well as understanding and modeling humans. This paper makes two contributions to the modeling and analysis of multimodal data in the context of user behavior analysis. First, it introduces the use of a concept lattice based data fusion technique for recognizing events. Concept lattices are very effective when enough labeled data samples are not available for supervised machine learning algorithms, but human knowledge is available to develop classification approaches for recognition. Life events encode activities of daily living, and environmental events encode states and state transitions in environmental variables. Second, it introduces a framework that detects frequent co-occurrence patterns as sequential and parallel relations among events from multiple event streams. We show the applicability of our approach in finding interesting human behavior patterns by using longitudinal mobile data collected from 23 users over 1–2 months.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://youtu.be/mmu9EFYFPjE.

References

  1. A data model and format for collecting and distributing eventinformation. https://iptc.org/standards/eventsml-g2/

  2. Aigner, W., Miksch, S., Müller, W., Schumann, H., Tominski, C.: Visualizing time-oriented data - a systematic view. Comput. Graph. 31(3), 401–409 (2007)

    Article  Google Scholar 

  3. Ankerst, M., Jones, D.H., Kao, A., Wang, C.: Datajewel: tightly integrating visualization with temporal data mining. In: VDM@ ICDM, p. 113 (2003)

    Google Scholar 

  4. Bao, L., Intille, S.S.: Activity recognition from user-annotated acceleration data. In: Ferscha, A., Mattern, F. (eds.) Pervasive 2004. LNCS, vol. 3001, pp. 1–17. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24646-6_1

    Chapter  Google Scholar 

  5. Bao, X., Gong, N.Z., Hu, B., Shen, Y., Jin, H.: Connect the dots by understanding user status and transitions. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, pp. 361–366. ACM (2014)

    Google Scholar 

  6. Biegel, G., Cahill, V.,: A framework for developing mobile, context-aware applications. In: Proceedings of the Second IEEE Annual Conference on Pervasive Computing and Communications, PerCom 2004, pp. 361–365. IEEE (2004)

    Google Scholar 

  7. Birkhoff, G., Birkhoff, G., Birkhoff, G., Birkhoff, G.: Lattice Theory, vol. 25. American Mathematical Society, New York (1948)

    MATH  Google Scholar 

  8. Dey, A.K., Abowd, G.D., Salber, D.: A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Hum. Comput. Interact. 16(2), 97–166 (2001)

    Article  Google Scholar 

  9. Ferreira, D., Kostakos, V., Dey, A.K.: AWARE: mobile context instrumentation framework. Front. ICT 2(6), 1–9 (2015)

    Google Scholar 

  10. Ganter, B.: Two Basic Algorithms in Concept Analysis. Springer, New York (2010)

    Book  MATH  Google Scholar 

  11. Gupta, A., Jain, R.: Managing event information: modeling, retrieval, and applications. Synth. Lect. Data Manag. 3(4), 1–141 (2011)

    Article  MathSciNet  Google Scholar 

  12. Gurrin, C., Smeaton, A.F., Doherty, A.R.: Lifelogging: personal big data. Found. Trends Inf. Retrieval 8(1), 1–125 (2014)

    Article  Google Scholar 

  13. Hochheiser, H., Shneiderman, B.: Interactive exploration of time series data. In: Jantke, K.P., Shinohara, A. (eds.) DS 2001. LNCS (LNAI), vol. 2226, pp. 441–446. Springer, Heidelberg (2001). doi:10.1007/3-540-45650-3_38

    Chapter  Google Scholar 

  14. Hudson, V.M., Schrodt, P.A., Whitmer, R.D.: A new kind of social science? Moving ahead with reverse wolfram models applied to event data. In: International Studies Association, Honolulu (2005)

    Google Scholar 

  15. Incel, O.D., Kose, M., Ersoy, C.: A review and taxonomy of activity recognition on mobile phones. BioNanoScience 3(2), 145–171 (2013)

    Article  Google Scholar 

  16. Jalali, L., Jain, R.: Bringing deep causality to multimedia data streams. In Proceedings of the 23rd Annual ACM Conference on Multimedia Conference, pp. 221–230. ACM (2015)

    Google Scholar 

  17. King, A.C., Hekler, E.B., Grieco, L.A., Winter, S.J., Sheats, J.L., Buman, M.P., Banerjee, B., Robinson, T.N., Cirimele, J.: Harnessing different motivational frames via mobile phones to promote daily physical activity and reduce sedentary behavior in aging adults. PloS one 8(4), e62613 (2013)

    Article  Google Scholar 

  18. Kuznetsov, S.O., Obiedkov, S.A.: Comparing performance of algorithms for generating concept lattices. J. Exper. Theoret. Artif. Intell. 14(2–3), 189–216 (2002)

    Article  MATH  Google Scholar 

  19. Liao, L., Fox, D., Kautz, H.: Extracting places and activities from GPS traces using hierarchical conditional random fields. Int. J. Robot. Res. 26(1), 119–134 (2007)

    Article  Google Scholar 

  20. Lin, J., Keogh, E., Wei, L., Lonardi, S.: Experiencing SAX: a novel symbolic representation of time series. Data Mining Knowl. Disc. 15(2), 107–144 (2007)

    Article  MathSciNet  Google Scholar 

  21. Nourine, L., Raynaud, O.: A fast algorithm for building lattices. Inf. Process. Lett. 71(5), 199–204 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  22. Oh, H., Jalali, L., Jain, R.: An intelligent notification system using context from real-time personal activity monitoring. In: 2015 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2015)

    Google Scholar 

  23. Oosthuizen, G.D.: The use of a lattice in knowledge processing (1992)

    Google Scholar 

  24. Plaisant, C., Milash, B., Rose, A., Widoff, S., Shneiderman, B.: Lifelines: visualizing personal histories. In: Proceedings of the SIGCHI Conference on Human Factors Incomputing Systems, pp. 221–227. ACM (1996)

    Google Scholar 

  25. Qiu, Z., Gurrin, C., Doherty, A.R., Smeaton, A.F.: A Real-Time Life Experience Logging Tool. Springer, New York (2012)

    Book  Google Scholar 

  26. Qiu, Z., Gurrin, C., Smeaton, A.F.: Evaluating access mechanisms for multimodal representations of lifelogs. In: Tian, Q., Sebe, N., Qi, G.-J., Huet, B., Hong, R., Liu, X. (eds.) MMM 2016. LNCS, vol. 9516, pp. 574–585. Springer, Heidelberg (2016). doi:10.1007/978-3-319-27671-7_48

    Chapter  Google Scholar 

  27. Scherp, A., Mezaris, V.: Survey on modeling and indexing events in multimedia. Multimedia Tools Appl. 70(1), 7–23 (2014)

    Article  Google Scholar 

  28. Srinivasan, V., Moghaddam, S., Mukherji, A., Rachuri, K.K., Xu, C., Tapia, E.M.: MobileMiner: mining your frequent patterns on your phone. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 389–400. ACM (2014)

    Google Scholar 

  29. Tominski, C.: Event based visualization for user centered visual analysis. Ph.D. thesis, University of Rostock (2006)

    Google Scholar 

  30. Valtchev, P., Missaoui, R., Lebrun, P.: A partition-based approach towards constructing Galois (concept) lattices. Discrete Math. 256(3), 801–829 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  31. Wang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., Zhou, X., Ben-Zeev, D., Campbell, A.T.: StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 3–14. ACM (2014)

    Google Scholar 

  32. Wang, R., Harari, G., Hao, P., Zhou, X., Campbell, A.T.: SmartGPA: how smartphones can assess and predict academic performance of college students. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 295–306. ACM (2015)

    Google Scholar 

  33. Westermann, U., Jain, R.: E - A generic event model for event-centric multimedia data management in eChronicle applications. In: Proceedings of the 22nd International Conference on Data Engineering Workshops. IEEE (2006)

    Google Scholar 

  34. Zenke, F., Agnes, E.J., Gerstner, W.: Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks. Nat. Commun. 6 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of California, Irvine, USA

    Laleh Jalali, Hyungik Oh & Ramesh Jain

  2. Santa Clara University, Santa Clara, USA

    Ramin Moazeni

Authors
  1. Laleh Jalali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyungik Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramin Moazeni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ramesh Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laleh Jalali .

Editor information

Editors and Affiliations

  1. Université Pierre et Marie Curie , Paris, France

    Mohamed Chetouani

  2. University of Pittsburgh , Pittsburgh, Pennsylvania, USA

    Jeffrey Cohn

  3. Bogazici University , Bebek, Istanbul, Turkey

    Albert Ali Salah

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Jalali, L., Oh, H., Moazeni, R., Jain, R. (2016). Human Behavior Analysis from Smartphone Data Streams. In: Chetouani, M., Cohn, J., Salah, A. (eds) Human Behavior Understanding. HBU 2016. Lecture Notes in Computer Science(), vol 9997. Springer, Cham. https://doi.org/10.1007/978-3-319-46843-3_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46843-3_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46842-6

  • Online ISBN: 978-3-319-46843-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation