Skip to main content
SpringerLink
Log in

Design principles of a stream-based framework for mobility analysis

  • Published:
GeoInformatica Aims and scope Submit manuscript

Abstract

Trajectory analysis is of crucial importance in several fields as social analysis, zoology, climatology or traffic monitoring. Over the last decade, the number of mobile systems and devices recording their positions has grown significantly generating a deluge of spatial and temporal data to analyze. This increasing volume of data raises numerous issues in terms of storage, processing and extraction of information. Previous works considering movement analysis have been mainly oriented towards either archived data processing and mining or continuous handling of incoming streams. The research developed in this pa- per introduces the design principles of a holistic approach combining real-time processing and archived data analysis to process mobility data “on the fly”. This solution aims to provide better results comparing to both purely offline and online approaches. This research considers distributed data and processing to be more efficient. The design principles are applied to maritime traffic analysis and a few representative examples are introduced to demonstrate the relevance of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Pallotta G, Vespe M, Bryan K (2013) Vessel pattern knowledge discovery from AIS data: a framework for anomaly detection and route prediction. Entropy 15(6):2218–2245

    Article  Google Scholar 

  2. Giannotti F, Nanni M, Pedreschi D, Pinelli F, Renso C, Rinzivillo S, Trasarti R (2011) Unveiling the complexity of human mobility by querying and mining massive trajectory data. VLDB J 20(5):695–719

    Article  Google Scholar 

  3. Shekhar S, Gunturi V, Evans MR et al. (2012) Spatial big-data challenges intersect- ing mobility and cloud computing. Proc Eleventh ACM Int Workshop Data Eng Wireless Mobile Access, MobiDE ’12, 1–6, New York, NY, USA. ACM

  4. Anselin L (1989) What is special about spatial data? alternative perspectives on spatial data analysis 63–77

  5. Vatsavai RR, Ganguly A, Chandola V et al. (2012) Spatiotemporal data mining in the era of big spatial data: algorithms and applications. Proc 1st ACM SIGSPATIAL Int Workshop Anal Big Geospatial Data, BigSpatial ’12, 1-10, New York, NY, USA. ACM

  6. Nguyen-Dinh L-V, Aref WG, Mokbel MF (2010) Spatio-temporal access methods: part 2 (2003 - 2010). IEEE Data Eng Bull 33(2):46–55

    Google Scholar 

  7. Patroumpas K (2013) Multi-scale window specification over streaming trajectories. J Spatial Inform Sci 7(1):45–75

    Google Scholar 

  8. Dean J, Ghemawat S (2004) Mapreduce: simplified data processing on large clusters. Proceedings of the 6th Conference on Symposium on Opearting Systems Design & Implementation - volume 6, OSDI’04. USENIX Association, Berkeley, p 10

    Google Scholar 

  9. Eldawy A, Mokbel MF (2015) The era of big spatial data. 31st IEEE Int Conf Data Eng Workshops, ICDE Workshops 2015, Seoul, South Korea 42–49

  10. Aji A, Wang F, Vo H, Lee R, Liu Q, Zhang X, Saltz J (2013) Hadoop gis: a high performance spatial data warehousing system over mapreduce. Proc VLDB Endow 6(11):1009–1020

    Article  Google Scholar 

  11. Lu J, G¨ uting RH (2013) Parallel SECONDO: practical and efficient mobility data processing in the cloud. Proc 2013 I.E. Int Conf Big Data, 6-9 October 2013, Santa Clara, CA, USA, 17–25

  12. Pelekis N, Theodoridis Y, Vosinakis S et al. (2006) Hermes - a framework for location-based data management. In Proc EDBT 1130–1134

  13. Mokbel MF, Xiong X, Hammad MA et al. (2005) Continuous query processing of spatio-temporal data streams in place. Geoinformatica 343–365

  14. Forlizzi L, G¨ uting RH, Nardelli E et al. (2000) A data model and data structures for moving objects databases. Proc 2000 ACM SIGMOD Int Conf Manag Data, SIGMOD ’00, 319–330, New York, NY, USA. ACM

  15. de Almeida VT, Guting RH, Behr T et al. (2006) Querying moving objects in secondo. Proc 7th Int Conf Mobile Data Manag, MDM ’06, pages 47–52. IEEE Computer Society

  16. Giannotti F, Nanni M, Pinelli F et al. (2007) Trajectory pattern mining. Proc 13th ACM SIGKDD Int Conf Knowledge Discov Data Mining, KDD ’07, 330–339. ACM

  17. Ma Q, Yang B, Qian W et al. (2009) Query processing of massive trajectory data based on mapreduce. Proc First Int CIKM Workshop Cloud Data Manag, CloudDb 2009, Hong Kong, China, November 2, 2009, 9–16

  18. Golab L, Ozsu MT (2003) Issues in data stream management. SIGMOD Rec., 5–14

  19. Yu Z, Liu Y, Yu X, Pu KQ (2015) Scalable distributed processing of K nearest neighbor queries over moving objects. IEEE Trans Knowl Data Eng 27(5):1383–1396

    Article  Google Scholar 

  20. Chandrasekaran S, Franklin M (2004) Remembrance of streams past: overload-sensitive management of archived streams. Proc Thirtieth Int Conf Very Large Data Bases, VLDB ’04, 348–359

  21. Dindar N, Lau BGP, Zal A et al. (2009) Dejavu: declarative pattern matching over live and archived streams of events. In Etintemel U, Zdonik SB, Kossmann D, Tatbul N, editors, SIGMOD Conference, pages 1023-1026. ACM

  22. Marz N (2013) Big data : principles and best practices of scalable realtime data systems. O’Reilly Media, [S.l.]

  23. Golab L, Johnson T (2014) Data stream warehousing. IEEE 30th Int Conf Data Eng, Chicago, ICDE 2014, IL, USA 1290–1293

  24. Condie T, Conway N, Alvaro P et al. (2010) Mapre- duce online. Proc 7th USENIX Conf Networked Syst Design Implement, NSDI'10, 21, Berkeley, CA, USA, 2010. USENIX Association

  25. Lam W, Liu L, Prasad S, Rajaraman A, Vacheri Z, Doan A (2012) Muppet: Mapreduce-style processing of fast data. Proc VLDB Endow 5(12):1814–1825

    Article  Google Scholar 

  26. Olston C, Chiou G, Chitnis L et al. (2011) Nova: continuous pig/hadoop workflows. In Proc 2011 ACM SIGMOD Int Conf Manag Data, SIGMOD ’11, pages 1081-1090, New York, NY, USA. ACM

  27. Zaharia M, Chowdhury M, Franklin MJ et al. (2010) Spark: cluster computing with working sets. 2nd USENIX Workshop Hot Topics Cloud Comput, HotCloud’10, Boston, MA, USA

  28. Zaharia M, Chowdhury M, Das T et al. (2012) Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. Proc 9th USENIX Conf Networked Syst Design Implement, NSDI’12, 2-2, Berkeley, CA, USA. USENIX Association

  29. Arasu A, Babcock B, Babu S et al (2004) Stream: the Stanford data stream management system. Technical Report 2004-20, Stanford InfoLab

  30. Zaharia M, Das T, Li H et al. (2013) Discretized streams: fault-tolerant streaming computation at scale. Proc Twenty-Fourth ACM Symp Oper Syst Principles, SOSP ’13, 423–438, New York, NY, USA. ACM

  31. Boykin PO, Ritchie S, O’Connell I, Lin J (2014) Summingbird: a framework for integrating batch and online mapreduce computations. PVLDB 7(13):1441–1451

    Google Scholar 

  32. Alexandrov A, Bergmann R, Ewen S, Freytag J, Hueske F, Heise A, Kao O, Leich M, Leser U, Markl V, Naumann F, Peters M, Rheinl¨ ander A, Sax MJ, Schelter S, Hoger M, Tzoumas K, Warneke D (2014) The stratosphere platform for big data ana- lytics. VLDB J 23(6):939–964

    Article  Google Scholar 

  33. Ewen S, Schelter S, Tzoumas S et al. (2013) Iterative parallel data processing with stratosphere: an inside look. Proc ACM SIGMOD Int Conf Manag Data, SIGMOD 2013, New York, NY, USA 1053–1056

  34. Ewen S, Tzoumas K, Kaufmann M et al. (2012) Spinning fast iterative data flows. CoRR, abs/1208.0088

  35. Hueske F, Peters M, Sax M et al. (2012) Opening the black boxes in data flow optimization. CoRR, abs/1208.0087

  36. Hueske F, Krettek A, Tzoumas K et al. (2013) Enabling operator reordering in data flow programs through static code analysis. CoRR, abs/1301.4200

  37. Technical characteristics for an automatic identification system using time division multiple access in the VHF maritime mobile frequency band. Recommendation ITU-R M.1371-5 (02/2014), 2014

  38. Vodas M, Pelekis N, Theodoridis Y, Ray C, Karkaletsis V, Petridis S, Miliou A (2013) Efficient ais data processing for environmentally safe shipping. SPOUDAI J Econ Bus 63(3-4):181–190

    Google Scholar 

  39. Ghanem TM, Elmagarmid AK, Larson P et al. (2010) Supporting views in data stream management systems. ACM Trans. Database Syst 35(1)

  40. Deshpande A, Ives Z, Raman V (2007) Adaptive query processing. Found Trends Databases 1(1):1–140

    Article  Google Scholar 

  41. Sakr MA, G¨ uting RH (2014) Group spatiotemporal pattern queries. GeoInformatica 18(4):699–746

    Article  Google Scholar 

  42. Abadi DJ, Carney D, Cetintemel U et al (2003) Aurora: a data stream management system. Proc 2003 ACM SIGMOD Int Conf Manag Data, San Diego, California, USA 666

  43. Shah MA, Hellerstein JM, Brewer EA et al. (2004) Highly-available, fault-tolerant, parallel dataflows. Proc ACM SIGMOD Int Conf Manag Data, Paris, France 827–838

  44. Sun X, Yaagoub A, Trajcevski G et al. (2013) P2est: parallelization philosophies for evaluating spatio-temporal queries. Proc 2nd ACM SIGSPATIAL Int Workshop Anal Big Geospatial Data, BigSpatial@SIGSPATIAL 2013, Orlando, FL, USA 47–54

  45. Patroumpas K, Sellis TK (2004) Managing trajectories of moving objects as data streams. Spatio-Temporal Database Manag, 2nd Int Workshop STDBM’04, Toronto, Canada 41–48

  46. Potamias M, Patroumpas K, Sellis TK et al. (2006) Sampling trajectory streams with spatiotemporal criteria. 18th Int Conf Scientific Statistical Database Manag, SSDBM 2006, Vienna, Austria, Proceedings 275–284

  47. Patroumpas K (2013) Multi-scale window specification over streaming trajectories. J Spatial Inform Sci 45–75

  48. Potamias M, Patroumpas K, Sellis TK et al. (2007) Online amnesic summarization of stream- ing locations. Adv Spatial Temp Databases, 10th Int Symp, SSTD 2007, Boston, MA, USA, Proceedings, 148–166

  49. Li Z (2014) Spatiotemporal pattern mining: algorithms and applications. Frequent Pattern Mining 283–306

  50. Chandrasekaran S, Franklin MJ (2003) Psoup: a system for streaming queries over streaming data. VLDB J 12(2):140–156

    Article  Google Scholar 

  51. Mokbel MF, Xiong X, Aref W et al. (2004) SINA: scalable incremental processing of continuous queries in spatio-temporal databases. Proc ACM SIGMOD Int Conf Manag Data, Paris, France 623–634

  52. Ghanem TM, Aref WG, Elmagarmid AK (2006) Exploiting predicate-window semantics over data streams. SIGMOD Record 35(1):3–8

    Article  Google Scholar 

  53. Xiong X, Elmongui HG, Chai X et al. (2007) Place: A distributed spatio- temporal data stream management system for moving objects. 8th Int Conf Mobile Data Manag (MDM 2007), Mannheim, Germany 44–51

  54. Mokbel MF, Aref WG (2008) SOLE: scalable on-line execution of continuous queries on spatio-temporal data streams. VLDB J 17(5):971–995

    Article  Google Scholar 

  55. Nehme RV, Rundensteiner EA (2006) SCUBA: scalable cluster-based algorithm for evaluating continuous spatio-temporal queries on moving objects. Adv Database Technol - EDBT 2006, 10th Int Conf Extend Database Technol, Munich, Germany, March 26-31, 2006, Proceedings, 1001–1019

  56. Zhang C, Huang Y, Grifn T et al. (2009) Querying geospatial data streams in SECONDO. 17th ACM SIGSPATIAL Int Symp Adv Geographic Inform Syst, ACM-GIS 2009, Seattle, Washington, USA, Proceedings 544–545

  57. Galic Z, Baranovic M, Krizanovic K, Meskovic E (2014) Geospatial data streams: formal framework and implementation. Data Knowl Eng 91:1–16

    Article  Google Scholar 

  58. Kazemitabar SJ, Demiryurek U, Ali MH, Akdogan A, Shahabi C (2010) Geospatial stream query processing using microsoft SQL server streaminsight. PVLDB 3(2):1537–1540

    Google Scholar 

  59. Ali MH, Gerea C, Raman BS, Sezgin B, Tarnavski T, Verona T, Wang P, Zab- back P, Kirilov A, Ananthanarayan A, Lu M, Raizman A, Krishnan R, Schindlauer R, Grabs T, Bjeletich S, Chandramouli B, Goldstein J, Bhat S, Li Y, Nicola VD, Wang X, Maier D, Santos I, Nano O, Grell S (2009) Microsoft CEP server and online behavioral targeting. PVLDB 2(2):1558–1561

    Google Scholar 

  60. Biem A, Bouillet E, Feng H et al. (2010) IBM infosphere streams for scalable, real-time, intelligent transportation services. Proc ACM SIGMOD Int Conf Manag Data, SIGMOD 2010, Indianapolis, Indiana, USA, June 6-10, 2010, pages 1093–1104

  61. Wolf JL, Bansal N, Hildrum K et al. (2008) SODA: an optimizing scheduler for large-scale stream-based distributed computer systems. Middleware 2008, ACM/IFIP/USENIX 9th Int Middleware Conf, Leuven, Belgium, Proceedings 306–325

  62. Khandekar R, Hildrum K, Parekh S et al. (2009) COLA: optimizing stream processing applications via graph partitioning. Middleware 2009, ACM/IFIP/USENIX, 10th Int Middleware Conf, Urbana, IL, USA, November 30 - December 4, 2009. Proceedings, 308–327

  63. Neumeyer L, Robbins B, Nair A et al. (2010) S4: distributed stream computing platform. Proc 2010 I.E. Int Conf Data Mining Workshops, ICDMW ’10, pages 170-177. IEEE Computer Society

  64. Garz A, Benczr AA, Sidl CI et al. (2013) Real-time streaming mobility analytics. In Hu X, Lin TY, Raghavan V, Wah BW, Baeza- Yates RA, Fox G, Shahabi C, Smith M, Q. Y. 0001, Ghani R, Fan W, Lempel R, Nambiar R, editors, BigData Conference, 697–702. IEEE

  65. Xiong X, Mokbel MF, Aref WG et al. (2005) SEA-CNN: scalable processing of continuous k-nearest neighbor queries in spatio-temporal databases. Proc 21st Int Conf Data Eng, ICDE 2005, Tokyo, Japan 643–654

  66. Kalashnikov DV, Prabhakar S, Hambrusch SE et al. (2002) Efficient evaluation of continuous range queries on moving objects. Database Expert Syst Applic, 13th Int Conf, DEXA 2002, Aix-en-Provence, France, September 2-6, 2002, Proceedings, 731–740

  67. Mokbel MF, Aref WG (2005) Gpac: generic and progressive processing of mobile queries over mobile data. Proc 6th Int Conf Mobile Data Manag, MDM ’05, 155–163, New York, NY, USA. ACM

  68. Deng K, Xie K, Zheng K et al. (2011) Trajectory indexing and retrieval. Comput Spatial Traject 35–60

  69. Patroumpas K, Artikis A, Katzouris N et al. (2015) Event recognition for maritime surveillance. Proc 18th Int Conf Extend Database Technol, EDBT 2015, Brussels, Belgium 629–640

  70. Balazinska M, Kwon Y, Kuchta N et al. (2007) Moirae: history-enhanced monitoring. CIDR 2007, Third Biennial Conf Innov Data Syst Res, Asilomar, CA, USA, January 7-10, 2007, Online Proceedings, pages 375–386

  71. Etienne L, Devogele T, Bouju A (2012) Spatio-temporal trajectory analysis of mobile objects following the same itinerary. Adv Geo-Spatial Inform Sci 10:47–57

    Google Scholar 

  72. Devogele T, Etienne L, Ray C et al. (2013) Maritime monitoring. Mobility Data: Model, Manag, Understand 221–239

  73. Han J, Pei J, Yin Y et al. (2000) Mining frequent patterns without candidate generation. Proc 2000 ACM SIGMOD Int Conf Manag Data, May 16-18, 2000, Dallas, Texas, USA., 1–12

  74. Morzy M (2007) Mining frequent trajectories of moving objects for location prediction. Mach Learn Data Mining Pattern Recognition, 5th Int Conf, MLDM 2007, Leipzig, Germany 2007, Proceedings, 667–680

  75. Le Guyader D, Ray C, Brosset D et al. (2016) Defining fishing grounds variability with Automatic Identification System (AIS) data. 2nd Int Workshop Maritime Flows Networks (WIMAKS’16), Paris, 2527, 2 pages

  76. Hammad MA, Mokbel MF, Ali MH et al. (2004) Nile: a query processing engine for data streams. Proc 20th Int Conf Data Eng, ICDE 2004, 30 March - 2 April 2004, Boston, MA, USA, 851

  77. Hammad MA, Franklin MJ, Aref WG et al. (2003) Scheduling for shared window joins over data streams. VLDB 297–308

  78. Hammad MA, Aref WG, Elmagarmid AK (2008) Query processing of multi-way stream window joins. VLDB J 17(3):469–488

    Article  Google Scholar 

  79. Ghanem TM, Hammad MA, Mokbel MF, Aref WG, Elmagarmid AK (2007) In- cremental evaluation of sliding-window queries over data streams. IEEE Trans Knowl Data Eng 19(1):57–72

    Article  Google Scholar 

  80. Elmongui HG, Mokbel MF, Aref WG et al. (2005) Spatio-temporal histograms. Adv Spatial Temp Databases, 9th Int Symp, SSTD 2005, Angra dos Reis, Brazil, August 22-24, 2005, Proceedings, pages 19–36

  81. Huang Y, Zhang C (2008) New data types and operations to support geo-streams. Geographic Inform Sci, 5th Int Conf, GIScience 2008, Park City, UT, USA, September 23-26, 2008. Proceedings, pages 106–118

  82. Huang Y, Zhang C (2009) Interval-based nearest neighbor queries over sliding windows from trajectory data. MDM 2009, Tenth Int Conf Mobile Data Manag, Taipei, Taiwan, 18-20 May 2009, 212–221

  83. Chandrasekaran S, Cooper O, Deshpande A et al. (2003) Telegraphcq: continuous dataflow processing. Proc 2003 ACM SIGMOD Int Conf Manag Data, San Diego, California, USA, June 9-12, 2003, page 668

  84. Pelekis N, Frentzos E, Giatrakos N et al. (2008) Hermes: aggregative lbs via a trajectory db engine. Proc 2008 ACM SIGMOD Int Conf Manag Data, SIGMOD ’08, 1255–1258, New York, NY, USA. ACM

  85. Avnur R, Hellerstein JM (2000) Eddies: continuously adaptive query processing. Proc 2000 ACM SIGMOD Int Conf Manag Data, May 16-18, 2000, Dallas, Texas, USA., 261–272

  86. Urhan T, Franklin MJ (2001) Dynamic pipeline scheduling for improving interactive query performance. VLDB 2001, Proc 27th Int Conf Very Large Data Bases, Roma, Italy 501–510

  87. Patroumpas K, Sellis TK (2011) Subsuming multiple sliding windows for shared stream computation. Adv Databases Inform Syst - 15th Int Conf, ADBIS 2011, Vienna, Austria. Proceedings, pages 56–69

  88. Patroumpas K, Sellis TK (2010) Multi-granular time-based sliding windows over data streams. TIME 2010 - 17th Int Symp Temporal Represent Reason, Paris, France 146–153

  89. Shah MA, Hellerstein JM, Chandrasekaran S et al. (2003) Flux: an adaptive partitioning operator for continuous query systems. Proc 19th Int Conf Data Eng, Bangalore, India 25–36

  90. Rundensteiner EA, Ding L, Sutherland TM et al. (2004) CAPE: continuous query engine with heterogeneous-grained adaptivity. Proc Thirtieth Int Conf Very Large Data Bases, Toronto, Canada, 1353–1356

  91. Zhu Y, Rundensteiner EA, Heineman GT et al. (2004) Dynamic plan migration for con- tinuous queries over data streams. Proc ACM SIGMOD Int Conf ManagData, Paris, France, 431–442

  92. Sutherland TM, Zhu Y, Ding L et al. (2005) An adaptive multi- objective scheduling selection framework for continuous query processing. Ninth Int Database Eng Appl Symp (IDEAS 2005), Montreal, Canada 445–454

  93. Nehme RV, Rundensteiner EA (2007) ClusterSheddy : load shedding using mov- ing clusters over spatio-temporal data streams. Adv Databases: Concepts, Syst Appl, 12th Int Conf Database Syst Adv Appl, DASFAA 2007, Bangkok, Thailand, April 9-12, 2007, Proceedings, 637–651

  94. Sutherland TM, Liu B, Jbantova M et al. (2005) D-CAPE: distributed and self-tuned continuous query processing. Proc 2005 ACM CIKM Int Conf Inform Knowledge Manag, Bremen, Ger- many 217–218

  95. Miller J, Raymond M, Archer J et al. (2011) An extensibility approach for spatio-temporal stream processing using microsoft stream insight. Adv Spatial Temporal Databases - 12th Int Symp, SSTD 2011, Minneapolis, MN, USA, August 24-26, 2011, Proceedings, 496–501

  96. Ali MH, Chandramouli B, Raman BS, Katibah E (2010) Spatio-temporal stream processing in microsoft streaminsight. IEEE Data Eng Bull 33(2):69–74

    Google Scholar 

  97. Meskovic E, Osmanovic D, Galic Z et al. (2014) Generating spatio-temporal streaming trajectories. 37th Int Convention Inform Commun Technol, Electron Microelectronics, MIPRO 2014, Opatija, Croatia, 1130–1135

Download references

Author information

Authors and Affiliations

  1. Naval Academy Research Lab, 29240, Brest, Cedex 9, France

    Loic Salmon & Cyril Ray

Authors
  1. Loic Salmon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cyril Ray
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Loic Salmon.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Salmon, L., Ray, C. Design principles of a stream-based framework for mobility analysis. Geoinformatica 21, 237–261 (2017). https://doi.org/10.1007/s10707-016-0256-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10707-016-0256-z

Keywords

Search

Navigation