TimeClassifier: a visual analytic system for the classification of multi-dimensional time series data

Abstract

Biologists studying animals in their natural environment are increasingly using sensors such as accelerometers in animal-attached ‘smart’ tags because it is widely acknowledged that this approach can enhance the understanding of ecological and behavioural processes. The potential of such tags is tempered by the difficulty of extracting animal behaviour from the sensors which is currently primarily dependent on the manual inspection of multiple time series graphs. This is time consuming and error-prone for the domain expert and is now the limiting factor for realising the value of tags in this area. We introduce TimeClassifier, a visual analytic system for the classification of time series data for movement ecologists. We deploy our system with biologists and report two real-world case studies of its use.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

References

  1. 1.

    Abdulla-Al-Maruf, A., Huang, H.H., Kawagoe, K.: Time series classification method based on longest common subsequence and textual approximation. In: Digital Information Management (ICDIM), 2012 Seventh International Conference on, pp. 130–137 (2012)

  2. 2.

    Alexander, R.M.: Models and the scaling of energy costs for locomotion. J. Exp. Biol. 208(9), 1645–1652 (2005)

    Article  Google Scholar 

  3. 3.

    Bidder, O.R., Campbell, H.A., Gmez-Laich, A., Urg, P., Walker, J., Cai, Y., Gao, L., Quintana, F., Wilson, R.P.: Love thy neighbour: automatic animal behavioural classification of acceleration data using the k-nearest neighbour algorithm. PLoS ONE 9(2), e88,609 (2014)

    Article  Google Scholar 

  4. 4.

    Bidder, O.R., Qasem, L.A., Wilson, R.P.: On higher ground: how well can dynamic body acceleration determine speed in variable terrain? PLoS ONE 7(11), 50556 (2012)

    Article  Google Scholar 

  5. 5.

    Blaas, J., Botha, C.P., Grundy, E., Jones, M.W., Laramee, R.S., Post, F.H.: Smooth graphs for visual exploration of higher-order state transitions. IEEE Trans. Vis. Comput. Graph. 15(6), 969–976 (2009)

    Article  Google Scholar 

  6. 6.

    Bouali, F., Devaux, S., Venturini, G.: Visual mining of time series using a tubular visualization. Vis. Comput. 1–16 (2014)

  7. 7.

    Buono, P., Aris, A., Plaisant, C., Khella, A., Shneiderman, B.: Interactive pattern search in time series. In: International Society for Optics and Photonics, Electronic Imaging 2005 (2005)

  8. 8.

    Ellis, K., Kerr, J., Godbole, S., Lanckriet, G., Wing, D., Marshall, S.: A random forest classifier for the prediction of energy expenditure and type of physical activity from wrist and hip accelerometers. Physiol. Meas. 35(11), 2191 (2014)

    Article  Google Scholar 

  9. 9.

    van den Elzen, S., van Wijk, J.: Baobabview: interactive construction and analysis of decision trees. In: Visual Analytics Science and Technology (VAST), 2011 IEEE Conference on, pp. 151–160 (2011). doi:10.1109/VAST.2011.6102453

  10. 10.

    Esling, P., Agon, C.: Time-series data mining. ACM Comput. Surv. 45(1), 12:1–12:34 (2012)

    Article  Google Scholar 

  11. 11.

    Gao, L., Campbell, H.A., Bidder, O.R., Hunter, J.: A web-based semantic tagging and activity recognition system for species’ accelerometry data. Ecol. Informatics 13(0), 47–56 (2013)

    Article  Google Scholar 

  12. 12.

    Gleiss, A.C., Wilson, R.P., Shepard, E.L.C.: Making overall dynamic body acceleration work: on the theory of acceleration as a proxy for energy expenditure. Methods Ecol. Evol. 2(1), 23–33 (2011)

    Article  Google Scholar 

  13. 13.

    Gregory, M., Shneiderman, B.: Shape identification in temporal data sets. In: Expanding the Frontiers of Visual Analytics and Visualization, pp. 305–321. Springer, Berlin (2012)

  14. 14.

    Grundy, E., Jones, M.W., Laramee, R.S., Wilson, R.P., Shepard, E.L.C.: Visualisation of sensor data from animal movement. Comput. Graph. Forum 28(3), 815–822 (2009)

    Article  Google Scholar 

  15. 15.

    Hao, M.C., Dayal, U., Keim, D.A., Schreck, T.: Importance-driven visualization layouts for large time series data. In: Information Visualization, 2005. INFOVIS 2005. IEEE Symposium on, pp. 203–210. IEEE (2005)

  16. 16.

    Hao, M.C., Marwah, M., Janetzko, H., Dayal, U., Keim, D.A., Patnaik, D., Ramakrishnan, N., Sharma, R.K.: Visual exploration of frequent patterns in multivariate time series. Inf. Vis. 11(1), 71–83 (2012)

    Article  Google Scholar 

  17. 17.

    Holz, C., Feiner, S.: Relaxed selection techniques for querying time-series graphs. In: UIST ’09: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, pp. 213–222. ACM, New York (2009)

  18. 18.

    Janicke, H., Bottinger, M., Mikolajewicz, U., Scheuermann, G.: Visual exploration of climate variability changes using wavelet analysis. IEEE Trans. Vis. Comput. Graph. 15(6), 1375–1382 (2009)

    Article  Google Scholar 

  19. 19.

    Keogh, E., Ratanamahatana, C.A.: Exact indexing of dynamic time warping. Knowl. Inf. Syst. 7(3), 358–386 (2005)

    Article  Google Scholar 

  20. 20.

    Kincaid, R.: Signallens: focus+context applied to electronic time series. Vis. Comput. Graph. IEEE Trans. 16(6), 900–907 (2010)

    Article  Google Scholar 

  21. 21.

    von Landesberger, T., Andrienko, G., Andrienko, N., Bremm, S., Kirschner, M., Wesarg, S., Kuijper, A.: Opening up the black box of medical image segmentation with statistical shape models. Vis. Comput. 29(9), 893–905 (2013)

    Article  Google Scholar 

  22. 22.

    Lanzone, M.J., Miller, T.A., Turk, P., Brandes, D., Halverson, C., Maisonneuve, C., Tremblay, J., Cooper, J., O’Malley, K., Brooks, R.P., Katzner, T.: Flight responses by a migratory soaring raptor to changing meteorological conditions. Biol. Lett. 8(5), 710–713 (2012)

    Article  Google Scholar 

  23. 23.

    Lewis, J.P.: Fast template matching. Vis. Interface 95, 120–123 (1995)

    Google Scholar 

  24. 24.

    Lin, J., Keogh, E., Lonardi, S.: Visualizing and discovering non-trivial patterns in large time series databases. Inf. Vis. 4(2), 61–82 (2005)

    Article  Google Scholar 

  25. 25.

    Liu, S., Cui, W., Wu, Y., Liu, M.: A survey on information visualization: recent advances and challenges. Vis. Comput. 30(12), 1373–1393 (2014)

    Article  Google Scholar 

  26. 26.

    Nathan, R., Spiegel, O., Fortmann-Roe, S., Harel, R., Wikelski, M., Getz, W.M.: Using tri-axial acceleration data to identify behavioral modes of free-ranging animals: general concepts and tools illustrated for griffon vultures. J. Exp. Biol. 215(6), 986–996 (2012)

    Article  Google Scholar 

  27. 27.

    Payne, N.L., Taylor, M.D., Watanabe, Y.Y., Semmens, J.M.: From physiology to physics: are we recognizing the flexibility of biologging tools? J. Exp. Biol. 217(Pt 3), 317–322 (2014)

    Article  Google Scholar 

  28. 28.

    Ratanamahatana, C., Lin, J., Gunopulos, D., Keogh, E., Vlachos, M., Das, G.: Data mining and knowledge discovery handbook 2010. In: Maimon, O., Rokach, l., 2nd edn. (2010)

  29. 29.

    Ropert-Coudert, Y., Wilson, R.P.: Trends and perspectives in animal-attached remote sensing. Front. Ecol. Environ. 3(8), 437–444 (2005)

    Article  Google Scholar 

  30. 30.

    Ryall, K., Lesh, N., Lanning, T., Leigh, D., Miyashita, H., Makino, S.: Querylines: approximate query for visual browsing. In: CHI ’05 Extended Abstracts on Human Factors in Computing Systems, CHI EA’05, pp. 1765–1768 (2005)

  31. 31.

    Sakai, M., Aoki, K., Sato, K., Amano, M., Baird, R.W., Webster, D.L., Schorr, G.S., Miyazaki, N.: Swim speed and acceleration measurements of short-finned pilot whales (Globicephala macrorhynchus) in Hawai’i. Mammal Study 36(1), 55–59 (2011)

    Article  Google Scholar 

  32. 32.

    Sakamoto, K.Q., Sato, K., Ishizuka, M., Watanuki, Y., Takahashi, A., Daunt, F., Wanless, S.: Can ethograms be automatically generated using body acceleration data from free-ranging birds? PLoS ONE 4(4), e5379 (2009)

    Article  Google Scholar 

  33. 33.

    Sato, K., Charrassin, J.B., Bost, C.A., Naito, Y.: Why do macaroni penguins choose shallow body angles that result in longer descent and ascent durations? J. Exp. Biol. 207(23), 4057–4065 (2004)

    Article  Google Scholar 

  34. 34.

    Shepard, E.L.C., Halsey, L.G.: Identification of animal movement patterns using tri-axial accelerometry. Endanger. Species Res. 10, 47–60 (2008)

    Article  Google Scholar 

  35. 35.

    Smith, S.W.: The Scientist and Engineer’s Guide to Digital Signal Processing. California Technical Publishing, San Diego (1997)

    Google Scholar 

  36. 36.

    Walker, J., Geng, Z., Jones, M., Laramee, R.S.: Visualization of large, time-dependent, abstract data with integrated spherical and parallel coordinates, pp. 43–47. Eurographics Association, Vienna (2012)

  37. 37.

    Ware, C., Arsenault, R., Plumlee, M., Wiley, D.: Visualizing the underwater behavior of humpback whales. IEEE Comput. Graph. Appl. 26(4), 14–18 (2006)

    Article  Google Scholar 

  38. 38.

    Watanuki, Y., Takahashi, A., Daunt, F., Wanless, S., Harris, M., Sato, K., Naito, Y.: Regulation of stroke and glide in a foot-propelled avian diver. J. Exp. Biol. 208(12), 2207–2216 (2005)

    Article  Google Scholar 

  39. 39.

    Weber, M., Alexa, M., Muller, W.: Visualizing time-series on spirals. In: Information Visualization, 2001. INFOVIS 2001. IEEE Symposium on, pp. 7–13

  40. 40.

    van Wijk, J., Van Selow, E.: Cluster and calendar based visualization of time series data. In: Information Visualization, 1999. (Info Vis ’99) Proceedings. 1999 IEEE Symposium on, vol. 140, pp. 4–9 (1999)

  41. 41.

    Wilson, R.P., Hustler, K., Ryan, P.G., Burger, A.E., Noldeke, E.C.: Diving birds in cold water: do archimedes and Boyle determine energetic costs? Am. Nat., pp. 179–200 (1992)

  42. 42.

    Wilson, R.P., Shepard, E., Liebsch, N.: Prying into the intimate details of animal lives: use of a daily diary on animals. Endanger. Species Res. 4, 123–137 (2008)

    Article  Google Scholar 

  43. 43.

    Zhao, J., Chevalier, F., Pietriga, E., Balakrishnan, R.: Exploratory analysis of time-series with chronoLenses. IEEE Trans. Vis. Comput. Graph. 17(12), 2422–2431 (2011)

    Article  Google Scholar 

Download references

Acknowledgments

This work was funded by an EPSRC doctoral training grant.

Author information

Affiliations

Authors

Corresponding author

Correspondence to James S. Walker.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 2 (mp4 19528 KB)

Supplementary material 1 (pdf 333 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Walker, J.S., Jones, M., Laramee, R.S. et al. TimeClassifier: a visual analytic system for the classification of multi-dimensional time series data. Vis Comput 31, 1067–1078 (2015). https://doi.org/10.1007/s00371-015-1112-0

Download citation

Keywords

  • Visual analytics
  • Time series analysis
  • Movement ecology