Skip to main content
SpringerLink
Log in

Privacy preserving recognition of object-based activities using near-infrared reflective markers

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

This paper presents a low-cost privacy preserving method for recognizing object-based activities through the use of near-infrared (NIR) reflective markers. The NIR markers are thin pieces of durable reflective material that can be attached to objects, with their movement captured using a webcam that has been modified to capture NIR images. The camera is modified to replace its standard IR-blocking filter with an IR-passing visible-light-blocking filter to allow us to preserve the privacy of users during recognition. Using this equipment, we attempt to achieve accurate recognition of several object-based activities by differentiating between markers based on their stripe or grid designs, as well as by capturing the usage of the objects based on their unique movement patterns. We evaluate our method by identifying kitchen activities using markers attached to common kitchen utensils.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Adib F, Kabelac Z, Katabi D, Miller RC (2014) 3D tracking via body radio reflections. In: symposium on networked systems design and implementation, vol 14. USENIX, pp 317–329

  2. Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. In: pervasive computing. Springer, pp 1–17

  3. Bradski G (2000) Opencv library. Dr. Dobb’s J Softw Tools 25:120–123

  4. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MATH  Google Scholar 

  5. Dernbach S, Das B, Krishnan NC, Thomas BL, Cook DJ (2012) Simple and complex activity recognition through smart phones. In: IEEE international conference on intelligent environments. IEEE, pp 214–221

  6. Fiala M (2005) Artag, a fiducial marker system using digital techniques. In: IEEE conference on computer vision and pattern recognition, vol 2. IEEE, pp 590–596

  7. Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn 47(6):2280–2292

    Article  Google Scholar 

  8. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. ACM SIGKDD Explor Newsl 11(1):10–18

    Article  Google Scholar 

  9. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision. Springer, pp 630–645

  10. Hooper CJ, Preston A, Balaam M, Seedhouse P, Jackson D, Pham C, Ladha C, Ladha K, Plötz T, Olivier P (2012) The french kitchen: task-based learning in an instrumented kitchen. In: ACM conference on ubiquitous computing. ACM, pp 193–202

  11. Ji Y, Ko Y, Shimada A, Nagahara H, Taniguchi RI (2012) Cooking gesture recognition using local feature and depth image. In: ACM multimedia. ACM, pp 37–42

  12. Jolliffe I (2002) Principal component analysis. Wiley Online Library

  13. Lei J, Ren X, Fox D (2012) Fine-grained kitchen activity recognition using RGB-D. In: ACM conference on ubiquitous computing. ACM, pp 208–211

  14. Li T, An C, Tian Z, Campbell AT, Zhou X (2015) Human sensing using visible light communication. In: International conference on mobile computing and networking. ACM, pp 331–344

  15. Nakazato Y, Kanbara M, Yokoya N (2005) Wearable augmented reality system using invisible visual markers and an IR camera. In: International symposium on wearable computers, pp 198–199

  16. Pham C, Jackson D, Schöning J, Bartindale T, Plötz T, Olivier P (2013) Foodboard: surface contact imaging for food recognition. In: ACM international joint conference on pervasive and ubiquitous computing. ACM, pp 749–752

  17. Plötz T, Moynihan P, Pham C, Olivier P (2011) Activity recognition and healthier food preparation. In: activity recognition in pervasive intelligent environments. Springer, Berlin, pp 313–329

  18. Rybok L, Friedberger S, Hanebeck UD, Stiefelhagen R (2011) The kit robo-kitchen data set for the evaluation of view-based activity recognition systems. In: IEEE-RAS international conference on humanoid robots. IEEE, pp 128–133

  19. Souza CR (2014) The accord.net framework. http://accord-framework.net/

  20. Spielberg A, Sample A, Hudson SE, Mankoff J, McCann J (2016) RapID: a framework for fabricating low-latency interactive objects with RFID tags. In: Conference on human factors in computing systems. ACM, pp 5897–5908

  21. Stein S, McKenna SJ (2013) Combining embedded accelerometers with computer vision for recognizing food preparation activities. In: International joint conference on pervasive and ubiquitous computing. ACM, pp 729–738

  22. Wagner D, Schmalstieg D (2007) Artoolkitplus for pose tracking on mobile devices. In: computer vision winter workshop

  23. Wagner J, van Halteren A, Hoonhout J, Plötz T, Pham C, Moynihan P, Jackson D, Ladha C, Ladha K, Olivier P (2011) Towards a pervasive kitchen infrastructure for measuring cooking competence. In: International conference on pervasive computing technologies for healthcare. IEEE, pp 107–114

  24. Yan Z, Chakraborty D, Misra A, Jeung H, Aberer K (2012) Sammple: detecting semantic indoor activities in practical settings using locomotive signatures. In: International symposium on wearable computers. IEEE, pp 37–40

  25. Yang L, Li Y, Lin Q, Li XY, Liu Y (2016) Making sense of mechanical vibration period with sub-millisecond accuracy using backscatter signals. In: International conference on mobile computing and networking. ACM, pp 16–28

Download references

Acknowledgements

This study is partially supported by JST CREST.

Author information

Authors and Affiliations

  1. Osaka University, 1-5, Yamadaoka, Suita-shi, Osaka, 565-0871, Japan

    Joseph Korpela & Takuya Maekawa

Authors
  1. Joseph Korpela
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Takuya Maekawa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joseph Korpela.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Korpela, J., Maekawa, T. Privacy preserving recognition of object-based activities using near-infrared reflective markers. Pers Ubiquit Comput 22, 365–377 (2018). https://doi.org/10.1007/s00779-017-1070-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-017-1070-9

Keywords

Search

Navigation