Skip to main content
SpringerLink
Log in

Platforms and Architectures for Distributed Smart Cameras

  • Chapter
  • First Online:
Distributed Embedded Smart Cameras

Abstract

Embedded computer vision places huge computational demands on smart cameras; in addition, these systems must often be designed to consume very low power and be inexpensive to manfuacture. In this chapter, we consider computational platforms for both smart cameras and networks of smart cameras. A platform is a combination of hardware and software that provides a set of features and services for an application space. We first compare a broad range of computing fabrics suitable for embedded computer vision: FPGAs, GPUs, video signal processors, and heterogeneous multiprocessor system-on-chip. We then look at approaches to the design of a platform for distributed services in a smart camera network.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Agbaria A, Kang DI, Singh K (2006) Lmpi: MPI for heterogeneous embedded distributed systems. In: 12th International conference on parallel and distributed systems 2006 (ICPADS 2006), vol 1, p 8. doi:10.1109/ICPADS.2006.56

  2. Arth C, Leistner C, Bischof H (2007) Object reacquisition and tracking in large-scale smart camera networks. In: First ACM/IEEE international conference on distributed smart cameras 2007 (ICDSC ’07), pp 156–163. doi:10.1109/ICDSC.2007.4357519

  3. Austin T, Blaauw D, Mahlke S, Mudge T, Chakrabarti C, Wolf W (2004) Mobile supercomputers. IEEE Comput 37(5):81–83

    Article  Google Scholar 

  4. Barnard ST, Thompson WB (1980) Disparity analysis of images. IEEE Trans Pattern Anal Mach Intell PAMI 2(4):333–340

    Google Scholar 

  5. Boddie J, Daryanani G, Eldumiati I, Gadenz R, Thompson J, Walters S, Pedersen R (1980) A digital signal processor for telecommunications applications. In: Solid-state circuits conference. Digest of technical papers. 1980 IEEE international, vol XXIII, pp 44–45. doi:10.1109/ISSCC.1980.1156117

  6. Burt P, Adelson E (1983) The laplacian pyramid as a compact image code. IEEE Trans Commun 31(4):532–540. doi:10.1109/TCOM.1983.1095851

    Article  Google Scholar 

  7. Clemons J, Jones A, Perricone R, Savarese S, Austin T (2011) Effex: an embedded processor for computer vision based feature extraction. In: Design automation conference (DAC), 2011. 48th ACM/EDAC/IEEE, pp 1020–1025

    Google Scholar 

  8. Corporation A (2014) Stratix v device overview. Tech Rep SV51001

    Google Scholar 

  9. Corporation X (2013) 7 Series fpgas overview. Tech Rep DS180

    Google Scholar 

  10. Devarajan D, Cheng Z, Radke R (2008) Calibrating distributed camera networks. Proc IEEE 96(10):1625–1639. doi:10.1109/JPROC.2008.928759

  11. Doblander A, Zoufal A, Rinner B (2009) A novel software framework for embedded multiprocessor smart cameras. ACM Trans Embed Comput Syst 8(3):24:1–24:30. doi:10.1145/1509288.1509296. http://doi.acm.org.prx.library.gatech.edu/10.1145/1509288.1509296

  12. Dutta S, O’Connor K, Wolf W, Wolfe A (1998) A design study of a 0.25-\({\upmu }m\) video signal processor. IEEE Trans Circuits Syst Video Technol 8(4):501–519 doi:10.1109/76.709414.

  13. Dutta S, Wolf W (1999) A circuit-driven design methodology for video signal-processing datapath elements. IEEE Trans VLSI Syst 7(2):229–240

    Article  Google Scholar 

  14. Dutta S, Wolf W, Wolfe A (1998) A methodology to evaluate memory architecture design tradeoffs for video signal processors. IEEE Trans Circuits Syst Video Technol 8(1):36–53

    Article  Google Scholar 

  15. Ercegovac MD, Lang T (2004) Digital arithmetic. Morgan-Kaufmann

    Google Scholar 

  16. Farabet C, Martini B, Corda B, Akselrod P, Culurciello E, LeCun Y (2011) Neuflow: a runtime reconfigurable dataflow processor for vision. In: IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW 2011), pp 109–116. doi:10.1109/CVPRW.2011.5981829

  17. Fritts J, Wolf W (1999) Understanding multimedia application characteristics for designing programmable media processors. In: Multimedia hardware architectures. SPIE

    Google Scholar 

  18. Fritts J, Wolf W (2000) Evaluation of static and dynamic scheduling for media processors. In: Proceedings, MICRO-33 MP-DSP2 workshop. ACM

    Google Scholar 

  19. Fung J, Mann S (2005) OpenVIDIA: parallel GPU computer vision. In: Multimedia ’05: proceedings of the 13th annual ACM international conference on multimedia

    Google Scholar 

  20. Fung J, Mann S (2008) Using graphics devices in reverse: Gpu-based image processing and computer vision. In: IEEE international conference on multimedia and Expo 2008, pp 9–12. doi:10.1109/ICME.2008.4607358

  21. Gropp W, Lusk EL, Skjellum A (1999) Using MPI: portable parallel programming with the message-passing interface, 2nd edn. MIT Press, Cambridge

    Google Scholar 

  22. Gudis E, Lu P, Berends D, Kaighn K, van der Wal G, Buchanan G, Chai S, Piacentino M (2013) An embedded vision services framework for heterogeneous accelerators. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW 2013), pp 598–603. doi:10.1109/CVPRW.2013.90

  23. Gudis E, Van der Wal G, Kuthirummal S, Chai S (2012) Multi-resolution real-time dense stereo vision processing in fpga. In: IEEE 20th annual international symposium on field-programmable custom computing machines (FCCM 2012), pp 29–32. doi:10.1109/FCCM.2012.15

  24. Hartley RI, Zisserman A (2004) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge

    Google Scholar 

  25. Hennessy JL, Patterson DA (2001) Computer architecture: a quantitative approach, 5th edn. Morgnn Kaufman, Los Altos

    Google Scholar 

  26. Hong K, Smaldone S, Shin J, Lillethun D, Iftode L, Ramachandran U (2011) Target container: a target-centric parallel programming abstraction for video-based surveillance. In: Fifth ACM/IEEE international conference on distributed smart cameras (ICDSC 2011), pp 1–8. doi:10.1109/ICDSC.2011.6042914

  27. Horprasesert T, Harwood D, Davis LS (1999) A statistical approach for real-time robust background subtraction and shadow detection. In: IEEE international conference on computer vision FRAME-RATE workshop

    Google Scholar 

  28. Instruments T (2001) Tms320c55x dsp reference guide. Tech Rep SPRU371D

    Google Scholar 

  29. Instruments T (2010) Tms320c674x dsp cpu and instruction set reference guide. Tech Rep SPRUFE8B

    Google Scholar 

  30. Instruments T (2013) Floating-point digital signal processor. Tech Rep SPRS675

    Google Scholar 

  31. Instruments T (2013) Tms320dm814x davinci digital video processors technical reference manual. Tech Rep SPRUGZ8E. Revision 5

    Google Scholar 

  32. Kanevsky A, Skjellum A, Rounbehler A (1998) MPI/RT-an emerging standard for high-performance real-time systems. In: Proceedings of the thirty-first Hawaii international conference on system sciences, vol 3, pp 157–166. doi:10.1109/HICSS.1998.656130

  33. Kim H, Wolf M (2010) Distributed tracking in a large-scale network of smart cameras. In: Proceedings of the fourth ACM/IEEE international conference on distributed smart cameras. ACM Press, p 816

    Google Scholar 

  34. Kushwaha M, Koutsoukos X (2010) 3d target tracking in distributed smart camera networks with in-network aggregation. In: Proceedings of the fourth ACM/IEEE international conference on distributed smart cameras (ICDSC ’10). ACM, New York, pp 25–32. doi:10.1145/1865987.1865992. http://doi.acm.org.prx.library.gatech.edu/10.1145/1865987.1865992

  35. Li E, Wang B, Yang L, ti Peng Y, Du Y, Zhang Y, Chiu YJ (2012) GPU and CPU cooperative accelaration for face detection on modern processors. In: IEEE international conference on multimedia and expo (ICME 2012), pp 769–775. doi:10.1109/ICME.2012.121

  36. Lin CH, Wolf M, Koutsoukos X, Neema S, Sztipanovits J (2010) System and software architectures of distributed smart cameras. ACM Trans Embed Comput Syst 9(4):38:1–38:30. doi:10.1145/1721695.1721704. http://doi.acm.org.prx.library.gatech.edu/10.1145/1721695.1721704

  37. Lowe D (1999) Object recognition from local scale-invariant features. In: The proceedings of the seventh IEEE international conference on computer vision, vol 2, pp 1150–1157. doi:10.1109/ICCV.1999.790410

  38. Lucas B, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: International joint conference on artificial intelligence. AAAI

    Google Scholar 

  39. Ly D, Saldana M, Chow P (2009) The challenges of using an embedded MPI for hardware-based processing nodes. In: International conference on field-programmable technology 2009 (FPT 2009), pp 120–127. doi:10.1109/FPT.2009.5377688

  40. McMahon T, Skjellum A (1996) eMPI/eMPICH: embedding MPI. In: Proceedings of MPI developer’s conference, 1996, Second, pp 180–184. doi:10.1109/MPIDC.1996.534111

  41. Morat J, Devernay F, Ibanez-Guzman J, Cornou S (2007) Evaluation method for automotive stereo-vision systems. In: Intelligent vehicles symposium, 2007 IEEE, pp 202–208. doi:10.1109/IVS.2007.4290115

  42. Nagendra P (2011) Performance characterization of automotive computer vision systems using graphics processing units (GUPs). In: International conference on image information processing (ICIIP 2011), pp 1–4. doi:10.1109/ICIIP.2011.6108951

  43. Oh S, Russell S, Sastry S (2004) Markov chain monte carlo data association for general multiple-target tracking problems. In: Proceedongs of the 43rd IEEE conference on decision and control

    Google Scholar 

  44. Peleg A, Weiser U (1996) Mmx technology extension to the intel architecture. Micro IEEE 16(4):42–50. doi:10.1109/40.526924

  45. Ruetz P, Tong P, Bailey D, Luthi D, H.Ang P (1992) A high-performance full-motion video compression chip set. IEEE Trans Circuits Syst Video Technol 2(2):111–122. doi:10.1109/76.143411

  46. Schlessman J, Lodato M, Ozer B, Wolf W (2007) Heterogeneous MPSoC architectures for embedded computer vision. In: 2007 IEEE international conference on multimedia and expo, pp 1870–1873. IEEE

    Google Scholar 

  47. Schlessman J, Saha S, Wolf W, Bhattacharya S (2005) An extended motion-estimation architecture applied to shape recognition. In: IEEE international conference on multimedia and expo 2005 (ICME 2005), pp 1504–1507. doi:10.1109/ICME.2005.1521718

  48. Schulte M, Stine J (1999) Approximating elementary functions with symmetric bipartite tables. IEEE Trans Comput 48(8):842–847. doi:10.1109/12.795125

  49. Semiconductor F (2012) Mpc5676r microcontroller data sheet. Tech Rep MPC5676R. Rev 3

    Google Scholar 

  50. Soderquist P, Leeser M (1997) Division and square root: choosing the right implementation. Micro IEEE 17(4):56–66. doi:10.1109/40.612224

  51. Stein G, Rushinek E, Hayun G, Shashua A (2005) A computer vision system on a chip: a case study from the automotive domain. In: IEEE computer society conference on computer vision and pattern recognition—workshops, 2005. CVPR workshops, pp 130–130. doi:10.1109/CVPR.2005.387

  52. Stone W, Sekercioglu Y (2010) Analysis of distributed smart camera calibration accuracy using minimal data transmission. In: IEEE international conference on communication systems (ICCS), 2010, pp 655–661. doi:10.1109/ICCS.2010.5686597

  53. Tallu D, John LK, Burger D (2003) Bottlenecks in multimedia pro-cessing with SIMD style extensions and multimedia enhancements. IEEE Trans Comput 52(8):1015–1031

    Article  Google Scholar 

  54. Velipasalar S, Schlessman J, Chen CY, Wolf W, Singh J (2006) SCCS: a scalable clustered camera system for multiple object tracking communicating via message passing interface. In: IEEE international conference on multimedia and expo, 2006, pp 277–280. doi:10.1109/ICME.2006.262452

  55. Velipasalar S, Wolf W (2005) Multiple object tracking and occlusion handling by information exchange between uncalibrated cameras. In: IEEE international conference on image processing 2005 (ICIP 2005), vol 2, pp II-418-21. doi:10.1109/ICIP.2005.1530081

  56. Velipasalar S, Wolf WH (2008) Frame-level temporal calibration of video sequences from unsynchronized cameras. Mach Vis Appl J. doi:10.1007/s00138-008-0122-6

  57. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition 2001 (CVPR 2001), vol 1, pp I-511–I-518. doi:10.1109/CVPR.2001.990517

  58. Wang G, Xiong Y, Yun J, Cavallaro J (2013) Accelerating computer vision algorithms using opencl framework on the mobile GPU—a case study. In: IEEE international conference on acoustics, speech and signal processing (ICASSP 2013), pp 2629–2633. doi:10.1109/ICASSP.2013.6638132

  59. Wang X, Leeser M (2010) Vfloat: a variable precision fixed- and floating-point library for reconfigurable hardware. ACM Trans Reconfigurable Technol Syst 3(3):16:1–16:34. doi:10.1145/1839480.1839486. http://doi.acm.org/10.1145/1839480.1839486

  60. Weber WD, Chou J, Swarbrick I, Wingard D (2005) A quality-of-service mechanism for interconnection networks in system-on-chips. In: Proceedings of design, automation and test in Europe, 2005, vol 2, pp 1232–1237. doi:10.1109/DATE.2005.33

  61. Wolf M (2014) High performance embedded computing, 2nd edn. Morgan Kaufmann, Los Altos

    Google Scholar 

  62. van der Wolf P, Geuzebroek J (2011) Soc infrastructures for predictable system integration. In: Design, automation test in Europe conference exhibition (DATE 2011), pp 1–6. doi:10.1109/DATE.2011.5763146

  63. van der Wolf P, Henriksson T (2008) Video processing requirements on SoC infrastructures. In: Design, automation and test in Europe 2008 (DATE ’08), pp 1124–1125. doi:10.1109/DATE.2008.4484827

  64. Wolf W, Jerraya A, Martin G (2008) Multiprocessor system-on-chip (MPSoC) technology. IEEE Trans Comput Aided Des Integr Circuits Syst 27(10):1701–1713. doi:10.1109/TCAD.2008.923415

  65. Wolf W, Ozer B, Lv T (2002) Smart cameras as embedded systems. IEEE Comput 35(9):48–53

    Article  Google Scholar 

  66. Xu J, Wolf W, Henkel J, Chakradhar S (2006) A design methodology for application-specific networks-on-chip. ACM Trans Embed Comput Syst 5(2):263–280. doi:10.1145/1151074.1151076. http://doi.acm.org.prx.library.gatech.edu/10.1145/1151074.1151076

  67. Xu J, Wolf W, Henkel J, Chakradhar S, Lv T (2004) A case study in networks-on-chip design for embedded video. In: Proceedings of design, automation and test in Europe conference and exhibition, 2004, vol 2, pp 770–775. doi:10.1109/DATE.2004.1268973

  68. Yang KM, Sun MT, Wu L (1989) A family of vlsi designs for the motion compensation block-matching algorithm. IEEE Trans Circuits Syst 36(10):1317–1325. doi:10.1109/31.44348

  69. Zitnick C, Kanade T (2000) A cooperative algorithm for stereo matching and occlusion detection. IEEE Trans Pattern Anal Mach Intell 22(7):675–684. doi:10.1109/34.865184

Download references

Acknowledgments

Thanks to Tom Conte of Georgia Tech for discussions on computer architecture.

Author information

Authors and Affiliations

  1. School of ECE, Georgia Institute of Technology, 777 Atlantic Drive NW, Atlanta, GA, 30332, USA

    Marilyn Wolf

Authors
  1. Marilyn Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marilyn Wolf .

Editor information

Editors and Affiliations

  1. University of Arkansas, Fayetteville, Arkansas, USA

    Christophe Bobda

  2. Syracuse University, Syracuse, New York, USA

    Senem Velipasalar

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this chapter

Cite this chapter

Wolf, M. (2014). Platforms and Architectures for Distributed Smart Cameras. In: Bobda, C., Velipasalar, S. (eds) Distributed Embedded Smart Cameras. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7705-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-7705-1_1

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-7704-4

  • Online ISBN: 978-1-4614-7705-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation