Multimodal image feature detection with ROI-based optimization for image registration

  • Rajesh NandalikeEmail author
  • H. Sarojadevi
Original Research Paper


Image registration plays an imperative part of multimodal video analysis system. In video surveillance applications, change in the environmental conditions makes the registration process hard. Use of multiple sensors makes the system more robust to environmental changes as compared to single sensor imaging system. Using multiple modalities such as infrared(IR)/thermal sensors and CMOS image sensors augment the sturdiness of the surveillance system. Here we propose hardware implementation of feature detection on Genesys 2 Kintex-7 FPGA for a multimodal surveillance system, which is robust in poor lighting conditions and affine changes. To reduce the processing time, a region of interest (ROI) is identified and feature extraction is performed in this region. Design optimization in hardware architecture resulted in achieving the real-time performance of image registration on HD 720p video.


Multimodal Image registration Thermal Image ROI Hardware architecture Hardware acceleration 



  1. 1.
    Dıaz, S., Soto, J.E., Inostroza, F., Godoy, S.E., Figueroa, M.: An embedded system for image segmentation and multimodal registration in noninvasive skin cancer screening. In: 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju Island, South Korea, South Korea (2017)Google Scholar
  2. 2.
    Davis, J.W., Sharma, V.: Background-subtraction using contour-based fusion of thermal and visible imagery. Comput. Vis. Image Underst. 106(2–3), 162–182 (2007)CrossRefGoogle Scholar
  3. 3.
    Inostroza, F., Cárdenas, J., Godoy, S.E.: Embedded multimodal registration of visible images on long-wave infrared video in real time. In: Euromicro Conference on Digital System Design (DSD), Limassol, Cyprus (2016)Google Scholar
  4. 4.
    Jie, Z.: A novel image registration algorithm using SIFT feature descriptors. In: International Conference on Smart City and Systems Engineering, Hunan, China (2016)Google Scholar
  5. 5.
    Yang, Z., Cohen, F.S.: Image registration and object recognition using affine invariants and convex hulls. IEEE Trans. Image Process. 8(7), 934–946 (1999)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Zitová, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)CrossRefGoogle Scholar
  7. 7.
    Zhu, Y., Cheng, S., Stankovic´, V., Stankovic, L.: Image registration using BP-SIFT. J. Vis. Commun. Image Represent. 24(4), 448–457 (2013)CrossRefGoogle Scholar
  8. 8.
    Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24(4), 325–376 (1992)CrossRefGoogle Scholar
  9. 9.
    Claus, C., Huitl, R., Rausch, J., Stechele, W.: Optimizing the SUSAN corner detection algorithm for a high speed FPGA implementation. In: FPL 2009. International Conference on Field Programmable Logic and Applications, Prague, Czech Republic (2009)Google Scholar
  10. 10.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. 60(2), 91–110 (2004)Google Scholar
  11. 11.
    David, J.W.: Dataset 03: OSU Color-Thermal Database, OTCBVS Benchmark Dataset Collection. (2004). Accessed 1 Sep 2017
  12. 12.
    Vourvoulakis, J., Kalomiros, J., Lygouras, J.: Fully pipelined FPGA-based architecture for real-time SIFT extraction. Microprocess. Microsyst. 40, 53–73 (2015)CrossRefGoogle Scholar
  13. 13.
    Mizuno, K., Kamino, T., Ariki, Y.: FPGA based accelerated orientation calculation in SIFT using LUTs. In: A Low-Power Real-Time SIFT Descriptor Generation Engine for Full-HDTV Video Recognition, vol. E94, no. C, pp. 448–457 (2013)Google Scholar
  14. 14.
    Wang, J., Zhong, S., Yan, L., Cao, Z.: An embedded system-on-chip architecture for real-time visual detection and matching. In: IEEE Transactions on Circuits and Systems for Video Technology (2013)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of E&CENitte Meenakshi Institute of TechnologyBengaluruIndia
  2. 2.Department of CS&ENitte Meenakshi Institute of TechnologyBengaluruIndia

Personalised recommendations