Special issue on advances on smart camera architectures for real-time image processing
- 418 Downloads
Smart cameras embed systems that perform image sensing and processing. Image processing on a smart camera starts at early vision, focused on enhancing a given feature in a scene, e.g., edge detection, and eventually ends up with image understanding and decision-making. Image sensors with companion reconfigurable chips like FPGAs, or CMOS vision chips that include image sensing and processing on the same IC, are examples of smart cameras. The design of such systems spans a great variety of fields, and levels, from hardware to software, ranging from CMOS vision sensors up to sensory systems, including image processing algorithms, hardware accelerators, and networks of smart embedded cameras.
Smart cameras are now finding their way into a number of niche applications once dominated by camera/PC software-based combinations. However, the choice of smart camera and development environment is dependent on the nature of the vision application, the cost and volume of the system in which the camera will be deployed, and the ease with which it is to be programmed.
In this special issue, we collect the best contributions of two editions of the Workshop on the Architecture of Smart Cameras (WASC) held in Santiago de Compostela (Spain) in 2015, and in Dijon (France), in 2016. Thus, this issue focuses on novel advances in architecture for image understanding, signal and image processing, configurable and FPGA-based architectures, VLSI systems, but also on languages, software environments, and programming tools and applications of smart cameras.
Six papers out of 13 submissions are collected in this Special Issue. The first four papers gather advances in new efficient image processing algorithms oriented to smart cameras. The last two papers are focused on efficient hardware design methodologies for smart cameras.
The first paper by Merwan Birem et al. introduces a new visual odometry method based on the Fourier transform to estimate the location and orientation of a robot with a ground-facing camera. The method is particularly suitable for images with few distinct features. Experimental results on different platforms validate the authors’ approach.
The second paper by A. Aguilar-González et al. introduces a new robust and still fast feature extraction algorithm suitable for indoor and outdoor scenarios, and for FPGA implementation, and thus for a smart camera. Authors show the feasibility of their approach with an FPGA architecture that reaches up to 44 frames per second with full HD images.
The third paper by C. Cruz-Martínez et al. introduces an approach to provide denser 3D maps of environments with large texture-less regions by means of super-pixel-based segmentation. Authors map their architecture onto a GPU architecture for: (1) super-pixel extraction with chromatic and depth information, (2) super-pixel segmentation, and (3) mapping of segmented regions to 3D points. The final sparse 3D map is built at 27 frames per second with an ORB-SLAM system.
The fourth paper by Roberto de Lima et al. is a work toward a smart camera for the fast extraction of high level structures such as planes, spheres or cubes by means of a novel methodology that combines stereo vision, binary descriptors and parallel processing on a GPU. Among other steps, authors’ approach avoids the frame-to-frame rectification stage of conventional stereo techniques through a semi-calibrated stereo rig, reaching up to 50 frames per second for the whole process.
The fifth paper by C. Li et al. addresses the performance gap between manual and high-level synthesis (HLS) design methodologies for real-time image processing on FPGA platforms. Authors show that their code optimization of the HLS methodology outperforms similar design flows like Polycomp and Vivado_HLS in terms of speed on four basic image processing test benches, namely 3 × 3 filter for RGB images, matrix product, image segmentation with the Sobel operator and stereo matching.
The sixth paper by C. Hartmann and D. Fey addresses the memory hierarchy for efficient implementations of image processing applications. The paper introduces new methods to classify image processing applications according to their memory access pattern and their mapping on memory architectures. Authors’ approach combines a simulation framework, namely the Heterogeneous Memory Simulator (HSIM) with an analytical framework, the Memory Analyzer (MemAn). Both of them are used to find the most suitable application-specific memory configuration of image processing applications in terms of processing time and energy consumption.
The guest editors conclude by expressing their sincere thanks to JRTIP’s Editors-in-Chief, Matthias F. Carlsohn and Nasser Kehtarnavaz, and to the entire team of the Journal of Real-Time Image Processing. Of course, this issue would not have been possible without the hard work of the authors who submitted their papers to this issue. Our sincere thanks also go to the reviewers, who did a superb job reviewing the submitted papers under very tight deadlines. We hope that JRTIP’s readers will find this issue illuminating. Happy reading!