Advertisement

A shadow detection and removal method for fruit recognition in natural environments

  • Rongbin Bu
  • Juntao XiongEmail author
  • Shumian Chen
  • Zhenhui Zheng
  • Wentao Guo
  • Zhengang Yang
  • Xiaoyun Lin
Article
  • 38 Downloads

Abstract

Effective shadow detection and shadow removal can improve the performance of fruit recognition in natural environments and provide technical support for agricultural intelligence. In this study, a superpixel segmentation method was used to divide an image into multiple small regions. Based on the superpixel segmentation results, the shadow regions and the shadowless regions of the orchard images under natural light were compared and studied. Seven shadow saliency features (SSF) were explored and analyzed for shadow detection. The SSF were used to enhance the shadow characteristics. Then, the genetic algorithm (GA) was used to optimize the parameters, and support vector machine recursive feature elimination (SVM-RFE) was used to determine the best feature combination for shadow detection. According to the best feature combination, the support vector machine (SVM) algorithm was used to determine whether each segment of the superpixel segmentation results belonged to the shadow region. Shadow removal was carried out on each detected shadow region, and a natural light image after shadow removal was obtained. Finally, the accuracy of shadow detection was tested. The experimental results showed that the average accuracy of the shadow detection algorithm in this study was 91.91%. As a result, the precision and recall for fruits recognition after shadow removal generally improved.

Keywords

Shadow detection Fruit detection Feature extraction Shadow removal Support vector machine 

Notes

Acknowledgements

This work is supported by the Natural Science Foundation of Guangdong (No. 2018A030313330), the Special Funds for the Cultivation of Guangdong College Students’ Scientific and Technological Innovation (“Climbing Program” Special Funds.) (pdjh2019a0073), the National Natural Science Foundation of China (Nos. 31201135, 31571568) and the Science and Technology Plan Project of Guangzhou (201802020032). The authors wish to thank the useful comments of the anonymous reviewers to this paper.

Author contributions

BR and XJ conceived and designed the experiments; BR designed the algorithm; BR, CS and ZZ performed the experiments and analyzed the data; BR, XJ, CS, ZZ, GW, YZ and LX wrote the paper.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflicts of interest.

References

  1. Adankon, M. M., & Cheriet, M. (2009). Support vector machine. In S. Z. Li & A. Jain (Eds.), Encyclopedia of biometrics (pp. 1303–1308). Boston, MA, USA: Springer.Google Scholar
  2. Baba, M., Mukunoki, M., & Asada, N. (2004). Shadow removal from a real image based on shadow density. In Proceedings of the ACM SIGGRAPH (pp. 8–12). New York, USA: ACM.Google Scholar
  3. Chang, C.-C., & Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), 1–27.  https://doi.org/10.1145/1961189.1961199.CrossRefGoogle Scholar
  4. Comaniciu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5), 603–619.  https://doi.org/10.1109/34.1000236.CrossRefGoogle Scholar
  5. Finlayson, G. D., Drew, M. S., & Lu, C. (2004). Intrinsic images by entropy minimization. In T. Pajdla & J. Matas (Eds.), Computer vision (ECCV) (pp. 582–595). Berlin, Heidelberg, Germany: Springer.Google Scholar
  6. Finlayson, G. D., Drew, M. S., & Lu, C. (2009). Entropy minimization for shadow removal. International Journal of Computer Vision, 85(1), 35–57.  https://doi.org/10.1007/s11263-009-0243-z.CrossRefGoogle Scholar
  7. Finlayson, G., Fredembach, C., & Drew, M. S. (2007). Detecting illumination in images. In 2007 IEEE 11th international conference on computer vision (ICCV) (pp. 1–8). Los Alamitos, CA, USA: IEEE.Google Scholar
  8. Finlayson, G. D., Hordley, S. D., & Drew, M. S. (2002). Removing shadows from images. In A. Heyden, G. Sparr, M. Nielsen, & P. Johansen (Eds.), Computer vision (ECCV) (pp. 823–836). Berlin, Heidelberg, Germany: Springer.Google Scholar
  9. Finlayson, G. D., Hordley, S. D., Lu, C., & Drew, M. S. (2006). On the removal of shadows from images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1), 59–68.  https://doi.org/10.1109/tpami.2006.18.CrossRefPubMedGoogle Scholar
  10. Freeman, W. T., Pasztor, E. C., & Carmichael, O. T. (2000). Learning low-level vision. International Journal of Computer Vision, 40(1), 25–47.  https://doi.org/10.1023/A:1026501619075.CrossRefGoogle Scholar
  11. Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc.Google Scholar
  12. Gong, H., & Cosker, D. (2017). User-assisted image shadow removal. Image and Vision Computing, 62, 19–27.  https://doi.org/10.1016/j.imavis.2017.04.001.CrossRefGoogle Scholar
  13. Gonzalez, R. C., & Woods, R. E. (2007). Digital image processing (3rd ed.). Upper Saddle River, NJ, USA: Prentice-Hall Inc.Google Scholar
  14. Gonzalez, R. C., Woods, R. E., & Eddins, S. L. (2003). Digital image processing using MATLAB. Upper Saddle River, NJ, USA: Prentice-Hall Inc.Google Scholar
  15. Guo, R. Q., Dai, Q. Y., & Hoiem, D. (2013). Paired regions for shadow detection and removal. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(12), 2956–2967.  https://doi.org/10.1109/tpami.2012.214.CrossRefPubMedGoogle Scholar
  16. Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46(1–3), 389–422.  https://doi.org/10.1023/a:1012487302797.CrossRefGoogle Scholar
  17. Hertog, W., Llenas, A., & Carreras, J. (2015). Optimizing indoor illumination quality and energy efficiency using a spectrally tunable lighting system to augment natural daylight. Optics Express, 23(24), 1564–1574.  https://doi.org/10.1364/oe.23.01564.CrossRefGoogle Scholar
  18. Jobson, D. J., Rahman, Z., & Woodell, G. A. (1997). A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing, 6(7), 965–976.  https://doi.org/10.1109/83.597272.CrossRefPubMedGoogle Scholar
  19. Khan, S. H., Bennamoun, M., Sohel, F., & Togneri, R. (2016). Automatic shadow detection and removal from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(3), 431–446.  https://doi.org/10.1109/tpami.2015.2462355.CrossRefPubMedGoogle Scholar
  20. Lalonde, J. F., Efros, A. A., & Narasimhan, S. G. (2010). Detecting ground shadows in outdoor consumer photographs. In K. Daniilidis, P. Maragos, & N. Paragios (Eds.), Computer vision (ECCV) (Vol. 6312, Lecture Notes in Computer Science, pp. 322–335). Berlin, Heidelberg, Germany: Springer.Google Scholar
  21. Lei, T., Jia, X. H., Zhang, Y. N., He, L. F., Meng, H. Y., & Nandi, A. K. (2018). Significantly fast and robust fuzzy c-means clustering algorithm based on morphological reconstruction and membership filtering. IEEE Transactions on Fuzzy Systems, 26(5), 3027–3041.  https://doi.org/10.1109/tfuzz.2018.2796074.CrossRefGoogle Scholar
  22. Levine, M. D., & Bhattacharyya, J. (2005). Removing shadows. Pattern Recognition Letters, 26(3), 251–265.  https://doi.org/10.1016/j.patrec.2004.10.021.CrossRefGoogle Scholar
  23. MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth berkeley symposium on mathematical statistics and probability (pp. 281–297). Berkeley, CA, USA: University of California Press.Google Scholar
  24. Matsushita, Y., Nishino, K., Ikeuchi, K., & Sakauchi, M. (2004). Illumination normalization with time-dependent intrinsic images for video surveillance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(10), 1336–1347.  https://doi.org/10.1109/tpami.2004.86.CrossRefPubMedGoogle Scholar
  25. Mo, N., Zhu, R. X., Yan, L., & Zhao, Z. (2018). Deshadowing of urban airborne imagery based on object-oriented automatic shadow detection and regional matching compensation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(2), 585–605.  https://doi.org/10.1109/jstars.2017.2787116.CrossRefGoogle Scholar
  26. Otsu, N. (1979). Threshold selection method from gray-level histograms. IEEE Transactions on Systems Man and Cybernetics, 9(1), 62–66.  https://doi.org/10.1109/tsmc.1979.4310076.CrossRefGoogle Scholar
  27. Phong, B. T. (1975). Illumination for computer generated pictures. Communications of the ACM, 18(6), 311–317.  https://doi.org/10.1145/360825.360839.CrossRefGoogle Scholar
  28. Shen, L., & Yeo, C. (2011). Intrinsic images decomposition using a local and global sparse representation of reflectance. In 2011 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 697–704). Los Alamitos, CA, USA: IEEE.Google Scholar
  29. Shen, L., Yeo, C., & Hua, B.-S. (2013). Intrinsic image decomposition using a sparse representation of reflectance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(12), 2904–2915.  https://doi.org/10.1109/tpami.2013.136.CrossRefPubMedGoogle Scholar
  30. Suh, H. K., Hofstee, J. W., & van Henten, E. J. (2018). Improved vegetation segmentation with ground shadow removal using an HDR camera. Precision Agriculture, 19(2), 218–237.  https://doi.org/10.1007/s11119-017-9511-z.CrossRefGoogle Scholar
  31. Sun, J., Tian, J. D., Du, Y. K., & Tang, Y. D. (2009). Retinex theory-based shadow detection and removal in single outdoor image. Industrial Robot: An International Journal, 36(3), 263–269.  https://doi.org/10.1108/01439910910950531.CrossRefGoogle Scholar
  32. Wang, G., Wei, Y., & Qiao, S. (2018). Equation solving generalized inverses. In G. Wang, Y. Wei, & S. Qiao (Eds.), Generalized inverses: Theory and computations (pp. 1–64). Singapore: Springer.CrossRefGoogle Scholar
  33. Wang, S.-T., Yuan, Y.-Y., Zhu, C.-Y., Kong, D.-M., & Wang, Y.-T. (2019). Discrimination of polycyclic aromatic hydrocarbons based on fluorescence spectrometry coupled with CS-SVM. Measurement, 139, 475–481.  https://doi.org/10.1016/j.measurement.2019.01.087.CrossRefGoogle Scholar
  34. Weiss, Y. (2001). Deriving intrinsic images from image sequences. In Proceedings eighth IEEE international conference on computer vision (ICCV 2001) (pp. 68–75). Los Alamitos, CA, USA: IEEE.  https://doi.org/10.1109/iccv.2001.937606.
  35. Wu, T. P., & Tang, C. K. (2005). A Bayesian approach for shadow extraction from a single image. In Tenth IEEE international conference on computer vision (ICCV) (pp. 480–487). Los Alamitos, CA, USA: IEEE.Google Scholar
  36. Zhu, J. J., Samuel, K. G. G., Masood, S. Z., & Tappen, M. F. (2010). Learning to recognize shadows in monochromatic natural images. In 2010 IEEE conference on computer vision and pattern recognition (pp. 223–230). Los Alamitos, CA, USA: IEEE.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Mathematics and InformaticsSouth China Agricultural UniversityGuangzhouChina

Personalised recommendations