Machine vision for counting fruit on mango tree canopies
- 1.1k Downloads
Machine vision technologies hold the promise of enabling rapid and accurate fruit crop yield predictions in the field. The key to fulfilling this promise is accurate segmentation and detection of fruit in images of tree canopies. This paper proposes two new methods for automated counting of fruit in images of mango tree canopies, one using texture-based dense segmentation and one using shape-based fruit detection, and compares the use of these methods relative to existing techniques:—(i) a method based on K-nearest neighbour pixel classification and contour segmentation, and (ii) a method based on super-pixel over-segmentation and classification using support vector machines. The robustness of each algorithm was tested on multiple sets of images of mango trees acquired over a period of 3 years. These image sets were acquired under varying conditions (light and exposure), distance to the tree, average number of fruit on the tree, orchard and season. For images collected under the same conditions as the calibration images, estimated fruit numbers were within 16 % of actual fruit numbers, and the F1 measure of detection performance was above 0.68 for these methods. Results were poorer when models were used for estimating fruit numbers in trees of different canopy shape and when different imaging conditions were used. For fruit-background segmentation, K-nearest neighbour pixel classification based on colour and smoothness or pixel classification based on super-pixel over-segmentation, clustering of dense scale invariant feature transform features into visual words and bag-of-visual-word super-pixel classification using support vector machines was more effective than simple contrast and colour based segmentation. Pixel classification was best followed by fruit detection using an elliptical shape model or blob detection using colour filtering and morphological image processing techniques. Method results were also compared using precision–recall plots. Imaging at night under artificial illumination with careful attention to maintaining constant illumination conditions is highly recommended.
KeywordsFruit detection Heuristic shape analysis K nearest neighbour Night-time imaging Segmentation Super-pixels Yield prediction
KW acknowledges support from Horticulture Innovation Australia (MT14048/ST15005).
- Chaivivatrakul, S., Moonrinta, J., & Dailey, M. N. (2010). Towards automated crop yield estimation—Detection and 3D reconstruction of pineapples in video sequences. In Proceedings of the Fifth International Conference on Computer Vision Theory and Applications (pp. 180–183). Angers: INSTICC Press.Google Scholar
- Davis, J., & Goadrich, M. (2006). The relationship between precision–recall and ROC curves. In Proceedings of the 23rd International Conference on Machine Learning (pp. 233–240). New York: ACM Publications.Google Scholar
- Hung, C., Nieto, J., Taylor, Z., Underwood, J., & Sukkarieh, S. (2013). Orchard fruit segmentation using multi-spectral feature learning. In Proceedings of the International Conference on Intelligent Robots and Systems (pp. 5314–5320). Tokyo: IEEE.Google Scholar
- Hung, C., Underwood, J., Nieto, J., & Sukkarieh, S. (2015). A feature learning based approach for automated fruit yield estimation. In Field and Service Robotics: Springer Tracts in Advanced Robotics (Vol 12) (pp.485–498).Google Scholar
- Moonrinta, J., Chaivivatrakul, S., Dailey, M. N., & Ekpanyapong, M. (2010). Fruit detection, tracking, and 3D reconstruction for crop mapping and yield estimation. In Proceedings of the International Conference on Intelligent Robots and Systems (pp. 1181–1186). Singapore: IEEE.Google Scholar
- Nuske, S., Achar, S., Bates, T., Narasimhan, S., & Singh, S. (2011) Yield estimation in vineyards by visual grape detection. In Proceedings of the International Conference on Intelligent Robots and Systems (pp. 2352–2358). San Fransico: IEEE.Google Scholar
- Payne, A., & Walsh, K. (2014). Machine vision in estimation of crop yield. In S. D. Gupta & Y. Ibaraki (Eds.), Plant image analysis: Fundamentals and applications. Boca Raton: CRC Press.Google Scholar
- Powers, D. M. W. (2011). Evaluation: from precision, recall and F-measure to ROC, informedness, markedness & correlation. Journal of Machine Learning Technologies, 2(1), 37–63.Google Scholar
- Qureshi, W. S., Satoh, S., Dailey, M. N., & Ekpanyapong, M. (2014). Dense segmentation of textured fruits in video sequences. In Proceedings of the 9th International Conference on Computer Vision Theory and Applications (pp. 441–447). Lisbon: INSTICC Press.Google Scholar
- Roy, A., Banerjee, S., Roy, D., & Mukhopadhyay, A. (2011) Statistical video tracking of pomegranate fruits. In Proceedings of the National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (pp. 227–230). Hubli: IEEE.Google Scholar
- Schillaci, G., Pennisi, A., Franco, F., & Longo, D. (2012). Detecting tomato crops in greenhouses using a vision based method. In Proceedings of International Conference on Safety, Health and Welfare in Agriculture and Agro (pp. 3–6). Ragusa: RAGUSA-SHWA.Google Scholar