Abstract
Functional-structural modeling and high-throughput phenomics demand tools for 3D measurements of plants. In this work, structure from motion is employed to estimate the position of a hand-held camera, moving around plants, and to recover a sparse 3D point cloud sampling the plants’ surfaces. Multiple-view stereo is employed to extend the sparse model to a dense 3D point cloud. The model is automatically segmented by spectral clustering, properly separating the plant’s leaves whose surfaces are estimated by fitting trimmed B-splines to their 3D points. These models are accurate snapshots for the aerial part of the plants at the image acquisition moment and allow the measurement of different features of the specimen phenotype. Such state-of-the-art computer vision techniques are able to produce accurate 3D models for plants using data from a single free moving camera, properly handling occlusions and diversity in size and structure for specimens presenting sparse canopies. A data set formed by the input images and the resulting camera poses and 3D points clouds is available, including data for sunflower and soybean specimens.
Chapter PDF
Similar content being viewed by others
References
Alenya, G., Dellen, B., Torras, C.: 3D modelling of leaves from color and ToF data for robotized plant measuring. In: 2011 IEEE International Conference on Robotics and Automation, pp. 3408–3414, May 2011
Alenya, G., Dellen, B., Foix, S., Torras, C.: Robotized Plant Probing: Leaf Segmentation Utilizing Time-of-Flight Data. IEEE Robotics & Automation Magazine 20(3), 50–59 (2013)
Baumberg, A., Lyons, A., Taylor, R.: 3D S.O.M.A commercial software solution to 3D scanning. Graphical Models 67(6), 476–495 (2005)
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-Up Robust Features (SURF). Computer Vision and Image Understanding 110(3), 346–359 (2008)
Bellasio, C., OlejnĂÄŤková, J., Tesa, R., Sebela, D., Nedbal, L.: Computer reconstruction of plant growth and chlorophyll fluorescence emission in three spatial dimensions. Sensors (Basel, Switzerland) 12(1), 1052–1071 (2012)
Biskup, B., Scharr, H., Schurr, U., Rascher, U.: A stereo imaging system for measuring structural parameters of plant canopies. Plant, Cell & Environment 30(10), 1299–1308 (2007)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(11), 1222–1239 (2001)
Chéné, Y., Rousseau, D., Lucidarme, P., Bertheloot, J., Caffier, V., Morel, P., Belin, E., Chapeau-Blondeau, F.: On the use of depth camera for 3D phenotyping of entire plants. Computers and Electronics in Agriculture 82, 122–127 (2012)
Dassot, M., Colin, A., Santenoise, P., Fournier, M., Constant, T.: Terrestrial laser scanning for measuring the solid wood volume, including branches, of adult standing trees in the forest environment. Computers and Electronics in Agriculture 89, 86–93 (2012)
Delagrange, S., Rochon, P.: Reconstruction and analysis of a deciduous sapling using digital photographs or terrestrial-LiDAR technology. Annals of Botany 108(6), 991–1000 (2011)
Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD, vol. 96, pp. 226–231 (1996)
Fiorani, F., Schurr, U.: Future scenarios for plant phenotyping. Annual Review of Plant Biology 64, 267–291 (2013)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 24(6), 381–395 (1981)
Flöry, S., Hofer, M.: Constrained curve fitting on manifolds. Computer-Aided Design 40(1), 25–34 (2008)
Foix, S., Alenya, G., Torras, C.: Lock-in Time-of-Flight (ToF) Cameras: A Survey. IEEE Sensors Journal 11(9), 1917–1926 (2011)
Freedman, B., Shpunt, A., Machline, M., Arieli, Y.: Depth mapping using projected patterns. US Patent 8,493,496, July 23, 2013
Furbank, R.T., Tester, M.: Phenomics-technologies to relieve the phenotyping bottleneck. Trends in Plant Science 16(12), 635–644 (2011)
Furukawa, Y., Ponce, J.: Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(8), 1362–1376 (2010)
Godin, C., Costes, E., Sinoquet, H.: A Method for Describing Plant Architecture which Integrates Topology and Geometry. Annals of Botany 84(3), 343–357 (1999)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, April 2004
van der Heijden, G., Song, Y., Horgan, G., Polder, G., Dieleman, A., Bink, M., Palloix, A., van Eeuwijk, F., Glasbey, C.: SPICY: towards automated phenotyping of large pepper plants in the greenhouse. Functional Plant Biology 39(11), 870 (2012)
Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments. The International Journal of Robotics Research 31(5), 647–663 (2012)
Ivanov, N., Boissard, P., Chapron, M., Andrieu, B.: Computer stereo plotting for 3-D reconstruction of a maize canopy. Agricultural and Forest Meteorology 75(1–3), 85–102 (1995)
Kaminuma, E., Heida, N., Tsumoto, Y., Yamamoto, N., Goto, N., Okamoto, N., Konagaya, A., Matsui, M., Toyoda, T.: Automatic quantification of morphological traits via three-dimensional measurement of Arabidopsis. The Plant Journal 38(2), 358–365 (2004)
Lang, A.: Leaf orientation of a cotton plant. Agricultural Meteorology 11(c), 37–51 (1973)
Leung, C., Appleton, B., Buckley, M., Sun, C.: Embedded voxel colouring with adaptive threshold selection using globally minimal surfaces. International Journal of Computer Vision 99(2), 215–231 (2012)
Lourakis, M.I.A., Argyros, A.A.: SBA: A software package for generic sparse bundle adjustment. ACM Transactions on Mathematical Software 36(1), 1–30 (2009)
Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)
Luxburg, U.: A tutorial on spectral clustering. Statistics and Computing 17(4), 395–416 (2007)
Mörwald, T.: Fitting trimmed B-splines to unordered point clouds. http://pointclouds.org/documentation/tutorials/bspline_fitting.php
Newcombe, R.a., Davison, A.J., Izadi, S., Kohli, P., Hilliges, O., Shotton, J., Molyneaux, D., Hodges, S., Kim, D., Fitzgibbon, A.: KinectFusion: Real-time dense surface mapping and tracking. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 127–136. IEEE, October 2011
Newcombe, R.A., Lovegrove, S.J., Davison, A.J.: DTAM: dense tracking and mapping in real-time. In: 2011 International Conference on Computer Vision, pp. 2320–2327, November 2011
Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. Advances in neural information ... (2002)
Paproki, A., Fripp, J., Salvado, O., Sirault, X., Berry, S., Furbank, R.: Automated 3D segmentation and analysis of cotton plants. In: 2011 International Conference on Digital Image Computing: Techniques and Applications, pp. 555–560, December 2011
Paproki, A., Sirault, X., Berry, S., Furbank, R., Fripp, J.: A novel mesh processing based technique for 3D plant analysis. BMC Plant Biology 12, 63 (2012)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12, 2825–2830 (2011)
Piegl, L., Tiller, W.: The NURBS Book. Monographs in Visual Communication, U.S. Government Printing Office (1997)
Rakocevic, M., Sinoquet, H., Christophe, A., Varlet-Grancher, C.: Assessing the Geometric Structure of a White Clover (Trifolium repens L.) Canopy using3-D Digitising. Annals of Botany 86(3), 519–526 (2000)
Rascher, U., Blossfeld, S., Fiorani, F., Jahnke, S., Jansen, M., Kuhn, A.J., Matsubara, S., Märtin, L.L.A., Merchant, A., Metzner, R., Müller-Linow, M., Nagel, K.A., Pieruschka, R., Pinto, F., Schreiber, C.M., Temperton, V.M., Thorpe, M.R., Dusschoten, D.V., Van Volkenburgh, E., Windt, C.W., Schurr, U.: Non-invasive approaches for phenotyping of enhanced performance traits in bean. Functional Plant Biology 38(12), 968 (2011)
Reuzeau, C., Frankard, V., Hatzfeld, Y., Sanz, A., Camp, W.V., Lejeune, P., Wilde, C.D., Lievens, K., de Wolf, J., Vranken, E., Peerbolte, R., Broekaert, W.: Traitmill™: a functional genomics platform for the phenotypic analysis of cereals. Plant Genetic Resources 4(01), 20–24 (2006)
Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings Third International Conference on 3-D Digital Imaging and Modeling, pp. 145–152 (2001)
Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9–13, 2011
Santos, T., Ueda, J.: Automatic 3D plant reconstruction from photographies, segmentation and classification of leaves and internodes using clustering 1. In: Sievänen, R., Nikinmaa, E., Godin, C., Anna Lintunen, P.N. (eds.) Proceedings of the 7th International Conference on Functional-Structural Plant Models, Saariselkä, Finland, pp. 95–97 (2013)
Santos, T.T., de Oliveira, A.A.: Image-based 3D digitizing for plant architecture analysis and phenotyping. In: SaĂşde, A.V., GuimarĂŁes, S.J.F. (eds.) Workshop on Industry Applications (WGARI) in SIBGRAPI 2012 (XXV Conference on Graphics, Patterns and Images). Ouro Preto (2012)
Scaramuzza, D., Fraundorfer, F.: Visual Odometry [Tutorial]. IEEE Robotics & Automation Magazine 18(4), 80–92 (2011)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888–905 (2000)
Sinoquet, H., Moulia, B., Bonhomme, R.: Estimating the three-dimensional geometry of a maize crop as an input of radiation models: comparison between three-dimensional digitizing and plant profiles. Agricultural and Forest Meteorology 55(3–4), 233–249 (1991)
Sirault, X., Fripp, J., Paproki, A., Kuffner, P., Nguyen, C., Li, R., Daily, H., Guo, J., Furbank, R.: PlantScan: a three-dimensional phenotyping platform for capturing the structural dynamic of plant development and growth. In: Proceedings of the 7th International Conference on FunctionalStructural Plant Models, Saariselkä, Finland, pp. 45–48 (2013)
Snavely, N., Seitz, S., Szeliski, R.: Modeling the World from Internet Photo Collections. International Journal of Computer Vision 80(2), 189–210 (2008)
Song, Y., Glasbey, C.A., van der Heijden, G.W.A.M., Polder, G., Dieleman, J.A.: Combining stereo and time-of-flight images with application to automatic plant phenotyping. In: Heyden, A., Kahl, F. (eds.) SCIA 2011. LNCS, vol. 6688, pp. 467–478. Springer, Heidelberg (2011)
Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment – a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) ICCV-WS 1999. LNCS, vol. 1883, p. 298. Springer, Heidelberg (2000)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic Supplementary Material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Santos, T.T., Koenigkan, L.V., Barbedo, J.G.A., Rodrigues, G.C. (2015). 3D Plant Modeling: Localization, Mapping and Segmentation for Plant Phenotyping Using a Single Hand-held Camera. In: Agapito, L., Bronstein, M., Rother, C. (eds) Computer Vision - ECCV 2014 Workshops. ECCV 2014. Lecture Notes in Computer Science(), vol 8928. Springer, Cham. https://doi.org/10.1007/978-3-319-16220-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-16220-1_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16219-5
Online ISBN: 978-3-319-16220-1
eBook Packages: Computer ScienceComputer Science (R0)