Abstract
Recent studies in neuropsychiatry have highlighted the correlation between facial and brain dysmorphologies. One way of simultaneously analysing the brain and the face of a subject is by reconstructing a whole-head 3D model from structural magnetic resonance imaging (sMRI). However, the use of different reconstruction protocols generates undesired orthogonal rotations of the 3D models. This is a likely situation in multicentric studies that hampers the combination of data from different centers. Although the original sMRI files contain the subject orientation, it is not always possible to access this data. To solve this issue, in this work we propose a novel method to estimate the orientation of 3D heads with rotations of 90\(^\circ \) or multiples thereof around any of the three Cartesian axes as a required step for generating a normalised dataset in terms of orientation. Our proposal creates 2D images from orthogonal projections of the 3D object, transforming orientation estimation into an image classification problem. Experimental results show that our method, using three orthographic views of the 3D head to create the projection image and ResNet50 for classification, achieves an accuracy of 99.7%, which corresponds to 0.15 mean absolute error in rotation, outperforming state-of-the-art point cloud registration methods like DeepBBS and PRNet.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Martínez-Abadías, N., et al.: Understanding brain/face integration from neuropsychiatric disorders. FASEB J. 34, 1–1 (2020). https://doi.org/10.1096/fasebj.2020.34.s1.05095
Myers, L., et al.: Minor physical anomalies in neurodevelopmental disorders: a twin study. Child Adolesc. Psychiatry Ment. Health 11, 57 (2017). https://doi.org/10.1186/s13034-017-0195-y
Hammond, P., Suttie, M.: Large-scale objective phenotyping of 3D facial morphology. Hum. Mutat. 33, 817–825 (2012). https://doi.org/10.1002/humu.22054
Li, M., et al.: Rapid automated landmarking for morphometric analysis of three-dimensional facial scans. J. Anat. 230, 607–618 (2017). https://doi.org/10.1111/joa.12576
Hallgrímsson, B., et al.: Automated syndrome diagnosis by three-dimensional facial imaging. Genet. Med. 22, 1682–1693 (2020). https://doi.org/10.1038/s41436-020-0845-y
Poldrack, R.A., Gorgolewski, K.J.: Making big data open: data sharing in neuroimaging. Nat. Neurosci. 17, 1510–1517 (2014). https://doi.org/10.1038/nn.3818
Abate, A., Bisogni, C., Castiglione, A., Nappi, M.: Head pose estimation: an extensive survey on recent techniques and applications. Pattern Recognit. 127, 108591 (2022). https://doi.org/10.1016/j.patcog.2022.108591
Zhang, Z., Dai, Y., Sun, J.: Deep learning based point cloud registration: an overview. Virtual Reality Intell. Hardw. 2, 222–246 (2020). https://doi.org/10.1016/j.vrih.2020.05.002
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Hou, B., et al.: 3-D reconstruction in canonical co-ordinate space from arbitrarily oriented 2-D images. IEEE Trans. Med. Imaging 37(8), 1737–1750 (2018). https://doi.org/10.1109/TMI.2018.2798801
Namburete, A.I.L. et al.: Fully-automated alignment of 3D fetal brain ultrasound to a canonical reference space using multi-task learning. Med. Image Anal. 46, 1–14 (2018). https://doi.org/10.1016/j.media.2018.02.006
Hezroni I., Drory, A., Giryes, R., Avidan S.: DeepBBS: deep best buddies for point cloud registration. In: 2021 International Conference on 3D Vision (3DV), London, United Kingdom, pp. 342–351 (2021). https://doi.org/10.1109/3DV53792.2021.00044
Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 239–256 (1992). https://doi.org/10.1109/34.121791
Aoki, Y., Goforth, H., Srivatsan, R.A., Lucey, S.: PointNetLK: robust & efficient point cloud registration using PointNet. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7156–7165 (2019). https://doi.org/10.1109/CVPR.2019.00733
Wang, Y., Solomon, J.M.: PRNet: self-supervised learning for partial-to-partial registration. In: Advances in Neural Information Processing Systems. Curran Associates, Inc. (2019)
Hempel, T., Abdelrahman, A.A., Al-Hamadi, A.: 6D rotation representation for unconstrained head pose estimation. In: 2022 IEEE International Conference on Image Processing (ICIP), pp. 2496–2500 (2022). https://doi.org/10.1109/ICIP46576.2022.9897219
Xu, Y., Jung, C., Chang, Y.: Head pose estimation using deep neural networks and 3D point clouds. Pattern Recogn. 121, 108210 (2022). https://doi.org/10.1016/j.patcog.2021.108210
Gomez-Donoso, F., Garcia-Garcia, A., Garcia-Rodriguez, J., Orts-Escolano, S., Cazorla, M.: LonchaNet: a sliced-based CNN architecture for real-time 3D object recognition. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 412–418 (2017). https://doi.org/10.1109/IJCNN.2017.7965883
Carlbom, I., Paciorek, J.: Planar geometric projections and viewing transformations. ACM Comput. Surv. 10, 465–502 (1978). https://doi.org/10.1145/356744.356750
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 84–90 (2017). https://doi.org/10.1145/3065386
Acknowledgements
The research was supported by the Joan Oró grant (FI 2022) from the DRU of the Generalitat de Catalunya and the European Social Fund (2023 FI-2 00160). The authors would also like to thank the Agència de Gestió d’Ajuts Universitaris i de Recerca (AGAUR) of the Generalitat de Catalunya (2021 SGR01396, 2021 SGR00706, 2021 SGR1475), the Spanish Ministry of Science, Innovation, and Universities under grant PID2020-113609RB-C21, and Fondation Jerome Lejeune under grant 2020b cycle-Project No.2001.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Heredia-Lidón, Á. et al. (2023). Automated Orientation Detection of 3D Head Reconstructions from sMRI Using Multiview Orthographic Projections: An Image Classification-Based Approach. In: Pertusa, A., Gallego, A.J., Sánchez, J.A., Domingues, I. (eds) Pattern Recognition and Image Analysis. IbPRIA 2023. Lecture Notes in Computer Science, vol 14062. Springer, Cham. https://doi.org/10.1007/978-3-031-36616-1_48
Download citation
DOI: https://doi.org/10.1007/978-3-031-36616-1_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36615-4
Online ISBN: 978-3-031-36616-1
eBook Packages: Computer ScienceComputer Science (R0)
