Abstract
An uncertainty score along with predictions of a deep learning model is necessary for acceptance and often mandatory to satisfy regulatory requirements. The predominant method to generating uncertainty scores is to utilize a Bayesian formulation of deep learning. In this paper, we present a plug-and-play method to obtain samples from an already optimized model. Specifically, we present a simple, albeit principled methodology, to generate a number of models by sampling along the eigen directions of the Hessian of the converged minimum. We demonstrate the utility of our methods on two challenging medical ultrasound imaging problems - cardiac view recognition and kidney segmentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Daxberger, E., Kristiadi, A., Immer, A., Eschenhagen, R., Bauer, M., Hennig, P.: Laplace redux - effortless Bayesian deep learning. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems. vol. 34, pp. 20089–20103 (2021)
Daxberger, E., Kristiadi, A., Immer, A., Eschenhagen, R., Bauer, M., Hennig, P.: Laplace redux-effortless Bayesian deep learning. Adv. Neural. Inf. Process. Syst. 34, 20089–20103 (2021)
Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real NVP. arXiv preprint arXiv:1605.08803 (2016)
Fort, S., Hu, H., Lakshminarayanan, B.: Deep ensembles: a loss landscape perspective. Cite arXiv:1912.02757 (2019)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: Proceedings of the 33rd International Conference on Machine Learning, pp. 1050–1059 (2016). JMLR.org
Graves, A.: Practical variational inference for neural networks. In: Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS 2011, pp. 2348–2356 (2011)
Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: International Conference on Machine Learning, pp. 1321–1330. PMLR (2017)
Kristiadi, A., Hein, M., Hennig, P.: Learnable uncertainty under Laplace approximations. In: Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence. Proceedings of Machine Learning Research, vol. 161, pp. 344–353. PMLR, 27–30 July 2021
Li, H., Xu, Z., Taylor, G., Studer, C., Goldstein, T.: Visualizing the loss landscape of neural nets. In: Advances in Neural Information Processing Systems, vol. 31. Curran Associates, Inc. (2018)
Naeini, M.P., Cooper, G., Hauskrecht, M.: Obtaining well calibrated probabilities using Bayesian binning. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Ritter, H., Botev, A., Barber, D.: A scalable Laplace approximation for neural networks. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=Skdvd2xAZ
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241 (2015)
Yao, Z., Gholami, A., Keutzer, K., Mahoney, M.W.: PyHessian: neural networks through the lens of the Hessian. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 581–590. IEEE (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ravishankar, H. et al. (2022). Stochastic Weight Perturbations Along the Hessian: A Plug-and-Play Method to Compute Uncertainty. In: Sudre, C.H., et al. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging. UNSURE 2022. Lecture Notes in Computer Science, vol 13563. Springer, Cham. https://doi.org/10.1007/978-3-031-16749-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-16749-2_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16748-5
Online ISBN: 978-3-031-16749-2
eBook Packages: Computer ScienceComputer Science (R0)
