Bayesian Neural Networks for Probabilistic Surrogate Models ﾖ Uncertainty Quantification, Propagation, and Sensitivity Analysis

Abstract

Neural networks are powerful function approximators which scale to problems having large input dimensionality. Bayesian neural networks are an interesting choice for surrogate models as they (1) natively enable performing sensitivity analyses by quantifying the derivative of the function's output with respect to its inputs, (2) are able to quantify heteroscesastic model prediction uncertainties, (3) are able to quantify the epistemic uncertainties associated with parameters. However, a main limitation is that they do not allow propagating uncertainties analytically from the model inputs to its outputs. The tractable approximate Gaussian inference method (TAGI) enables propagating uncertainties analytically from the model inputs to its outputs, making it suited to be used as a surrogate model for probabilistic setups. One key limitation of TAGI is that, up to now, it relied on locally linearized activation functions. The result of that approximation is that the input uncertainties only affect the output variances without modifying the output expected values, and the output variances are only accurate for small magnitudes of input-uncertainties. The objectives of this paper are twofold: first it is to introduce the TAGI method for Bayesian neural network to the surrogate modelling community and second, to present a new method based on a mixture of truncated Gaussians to replace the local linearization in order to accurately propagate uncertainties through Bayesian neural networks.