Uncertainty-Aware Deep Neural Network Training for Imbalanced Geochemical Data Distributions

• Ali Dashti, 
• Michael Trumpp, 
• Lars H. Ystroem, 
• Valentin Goldberg, 
• Nancy Seimetz & 
• Fabian Nitschke
  Abstract

The growing interest in raw material extraction, particularly in trace elements, highlights the need for innovative geochemical modeling techniques to predict element concentrations accurately. This paper explores the predictive capabilities of a deep neural network (DNN) in estimating the concentrations of 20 trace elements based on 11 major elements and pH values. Using data from the BrineMine project, we applied DNNs to a challenging dataset characterized by a small sample size and imbalanced distributions. In total, 1000 independent DNN models were generated to address prediction accuracy and uncertainty instead of relying on a single model. Two preprocessing methods, including synthetic minority over-sampling technique for regression with Gaussian noise (SMOGN) statistical transformation, were applied to improve the accuracy and decrease uncertainty further. Despite issues such as low initial correlations between input features and target variables, imbalanced data distributions, and extremely low concentrations, the DNN models provided reliable and robust results, except for Cu and V. For 13 trace elements, the DNN models achieved acceptable reliability with R² > 0.8. Analyzing the weight distribution of the DNN revealed that input features with high cross-correlation are prone to sharing the same information. While input features such as Fe, pH, and Mg are highly correlated to several target variables, accumulated local effects (ALE) scores indicate that Li has the highest influence, as it is the only input feature with a high correlation coefficient to some of the target variables.