Vector Researcher Geoff Pleiss Enhances ML Model Accuracy Through Uncertainty Quantification

Geoff Pleiss, a statistician at the University of British Columbia and Canada CIFAR AI Chair affiliated with the Vector Institute, is conducting critical research into quantifying uncertainty in machine learning models. His work addresses the 'known unknowns' of AI predictions, aiming to determine how much trust users can place in model outputs. This research is particularly vital for safety-critical applications, such as self-driving cars requiring human intervention alerts, and healthcare systems detecting out-of-distribution data. Additionally, Pleiss explores sequential decision-making and reinforcement learning, where uncertainty helps balance exploitation of known successful strategies against exploration of new possibilities, such as in drug development. A significant challenge identified is that modern neural networks are large, expensive, and often homogenous, making traditional uncertainty techniques ineffective. Pleiss employs 'deep ensembling,' training multiple networks with random variations to measure prediction variance. However, he notes that these networks often collapse into similar predictions despite architectural differences, highlighting the complexity of extracting reliable uncertainty metrics from contemporary AI systems.