“Role of Calibration in Uncertainty-based Referral for Deep Learning” & “Supervised Machine Learning Workflows for Electronic Health Records”

Ruotao Zhang, MSc: “Role of Calibration in Uncertainty-based Referral for Deep Learning” The uncertainty in predictions from deep neural network analysis of medical imaging is challenging to assess but potentially important to include in subsequent decision making. Using data from diabetic retinopathy detection, we present an empirical evaluation of model performance and the impact of uncertainty-based referral, an approach that prioritizes referral of observations based on the magnitude of a measure of uncertainty. We consider several configurations of network architecture, method for uncertainty estimation, and training data size. We identify a strong relationship between the effectiveness of uncertainty-based referral and having a well-calibrated model. This is especially relevant as complex deep neural networks tend to have high calibration errors. Finally, we provide evidence that post-calibration of the neural network can improve uncertainty-based referral. Dilum Aluthge, MD, PhD student: “Supervised Machine Learning Workflows for Electronic Health Records” Supervised machine learning can be used to develop clinical decision support systems for use in electronic health records (EHRs). The first portion of the talk will provide an overview of the supervised machine learning workflow. The second portion will present an example application of classification using EHR data, specifically the problem list and medication list from a patient’s chart.