FDA Updates
July 1, 2023
Sophia Genetics
July 1, 2023Article of the Month – July 2023
Artificial Intelligence Predictive Analytics in the Management of Outpatient MRI Appointment No-Shows
A, , ,
Objectives
Outpatient appointment no-shows are a common problem. Artificial intelligence predictive analytics can potentially facilitate targeted interventions to improve efficiency. We describe a quality improvement project that uses machine learning techniques to predict and reduce outpatient MRI appointment no-shows.
Materials and Methods:
Anonymized records from 32,957 outpatient MRI appointments between 2016 and 2018 were acquired for model training and validation, along with a holdout test set of 1080 records from January 2019. The overall no-show rate was 17.4%. A predictive model developed with XGBoost, a decision tree-based ensemble machine learning algorithm using a gradient boosting framework, was deployed after evaluating various machine learning algorithms. The simple intervention measure of using telephone call reminders for patients with the top 25% highest risk of an appointment no-show, as predicted by the model, was implemented over six months.
Results
The ROC AUC for the predictive model was 0.746 with an optimized F1 score of 0.708; at this threshold, the precision and recall were 0.606 and 0.852, respectively. The AUC for the holdout test set was 0.738 with an optimized F1 score of 0.721; at this threshold, the precision and recall were 0.605 and 0.893, respectively. The no-show rate six months after deployment of the predictive model was 15.9% compared with 19.3% in the preceding 12-month preintervention period, corresponding to a 17.2% improvement from the baseline no-show rate (p < 0.0001). The no-show rates of contactable and noncontactable patients in the group at high risk of appointment no-shows, as predicted by the model, were 17.5% and 40.3%, respectively (p < 0.0001).
Conclusions
Machine learning predictive analytics perform moderately well in predicting complex problems involving human behavior using a modest amount of data with basic feature engineering, and they can be incorporated into routine workflow to improve healthcare delivery.
Relevance to Healthcare Field
Hospital appointment no-shows burden healthcare systems worldwide. No-shows waste limited healthcare resources, resulting in long appointment lead times and patients being denied timely care. This study implemented an improvement project to predict outpatient MRI appointment no-shows. It was possible to develop a moderately well-performing machine learning predictive model suitable for improving healthcare delivery and directing future efforts to maximize healthcare resource availability.