Article of the Month – October 2023

Published by Daniel at October 15, 2023

Performance of Machine Learning Algorithms for Predicting Progression to Dementia in Memory Clinic Patients

JAMA Network Open – Psychiatry

Importance

Machine learning algorithms could be used as the basis for clinical decision-making aids to enhance clinical practice.

Objectives

To assess the ability of machine learning algorithms to predict dementia incidence within 2 years compared with existing models and determine the optimal analytic approach and number of variables required.

Design, Setting, and Participants

This prognostic study used data from a prospective cohort of 15 307 participants without dementia at baseline to perform a secondary analysis of factors that could be used to predict dementia incidence. Participants attended National Alzheimer Coordinating Center memory clinics across the United States between 2005 and 2015. Analyses were conducted from March to May 2021.

Exposure

258 variables spanning domains of dementia-related clinical measures and risk factors.

Main Outcomes and Measures:

The main outcome was incident all-cause dementia diagnosed within 2 years of baseline assessment.

Results:

In a sample of 15 307 participants (mean [SD] age, 72.3 [9.8] years; 9129 [60%] women and 6178 [40%] men) without dementia at baseline, 1568 (10%) received a diagnosis of dementia within 2 years of their initial assessment. Compared with 2 existing models for dementia risk prediction (i.e., Cardiovascular Risk Factors, Aging, and Incidence of Dementia Risk Score, and the Brief Dementia Screening Indicator), machine learning algorithms were superior in predicting incident all-cause dementia within 2 years. The gradient-boosted trees algorithm had a mean (SD) overall accuracy of 92% (1%), sensitivity of 0.45 (0.05), specificity of 0.97 (0.01), and area under the curve of 0.92 (0.01) using all 258 variables. Analysis of variable importance showed that only 6 variables were required for machine learning algorithms to achieve an accuracy of 91% and area under the curve of at least 0.89. Machine learning algorithms also identified up to 84% of participants who received an initial dementia diagnosis that was subsequently reversed to mild cognitive impairment or cognitively unimpaired, suggesting possible misdiagnosis.

The six Predictive Variables and Their Ranks After Substituting for Highly Correlated Variables

Notes: Colors correspond to the following categories: Green: Neuropsychological Battery Summary scores; Yellow: Clinical judgment of symptoms; Blue: Clinical dementia rating; Orange: Subject demographics. Abbreviations: RF: Random Forest, LR: Logistic Regression, SVM: Support Vector Machine, XGB: Gradient-boosted Trees.

Cumulative Distribution Functions of the Classification Scores from Machine Learning Models

Conclusions And Relevance:

These findings suggest that machine learning algorithms could accurately predict incident dementia within 2 years in patients receiving care at memory clinics using only 6 variables. These findings could be used to inform the development and validation of decision-making aids in memory clinics.

Relevance to Healthcare Field

Dementia is the loss of cognitive functioning that interferes with a person’s life and activities. According to the World Health Organization, there are 55 million people with dementia worldwide, and it is the seventh leading cause of death. Patient diagnosis and classification with different dementia subtypes can be challenging for clinicians. This study uses a Machine Learning model that applies four different algorithms to determine if the patient will develop dementia or not. Compared to the CAIDE and BDSI (dementia risk prediction models, within 20 and six years, respectively), this study can predict dementia faster (within 29 months) and more accurately. Because dementia affects not only the patient but also their family members and caregivers, being able to predict when this condition will develop can help in managing conditions that can be preventable, such as Vascular Dementia, or provide the appropriate treatment and training for those types that cannot be stopped from progressing, such as Alzheimer’s Dementia.