Microsoft A.I
January 1, 2022United Kingdom
February 1, 2022Article of the Month – January 2022
Diagnostic capacity of skin tumor artificial intelligence assisted decision-making software in real-world clinical settings
Diagnostic capacity of skin tumor artificial intelligence assisted decision-making software in real-world clinical settings
Cheng-Xu Li, Wen-Min Fei , Chang-Bing Shen , Zi-Yi Wang , Yan Jing , Ru-Song Meng ,Yong Cui
Background
Youzhi artificial intelligence (AI) software is the AI-assisted decision-making system for diagnosing skin tumors. The high diagnostic accuracy of Youzhi AI software was previously validated in specific datasets. The objective of this study was to compare the performance of diagnostic capacity between Youzhi AI software and dermatologists in real-world clinical settings.
Methods
A total of 106 patients who underwent skin tumor resection in the Dermatology Department of China-Japan Friendship Hospital from July 2017 to June 2019 and were confirmed as skin tumors by pathological biopsy were selected. Dermoscopy and clinical images of 106 patients were diagnosed by Youzhi AI software and dermatologists at different dermoscopy diagnostic levels. The primary outcome was to compare the diagnostic accuracy of the Youzhi AI software with that of dermatologists and that measured in the laboratory using specific data sets. The secondary results included the sensitivity, specificity, positive predictive value, negative predictive value, F-measure, and Matthews correlation coefficient of Youzhi AI software in the real world.
Results
The diagnostic accuracy of Youzhi AI software in real-world clinical settings was lower than that of the laboratory data (P < 0.001). The output result of Youzhi AI software has good stability after several tests. Youzhi AI software diagnosed benign and malignant diseases by recognizing dermoscopic images and diagnosed disease types with higher diagnostic accuracy than by recognizing clinical images (P = 0.008, P = 0.016, respectively). Compared with dermatologists, Youzhi AI software was more accurate in the diagnosis of skin tumor types through the recognition of dermoscopic images (P = 0.01). By evaluating the diagnostic performance of dermatologists under different modes, the diagnostic accuracy of dermatologists in diagnosing disease types by matching dermoscopic and clinical images was significantly higher than that by identifying dermoscopic and clinical images in a random sequence (P = 0.022). The diagnostic accuracy of dermatologists in the diagnosis of benign and malignant diseases by recognizing dermoscopic images was significantly higher than that of recognizing clinical images (P = 0.010).
Conclusions
The diagnostic accuracy of Youzhi AI software for skin tumors in real-world clinical settings was not as high as that of using special data sets in the laboratory. However, there was no significant difference between the diagnostic capacity of Youzhi AI software and the average diagnostic capacity of dermatologists. It can provide assistant diagnostic decisions for dermatologists in the current state.
Relevance to Healthcare Field
Artificial intelligence (AI) uses algorithms and software to interpret and analyze information without human intervention. It has increasingly been used in the healthcare field as a diagnostic tool, but it requires further development. This study is evidence of the need for more technological advancements in the AI field because it can determine that, despite the software’s updates, we still need human analysis for certain samples to be diagnosed correctly. This is especially true in the dermatologic area, where there is a wide variation of skin lesions that require an accurate diagnosis to determine the most appropriate management.