Malaria RDT (mRDT) interpretation accuracy by frontline health workers compared to AI in Kano state, Nigeria
Read the publication
Background
Although malaria is preventable and treatable, it continues to be a significant cause of illness and death. Early diagnosis through testing is critical in reducing malaria-related morbidity and mortality. Malaria rapid diagnostic tests (mRDTs) are preferred for their ease of use, sensitivity, and rapid results, yet misadministration and misinterpretation errors persist. This study investigated whether pairing an existing application with an AI-based software could enhance interpretation accuracy among Frontline Healthcare Workers (FHWs) in Kano State, Nigeria.
Methods
A comparative analysis was conducted, examining mRDT interpretations by FHWs, trained expert mRDT reviewers (Panel Readers), and AI-based computer vision algorithms. The accuracy comparisons included: (1) AI interpretation versus Panel Read interpretation, (2) FHW interpretation versus Panel Read interpretation, (3) FHW interpretation versus AI interpretation, and (4) AI performance on faint positive lines. Accuracy was reported as a weighted F1 score, reflecting the harmonic mean of recall (sensitivity) and precision (positive predictive value).
Results
The AI algorithm demonstrated high accuracy, matching Panel Read interpretations correctly for positives 96.38% of the time and negatives 97.12%. FHW interpretations agreed with the Panel Read 96.82% on positives and 94.31% on negatives. Comparison of FHW and AI interpretations showed 97.52% agreement on positives and 93.38% on negatives. The overall accuracy was higher for AI (weighted F1 score of 96.4) compared to FHWs (95.3). Notably, the AI accurately identified 90.2% of 163 faint positive mRDTs, whereas FHWs correctly identified 76.1%.
Conclusion
AI-based computer vision algorithms performed comparably to trained and experienced FHWs and exceeded FHW performance in identifying faint positives. These findings demonstrate the potential of AI technology to enhance the accuracy of mRDT interpretation, thereby improving malaria diagnosis and reporting accuracy in malaria-endemic, resource-limited settings.
