Skip to content
1887

Journal of Medical Microbiology logo Journal of Medical Microbiology

Volume 74, Issue 4

Accuracy of classification of urinary Gram-stain findings by a computer-aided diagnosis app compared with microbiology specialists Open Access

Abstract

Timely and accurate diagnosis of bacterial infections enables early administration of appropriate antimicrobial treatment and improved outcomes.

The accuracy of computer-aided diagnosis (CAD) for identifying organisms on urine Gram stains has not been compared with that of microbiology specialists (MS).

To compare the interpretation of urine Gram-stain results by MS and a CAD app designed using artificial intelligence.

Urine specimens from patients with urinary tract infections were used and collected at two tertiary hospitals between 1 April and 31 December 2022. Using non-inferiority analysis to assess whether CAD was non-inferior to expert interpretation, CAD-predicted microscopic findings of the Gram-stained slide generated from iPhone camera images from two hospitals were compared with those from ten MS. A total of 153 images were taken from each hospital, and CAD interpreted a total of 306. The primary endpoint was the prediction accuracy based on the morphology of the Gram-stained bacteria.

The accuracy (95% confidence interval) of MS and CAD predictions was 83.0% (81.6%–84.3%) and 87.9% (83.7%–91.3%), respectively, with a difference of –4.93% (–8.43% to –0.62%) indicating non-inferiority of CAD.

CAD was non-inferior to MS predictions for identifying Gram-stained pathogens; therefore, CAD was suggested to have the potential for guiding empirical antibiotic selection in patients with urinary tract infections.

  • Received:
  • Accepted:
  • Published Online:
Keyword(s): artificial intelligence , computer-aided diagnosis , Gram staining , non-inferiority trial and urinary tract infection (UTI)
Funding
This study was supported by the:
  • Japan Agency for Medical Research and Development (Award JP23hk0102076)
    • Principle Award Recipient: KeiYamamoto
© 2025 The Authors
  • This is an open-access article distributed under the terms of the Creative Commons Attribution License. The Microbiology Society waived the open access fees for this article.
Loading

Article metrics loading...

/content/journal/jmm/10.1099/jmm.0.002008
2025-04-23
2025-05-24
Loading full text...

Full text loading...

/deliver/fulltext/jmm/74/4/jmm002008.html?itemId=/content/journal/jmm/10.1099/jmm.0.002008&mimeType=html&fmt=ahah

References

  1. Im Y, Kang D, Ko R-E, Lee YJ, Lim SY et al. Time-to-antibiotics and clinical outcomes in patients with sepsis and septic shock: a prospective nationwide multicenter cohort study. Crit Care 2022; 26:19 [View Article] [PubMed]
     [Google Scholar]
  2. Ohnuma T, Chihara S, Costin B, Treggiari MM, Bartz RR et al. Association of appropriate empirical antimicrobial therapy with in-hospital mortality in patients with bloodstream infections in the US. JAMA Netw Open 2023; 6:e2249353 [View Article] [PubMed]
     [Google Scholar]
  3. Timbrook TT, Morton JB, McConeghy KW, Caffrey AR, Mylonakis E et al. The effect of molecular rapid diagnostic testing on clinical outcomes in bloodstream infections: a systematic review and meta-analysis. Clin Infect Dis 2017; 64:15–23 [View Article] [PubMed]
     [Google Scholar]
  4. Baltekin Ö, Boucharin A, Tano E, Andersson DI, Elf J. Antibiotic susceptibility testing in less than 30 min using direct single-cell imaging. Proc Natl Acad Sci USA 2017; 114:9170–9175 [View Article] [PubMed]
     [Google Scholar]
  5. Arienzo A, Murgia L, Cellitti V, Ferrante V, Stalio O et al. A new point-of-care test for the rapid antimicrobial susceptibility assessment of uropathogens. PLoS One 2023; 18:e0284746 [View Article] [PubMed]
     [Google Scholar]
  6. Wiwanitkit V, Udomsantisuk N, Boonchalermvichian C. Diagnostic value and cost utility analysis for urine Gram stain and urine microscopic examination as screening tests for urinary tract infection. Urol Res 2005; 33:220–222 [View Article] [PubMed]
     [Google Scholar]
  7. Williams GJ, Macaskill P, Chan SF, Turner RM, Hodson E et al. Absolute and relative accuracy of rapid urine tests for urinary tract infection in children: a meta-analysis. Lancet Infect Dis 2010; 10:240–250 [View Article] [PubMed]
     [Google Scholar]
  8. Taniguchi T, Tsuha S, Shiiki S, Narita M. Gram-stain-based antimicrobial selection reduces cost and overuse compared with Japanese guidelines. BMC Infect Dis 2015; 15:458 [View Article] [PubMed]
     [Google Scholar]
  9. Guarner J, Street C, Matlock M, Cole L, Brierre F. Improving Gram stain proficiency in hospital and satellite laboratories that do not have microbiology. Clin Chem Lab Med 2017; 55:458–461 [View Article] [PubMed]
     [Google Scholar]
  10. Munson E, Block T, Basile J, Hryciuk JE, Schell RF. Mechanisms to assess Gram stain interpretation proficiency of technologists at satellite laboratories. J Clin Microbiol 2007; 45:3754–3758 [View Article] [PubMed]
     [Google Scholar]
  11. Shime N, Yanagihara K, Watanabe M, Morita M, Sasaki M et al. A multicenter survey for microbiological testing in emergency department. J Japan Soc Clin Microbiol 2019; 29:28–31
     [Google Scholar]
  12. Ikegaki S, Ohji G, Yamamoto K, Ohnuma K, Ebisawa K et al. O-149: a study of differences in prediction accuracy in urine Gram stain decipherment by decipherment method and rater. In The 97th annual meeting of the Japanese Association for Infectious Diseases, YokohamaApril28 2023
     [Google Scholar]
  13. Bartholomew JW, Mittwer T. The Gram stain. Bacteriol Rev 1952; 16:1–29 [View Article]
     [Google Scholar]
  14. Hicks SA, Strümke I, Thambawita V, Hammou M, Riegler MA et al. On evaluation metrics for medical applications of artificial intelligence. Sci Rep 2022; 12:5979 [View Article] [PubMed]
     [Google Scholar]
  15. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. Biometrics 1977; 33:159–174 [View Article]
     [Google Scholar]
  16. Feinstein AR, Cicchetti DV. High agreement but low kappa: I. the problems of two paradoxes. J Clin Epidemiol 1990; 43:543–549 [View Article] [PubMed]
     [Google Scholar]
  17. Smith KP, Kang AD, Kirby JE. Automated interpretation of blood culture Gram stains by use of a deep convolutional neural network. J Clin Microbiol 2018; 56:e01521-17 [View Article] [PubMed]
     [Google Scholar]
  18. Wang Z, Zhang L, Zhao M, Wang Y, Bai H et al. Deep neural networks offer morphologic classification and diagnosis of bacterial vaginosis. J Clin Microbiol 2021; 59:e02236-20 [View Article] [PubMed]
     [Google Scholar]
  19. Samuel LP, Balada-Llasat JM, Harrington A, Cavagnolo R. Multicenter assessment of Gram stain error rates. J Clin Microbiol 2016; 54:1442–1447 [View Article] [PubMed]
     [Google Scholar]
  20. Ichikawa L, Ishigaki S, Matsumura M, Asahara M. Case study of morphological changes of Corynebacterium species to gram-positive cocci. Japan J Technol 2018; 67:299–306
     [Google Scholar]
  21. Grubb TC. Coccus forms of Corynebacterium diphtheriae. J Infect Dis 1935; 56:64–77 [View Article]
     [Google Scholar]
  22. Pendergrast J, Marrie T. Reasons for choice of antibiotic for the empirical treatment of CAP by Canadian infectious disease physicians. Can J Infect Dis 1999; 10:337–345 [View Article] [PubMed]
     [Google Scholar]
  23. Yoshimura J, Yamakawa K, Ohta Y, Nakamura K, Hashimoto H et al. Effect of Gram stain-guided initial antibiotic therapy on clinical response in patients with ventilator-associated pneumonia: the GRACE-VAP randomized clinical trial. JAMA Netw Open 2022; 5:e226136 [View Article] [PubMed]
     [Google Scholar]
  24. Yodoshi T, Matsushima M, Taniguchi T, Kinjo S. Utility of point-of-care Gram stain by physicians for urinary tract infection in children ≤36 months. Medicine 2019; 98:e15101 [View Article] [PubMed]
     [Google Scholar]
  25. Taniguchi T, Tsuha S, Shiiki S, Narita M. Point-of-care urine Gram stain led to narrower-spectrum antimicrobial selection for febrile urinary tract infection in adolescents and adults. BMC Infect Dis 2022; 22:198 [View Article] [PubMed]
     [Google Scholar]
  26. Hasegawa S, Livorsi DJ, Perencevich EN, Church JN, Goto M. Diagnostic accuracy of hospital antibiograms in predicting the risk of antimicrobial resistance in Enterobacteriaceae isolates: a nationwide multicenter evaluation at the Veterans Health Administration. Clin Infect Dis 2023; 77:1492–1500 [View Article] [PubMed]
     [Google Scholar]
  27. Shigemura K, Tanaka K, Okada H, Nakano Y, Kinoshita S et al. Pathogen occurrence and antimicrobial susceptibility of urinary tract infection cases during a 20-year period (1983-2002) at a single institution in Japan. Jpn J Infect Dis 2005; 58:303–308 [PubMed]
     [Google Scholar]
  28. Murdoch DR, Greenlees RL. Rapid identification of Staphylococcus aureus from BacT/ALERT blood culture bottles by direct Gram stain characteristics. J Clin Pathol 2004; 57:199–201 [View Article] [PubMed]
     [Google Scholar]
  29. Hadano Y, Isoda M, Ishibashi K, Kakuma T. Validation of blood culture gram staining for the detection of Staphylococcus aureus by the “oozing sign” surrounding clustered gram-positive cocci: a prospective observational study. BMC Infect Dis 2018; 18:490 [View Article] [PubMed]
     [Google Scholar]
  30. Kondo S, Yamada T, Misawa S, Nakamura A, Oguri T. Morphological identification of Staphylococcus spp. from blood culture bottles based on direct Gram staining. Kansenshogaku Zasshi 2008; 82:656–657 [View Article] [PubMed]
     [Google Scholar]
  31. Hayashi T, Yoshida M. The peanuts sign: a novel clue to the rapid discrimination of penicillin-susceptible Enterococci based on Gram staining findings. Kansenshogaku Zasshi 2019; 93:306–311 [View Article]
     [Google Scholar]
  32. Bartlett RC, Mazens-Sullivan MF, Lerer TJ. Differentiation of Enterobacteriaceae, Pseudomonas aeruginosa, and Bacteroides and Haemophilus species in gram-stained direct smears. Diagn Microbiol Infect Dis 1991; 14:195–201 [View Article] [PubMed]
     [Google Scholar]
  33. Ministry of Health, Labour and Welfare, Japan Japan Nosocomial Infections Surveillance (JANIS). n.d https://janis.mhlw.go.jp/english/index.asp accessed 21 November 2023
  34. Wilson ML, Gaido L. Laboratory diagnosis of urinary tract infections in adult patients. Clin Infect Dis 2004; 38:1150–1158 [View Article] [PubMed]
     [Google Scholar]
/content/journal/jmm/10.1099/jmm.0.002008
Loading
Accuracy of classification of urinary Gram-stain findings by a computer-aided diagnosis app compared with microbiology specialists
J. Med. Microbiol. 74, 002008 (2025); https://doi.org/10.1099/jmm.0.002008
/content/journal/jmm/10.1099/jmm.0.002008
/content/journal/jmm/10.1099/jmm.0.002008
Loading

Data & Media loading...

Supplements

Supplementary material 1

PDF

Most cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error