1887

Microbial Genomics logo

Volume 6, Issue 2

Discordant bioinformatic predictions of antimicrobial resistance from whole-genome sequencing data of bacterial isolates: an inter-laboratory study Open Access

Abstract

Antimicrobial resistance (AMR) poses a threat to public health. Clinical microbiology laboratories typically rely on culturing bacteria for antimicrobial-susceptibility testing (AST). As the implementation costs and technical barriers fall, whole-genome sequencing (WGS) has emerged as a ‘one-stop’ test for epidemiological and predictive AST results. Few published comparisons exist for the myriad analytical pipelines used for predicting AMR. To address this, we performed an inter-laboratory study providing sets of participating researchers with identical short-read WGS data from clinical isolates, allowing us to assess the reproducibility of the bioinformatic prediction of AMR between participants, and identify problem cases and factors that lead to discordant results. We produced ten WGS datasets of varying quality from cultured carbapenem-resistant organisms obtained from clinical samples sequenced on either an Illumina NextSeq or HiSeq instrument. Nine participating teams (‘participants’) were provided these sequence data without any other contextual information. Each participant used their choice of pipeline to determine the species, the presence of resistance-associated genes, and to predict susceptibility or resistance to amikacin, gentamicin, ciprofloxacin and cefotaxime. We found participants predicted different numbers of AMR-associated genes and different gene variants from the same clinical samples. The quality of the sequence data, choice of bioinformatic pipeline and interpretation of the results all contributed to discordance between participants. Although much of the inaccurate gene variant annotation did not affect genotypic resistance predictions, we observed low specificity when compared to phenotypic AST results, but this improved in samples with higher read depths. Had the results been used to predict AST and guide treatment, a different antibiotic would have been recommended for each isolate by at least one participant. These challenges, at the final analytical stage of using WGS to predict AMR, suggest the need for refinements when using this technology in clinical settings. Comprehensive public resistance sequence databases, full recommendations on sequence data quality and standardization in the comparisons between genotype and resistance phenotypes will all play a fundamental role in the successful implementation of AST prediction using WGS in clinical microbiology laboratories.

  • Received:
  • Accepted:
  • Published Online:
Keyword(s): antimicrobial resistance , antimicrobial-susceptibility testing , bioinformatics , carbapenem resistance and whole-genome sequencing
Funding
This study was supported by the:
  • EU Horizon 2020 (Award 675412)
    • Principle Award Recipient: Andreu Coello Pelegrin
  • European Metrology Programme for Innovation and Research (Award HLT07)
    • Principle Award Recipient: Ronan M Doyle
© 2020 The Authors
  • This is an open-access article distributed under the terms of the Creative Commons Attribution License.
Loading

Article metrics loading...

/content/journal/mgen/10.1099/mgen.0.000335
2020-02-12
2024-04-28
Loading full text...

Full text loading...

/deliver/fulltext/mgen/6/2/mgen000335.html?itemId=/content/journal/mgen/10.1099/mgen.0.000335&mimeType=html&fmt=ahah

References

  1. O’Neill J. Tackling Drug-Resistant Infections Globally: Final Report and Recommendations London: Review on Antimicrobial Resistance; 2016
     [Google Scholar]
  2. World Health Organization Global Action Plan on Antimicrobial Resistance ( http://www.who.int/iris/bitstream/10665/193736/1/9789241509763_eng.pdf) Geneva: World Health Organization; 2015
     [Google Scholar]
  3. Fleming A. Classics in infectious diseases: on the antibacterial action of cultures of a Penicillium, with special reference to their use in the isolation of B. influenzae by Alexander Fleming, reprinted from the British Journal of Experimental Pathology 10:226-236, 1929. Rev Infect Dis 1980; 2:129–139
     [Google Scholar]
  4. Archer GL, Pennell E. Detection of methicillin resistance in staphylococci by using a DNA probe. Antimicrob Agents Chemother 1990; 34:1720–1724 [View Article]
     [Google Scholar]
  5. Marlowe EM, Novak-Weekley SM, Cumpio J, Sharp SE, Momeny MA et al. Evaluation of the Cepheid Xpert MTB/RIF assay for direct detection of Mycobacterium tuberculosis complex in respiratory specimens. J Clin Microbiol 2011; 49:1621–1623 [View Article]
     [Google Scholar]
  6. Hays JP, Mitsakakis K, Luz S, van Belkum A, Becker K et al. The successful uptake and sustainability of rapid infectious disease and antimicrobial resistance point-of-care testing requires a complex 'mix-and-match' implementation package. Eur J Clin Microbiol Infect Dis 2019; 38:1015–1022 [View Article]
     [Google Scholar]
  7. van Belkum A, Bachmann TT, Lüdke G, Lisby JG, Kahlmeter G et al. Developmental roadmap for antimicrobial susceptibility testing systems. Nat Rev Microbiol 2019; 17:51–62 [View Article]
     [Google Scholar]
  8. Török ME, Peacock SJ. Rapid whole-genome sequencing of bacterial pathogens in the clinical microbiology laboratory – pipe dream or reality?. J Antimicrob Chemother 2012; 67:2307–2308 [View Article]
     [Google Scholar]
  9. Zankari E, Hasman H, Kaas RS, Seyfarth AM, Agersø Y et al. Genotyping using whole-genome sequencing is a realistic alternative to surveillance based on phenotypic antimicrobial susceptibility testing. J Antimicrob Chemother 2013; 68:771–777 [View Article]
     [Google Scholar]
  10. Ellington MJ, Ekelund O, Aarestrup FM, Canton R, Doumith M et al. The role of whole genome sequencing in antimicrobial susceptibility testing of bacteria: report from the EUCAST Subcommittee. Clin Microbiol Infect 2017; 23:2–22 [View Article]
     [Google Scholar]
  11. Tagini F, Greub G. Bacterial genome sequencing in clinical microbiology: a pathogen-oriented review. Eur J Clin Microbiol Infect Dis 2017; 36:2007–2020 [View Article]
     [Google Scholar]
  12. Rossen JWA, Friedrich AW, Moran-Gilad J. ESCMID Study Group for Genomic and Molecular Diagnostics (ESGMD) Practical issues in implementing whole-genome-sequencing in routine diagnostic microbiology. Clin Microbiol Infect 2018; 24:355–360
     [Google Scholar]
  13. Moran-Gilad J. How do advanced diagnostics support public health policy development?. Euro Surveill 2019; 24:1900068 [View Article]
     [Google Scholar]
  14. Votintseva AA, Bradley P, Pankhurst L, Del Ojo Elias C, Loose M et al. Same-day diagnostic and surveillance data for tuberculosis via whole-genome sequencing of direct respiratory samples. J Clin Microbiol 2017; 55:1285–1298
     [Google Scholar]
  15. Doyle RM, Burgess C, Williams R, Gorton R, Booth H et al. Direct whole-genome sequencing of sputum accurately identifies drug-resistant Mycobacterium tuberculosis faster than MGIT culture sequencing. J Clin Microbiol 2018; 56:e00666-18
     [Google Scholar]
  16. Charalampous T, Kay GL, Richardson H, Aydin A, Baldan R et al. Nanopore metagenomics enables rapid clinical diagnosis of bacterial lower respiratory infection. Nat Biotechnol 2019; 37:783792 [View Article]
     [Google Scholar]
  17. Hendriksen RS, Bortolaia V, Tate H, Tyson GH, Aarestrup FM et al. Using genomics to track global antimicrobial resistance. Front Public Health 2019; 7:242 [View Article]
     [Google Scholar]
  18. Zankari E, Hasman H, Cosentino S, Vestergaard M, Rasmussen S et al. Identification of acquired antimicrobial resistance genes. J Antimicrob Chemother 2012; 67:2640–2644 [View Article]
     [Google Scholar]
  19. Clausen PTLC, Zankari E, Aarestrup FM, Lund O. Benchmarking of methods for identification of antimicrobial resistance genes in bacterial whole genome data. J Antimicrob Chemother 2016; 71:2484–2488 [View Article]
     [Google Scholar]
  20. Hunt M, Mather AE, Sánchez-Busó L, Page AJ, Parkhill J et al. ARIBA: rapid antimicrobial resistance genotyping directly from sequencing reads. Microb Genom 2017; 3:mgen.0.000131 [View Article]
     [Google Scholar]
  21. Stoesser N, Batty EM, Eyre DW, Morgan M, Wyllie DH et al. Predicting antimicrobial susceptibilities for Escherichia coli and Klebsiella pneumoniae isolates using whole genomic sequence data. J Antimicrob Chemother 2013; 68:2234–2244 [View Article]
     [Google Scholar]
  22. Tyson GH, McDermott PF, Li C, Chen Y, Tadesse DA et al. WGS accurately predicts antimicrobial resistance in Escherichia coli . J Antimicrob Chemother 2015; 70:2763–2769 [View Article]
     [Google Scholar]
  23. Lemon JK, Khil PP, Frank KM, Dekker JP. Rapid Nanopore sequencing of plasmids and resistance gene detection in clinical isolates. J Clin Microbiol 2017; 55:3530–3543 [View Article]
     [Google Scholar]
  24. Greig DR, Dallman TJ, Hopkins KL, Jenkins C. MinION nanopore sequencing identifies the position and structure of bacterial antibiotic resistance determinants in a multidrug-resistant strain of enteroaggregative Escherichia coli . Microb Genom 2018; 4:mgen.0.000213 [View Article]
     [Google Scholar]
  25. Allix-Béguec C, Arandjelovic I, Bi L, Beckert P, Bonnet M et al. Prediction of susceptibility to first-line tuberculosis drugs by DNA sequencing. N Engl J Med 2018; 379:1403–1415 [View Article]
     [Google Scholar]
  26. Aires-de-Sousa M, Boye K, de Lencastre H, Deplano A, Enright MC et al. High interlaboratory reproducibility of DNA sequence-based typing of bacteria in a multicenter study. J Clin Microbiol 2006; 44:619–621 [View Article]
     [Google Scholar]
  27. Mellmann A, Andersen PS, Bletz S, Friedrich AW, Kohl TA et al. High interlaboratory reproducibility and accuracy of next-generation-sequencing-based bacterial genotyping in a ring trial. J Clin Microbiol 2017; 55:908–913 [View Article]
     [Google Scholar]
  28. Wood DE, Salzberg SL. Kraken: ultrafast metagenomic sequence classification using exact alignments. Genome Biol 2014; 15:R46 [View Article]
     [Google Scholar]
  29. Breitwieser FP, Baker DN, Salzberg SL. KrakenUniq: confident and fast metagenomics classification using unique k-mer counts. Genome Biol 2018; 19:198 [View Article]
     [Google Scholar]
  30. Ondov BD, Treangen TJ, Melsted P, Mallonee AB, Bergman NH et al. Mash: fast genome and metagenome distance estimation using MinHash. Genome Biol 2016; 17:132 [View Article]
     [Google Scholar]
  31. Kim D, Song L, Breitwieser FP, Salzberg SL. Centrifuge: rapid and sensitive classification of metagenomic sequences. Genome Res 2016; 26:1721–1729
     [Google Scholar]
  32. Bankevich A, Nurk S, Antipov D, Gurevich AA, Dvorkin M et al. SPAdes: a new genome assembly algorithm and its applications to single-cell sequencing. J Comput Biol 2012; 19:455–477 [View Article]
     [Google Scholar]
  33. Wick RR, Judd LM, Gorrie CL, Holt KE. Unicycler: resolving bacterial genome assemblies from short and long sequencing reads. PLoS Comput Biol 2017; 13:e1005595 [View Article]
     [Google Scholar]
  34. Tritt A, Eisen JA, Facciotti MT, Darling AE. An integrated pipeline for de novo assembly of microbial genomes. PLoS One 2012; 7:e42304 [View Article]
     [Google Scholar]
  35. Jia B, Raphenya AR, Alcock B, Waglechner N, Guo P et al. Card 2017: expansion and model-centric curation of the comprehensive antibiotic resistance database. Nucleic Acids Res 2017; 45:D566–D573 [View Article]
     [Google Scholar]
  36. de Man TJB, Limbago BM. SSTAR, a stand-alone easy-to-use antimicrobial resistance gene predictor. mSphere 2016; 1:e00050-15
     [Google Scholar]
  37. Inouye M, Dashnow H, Raven L-A, Schultz MB, Pope BJ et al. SRST2: rapid genomic surveillance for public health and hospital microbiology Labs. Genome Med 2014; 6:90 [View Article]
     [Google Scholar]
  38. Gupta SK, Padmanabhan BR, Diene SM, Lopez-Rojas R, Kempf M et al. ARG-ANNOT, a new bioinformatic tool to discover antibiotic resistance genes in bacterial genomes. Antimicrob Agents Chemother 2014; 58:212–220 [View Article]
     [Google Scholar]
  39. Mason A, Foster D, Bradley P, Golubchik T, Doumith M et al. Accuracy of different bioinformatics methods in detecting antibiotic resistance and virulence factors from Staphylococcus aureus whole-genome sequences. J Clin Microbiol 2018; 56:e01815-17 [View Article]
     [Google Scholar]
  40. Ramirez M, Tolmasky M. Amikacin: uses, resistance, and prospects for inhibition. Molecules 2017; 22:2267 [View Article]
     [Google Scholar]
  41. Garijo D, Kinnings S, Xie L, Xie L, Zhang Y et al. Quantifying reproducibility in computational biology: the case of the tuberculosis drugome. PLoS One 2013; 8:e80278 [View Article]
     [Google Scholar]
  42. Loman N, Watson M. So you want to be a computational biologist?. Nat Biotechnol 2013; 31:996–998 [View Article]
     [Google Scholar]
  43. Sandgren A, Strong M, Muthukrishnan P, Weiner BK, Church GM et al. Tuberculosis drug resistance mutation database. PLoS Med 2009; 6:e2 [View Article]
     [Google Scholar]
  44. Flandrois J-P, Lina G, Dumitrescu O. MUBII-TB-DB: a database of mutations associated with antibiotic resistance in Mycobacterium tuberculosis . BMC Bioinformatics 2014; 15:107 [View Article]
     [Google Scholar]
  45. Feldgarden M, Brover V, Haft DH, Prasad AB, Slotta DJ et al. Validating the AMRFinder tool and resistance gene database by using antimicrobial resistance genotype-phenotype correlations in a collection of isolates. Antimicrob Agents Chemother 2019; 63:e00483-19
     [Google Scholar]
  46. Miotto P, Tessema B, Tagliani E, Chindelevitch L, Starks AM et al. A standardised method for interpreting the association between mutations and phenotypic drug resistance in Mycobacterium tuberculosis . Eur Respir J 2017; 50:1701354 [View Article]
     [Google Scholar]
  47. Balloux F, Brønstad Brynildsrud O, van Dorp L, Shaw LP, Chen H et al. From theory to practice: translating whole-genome sequencing (WGS) into the clinic. Trends Microbiol 2018; 26:1035–1048 [View Article]
     [Google Scholar]
http://instance.metastore.ingenta.com/content/journal/mgen/10.1099/mgen.0.000335
Loading
Discordant bioinformatic predictions of antimicrobial resistance from whole-genome sequencing data of bacterial isolates: an inter-laboratory study
M Gen 6, e000335 (2020); https://doi.org/10.1099/mgen.0.000335
/content/journal/mgen/10.1099/mgen.0.000335
/content/journal/mgen/10.1099/mgen.0.000335
Loading

Data & Media loading...

Supplements

Supplementary material 1

EXCEL

Supplementary material 2

PDF

Most cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error