Software testing in microbial bioinformatics: a call to action

Boas C.L. van der Putten; C. I. Mendes; Brooke M. Talbot; Jolinda de Korne-Elenbaas; Rafael Mamede; Pedro Vila-Cerqueira; Luis Pedro Coelho; Christopher A. Gulvik; Lee S. Katz; The ASM NGS 2020 Hackathon participants

doi:10.1099/mgen.0.000790

Volume 8, Issue 3

Other

Open Access

Software testing in microbial bioinformatics: a call to action

Boas C.L. van der Putten^1,2, C. I. Mendes³, Brooke M. Talbot⁴, Jolinda de Korne-Elenbaas^1,5, Rafael Mamede³, Pedro Vila-Cerqueira³, Luis Pedro Coelho^6,7, Christopher A. Gulvik⁸, Lee S. Katz^9,10 and The ASM NGS 2020 Hackathon participants
View Affiliations Hide Affiliations

Affiliations: ¹ Department of Medical Microbiology, Amsterdam UMC, University of Amsterdam, the Netherlands ² Department of Global Health, Amsterdam Institute for Global Health and Development, Amsterdam UMC, University of Amsterdam, the Netherlands ³ Instituto de Microbiologia, Instituto de Medicina Molecular, Faculdade de Medicina, Universidade de Lisboa, Lisboa, Portugal ⁴ Department of Biological and Biomedical Sciences, Emory University, Atlanta, GA, USA ⁵ Department of Infectious Diseases, Public Health Laboratory, Public Health Service of Amsterdam, the Netherlands ⁶ Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, PR China ⁷ Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence, PR China ⁸ Bacterial Special Pathogens Branch, Division of High-Consequence Pathogens and Pathology, Centers for Disease Control and Prevention, Atlanta, GA, USA ⁹ Enteric Diseases Laboratory Branch, Division of Foodborne, Waterborne, and Environmental Diseases, Centers for Disease Control and Prevention, Atlanta, GA, USA ¹⁰ Center for Food Safety, University of Georgia, Griffin, GA, USA
*Correspondence: Boas C.L. van der Putten, [email protected]
Published: 08 March 2022 https://doi.org/10.1099/mgen.0.000790

Abstract

Computational algorithms have become an essential component of research, with great efforts by the scientific community to raise standards on development and distribution of code. Despite these efforts, sustainability and reproducibility are major issues since continued validation through software testing is still not a widely adopted practice. Here, we report seven recommendations that help researchers implement software testing in microbial bioinformatics. We have developed these recommendations based on our experience from a collaborative hackathon organised prior to the American Society for Microbiology Next Generation Sequencing (ASM NGS) 2020 conference. We also present a repository hosting examples and guidelines for testing, available from https://github.com/microbinfie-hackathon2020/CSIS.

Received: 14/10/2021
Accepted: 02/02/2022
Published Online: 08/03/2022

Keyword(s): computational biology , continuous integration and software testing

Funding

This study was supported by the:

Fundação para a Ciência e a Tecnologia (Award 2020.08493.BD)
- Principle Award Recipient: RafaelMamede
ZJLab
- Principle Award Recipient: LuisPedro Coelho
Shanghai Municipal Science and Technology Major Project (Award 2018SHZDZX01)
- Principle Award Recipient: LuisPedro Coelho
Fundação para a Ciência e a Tecnologia (Award SFRH/BD/129483/2017)
- Principle Award Recipient: CInes Mendes

This is an open-access article distributed under the terms of the Creative Commons Attribution License.

Article metrics loading...

/content/journal/mgen/10.1099/mgen.0.000790

2022-03-08

2024-04-19

Full text loading...

/deliver/fulltext/mgen/8/3/mgen000790.html?itemId=/content/journal/mgen/10.1099/mgen.0.000790&mimeType=html&fmt=ahah

References

Stodden V, Seiler J, Ma Z. An empirical analysis of journal policy effectiveness for computational reproducibility. Proc Natl Acad Sci U S A 2018; 115:2584–2589 [View Article] [PubMed]
[Google Scholar]
Krafczyk M, Shi A, Bhaskar A, Marinov D, Stodden V. Scientific tests and continuous integration strategies to enhance reproducibility in the scientific software context. In Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems Phoenix, AZ, USA: Association for Computing Machinery; 2019 pp 23–28
[Google Scholar]
Chang G, Roth CB, Reyes CL, Pornillos O, Chen Y-J et al. Retraction. Science 2006; 314:1875 [View Article] [PubMed]
[Google Scholar]
Hall BG, Salipante SJ. Retraction: Measures of clade confidence do not correlate with accuracy of phylogenetic trees. PLoS Comput Biol 2007; 3:e158 [View Article] [PubMed]
[Google Scholar]
Busby B, Lesko M. August 2015 and January 2016 Hackathon participants Federer L. Closing gaps between open software and public data in a hackathon setting: User-centered software prototyping. F1000Res 2016; 5:672 [View Article] [PubMed]
[Google Scholar]
Gruening B, Sallou O, Moreno P, da Veiga Leprevost F, Ménager H et al. Recommendations for the packaging and containerizing of bioinformatics software. F1000Res 2018; 7:742 [View Article] [PubMed]
[Google Scholar]
Trockman A, Zhou S, Kästner C, Vasilescu B. Adding sparkle to social coding: an empirical study of repository badges in the npm ecosystem. In Proceedings of the 40th International Conference on Software Engineering Gothenburg, Sweden: Association for Computing Machinery; 2018 pp 511–522
[Google Scholar]
Karimzadeh M, Hoffman MM. Top considerations for creating bioinformatics software documentation. Brief Bioinform 2018; 19:693–699 [View Article] [PubMed]
[Google Scholar]
O’Toole Á, Scher E, Underwood A, Jackson B, Hill V et al. Assignment of epidemiological lineages in an emerging pandemic using the pangolin tool. Virus Evol 2021; 7:veab064 [View Article] [PubMed]
[Google Scholar]
Silva M, Machado MP, Silva DN, Rossi M, Moran-Gilad J et al. chewBBACA: A complete suite for gene-by-gene schema creation and strain identification. Microb Genom 2018; 4: [View Article] [PubMed]
[Google Scholar]

http://instance.metastore.ingenta.com/content/journal/mgen/10.1099/mgen.0.000790

Software testing in microbial bioinformatics: a call to action

M Gen 8, 000790 (2022); https://doi.org/10.1099/mgen.0.000790

/content/journal/mgen/10.1099/mgen.0.000790

Data & Media loading...

Supplements

Volume 8, Issue 3

Other

Open Access

Software testing in microbial bioinformatics: a call to action

Abstract

Funding

Supplementary material 1

Most read this month

Most cited Most Cited RSS feed

ResFinder – an open online resource for identification of antimicrobial resistance genes in next-generation sequencing data and prediction of phenotypes from genotypes

Bakta: rapid and standardized annotation of bacterial genomes via alignment-free sequence identification

MOB-suite: software tools for clustering, reconstruction and typing of plasmids from draft assemblies

Completing bacterial genome assemblies with multiplex MinION sequencing

SNP-sites: rapid efficient extraction of SNPs from multi-FASTA alignments

ClermonTyping: an easy-to-use and accurate in silico method for Escherichia genus strain phylotyping

Identification of Klebsiella capsule synthesis loci from whole genome data

Emergence, molecular mechanisms and global spread of carbapenem-resistant Acinetobacter baumannii

Microreact: visualizing and sharing data for genomic epidemiology and phylogeography

The diversity of Klebsiella pneumoniae surface polysaccharides