A systematic assessment of current genome-scale metabolic reconstruction tools

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

Background: Several genome-scale metabolic reconstruction software platforms have been developed and are being continuously updated. These tools have been widely applied to reconstruct metabolic models for hundreds of microorganisms ranging from important human pathogens to species of industrial relevance. However, these platforms, as yet, have not been systematically evaluated with respect to software quality, best potential uses and intrinsic capacity to generate high-quality, genome-scale metabolic models. It is therefore unclear for potential users which tool best fits the purpose of their research. Results: In this work, we perform a systematic assessment of current genome-scale reconstruction software platforms. To meet our goal, we first define a list of features for assessing software quality related to genome-scale reconstruction. Subsequently, we use the feature list to evaluate the performance of each tool. To assess the similarity of the draft reconstructions to high-quality models, we compare each tool's output networks with that of the high-quality, manually curated, models of Lactobacillus plantarum and Bordetella pertussis, representatives of gram-positive and gram-negative bacteria, respectively. We additionally compare draft reconstructions with a model of Pseudomonas putida to further confirm our findings. We show that none of the tools outperforms the others in all the defined features. Conclusions: Model builders should carefully choose a tool (or combinations of tools) depending on the intended use of the metabolic model. They can use this benchmark study as a guide to select the best tool for their research. Finally, developers can also benefit from this evaluation by getting feedback to improve their software.

Original languageEnglish
Article number158
JournalGenome Biology
Volume20
Issue number1
DOIs
Publication statusPublished - 7 Aug 2019

Fingerprint

Software
genome
Genome
software
Benchmarking
Lactobacillus plantarum
Bordetella pertussis
Pseudomonas putida
Gram-Negative Bacteria
Research
Gram-negative bacteria
pathogen
microorganism
microorganisms
bacterium
pathogens

Keywords

  • Bordetella pertussis
  • Genome-scale metabolic models
  • Genome-scale metabolic reconstruction
  • Lactobacillus plantarum
  • Pseudomonas putida
  • Systematic evaluation

Cite this

@article{68f5a93000fe407cbbf880ec707f0c9f,
title = "A systematic assessment of current genome-scale metabolic reconstruction tools",
abstract = "Background: Several genome-scale metabolic reconstruction software platforms have been developed and are being continuously updated. These tools have been widely applied to reconstruct metabolic models for hundreds of microorganisms ranging from important human pathogens to species of industrial relevance. However, these platforms, as yet, have not been systematically evaluated with respect to software quality, best potential uses and intrinsic capacity to generate high-quality, genome-scale metabolic models. It is therefore unclear for potential users which tool best fits the purpose of their research. Results: In this work, we perform a systematic assessment of current genome-scale reconstruction software platforms. To meet our goal, we first define a list of features for assessing software quality related to genome-scale reconstruction. Subsequently, we use the feature list to evaluate the performance of each tool. To assess the similarity of the draft reconstructions to high-quality models, we compare each tool's output networks with that of the high-quality, manually curated, models of Lactobacillus plantarum and Bordetella pertussis, representatives of gram-positive and gram-negative bacteria, respectively. We additionally compare draft reconstructions with a model of Pseudomonas putida to further confirm our findings. We show that none of the tools outperforms the others in all the defined features. Conclusions: Model builders should carefully choose a tool (or combinations of tools) depending on the intended use of the metabolic model. They can use this benchmark study as a guide to select the best tool for their research. Finally, developers can also benefit from this evaluation by getting feedback to improve their software.",
keywords = "Bordetella pertussis, Genome-scale metabolic models, Genome-scale metabolic reconstruction, Lactobacillus plantarum, Pseudomonas putida, Systematic evaluation",
author = "Mendoza, {Sebasti{\'a}n N.} and Olivier, {Brett G.} and Douwe Molenaar and Bas Teusink",
year = "2019",
month = "8",
day = "7",
doi = "10.1186/s13059-019-1769-1",
language = "English",
volume = "20",
journal = "Genome Biology",
issn = "1465-6906",
publisher = "Springer Verlag",
number = "1",

}

A systematic assessment of current genome-scale metabolic reconstruction tools. / Mendoza, Sebastián N.; Olivier, Brett G.; Molenaar, Douwe; Teusink, Bas.

In: Genome Biology, Vol. 20, No. 1, 158, 07.08.2019.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - A systematic assessment of current genome-scale metabolic reconstruction tools

AU - Mendoza, Sebastián N.

AU - Olivier, Brett G.

AU - Molenaar, Douwe

AU - Teusink, Bas

PY - 2019/8/7

Y1 - 2019/8/7

N2 - Background: Several genome-scale metabolic reconstruction software platforms have been developed and are being continuously updated. These tools have been widely applied to reconstruct metabolic models for hundreds of microorganisms ranging from important human pathogens to species of industrial relevance. However, these platforms, as yet, have not been systematically evaluated with respect to software quality, best potential uses and intrinsic capacity to generate high-quality, genome-scale metabolic models. It is therefore unclear for potential users which tool best fits the purpose of their research. Results: In this work, we perform a systematic assessment of current genome-scale reconstruction software platforms. To meet our goal, we first define a list of features for assessing software quality related to genome-scale reconstruction. Subsequently, we use the feature list to evaluate the performance of each tool. To assess the similarity of the draft reconstructions to high-quality models, we compare each tool's output networks with that of the high-quality, manually curated, models of Lactobacillus plantarum and Bordetella pertussis, representatives of gram-positive and gram-negative bacteria, respectively. We additionally compare draft reconstructions with a model of Pseudomonas putida to further confirm our findings. We show that none of the tools outperforms the others in all the defined features. Conclusions: Model builders should carefully choose a tool (or combinations of tools) depending on the intended use of the metabolic model. They can use this benchmark study as a guide to select the best tool for their research. Finally, developers can also benefit from this evaluation by getting feedback to improve their software.

AB - Background: Several genome-scale metabolic reconstruction software platforms have been developed and are being continuously updated. These tools have been widely applied to reconstruct metabolic models for hundreds of microorganisms ranging from important human pathogens to species of industrial relevance. However, these platforms, as yet, have not been systematically evaluated with respect to software quality, best potential uses and intrinsic capacity to generate high-quality, genome-scale metabolic models. It is therefore unclear for potential users which tool best fits the purpose of their research. Results: In this work, we perform a systematic assessment of current genome-scale reconstruction software platforms. To meet our goal, we first define a list of features for assessing software quality related to genome-scale reconstruction. Subsequently, we use the feature list to evaluate the performance of each tool. To assess the similarity of the draft reconstructions to high-quality models, we compare each tool's output networks with that of the high-quality, manually curated, models of Lactobacillus plantarum and Bordetella pertussis, representatives of gram-positive and gram-negative bacteria, respectively. We additionally compare draft reconstructions with a model of Pseudomonas putida to further confirm our findings. We show that none of the tools outperforms the others in all the defined features. Conclusions: Model builders should carefully choose a tool (or combinations of tools) depending on the intended use of the metabolic model. They can use this benchmark study as a guide to select the best tool for their research. Finally, developers can also benefit from this evaluation by getting feedback to improve their software.

KW - Bordetella pertussis

KW - Genome-scale metabolic models

KW - Genome-scale metabolic reconstruction

KW - Lactobacillus plantarum

KW - Pseudomonas putida

KW - Systematic evaluation

UR - http://www.scopus.com/inward/record.url?scp=85072052116&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072052116&partnerID=8YFLogxK

U2 - 10.1186/s13059-019-1769-1

DO - 10.1186/s13059-019-1769-1

M3 - Article

VL - 20

JO - Genome Biology

JF - Genome Biology

SN - 1465-6906

IS - 1

M1 - 158

ER -