Pathogen metadata platform: software for accessing and analyzing pathogen strain information
- 959 Downloads
Pathogen metadata includes information about where and when a pathogen was collected and the type of environment it came from. Along with genomic nucleotide sequence data, this metadata is growing rapidly and becoming a valuable resource not only for research but for biosurveillance and public health. However, current freely available tools for analyzing this data are geared towards bioinformaticians and/or do not provide summaries and visualizations needed to readily interpret results.
We designed a platform to easily access and summarize data about pathogen samples. The software includes a PostgreSQL database that captures metadata useful for disease outbreak investigations, and scripts for downloading and parsing data from NCBI BioSample and BioProject into the database. The software provides a user interface to query metadata and obtain standardized results in an exportable, tab-delimited format. To visually summarize results, the user interface provides a 2D histogram for user-selected metadata types and mapping of geolocated entries. The software is built on the LabKey data platform, an open-source data management platform, which enables developers to add functionalities. We demonstrate the use of the software in querying for a pathogen serovar and for genome sequence identifiers.
This software enables users to create a local database for pathogen metadata, populate it with data from NCBI, easily query the data, and obtain visual summaries. Some of the components, such as the database, are modular and can be incorporated into other data platforms. The source code is freely available for download at https://github.com/wchangmitre/bioattribution.
KeywordsPathogen Metadata BioSample LabKey Biosurveillance Geocoding PostgreSQL Java
Document object model
Geogenomic mutational atlas of pathogens
Genomic standards consortium
Minimum information about a genome sequence
Minimum information about a marker gene sequence
Minimum information about a metagenome sequence
Minimum information about any sequence
National center for biotechnology information
Sequence read archive
Virus pathogen database and analysis resource
With advances in DNA sequencing technology, whole genome sequencing of pathogen strains from disease outbreaks is becoming routine. These advances are resulting in enormous growth in the amount of publicly available pathogen nucleotide sequence data. One critical component of this data is high-quality metadata about biological samples. This metadata includes information about where the sample originated and the sample’s phenotypic properties. These types of features include, but are not limited to, geolocation data, isolation source, collection date, the organization performing collection, sample and strain names, and drug or vaccine resistance information. Pathogen sample metadata presents new opportunities for diagnostic and treatment discovery, biosurveillance, and public health investigations. In order for many of these opportunities to be realized, pathogen metadata needs to be made easily accessible to those beyond the bioinformatics community.
There has been significant growth in the capture and sharing of pathogen metadata. The Genomic Standards Consortium (GSC) has developed a set of “Minimal Information about any Sequence” (MIxS) checklists for genomes (MIGS), including checklists specifically for pathogen samples [1, 2]. Recently, a consortium of pathogen-sequencing institutions created a new metadata standard for pathogens, called the GSCID/BRC (Genome Sequencing Centers for Infectious Diseases and Bioinformatics Resource Centers) Project and Sample Application Standard . Repositories for pathogen metadata have also been created. The National Center for Biotechnology Information (NCBI) maintains the BioSample and BioProject databases , which contain metadata about biological samples and projects, respectively. This data is typically submitted by investigators in concert with submission of nucleotide sequence data. BioSample and BioProject databases exchange data with their European and Japanese counterparts . The Pathosystems Resource Integration Center (PATRIC) and the Virus Pathogen Database and Analysis Resource (ViPR) also provide standardized metadata for some pathogenic bacterial and viral genomes, respectively [6, 7]. The Genomes Online Database (GOLD) , developed at the Department of Energy Joint Genomes Institute, is a manually curated warehouse of metadata about sequencing experiments following the MIxS standards. There have also been a number of tools developed to query and retrieve this metadata. The Entrez system at the NCBI  provides an interface for searching and filtering query results, and tools such as BioPython , BioPerl , and BioJava  provide functionality for interfacing with these web services. SRAdb enables access to the Sequence Read Archive metadata using R .
For biosurveillance and public health endeavors, there are advantages to hosting an independent data platform incorporating publicly available pathogen metadata. In particular, this allows institutions to integrate other data critical for the mission and analyze it in concert with NCBI sample data. For biosurveillance and public health, the joint analysis of pathogen metadata and epidemiological data will be particularly important. Institutions may also have additional pathogen sample data not associated with genomes, or sample data an institution does not want to make public to be analyzed in concert with publicly available data. Furthermore, a separate database allows institutions to customize the database by further standardizing data or adding data fields and tables.
This manuscript describes a web server application designed to make pathogen metadata readily accessible to biologists, biosurveillance analysts, and public health investigators without requiring computer programming. The software includes a database for the capture of pathogen metadata, scripts to populate the database with metadata from NCBI BioSample and BioProject and a user interface to query, obtain standardized metadata, and visually summarize results.
The sample metadata database schema
Overview of the tables in the sample metadata database
Identity of a sample, including strain name, serovar, and submission date
Where, when and what type of environment the sample was collected from
Information about the human host for clinical samples, such as age and gender
Information about non-human hosts for environmental samples
Methods used for obtaining and identifying a sample
Information about the project associated with the strain
Links projects to samples
Information about the organization that submitted the information about a strain
Information about the organization that collected the strain
Links a sample to publications by PubMed Identifier
Stores source and id pairs for documents and databases that reference a sample
Scripts to import, parse, and standardize metadata from NCBI
The import of NCBI metadata into the metadata database is handled in four steps. In the first step, performed by the DataDownload.sh script, the BioProject and BioSample XML files are downloaded from the NCBI FTP server. Next, the DataSplit.sh script splits the single XML file provided by NCBI into multiple files containing a subset of the nodes relevant to the database schema for more efficient parsing. Parsing is performed by a Java program, which uses a document object model (DOM) parser to map the XML files to Java classes, create tables, and load the data into the database. When the BioProject and BioSample XML schemas are changed by NCBI, the parser code will need to be updated to reflect the changes. The DataMapping.sh script calls the parser and pre-parses the XML files to create a mapping between BioProject and BioSample files. Finally, the DataUpdate.sh calls the parser twice – once to create the database, and once to load the data into the database.
LabKey module for database query and visualization
In addition to the built-in table and graph views from LabKey, the module adds the ability to summarize the results of a query in the form of a 2D histogram. The visualization, which is built using D3.js , creates a two-dimensional histogram using two variables selected by the user. The visualization is interactive, allowing the user to mouse over to see the exact count for any given combination.
In addition to the 2D histogram view, the software provides functionality to geocode based on any column in a List (LabKey’s user-created database tables) and display the results on a map. In this distribution, the geocoding and mapping is performed using a Google Maps API (https://developers.google.com/maps/), though this could be changed by the end user to use a geospatial analysis package of their choice.
Results and discussion
In this section, we highlight two examples showing how the Pathogen Metadata Platform can be used in the investigation of disease outbreaks. In these examples, the database has been populated with data from NCBI on October 27, 2014. Time to populate the database will depend on the current size of BioSample and BioProject, connection speed, parameters used for splitting, and processor speed. On our system, upload time for the database in May 2016, with size 4.7 GB, was less than 16 h.
Identifying and Summarizing Strain Data for a Pathogen Species
Obtaining and visualizing information about closely related pathogen strains
In this example, investigators have sequenced a pathogen sample from a patient and performed phylogenetic analyses using RAxML , phylogenetic software that uses a maximum likelihood approach. This identified 22 Salmonella enterica serovar Typhimurium genomes from NCBI that are closely related to the patient’s strain. The investigator wants to know where and what type of environments these closely related strains came from.
The collection locations of these strains are then mapped. To do this, the table of results is exported as a LabKey list. The “Strain Geography” tab within the LabKey Module allows the user to select this list, along with the column containing the location information to be passed to the geocoder. A map is then presented, with each strain with a location returned by the geocoder displayed as a point on the map (Fig. 2d). Here, we see that the majority of the closely-related strains found within the United States are located in the northeast.
Relationship to other resources
The Pathogen Metadata Platform offers a few advantages relative to other currently available resources. First, once installed, the platform provides an easy way to query and obtain tables of standardized metadata. In this respect is it similar to capabilities offered in ViPR for some virus genomes , and in PATRIC for assembled bacterial genomes , but provides access to all sample entries in BioSample including for the growing number associated with unassembled genomic data. Second, the platform integrates mapping of geographical locations for genomes from a large database. Available software for mapping geolocations of pathogen genomes includes Supramap, which superimposes phylogenies onto a map , and GoMap, which is currently implemented to map HIV strains with drug resistance mutation information . Unlike these, the Pathogen Metadata Platform links mapping with all samples from BioSample, though without a DNA analysis component. In addition, the platform provides interactive 2D histograms to show the variables most strongly associated with the queried pathogen, such as types of environments the pathogen is frequently collected from. Interactive summary figures for pathogen genome metadata have not been incorporated into other webserver applications yet, but provide a way to understand pathogen context quickly, especially when there are large numbers of genomes per species.
The Pathogen Metadata Platform provides functionalities for parsing and loading metadata from NCBI into a relational schema, as well as query and visualization capabilities. This open-source software is modular, such that some components can be individually incorporated into other platforms and modified for specific purposes. For example, the metadata database could be used with other software, and data from sources other than NCBI can be added to it. In addition, the software is extensible, and the LabKey platform provides the opportunity to develop modules for additional analyses. We believe this software will be particularly useful as a complement to DNA analyses, as it has been in our own research. The platform could be paired with easy-to-use DNA analysis software that assesses the relatedness of pathogen strains to enable biosurveillance and public health investigations.
Availability and requirements
Project Name: Pathogen Metadata Platform
Project Home Page: https://github.com/wchangmitre/bioattribution
Operating system: Linux
Programming Environment: Java, SQL
Requirements: A working installation of LabKey Server and PostgreSQL database server
License: Apache License
We would like to thank Marc Colosimo for review of the platform, Marcia Lazo for testing the software installation, and Lynette Hirschman for helpful advice on this work. Approved for Public Release; Distribution Unlimited. Case Number 15- 3669 ©2015 The MITRE Corporation. ALL RIGHTS RESERVED.
This work was supported by the MITRE Innovation Program.
Availability of data and material
The software for the Pathogen Metadata Platform is freely available at https://github.com/wchangmitre/bioattribution.
WC built the database, implemented data import and data standardization, built the queries, contributed to the LabKey user interfaces, geocoding utility, system design, and co-wrote the manuscript. MP conceived of the initial system, contributed to the system design, and co-wrote the manuscript. CG built the 2D histogram component, and contributed to the queries, LabKey user interfaces and geocoding utility. TK selected metadata types for the database, contributed to system design, tested the software, and co-wrote the manuscript. All authors read and approved the final manuscript.
The authors declare no competing interests.
Consent for publication
Ethics approval and consent to participate
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.