Calling on a million minds for community annotation in WikiProteins
- 34k Downloads
WikiProteins enables community annotation in a Wiki-based system. Extracts of major data sources have been fused into an editable environment that links out to the original sources. Data from community edits create automatic copies of the original data. Semantic technology captures concepts co-occurring in one sentence and thus potential factual statements. In addition, indirect associations between concepts have been calculated. We call on a 'million minds' to annotate a 'million concepts' and to collect facts from the literature with the reward of collaborative knowledge discovery. The system is available for beta testing at http://www.wikiprofessional.org.
KeywordsAdditional Data File Resource Description Framework Text Mining Concept Space Tegafur
Unified Medical Language System.
Rationale and overview
This paper aims to explain an experimental system for community annotation and collaborative knowledge discovery called WikiProteins.
The exploding number of papers abstracted in PubMed [1, 2] has prompted many attempts to capture information automatically from the literature and from primary data into a computer readable, unambiguous format. When done manually and by dedicated experts, this process is frequently referred to as 'curation'. The automated computational approach is broadly referred to as text mining. The term text mining itself is ambiguous in that it means very different things to different people . In a recent debate there is a perceived controversy between pure text mining approaches to recover facts from texts and the manual curation approach [3, 4]. We propose here that a combination of text mining and subsequent community annotation of relationships between concepts in a collaborative environment is the way forward .
The future outlook to integrate data mining (for instance gene co-expression data) with literature mining, as formulated in the review by Jensen et al. , is at the core of what we aim for at the text mining/data mining interface. To support the capturing of qualitative as well as quantitative data of different natures into a light, flexible, and dynamic ontology format, we have developed a software component called Knowlets™. The Knowlets combine multiple attributes and values for relationships between concepts.
The first section of this article describes the WikiProteins application and rationale in general terms. The second section describes three user scenarios enabled by the current status of the Knowlet-based Wiki system. In the third section (provided as Additional data file 1) a more detailed technical description of the system is given.
WikiProteins is a web-based, interactive and semantically supported workspace based on Wiki pages and connected Knowlets of over one million biomedical concepts, selected from authorities such as the Unified Medical Language System (UMLS) , UniProtKB/Swiss-Prot  IntAct  and the Gene Ontology (GO) . Progressively more biological databases and ontologies, such as the Genetic Association Database, can be added , although not all of these may have an authoritative status. The terminological data derived from these resources has been entered and mapped to unique concept identifiers in a Wiki-based terminology system called OmegaWiki . More detailed information regarding biomedical concepts can be viewed in the WikiProteins user interface.
Knowlets and their connections can be exported into standard ontology and web languages such as the Resource Description Framework (RDF) and the Web Ontology Language (OWL) . Therefore, any application using these languages will enable the use of Knowlet output for reasoning and querying with programmes such as the SPARQL Protocol and RDF Query Language . The concept space is provided in open access. The system performs a recalculation of the semantic relationships in the entire biomedical concept space at regular intervals.
The Knowlet forms a 'related concept cloud' around a given concept, where each relationship is attributed with a semantic association with a given value. Spurious co-occurrences between concepts of specific semantic types, such as a drug and a disease or a protein and a tissue, in one and the same sentence are rare. Such co-occurrences may still occur, for instance, based on erroneous mapping of ambiguous terms to the wrong concepts. Spurious correlations can be reported and corrected by the community in WikiProteins.
Filters can be applied by users so that only associations between semantic types of their specific interest are shown. Currently, the following semantic groups are supported: anatomy, chemicals, diseases, organisms, proteins (and their genes), and a general class of 'others' (all other semantic types classified in the UMLS ). In addition, Knowlets can be viewed with a 'background mode' filter to mainly show factual and strong co-occurrence associations, and with a 'discovery mode' filter where more weight is given to indirect associations.
The new Wiki component
In WikiProteins, for each source concept a unique Wiki page has been created describing the preferred thesaurus term, the synonyms, one or more definitions and the annotations as derived from authoritative databases.
Registered users can edit records from an authoritative database and change, correct or add data to that record. Upon saving the data, however, a new (copied) record in the community database is created, which can be viewed alongside the original data from the authoritative sources. Thus, the authority and the integrity of the participating authoritative sources are protected. Multiple threads of authorities and the community can be edited separately and can be converged again based on consensus. Several authoritative sources collaborating in this initiative have already indicated that they will formally recognize authors who have contributed significantly to the annotation and refinement of the information on certain concepts, such as proteins.
The first round of indexing and Knowlet creation has yielded over one million biomedical concepts in the Knowlet database, as well as the Knowlets of well over one million authors who currently have publications in PubMed. By matching concept Knowlets with author Knowlets it is now conceivable that those 'million minds' will annotate the few Knowlets most central to their expertise.
Combination of the Wiki and the Knowlet technologies enables the creation of an environment where scientists can combine their daily practice of knowledge discovery with close to real time collaborative comments and annotations. Edits that influence semantic associations will be reported automatically to other interested colleagues, as well as to the owners of the participating authoritative databases. The anticipation is that these resources will be amended, based on the community activities in the Wiki-environment.
Annotators of authoritative sources can use the information in the community database to facilitate their curation work and they can choose to record their activities in the community version as normal edits or comments. The community will judge the newly entered and amended data for credibility, as well as re-edit them where appropriate. This holds for updates in the authoritative source as well as for the community edits.
All edits can be viewed in the community history pages with real names of the editors associated. Thus, the level of expertise of the editor can be revealed easily: the person is a formal annotator, has many publications on the subject, is a formal guardian of this concept, and so on. Because of the formal registration, appropriate credits can also be given to active community annotators. The editor can also add peer reviewed references to the comments, to increase credibility and general acceptance of the edit. The expertise level of contributing community members can be judged from the publications associated with their name and the Knowlet based on their publications. Embryonic functionality review expert profiles will be available in the first launch of WikiProteins. Full social networking aspects, including several parameters relating to level of expertise and official 'guardianship' of certain concepts will be developed in close collaboration with a growing consortium of active users in order to serve the best practises developed.
Concepts for which no terms are present and defined in OmegaWiki are not identified by the Peregrine indexer and thus not highlighted in web pages. Registered users can manually select them in the page and start a new concept page in WikiProteins with one click. A definition of the selected expression will give it defined meaning status and unless the community rejects the entry, the term will soon be considered a valid concept. Each term added in WikiProteins will be synchronized with OmegaWiki, where translations and other terminological additions can be given. The Peregrine indexer will soon highlight newly added concepts, but they will be marked as 'under construction' for a given period of time. When text and references are added to the concept page, the Knowlet of the new concept can be created.
The biomedical literature contains pertinent 'facts', that is, statements of relationships between concepts that are generally considered to be scientifically 'accepted'. Each new article contains many repetitious factual statements, with references, along with a limited number of 'novel' facts. New facts will frequently also cause novel co-occurrences. As a consequence of removing factual redundancy, the number of unique facts (and thus the concept space) expands with only a fraction of the total number of sentences in the biomedical literature (Figure 1; see the 'Rationale and overview' section).
A growing subset of these relevant facts, such as the described functions of proteins, protein-protein interactions or protein-disease relationships, have already been annotated and curated in open access databases and ontologies, such as the UMLS and UniProtKB/Swiss-Prot, IntAct, and GO Annotation. These and other on-line resources have become indispensable tools for current biomedical research. However, the rate of growth of high throughput data and published information in the life sciences renders comprehensive and timely annotation of the literature for actual facts by any central team of experts an unachievable goal. Computer assistance in the annotation process is, therefore, urgently needed.
Recognizing concepts in free text is not trivial, not even for human readers, let alone for computers. The yeast protein CLB2 is an instructive example. The (incorrectly spelled) term 'Clb2', used as an example in , when typed into UniProtKB/Swiss-Prot, leads to 25 entries. One is the correct concept - the gene coding for G2/mitotic-specific cyclin-2 (see Figure 3 for its WikiProteins page) - but the incorrect synonym used by the original authors is not listed in the corresponding Swiss-Prot record, neither as a synonym of the corresponding gene name nor of its protein. But Clb2 is, for instance, also a synonym for emb-9, which encodes the Collagen alpha-1(IV) chain in Caenorhabditis elegans.
In the Saccharomyces Genome Database , the formal name of the gene is CLB2, and the synonym Clb2 is not listed; however, the query term Clb2 leads to the correct gene. A focused database like Saccharomyces Genome Database can let its internal search engine be case insensitive and find CLB2 based on the query term Clb2, but in a wider context, case insensitivity leads to aggravation of the ambiguity problem. For example, in PubMed, the query 'Clb2' delivers papers on dental self-etching primers such as 'Clearfil Liner Bond 2' [PMID: 9522695, 12601887], on the Clb1 gene in the fungal pathogen Ustilago maydis [PMID: 14679309] and on calcineurin B-like proteins, such as CLB1 in Arabidopsis [PMID: 14617077].
For computational meta-analysis this ambiguity is a severe limitation. In earlier microarray case studies we typically found that roughly 40% of all gene names in our lists have homonymy problems of some sort (unpublished data). Most of the re-writing rules to improve 'fuzzy' recall of gene and protein names have negative effects on precision and only marginal positive effects on recall . Thus, non-standardized use of terms in the literature induces vast problems of homonymy and these are not easy to solve.
In WikiProteins, various algorithms have been implemented to keep the homonym problem to the minimum achievable with the current techniques for word sense disambiguation . However, false positives for co-occurrence of two concepts in a sentence based on homonyms still happens occasionally and will be a disturbing factor in WikiProteins also. In contrast to 'read only' sources on the web, in WikiProteins, users are able to enrich the terminology system, thus improving concept recognition in future instances of indexing the same records.
In the natural language of standard scientific literature, the majority of simple facts have been described within one sentence, although in some cases a factual statement may be spread over multiple sentences. Attempting to mine these 'scrambled facts', in early case studies, only marginally increased the recall of actual facts and introduced many errors . Attempts to mine multiple sentences and paragraphs in the broad biomedical literature for all individual instances of a unique factual statement have met with limited success and, in fact, may have very little added value for meta-analysis of the literature as a whole . Unless the fact is very new, multiple instances of statements in sequential publications are only of use, from a computational point of view, to increase the likelihood that the statement is a consolidated fact. For well established facts one does not need to find the very last instance of the factual statement in all papers to be able to present the fact correctly in an ontological format such as the Knowlet. We have chosen, therefore, to analyse texts at the sentence level and accept the trade off with optimal recall of individual statements.
For Knowlet construction the number of sentences found affects the value of the C parameter (Figure 1), but in many instances where the C parameter is positive, there is either factual or associative information involved in the computation of the semantic association. Logical co-occurrences suggested by the mining technologies as 'potential facts' are actively presented to registered experts for community annotation. Where possible, confirmation of factual status should be reported in the Wiki with references to sentences in the peer reviewed literature as supporting evidence.
An additional major limitation of classic text mining approaches is that much of the relevant text is securely behind the firewalls of publishers and is not easily accessible for automated indexing. This is another reason why it is not possible to exclusively rely on computational text mining as a definitive source for facts. In fact, roughly 60% of protein-protein interactions mined from Swiss-Prot and IntAct cannot be found co-occurring in a PubMed sentence or even an abstract (H van Haagen and A Botelho-Bovo, in preparation). This should not be considered surprising, as much of the information leading to those annotations came from full text articles, and within these from tables and figures, many of which are not suited for computer indexing. Thus, a large, intrinsically motivated community of experts is needed to accelerate the curation and annotation process of mined 'potential facts'. Copying of relevant sentences from full text literature with reference to the original article is one of the goals of WikiProteins. Easy tools for recognition of new co-occurrences (that is, not occurring in PubMed), but only in full text articles, are under development. Digital object identifiers of the underpinning articles can be downloaded in the Wiki environment to support factual statements by registered scientists. As more new relationships are validated, this approach may lead to collaborative knowledge discovery. This iterative human-machine interaction is a perceived central aspect of community annotation.
Based partly on the concerns described above, several attempts have already been made to involve the scientific community in annotation [19, 20, 21, 22], but so far with limited success. We postulate that this slow adoption of collaboration via web services is due both to the perception of immature applications for annotation and to the fact that distributed annotation is widely perceived by busy scientists as a service to their colleagues only, and much less as a crucial activity for their own research work with immediate positive returns. However, community annotation aims to create and support stable and growing communities of interest around certain concepts, such as genes/proteins, pathways, diseases and drugs, with incentives for keeping information fully up to date.
Several colleagues have recently communicated a spontaneously growing activity in the current Wikipedia environment to annotate protein and RNA related pages (A Bateman, personal communication). WikiProteins is automatically linked to such community annotations in Wikipedia through the on the fly concept recognition. More direct mapping approaches are being developed. This hyper-linking allows annotations in both environments to be captured in the concept space.
It should be emphasized that editing in Wikipedia is not restricted to traceable registered users and that Wikipedia is meant to represent a neutral point of view. WikiProteins is complementary in that it provides a more structured environment where more original data and scientific debate can be accommodated, as well as a direct collaboration with authoritative sources. We anticipate, therefore, a co-existence and complementary role for Wikipedia and WikiProteins.
A second user scenario is the use of WikiProteins to browse quickly through the concept space for interesting relationships.
To demonstrate the current status of the Knowlet based system we will use the following sentence from PMID 15920482: "Mitotic cyclin (Clb2)-bound Cdc28 (Cdk1 homolog) directly phosphorylated Swe1 and this modification served as a priming step to promote subsequent Cdc5-dependent Swe1 hyperphosphorylation and degradation." Jensen et al  discussed this example in their review and made the following statement regarding this sentence: "Current ad hoc IR systems are not able to retrieve our example sentence when they are given the query 'yeast cell cycle'. Instead, this could be achieved by realizing that 'yeast' is a synonym for S. cerevisiae, that 'cell cycle' is a Gene Ontology term and that the word Cdc28 refers to a S. cerevisiae protein, and finally, by looking up the gene ontology terms that relate to Cdc28 to connect it to the yeast cell cycle. Although this will not be easy, we see this form of query expansion as the next logical step for ad hoc IR." WikiProteins is not to be perceived as an information retrieval (IR) system, but it is illustrated below that the concept space may nevertheless serve this stated need.
First, when the full abstract  is put into the concept recognition window, the ambiguity in the language becomes quite apparent. 'S. cerevisiae' is called 'budding yeast' in the title and the only protein mentioned there is 'Swe1/Wee1'. Furthermore, the authors of this abstract have used several constructs that make text mining difficult as they enter conjugate terms such as 'mitotic cyclin (Clb2)-bound Cdc28 (Cdk1 homolog)', 'Clb2-Cdc28', 'Clb2-Cdc28-phosphorylated Swe1', 'Cdc28/Cdk1', and 'Cdc5/Polo'. Many difficulties are introduced by using non-preferred names for genes and proteins and, particularly, by using dashes and slashes that are not parts of the gene symbol, but are simply separators for conjugated terms. The text further mentions that Wee1 is a protein kinase.
Despite this high degree of ambiguity in the terminology in the test abstract 15920482, the Peregrine indexer recognizes several meaningful concepts in the abstract: the proteins Serine/threonine protein kinase; Wee1 like protein kinase; Protein arginine N-methyltransferase HSL7: Cell division control protein 2, based on the synonyms Cdk1 and Cdc28; the concepts bud neck, and mitotic entry; the GO term cyclin-dependent protein kinase regulator activity; Polo-Box domain, phosphorylation; and the organism Saccharomyces. A click on the PMID 15920482 will lead to the concept web-linked version of the abstract.
When the selected sentence is taken by itself for indexing, only one of the proteins is correctly recognized by the indexer. Nevertheless 'cell cycle' and 'mitosis' are central concepts in the resulting Knowlet. The connection to 'yeast' disappears, which is due to the poor species-specific recognition of proteins in the sentence and the absence of a reference to yeast in the sentence itself.
As a second example, the respective proteins from the case study sentence were mapped with the WikiProteins dictionary look up to the following concepts with the preferred terms: Clb2 = G2/mitotic-specific cyclin-2 (S. cerevisiae) Swiss-Prot P24869; Cdc28 = Cell division control protein 28 (S. cerevisiae) Swiss-Prot P00546; Cdk1 = homolog of Cdc28; Swe1 = Mitosis inhibitor protein kinase SWE1 (S. cerevisiae) Swiss-Prot P32944; Cdc5 = Cell cycle serine/threonine-protein kinase CDC5/MSD2 (S. cerevisiae) Swiss-Prot P32562
The main conclusion from this particular example is that the future aim to associate the selected sentence with the concepts 'yeast' and 'cell cycle' is, in fact, not primarily hampered by the fact that the two terms or their synonyms are not mentioned in the sentence. With this level of language complexity and ambiguity, the problem is more related to the lack of adequate computer-recognition of (wrongly spelled) terms (see also the 'Rationale and overview' section). Methods that take context and factual knowledge from databases into account, like the one described here, will relate the case study sentence to the desired terms.
It should be emphasized again that creating a factual and associated concept space around 'yeast cell cycle' with appropriate links to supporting sentences for each edge in the network is a more useful approach to knowledge discovery than the retrieval of a single sentence.
Collaborative knowledge discovery
The third scenario serves to demonstrate the potential for knowledge discovery using the WikiProteins resource and community annotation.
When PubMed was consulted, out of 2,991 abstracts on tegafur, several mentioned the enzyme as a target for the drug. An 'AND' query with 'malaria' and TS yields 55 abstracts among which is the article 'Evaluation of the activities of pyrimethamine analogs against Plasmodium vivax and Plasmodium falciparum dihydrofolate reductase-thymidylate synthase (TS) using in vitro enzyme inhibition and bacterial complementation assays' by Bunyarataphan et al. . This abstract contains the sentence: "The 50% inhibitory concentrations derived from PvDHFR-TS-dependent bacteria were correlated with their corresponding inhibition constants (Ki) from an enzyme inhibition assay, pointing to the likelihood that the potent enzyme inhibitors will also have potent anti-malarial activities." The procedure described has correctly revealed an indirect association in the concept space that could indicate that tegafur is a candidate anti-malarial drug.
Final considerations and future outlook
Collaborative knowledge discovery and alerts as a major incentive
The system presented here is the very first start of an environment in which social networks of scientists with a common interest can collaborate with the aim of making the representation of 'their' part of the concept space more accurate, which has the immediate potential to uncover and alert them to previously evasive relationships in the process. For instance, in the hypothetical situation that it was recently discovered that tegafur inhibits the function of TS and that the first paper about this association with the enzyme was indexed in WikiProteins, experts who had saved an 'antimalarials' Knowlet in their system would be alerted, because the concept 'tegafur' would have entered the concept space of that Knowlet.
Moreover, the concept space of any drug or gene list will naturally contain a number of associated diseases that may be interesting for further study. In the light of the recent commentary by Chong and Sullivan , this could be approached systematically to generate candidate diseases for drugs in a more sophisticated way than has been done previously [24, 25].
Wikiprofessional, a crucial future element of community annotation
The concept profiles of more than one million individual authors from PubMed have already been pre-constructed. Using disambiguation algorithms as partly described before , as many papers as possible for each unique author have been automatically collected. With these papers, an 'Author concept profile' has been constructed and, subsequently, an Author Knowlet. These will be augmented by a highly curated database from Latin America, CV Lattes , which contains considerable overlap with the PubMed Author collection, but will also enrich the system with authors that cannot be easily found via PubMed. The concept space of experts enables WikiProteins and any other conceived professional Wiki to assign any fact, potential fact or potentially interesting associative concept combination to a natural community of interest, namely to that group of authors that share most of the concepts in question in their personal publication Knowlet. Thereby, the system can target alerts to specific and knowledgeable groups of experts. Even if only a small percentage of these experts reacts by commenting on a suggested fact or an interesting new association beyond direct co-occurrence, the facts can become 'community reviewed' and approved by a list of experts until they can be accepted as 'factual'. By participating in the system with an author approved Knowlet, scientists can contribute to the collection and approval of facts from the literature, but they can also confirm 'factuality' of (C+) or even only (A+) relationships in their expertise area, which will influence the overall concept space.
The first release of WikiProteins contains an embryonic version of what is intended to be developed into a fully functional WikiProfessionals in 2008 and beyond. Users are able to review their pre-constructed (recent) publication list and create their Knowlet before registration. With an increasing number of authors having curated their own Knowlet(s) in the system, creating communities of expertise and indicating their availability for comments and peer review, instant messaging and web conferencing will become available in the system. The system also bears great potential to create a unique author ID, a stated need in all publishing environments.
External indexers, databases and ontologies
The data contained in the open content Wiki-environment are open for any research group to be analysed with their own taggers, indexers and text analysis algorithms. Generating additional data by such efforts can capture the relevant factual, co-occurrence and associative relationships in the publicly available system. In addition, all owners of authoritative databases or biomedical ontologies are invited to connect to the Wiki system to enable community-assisted enrichment of their resource. The National Center for Biomedical Ontology (NCBO) will facilitate these efforts.
A consortium has been formed to construct the first public, interactive web service providing both factually and potentially associated concepts, including proteins and genes. Based on frequent meta-analysis of biomedical ontologies, published databases, the Wiki and new sentences collected by the community from the broad scientific literature, the system will actively push new or suggested associations to experts for review and annotation in a Wiki-environment. The resulting community annotation layer is provided in addition to authoritative sources, not as a replacement. It is freely available to all interested parties, and special provisions are made for the participation of colleagues from developing regions.
Curated sources such as UMLS, UniProtKB/Swiss-Prot, IntAct and GO will mine the community layer for material to improve the quality of the records in their own platforms. It is believed that there is room for a persistent co-existence of an editable (Wiki-) layer of provisional annotation along with one or more sanctioned layers of 'established knowledge', defined here as 'authoritative sources'. Additionally, complementary resources can be linked to the system.
It is anticipated that WikiProteins will accelerate computationally assisted, collaborative knowledge discovery in conjunction with annotation of factual information from the literature, from rough data and from direct experimental evidence. The various filters on the concept space reduce the risk for spurious co-occurrences and associations to a minimum, but they also allow users to explore less obvious connections if so desired.
The indexing so far has revealed that the 'million minds' approach can be taken quite literally and the consortium invites the currently active scientific community to annotate minimally one Knowlet on which they are an expert. In doing so, they may also include sentences from articles that are not available in open access and are, therefore, inaccessible to public text mining, with proper reference to the original article.
Potential further developments
In terms of content presently represented in the Wiki and the externally indexed resources, the prototypic version of WikiProteins is only a start. This automatically translates to the richness and the quality of the current Knowlets. The consortium intends to add progressively more authoritative resources to the community annotation system, but in the Wiki-spirit the community is encouraged to bring in new technologies and content into this open access environment. In principle, all high quality resources describing interactions between biologically meaningful concepts could greatly benefit from inclusion in this environment, and their integrity and data-ownership is guaranteed via the authoritative source protection. Although all data in the community database of the Wiki are subject to the GPL-CC-by license (freely downloadable and re-usable), individual authoritative databases retain their own copyright and can put restrictions on the use of their original data supplied to the Wiki for Professionals environment.
The consortium can assist new candidate authoritative sources with technical advice on the development of dedicated import scripts of (selected) data from the source database. Upper ontologies will secure data consistency. Updating of the individual authoritative source will take place through dedicated update scripts for each source, which are anticipated to remain the prime responsibility of the respective database owners. The consortium will also develop technology to allow a more 'federated' approach where original data from authoritative sources are no longer physically imported into the Wiki database but are 'Wikified' in the founder sites.
Once widely used and augmented, this resource could become an open, yet quality assured and comprehensive, environment for collaborative reference and knowledge discovery.
Additional data files
The following additional data are available with the online version of this paper. Additional data file 1 provides a more detailed technical description of the construction of Knowlets and the Wiki system.
This work was financially supported by Knewco, Inc., and by the Centre for Medical Systems Biology, through the Biorange grant SP 3.5.1. of the Netherlands Bioinformatics Centre and the Netherlands Genomics Initiative. Christine Chichester was also supported by the Leiden University Medical Centre. Erik van Mulligen and Barend Mons are also supported by the Erasmus Medical Centre, department of Medical Informatics. Olivier Bodenreider critically reviewed the manuscript in various stages. The Peregrine indexer was originally developed by Martijn Schuemie at the Erasmus Medical Centre in Rotterdam.
- 6.Bodenreider O: The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res. 2004, D267-D270. 10.1093/nar/gkh061. 32 DatabaseGoogle Scholar
- 7.Wu CH, Apweiler R, Bairoch A, Natale DA, Barker WC, Boeckmann B, Ferro Gasteiger E, Huang H, Lopez R, Magrane M, Martin MJ, Mazumder R, O'Donovan C, Redaschi N, Suzek B: The Universal Protein Resource (UniProt): an expanding universe of protein information. Nucleic Acids Res. 2006, D187-D191. 10.1093/nar/gkj161. 34 DatabaseGoogle Scholar
- 8.Hermjakob H, Montecchi-Palazzi L, Lewington C, Mudali S, Kerrien S, Orchard S, Vingron M, Roechert B, Roepstorff P, Valencia A, Margalit H, Armstrong J, Bairoch A, Cesareni G, Sherman D, Apweiler R: IntAct: an open source molecular interaction database. Nucleic Acids Res. 2004, D452-D455. 10.1093/nar/gkh052. 32 DatabaseGoogle Scholar
- 9.Harris MA, Clark J, Ireland A, Lomax J, Ashburner M, Foulger R, Eilbeck K, Lewis S, Marshall B, Mungall C, Richter J, Rubin GM, Blake JA, Bult C, Dolan M, Drabkin H, Eppig JT, Hill DP, Ni L, Ringwald M, Balakrishnan R, Cherry JM, Christie KR, Costanzo MC, Dwight SS, Engel S, Fisk DG, Hirschman JE, Hong EL, Nash RS, Sethuraman A, Theesfeld CL, Botstein D, Dolinski K, et al: The Gene Ontology (GO) database and informatics resource. Nucleic Acids Res. 2004, D258-261. 32 DatabaseGoogle Scholar
- 11.OmegaWiki. [http://www.omegawiki.org]
- 13.SPARQL. [http://en.wikipedia.org/wiki/SPARQL]
- 15.Saccharomyces Genome Database. [http://www.yeastgenome.org]
- 19.Nagaraj SH, Deshpande N, Gasser RB, Ranganathan S: ESTExplorer: an expressed sequence tag (EST) assembly and annotation platform. Nucleic Acids Res. 2007, W143-W147. 10.1093/nar/gkm378. 35 Web ServerGoogle Scholar
- 24.Weeber M, Vos R, Klein H, De Jong-Van Den Berg LT, Aronson AR, Molema G: Generating hypotheses by discovering implicit associations in the literature: a case report of a search for new potential therapeutic uses for thalidomide. J Am Med Inform Assoc. 2003, 10: 252-259. 10.1197/jamia.M1158.PubMedPubMedCentralCrossRefGoogle Scholar
- 25.Weeber M, Klein H, Aronson AR, Mork JG, de Jong-van den Berg LT, Vos R: Text-based discovery in biomedicine: the architecture of the DAD-system. Proc AMIA Symp. 2000, 903-907.Google Scholar
- 26.Torvik VI, Weeber M, Swanson DR, Smalheiser NR: A probabilistic similarity metric for Medline records: a model for author name disambiguation. AMIA Ann Symp Proc. 2003, 1033-Google Scholar
- 27.CVLattes. [http://lattes.cnpq.br/eng/]
- 28.Salvador Declaration on Open Access. [http://www.icml9.org/channel.php?lang=en&channel=91&content=439]
- 30.Weeber M, Schijvenaars BJ, Van Mulligen EM, Mons B, Jelier R, Eijk Van Der CC, Kors JA: Ambiguity of human gene symbols in LocusLink and MEDLINE: creating an inventory and a disambiguation test collection. AMIA Annu Symp Proc. 2003, 704-708.Google Scholar
- 33.Jelier R, Jenster G, Dorssers LC, Wouters BJ, Hendriksen PJ, Mons B, Delwel R, Kors JA: Text-derived concept profiles support assessment of DNA microarray data for acute myeloid leukemia and for androgen receptor stimulation. BMC Bioinformatics. 2007, 8: 14-10.1186/1471-2105-8-14.PubMedPubMedCentralCrossRefGoogle Scholar
- 38.Freebase. [http://www.freebase.com]
- 39.Semantic Mediawiki. [http://meta.wikimedia.org/wiki/Semantic_MediaWiki]
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.