High-quality research produces results that can be confidently used as the basis for generating new knowledge or for application purposes (e.g. the development of a new drug). When choosing collaborators the trust in the quality of their research outputs is a key factor. Thus high-quality research and successful collaborative research go hand in hand. Robustness of results is increased when it can be confirmed by combining different methods and tools and even more when results are obtained while sharing and learning different approaches and languages of science. When doing collaborative research, there is the best opportunity to combine the different experiences and expertise of all partners by design to create a more efficient and effective environment conductive for high-quality research.
Most IMI consortia perform collaborative research that is highly multidisciplinary and where research teams work jointly across the public and private sector. For example, the IMI “Methods for systematic next generation oncology biomarker development”, ONCOTRACK project (Methods for Systematic Next Generation Oncology Biomarker Development ONcoTRACK Project 2016), is a precompetitive research project that was created to tackle the general problem of identification and validation of clinically robust biomarkers in oncology (Schütte et al. 2017). The project team included eight EFPIA industry teams, nine universities/research institutions and four SMEs. Among the outputs of this consortium is a high-throughput screening platform for three-dimensional patient-derived colon cancer organoid cultures. The platform has been fully validated for assay robustness and reproducibility, an achievement only possible via the collaborative efforts of the multidisciplinary industry-academia team, built on a foundation of trust in the quality of the work of each partner with criteria agreed and applied from the very beginning of the partnership (Boehnke et al. 2016).
However, such complex cross-fertilisation comes with some caveats. It must not be underestimated that, for such collaborative efforts to succeed, all partners need time and good will to adjust to each other’s way of working and thinking. Technical jargon can be very different; the same acronym may have very different meanings (e.g. API: “Active pharmaceutical ingredient” or “application programming interface”). The time and resources necessary for this preparatory work should be considered carefully with attention to aspects such as a good communication platform and legal support. It can be a lengthy and challenging process: timelines should be adjusted accordingly, and expectations!
Trust is the uppermost key factor of success for a research collaboration. In a collaboration, researchers depend for success on both their own results and those of their partners. Since it might be challenging and impracticable to share fully details of work going on in different laboratories, the application of high-quality standards, understood and agreed by all partners from the very start of the collaborative work, is an important enabler for a successful partnership. Once such standards are in place, there is higher motivation and opportunity for the achievement of robust high-quality results and conclusions.
Collaborative research has enabled research programmes at an unprecedented scale. Typical examples are those from genetic research. Here, large-scale collaborative genome studies have delivered huge amount of data opening new avenues for the understanding of disease biology via the use and reuse of these data by many scientists. A stringent adherence to good research practice and quality control is necessary when working at such scale. Among the many programmes is worth mentioning the “Encyclopedia of DNA Elements, ENCODE initiative funded by the National Human Genome Research Institute” (Encyclopedia of DNA Elements (ENCODE) 2019). ENCODE aims to identify all functional elements in the human and mouse genomes and make them available through the project’s freely accessible database. The ENCODE project has developed standards for each experiment type to ensure high-quality, reproducible data and novel algorithms to facilitate analysis (ENCODE Consortium 2017). As a result of outreach and collaboration, enabled by such quality-driven approach, ENCODE has been highly successful, and its data are widely used to deliver high-quality publications.
While standards are needed to deliver a successful collaboration, conversely collaborative research can be a powerful tool to boost the development and implementation of standards and interoperability of results, which again significantly enhances research quality as shown by two further examples below.
Modelling and simulation (M&S), a technology providing the basis for informed, quantitative decision-making, is of high importance in modern drug development. A lack of common tools, languages and ontologies for M&S often leads to inefficient reuse of data and duplication of effort by academic, industrial and regulatory stakeholders, as well as hindering research quality. The IMI “Drug Disease Model Resources”, DDMoRe (Drug Disease Model Resources DDMoRe Project 2012), consortium delivered a set of integrated tools, exchange standards and training to improve the quality and cost-effectiveness of model-informed decision-making for pharmaceutical research and development. The set of standards has been designed both for model and workflow encoding and for storage and transfer of models and associated metadata. One of the key products the project developed based on these standards is the publicly available DDMoRe model repository (DDMoRe Model Repository 2017). It provides access to more than 100 annotated and “ready to use” pharmacokinetic (PK), pharmacodynamic (PD), PK/PD, physiologically based PK (PBPK), statistical and systems biology models applied in different therapeutic areas like oncology, diabetes and neuroscience. The model repository content is quality assured by experts from the DDMoRe model review group, who provides on-demand impartial review and assesses the model’s technical validity and reproducibility. The models that pass the review are certified and can be confidently reused by anyone either commercially or for research purposes.
Rheumatoid arthritis (RA) is a very common and debilitating condition due to many underlying disease mechanisms, thus the plethora of animal models of which the translatability and reproducibility is not well established. The IMI “Be The Cure” BTCure (“Be The Cure” BTCure Project 2012) consortium developed an infrastructure to standardise procedures to generate and interpret commonly used RA animal models, as well as to generate new types of RA animal models. Their work has shown how data obtained from these models might lack quality and reproducibility, due to insufficient documentation and nomenclature, wrong presentation of results and data, as well as the selection of inappropriate models and strains. Most importantly, the BTCure team proposed relevant solutions and developed training material to improve the quality of RA models (Holmdahl 2015). The achievement has been possible only by the joint collaborative work of key opinion leaders in the field of RA to build a critical mass of experts and achieve consensus via a series of dedicated workshops (e.g. BTCure Consortium 2012).
High-quality research has external validity. This does not mean its results are fully reproducible in an identical manner anywhere and anytime but that each result comes together with the awareness and understanding on the potential contextual factors that determine the variation over space or time. Collaborative research enables several partners in a consortium to replicate each other results and then share the obtained knowledge. This dramatically increases the robustness of the research outputs allowing to understand factors impinging on reproducibility of results.
For example, drug-induced liver injury (DILI) is a serious issue not only for patients and health-care professionals but also for the pharmaceutical industry and regulatory authorities. Human-specific and idiosyncratic adverse reactions are often detected only at the clinical and post-marketing stages leading to costly termination of drug development and risk for the patients with black box warnings or even withdrawal of drugs from the market. DILI remains a significant problem in drug development, suggesting that currently used in vitro models are not appropriate for effective screening. The IMI “Mechanism-Based Integrated Systems for the Prediction of Drug-Induced Liver Injury” MIP-DILI (Mechanism-Based Integrated Systems for the Prediction of Drug-Induced Liver Injury MIP-DILI Project 2012) consortium has run a comprehensive, multicentre, unbiased assessment to test this unequivocally. The consortium used a panel of compounds implicated in DILI in man, in order to determine whether any of these simple cell models per se are actually predictive of human DILI. Furthermore, by using a small panel of DILI- and non-DILI-implicated compounds and basic measures of cell health, it monitored reproducibility across different sites, thereby ensuring that data should be more definitive than any currently available (Sison-Young et al. 2017).
Multicentre collaborations can expose systematic biases and identify critical factors to be standardised. Human-induced pluripotent stem cells (iPSCs) are powerful tools for novel in vitro models in basic science and drug discovery. iPSCs need to be differentiated using lengthy complex procedures with increased possibility for variability and noise in the results. The IMI “Stem cells for biological assays of novel drugs and predictive toxicology” (STEMBANCC) project (Stem Cells for Biological Assays of Novel Drugs and Predictive Toxicology Project 2014) runs a unique assessment of the inter- and intra-laboratory reproducibility of transcriptomic and proteomic read-outs using two iPSC lines at five independent laboratories in parallel. By achieving larger sample numbers in a collaborative approach with cross-laboratory studies, the team could detect identifiable sources of variation that investigators can control. This study also strongly advocates for transparency via disclosure of identified variation-inflating confounders in published iPSC differentiation protocols (Volpato et al. 2018).