Quality in Non-GxP Research Environment
- 2.1k Downloads
Abstract
There has been increasing evidence in recent years that research in life sciences is lacking in reproducibility and data quality. This raises the need for effective systems to improve data integrity in the evolving non-GxP research environment. This chapter describes the critical elements that need to be considered to ensure a successful implementation of research quality standards in both industry and academia. The quality standard proposed is founded on data integrity principles and good research practices and contains basic quality system elements, which are common to most laboratories. Here, we propose a pragmatic and risk-based quality system and associated assessment process to ensure reproducibility and data quality of experimental results while making best use of the resources.
Keywords
ALCOA+ principles Data integrity Data quality EQIPD European Quality in Preclinical Data European Union’s Innovative Medicines Initiative Experimental results Good research practice IMI Non-GxP research environment Quality culture Reproducibility Research quality standard Research quality system Risk-based quality system assessment Transparency1 Why Do We Need a Quality Standard in Research?
Over the past decades, numerous novel technologies and scientific innovation initiated a shift in drug discovery and development models. Progress in genomics and genetics technologies opened the door for personalized medicine. Gene and targeted therapies could give the chance of a normal life for genetically diseased patients. For example, adeno-associated viruses, such as AAV9, are currently used to create new treatments for newborns diagnosed with spinal muscular atrophy (SMA) (Mendell et al. 2017; Al-Zaidy et al. 2019). Similarly, the use of clustered regularly interspaced short palindromic repeats (CRISPR) (Liu et al. 2019) or proteolysis targeting chimeras (PROTACs) (Caruso 2018) is leading to novel cancer therapy developments. The broader use of digitalization, machine learning and artificial intelligence (AI) (Hassanzadeh et al. 2019) in combination with these technologies will revolutionize the drug discovery and clinical study design and accelerate drug development (Pangalos et al. 2019).
Regulators all over the world are closely monitoring these breakthrough scientific advances and drug development revolution. While they evaluate the great promise of innovative medicines, they also raise questions about potential safety risks, ethics and environment. Consequently, new ethical laws and regulations are emerging to mitigate the risks without slowing down innovation. For example, the UK Human Tissue Act became effective in 2006, followed by the Swiss Human Research Act in January 2014 (Swiss-Federal-Government, Effective 1 January 2014); the EU General Data Protection Regulation (No.679/2016, the GDPR) came into effect on May 25, 2018 (EMA 2018a); and the guideline on good pharmacogenomics practice has been in effect since September 2018 (EMA 2018b).
This is exemplified by the EMA Network Strategy to 2020 (EMA 2015), which aims both to promote innovation and to better understand associated risks, in order to provide patients with safe and novel drugs or treatments on the market more rapidly.
This evolving research and regulatory environment, along with many other new challenges, such as aggressive patent litigation cases, increasing burden for approval and reimbursement of new molecular entities (NMEs), challenging market dynamics and high societal pressure enforce radical changes in the research and drug development models of the pharmaceutical industry (Gautam and Pan 2016). In response, most of the pharmaceutical companies have refocused on portfolio management, acquired promising biotechnology companies and developed research collaborations with academia (Palmer and Chaguturu 2017). The goal is to speed up drug development in order to deliver new drugs and new treatments to their patients and customers. Thus, transition from research to drug development should be more efficient. To do so, robust data quality, integrity and reproducibility became essential, and the development of a quality culture across the entire value chain emerged to be critical. Indeed, while many drug development areas already applied the various good practice (GxP) standards and guidances, no recognized quality standard governed discovery and early development. Conversely, discovery activities had to comply with many regulations, such as biosafety, controlled substances and data privacy; thus, there was a real risk of exposure in non-GXP research.
In order to mitigate these newly emerging risks and speed up drug development, some pharmaceutical companies decided to develop their own internal research quality standard (RQS), based on good scientific practices and data integrity, to promote robust science and data quality. The foundations of RQS were the WHO: “Quality Practices in Basic Biomedical Research” (WHO 2005), first published in 2001, and the “Quality in Research Guideline for working in non-regulated research”, published by the British Research Quality Association RQA, in 2006 and revised in 2008 and 2014 (RQA-Working-Party-on-Quality-in-Non-Regulated-Research 2014).
Academic research institutions and laboratories are as committed as their pharmaceutical counterparts to good scientific practices but are largely operating without defined standards. Many universities hold their scientists accountable for good scientific practices, which are mainly focused on preventing misconduct and promoting a collaborative environment. Academic output is measured by the amount of publications, often in prestigious journals. Peer review of manuscripts is seen by academics as the main quality control element. During the last decade, the replication and reproducibility crisis in biomedical sciences has exposed severe quality problems in the planning and conduct of research studies in both academia and pharmaceutical industry. Academic crisis response elements include public transparency measures such as preregistration, open-access publication and open data (Kupferschmidt 2018; Levin et al. 2016).
As a result of the replication crisis, which hinges on poor quality of experimental design and resulting data, quality management now has a historic chance to be introduced in the academic biomedical world. Such a system incorporates openness and transparency as key elements for quality assurance (Dirnagl et al. 2018).
2 Critical Points to Consider Before Implementing a Quality Standard in Research
2.1 GxP or Non-GxP Standard Implementation in Research?
Many activities performed in discovery phase and early development are not conducted under GxP standard but need to comply with a number of regulations. Thus, the implementation of an early phase quality standard could help to mitigate the gap and reduce risk exposure. A simple solution could be to apply good laboratory practice (GLP) standards to all research activities in order to mitigate the gap of quality standard.
The classical GxP standards were often born reactively, out of disaster and severe malpractices, which compromised human health. The GLP, for example, originate from the early 1970s, when the Food and Drug Administration (FDA) highlighted several compliance findings in preclinical studies in the USA, such as mis-identification of control and treated animals, suppressed scientific findings, data inventions, dead animal replacements and mis-dosing of test animals. These cases emphasized the need for better control of safety data to minimize risk, in study planning and conduct, in order to both improve the data reliability and protect study participant life. As a result, the FDA created the GLP regulations, which became effective on June 20, 1979. The FDA also launched their Bioresearch Monitoring Program (BIMO), which aimed to conduct routine inspection and data reviews of nonclinical laboratories, in order to evaluate their compliance with the FDA GLP regulation requirements (FDA 1979). Thereafter, the Organisation for Economic Co-operation and Development (OECD) launched their GLP regulation in Europe. Each country, which adopted GLP into their law, tended to add some specificities to their application of GLPs.
Regulated research, which delivers data directly supporting patient safety, is one research area, where GLP were mostly implemented successfully to ensure data integrity and reliability for regulatory approval. Accredited regulatory research laboratories employ continuously trained personnel to perform mainly routine analysis, following defined standard operating procedures (SOPs). Regulatory activities are systematically reviewed/audited by quality assurance groups and inspected by regulators. Thus, developing and maintaining GLP standards needs resources from both research laboratories and regulatory bodies.
In contrast, early discovery research rarely delivers results, which directly impact human health. Therefore the implementation of GxP standards might not be required by the scope of discovery activities (Hickman et al. 2018). However, discovery science would benefit from the use of best scientific practices and quality standards, in order to enhance research robustness and effectiveness and proactively achieve compliance. Many discovery laboratories, hosted either in academia, small biotechs or industries, use cutting-edge technologies, constantly develop novel methods and need the flexibility that GxP standards do not offer. Furthermore, when resources are limited, as often in academia, the implementation of GxP standards is often unbearable. In addition, governmental oversight would increase the burden on the part of the regulatory agencies to come up with specific regulations, check documentation and perform additional inspections.
Therefore the main argument for not extending GxP regulation to non-GxP research is that it would stifle the creativity of researchers, slow down innovation and seriously limit early discovery research. Pragmatic, risk-based and science-driven research quality standards could fit with the discovery activities’ scope and requirement of this research activity and ensure data integrity while saving resources.
2.1.1 Diverse Quality Mind-Set
The success of the development and implementation of a research quality standard relies first on understanding the mind-set of GxP group associates and non-GxP researchers.
Experienced GxP scientists, working in conventional science performing routine well-developed and validated assays, generally apply standards consistently and straightforwardly. Risks in such GxP areas are pretty well understood and predicate rules apply. GxP researchers are used to audits and regulatory inspections. Quality assurance departments usually have these activities under strict scrutiny and help to ensure that study documentation is ready for inspection.
In early discovery, the oversight of quality professionals might be lighter. The scientists might be less familiar with audit or inspections. Thus, many pharma companies have implemented clear internal research guidelines, and a number of universities have dedicated teams both to ensure data integrity and to conduct scientists training.
Academic researchers operate under laboratory conditions similar to those in industrial non-GxP research and are united in their commitment to produce high-quality data. There are academic institutional and funder requirements to preserve research data for at least 10 years after a research project ended, many of which support scientific publications. However, there are varying levels of requirements for documentation, aside from laboratory notebooks, which are still in paper format at most universities, despite the fact that most data are nowadays created and preserved in digital format. But the documentation practices are slowly adapting in academic research laboratories: electronic laboratory notebooks are gaining popularity (Dirnagl and Przesdzing 2016), and more and more institutions are willing to cover licensing costs for their researchers (Kwok 2018). Another group of academic stakeholders are funders, who have tightened the requirements in the application phase. Grant application should include data management plans describing processes to collect, preserve data and ensure their public access. These promising developments might mark the beginning of documentation quality standards in academic biomedical research.
2.2 Resource Constraints
The development of phase-appropriate standards, which provide enough flexibility for innovation and creativity while using best practices ensuring documentation quality and data integrity, is complex and requires time and resources. Thus, both a consistent senior management support and a strong partnership between quality professionals and research groups are mandatory to succeed in both the implementation and the maintenance of the research quality standard.
Research groups, which have the right quality culture/mind-set, could require less inputs from a quality organization.
While these requirements are relatively easy to implement in a pharmaceutical setting, the current academic research environment presents a number of hindrances: usually, academic institutions transfer the responsibilities for data integrity to the principal investigators. While many universities have quality assurance offices, their scope might be limited to quality of teaching and not academic research. Internal and external funding sources do not always support a maintainable quality assurance structure needed to achieve research quality characteristics including robustness, reproducibility and data integrity (Begley et al. 2015). However, more and more academia are increasing their efforts to address research quality.
3 Non-GxP Research Standard Basics
The foundation of any quality standards in regulated and non-regulated environments are good documentation practices, based on data integrity principles, named ALCOA+. Thus, a non-GxP Research standard should focuses on data integrity and research reproducibility. The rigor and frequency of its application need to be adapted to the research phase to which it is applied: in early discovery, focus is laid on innovation, protection of intellectual property and data integrity. In contrast, many other elements have to be consistently implemented, such as robust method validation, equipment qualification in nonclinical confirmatory activities or clinical samples analysis under exploratory objectives of clinical protocols and early development.
3.1 Data Integrity Principles: ALCOA+
Principle | Meaning |
|---|---|
Attributable | The source of data is identified: who/when created a record and who/when/why changed a record |
Legible | Information is clear and readable. In other words, documentation is comprehensive and understandable without need for specific software or knowledge |
Contemporaneous | Information is recorded at the time of data generation and/or event observation |
Original | Source information is available and preserved in its original form |
Accurate | There are no errors or editing without documented amendments |
Principle | Meaning |
|---|---|
Complete | All data is recorded, including repeat or reanalysis performed |
Available | Data is available and accessible at any time for review or audit and for the lifetime of the record |
Consistent | Harmonized documentation process is constantly applied |
Enduring | Data is preserved and retrievable during its entire lifetime |
In order to ensure data integrity and compliance with ALCOA+ principles, all scientific and business practices should underpin the RQS. This standard needs to contain a set of essential quality system elements that can be applied to all types of research, in a risk-based and flexible manner. At a minimum, the following elements should be contained.
3.2 Research Quality System Core Elements
3.2.1 Management and Governance
Management support is critical to ensure that resources are allocated to implement, maintain and continuously improve processes to ensure sustained compliance with RQS. Roles and responsibilities should be well defined, and scientists should be trained accordingly. Routine quality system assessments, conducted by QA and/or scientists themselves, should be also implemented.
3.2.2 Secure Research Documentation and Data Management
Scientists should document their research activities by following the ALCOA+ principles, in a manner to allow reproducibility and straightforward data reconstruction of all activities. Data management processes should ensure long-term data security and straightforward data retrieval.
3.2.3 Method and Assay Qualification
Methods and key research processes should be consistently documented and available for researchers conducting the activity. Assay acceptance/rejection criteria should be predefined. Studies should be well designed to allow statistical relevance. Routine QC and documented peer reviews of research activities and results should be conducted to ensure good scientific quality and reliability. Any change to the method should be documented.
3.2.4 Material, Reagents and Samples Management
Research materials, reagents and samples should be fit for purpose and documented in a manner to permit reproducibility of the research using equivalent items with identical characteristics. Their integrity should be preserved through their entire life cycle until their disposal, which should be consistent with defined regulation or guidance. Research specimens should be labelled to facilitate traceability and storage conditions.
3.2.5 Facility, Equipment and Computerized System Management
Research facilities should be fit for their research activity purpose and provide safe and secure work environments. Research equipment and computerized system, used in the laboratory, should be suitable for the task at hand and function properly. Ideally, their access should be restricted to trained users only, and an activity log should be maintained to increase data traceability.
3.2.6 Personnel and Training Records Management
Research personnel should be competent, trained to perform their research functions in an effective and safe manner. Ideally, in industry environment, personnel and training records should be maintained and available for review.
3.2.7 Outsourcing/External Collaborations
The RQS should be applied to both internal and external activities (conducted by other internal groups, external research centres, academic laboratories or service providers). Agreement to comply with requirements of RQS should be signed off before starting any research work with research groups outside of the organization. Assessment and qualification of an external partner’s quality system are recommended and should be conducted in a risk-based manner (Volsen et al. 2014).
3.3 Risk- and Principle-Based Quality System Assessment Approach
The risk-based and principle-based approaches are the standard biopharma industry quality practice to balance resources, business needs and process burden in order to maximize the impact of an assessment. The risk-based approach is essentially an informed and intelligent way to prioritize frequency and type of assessment (remote, on-site) across a large group of service providers.
CAR-NK and CAR-T Immuno-oncology therapy is an example where patient is treated with his own or donor’s modified cells. Multiple standards and regulations apply. Researchers perform experiments under a combination of sections of good clinical practice (GCP) and good tissue practice (GTP) in a hospital setting (Tang et al. 2018a, b).
Digital therapeutics are another emerging biopharmaceutical field (Pharmaceuticalcommerce.com 2019). Developers utilize knowledge of wearable medical devices, artificial intelligence and cloud computing to boost the effectiveness of traditional chemical or biological drugs or create standalone therapies. As software becomes a part of treatment, it brings a host of nontraditional quality challenges such as health authority pre-certification, management of software updates and patient privacy when using their own devices.
For the above examples, it is important to adhere to ALCOA+ principles as no single quality standard can cover all the needs.
As quality is by design a support function to serve the needs of researchers, business and traditional quality risk factors need to come together when calculating an overall score.
A simple 3X4 Failure Mode and Effects Analysis (FMEA) – like risk matrix – can be constructed using the following example:
- A pharmaceutical company wants to use an external service provider and works on coded human tissue, which is a regulated activity by law, in several countries, such as Switzerland and the UK:
Quality risk factor 1. Severity is medium.
- This laboratory was already audited by the quality assurance of the pharmaceutical company, and gaps were observed in data security and integrity. Remediation actions were conducted by this laboratory to close these gaps:
Quality risk factor 2. Severity is high.
- The planned activity will be using a well-established method that the pharma company needs to transfer to the Swiss laboratory. Since the method need to be handoff, the risk is medium:
Business risk factor 1. Severity is medium.
- The data generated by the laboratory may be used later in an Investigational New Drug (IND) Application. This is a submission critical, and it will be filed to Health Authorities.
Business risk factor 2. Severity is high.
Risk factor | Severity | ||
|---|---|---|---|
Low | Medium | High | |
Quality 1 | 1 | 3 | 9 |
Quality 2 | 1 | 3 | 9 |
Business 1 | 1 | 3 | 9 |
Business 2 | 1 | 3 | 9 |
The risk matrix is balanced for quality and business components. Final business risk is calculated as a product of two business component severity scores such as medium × high = 3 × 9 = 27. Quality risk is calculated in the same fashion.
4 How Can the Community Move Forward?
The improvement of research reproducibility is not only about process implementation but also about promoting quality culture. The research community needs to join force to build a harmonized and recognized quality culture in research, providing tools, guidelines and policies to ensure data quality and research reproducibility.
4.1 Promoting Quality Culture
A process might be far easier for building systems than building a culture of quality. Very often goals are set around cost, speed and productivity. But what is the cost of working on poor processes and with low quality?
In the Oxford dictionary, culture is defined as “The ideas, customs and social behaviour of a particular people or society” and quality as “The standard of something as measured against other things of a similar kind; the degree of excellence of something” (Oxford-Dictionary 2019). So what are the building blocks, which could allow the research community to build a strong quality culture and which elements could influence scientist’s behaviours to strive for research excellence?
4.1.1 Raising Scientist Awareness, Training and Mentoring
In order to embark on the quality journey, researchers should understand the benefits of embracing robust quality:
First Benefit: Help Ensure Their Sustained Success
Great science can lead to patents, publications, key portfolio management decisions, scientific advances and drug submissions. Robust processes position researcher for sustained success, preserving their scientific credibility and enabling, for example, to defend their patent against litigation, make the right decisions, answer regulator’s questions.
Second Benefit: Serve Patients and Advance Scientific Knowledge
The main researcher focus, which fuels their motivation to innovate and go forward, is to advance scientific knowledge and discover new pathways, new drugs and new treatment. Efficient processes enhance research effectiveness and lead to scientific discoveries. Data integrity supports good science, drug safety, products and treatment development for patients and customers.
Once awareness is raised, researchers need to be trained on basic documentation processes and good scientific practices to ensure data integrity and quality. Targeted training should be added on new guidelines, processes and regulations applied to their specific activities (e.g. human tissue use, natural products, pharmacogenomics activities).
4.1.2 Empowering of Associates
The best way to engage researchers is to empower them to perform some changes in order to improve processes and systems. These changes need to be documented, fit for purpose and organized within the quality framework, managed and governed by the senior management. Managers should lead by example, embrace the change in quality culture and interact more with their staff during study planning or laboratory meetings. They should also encourage people to speak up when they observe inaccuracies in the results or potential fraud.
4.1.3 Incentives for Behaviours Which Support Research Quality
A culture that emphasizes research quality can be fostered by providing appropriate incentives for certain behaviours that are aligned with the quality objectives. Such incentives can come in form of promotions, monetary rewards or public recognition. Awards for best practices to ensure data integrity could be a start. Not all incentives must be endured. Some are only necessary to introduce or change a certain practice. Incentives permit an uptake to be measured and the more visible incentives within an institution improve the reach. There is a great variability in effectiveness of a certain incentive. Questionnaires are a useful instrument to find out which incentives are effective for a certain target research population. Any incentives that do not promote quality need to be critically evaluated by the management (Lesmeister 2018; Finkel 2019).
4.1.4 Promoting a Positive Error Culture
“Error is human” and errors will happen in any research laboratory environment, no matter what precautions are taken. However, errors can be prevented from reoccurring and serve as teaching examples for quality assurance and risk management. For this to happen, a positive error culture needs to be created by leaders that embrace learning and do not punish reported errors. The possibility of anonymous reporting is a crucial element as a seed for community trust, so error reporting is not used for blaming and shaming. Next, a guided discussion of reported errors with the laboratory personnel needs to take place, and potential consequences can be discussed. Such a community effort empowers laboratory workers and makes them part of the solution.
An example of a system to manage errors is the free “Laboratory Critical Incident and Error Reporting System” (LabCIRS) software which permits to record all incidents anonymously and to analyse, discuss and communicate them (Dirnagl and Bernard 2018).
4.2 Creating a Recognized Quality Standard in Research: IMI Initiative – EQIPD
Large pharmaceutical companies, service providers and academia are facing the same challenges. They need to manage budget and portfolio, keep credibility and serve customers and patients. Research reproducibility, accuracy and integrity are a benefit to all. For the first time, an Innovative Medicines Initiative (IMI 2008) project on quality was launched in October 2017, named European Quality In Preclinical Data (EQIPD 2017). EQIPD is a 3-year project co-funded by the EU’s Innovative Medicines Initiative (IMI 2008) and the European Federation of Pharmaceutical Industries and Associations (EFPIA).
Pharmaceutical companies and academia joined forces to foster a quality culture and develop a “unified non-GxP research quality standard”, which is expect to be released in 2020 (Steckler et al. 2018; Macleod and Steckler 2019).
The aim of this project is to establish best practices, primarily in the preclinical neuroscience field but also applicable to the overall non-GxP research, that are harmonized across the pharmaceutical industry to improve data quality and reproducibility in discovery and exploratory research. The EQIPD members are working together to develop simple and sustainable solutions to facilitate implementation of robust research quality systems and expansion of knowledge on principles necessary to address robustness and quality.
4.3 Funders Plan to Enhance Reproducibility and Transparency
The NIH proposed first to implement a mandatory training regarding result reproducibility and transparency and good experimental design. Starting in 2019, the NIH research grant applications now have to include components that address reproducibility, rigor and transparency. Applications must include measures to ensure robust and unbiased experimental design, methodology, analysis, interpretation and reporting of results. More relevant biological models should be considered, and the rigor of prior research that the application is based on should be reviewed. NIH asked publishers to get more involved, promote peer-review and data disclosure. In addition, the whole research community is encouraged to work together in order to improve research reproducibility (National-Institutes-of-Health-NIH 2019).
European funders as well aim to enhance reproducibility, mainly by increased transparency and public data availability of research results. The most prominent EU project with that goal is the European Open Science Cloud (EOSC 2018). A key feature of the EOSC is that the shared data conforms to the FAIR criteria: findable, accessible, interoperable and reusable (Wilkinson et al. 2016, 2019). Also at the national funder level, more calls of applications emerge that specifically address scientific rigor and robustness in non-GLP research (German-Federal-Ministry-of-Education-and-Research 2018).
5 Conclusion
In conclusion, the strategic collaboration between pharmaceutical companies, service providers and academia is critical to help develop both quality culture and standards in research, which could help enhance research reproducibility and data integrity. As resources are often limited, a pragmatic quality system combined with a risk-based approach could mitigate the gaps and proactively address the ever-changing regulatory environment, which continuously expands quality expectations.
References
- Al-Zaidy S, Pickard AS, Kotha K, Alfano LN, Lowes L, Paul G, Church K, Lehman K, Sproule DM, Dabbous O, Maru B, Berry K, Arnold WD, Kissel JT, Mendell JR, Shell R (2019) Health outcomes in spinal muscular atrophy type 1 following AVXS-101 gene replacement therapy. Pediatr Pulmonol 54(2):179–185PubMedGoogle Scholar
- Begley CG, Buchan AM, Dirnagl U (2015) Robust research: institutions must do their part for reproducibility. Nature 525(7567):25–27CrossRefGoogle Scholar
- Caruso C (2018) Arvinas, pfizer team up on PROTACs. Cancer Discov 8(4):377–378Google Scholar
- Dirnagl U, Bernard R (2018) Errors and error management in biomedical research. In: Hagen JU (ed) How could this happen? Managing errors in organizations. Springer, Cham, pp 149–160CrossRefGoogle Scholar
- Dirnagl U, Przesdzing I (2016) A pocket guide to electronic laboratory notebooks in the academic life sciences. F1000Res 5:2CrossRefGoogle Scholar
- Dirnagl U, Kurreck C, Castanos-Velez E, Bernard R (2018) Quality management for academic laboratories: burden or boon? Professional quality management could be very beneficial for academic research but needs to overcome specific caveats. EMBO Rep 19(11):e47143CrossRefGoogle Scholar
- EMA (2015) Eu-medicines-agencies-network-strategy-2020-working-together-improve-health. https://www.ema.europa.eu/documents/other/eu-medicines-agencies-network-strategy-2020-working-together-improve-health_en.pdf. Accessed 24 May 2019
- EMA (2018a) General data protection regulation. Directive 95/46/EC. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679. Accessed 24 May 2019
- EMA (2018b) Guideline on good pharmacogenomic practice (EMA/CHMP/718998/2016). http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2018/03/WC500245944.pdf. Accessed 24 May 2019
- EOSC (2018) European Open Science Cloud (EOSC). https://www.eosc-portal.eu/. Accessed 24 May 2019
- EQIPD (2017) European quality in preclinical data. https://quality-preclinical-data.eu/. Accessed 24 May 2019
- FDA (1979) FDA issues good laboratory practice (GLP) regulations. https://www.fda.gov/inspections-compliance-enforcement-and-criminal-investigations/compliance-program-guidance-manual-cpgm/bioresearch-monitoring-program-bimo-compliance-programs. Accessed 24 May 2019
- Finkel A (2019) To move research from quantity to quality, go beyond good intentions. Nature 566(7744):297CrossRefGoogle Scholar
- Gautam A, Pan X (2016) The changing model of big pharma: impact of key trends. Drug Discov Today 21(3):379–384CrossRefGoogle Scholar
- German-Federal-Ministry-of-Education-and-Research (2018) Richtlinie zur Förderung von konfirmatorischen präklinischen Studien. https://www.gesundheitsforschung-bmbf.de/de/8344.php. Accessed 24 May 2019
- Good Automated Manufacturing Practice Forum I. S. f. P. E.-I (2012) ISPE GAMP® good practice guide: a risk-based approach to GxP compliant laboratory computerized systems, 2nd ed.Google Scholar
- Hassanzadeh P, Atyabi F, Dinarvand R (2019) The significance of artificial intelligence in drug delivery system design. Adv Drug Deliv Rev. https://doi.org/10.1016/j.addr.2019.05.001 CrossRefGoogle Scholar
- Hickman S, Ogilvie B, Patterson T (2018) Good laboratory practice – to GLP or not to GLP? Drug Discov World 19:61–66Google Scholar
- IMI (2008) Innovative medicines initiative. http://www.imi.europa.eu/content/mission. Accessed 24 May 2019
- Kupferschmidt K (2018) More and more scientists are preregistering their studies. Should you? https://www.sciencemag.org/news/2018/09/more-and-more-scientists-are-preregistering-their-studies-should-you. Accessed 24 May 2019
- Kwok R (2018) How to pick an electronic laboratory notebook. Nature 560:269–270CrossRefGoogle Scholar
- Lesmeister F (2018) How to create a culture of quality improvement. https://www.bcg.com/en-ch/publications/2018/how-to-create-culture-quality-improvement.aspx. Accessed 24 May 2019
- Levin N, Leonelli S, Weckowska D, Castle D, Dupre J (2016) How do scientists define openness? Exploring the relationship between open science policies and research practice. Bull Sci Technol Soc 36(2):128–141CrossRefGoogle Scholar
- Liu B, Saber A, Haisma HJ (2019) CRISPR/Cas9: a powerful tool for identification of new targets for cancer treatment. Drug Discov Today 24(4):955–970CrossRefGoogle Scholar
- Macleod M, Steckler T (2019) A European initiative to unclog pipeline for new medicines. Nature 568(7753):458CrossRefGoogle Scholar
- Mendell JR, Al-Zaidy S, Shell R, Arnold WD, Rodino-Klapac LR, Prior TW, Lowes L, Alfano L, Berry K, Church K, Kissel JT, Nagendran S, L’Italien J, Sproule DM, Wells C, Cardenas JA, Heitzer MD, Kaspar A, Corcoran S, Braun L, Likhite S, Miranda C, Meyer K, Foust KD, Burghes AHM, Kaspar BK (2017) Single-dose gene-replacement therapy for spinal muscular atrophy. N Engl J Med 377(18):1713–1722CrossRefGoogle Scholar
- National-Institutes-of-Health-NIH (2019) Guidance: rigor and reproducibility in grant applications. https://grants.nih.gov/policy/reproducibility/guidance.htm. Accessed 24 May 2019
- Oxford-Dictionary (2019) LEXICO dictionary powered by Oxford. https://www.lexico.com/en.
- Palmer M, Chaguturu R (2017) Academia–pharma partnerships for novel drug discovery: essential or nice to have? https://doi.org/10.1080/17460441.2017.1318124 CrossRefGoogle Scholar
- Pangalos M, Rees S et al (2019) From genotype to phenotype: leveraging functional genomics in oncology. Drug Discov World Spring 2019Google Scholar
- Pharmaceuticalcommerce.com (2019) Prescription digital therapeutics are entering the healthcare mainstream. https://pharmaceuticalcommerce.com/brand-marketing-communications/prescription-digital-therapeutics-are-entering-the-healthcare-mainstream/. Accessed 07 June 2019
- RQA-Working-Party-on-Quality-in-Non-Regulated-Research (2014) Guidelines for quality in non-regulated research. https://www.therqa.com/resources/publications/booklets/guidelines-for-quality-in-non-regulated-scientific-research-booklet/. Accessed 24 May 2019
- Steckler T, Macleod M, Kas MJH, Gilis A, Wever KE (2018) European quality in preclinical data (EQIPD): een breed consortium voor het verbeteren van de kwaliteit van proefdieronderzoek. Biotech 57(2):18–23Google Scholar
- Swiss-Federal-Government (Effective 1 January 2014) Human Research Act HRA. https://www.admin.ch/opc/en/classified-compilation/20061313/201401010000/810.30.pdf. Accessed 24 May 2019
- Tang X, Yang L, Li Z, Nalin AP, Dai H, Xu T, Yin J, You F, Zhu M, Shen W, Chen G, Zhu X, Wu D, Yu J (2018a) Erratum: first-in-man clinical trial of CAR NK-92 cells: safety test of CD33-CAR NK-92 cells in patients with relapsed and refractory acute myeloid leukemia. Am J Cancer Res 8(9):1899PubMedPubMedCentralGoogle Scholar
- Tang X, Yang L, Li Z, Nalin AP, Dai H, Xu T, Yin J, You F, Zhu M, Shen W, Chen G, Zhu X, Wu D, Yu J (2018b) First-in-man clinical trial of CAR NK-92 cells: safety test of CD33-CAR NK-92 cells in patients with relapsed and refractory acute myeloid leukemia. Am J Cancer Res 8(6):1083–1089PubMedPubMedCentralGoogle Scholar
- Volsen SG, Masson MM, Bongiovanni S (2014) A diligent process to evaluate the quality of outsourced, non GxP. Quasar 128:16–20Google Scholar
- WHO (2005) Quality practices in basic biomedical research (QPBR). https://www.who.int/tdr/publications/training-guideline-publications/handbook-quality-practices-biomedical-research/en/. Accessed 24 May 2019
- Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten JW, da Silva Santos LB, Bourne PE, Bouwman J, Brookes AJ, Clark T, Crosas M, Dillo I, Dumon O, Edmunds S, Evelo CT, Finkers R, Gonzalez-Beltran A, Gray AJ, Groth P, Goble C, Grethe JS, Heringa J, Hoen PA, Hooft R, Kuhn T, Kok R, Kok J, Lusher SJ, Martone ME, Mons A, Packer AL, Persson B, Rocca-Serra P, Roos M, van Schaik R, Sansone SA, Schultes E, Sengstag T, Slater T, Strawn G, Swertz MA, Thompson M, van der Lei J, van Mulligen E, Velterop J, Waagmeester A, Wittenburg P, Wolstencroft K, Zhao J, Mons B (2016) The FAIR guiding principles for scientific data management and stewardship. Sci Data 3:160018CrossRefGoogle Scholar
- Wilkinson MD, Dumontier M, Jan Aalbersberg I, Appleton G, Axton M, Baak A, Blomberg N, Boiten JW, da Silva Santos LB, Bourne PE, Bouwman J, Brookes AJ, Clark T, Crosas M, Dillo I, Dumon O, Edmunds S, Evelo CT, Finkers R, Gonzalez-Beltran A, Gray AJG, Groth P, Goble C, Grethe JS, Heringa J, Hoen PAC, Hooft R, Kuhn T, Kok R, Kok J, Lusher SJ, Martone ME, Mons A, Packer AL, Persson B, Rocca-Serra P, Roos M, van Schaik R, Sansone SA, Schultes E, Sengstag T, Slater T, Strawn G, Swertz MA, Thompson M, van der Lei J, van Mulligen E, Jan V, Waagmeester A, Wittenburg P, Wolstencroft K, Zhao J, Mons B (2019) Addendum: the FAIR Guiding Principles for scientific data management and stewardship. Sci Data 6(1):6CrossRefGoogle Scholar
- Woollen SW (2010) Data quality and the origin of ALCOA. http://www.southernsqa.org/newsletters/Summer10.DataQuality.pdf. Accessed 24 May 2019
Copyright information
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.