1 Why Do We Need a Quality Standard in Research?

Over the past decades, numerous novel technologies and scientific innovation initiated a shift in drug discovery and development models. Progress in genomics and genetics technologies opened the door for personalized medicine. Gene and targeted therapies could give the chance of a normal life for genetically diseased patients. For example, adeno-associated viruses, such as AAV9, are currently used to create new treatments for newborns diagnosed with spinal muscular atrophy (SMA) (Mendell et al. 2017; Al-Zaidy et al. 2019). Similarly, the use of clustered regularly interspaced short palindromic repeats (CRISPR) (Liu et al. 2019) or proteolysis targeting chimeras (PROTACs) (Caruso 2018) is leading to novel cancer therapy developments. The broader use of digitalization, machine learning and artificial intelligence (AI) (Hassanzadeh et al. 2019) in combination with these technologies will revolutionize the drug discovery and clinical study design and accelerate drug development (Pangalos et al. 2019).

Regulators all over the world are closely monitoring these breakthrough scientific advances and drug development revolution. While they evaluate the great promise of innovative medicines, they also raise questions about potential safety risks, ethics and environment. Consequently, new ethical laws and regulations are emerging to mitigate the risks without slowing down innovation. For example, the UK Human Tissue Act became effective in 2006, followed by the Swiss Human Research Act in January 2014 (Swiss-Federal-Government, Effective 1 January 2014); the EU General Data Protection Regulation (No.679/2016, the GDPR) came into effect on May 25, 2018 (EMA 2018a); and the guideline on good pharmacogenomics practice has been in effect since September 2018 (EMA 2018b).

This is exemplified by the EMA Network Strategy to 2020 (EMA 2015), which aims both to promote innovation and to better understand associated risks, in order to provide patients with safe and novel drugs or treatments on the market more rapidly.

This evolving research and regulatory environment, along with many other new challenges, such as aggressive patent litigation cases, increasing burden for approval and reimbursement of new molecular entities (NMEs), challenging market dynamics and high societal pressure enforce radical changes in the research and drug development models of the pharmaceutical industry (Gautam and Pan 2016). In response, most of the pharmaceutical companies have refocused on portfolio management, acquired promising biotechnology companies and developed research collaborations with academia (Palmer and Chaguturu 2017). The goal is to speed up drug development in order to deliver new drugs and new treatments to their patients and customers. Thus, transition from research to drug development should be more efficient. To do so, robust data quality, integrity and reproducibility became essential, and the development of a quality culture across the entire value chain emerged to be critical. Indeed, while many drug development areas already applied the various good practice (GxP) standards and guidances, no recognized quality standard governed discovery and early development. Conversely, discovery activities had to comply with many regulations, such as biosafety, controlled substances and data privacy; thus, there was a real risk of exposure in non-GXP research.

In order to mitigate these newly emerging risks and speed up drug development, some pharmaceutical companies decided to develop their own internal research quality standard (RQS), based on good scientific practices and data integrity, to promote robust science and data quality. The foundations of RQS were the WHO: “Quality Practices in Basic Biomedical Research” (WHO 2005), first published in 2001, and the “Quality in Research Guideline for working in non-regulated research”, published by the British Research Quality Association RQA, in 2006 and revised in 2008 and 2014 (RQA-Working-Party-on-Quality-in-Non-Regulated-Research 2014).

Academic research institutions and laboratories are as committed as their pharmaceutical counterparts to good scientific practices but are largely operating without defined standards. Many universities hold their scientists accountable for good scientific practices, which are mainly focused on preventing misconduct and promoting a collaborative environment. Academic output is measured by the amount of publications, often in prestigious journals. Peer review of manuscripts is seen by academics as the main quality control element. During the last decade, the replication and reproducibility crisis in biomedical sciences has exposed severe quality problems in the planning and conduct of research studies in both academia and pharmaceutical industry. Academic crisis response elements include public transparency measures such as preregistration, open-access publication and open data (Kupferschmidt 2018; Levin et al. 2016).

As a result of the replication crisis, which hinges on poor quality of experimental design and resulting data, quality management now has a historic chance to be introduced in the academic biomedical world. Such a system incorporates openness and transparency as key elements for quality assurance (Dirnagl et al. 2018).

2 Critical Points to Consider Before Implementing a Quality Standard in Research

2.1 GxP or Non-GxP Standard Implementation in Research?

Many activities performed in discovery phase and early development are not conducted under GxP standard but need to comply with a number of regulations. Thus, the implementation of an early phase quality standard could help to mitigate the gap and reduce risk exposure. A simple solution could be to apply good laboratory practice (GLP) standards to all research activities in order to mitigate the gap of quality standard.

The classical GxP standards were often born reactively, out of disaster and severe malpractices, which compromised human health. The GLP, for example, originate from the early 1970s, when the Food and Drug Administration (FDA) highlighted several compliance findings in preclinical studies in the USA, such as mis-identification of control and treated animals, suppressed scientific findings, data inventions, dead animal replacements and mis-dosing of test animals. These cases emphasized the need for better control of safety data to minimize risk, in study planning and conduct, in order to both improve the data reliability and protect study participant life. As a result, the FDA created the GLP regulations, which became effective on June 20, 1979. The FDA also launched their Bioresearch Monitoring Program (BIMO), which aimed to conduct routine inspection and data reviews of nonclinical laboratories, in order to evaluate their compliance with the FDA GLP regulation requirements (FDA 1979). Thereafter, the Organisation for Economic Co-operation and Development (OECD) launched their GLP regulation in Europe. Each country, which adopted GLP into their law, tended to add some specificities to their application of GLPs.

Regulated research, which delivers data directly supporting patient safety, is one research area, where GLP were mostly implemented successfully to ensure data integrity and reliability for regulatory approval. Accredited regulatory research laboratories employ continuously trained personnel to perform mainly routine analysis, following defined standard operating procedures (SOPs). Regulatory activities are systematically reviewed/audited by quality assurance groups and inspected by regulators. Thus, developing and maintaining GLP standards needs resources from both research laboratories and regulatory bodies.

In contrast, early discovery research rarely delivers results, which directly impact human health. Therefore the implementation of GxP standards might not be required by the scope of discovery activities (Hickman et al. 2018). However, discovery science would benefit from the use of best scientific practices and quality standards, in order to enhance research robustness and effectiveness and proactively achieve compliance. Many discovery laboratories, hosted either in academia, small biotechs or industries, use cutting-edge technologies, constantly develop novel methods and need the flexibility that GxP standards do not offer. Furthermore, when resources are limited, as often in academia, the implementation of GxP standards is often unbearable. In addition, governmental oversight would increase the burden on the part of the regulatory agencies to come up with specific regulations, check documentation and perform additional inspections.

Therefore the main argument for not extending GxP regulation to non-GxP research is that it would stifle the creativity of researchers, slow down innovation and seriously limit early discovery research. Pragmatic, risk-based and science-driven research quality standards could fit with the discovery activities’ scope and requirement of this research activity and ensure data integrity while saving resources.

2.1.1 Diverse Quality Mind-Set

The success of the development and implementation of a research quality standard relies first on understanding the mind-set of GxP group associates and non-GxP researchers.

Experienced GxP scientists, working in conventional science performing routine well-developed and validated assays, generally apply standards consistently and straightforwardly. Risks in such GxP areas are pretty well understood and predicate rules apply. GxP researchers are used to audits and regulatory inspections. Quality assurance departments usually have these activities under strict scrutiny and help to ensure that study documentation is ready for inspection.

In early discovery, the oversight of quality professionals might be lighter. The scientists might be less familiar with audit or inspections. Thus, many pharma companies have implemented clear internal research guidelines, and a number of universities have dedicated teams both to ensure data integrity and to conduct scientists training.

Academic researchers operate under laboratory conditions similar to those in industrial non-GxP research and are united in their commitment to produce high-quality data. There are academic institutional and funder requirements to preserve research data for at least 10 years after a research project ended, many of which support scientific publications. However, there are varying levels of requirements for documentation, aside from laboratory notebooks, which are still in paper format at most universities, despite the fact that most data are nowadays created and preserved in digital format. But the documentation practices are slowly adapting in academic research laboratories: electronic laboratory notebooks are gaining popularity (Dirnagl and Przesdzing 2016), and more and more institutions are willing to cover licensing costs for their researchers (Kwok 2018). Another group of academic stakeholders are funders, who have tightened the requirements in the application phase. Grant application should include data management plans describing processes to collect, preserve data and ensure their public access. These promising developments might mark the beginning of documentation quality standards in academic biomedical research.

2.2 Resource Constraints

The development of phase-appropriate standards, which provide enough flexibility for innovation and creativity while using best practices ensuring documentation quality and data integrity, is complex and requires time and resources. Thus, both a consistent senior management support and a strong partnership between quality professionals and research groups are mandatory to succeed in both the implementation and the maintenance of the research quality standard.

Research groups, which have the right quality culture/mind-set, could require less inputs from a quality organization.

While these requirements are relatively easy to implement in a pharmaceutical setting, the current academic research environment presents a number of hindrances: usually, academic institutions transfer the responsibilities for data integrity to the principal investigators. While many universities have quality assurance offices, their scope might be limited to quality of teaching and not academic research. Internal and external funding sources do not always support a maintainable quality assurance structure needed to achieve research quality characteristics including robustness, reproducibility and data integrity (Begley et al. 2015). However, more and more academia are increasing their efforts to address research quality.

3 Non-GxP Research Standard Basics

The foundation of any quality standards in regulated and non-regulated environments are good documentation practices, based on data integrity principles, named ALCOA+. Thus, a non-GxP Research standard should focuses on data integrity and research reproducibility. The rigor and frequency of its application need to be adapted to the research phase to which it is applied: in early discovery, focus is laid on innovation, protection of intellectual property and data integrity. In contrast, many other elements have to be consistently implemented, such as robust method validation, equipment qualification in nonclinical confirmatory activities or clinical samples analysis under exploratory objectives of clinical protocols and early development.

figure a

3.1 Data Integrity Principles: ALCOA+

Essential principles ensuring data integrity throughout the lifecycle are commonly known by the acronym “ALCOA”. Stan Woollen first introduced this acronym in the early 1990s when he worked at the Office of Enforcement, in the USA. He used it to memorize the five key elements of data quality when he presented the GLP and FDA’s overall BIMO program (Woollen 2010). Since then, QA professionals used commonly the acronym ALCOA to discuss data integrity. Later on, four additional elements, extracted from the Good Automated Manufacturing Practice (GAMP) guide “A Risk-Based Approach to GxP Complaint Laboratory Computerized Systems” (Good Automated Manufacturing Practice Forum 2012), completed the set of integrity principles (ALCOA+). The ALCOA+ consists of a set of principles, which underpins any quality standards:

Principle

Meaning

Attributable

The source of data is identified: who/when created a record and who/when/why changed a record

Legible

Information is clear and readable. In other words, documentation is comprehensive and understandable without need for specific software or knowledge

Contemporaneous

Information is recorded at the time of data generation and/or event observation

Original

Source information is available and preserved in its original form

Accurate

There are no errors or editing without documented amendments

Additional elements:

Principle

Meaning

Complete

All data is recorded, including repeat or reanalysis performed

Available

Data is available and accessible at any time for review or audit and for the lifetime of the record

Consistent

Harmonized documentation process is constantly applied

Enduring

Data is preserved and retrievable during its entire lifetime

In order to ensure data integrity and compliance with ALCOA+ principles, all scientific and business practices should underpin the RQS. This standard needs to contain a set of essential quality system elements that can be applied to all types of research, in a risk-based and flexible manner. At a minimum, the following elements should be contained.

3.2 Research Quality System Core Elements

3.2.1 Management and Governance

Management support is critical to ensure that resources are allocated to implement, maintain and continuously improve processes to ensure sustained compliance with RQS. Roles and responsibilities should be well defined, and scientists should be trained accordingly. Routine quality system assessments, conducted by QA and/or scientists themselves, should be also implemented.

3.2.2 Secure Research Documentation and Data Management

Scientists should document their research activities by following the ALCOA+ principles, in a manner to allow reproducibility and straightforward data reconstruction of all activities. Data management processes should ensure long-term data security and straightforward data retrieval.

3.2.3 Method and Assay Qualification

Methods and key research processes should be consistently documented and available for researchers conducting the activity. Assay acceptance/rejection criteria should be predefined. Studies should be well designed to allow statistical relevance. Routine QC and documented peer reviews of research activities and results should be conducted to ensure good scientific quality and reliability. Any change to the method should be documented.

3.2.4 Material, Reagents and Samples Management

Research materials, reagents and samples should be fit for purpose and documented in a manner to permit reproducibility of the research using equivalent items with identical characteristics. Their integrity should be preserved through their entire life cycle until their disposal, which should be consistent with defined regulation or guidance. Research specimens should be labelled to facilitate traceability and storage conditions.

3.2.5 Facility, Equipment and Computerized System Management

Research facilities should be fit for their research activity purpose and provide safe and secure work environments. Research equipment and computerized system, used in the laboratory, should be suitable for the task at hand and function properly. Ideally, their access should be restricted to trained users only, and an activity log should be maintained to increase data traceability.

3.2.6 Personnel and Training Records Management

Research personnel should be competent, trained to perform their research functions in an effective and safe manner. Ideally, in industry environment, personnel and training records should be maintained and available for review.

3.2.7 Outsourcing/External Collaborations

The RQS should be applied to both internal and external activities (conducted by other internal groups, external research centres, academic laboratories or service providers). Agreement to comply with requirements of RQS should be signed off before starting any research work with research groups outside of the organization. Assessment and qualification of an external partner’s quality system are recommended and should be conducted in a risk-based manner (Volsen et al. 2014).

3.3 Risk- and Principle-Based Quality System Assessment Approach

The risk-based and principle-based approaches are the standard biopharma industry quality practice to balance resources, business needs and process burden in order to maximize the impact of an assessment. The risk-based approach is essentially an informed and intelligent way to prioritize frequency and type of assessment (remote, on-site) across a large group of service providers.

The principle-based trend reflects the fact that it may not be possible to anticipate and prescriptively address a myriad of emerging nuances and challenges in a rapidly evolving field. Cell and gene therapy (e.g. CAR-NK and CAR-T), digital medicine, complex drug/device interfaces and new categories of biomarkers are just some of the recent examples demanding a flexible and innovative quality mind-set:

  • CAR-NK and CAR-T Immuno-oncology therapy is an example where patient is treated with his own or donor’s modified cells. Multiple standards and regulations apply. Researchers perform experiments under a combination of sections of good clinical practice (GCP) and good tissue practice (GTP) in a hospital setting (Tang et al. 2018a, b).

  • Digital therapeutics are another emerging biopharmaceutical field (Pharmaceuticalcommerce.com 2019). Developers utilize knowledge of wearable medical devices, artificial intelligence and cloud computing to boost the effectiveness of traditional chemical or biological drugs or create standalone therapies. As software becomes a part of treatment, it brings a host of nontraditional quality challenges such as health authority pre-certification, management of software updates and patient privacy when using their own devices.

For the above examples, it is important to adhere to ALCOA+ principles as no single quality standard can cover all the needs.

As quality is by design a support function to serve the needs of researchers, business and traditional quality risk factors need to come together when calculating an overall score.

A simple 3X4 Failure Mode and Effects Analysis (FMEA) – like risk matrix – can be constructed using the following example:

Suppose that:

  • A pharmaceutical company wants to use an external service provider and works on coded human tissue, which is a regulated activity by law, in several countries, such as Switzerland and the UK:

    • Quality risk factor 1. Severity is medium.

  • This laboratory was already audited by the quality assurance of the pharmaceutical company, and gaps were observed in data security and integrity. Remediation actions were conducted by this laboratory to close these gaps:

    • Quality risk factor 2. Severity is high.

  • The planned activity will be using a well-established method that the pharma company needs to transfer to the Swiss laboratory. Since the method need to be handoff, the risk is medium:

    • Business risk factor 1. Severity is medium.

  • The data generated by the laboratory may be used later in an Investigational New Drug (IND) Application. This is a submission critical, and it will be filed to Health Authorities.

    • Business risk factor 2. Severity is high.

Risk factor

Severity

Low

Medium

High

Quality 1

1

3

9

Quality 2

1

3

9

Business 1

1

3

9

Business 2

1

3

9

The risk matrix is balanced for quality and business components. Final business risk is calculated as a product of two business component severity scores such as medium × high = 3 × 9 = 27. Quality risk is calculated in the same fashion.

figure b

4 How Can the Community Move Forward?

The improvement of research reproducibility is not only about process implementation but also about promoting quality culture. The research community needs to join force to build a harmonized and recognized quality culture in research, providing tools, guidelines and policies to ensure data quality and research reproducibility.

4.1 Promoting Quality Culture

A process might be far easier for building systems than building a culture of quality. Very often goals are set around cost, speed and productivity. But what is the cost of working on poor processes and with low quality?

In the Oxford dictionary, culture is defined as “The ideas, customs and social behaviour of a particular people or society” and quality as “The standard of something as measured against other things of a similar kind; the degree of excellence of something” (Oxford-Dictionary 2019). So what are the building blocks, which could allow the research community to build a strong quality culture and which elements could influence scientist’s behaviours to strive for research excellence?

4.1.1 Raising Scientist Awareness, Training and Mentoring

In order to embark on the quality journey, researchers should understand the benefits of embracing robust quality:

First Benefit: Help Ensure Their Sustained Success

Great science can lead to patents, publications, key portfolio management decisions, scientific advances and drug submissions. Robust processes position researcher for sustained success, preserving their scientific credibility and enabling, for example, to defend their patent against litigation, make the right decisions, answer regulator’s questions.

Second Benefit: Serve Patients and Advance Scientific Knowledge

The main researcher focus, which fuels their motivation to innovate and go forward, is to advance scientific knowledge and discover new pathways, new drugs and new treatment. Efficient processes enhance research effectiveness and lead to scientific discoveries. Data integrity supports good science, drug safety, products and treatment development for patients and customers.

Once awareness is raised, researchers need to be trained on basic documentation processes and good scientific practices to ensure data integrity and quality. Targeted training should be added on new guidelines, processes and regulations applied to their specific activities (e.g. human tissue use, natural products, pharmacogenomics activities).

4.1.2 Empowering of Associates

The best way to engage researchers is to empower them to perform some changes in order to improve processes and systems. These changes need to be documented, fit for purpose and organized within the quality framework, managed and governed by the senior management. Managers should lead by example, embrace the change in quality culture and interact more with their staff during study planning or laboratory meetings. They should also encourage people to speak up when they observe inaccuracies in the results or potential fraud.

4.1.3 Incentives for Behaviours Which Support Research Quality

A culture that emphasizes research quality can be fostered by providing appropriate incentives for certain behaviours that are aligned with the quality objectives. Such incentives can come in form of promotions, monetary rewards or public recognition. Awards for best practices to ensure data integrity could be a start. Not all incentives must be endured. Some are only necessary to introduce or change a certain practice. Incentives permit an uptake to be measured and the more visible incentives within an institution improve the reach. There is a great variability in effectiveness of a certain incentive. Questionnaires are a useful instrument to find out which incentives are effective for a certain target research population. Any incentives that do not promote quality need to be critically evaluated by the management (Lesmeister 2018; Finkel 2019).

4.1.4 Promoting a Positive Error Culture

“Error is human” and errors will happen in any research laboratory environment, no matter what precautions are taken. However, errors can be prevented from reoccurring and serve as teaching examples for quality assurance and risk management. For this to happen, a positive error culture needs to be created by leaders that embrace learning and do not punish reported errors. The possibility of anonymous reporting is a crucial element as a seed for community trust, so error reporting is not used for blaming and shaming. Next, a guided discussion of reported errors with the laboratory personnel needs to take place, and potential consequences can be discussed. Such a community effort empowers laboratory workers and makes them part of the solution.

An example of a system to manage errors is the free “Laboratory Critical Incident and Error Reporting System” (LabCIRS) software which permits to record all incidents anonymously and to analyse, discuss and communicate them (Dirnagl and Bernard 2018).

4.2 Creating a Recognized Quality Standard in Research: IMI Initiative – EQIPD

Large pharmaceutical companies, service providers and academia are facing the same challenges. They need to manage budget and portfolio, keep credibility and serve customers and patients. Research reproducibility, accuracy and integrity are a benefit to all. For the first time, an Innovative Medicines Initiative (IMI 2008) project on quality was launched in October 2017, named European Quality In Preclinical Data (EQIPD 2017). EQIPD is a 3-year project co-funded by the EU’s Innovative Medicines Initiative (IMI 2008) and the European Federation of Pharmaceutical Industries and Associations (EFPIA).

Pharmaceutical companies and academia joined forces to foster a quality culture and develop a “unified non-GxP research quality standard”, which is expect to be released in 2020 (Steckler et al. 2018; Macleod and Steckler 2019).

The aim of this project is to establish best practices, primarily in the preclinical neuroscience field but also applicable to the overall non-GxP research, that are harmonized across the pharmaceutical industry to improve data quality and reproducibility in discovery and exploratory research. The EQIPD members are working together to develop simple and sustainable solutions to facilitate implementation of robust research quality systems and expansion of knowledge on principles necessary to address robustness and quality.

4.3 Funders Plan to Enhance Reproducibility and Transparency

The NIH proposed first to implement a mandatory training regarding result reproducibility and transparency and good experimental design. Starting in 2019, the NIH research grant applications now have to include components that address reproducibility, rigor and transparency. Applications must include measures to ensure robust and unbiased experimental design, methodology, analysis, interpretation and reporting of results. More relevant biological models should be considered, and the rigor of prior research that the application is based on should be reviewed. NIH asked publishers to get more involved, promote peer-review and data disclosure. In addition, the whole research community is encouraged to work together in order to improve research reproducibility (National-Institutes-of-Health-NIH 2019).

European funders as well aim to enhance reproducibility, mainly by increased transparency and public data availability of research results. The most prominent EU project with that goal is the European Open Science Cloud (EOSC 2018). A key feature of the EOSC is that the shared data conforms to the FAIR criteria: findable, accessible, interoperable and reusable (Wilkinson et al. 2016, 2019). Also at the national funder level, more calls of applications emerge that specifically address scientific rigor and robustness in non-GLP research (German-Federal-Ministry-of-Education-and-Research 2018).

5 Conclusion

In conclusion, the strategic collaboration between pharmaceutical companies, service providers and academia is critical to help develop both quality culture and standards in research, which could help enhance research reproducibility and data integrity. As resources are often limited, a pragmatic quality system combined with a risk-based approach could mitigate the gaps and proactively address the ever-changing regulatory environment, which continuously expands quality expectations.