Managing Life Science Innovations in Public Research Through Holistic Performance Measures
Addressing the innovation gap is today considered the third task for public research organizations (PROs) in addition to their traditional tasks of research and teaching. Thus, PROs need to adapt their strategies and research management organization so that more innovative ideas from research will enter the market for the benefit of society. Innovation can thus be defined as value-creating novelties. The commercialization of research results is usually managed through technology transfer offices (TTOs), serving as an interface to industry. How PROs create value is increasingly subject to performance measurement and performance-based budgeting. Applying holistic measures will help adjust the overall strategy of the PRO in the direction of innovation and balance multiple interests and goals. Holistic performance measurement is based on the four dimensions of the decision-oriented model of research production (input, processes, output, and outcome) corresponding to the pillars of innovation. In this model, patenting is a key innovation process in academic life sciences that arises from the co-production between researchers and TTOs.
KeywordsTechnology Transfer Technology Transfer Office Innovation Chain Public Research Organization Technology Transfer Activity
Creative resources in the life sciences in academic research and in industry are not being used efficiently. This phenomenon, often described as the innovation gap or science-to-market gap , is very much to the detriment of the economy and society. The core strategy of the European Union’s “Europe 2020” features the “Innovation Union” as one of seven flagship initiatives “to ensure that innovative ideas can be turned into products and services that create growth and jobs .” To meet the challenges of the global knowledge society, European governments and other public funding bodies increasingly demand that public research organizations, including universities (together called PROs), exploit their results from invention and discovery through technology transfer (TT). This so-called “third task”—in addition to the tasks of research and teaching—enable PROs to capitalize more fully on public funding for the benefit of society. Innovations stemming from public research will be defined in this chapter as value-creating novelties .
The commercialization activities arising out of public research are typically performed by a technology transfer office (TTO). In essence, the task of a TTO is to find a commercial partner willing and able to develop and further test novel ideas into product solutions that people are willing to pay for, thereby creating value. TT is defined by the Organization for Economic Co-operation and Development (OECD) as intellectual property (IP) management, whose purpose is “to identify, protect, exploit and defend intellectual property .”
Focus on relevant stakeholders
Balance between the resulting demands (as far as possible)
Integration in a systems approach, and
Sustainable and long term in orientation.
2 Research Production and Innovation
Holistic performance measurement of TT must take into account all dimensions at the level of the PRO. Detailed information is needed about the causal linkages of the setting in which innovation occurs, the processes of innovation, and the outcome of innovation.
PROs are an essential component in the innovation chain. Faculty and researchers produce basic novel knowledge, some of which is inventive and patentable. The bulk of these inventions occur early in the discovery phase of the innovation life cycle. Far from being a true product, they require additional research and experiments to further refine and advance these early inventions and to strengthen the patent application prior to commercialization. Today, many TTOs have created and manage a gap fund to finance such proof-of-concept studies to reduce risk and further advance their inventions in a bid to make them more attractive to industry.
One causal linkage deserves special emphasis. Patents as standard innovation indicators play a central role in the commercialization of life science inventions because only patents provide the temporary exclusivity and protection required by the pharmaceutical industry to recoup its huge investments in drug development. Thus patenting in academia deserves close scrutiny. This is because inventing and patenting are in fact two separate phenomena  that require different strategies and processes. Responsibilities must be clearly allocated to ascertain who is in control of which part of the innovation process.
Usually it is the researcher’s decision to disclose their invention that triggers the commercialization process at the TTO. If the researcher decides only to publish and not to reveal the invention to the TTO, their invention will be lost for commercialization since it will become public knowledge and no longer be proprietary. Thus, failing to disclose inventions represents the first gap in the innovation chain. It is a myth to believe that a TT manager can just “walk the halls” and pick inventions like cherries that after commercial evaluation will be patented and successfully marketed. Researchers themselves are responsible for disclosing their invention to the TTO. Therefore, invention disclosures constitute the only direct measure of researchers’ innovation activities. By contrast, the number of patents (patent applications, patents granted) reflects the output of research as a whole, including the TTO’s activities. This accords with the assistant role of the TTO as described in the “assisted innovation model” . The TTO forms both an integral part of the research production process and the linear innovation chain, facilitating moving inventions further along the value chain, thus closing the gap .
As patents are created as a result of the co-production between the researchers and their TTO, a PRO must take a twofold approach to capture the value of its investment in research and discovery: (1) ensure that all possible inventions are disclosed and (2) maintain a TTO ready and able to mine these inventions successfully. However, bearing in mind that the chances of meaningful commercial success are very low (estimated at below 1% of all invention disclosures), success can be defined as collaborations with industry, licensing to an established company, or founding a spin-off. Different inputs will yield different outputs and outcomes, just as the efficiency and effectiveness of processes will differ with expertise and experience. Although several indicators and benchmarks already exist for TT, they must be evaluated (holistically) at the level of the PRO.
Structure and resources (funding, personnel) available for research and TT, including the support of the commercialization process.
Research production of new knowledge includes all research activities, specifically inventions, plus all services offered by the TTO. If inventions are not disclosed to the TTO, internal innovation gaps may arise, leading to an early break in the innovation chain.
Productivity and efficiency of the PRO with respect to publications, patents, and commercialization results such as licensing contracts.
The overall impact on society can be divided into institutional impact and market impact. Only market impact can be associated with innovation (any value generated such as licensing income can be reinvested in research input).
3 Conceptual Design of Holistic Performance Measurement in Technology Transfer
Analysis of the interests of the relevant stakeholders: government, management, and the researchers themselves
Alignment of possible goals to respective indicators
Examples of evaluation methods applicable to TT
3.1 Stakeholders in Technology Transfer
The most relevant stakeholders in the innovation process in the academic context are the government or other funding agencies, the PRO management, and the researchers themselves. Each of them will have a different interest in innovation, thus producing multiple, potentially conflicting goals.
The Government: The government (or other public funding bodies) funds the production of new knowledge, stipulating that funding be used sparingly and economically by maximizing desirable side effects (such as TT and international collaborations, equal opportunities, and promoting young researchers) and at the same time minimizing risks. The government might also fund specific programs to foster innovation (such as for establishing spin-off companies).
Management: The PRO’s management is responsible for the overall strategy and budget from public money.1 The management sets priorities and balances the multiple interests of the funding bodies and other relevant interest groups. The strategy is often devised together with the so-called visible scientists whose expert knowledge constitutes the most important capital of a PRO. Managing such experts, who thrive on the privilege of academic freedom, is admittedly challenging. Since the research budget is generally decreasing rather than increasing, any change in strategy will result in reallocating funds, such as for those for innovation, at the expense of other activities. Without management’s support it might be difficult to pursue commercialization activities efficiently. Management might also prefer short-term visible achievements over long-term investments in innovation. As such, management might choose to establish collaborations with industry creating immediate research money, rather than investing long term in the infrastructure for TT. This is regardless of how small the investment is compared with research expenses. Ultimately, some PROs may wisely do both.
Faculty: Researchers and inventors certainly constitute the most important stakeholders in innovation. It is well known that their motivation influences the outcome of TT [9, 10]. Researchers have three ways to convey information about their work results—publish, patent or do both. However, not all researchers are equally motivated to patent. A researcher is primarily motivated by “solving the puzzle” , that is, answering scientific questions not necessarily connected to patenting or commercialization. Second, researchers are driven to receive recognition from their peers and to establish their reputation within their scientific community. Third, money does not seem to be the primary motivation for most researchers because most research fails to generate patentable and commercially viable inventions. Therefore, although inventors usually receive a share of about 30% of the revenues received by the PRO, this motivation is the least powerful. This share can be substantial in a very few cases, but might take a long time to materialize due to long development times.
The propensity to disclose inventions differs among researchers. Those performing basic research are less motivated to patent compared with those who are willing to invest additional time to validate their invention. This also holds true for emerging young talents who first have to build their scientific reputation. In essence, researchers will be motivated to engage in commercial activities the more they feel their resulting share will be adequate in relation to the efforts required . Role models (and potentially feelings of envy) and the fact that some researchers like to “gamble” are further factors affecting researchers’ propensity to patent.
3.2 Indicators and Goals in Technology Transfer
Bearing the stakeholders in mind, the most visible indicator of added value from public research is reflected in the millions of dollars in licensing revenues that some of the large American universities receive. However, this does not automatically mean that these universities and their TTOs are best in class or best in process. This is because very few extremely successful (blockbuster) inventions can generate huge licensing revenues, whereas the bulk of inventions produce only a lower income. As a result, only 16% of TTOs in the United States operate profitably, while more than 50% accrue losses . Moreover, licensing revenues produced by public research constitute only a fraction of the impact and benefit to the economy and society. They reflect market success, which can be neither planned nor influenced by the TTO or PRO. Using licensing revenues as the main indicator of the quality of the TTO or outcome of research is thus not recommended .
Three indicators highlight the commercial potential of public research and constitute the output of research and the prerequisites for commercialization: (1) number of invention disclosures, (2) number of patent applications, and (3) number of patents granted.
Three indicators reflect the commercialization of public research by companies: (4) number of new licensing agreements, (5) number of spin-offs and (6) gross licensing income (in Euros or dollars) and thus measure the outcome of TT.
In addition, one indicator was introduced that serves as a process indicator: (7) the number of new collaboration agreements with industry (excluding consortia being funded by the EC or other public money). This makes it possible to determine the focus of the transfer activities (i.e., collaboration with industry or licensing).
Care must be taken to measure performance with indicators relevant to the goals and strategy of a PRO. Measuring the wrong things may subsequently lead innovation efforts down the wrong path. Four goals in TT are usually identified: service, transfer, profit, and regional development. Each goal can be further broken down into strategic and operational goals and connected with one key indicator as follows .
Service: Service for the researchers as customers means focusing on the internal relationships. This is done, for example, by delivering prompt service, providing expertise in business development in specific research areas, and minimizing transaction times. The key indicator for service is the satisfaction survey conducted for a customer. Note: Because the researcher may demand services that far exceed the budget, satisfaction surveys could be conducted cost-effectively via a focused interest group.
Transfer: Maximizing transfer means maximizing marketing to industry (i.e., external relationships). For each invention the TTO assesses the most potent way of commercialization, giving preference to exclusive licenses and web-based click-licensing. Researchers should be highly motivated and hand in as many invention disclosures as possible. The number of transferred technologies represents the key indicator for transfer. Note: This is a composite indicator and not so easy to assess. Alternatively, the number of license deals can be used as a more direct indicator.
Profit: Maximizing profit involves focusing on large markets with a high potential income. It is achieved, for example, by aggressively collecting royalties and pursuing IP infringers. The key indicator for profit is gross revenue minus direct costs. Note: Although this goal is pursued primarily by only 10% of the TTOs , revenues will always be an important measure of innovation.
Regional Development: Creating jobs in the region in particular is done by providing extensive coaching and teaching on how to set up a company and establishing links to science parks and investors. The key indicator here is the number of spin-offs and the number of jobs created. Note: Universities usually have a closer affinity to their region than non-university research centers.
Rather than just listing the four goals, it is preferable to identify and pursue one primary goal . However, all of the above goals are essential to TT activities, and none of them can be ignored. Therefore, it is important to balance these goals and set priorities that can be changed from time to time if deemed necessary. Goals 1 and 2 are both personnel-intensive, because they require extensive marketing efforts. Although the extent to which they are pursued might be restricted by staff and budgetary limitations, marketing (internal and external) needs to be continuously improved.
3.3 Evaluation Methods and Best Practices
The above analysis on stakeholders’ interests, goals, and indicators provides a systematic framework for holistic performance measurement. Making use of the concept outlined above, a PRO will first need to design a strategy geared to innovation and then start measuring performance operationally. Thus, prior to evaluation, the expectations, purpose, and goals will be laid out and provide the necessary guidance.
Strategically, after reaching agreement on what is relevant in innovation or TT, a PRO will formulate its mission statement and goals or even stipulate them in its statues or policies. The process may well create tensions and trigger discussions on the importance of traditional academic values versus engaging in entrepreneurial activities and creating added value. However, most, if not all, PROs are subject to performance-based budgeting which increasingly includes patent indicators and other innovation measures.
Operationally, a PRO will need to initiate an evaluation cycle or integrate the performance measurement of the TT activities into its own evaluation culture and make use of established evaluation methods. At the Deutsches Krebsforschungszentrum the standard process every 5 years includes self-evaluation, benchmarking with others, and external review. This process is described in more detail below.
Self-evaluation: Self-evaluation involves an analysis of the current situation, for example by summarizing TT results in a business report or plan, including an estimate of the value of the patent portfolio and a forecast of potential licensing income. A SWOT analysis is a helpful tool to visualize strengths, weaknesses, opportunities and threats and to devise action items.
Benchmarking: Benchmarking compares one’s own performance with others, searching for the best processes, activities, and results. The aim is to learn from them to improve one’s own processes, activities, and results. Quantitative data are available from the AUTM survey. Other data may be obtained from the literature [3, 9, 15, 16]. For a true comparison, the data must be standardized either with respect to the number of researchers potentially producing patentable results or the amount of research dollars received in relevant fields (engineering, medicine, life sciences, natural sciences, computing, and so on).
External Evaluation: External evaluation might be conducted as a peer review on site. Peer review is admittedly time-consuming and has often been criticized but it remains the oldest and the most widely accepted qualitative tool for monitoring research. If done regularly in a standardized manner, such evaluation will provide a feedback mechanism for continuous improvement and learning. For the evaluation of TT activities a mixture of the following types of experts with different professional specialties should avoid creating any bias: administrative or scientific directors of a PRO, directors of TT offices, representatives from industry and venture capital and, ideally, reputable researchers with commercial experience.
Such external evaluation may even be performed to support management in devising a strategy for TT and mapping the ensuing operational steps. Management will traditionally include visible scientists and relevant committees. Such on-site peer review will result in a written expert’s opinion on topics such as (1) culture and ethos, including mission, self-image, internal and external outreach, (2) governance of the TTO, (3) business activities (IP management, collaborations with industry, and startup formation), (4) business development activities including gap or validation funding, (5) strategic orientation for the future, and (6) staffing.
Key questions in four dimensions to holistically measure performance and innovative activity of a PRO (modified from )
Resources sufficient to create innovation?
Gap funding available?
Professional TTO available?
Research budget and personnel
TTO budget and personnel
TTO experience sufficient to support researchers in innovation?
Licensing manager sufficiently supported by specialists and partners?
Which services are offered by the TTO to the PRO and the researcher?
Customer satisfaction survey (researchers)
Transparency of processes, successes, and activities of the TTO
Internal/external connectivity of the TTO?
Customer satisfaction survey (including industry)
How efficiently does the TTO transfer input to output?
Ratio of patents and licenses to invention disclosures or per licensing manager
How efficiently does the TTO produce results?
Ratio of licensing contracts per licensing manager
Time to deal
Licensing income minus costs
Which services are offered by the TTO to the PRO and the researcher?
Dependency of income on largest product
Number of contacts of the TTO with researchers
Transparency of processes, successes, and TTO activities
Products in the market
Number of spin-offs
4 Conclusions and Recommendations
In summary, this chapter has detailed the elements of holistic performance measurement for PROs to fulfill their roles of fostering, supporting, and enabling innovation and managing their “third task” activities. Although much has already been achieved there is still untapped potential for innovation from academia. Unraveling this potential is in fact highly complex. The decisive measurable first step for researchers is disclosing their inventions before publishing. The number of unreported cases is not known, but they constitute the innovation potential to be addressed. Patenting and publishing can be performed simultaneously. However, because researchers are under great pressure to publish their findings, there might not be enough time to exemplify promising inventions prior to patenting, thus making it difficult to produce strong patents for commercialization. TTOs are more directly involved in the innovation process and are thus a genuine organizational mechanism that closes the gap between invention and innovation. More still needs to be learned about the consequences of introducing measures, such as performance-based budgeting. It is hoped—and there is reason to hope—that fostering innovation will not lead to a decrease in the quality of basic research.
Integrating innovation as a third task into any research organization requires a holistic approach, because innovation may be perceived as counter to the tradition and culture of academic institutions. However, changes are necessary to increase innovation activities within a PRO for the benefit of society. Changes are already happening with novel therapeutic concepts, such as personalized or translational medicine. PROs have assumed responsibilities in early drug discovery formerly only vested with the pharmaceutical industry, such as establishing high-throughput screening facilities, medicinal chemistry, and early clinical trials . The key aspect of this public commitment is the alarmingly decreasing number of innovative new medicines over a period of many years. This downward trend is detrimental to the healthcare system and to patients . However, companies such as Genentech are still very successfully developing new and innovative drugs. There is reason to believe that their special skills in turning academia’s novel ideas into novel drugs constitute the key factor for success.
Inspired by this example, many pharmaceutical companies are now creating novel collaboration models with academic institutions in early drug discovery. Both pharmaceutical companies and academia are striving to overcome cultural and organizational barriers and boundaries between them and thus are becoming more flexible in interacting with one another. Learning to integrate the best of both worlds seems like a promising way to create added value. For example, the collaborative innovation alliance between the Deutsches Krebsforschungszentrum and Bayer Healthcare can be highlighted where joint projects are performed together along defined milestones, directly feeding successful projects into the internal Bayer pipeline. This risk-and-reward sharing partnership model is based on interactions at all organizational levels in an open and mutually beneficial atmosphere of an exchange of ideas . In a few years’ time the ultimate measure of success of such academic institutions and industry alliances will be the introduction of innovative treatments into patient care.
In the United States and Europe researchers have significant discretion in raising their own grant funding independently from the PRO management.
- 2.Europe 2020 Flagship Initiative Innovation Union SEC (2010) 116, COM (2010) 546 final, p 6. http://ec.europa.eu/research/innovation-union/pdf/innovation-union-communication_en.pdf. Accessed 23 Oct 2012
- 3.Levy D, Wasden C, Reich A (2011) If innovation isn’t measured can it be managed? How universities manage innovation through disciplined and novel measures. Price Waterhouse Coopers ReportGoogle Scholar
- 4.OECD (2003) Turning science into business: patenting and licensing at public research organizations. OECD, Paris. doi:10.1787/9789264100244-enGoogle Scholar
- 6.Goktepe D, Mahangaonkar P (2008) What do scientists want: money or fame? Jena Economic Research Papers, No. 2008–032Google Scholar
- 7.Etzkovitz H, Goktepe D (2005) The co-evolution of the university technology transfer office and the linear model of innovation. Paper to be presented at the DRUID tenth anniversary summer conference, Copenhagen, Denmark, June 27–29, pp 1–17Google Scholar
- 9.Abrams I, Leung G, Stevens AJ (2009) How are U.S. technology transfer offices tasked and motivated—is it all about the money? Res Manag Rev 17(1):18–50Google Scholar
- 10.Stevens A (2003) 20 years of academic licensing—royalty income and economic impact. J Licensing Exec Soc Int (les Nouvelles) 38:133–140Google Scholar
- 11.Merrill SA, Mazza A-M (eds) (2010) Managing university intellectual property in the public interest. Committee on Management of University Intellectual Property: lessons from a generation of experience, research and dialogue. The National Academies Press (NAP), WashingtonGoogle Scholar
- 12.Kordal R, Sanga A, Smith R (eds) AUTM U.S. licensing activity survey: FY2009. Survey summary. Association for University Technology Managers, Deerfield, IL. www.autm.net. Accessed 23 Oct 2012
- 13.European Commission: Metrics for knowledge transfer from public research organizations in Europe. Expert group report, EUR 23894, 2009, EUROPEAN COMMISSION Directorate-General for Research, http://ec.europa.eu/invest-in-research/pdf/download_en/knowledge_transfer_web.pdf. Accessed 23 Oct 2012
- 14.Sharer M, Faley TL (2008) The strategic management of the technology transfer function—aligning goals with strategies, objectives and tactics. J Licensing Exec Soc Int (les Nouvelles) 43:170–179Google Scholar
- 15.Batrla R, Licht G (2004) Technologietransfer im Vergleich. Eine Fallstudie. Wissenschaftsmanagement 6:12–17Google Scholar
- 16.Heher AD (2007) Benchmarking of technology transfer offices and what it means for developing countries. In: Krattiger A (ed) Intellectual property management in health and agricultural innovation: a handbook of best practices, vol 1. MIHR-USA, Davis. Chapter no. 3.5, pp 207–228Google Scholar
- 18.Paul SM, Mytelka DS, Dunwiddie CT, Persinger CC, Munos BH, Lindborg SR, Schacht A (2010) How to improve R&D productivity: the pharmaceutical industry’s grand challenge. Nat Rev Drug Discov 9:203–214Google Scholar
- 19.Wellenreuther R, Keppler D, Mumberg D, Ziegelbauer K, Lessl M (2012) Promoting drug discovery by collaborative innovation: a novel risk- and reward-sharing partnership between the German Cancer Research Center and Bayer Healthcare. Drug Discov Today 17(21–22):1242–1248. doi:10.1016/j.drudis.2012.04.004CrossRefGoogle Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.