Note from the Editor-in-Chief:

At Clinical Orthopaedics and Related Research®, we are proud of our team of statisticians and methodologists. This group’s expertise helps us to get the most out of the increasing number of papers we receive that involve complex study designs and advanced analytic techniques. As most of our statistics editors are neither clinicians nor musculoskeletal laboratory scientists, they look at research from a different angle than do most of our readers and writers. We believe it is important for readers to get a view from this perspective from time to time. The author of this editorial, Mitchell Maltenfort PhD, is our Deputy Editor for methodology and statistics.

—Seth S. Leopold MD

Readers expect that high-quality academic journals will apply rigorous standards in the course of their editorial and peer-review processes. Although no author relishes revising either manuscripts or analyses, such revisions help ensure that the results can be relied on by those readers who might use the research in practice. But excessive criticism, as every author knows, can tend to dampen one’s enthusiasm and make perseverance on behalf of the cause a real challenge. Even in technical fields like surgery, storytelling with data—finding possible explanations for the data and presenting these to a general audience—calls for passion, creativity, and curiosity. How does a scientist (or clinician-scientist) balance these forces in order to remain excited about research while keeping the quality high?

A variety of motivations propel research scientists. Established clinical researchers want to help their patients while continuing to advance professionally, while newer clinical researchers additionally need to establish a reputation. Funded investigators are further obliged to convert dollars into discoveries, in the hope of garnering future support. So how much research is driven by an investigator who simply says, “I want to know …”?

John Tukey, a 20th-century mathematician and statistician once said, “The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data” [4]. In the prior paragraph, I listed some sources of that aching desire. So what happens if a study’s results are inconclusive? Evidence suggests that researchers may not pursue publication [3], and that when they do, journals may make it harder for them to achieve it [1]. But as the editor of this journal has recently reminded CORR ® readers [2], a no-difference study can offer as much information as one with a “positive” result: Perhaps the original study hypothesis was flawed, or perhaps the study design was not appropriate to answer the research question. And so “inconclusive” findings move knowledge ahead, at least incrementally. Unfortunately, such findings are often seen as failures—even when no one has actually failed at anything.

In other fields, taking small steps, going through multiple drafts, and rerunning variations on an early experiment all are perceived as necessary for success. Edison’s famous quote about searching for the light bulb was “I have not failed. I’ve just found 10,000 ways that won’t work.” In modern times, “Fail early, fail often” has become a mantra for business.

But clinicians responsible for a patient’s life or health are trained to believe the only acceptable outcome is success. In research, that sometimes is mistranslated as a “publishable” result, even if that specific result was not what the study originally set out to find. Further, when time is tight—a situation familiar to residents, busy surgeons, or anyone who has faced grant deadlines—there may not be an opportunity to try again, at least not with a new study. This is why authors mine existing data for associations and perform seemingly endless subgroup analyses in order to find something “statistically significant”—which is perceived to equate to something publishable. Unfortunately, this introduces another problem: Statistical tests based on such analyses are likelier to reflect random chance (technically called “inflated Type 1 error”) than they are to be clinically relevant. The problem of avoiding misleading findings is why studies face multiple levels of criticism in the first place. Because peer review cannot, and arguably should not, detect or direct the prepublication thoughts of the authors, preemptive approaches may be necessary to address these pressures and remove their adverse effects.

I would argue that removing some of the drivers of Tukey’s “aching desire” would do as much as peer review does to maintain research quality. I have seen what happens when a research fellow discovers that the question or the tools being used to answer it are interesting for their own sake: Research stops being a jungle the young scientist is lost in, and starts being a beautiful place full of trees to climb up and look out from. The young researcher’s work improves and he or she enjoys it more.

The consistent element among these converts is that they became more involved in and attentive to the planning of each research project. Both the overall question and the details of implementation become exciting. What information is available? What analyses will be most clear and credible? How might the results affect decision-making for clinicians or hospitals? This participation in and control of the research design fosters a sense of ownership, which leads to increased enthusiasm. Actively thinking about data acquisition and analysis can also avoid delays, detours, and frustrations that would arise without adequate planning.

Writing this, it belatedly occurred to me to look up the roots of the word “curious.” In Latin, its root means “careful.” I started this essay considering curiosity and criticism as opposing forces. Perhaps instead curiosity and criticism are similar attention to detail, just from different perspectives: Curiosity works from the ground up and fits pieces together, while criticism reflects on the finished work to see to what degree the pieces indeed did fit. Opposed? No more than two hands in a handshake.