More insight into the fate of biomedical meeting abstracts: a systematic review
- 1.5k Downloads
It has been estimated that about 45% of abstracts that are accepted for presentation at biomedical meetings will subsequently be published in full. The acceptance of abstracts at meetings and their fate after initial rejection are less well understood. We set out to estimate the proportion of abstracts submitted to meetings that are eventually published as full reports, and to explore factors that are associated with meeting acceptance and successful publication.
Studies analysing acceptance of abstracts at biomedical meetings or their subsequent full publication were searched in MEDLINE, OLDMEDLINE, EMBASE, Cochrane Library, CINAHL, BIOSIS, Science Citation Index Expanded, and by hand searching of bibliographies and proceedings. We estimated rates of abstract acceptance and of subsequent full publication, and identified abstract and meeting characteristics associated with acceptance and publication, using logistic regression analysis, survival-type analysis, and meta-analysis.
Analysed meetings were held between 1957 and 1999. Of 14945 abstracts that were submitted to 43 meetings, 46% were accepted. The rate of full publication was studied with 19123 abstracts that were presented at 234 meetings. Using survival-type analysis, we estimated that 27% were published after two, 41% after four, and 44% after six years. Of 2412 abstracts that were rejected at 24 meetings, 27% were published despite rejection. Factors associated with both abstract acceptance and subsequent publication were basic science and positive study outcome. Large meetings and those held outside the US were more likely to accept abstracts. Abstracts were more likely to be published subsequently if presented either orally, at small meetings, or at a US meeting. Abstract acceptance itself was strongly associated with full publication.
About one third of abstracts submitted to biomedical meetings were published as full reports. Acceptance at meetings and publication were associated with specific characteristics of abstracts and meetings.
KeywordsAcceptance Rate Publication Rate Subsequent Publication Full Report Science Citation Index Expand
Dissemination of results from research is a key feature of biomedical science. Over 40 years ago, the filtering process from the stage of designing a study until publication of the results was likened to a biological metabolism . Abstracts that are presented at biomedical meetings often provide the first evidence of a research activity. Since abstracts are usually published in proceeding volumes or in journal supplements, they can be followed up until successful publication as full reports. Thus, through follow-up of abstracts from meeting submission to subsequent full publication, some aspects of this "metabolism"  can be studied. Systematic review of reports that focused on the follow-up of meeting abstracts identified some specific characteristics of the abstracts that were associated with successful publication [2, 3]. The average publication rate of abstracts was shown to be about 45% only, suggesting that more than half were not fully published after presentation at the meetings [2, 3]. Despite this important finding, some elements of the selection process of scientific data remained unclear. For instance, it is not well understood what happens to abstracts submitted for presentation at meetings. The rate of meeting acceptance has not been subject to systematic review, and factors that determine acceptance of abstracts at meetings have not been systematically investigated. Finally, the rate of abstracts that are rejected at meetings but still published is unclear. This additional knowledge would improve our understanding on the "metabolism" of scientific data. Our study focused on five objectives. First, to estimate, based on published data, the acceptance rate of abstracts submitted to biomedical meetings. Second, to better describe the time-course of subsequent full publication of abstracts. Third, to identify abstract and meeting characteristics that are associated with abstract acceptance at meetings and their subsequent publication. Fourth, to investigate the fate of rejected abstracts. And fifth, to estimate the rate of abstracts that are submitted to biomedical meetings and, whether accepted at the meetings or not, subsequently published in full.
Dissemination of biomedical information
We searched in MEDLINE, OLDMEDLINE, EMBASE, Cochrane Library, CINAHL, and BIOSIS for studies that provided data on Flows 1 to 3. Searches were conducted until March 31, 2001 without restriction for language or publication format. The search strategy included multiple free text keywords and combinations thereof: abstract, summary report, poster, poster presentation, oral presentation, podium presentation, presented, meeting, congress, conference, convention, submission, submitted, acceptance, accepted, acceptance rate, rejection, rejected, publication, published, publication rate, and publication pattern. Fate was searched as a title word. Citations of previously published similar analysis  were searched using Science Citation Index Expanded . Bibliographies of retrieved articles, and the proceedings of the first three International Congresses on Peer Review in Biomedical Publication were checked. Authors were contacted to clarify ambiguity in the reports and to provide supplementary data if necessary. Internet searches were conducted for additional information on the meetings.
Inclusion / exclusion criteria
We included reports on the acceptance of abstracts submitted to biomedical meetings and follow-up studies on the full publication of abstracts submitted to or accepted at biomedical meetings. Only studies on individual and identifiable biomedical meetings were included; studies on large thematic collections of abstracts from a variety of meetings were excluded. We checked whether there was overlap in the included meetings. Reports on meetings in social science including psychology were excluded.
Reported details on follow-up period, the searched databases and other sources, and the procedure used to match an abstract with its corresponding full publication were identified as indicators of methodological quality of follow-up studies. For inclusion, these key components had to be described adequately.
For each identifiable meeting we extracted information on the most frequently reported meeting characteristics; these were year, country, size, and speciality of the meeting. We hypothesised that they may be associated with meeting acceptance and full publication of abstracts. Size was defined as the number of abstracts presented at the meeting. Data from related specialities (for instance, anaesthesia and emergency medicine) were combined. When a report provided information on more than one meeting, data were extracted separately for each meeting.
We also extracted dichotomous data on abstract characteristics that were potentially associated with acceptance or subsequent publication, for instance, study outcome (positive / negative), topic of research (basic / clinical), and type of presentation (oral / poster). Definitions of these characteristics were taken as provided in the original reports.
Two investigators extracted all relevant data independently; discrepancies were resolved by discussion with the other investigators.
Acceptance rate was defined as ratio of number of accepted abstracts to number of submitted abstracts. Multiple logistic regression was used to investigate associations between abstract acceptance and meeting characteristics. Results were expressed as adjusted odds ratios (OR) with 95% confidence intervals (95%CI).
Publication rate was defined as ratio of number of subsequently published abstracts to total number of accepted abstracts. The delays until full publication were used as time-to-event data for survival-type analyses. Full publication was defined as the event; abstracts that remained unpublished were censored at time of last follow-up. Results were plotted as Kaplan-Meier (product limit) curves. A follow-up interval was defined as the delay between time points at which publication rates were reported during the follow-up period. For instance, if publication rates were reported for each year elapsed since the date of the meeting, the follow-up interval was one year. The average follow-up interval was calculated for each included meeting. We assumed that publication occurred always at the end of the follow-up intervals. Since average follow-up intervals of meetings varied, we performed sensitivity analyses using different maximum lengths of average follow-up intervals. Differences in subsequent full publication according to meeting characteristics were investigated using logrank tests.
Data on abstract characteristics were combined using the Mantel-Haenszel method and a fixed-effect model. Results were expressed as odds ratios (OR) with 95% confidence intervals (95%CI).
Retrieved and analysed studies
We retrieved 84 potentially relevant reports. We excluded 14: three were not about biomedical meetings [5, 6, 7], six were on collections of abstracts across meetings that were not identifiable individually [8, 9, 10, 11, 12, 13]; and in five, relevant data could not be extracted [14, 15, 16, 17, 18]. We analysed data from 64 studies published in 70 reports [1, 3, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86] (see Additional file 1). Four of those studies were published in two full reports each and with partial overlap of the data [1, 45, 46, 50, 51, 62, 74, 84]. Another study was published in three abstracts with substantial redundancy of the reported data [79, 80, 81]. All redundant data were disregarded.
Flow 1 – From submission to acceptance
Flow 1, from submission to acceptance. Meeting characteristics associated with abstract acceptance
Number of abstracts
95% conf. interval
Year (median: 1990)
1990 or before
Size (median: 89 abstracts)
Smaller than median
Larger than median
Flow 1, from submission to acceptance. Meta-analyses of abstract characteristics associated with abstract acceptance.
Accepted / submitted abstracts
Accepted / submitted abstracts
Odds ratio (95% conf. interval)
131 / 216(61%)
Flow 2 – From acceptance to subsequent publication
Fifty studies provided information on Flow 2 (Additional file 1, Figure 1). They reported on 19123 abstracts presented at 234 biomedical meetings held between 1957 and 1998 (median, 1992). The majority (53%) of meeting specialities were surgery (25%), internal medicine (13%), anaesthesia/emergency medicine (9%), and paediatrics (6%); the remaining 47% comprised a broad range of 27 different specialities.
Flow 2, from abstract acceptance to subsequent publication. Meta-analyses of meeting characteristics associated with full publication
Fully published / accepted abstracts
Fully published / accepted abstracts
Odds ratio (95% conf. interval)
262 / 471 (56%)
Flow 3 – Full publication despite meeting rejection
Nine studies provided information on Flow 3, with eight of those reporting on analysable data on the fate of both accepted and rejected abstracts (Additional file 1, Figure 1). These eight studies reported on 4414 abstracts submitted to 24 biomedical meetings held between 1976 and 1995. Most meetings were in paediatrics (50%), internal medicine (25%), or surgery (17%). Follow-up periods ranged from 36 months  to 88 months . Full publications were identified for 1042 (52%) of 2002 accepted abstracts, and for 641 (27%) of 2412 rejected abstracts, respectively. Abstract acceptance was strongly associated with full publication (OR 2.95, 95%CI 2.59–3.36).
In four studies, authors were queried on the reasons why their manuscript had not been published [3, 75, 82, 84]. Of 549 valid responses, lack of time was the most cited reason (31%), followed by low priority (21%), prior rejection (10%), problems with co-authors (9%), anticipated rejection (8%), and negative study results (3%).
How many abstracts get published?
The number of submitted abstracts (accepted or not) that were eventually published in full could be estimated in two different ways. First, in 11 studies (39 meetings), abstracts were followed up from submission to full publication [73, 74, 75, 76, 77, 78, 79, 82, 83, 85, 86]. Follow-up times ranged from 8 to 127 months. Of 7672 abstracts that were submitted to biomedical meetings, 2415 (31%) were eventually published in full.
The second estimation yielded a very similar number. Of all submitted abstracts, about 46% were accepted for presentation, while 54% were rejected (Flow 1). Of the accepted abstracts, 44% (20% of all submissions) were subsequently published as full reports within six years (Flow 2). Of the rejected abstracts, 27% (15% of all submissions) were published as full reports despite rejection (Flow 3). Taken together, of all submitted abstracts, 35% were eventually published.
About one third of abstracts submitted to biomedical meetings is eventually published as full reports. This estimation was derived from two different methodological approaches and both gave very similar results. It is based on the assumption that the analysed abstracts are representative samples of all abstracts. Also, for rejected abstracts, the time-course until publication remained obscure. However, the estimate implies that about 2/3 of all submitted biomedical research abstracts will not get published. Selective underreporting of research may be seen as a form of scientific misconduct , and it may result in publication bias. It is important, therefore, to understand why some abstracts are accepted at meetings and others are not, and why some of the accepted abstracts are published subsequently and others are not.
Ideally, all published full reports are based on data from sound scientific studies. Accordingly, peer review of meetings and journals may be regarded as a putatively laudable mechanism to filter out invalid studies. Abstract acceptance and oral presentation were associated with subsequent full publication. One might hypothesise that only the most valid studies would be selected for presentation at the meetings, and that among those only the best would be chosen for oral presentation and for subsequent full publication. Unfortunately, there is no reliable evidence to support this view. One report found prospective study design, affiliation with a university, and randomisation to be associated with abstract acceptance . Another report showed an association between the originality of a study and abstract acceptance . However, in the same report, a quality score was not significantly associated with abstract acceptance .
We identified five factors that possibly play a role when abstracts are filtered on their way from meeting submission to subsequent publication. First, abstracts that reported on a positive study outcome were more likely to be published subsequently, as already shown in a previous analysis [2, 3]. Positive outcome was defined differently in the included studies; we decided to combine these data despite diverging definitions. When authors were asked why their study had remained unpublished, less than 5% indicated that a negative study result was the reason [82, 84]. Low priority, anticipated rejection, or lack of time were more often cited, but these reasons may be related to negative study outcome [82, 84]. When investigators were surveyed for reasons of non-completion or non-submission of work initially presented at a meeting, similar answers were found . Our analysis provides evidence that positive outcome bias is likely to occur already at an earlier stage, i.e. at abstract submission. Peers who select abstracts submitted to biomedical meetings might favour studies that report on positive results. The authors themselves may be not only less enthusiastic about publication of negative study results , but also about their prior submission to scientific meetings.
Second, abstracts that reported on basic (as opposed to clinical) research were more likely to be accepted at biomedical meetings and to be published subsequently. Differences in the quality of conducting and reporting of basic and clinical research may explain this finding. However, in both domains, studies that investigated the relationship between the quality of abstracts and their likelihood of publication came to contradictory conclusions [22, 90]. Whether the abstracts included in the analysed follow-up studies were representative for basic or clinical research could not be assessed. It is conceivable that, in basic research, only the most important findings were submitted for presentation and later published. When chairpersons and senior research advisors were asked to rate robustness and quality of research projects performed in their departments, they consistently rated basic research highest . Assuming a significant involvement of these research leaders in the reviewing of meeting abstracts and journal manuscripts, this implies the existence of a bias in favour of basic research.
Third, meetings comprising a larger number of abstracts had higher acceptance rates. But abstracts presented at smaller meetings were more likely to be published subsequently. Since meeting organisers often wish to attract the maximum number of attendees, a less rigorous selection of abstracts may result for larger meetings. At smaller meetings the peer review process may be more stringent, leading to selection of higher-quality papers that, in turn, would be more likely to be published eventually. However, numerous factors determine the size of a meeting and make the relationship with the acceptance of abstracts difficult to interpret.
Fourth, abstracts submitted to US meetings were accepted less often, but when accepted, were more likely to be published subsequently. This suggests that filtering mechanisms at US meetings are different from those at non-US meetings, but once again other underlying factors may play a role.
Fifth, the most frequently investigated specialities had different rates of acceptance and publication. Internal medicine and paediatrics had higher abstract acceptance rates than surgery. Paediatrics and surgery meetings had higher full publication rates than anaesthesia/emergency medicine meetings.
The analysed reports did not allow to explain these disparities.
Choosing an appropriate statistical approach for combining the time-to-event data presented a methodological challenge since publication rates were reported in widely varying average follow-up intervals. In previous analyses a single publication rate after the median follow-up time was calculated [2, 3]. We estimated the time-course of publication rates using the techniques of survival analysis. Then we performed sensitivity analyses by excluding data from reports with less detailed follow-up intervals. Based on reports with the most detailed follow-up (intervals ≤ 1 year), the publication rates increased to about 44% at six years after meeting presentation. When data from reports with longer follow-up intervals were included in the analyses, estimates differed. Generally, publication rates were lower for observation periods up to six years, and were higher for those longer than six years. Thus, for this type of combined time-to-event analyses, there may be an argument to only include data from studies with short follow-up intervals, i.e. more detailed follow-up.
Often, acceptance rates were reported as a secondary outcome of studies that investigated the functioning of scientific meeting committees. Therefore, we may have missed other relevant studies, and the retrieved studies may not be representative. Also, the methodology of included studies varied with regard to the length of follow-up after a meeting, follow-up intervals, type and number of data sources, and criteria of matching between abstracts and subsequent full reports. In an attempt to take this heterogeneity into account, we conducted sensitivity analyses with subgroups of studies with different maximum follow-up intervals. This resulted in different Kaplan-Meier curves.
We decided pre hoc to exclude studies on collections of abstracts that originated from a variety of meetings, since they did not provide information on the individual meetings. Two studies with good methodological quality were among those [8, 9]. The publication rate of abstracts from the Oxford Database of Perinatal Trials was reported to be 39% after at least four years of follow-up . For abstracts collected by the Cochrane Cystic Fibrosis Group, the publication rate after five years was 40% .
There are still unresolved questions, and these could be addressed in further studies. For instance, we did not study what happens to scientific data before abstract submission, nor did we look at alternative pathways of dissemination. An unknown number of manuscripts are submitted to journals directly and arrive at full publication without former presentation at meetings. Also, criteria applied by peers to select abstracts for meeting presentation are not well understood. Finally, we could not draw any conclusions on the relationship between study quality and the likelihood of abstract acceptance and subsequent full publication.
Of abstracts that are submitted to biomedical meetings, only about one third is subsequently published in full. Both meeting acceptance and full publication are associated with specific characteristics of the abstracts themselves and of the meetings. Our study confirms knowledge from previous similar analyses that concentrated on the flow of scientific data from presentation at a meeting to subsequent full publication [2, 3]. It integrates this previous knowledge with additional relevant information, i.e. the rate of abstract acceptance at meetings, and the fate of rejected abstracts. Thus, our analysis adds to an improved understanding of how scientific data are disseminated.
Dr Tramèr is the recipient of a PROSPER grant (N° 3233-051939.97/2) from the Swiss National Science Foundation. Dr von Elm's salary was provided by the Swiss National Science Foundation (N° 3200-052199.97/1 and 3200-064800.01/1). We thank the numerous authors who responded to our enquiry. We are grateful to Daniel Haake from the Library of the Geneva Medical School for his help in searching electronic databases, and to Greta Poglia who assisted in extracting and cross-checking data. Preliminary results of this study were presented at the 4th International Congress on Peer Review in Biomedical Publication, Barcelona/Spain, September 14–16, 2001.
- 2.Scherer RW, Langenberg P: Full publication of results initially presented in abstracts (Cochrane Methodology Review). The Cochrane Library. 2001, 3-Google Scholar
- 4.Science Citation Index Expanded. Institute for Scientific Information. Philadelphia/USA. March 15, 2001, [http://www.isinet.com/isi/products/citation/scie/index.html]
- 10.Dickersin K, Min Y-I: NIH clinical trials and publication bias. Online J Curr Clin Trials. 1993, Doc No 50:Google Scholar
- 26.Short EM: The role of abstracts and meeting presentations in the communication of scientific information. Clin Res. 1982, 30: 1-8.Google Scholar
- 30.Wuerz RE, Holliman CJ: Attendance and research abstract activity at the 1993 annual meetings of the academic medical society [abstract & unpublished manuscript]. Acad Emerg Med. 1994, 1 (Suppl): A59-Google Scholar
- 35.Ciesla MC, Massarani-Wafai R, Wojcik EM: What cytopathology abstracts become published papers in peer-reviewed journals? [abstract]. 48th Annual Scientific Meeting of the American Society of Cytopathology. Acta Cytol. 2000, 44: 927-928.Google Scholar
- 42.Ensom MHH, Walker SE: From abstract to publication: what makes the grade?. Can J Hosp Pharm. 1998, 51: 280-284.Google Scholar
- 45.Goldman L, Loscalzo A: Fate of cardiology research originally published in abstract form. 1980, 303: 255-259.Google Scholar
- 46.Goldman L, Loscalzo A: Publication rates of research originally presented in abstract form in three subspecialities of internal medicine. Clin Res. 1982, 30: 13-17.Google Scholar
- 56.Maleck WH, Koetter KP, Lenz M: Are papers presented at prehospital care research forum published in Medline-indexed journals? [abstract]. In Proceedings of the Prehospital Care Research Forum, March 11–14, 1998, Baltimore. Carlsbad (CA): Jems Communications. 1998, S-12- , Baltimore. Carlsbad (CA): Jems CommunicationsGoogle Scholar
- 57.Maleck WH, Petroianu GA, Jatzko A: Are papers presented at PECEMS published in Medline-indexed journals? [abstract]. In Proceedings of the Fourth Pan-European Conference on Emergency Medical Systems, Aug 23–27. 1998, , Opatija, Croatia. 20Google Scholar
- 64.Ramsey G, Sherman LA: The publication rate of scientific abstracts presented at the American Association of Blood Banks Annual Meeting [abstract]. 48th Annual Meeting of the AABB; Nov. 11–15, 1995; New Orleans, USA. Transfusion. 1995, 35 (Suppl 10): S1-Google Scholar
- 70.Sebel PS, Duensing SJ: Abstracts from the IARS Clinical and Scientific Congress: proportion published and time to publication [abstract]. Anesth Analg. 2001, 92: S110-Google Scholar
- 79.Ohlsson A, Walia R: Differences between information provided in abstracts of randomized controlled trials in neonates submitted to the annual meeting of the American Pediatric Society and the Society for Pediatric Research (APS/SPR) and final publications – implications for meta-analysis [abstract]. Abstr Book Cochrane Colloq. 1999, 7: 19-Google Scholar
- 81.Walia R, Ohlsson A: Clinical trials in perinatal-neonatal medicine: Outcome of abstracts submitted for annual conference of the American Pediatric Society and the Society for Pediatric Research – limited to randomized control trials in neonatology [abstract]. Abstr Book Cochrane Colloq. 1999, 7: 56-Google Scholar
- 82.Timmer A: Publication bias in gastroenterological research [MSc-Thesis]. Calgary/Canada: University of Calgary. 1999Google Scholar
- 83.Todd NW: Selection of abstracts for an otolaryngology meeting: Correlation of blinding with subsequent publication. In Proceedings of the Third International Congress on Biomedical Peer Review and Global Communications, Sep 18–20. 1997, May 10, 2002, [http://www.ama-assn.org/public/peer/sepu.htm] ; Prague, Czech RepublicGoogle Scholar
- 84.Weber EJ, Callaham ML, Wears RL, Barton C, Young G: Unpublished research from a medical specialty meeting: why investigators fail to publish. 1998, 280: 257-259.Google Scholar
- 86.Duchini A, Genta RM: From abstract to peer-reviewed article: the fate of abstracts submitted to the DDW [abstract]. Gastroenterology. 1997, 112 (suppl): A12-Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/3/12/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.