Skip to main content
SpringerLink
Log in

The assessment of publication pressure in medical science; validity and reliability of a Publication Pressure Questionnaire (PPQ)

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

To determine content validity, structural validity, construct validity and reliability of an internet-based questionnaire designed for assessment of publication pressure experienced by medical scientists.

Methods

The Publication Pressure Questionnaire (PPQ) was designed to assess psychological pressure to publish scientific papers. Content validity was evaluated by collecting independent comments from external experts (n = 7) on the construct, comprehensiveness and relevance of the PPQ. Structural validity was assessed by factor analysis and item response theory (IRT) using the generalized partial credit model. Pearson’s correlation coefficients were calculated to assess potential correlations with the emotional exhaustion and depersonalization subscales of the Maslach Burnout Inventory (MBI). Single test reliability (lambda2) was obtained from the IRT analysis.

Results

Content validity was satisfactory. Confirmatory factor analysis did not support the presence of three initially assumed separate domains of publication pressure (i.e., personally experienced publication pressure, publication pressure in general, pressure on position of scientist). After exclusion of the third domain (six items), we performed exploratory factor analysis and IRT. The goodness-of-fit statistics for the IRT assuming a single dimension were satisfactory when four items were removed, resulting in 14 items of the final PPQ. Correlations with the emotional exhaustion and depersonalization scales of the MBI were 0.34 and 0.31, respectively, supporting construct validity. Single test administration reliability lambda2 was 0.69 and 0.90 on the test scores and expected a posteriori scores, respectively.

Conclusion

The PPQ seems a valid and reliable instrument to measure publication pressure among medical scientists.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Gannon, F. (2000). The impact of the impact factor. EMBO Reports, 1(4), 293.

    Article  CAS  PubMed Central  Google Scholar 

  2. Retzer, V., & Jurasinski, G. (2009). Towards objectivity in research evaluation using bibliometric indicators—A protocol for incorporating complexity. Basic and Applied Ecology, 10(5), 393–400.

    Article  Google Scholar 

  3. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    CAS  PubMed Central  PubMed  Google Scholar 

  4. Bedeian, A. G., Van Fleet, D. D., & Hyman, H. H. (2009). Scientific achievement and editorial-board membership. Organizational Research Methods, 12(211), 238.

    Google Scholar 

  5. Bird, S. J. (2006). Research ethics, research integrity and the responsible conduct of research. Science and Engineering Ethics, 12(411), 412.

    Google Scholar 

  6. Glick, W. H., Miller, C. C., & Cardinal, L. B. (2007). Making a life in the field of organization science. Journal of Organizational Behavior, 28, 817–835.

    Article  Google Scholar 

  7. Anderson, M. S., Ronning, E. A., De, V. R., & Martinson, B. C. (2007). The perverse effects of competition on scientists’ work and relationships. Science and Engineering Ethics, 13(4), 437–461.

    Article  PubMed  Google Scholar 

  8. Anderson, M. S., Martinson, B. C., & De Vries, R. (2007). Normative dissonance in science: Results from a notional survey of U.S. Scientists. Journal of Empirical Research on Human Research Ethics, 2(4), 3–14.

    Article  CAS  PubMed  Google Scholar 

  9. Miller, A. N., Taylor, S. G., & Bedeian, A. G. (2011). Publish or perish: Academic life as management faculty live it. Career Development International, 16(5), 422–445.

    Article  Google Scholar 

  10. Fanelli, D. (2010). Do pressures to publish increase scientists’ bias? An empirical support from US States Data. PLoS One, 5(4), e10271.

    Article  PubMed Central  PubMed  Google Scholar 

  11. West, C. P., Tan, A. D., Habermann, T. M., Sloan, J. A., & Shanafelt, T. D. (2009). Association of resident fatigue and distress with perceived medical errors. JAMA, 302(12), 1294–1300.

    Article  CAS  PubMed  Google Scholar 

  12. Prins, J. T., Hoekstra-Weebers, J. E., Gazendam-Donofrio, S. M., Dillingh, G. S., Bakker, A. B., Huisman, M., et al. (2010). Burnout and engagement among resident doctors in the Netherlands: A national study. Medical Education, 44(3), 236–247.

    Article  PubMed  Google Scholar 

  13. Shanafelt, T. D., Boone, S., Tan, L., Dyrbye, L. N., Sotile, W., Satele, D., et al. (2012). Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Archives of Internal Medicine, 20, 1–9.

    Google Scholar 

  14. van Dalen, H. P., & Henkens, K. (2012). Intended and unintended consequences of a publish-or-perish culture: A worldwide survey. Journal of the American Society for Information Science and Technology, 63(7), 1282–1293.

    Article  Google Scholar 

  15. Streiner, D. L., & Norman, G. R. (2008). Health measurement scales: A practical guide to their development and use. Oxford: Oxford University Press.

    Book  Google Scholar 

  16. Tijdink, J. K., Vergouwen, A. C., & Smulders, Y. M. (2013). Publication pressure and burn out among Dutch medical professors: A nationwide survey. PLoS One, 8(9), e73381.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  17. Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., et al. (2010). The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: An international Delphi study. Quality of Life Research, 19(4), 539–549.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12(1), 58–79.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  19. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.

    Article  Google Scholar 

  20. Mplus user’s guide [computer program]. Version 6.12. Los Angeles: Muthén & Muthén; 2007.

  21. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists (1st ed.). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  22. Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159–176.

    Article  Google Scholar 

  23. Baker, F. B. (2001). The basics of item response theory (2nd ed.). College Park: ERIC Clearinghouse on Assessment and Evaluation.

    Google Scholar 

  24. Glas, C. A. W. (1999). Modification indices for the 2-PL and the nominal response model. Psychometrika, 64(3), 273–294.

    Article  Google Scholar 

  25. van Groen, M. M., ten Klooster, P. M., Taal, E., van de Laar, M. A. F. J., & Glas, C. A. W. (2010). Application of the health assessment questionnaire disability index to various rheumatic diseases. Quality of Life Research, 19, 1255–1263.

    Article  PubMed Central  PubMed  Google Scholar 

  26. Preliminary manual of the software program Multidimensional Item Response Theory (MIRT) [computer program]. Version 1.01. Enschede: University of Twente; 2010.

  27. Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120.

    Article  PubMed Central  PubMed  Google Scholar 

  28. Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal and structural equation models. Boca Raton, FL: Chapman & Hall/CRC Press.

    Book  Google Scholar 

  29. Maslach, C., Jackson, S. E., & Leiter, M. P. (1996). Maslach Burnout Inventory manual (3rd ed.). Palo Alto, CA: Consulting Psychologists Press.

    Google Scholar 

  30. Schaufeli, W. B., & van Dierendonk, D. (2000). Utrechtse Burnout Schaal (UBOS), handleiding [Utrecht Burnout Scale, manual]. Utrecht: Swets & Zeitlinger.

    Google Scholar 

  31. Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in web- or internet-based surveys. Educational and Psychological Measurement, 60, 821–836.

    Article  Google Scholar 

  32. Wicherts, J. M. (2011). Psychology must learn a lesson from fraud case. Nature, 480(7375), 7.

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Internal Medicine, VU University Medical Centre, PO Box 7057, 1007 MB, Amsterdam, The Netherlands

    J. K. Tijdink & Y. M. Smulders

  2. Department of Psychiatry, Tergooi Hospital, Blaricum, The Netherlands

    J. K. Tijdink

  3. Department of Psychiatry, St. Lucas Andreas Hospital, Amsterdam, The Netherlands

    A. C. M. Vergouwen

  4. Department of Epidemiology and Biostatistics, VU University Medical Center, Amsterdam, The Netherlands

    H. C. W. de Vet & D. L. Knol

  5. EMGO+ Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands

    H. C. W. de Vet

Authors
  1. J. K. Tijdink
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Y. M. Smulders
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. C. M. Vergouwen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. H. C. W. de Vet
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. D. L. Knol
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. K. Tijdink.

Appendix

Appendix

See Table 4.

Table 4 Publication Pressure Questionnaire
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tijdink, J.K., Smulders, Y.M., Vergouwen, A.C.M. et al. The assessment of publication pressure in medical science; validity and reliability of a Publication Pressure Questionnaire (PPQ). Qual Life Res 23, 2055–2062 (2014). https://doi.org/10.1007/s11136-014-0643-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-014-0643-6

Keywords

Search

Navigation