Skip to main content
SpringerLink
Log in

Precision and Recall of Search Strategies for Identifying Studies on Work-Related Psychosocial Risk Factors in PubMed

  • Published:
Journal of Occupational Rehabilitation Aims and scope Submit manuscript

Abstract

Purpose

This study aims to report on the effectiveness of various search strategies and keywords to find studies on work-related psychosocial risk factors (PRF) in the PubMed bibliographic database.

Methods

We first selected by hand-searching 191articles published on PRF and indexed in PubMed. We extracted 30 relevant MeSH terms and 38 additional textwords. We then searched PubMed combining these 68 keywords and 27 general keywords on work-related factors. Among the 2953 articles published in January 2020, we identified 446 articles concerning exposure to PRF, which were gathered in a Gold Standard database. We then computed the Recall, Precision, and Number Needed to Read of each keyword or combination of keywords.

Results

Overall, 189 search-words alone or in combination were tested. The highest Recall with a single MeSH term or textword was 43% and 35%, respectively. Subsequently, we developed two different search strings, one optimizing Recall while keeping Precision acceptable (Recall 98.2%, Precision 5.9%, NNR 16.9) and one optimizing Precision while keeping Recall acceptable (Recall 73.1%, Precision 25.5%, NNR 9.7).

Conclusions

No single MeSH term is available to identify relevant studies on PRF in PubMed. Locating these types of studies requires the use of various MeSH and non-MeSH terms in combination to obtain a satisfactory Recall. Nevertheless, enhancing the Recall of search strategies may lead to lower Precision, and higher NNR, although with a non-linear trend. This factor must be taken into consideration when searching PubMed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

The datasets (Gold Standard database) analyzed in the current study are available as a public collection in Pubmed: https://www.ncbi.nlm.nih.gov/sites/myncbi/1XUp-adEoi/collections/62100220/public/.

References

  1. Niedhammer I, Bertrais S, Witt K. Psychosocial work exposures and health outcomes: a meta-review of 72 literature reviews with meta-analysis. Scand J Work Environ Health. 2021;47:489–508.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Duchaine CS, Aubé K, Gilbert-Ouimet M, Vézina M, Ndjaboué R, Massamba V, et al. Psychosocial stressors at work and the risk of sickness absence due to a diagnosed mental disorder: a systematic review and meta-analysis. JAMA Psychiat. 2020;77:842–851.

    Article  Google Scholar 

  3. European Agency for Safety and Health at Work, Brun E, Milczarek M, editors. Expert forecast on emerging psychosocial risks related to occupational safety and health. Luxembourg: Office for Official Publications of the European Communities; 2007.

  4. Dunn K, Marshall JG, Wells AL, Backus JEB. Examining the role of MEDLINE as a patient care information resource: an analysis of data from the Value of Libraries study. J Med Libr Assoc. 2017;105:336–346.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Sampson M, de Bruijn B, Urquhart C, Shojania K. Complementary approaches to searching MEDLINE may be sufficient for updating systematic reviews. J Clin Epidemiol. 2016;78:108–115.

    Article  PubMed  Google Scholar 

  6. Rollin L, Darmoni S, Caillard J-F, Gehanno J-F. Searching for high-quality articles about intervention studies in occupational health–what is really missed when using only the Medline database? Scand J Work Environ Health. 2010;36:484–487.

    Article  PubMed  Google Scholar 

  7. Halladay CW, Trikalinos TA, Schmid IT, Schmid CH, Dahabreh IJ. Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. J Clin Epidemiol. 2015;68:1076–1084.

    Article  PubMed  Google Scholar 

  8. Cooper C, Varley-Campbell J, Booth A, Britten N, Garside R. Systematic review identifies six metrics and one method for assessing literature search effectiveness but no consensus on appropriate use. J Clin Epidemiol. 2018;99:53–63.

    Article  PubMed  Google Scholar 

  9. Mattioli S, Zanardi F, Baldasseroni A, Schaafsma F, Cooke RMT, Mancini G, et al. Search strings for the study of putative occupational determinants of disease. Occup Environ Med. 2010;67:436–443.

    Article  CAS  PubMed  Google Scholar 

  10. Mattioli S, Gori D, Di Gregori V, Ricotta L, Baldasseroni A, Farioli A, et al. PubMed search strings for the study of agricultural workers’ diseases. Am J Ind Med. 2013;56:1473–1481.

    Article  PubMed  Google Scholar 

  11. Mattioli S, Farioli A, Cooke RMT, Baldasseroni A, Ruotsalainen J, Placidi D, et al. Hidden effectiveness? Results of hand-searching Italian language journals for occupational health interventions. Occup Environ Med. 2012;69:522–524.

    Article  PubMed  Google Scholar 

  12. Gehanno J-F, Rollin L, Le Jean T, Louvel A, Darmoni S, Shaw W. Precision and recall of search strategies for identifying studies on return-to-work in Medline. J Occup Rehabil. 2009;19:223–230.

    Article  PubMed  Google Scholar 

  13. Verbeek J, Salmi J, Pasternack I, Jauhiainen M, Laamanen I, Schaafsma F, et al. A search strategy for occupational health intervention studies. Occup Environ Med. 2005;62:682–687.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Taouk Y, Spittal MJ, LaMontagne AD, Milner AJ. Psychosocial work stressors and risk of all-cause and coronary heart disease mortality: a systematic review and meta-analysis. Scand J Work Environ Health. 2020;46:19–31.

    Article  CAS  PubMed  Google Scholar 

  15. van der Molen HF, Nieuwenhuijsen K, Frings-Dresen MHW, de Groene G. Work-related psychosocial risk factors for stress-related mental disorders: an updated systematic review and meta-analysis. BMJ Open. 2020;10:e034849.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Schaafsma F, Hulshof C, Verbeek J, Bos J, Dyserinck H, van Dijk F. Developing search strategies in Medline on the occupational origin of diseases. Am J Ind Med. 2006;49:127–137.

    Article  PubMed  Google Scholar 

  17. Kok R, Verbeek JAHM, Faber B, Dijk FJH, Hoving JL. A search strategy to identify studies on the prognosis of work disability: a diagnostic test framework. BMJ Open. 2015;5:e006315.

    Article  PubMed  PubMed Central  Google Scholar 

  18. van de Glind EMM, van Munster BC, Spijker R, Scholten RJPM, Hooft L. Search filters to identify geriatric medicine in Medline. J Am Med Inform Assoc. 2012;19:468–472.

    Article  PubMed  Google Scholar 

  19. Wilczynski NL, Haynes RB. Optimal search filters for detecting quality improvement studies in Medline. BMJ Qual Saf. 2010;19:e31–e31.

    Article  CAS  Google Scholar 

  20. Wilczynski NL. Optimal search strategies for detecting health services research studies in MEDLINE. Can Med Assoc J. 2004;171:1179–1185.

    Article  Google Scholar 

  21. Haafkens J, Moerman C, Schuring M, van Dijk F. Searching bibliographic databases for literature on chronic disease and work participation. Occup Med (Lond). 2006;56:39–45.

    Article  PubMed  Google Scholar 

  22. Gehanno JF, Paris C, Thirion B, Caillard JF. Assessment of bibliographic databases performance in information retrieval for occupational and environmental toxicology. Occup Environ Med. 1998;55:562–566.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Syst Rev. 2012;1:19.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Wagner M, Rosumeck S, Küffmeier C, Döring K, Euler U. A validation study revealed differences in design and performance of MEDLINE search filters for qualitative research. J Clin Epidemiol. 2020;120:17–24.

    Article  PubMed  Google Scholar 

  25. Rice DB, Kloda LA, Levis B, Qi B, Kingsland E, Thombs BD. Are MEDLINE searches sufficient for systematic reviews and meta-analyses of the diagnostic accuracy of depression screening tools? A review of meta-analyses. J Psychosom Res. 2016;87:7–13.

    Article  PubMed  Google Scholar 

Download references

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Institute of Occupational Medicine, Rouen University Hospital, 1 Rue de Germont, 76000, Rouen, France

    Jean-Francois Gehanno & Laetitia Rollin

  2. Laboratory of Medical Informatics and Knowledge Engineering in e-Health, Inserm, Rouen University, Sorbonne University, University of Paris 13, LIMICS, Paris, France

    Jean-Francois Gehanno & Laetitia Rollin

  3. Centre de Consultations de Pathologie Professionnelle, CHRU de Nancy, Vandoeuvre les Nancy, France

    Isabelle Thaon

  4. Hospital University Center of Saint-Etienne, Université Lyon 1, Université de St Etienne, Université Gustave Eiffel-IFSTTAR, 42005, Saint-Etienne, France

    Carole Pelissier

  5. UMRESTTE UMR-T9405, 42005, Saint-Etienne, France

    Carole Pelissier

Authors
  1. Jean-Francois Gehanno
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isabelle Thaon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carole Pelissier
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Laetitia Rollin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study’s conception and design. Material preparation, data collection and analysis were performed by J-FG, IT, CP and LR. The first draft of the manuscript was written by J-FG and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jean-Francois Gehanno.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gehanno, JF., Thaon, I., Pelissier, C. et al. Precision and Recall of Search Strategies for Identifying Studies on Work-Related Psychosocial Risk Factors in PubMed. J Occup Rehabil 33, 776–784 (2023). https://doi.org/10.1007/s10926-023-10110-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10926-023-10110-w

Keywords

Search

Navigation