Skip to main content
Log in

The relation between the quality of research, researchers’ experience, and their academic environment

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

This article investigates to what extent researchers’ experience and the solidity of their academic environment influence the quality of their research. The hypotheses are derived from the assumptions that experience matters for quality research and that there are great intellectual synergies to be obtained from interacting with many colleagues who are active researchers. All articles published between 2000 and 2006 in five leading transportation journals are included in the analysis, and their research quality is measured by the number of times each article is cited by August 2016. Controlling for other factors influencing citations, such as article age and the number of references, the most important finding is that both experience and academic environment matter for performing quality research. When the authors’ experience, measured by the number of previous publications, increases by 1% from its average level, their published articles are expected to garner 0.31% more citations. Moreover, when the research activity at the unit to which the authors are affiliated, measured by the unit’s total number of publications, increases by 1% from its average level, the number of times their articles are cited will increase by 0.19%. This signals that, relatively speaking, the researchers’ own experience and merits mean more than the academic environment with regard to producing high-quality research. The above results enable us to discuss how researchers’ experience can compensate for working in less active academic communities holding research quality constant.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. A thorough review on citing motives is provided by Bornmann and Daniel (2008).

  2. From SCImago (2015), we deduced the following values of the weights: \(v_{\text{PA}} = 1.131,v_{\text{TR}} = 1.261, v_{\text{TS}} = 1.405\) and \(v_{\text{PB}} = 1.556.\) A similar method has been used in Norway to measure the research activity (publication points) at different universities and colleges; see Aagaard et al. (2014).

  3. It is obvious that the value of AN has nothing to do with an article’s quality. This reveals one weakness of using citation counts as a quality indicator.

  4. The five journals from which the articles analysed in this study were drawn are all in the top quartile of 119 transportation journals as ranked by SCImago (2015). This confirms that our selected journals are of good quality.

  5. The \({\text{MRT }}\) defined here is equivalent with the concept “marginal rate of technical substitution.” It is an important concept when it comes to describing the characteristics of different production functions. See, for example, Nicholson (1998).

  6. Problems of ranking researchers’ merits across different fields are discussed in Perry and Reny (2016).

  7. Daniel McFadden and James Heckman were awarded the Nobel Prize in economics in 2000. An extensive review of the developments in the economics of transportation is given in Winston (1985).

References

  • Aagaard, K., Bloch, C., Schneider, J. W., Henriksen, D., Ryan, T. K., & Lauridsen, P. S. (2014). Evaluation of the Norwegian publication indicator—English summary. Aarhus: Danish Centre for Studies in Research and Research Policy, Aarhus University.

    Google Scholar 

  • Abramo, G., & D’Angelo, C. A. (2015). The relationship between the number of authors of a publication, its citations and the impact factor of the publishing journal: Evidence from Italy. Journal of Informetrics, 9(4), 746–761. https://doi.org/10.1016/j.joi.2015.07.003.

    Article  Google Scholar 

  • Aksnes, D. W. (2003). Characteristics of highly cited papers. Research Evaluation, 12(3), 159–170. https://doi.org/10.3152/147154403781776645.

    Article  Google Scholar 

  • Beaver, D. D. (2004). Does collaborative research have greater epistemic authority? Scientometrics, 60(3), 399–408.

    Article  Google Scholar 

  • Becker, G. S. (1964). Human capital: A theoretical and empirical analysis with special reference to education. Chicago: The University of Chicago Press.

    Google Scholar 

  • Bergh, D. D., Perry, J., & Hanke, R. (2006). Some predictors of SMJ article impact. Strategic Management Journal, 27(1), 81–100.

    Article  Google Scholar 

  • Bertocchi, G., Gambardella, A., Jappelli, T., Nappi, C. A., & Peracchi, F. (2015). Bibliometric evaluation vs. informed peer review: Evidence from Italy. Research Policy, 44(2), 451–466. https://doi.org/10.1016/j.respol.2014.08.004.

    Article  Google Scholar 

  • Bonaccorsi, A., & Daraio, C. (2003). Age effects in scientific productivity: The case of the Italian National Research Council (CNR). Scientometrics, 58(1), 49–90.

    Article  Google Scholar 

  • Bornmann, L., & Daniel, H. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80.

    Article  Google Scholar 

  • Diamond, A. M. (1986). What is a citation worth? The Journal of Human Resources, 21(2), 200–215. https://doi.org/10.2307/145797.

    Article  Google Scholar 

  • Didegah, F., & Thelwall, M. (2013). Determinants of research citation impact in nanoscience and nanotechnology. Journal of the American Society for Information Science and Technology, 64(5), 1055–1064.

    Article  Google Scholar 

  • Ebadi, A., & Schiffauerova, A. (2015). How to become an important player in scientific collaboration networks? Journal of Informetrics, 9(2015), 809–825.

    Article  Google Scholar 

  • Fox, J. (2016). Applied regression analysis and generalized linear models. Beverly Hills: Sage.

    Google Scholar 

  • Garfield, E. (1962). Can citation indexing be automated? Essays of an Information Scientist, 1, 84–90.

    Google Scholar 

  • Garfield, E. (1998). Random thoughts on citationology its theory and practice. Scientometrics, 43(1), 69–76. https://doi.org/10.1007/bf02458396.

    Article  Google Scholar 

  • Gazni, A., & Didegah, F. (2011). Investigating different types of research collaboration and citation impact: A case study of Harvard University’s publications. Scientometrics, 87(2), 251–265. https://doi.org/10.1007/s11192-011-0343-8.

    Article  Google Scholar 

  • Habibzadeh, F., & Yadollahie, M. (2010). Are shorter article titles more attractive for citations? Crosssectional study of 22 scientific journals. Croatian Medical Journal, 51(2), 165–170.

    Article  Google Scholar 

  • Hair, J. F. (1998). Multivariate data analysis. Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Hanssen, T.-E. S., & Jørgensen, F. (2014). Citation counts in transportation research. European Transport Research Review, 6(2), 205–212.

    Article  Google Scholar 

  • Hanssen, T.-E. S., & Jørgensen, F. (2015). The value of experience in research. Journal of Informetrics, 9(1), 16–24.

    Article  Google Scholar 

  • Hensher, D. A., & Greene, W. H. (2003). The mixed logit model: The state of practice. Transportation, 30(2), 133–176. https://doi.org/10.1023/a:1022558715350.

    Article  Google Scholar 

  • Horta, H., Cattaneo, M., & Meoli, M. (2016). PhD funding as a determinant of PhD and career research performance. Studies in Higher Education. https://doi.org/10.1080/03075079.2016.1185406.

    Google Scholar 

  • Jamali, H. R., & Nikzad, M. (2011). Article title type and its relation with the number of downloads and citations. Scientometrics, 88(2), 653–661.

    Article  Google Scholar 

  • Judge, T. A., Cable, D. M., Colbert, A. E., & Rynes, S. L. (2007). What causes a management article to be cited-article author, or journal? Academy of Management Journal, 50(3), 491–506.

    Article  Google Scholar 

  • Kenna, R., & Berche, B. (2011). Critical mass and the dependency of research quality on group size. Scientometrics, 86(2), 527–540.

    Article  Google Scholar 

  • Kunnskapsdepartementet. (2014). Langtidsplan for forskning og høyere utdanning 2015–2024. Meld. St. 7 (2015–2024) Oslo.

  • Kyvik, S. (1995). Are big university departments better than small ones? Higher Education, 30(3), 295–304.

    Article  Google Scholar 

  • Lariviere, V., Gingras, Y., Sugimoto, C. R., & Tsou, A. (2015). Team size matters: Collaboration and scientific impact since 1900. Journal of the Association for Information Science and Technology, 66(7), 1323–1332.

    Article  Google Scholar 

  • Lokker, C., McKibbon, K. A., McKinlay, R. J., Wilczynski, N. L., & Haynes, R. B. (2008). Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: Retrospective cohort study. BMJ, 336(7645), 655–657.

    Article  Google Scholar 

  • Martin, B. R., & Irvine, J. (1983). Assessing basic research. Research Policy, 12(2), 61–90. https://doi.org/10.1016/0048-7333(83)90005-7.

    Article  Google Scholar 

  • Neuhaus, C., & Daniel, H. (2008). Data sources for performing citation analysis: An overview. Journal of Documentation, 64(2), 193–210. https://doi.org/10.1108/00220410810858010.

    Article  Google Scholar 

  • Nicholson, W. (1998). Microeconomic theory: Basic principles and extensions. Forth Worth: Dryden Press.

    Google Scholar 

  • NOU: 2016: 3. (2016). Ved et vendepunkt: Fra ressursøkonomi til kunnskapsøkonomi. Produktivitetskommisjonens andre rapport, Oslo.

  • Onodera, N., & Yoshikane, F. (2015). Factors affecting citation rates of research articles. Journal of the Association for Information Science and Technology, 66(4), 739–764. https://doi.org/10.1002/asi.23209.

    Article  Google Scholar 

  • Paiva, C. E., Lima, J. P. S. N., & Paiva, B. S. R. (2012). Articles with short titles describing the results are cited more often. Clinics, 67(5), 509–513.

    Article  Google Scholar 

  • Patterson, M. S., & Harris, S. (2009). The relationship between reviewers’ quality-scores and number of citations for papers published in the journal physics in medicine and biology from 2003–2005. Scientometrics, 80(2), 343–349. https://doi.org/10.1007/s11192-008-2064-1.

    Article  Google Scholar 

  • Perry, M., & Reny, P. J. (2016). How to count citations if you must. American Economic Review, 106(9), 2722–2741.

    Article  Google Scholar 

  • Ringelmann, M. (1913). Recherches sur les moteurs animés: Travails de l`homme. Annales del Institut National Agronomique, 2, 2–39.

    Google Scholar 

  • Ronda-Pupo, G. A., & Katz, J. S. (2016). The power-law relationship between citation-based performance and collaboration in management journals: A scale-independent approach. Journal of the Association for Information Science and Technology, 67(10), 2565–2572.

    Article  Google Scholar 

  • SCImago. (2015). SJR—SCImago Journal & Country Rank. http://www.scimagojr.com. Accessed January 20, 2015.

  • Smithson, M., & Merkle, E. C. (2013). Generalized linear models for categorical and continuous limited dependent variables. Boca Raton, FL: CRC Press, Taylor & Francis Group.

    MATH  Google Scholar 

  • Stankiewicz, R. (1979). The size and age of Swedish academic research groups and their scientific performance. In F. M. Andrews (Ed.), Scientific productivity. The effectiveness of research groups in six countries. Cambridge/Paris: Cambridge University Press/UNESCO.

    Google Scholar 

  • Steiner, I. D. (1972). Group process and productivity (social psychology). New York: Academic Press.

    Google Scholar 

  • Stremersch, S., Verniers, I., & Verhoef, P. C. (2007). The quest for citations: Drivers of article impact. Journal of Marketing, 71(3), 171–193.

    Article  Google Scholar 

  • Subotic, S., & Mukherjee, B. (2014). Short and amusing: The relationship between title characteristics, downloads, and citations in psychology articles. Journal of Information Science, 40(1), 115–124.

    Article  Google Scholar 

  • Tregenza, T. (1997). Darwin a better name than Wallace? Nature, 385(6616), 480.

    Article  Google Scholar 

  • Vabø, A., & Kårstein, A. (2014). Robuste fagmiljøer. En litteraturgjennomgang. Oslo: Nordic Institute for Studies in Innovation, Research and Education.

    Google Scholar 

  • Vanclay, J. K. (2013). Factors affecting citation rates in environmental science. Journal of Informetrics, 7(2), 265–271.

    Article  Google Scholar 

  • Vintzileos, A. M., & Ananth, C. V. (2010). How to write and publish an original research article. American Journal of Obstetrics and Gynecology, 202(4), 344.e341–344.e346.

    Article  Google Scholar 

  • Vygotsky, L. (1962). Thought and language. Cambridge, MA: MIT Press.

    Book  Google Scholar 

  • Wallmark, J. T., Eckerstein, S., Langered, B., & Holmqvist, H. E. S. (1973). Increase in efficiency with size of research teams. IEEE Transactions on Engineering Management, EM-20(3), 80–86.

    Article  Google Scholar 

  • Walters, G. D. (2006). Predicting subsequent citations to articles published in twelve crime-psychology journals: Author impact versus journal impact. Scientometrics, 69(3), 499–510.

    Article  Google Scholar 

  • Webster, G. D., Jonason, P. K., & Schember, T. O. (2009). Hot topics and popular papers in evolutionary psychology: Analyses of title words and citation counts in evolution and human behavior, 1979–2008. Evolutionary Psychology, 7(3), 348–362.

    Article  Google Scholar 

  • Winston, C. (1985). Conceptual developments in the economics of transportation: An interpretive survey. Journal of Economic Literature, 23(1), 57–94.

    Google Scholar 

  • Woelert, P. (2013). The ‘economy of memory’: Publications, citations, and the paradox of effective research governance. Minerva, 51(3), 341–362. https://doi.org/10.1007/s11024-013-9232-z.

    Article  Google Scholar 

  • Wuchty, S., Jones, B. F., & Uzzi, B. (2007). The increasing dominance of teams in production of knowledge. Science, 316(5827), 1036–1039. https://doi.org/10.1126/science.1136099.

    Article  Google Scholar 

  • Yitzhaki, M. (2002). Relation of the title length of a journal article to the length of the article. Scientometrics, 54(3), 435–447.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Thor-Erik Sandberg Hanssen.

Appendix: Deducing some mathematical properties of the model used

Appendix: Deducing some mathematical properties of the model used

It follows from Eq. (2) that the expressions for the first and second derivatives are \(\frac{{\partial {\text{CI}}}}{\partial i} = \beta_{i} \cdot {\text{CI}} \cdot i^{{\tau_{i} - 1}}\), \(\frac{{\partial {\text{CI}}}}{\partial j} = \beta_{j} {\text{CI}}\), \(\frac{{\partial^{2} {\text{CI}}}}{{\partial i^{2} }} = \beta_{i} \cdot {\text{CI}} \cdot i^{{\tau_{i} - 2}} \left[ {\beta_{i} \cdot i^{{\tau_{i} }} + \tau_{i} - 1} \right]\) and \(\frac{{\partial^{2} {\text{CI}}}}{{\partial j^{2} }} = \beta_{j}^{2} {\text{CI}}\), \(i = \left\{ {{\text{PI}},{\text{PP}},{\text{RF}}} \right\},\;j = \left\{ {{\text{AN}},{\text{YP}},{\text{WT}}} \right\}\). When \(0 < \tau_{i} < 1\) it implies that \(\frac{{\partial {\text{CI}}}}{\partial i},\frac{{\partial {\text{CI}}}}{\partial j} \ge \left( < \right) 0\) when \(\beta_{i} ,\beta_{j} \ge \left( < \right)0\), \(\frac{{\partial^{2} {\text{CI}}}}{{\partial i^{2} }} > 0\) when \(\beta_{i} < {0\,{\text{or}}\,\beta_{i } } > 0\;{\text{and}}\,i > \left[ {\frac{{1 - \tau_{i} }}{{\beta_{i} }}} \right]^{{1/\tau_{i} }} , \frac{{\partial^{2} {\text{CI}}}}{{\partial i^{2} }} < 0\) otherwise. When \(\tau_{\text{PI}}\) and \(\tau_{\text{PP}}\) are set to the stated values in Eq. (2). \(\frac{{\partial^{2} {\text{CI}}}}{{\partial {\text{PI}}^{2} }},\frac{{\partial^{2} {\text{CI}}}}{{\partial {\text{PP}}^{2} }} < 0\) when PI < 278 and PP < 690, whilst \(\frac{{\partial^{2} {\text{CI}}}}{{\partial {\text{RF}}^{2} }} < 0\) when RF < 2.54. Moreover, \(\frac{{\partial^{2} {\text{CI}}}}{{\partial j^{2} }} > 0\) for all values of \(\beta_{j}\). The derivatives show that neither are constant but rather vary with the values of \(i\) and j. The absolute change in \({\text{CI}}\) when one variable changes by one unit is thus dependent on the values of all explanatory variables in the model.

To obtain a good interpretation of the \(\beta\)-values, we can focus on the elasticity values. The above derivatives imply that \({\text{EL}}_{i} {\text{CI}} = \frac{{\partial {\text{CI}}}}{\partial i}\frac{i}{\text{CI}} = \beta_{i} \cdot i^{{\tau_{i} }}\) and \({\text{EL}}_{j} {\text{CI}} = \frac{{\partial {\text{CI}}}}{\partial j}\frac{j}{\text{CI}} = \beta_{j} \cdot j\) where \({\text{EL}}_{i} {\text{CI}}\) and \({\text{EL}}_{j} {\text{CI}}\) denote the elasticity of \({\text{CI}}\) with respect to \(i\) and \(j\), respectively. Hence, the absolute values of the elasticities of \({\text{CI}}\) with respect to \(i, i = \left\{ {{\text{PI}},{\text{PP}},{\text{RF}}} \right\}\) increase concavely with i, whereas the elasticities of \({\text{CI}}\) with respect to \(j, j = \left\{ {{\text{AN}},{\text{YP}}, \ldots } \right\}\) increase proportionally with \(j\).

From the above derivatives it also follows that \(\left( {\frac{{\frac{{\partial {\text{CI}}}}{\partial i}}}{\text{CI}}} \right) = \beta \cdot i^{{\tau_{i} - 1}}\) and \(\left( {\frac{{\frac{{\partial {\text{CI}}}}{\partial j}}}{\text{CI}}} \right) = \beta_{j} .\) Thus, an increase in i by one unit will always increase \({\text{CI}}\) by \(\left( {100 \cdot i^{\tau_{i} - 1} \cdot \beta_{i} } \right) \%\), while an increase in \(j\) by one unit will always increase \({\text{CI}}\) by approximately (\(100 \cdot \beta_{j} )\)%. Our model specification implies that the relative changes in \({\text{CI}}\) when \(i\) and j change by one unit, are independent of the values of the other explanatory variables. Moreover, the relative changes in j, \(j = \left\{ {{\text{AN}},{\text{YP}},{\text{WT}}} \right\}\) are independent of their own values whereas the relative changes in i, \(i = \left\{ {{\text{PI}},{\text{PP}},{\text{RF}}} \right\}\) decrease with their own values.

Equation (2) implicitly defines \({\text{PP}}\) as a function of \({\text{PI}}\) for given values of \({\text{CI}} ({\text{CI}}^{*} )\) and of the other explanatory variables (\({\text{AN}}^{*} ,{\text{RF}}^{*} ,{\text{WT}}^{*} ,{\text{YP}}^{*} ,{\text{TR}}^{*} ,{\text{TA}}^{*} ,{\text{TB}}^{*} ,{\text{TS}}^{*} )\); that is

$${\text{PP}} = F\left( {\text{PI}} \right) =\left[\frac{{\ln {\text{CI}} - \beta_{PI} {\text{PI}}^{{\tau_{\text{PI}} }} - q}}{{\beta_{\text{PP}} }}\right]^{{1/\tau_{\text{PP}} }}$$

in which q = \(\beta_{0} + \beta_{\text{RF}} {\text{RF}}^{0.37*} + \beta_{\text{AN}} {\text{AN}}^{*} + \beta_{\text{WT}} {\text{WT}}^{*} + \beta_{\text{YP}} {\text{YP}}^{*} + \beta_{\text{TR}} {\text{TR}}^{*} + \beta_{\text{PA}} {\text{PA}}^{*} +\, \beta_{\text{PB}} {\text{PB}}^{*} + \beta_{\text{TS}} {\text{TS}}^{*}\)

From the F(PI) relationship above we can deduce the marginal rate of substitution \(\left( {\text{MRT}} \right)\) between \({\text{PP}}\) and \({\text{PI}}\):

$${\text{MRT}} = \frac{{\partial {\text{PP}}}}{{\partial {\text{PI}}}} = - \frac{{\beta_{\text{PI}} }}{{\beta_{\text{PP}} }} \cdot \frac{{\tau_{\text{PI}} }}{{\tau_{\text{PP}} }} \cdot \frac{{{\text{PP}}^{{1 - \tau_{\text{PP}} }} }}{{{\text{PI}}^{{1 - \tau_{\text{PI}} }} }}$$

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hanssen, TE.S., Jørgensen, F. & Larsen, B. The relation between the quality of research, researchers’ experience, and their academic environment. Scientometrics 114, 933–950 (2018). https://doi.org/10.1007/s11192-017-2580-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-017-2580-y

Keywords

Navigation