Abstract
In the Italian context of research evaluation, research output in the Social Sciences and Humanities (SSH) presents problematic issues in classification, particularly for the Humanities. In the case of the Italian context, in fact, the SSH area features an extremely varied spectrum of disciplinary domains. The Italian national evaluation system for the Humanities applies a system in which research results are classified in correspondingly varied manner, based on the disciplinary affiliation of the researcher. This has problematic consequences in terms of increasing the numbers of items included in the classification, and still more severe ones concerning the quality criteria to be applied to each item. A particularly critical issue is that the criteria vary depending on the disciplinary domain under evaluation.
Beginning from an illustration of the problems specific to the Italian context, the chapter proceeds to a comparative analysis of the research quality criteria for books in the SSH area as applied in several countries and international contexts. The analysis adopts a methodological approach that can be applied to assessment of other national and international classification systems, leading towards a shared European response to the problem of evaluation, in particular for books and book parts. The investigation articulates the dimensions of quality identified in both literature and practice, and offers a structured framework for analysis of the role of books in the evaluation process.
The discourse is based on the case of the Italian SSH disciplinary domains and national evaluation exercise, which can be broadened to international and other national SSH research domains and, although the chapter deals with overall SSH, the discussion focuses on the Humanities. The Italian experience in this area is particularly strong, given the uniquely long history of the research area and its ramified development. The sources selected for analysis are the international ISO standards, national guidelines from Spain, Australia and New Zealand, and Italian academic classifications from the University of Turin and University of Bologna. The comparison is based on the classification of books and monographs as practiced under Italian national guidelines for research assessment.
This is a preview of subscription content, log in via an institution to check access.
Notes
- 1.
Research and experimental development (R&D) comprise creative work undertaken on a systematic basis in order to increase the stock of knowledge, including knowledge of man, culture and society, and the use of this stock of knowledge to devise new applications.
- 2.
Up to 12 or 18 publications in non-bibliometric areas, depending on whether the candidate tendered for role of associate professor or full professor.
- 3.
As occurs, for example, in Norway with CRIStin. In Italy a first important step has been made the census of research products operated by SUA-RD and the obligation for each scholar to provide an ORCID identifier within the VQR 2011–2014.
- 4.
- 5.
The classes are: AA (= 1.5), A (= 1), B (= 0.6), C (= 0.3), D (= 0.05).
References
ANEP, FECYT. (2007). Criterios de calidad en la investigación en humanidades. http://www.idi.mineco.gob.es/stfls/MICINN/Investigacion/FICHEROS/2007-criterios-hh.pdf. Accessed 4 Sept 2016.
ANVUR. (2011). Valutazione della qualità della ricerca 2004–2010 (VQR 2004–2010): bando di partecipazione. http://www.anvur.org/attachments/article/122/bando_vqr_def_07_11.pdf. Accessed 4 Sept 2016.
ANVUR. (2013). Autovalutazione, valutazione e accreditamento del sistema universitario italiano. www.anvur.org/attachments/article/26/documento_finale_28_01_13.pdf. Accessed 9 Sept 2016.
ANVUR. (2014). Linee guida per la compilazione della SUA-RD. Allegato A: Ridefinizione delle tipologie di prodotti della ricerca. http://www.anvur.org/attachments/article/26/Linee%20Guida%20SUA_RD%20ALLEGATO%20.pdf. Accessed 9 Sept 2016.
ANVUR. (2015a). Valutazione della qualità della ricerca 2011–2014 (VQR 2011–2014): bando di partecipazione. http://www.anvur.org/attachments/article/825/Bando%20VQR%202011-2014_secon~.pdf. Accessed 9 Sept 2016.
ANVUR. (2015b). Funzionalità e utilità della SUA-RD. http://www.anvur.org/attachments/article/26/SUA-RD20150121_29_1_2015.pdf. Accessed 4 Sept 2016.
ANVUR. (2016a). Linee guida per la compilazione della Scheda Unica Annuale della Ricerca Dipartimentale 2014, 2015 e 2016. http://www.anvur.org/attachments/article/26/Linee_GuidaSUA_RD_2014-20~.pdf. Accessed 4 Sept 2016.
ANVUR. (2016b). Rapporto biennale sullo stato del sistema universitario e della ricerca 2016. http://www.anvur.it/attachments/article/1045/ANVUR_Rapporto_INTEGRALE_~.pdf. Accessed 4 Sept 2016.
Australian Department of Education and Training. (2015). Higher education research data collection: Specifications for the collection of 2014 data. http://docs.education.gov.au/documents/2015-higher-education-research-data-collection-specifications. Accessed 4 Sept 2016.
Basili, C. (2003). La diffusione dei risultati scientifici. In A. M. Scarda (Ed.), Rapporto sul sistema scientifico e tecnologico in Italia. Elementi per un’analisi (pp. 191–202). Milano: Angeli.
Bennett, S. (1998). Reprint: Just-in-time scholarly monographs. The Journal of Electronic Publishing, 4(1). doi:https://doi.org/10.3998/3336451.0004.103.
Broadus, R. N. (1971). The literature of the social sciences: A survey of citation studies. International Social Sciences Journal, 23, 236–243.
Capaccioni, A. (2014). La monografia scientifica e le sfide dell’accesso aperto. AIB Studi, 54, 201–211. doi:https://doi.org/10.2426/aibstudi-10084.
Chodorow, S. (1999). The pace of scholarship, the scholarly career, and the monograph: The once and future monograph. In M. C. Case, & M. M. Case (Eds.), The specialized scholarly monograph in crisis, or, how can I get tenure if you won’t publish my book? Proceedings of a conference sponsored by American Council of Learned Societies, Association of American University Presses, and Association of Research Libraries. Washington, DC, September 11–12, 1997. Association of Research Libraries, Washington, DC. http://old.arl.org/resources/pubs/specscholmono/Chodorow.shtml. Accessed 10 July 2016.
CIVR. (2007). Relazione finale VTR 2001–2003. http://vtr2006.cineca.it/php5/relazione_civr/output/totale.pdf. Accessed 4 Sept 2016.
Cronin, B., & La Barre, K. (2004). Mickey Mouse and Milton: Book publishing in the humanities. Learned Publishing, 17, 85–98.
Crossick, G. (2015). Monographs and open access: A report to HEFCE. http://www.hefce.ac.uk/pubs/rereports/year/2015/monographs/. Accessed 4 Sept 2016.
Crossick, G. (2016). Monographs and open access. Insights, 29, 14–19. doi:https://doi.org/10.1629/uksg.280.
CUN. (2013). Proposal «Criteri identificanti il carattere scientifico delle pubblicazioni e degli altri prodotti della ricerca». https://www.cun.it/uploads/4532/proposta_cun_criteri_scientificit%C3%A0.pdf ? Accessed 4 Sept 2016.
Darnton, R. (1999). A program for reviving the monograph. Perspectives on History, 37. https://www.historians.org/publications-and-directories/perspectives-on-history/march-1999/a-program-for-reviving-the-monograph. Accessed 4 Sept 2016.
Darnton, R. (2009). The case for books: Past, present, and future. New York: Public Affairs.
De Santis, I. (ed) (2006). Linee guida per la valutazione della ricerca. http://vtr2006.cineca.it/documenti/linee_guida.pdf. Accessed 9 Sept 2016.
de Solla Price, D. J. (1970). Citation measures of hard science, soft science, technology, and nonscience. In C. E. Nelson & D. K. Pollock (Eds.), Communication among scientists and engineers (pp. 3–22). Lexington: Heath Lexington Books.
Elliott, M. A. (2015). The future of the monograph in the digital era: A report to the Andrew W. Mellon foundation. The Journal of Electronic Publishing, 18(4). doi: https://doi.org/10.3998/3336451.0018.407.
EU High Level Expert Group. (ed) (2015). Commitment and Coherence: Essential ingredients for success in science and innovation: Ex-Post-evaluation of the 7th EU framework programme (2007–2013). http://ec.europa.eu/research/evaluations/pdf/fp7_final_evaluation_expert_group_report.pdf. Accessed 4 Sept 2016.
Gimenez-Toledo, E., Mañana-Rodriguez, J., & Engels, T. et al. (2015). The evaluation of scholarly books as research output. Current developments in Europe. In A. A. Salah, Y. Tonta, A. A. Akdag Salah, C. Sugimoto, & U. Al (Eds.), Proceedings of the 15th International Society for Scientometrics and Informetrics Conference, Istanbul, Turkey, 29th June to 4th July, 2015. Bogazici University, Instambul, pp. 469–476. http://curis.ku.dk/ws/files/141056396/Giminez_Toledo_etal.pdf.
Giménez-Toledo, E., Mañana-Rodríguez, J., Engels, T., et al. (2016). Taking scholarly books into account: Current developments in five European countries. Scientometrics, 107, 685–699.
Hicks, D. (2004). The four literatures of social science. In H. Moed, W. Glanzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 473–496). Dordrecht: Kluwer Academic Publisher. http://works.bepress.com/diana_hicks/16/.
Hicks, D. (2006). The dangers of partial bibliometric evaluation in the social sciences. Economia Politica, 23(2), 145–162. https://doi.org/10.1428/22461.
Huang, M., & Chang, Y. (2008). Characteristics of research output in social sciences and humanities: From a research evaluation perspective. Journal of the American Society for Information Science and Technology, 59(11), 1819–1828. https://doi.org/10.1002/asi.20885.
IFLA. (2011). ISBD: International standard bibliographic description, consolidated edition. http://www.ifla.org/files/assets/cataloguing/isbd/isbd-cons_20110321.pdf. Accessed 4 Sept 2016.
International Organization for Standardization. (2001). ISO 5127: Information and documentation – Vocabulary. Geneva: ISO.
International Organization for Standardization. (2004). ISO 12615: Bibliographic references and source identifiers for terminology work. Geneva: ISO.
International Organization for Standardization. (2005). ISO 2108: Information and documentation – International standard book number (ISBN). Geneva: ISO.
International Organization for Standardization. (2014). ISO/DIS 5127: Information and documentation – Foundation and vocabulary. Geneva: ISO.
Joseph, R. P. (2015). Higher education book publishing, from print to digital: A review of the literature. Publishing Research Quarterly, 31, 264–274. https://doi.org/10.1007/s12109-015-9429-0.
Ochsner, M., Hug, S. E., & Daniel, H. D. (2016). Humanities scholars’ conceptions of research quality. In M. Ochsner, S. E. Hug, & H. D. Daniel (Eds.), Research assessment in the humanities: Towards criteria and procedures (pp. 43–69). Cham: Springer Open. https://doi.org/10.1007/978-3-319-29016-4.
OECD. (2002). Frascati manual: Proposed standard practice for surveys on research and experimental development. Paris: OECD. http://www.oecd-ilibrary.org/science-and-technology/frascati-manual-2002_9789264199040-en.
Tertiary Education Commission. (2013). Performance-based research fund: Quality evaluation guidelines 2012. https://www.tec.govt.nz/Documents/Publications/PBRF-Quality-Evaluation-Guidelines-2012.pdf
Thompson, J. W. (2002). The death of the scholarly monograph in the humanities? Citation patterns in literary scholarship. Libri, 52, 121–136.
University of Bologna. (2013). Definizione e principali criteri di valutazione dei prodotti della ricerca. http://www.catalogo-ugov.unibo.it/. Accessed 4 Sept 2016.
University of Turin. (2012). Rapporto sull’attribuzione dei pesi tonali ai prodotti della ricerca (Rev. 04). Accessed 4 Sept 2016.
University of Turin. (2013). Nuove tipologie U-GOV catalogo (rev. 03). Accessed 4 Sept 2016.
Williams, P., Stevenson, I., Nicholas, D., et al. (2009). The role and future of the monograph in arts & humanities research. ASLIB Proceedings, 61, 67–82.
Willinsky, J. (2009). Toward the design of an open monograph press. The Journal of Electronic Publishing, 12(1). http://quod.lib.umich.edu/j/jep/3336451.0012.103?view=text;rgn=main.
Zitt, M., & Bassecoulard, E. (2008). Challenges for scientometric indicators: Data demining, knowledge-flow measurements and diversity issues. Ethics in Science and Environmental Politics, 8(1), 49–60. https://doi.org/10.3354/esep00092.
Zuccala, A. A., Verleysen, F. T., Cornacchia, R., & Engels, T. C. E. (2015). Altmetrics for the humanities: Comparing Goodreads reader ratings with citations to history books. Aslib Journal of Information Management, 67, 320–336.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this chapter
Cite this chapter
Basili, C., Lanzillo, L. (2018). Research Quality Criteria in the Evaluation of Books. In: Bonaccorsi, A. (eds) The Evaluation of Research in Social Sciences and Humanities. Springer, Cham. https://doi.org/10.1007/978-3-319-68554-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-68554-0_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68553-3
Online ISBN: 978-3-319-68554-0
eBook Packages: Social SciencesSocial Sciences (R0)