logo

AGENZIA NAZIONALE DI VALUTAZIONE
DEL SISTEMA UNIVERSITARIO E DELLA RICERCA

Home > Pubblicazioni > Ricerca sulla valutazione

Pubblicazioni / Ricerca sulla valutazione

  • Nappi CA and Poggi G
    (2015) Rassegna Italiana di Valutazione , 59

    L’indicatore di voto standardizzato di dipartimento basato sull’esercizio VQR 2004 2010

  • Anfossi A, Ciolfi A, Costa F, Parisi G and Benedetto S
    (2016) Scientometrics

    Large-scale assessment of research outputs through a weighted combination of bibliometric indicators

    The paper describes a method to combine the information on the number of citations and the relevance of the publishing journal (as measured by the Impact Factor or similar impact indicators) of a publication to rank it with respect to the world scientific production in the specific subfield. The linear or non-linear combination of the two indicators is represented on the scatter plot of the papers in the specific subfield in order to immediately visualize the effect of a change in weights. The final rank of the papers is therefore obtained by partitioning the two-dimensional space through linear or higher order curves. The procedure is intuitive and versatile since it allows, after adjusting few parameters, an automatic and calibrated assessment at the level of the subfield. The derived evaluation is homogeneous among different scientific domains and can be used to address the quality of research at the departmental (or higher) levels of aggregation. We apply this method, that is designed to be feasible on a scale typical of a national evaluation exercise and to be effective in terms of cost and time, to some instances of the Thomson Reuters Web of Science database and discuss the results in view of what was done recently in Italy for the Evaluation of Research Quality exercise 2004-2010. We show how the main limitations of the bibliometric methodology used in that context can be easily overcome.

    DOI: 10.1007/s11192-016-1882-9

  • Bertocchi G, Gambardella A, Jappelli T, Nappi CA and Peracchi F
    (2015) Research Policy , 44(2): 451-466

    Bibliometric evaluation vs. informed peer review: Evidence from Italy

    A relevant question for the organization of large-scale research assessments is whether bibliometric evaluation and informed peer review yield similar results. In this paper, we draw on the experience of the panel that evaluated Italian research in Economics, Management and Statistics during the national assessment exercise (VQR) relative to the period 2004-2010. We exploit the unique opportunity of studying a sample of 590 journal articles randomly drawn from a population of 5681 journal articles (out of nearly 12,000 journal and non-journal publications), which the panel evaluated both by bibliometric analysis and by informed peer review. In the total sample we find fair to good agreement between informed peer review and bibliometric analysis and absence of statistical bias between the two. We then discuss the nature, implications, and limitations of this correlation. (C) 2014 Elsevier B.V. All rights reserved.

    DOI: 10.1016/j.respol.2014.08.004

Pubblicazioni