Paper short abstract:

How does the organisation of interdisciplinary evaluation panels influence the evaluation of specific disciplines in the social sciences? Our study of the European Research Council highlights potential conflicts and complementarities between epistemic styles and evaluation cultures.

Paper long abstract:

The European Research Council (ERC) is one of the most important research funding organisations in Europe. Every year, reviewing panels composed of researchers from various research fields gather together to decide which research projects should receive funding. ERC’s evaluation panels are largely interdisciplinary, and decisions to select a proposal are rarely made by panelists who belong to the same discipline as the proposal being evaluated. How does the organisation of interdisciplinary evaluation panels influence the evaluation of specific disciplines and what are the potential epistemic consequences of interdisciplinary panels in the social sciences? To answer this question, we draw on interviews conducted with ERC panelists, combined with the analysis of the composition of social science panels since the creation of the ERC in 2007. We first show that the composition of evaluation panels has largely changed over time to accommodate disciplinary differences, leading to the inclusion, regrouping and exclusion of specific disciplines. We then identify three types of tensions and compatibilities between disciplines, impacting the evaluation of different research quality criteria, and the significance attributed to specific disciplines within the panels. Ultimately, our study of the organisation of ERC evaluation panels highlights potential conflicts and complementarities between epistemic styles and evaluation cultures in the social sciences. It invites researchers and reviewers to be more mindful of these differences and proximities in order to promote more careful and respectful evaluation practices.