Short Abstract

How can we better evaluate the implementation of narrative CVs for researchers? Panel contributions will cover: metadata and surveys to assess implementation; NCV textual analysis; and effects on review panel behaviours.

Long Abstract

In a context of continued momentum for improving research cultures, and reforming research assessment as one route to achieving this, many funders and institutions are piloting or implementing Narrative CVs (NCVs). In parallel to their emergence as a key approach in research assessment reform, there is increased interest in studying NCV implementation, from evaluating implementation processes and user perceptions to understanding behavioural changes and longer-term impacts.

In this session, each discussant will use a 10-min opening statement to explore one dimension of the evaluation of NCV implementation:

- Miriam Kip (BIH QUEST Center at Charité – Universitätsmedizin Berlin) - Identifying the metadata and processes needed to assess NCV implementation and early signs of change

- Shin-Ren Pan (KU Leuven) - Measuring researcher priorities in NCVs using Relative Topic Attention (RTA)

- Anne Jorstad (SNSF) - The impact of NCVs on peer review

- Judit Varga (Leiden University) - Studying effects on review panel behaviours of introducing NCVs

In conversation with the audience, and facilitated by Karen Stroobants (CultureBase) and Wolfgang Kaltenbrunner (Leiden University), the panel will then engage in a reflective discussion on some of the contextual and methodological challenges associated with studying NCVs and their implementation into institutional workflows. It will also touch on learnings from tested study designs, opportunities for further exploration of longer-term impacts such as behavioural changes, and reflections on limitations to be aware of as we collectively built the NCV evidence base.