Short Abstract

Whole systems approaches and realist evaluation are positioned as antidotes to reductionist methods, due to their preoccupation with understanding the role of multiple layers of context and causal forces. Here, we communciate findings about how practitioners value and use them to inform their work.

Description

Pressing healthcare issues, and health inequalities are recognised as complex issues that are irreducible to their constitute parts. Appropriate evaluation within complex areas, including whole systems approaches and realist evaluation, have burgeoning credibility in their ability to account for learning and innovation across complex issues. However, deployment is often fraught with challenges and understanding how stakeholders become engaged in these approaches and integrate cycles of learning is lacking. Questions exist surrounding how and in what ways stakeholders react to this “participation” in complex congruent evaluation and how this evidence is valued and used. The aim of this research was to understand how large scale transformation of whole systems realist practice and evaluation occurs, for whom, and in what circumstances.

The National Evaluation and Learning Partnership, commissioned by Sport England, have worked collaboratively with a wide range of place partnerships engaged in whole system place-based approaches to tackle physical inactivity. The team have supported places to explore how Place Partnerships can build capacity to undertake appropriate evaluation. A focus of the work has been to substantially raise capability in whole systems realist evaluation. Drawing upon a bricolage of participatory evaluation methods, this approach has worked with places to appreciate importance of complexity, the conditions for change, and then enable them to operationalise realist informed evaluation methods. In this paper we reflect on the findings from 11 realist interviews with stakeholders who have been engaged in this place partnership journey to explore ideas on how, capability and consciousness may develop to inform everyday decision making and delivery.

Emerging results verify that places initially require an increased recognition on the need to accept uncertainty and alternative evaluation approaches. A prominent feature was the need for a senior leader who advocates for, supports, and facilitates change by “feeding the beast” of traditional ways of thinking whilst highlighting the need for broader ways of capturing impact. Another resource influencing change was the presence of a credible external voice who “fights the corner” of innovative ways of thinking. Findings indicate that once places understand complexity, they become ready to alter practices. The influence of funder expectations, engrained beliefs on evaluation as a performance metric, and the role of share social spaces for knowledge exchange prominent. Commissioned activities and external frameworks can be persuasive due to the competitive landscape, meaning organisations will conform to meet the funders requirements. However, in other instances without enforcing expectations some used the approach to embellish their work. The evolution of places to being reflexive with cycles of learning was not as discrete. This was often complicated by the various levels of the system and trying to influence multiple varying agendas. Often, this cross boundary work required “translation” which many within the system found alien.

Sustainable uptake of whole systems place based realist work is influenced by historical practices of evaluation, enduring beliefs about practice, the funding landscape, the provision of external support and social spaces, the wider stakeholder belief system, and the interplay of senior and middle management in discursive ways.