Send message to Author

Paper short abstract

This presentation demonstrates how developmental evaluation and a novel, complexity-aware VFI were combined to accommodate uncertainty, diverse forms of value, and evidence to enhance learning and adaptation in a complex, rapidly changing public-sector systems change initiative.

Paper long abstract

The Department of Health, Disability and Ageing in Australia is comprehensively reforming the aged care system, including investing $77 million in a networked stewardship approach. Such a dynamic and complex policy initiative creates challenges for evaluation that many evaluators and commissioners will be similarly grappling with. Recognising the need for an innovative and adaptive approach, the Australian Government commissioned a three-year developmental evaluation that included a Value for Money/Investment (VFI) component.

We will present how we combined developmental evaluation and complexity-aware VFI to bridge the gap between operational learning and adaptation and future policy decision-making. We will share how our approach accommodated uncertainty, integrated diverse forms of evidence and stakeholder views, and enhanced the capacity for learning and adaptation in complex and rapidly changing systems-change initiatives. This supports the conference sub-themes of ‘influencing policy and programme change through evaluation’ and ‘building evaluation cultures’ by demonstrating how iterative and deliberative evaluation cycles supported decision-making, learning, and the delivery of a complex and dynamic initiative.

We will pay particular attention to how we used participatory and user-centred methods to support reflection and learning, including in our rubric-based VFI approach, which was grounded in democratic deliberation. Our VFI evaluation used a novel approach to considering the diverse types of value created through complex and systems-change initiatives, including the potential value to create future systems change, which is often overlooked. We will also share the challenges and practical strategies we used to manage the resource and time demands of iterative cycles and deliberative processes, as well as the need for evaluator competencies in systems thinking, facilitation, and relational awareness. This will be a highly practical session, as well as presenting some new thinking and lessons learned about combining developmental and VFI approaches.

Send message to Authors

Paper short abstract

Our evaluation has focused on learning for the practical application to programme redesign as well as contribute learning to inform the FCDO's future humanitarian priorities in Yemen and enhance evidence on interventions in similar fragile contexts within the humanitarian-development nexus.

Paper long abstract

Title: How the Food Security Safety Net (FSSN) Evaluation in Yemen is Translating Evidence on Cash and Livelihoods Programmes into FCDO Policy and Programme Change

Our evaluation of the FSSN programme (2022–2026), examines whether the cash transfers and livelihoods support delivered by the Social Fund for Development (SFD), the BRIGHTLY consortium and World Food Programme (WFP) have reduced food insecurity and improved medium-term household resilience in Yemen. Our evaluation has focused on learning for the practical application to programme redesign; the mixed methods evaluation combines before and after Food Consumption Score (FCS) and Reduced Coping Strategy Index (rCSI) comparisons, contribution analysis, Total Cost to Transfer Ratio (TCTR) and value for money assessment. The qualitative element of the research aims to provide answers as to why interventions have been successful or not. We have adapted the evaluation to manage access constraints in the North and South of Yemen (De-facto Authority controlled areas and Internationally Recognised Government areas). We have explicitly designed the evaluation to focus on learning over accountability to inform cost-effective food security approaches amid Yemen’s evolving and volatile context. The evaluation will contribute learning to inform the Foreign, Commonwealth & Development Office’s (FCDO) future humanitarian priorities in Yemen and enhance evidence on interventions in similar fragile contexts within the humanitarian-development nexus.

Relevance to conference theme — Influencing policy and programme change through evaluation:

This session addresses Theme 1: “Influencing policy & programmes” directly by showcasing how an evaluation conducted in one of the world’s most volatile humanitarian contexts has been structured to: (a) generate evidence that speaks to decision windows of policy makers and donors (VfM of cash modalities; transition pathways from emergency cash to livelihoods); (b) barriers to uptake (data access limits from WFP, restricted qualitative research in DFA areas, operational and security constraints). We will demonstrate how method choices (contribution analysis where attribution is infeasible; paired sample FCS comparisons; TCTR) were selected to maximise policy relevance given contextual constraints and reflect on lessons about the scale and limits of influence in fragile settings. We will also discuss how we worked closely with the client to tailor the findings of the evaluation to ensure that the evaluation was providing answers that would help to directly inform policy decisions.

Objective of session and how it will be delivered (20-minute single slot):

We will present the evaluation’s redesign oriented evaluation questions and methodology, highlighting key areas where findings will feed directly into FCDO redesign decisions: targeting and inclusion criteria to better reach the most vulnerable; modality mix and transition design (which combinations of cash transfer types and livelihoods produce sustainable food security and for which groups); and finally value for money and delivery efficiency (TCTR and VfM analysis). Our presentation will be timely and framed around the uncertainty in the FCDO funding landscape and how similar humanitarian programmes in fragile contexts can ensure more effective approaches for the best cost-efficiency.