Short Abstract

Evaluating sustainability is complex and lacks regularity and depth beyond ‘results sustained’. The UKES award winning EAGI approach recently informed the baseline evaluation design for the FCDO BRAVE programme. It was adapted into a scorecard aiming to support a robust sustainability assessment.

Description

Evaluating sustainability is a complex issue that lacks regularity and depth beyond an assessment of ‘results sustained’. This session will present lessons from the design and application of an innovative sustainability scorecard on a large scale and multi component resilience programme in Pakistan (FCDO Building Resilience and Addressing Vulnerability to Emergencies or BRAVE).

This session offers to explore how innovative concepts can strengthen evaluations, particularly how practitioners address the important assessment of programme sustainability.

We will first present the scorecard, starting with presenting the complexity and system-thinking theories in which it is grounded. The scorecard was directly informed by the core arguments presented in the UKES award winning Environmental Approach for Generational Impact (EAGI) Working Paper. Structured as a rubric with tailored progress markers, the scorecard captures progress across three levels (initially defined in the working paper): from programme objectives, through systemic change and toward generational impact.

In preparation for BRAVE’s baseline evaluation, the EAGI author and Evaluation Lead worked together to embed tailored lines of inquiry into Third-Party Monitoring (TPM) tools, beneficiary feedback surveys, and Key Informant Interview (KII) guides. Following data collection and analysis, the EAGI author (also Scorecard lead advisor on the programme) conducted an initial baseline assessment using TPM and evaluation evidence. This baseline provided an early view of how sustainability considerations have been integrated into programme design and delivery. In future evaluations, the BRAVE MEL team will continue to track progress against the three scorecard levels. In parallel, the Scorecard is being used to inform a robust assessment of BRAVE’s contribution to transformational change, as required under its climate finance mandate.

This session will provide an overview of the EAGI approach and demonstrate how it was adapted into a programme-relevant scorecard. Specifically, the presenters will explain what design decisions were made to align the scorecard to the needs of the programme, namely the decision to use rubrics, with programme specific progress markers integrating language from the BRAVE’s Theory of Change and the ICF KPI 15 on the likelihood of achieving transformational change. Presenters will close the presentation by sharing lessons learned from translating a conceptual framework into a theory-based, mixed-method evaluation tool focused on climate resilience.