Short Abstract

This session examines how two UNICEF Kosovo evaluations influenced organizational decisions and government policy dialogue. The lead evaluator and UNICEF's Evaluation Manager jointly trace the evaluation journey from design to impact, sharing practical lessons on maximizing evaluation utility.

Description

The session will focus upon how two evaluations commissioned by UNICEF Kosovo have contributed to shape decisions within UNICEF Kosovo, and notably, been used to influence UNICEF’s policy dialogue with the government of Kosovo. The two evaluations (which will be discussed in a combined way) are:

• Evaluation of the UNICEF Kosovo Programmatic Strategies for Leaving No One Behind: Municipality Approach – completed August 2024

• Evaluation of UNICEF Kosovo Investing in the Early Years Project (2023-2025) - ongoing

The session will be prepared and presented jointly by myself (Team Lead - consultant) who led both evaluations, and the Evaluation Manager at UNICEF Kosovo. It will focus on the ‘journey’ of designing , developing , reporting on and disseminating / communicating the evaluation.

First, the session will focus on the design and development of the report , showing how it was designed with intended users and uses in mind. It will refer to a Utilization-Focused Evaluation (Michael Quinn Patton). A visual timeline of the process will be used, with identification of key junctures in the process to optimise potential utility such as identifying and engaging primary intended users early, gaining feedback and ownership at key stages.

We shall then focus on the dissemination/ uptake process. The UNICEF Evaluation Manager will share the process (unseen by the Evaluator) and how it has been used to influence internal and government policy and programming, and any changes brought about within the subthemes. We will use the Evaluation Influence Framework (Mark, Henry & Julnes) to structure including results use (decisions based on findings), and influence (broader changes in thinking, discourse, policy). The presentation will consider in particular findings related to (likely) (i) collaborative and multisectoral planning, and (ii) integrating equity issues or ‘leaving no one behind’ within policy/ programming.

We will also share the facilitators/ obstacles for the uptake of messaging from the evaluation. We may organise these around:

• Organizational factors (UNICEF's evaluation culture, leadership support)

• Relational factors (trust with government, stakeholder buy-in)

• Technical factors (quality of evidence, clarity of recommendations)

• Contextual factors (political timing, policy windows)

• Design factors (participatory approaches, communication strategies)

We will also consider the "null hypothesis": What might have happened without these evaluations?

The session will represent co-learning between the evaluation report ‘developer’ and ‘user/s’.