Short Abstract

We will present an initial set of universal metrics and KPIs that can be used in conjunction with SMART objectives, along with carefully specified rules and guidelines, for evaluating public innovation investments and facilitating comparability and benchmarking across programs..

Description

Policy and programme evaluations vary widely across geographies, scales, and sectors. National governments, regional authorities, and international organisations have each developed their own frameworks to assess the effectiveness and impact of interventions. It is broadly recognised that, for high-quality evaluations of policies and programmes, robust planning, measurement, and attribution principles must be established from the outset, combining operational practicality with internationally accepted standards.

In the United Kingdom, the combination of the Magenta Book and the Green Book prescribes consistent cost-benefit analysis and economic appraisal for public investment decisions, providing a coherent framework that could ensure appropriate evaluation and accountability for public programmes. Both books recommend the use of SMART (Specific, Measurable, Achievable, Relevant, and Time-bound) objectives, though they stop short of mandating them.

An extensive survey of innovation programmes across the EU, UK, US, Japan, and Singapore found that, although most initiatives align with strategic priorities, SMART objectives are rarely used at the outcome and impact stages. Instead, broad qualitative goals are often employed, with a recurring emphasis on outputs, while longer-term outcomes are less frequently defined and seldom quantified with targets.

Evaluation methodologies also vary considerably, ranging from quantitative approaches (surveys, administrative data analysis, econometric modelling) to qualitative methods (case studies, interviews, process tracing). Causal attribution and data granularity differ substantially, from project-level to portfolio- or policy-level measurement. These differences limit clarity and cross-programme comparability and learning.

To address these challenges, we propose the development of a common set of universal metrics/KPIs, forming the basis of a shared framework for assessment, reporting, and structured learning. Such metrics should be relevant, clearly defined, measurable, and practically collectable, capturing outputs and outcomes applicable across sectors, technologies, and organisational contexts. They should also support causal assessment, enabling evaluators to distinguish programme-driven effects from external factors, while incorporating granularity and normalisation to allow fair comparison across programmes of different sizes, durations, and funding scales.

When used alongside SMART objectives, these universal metrics could offer both consistency and specificity—the metrics enabling comparability and benchmarking across programmes, while SMART objectives anchor evaluation in measurable, time-bound targets. Proper normalisation and granularity further strengthen usability, ensuring evaluations remain meaningful across diverse contexts without obscuring essential differences in scale or scope.

This integrated framework enables policymakers and programme managers to define realistic objectives, track progress systematically, and communicate results transparently. It also supports robust ex-post evaluation and evidence-based learning, facilitating long-term understanding of programme effectiveness and impact. Attention should also be given to the inclusion of caveats relating to data quality, contexts, heterogeneity, measurement units, response rates and evaluation objectives.

Recognising this potential, we present an initial set of universal metrics/KPIs as a starting point for discussion and application across innovation programmes, aiming to improve consistency in measurement, monitoring, management, and evaluation. It is foreseen that consistency in definitions, context adjustment, awareness of scale effects, and recognition of time-lags or emergent outcomes will also be needed to enhance learning and maximise the societal value of public innovation investments.