Paper short abstract:

This paper applies a mixed-methods approach to evaluate Indonesia’s social forestry programme. Combining quantitative quasi-experimental methods with qualitative realist synthesis, we highlight challenges and strengths of integrating methods, and its value at the science-policy interface.

Paper long abstract:

Current impact evaluations designed to inform policies for better environmental and social outcomes often focus on quantifying the effects of policy interventions. However, qualitative methods—despite their value in systematically exploring causal processes and addressing the perspectives and underlying assumptions of different actors—are less commonly integrated into evaluation designs. Combining quantitative and qualitative methods in impact evaluations offers a broader and more diverse understanding of impacts, enriching the evidence base and enhancing relevance at the science-policy interface.

In our paper, we reflect on the application of a mixed-methods approach to assess the impact of Indonesia’s social forestry programme-a nation-wide initiative aiming to improve the sustainable management of forests, alleviate poverty, and prevent land conflicts. Quantitative methods are used to measure the impact of social forestry on deforestation and the well-being of local communities. We combine this method with a realist synthesis method to address the mechanisms through which social forestry is contributing, or hindering, its intended goals, and further investigate the context through which initiatives succeed or fail, and for whom.

We discuss some of the challenges of integrating qualitative and quantitative methods, particularly the paradigmatic tensions; and the difficulty of reconciling spatial scales when evaluating impacts. Furthermore, we highlight the strengths of a mixed methods approach in foregrounding the importance of context, rigorously interrogating the question of why an intervention has or hasn’t worked and moving beyond the narrow focus on measuring ‘how much’.