Send message to Authors

Short abstract:

We present a framework for designing and conducting project-accompanying formative evaluation of impact-oriented transdisciplinary research in real-world laboratories. Referring to examples from two research projects we illustrate the application of our evaluation design.

Long abstract:

Scientific literature provides various approaches to conceptualize and assess the societal impacts of transdisciplinary research. However, there remains a notable gap in strategies to foster impact orientation of real-world laboratories (RWLs) and evaluate their effectiveness. In this paper session, we propose a framework for designing and conducting formative evaluations of RWLs. Our evaluation approach aims to systematically document achieved effects and shape the strategic direction of RWLs within project management.

By promoting high quality of the transdisciplinary research process and using Theory of Change (ToC) to formulate and reflect on desired effects, our evaluation seeks to establish impact orientation from the beginning of the project. We clarify assumptions about impact pathways and foster a shared understanding among team members regarding the interplay between technical and social elements. Our set of methods encompasses steps from developing project-specific ToCs to continuous data collection and reflection on evaluation outcomes.

We illustrate the application of our evaluation approach with examples from two RWLs that we have accompanied as evaluators. The “Biodiversity Valuing & Valuation” project brings together science and business to develop tools for effective biodiversity management. The “UCKER Warentakt” project aims to introduce new logistics services to strengthen local public transport, reduce car traffic, and improve the supply of goods in a rural area in northeast Germany.

Formative evaluation presents challenges for project participants, including categorizing outputs and effects and understanding their significance for project success. Additionally, we address limitations of our approach, including data quality and the feasibility of data collection.