Exploring Unintended Consequences in STI Evaluations and Monitoring

Towards a Framework for Unintended Consequences and Its Use in Evaluation and Monitoring Exercises

Abstract

With the turn to mission orientation and transformative policies, STI funding organisations increasingly aim for impacts beyond the research sector. In consequence, in the STI evaluation community an intensive discussion has emerged on the nature of intended effects of research funding proposing new concepts and methods for modelling and measuring these intended effects. In contrast, the understanding of unintended effects – despite the recognition of their importance - has remained limited. This paper seeks to address this gap by exploring and consolidating the dispersed knowledge on unintended consequences by a multifaceted approach. We perform a systematic literature analysis focusing on the characteristics of unintended consequences and explore how unintended effects have been addressed in different evaluation guidelines and evaluation studies. Based on this review, we conclude that the diversity of unintended consequences is a major bottleneck to advance both theory and practice. To overcome this challenge, this paper develops a reflection tool consisting of guiding questions that can be used to navigate through the diversity of unintended consequences.