Systems Analysis in Evaluation: The unfulfilled promise

Abstract

Our paper addresses the question, why systemic approaches have only played a modest role in impact evaluations of in- novation and technology programmes so far and examines possible reasons for this shortcoming, as well as discussing solutions that could be offered to remedy the existing deficit. While the need for a systemic approach to evaluations has been stressed quite often, the methodological challenges and reasons for the lack of systemic evaluations in practice have to our knowledge not yet been addressed in a systematical manner. This contribution is conceptual in nature and based on a review of the research literature on the use of systemic approaches in evaluations of the impact of R&I policy programmes. The analysis shows that the use of systemic methods encounters both epistemological and institutional obstacles. Suggestions are made for the further development of the methodological repertoire by including suitable systemic approaches.