Implicated in the Indicator Game? An Experimental Debate

Abstract

The rise of new modes evaluating academic work has substantially changed institutions and cultures of knowledge production. This has been reflected and criticized in the literature in STS and beyond. For STS scholars, these debates (should) however have an even more specific dimension. Many of us are experts on aspects of these changes. But at the same time, we too are part of the processes we are analyzing, and often criticizing. To put it slightly provocatively, often we cannot avoid playing the very same game that we scrutinize. This creates tensions that many of us reflect on, and it certainly has created many implicit and explicit normative stances on how to deal with them. Yet it seems that so far there has been little room in our field to reflect on and exchange this particular kind of experience-based knowledge. There are many different ways to engage with the dynamics of evaluation, measurement and competition in contemporary academia, or to play what we refer to colloquially here as the “indicator game.” With this debate, we would like to give room to the expression and discussion of some of these ways. This text is the introduction and prompt to an experimental debate. We discuss the state of the academic discussion on the impact of indicator-based evaluation on academic organization, epistemic work and identities. We use insights from these debates to raise questions for how STS and STSers themselves deal with the indicator game. In conclusion, we summarize our contributors’ arguments and propose the concept of “evaluative inquiry” as a new way of representing the quality of STS work in evaluative contexts.