New research from the Department of Accounting offers insights into the digital transformation of the public sector by investigating the influence of digitalization on the process of benchmarking and, in particular, the peer selection process. The use of information technology is extensively embedded in the public sector bringing efficiency and speed to the delivery of public services. However, the use of information technology also raises concerns—ranging from data security to privacy breaches and algorithmic governance.

Based on a case study conducted by Johan Graaf, Kalle Kraus and Wai-Fong Chua at the Department of Accounting, new research investigates the impact of a digital database and benchmarking device – Kolada, used to compare the performance of Swedish municipalities. Tracing the introduction, development, and use of the digital database, the study moves beyond a focus on “technical problems” and shows how digitalized benchmarking was not a politically neutral or asocial process.

The emergence of benchmarking has followed the rise of new public management and the “modernization” of the public sector, similar to many other management control techniques. Consequently, different types of public sector organizations have seen the adoption of benchmarking, often in response to a lack of generally accepted evaluation criteria for operations. Such relative evaluations have been assumed to reduce uncertainty regarding an organization’s quality, productivity and efficiency, thereby enabling decision makers to implement more focused strategies for improving organizational performance.

One of the key impediments to benchmarking of the Swedish municipalities in the past was that information was collected, arranged, and made available by over 50 different organizations (e.g., various governmental agencies). Following the introduction of Kolada, all Swedish municipal statistics are held in one database, irrespective of their source. The database contains thousands of different performance indicators for hundreds of municipalities, all of which can be used to quickly compare and rank any selection of municipalities. The Comparer function of the Kolada guides users by dividing a municipality into different service areas, to then display a selection of indicators for comparison.

For quality controllers, working in central administrative departments of municipalities, Kolada enabled a more abstract, acontextual form of algorithmically supported benchmarking that was quickly performed from office desks. The quality controllers viewed Kolada’s comparative options as enabling a more fact-based, “objective” choice of comparative peers, which aimed to appeal to a broad set of decision makers. However, politicians and practitioners “on the ground,” in comparison, were interested in detailing what peers (as competitors and/or role models) were doing and how they were improving their services. Being interested in “applied science”, Kolada’s outputs—which lacked localized detail—failed to persuade the practitioners. Pure science was too generic, too decontextualized, and the more singular forms of comparability generated by Kolada’s digital algorithms were considered less important, both cognitively and affectively, than “imperfect” and localized forms of comparison with neighbours that had constituted past practices.

The findings indicate that choosing peers that “matter” is a critical decision due to the affective ties in the benchmarking process. The benchmarking peers that mattered over time would turn into role models, collaborators, competitors, and much more, and this engaged a broad range of stakeholders and associated emotions. The risk in employing benchmarking without affective ties to those who are selected as “peers” is that attention is put on the indicator values and average scores rather than on the learning process.

Furthermore, the study suggests that flexibility and freedom in the peer selection process can counteract the some of the more problematic features of benchmarking (e.g., competitiveness and gaming behaviour). Different peer groups are associated with different legacies and afford different responses to the indicators. Such flexibility in peer selection could thus aid public sector organizations to accommodate the many different demands and needs of their stakeholders and create some independence from a single perspective on performance.

Access the research article here

Researchers

Johan Graaf

Department of Accounting, Stockholm School of Economics

Kalle Kraus

Department of Accounting, Stockholm School of Economics

Wai Fong Chua

Department of Accounting, Stockholm School of Economics

Discipline of Accounting, The University of Sydney