About the Author... James Moses Omara-Ogwang is a Compliance Engineer and acting Manager of Technical Regulation at the Electricity Regulatory Authority of Uganda. His work entails ensuring, through inspections, that Licensees in the electricity sector abide by their license obligations, observe applicable standards, and operate according to good professional practice. He also advises investors intending to join the electricity sector in Uganda. Mr. Omara is an IP3 alumnus.

Establishing a Benchmarking System: Achievements and Lessons Learned from the Uganda Electricity Sector James Moses Omara-Ogwang Abstract The electricity sector in Uganda was unbundled in 2001 by passing into law a new Electricity Act in 1999. This Act also provided for the formation of the Electricity Regulatory Authority (ERA) in 2001. The Act provided for ERA's functions, which were premised on regulating the newly restructured electricity sector. Because of this, ERA had to develop key performance indicators to be used in a benchmarking system as part of the regulatory process. This involved undergoing a protracted consultation process with the key stakeholders in the electricity sector.

I. Introduction

"Agreements get better results than Arguments." (Anonymous)

The Electricity Act, 1999¹ , besides liberalising and privatising the electricity sector as its main thrust, also provided for the formation of the Electricity Regulatory Authority (ERA) in Uganda. As a consequence ERA was formed in 2000, and the then vertically integrated electricity utility, Uganda Electricity Board (UEB), was unbundled into independent generation, transmission and distribution companies in 2001. The Act provided for the functions of ERA, one of which is:

"To develop and enforce performance standards for the generation, transmission and distribution of electricity" (The Electricity Act, 1999, S11 (i))

Because of this mandate ERA resolved to develop a benchmarking system that could be uniformly applied in the electricity sector. This article attempts to summarise the experience ERA has gone through to establish the benchmarking system up to the point where it is on the verge of being launched, as well as the lessons learned.

The Beginning

ERA did not go straight into preparing and rolling out the benchmarking system but established a Reporting Schedule (RS) in 2003 that contained a set of reporting requirements for Licensees² . This was a template through which the Licensees could report on their licensed activities and was supposed to be filled in and submitted quarterly.

The RS enabled ERA to keep track of the licensed activities of the Licensees through establishment of trends. It did not incorporate a system of measuring their performance.

It is important to point out that prior to the issuing out of the RS's, ERA was still receiving reports from the Licensees though in an un-coordinated manner. The RS's were meant to provide a uniform format which the Licensees could follow thus making it easier for ERA to extract the required information.

ERA subsequently agreed with the Licensees that, as a means of making the RS's more meaningful and useful to all the stakeholders, a set of key performance indicators (KPI) should be drawn up, which would be worked out from the data coming through the RS's. ERA could then use these indicators to measure the performance of the Licensees, and even publish them publicly.

The Consultation Process for the Reporting Schedules (RS's) and Key Performance Indicators (KPI)

Preparing the Reporting Schedules

Prior to the launch of the RS's and KPI's there were a series of consultation processes that ERA had to go through with the Licensees, and also the wider public³ this time.

At the first consultation level, ERA engaged a consultant to draw up a template that would eventually be developed into the RS. To achieve this, the consultant had to have an idea of what kind of information ERA wanted to obtain from the Licensees as well as the information that the Licensees actually had. It also had to incorporate technical, commercial and financial information, which would be summarised to a level that made them not onerous to the Licensees but at the same time contained all the information that ERA required. This was not a simple balancing act!

The result was a set of RS's that ERA presented to the Licensees and the wider public in series of workshops at the second consultation level. These workshops had three aims:

• Receive ideas from the Licensees on how the schedules could be made more user-friendly.
• To agree on the final template to be used by the Licensees
• To establish the responsibility levels for preparing and submitting these reports

All parties agreed that it was important that an appropriate responsibility level be established for the RS's so that Licensees would take them seriously. It was therefore agreed that the top management in each organisation would take responsibility for having the RS templates filled in, endorsed and submitted to ERA in good time. I'm glad to note that this is still working well up to now.

After the workshops the RS's were agreed upon and adopted. It is also in one of these workshops (the last one!) that the idea of drawing up key performance indicators was flagged off. It was suggested that a list of benchmarks that could be applied to the Ugandan electricity sector be drawn up and these benchmarks would be worked out from the data provided in the RS's. Note that the Reporting Schedules have not been discarded at all! Instead they are being refined to enable better and more accurate capture of data.

Drawing up the Indicators

At the third consultation level in 2004, ERA engaged a consultant to examine the RS as well as other internationally accepted indicators and come up with a set of KPIs that could be incorporated into the benchmarking system we intended to adopt. Here again the consultant had to perform a serious balancing act! The following issues had to be dealt with:

• The KPI's had to be worked out from the already existing RS's
• These KPI's had to have a "local flavour" yet be internationally competent
• The KPI's had to be few but enough to provide a fair measure of the performance of the Licensees

The consultant proposed a set of 45 indicators from 10 "measurement areas" of the electricity sector. These were increased to 62 after adding indicators that were "peculiar" to the local situation but which could not be benchmarked internationally. Stakeholders debated the initial list of KPI's in a workshop at the fourth consultation level. The result of this debate was a final list of 53 indicators, which included commercial, technical and financial indicators. This number is not fixed and is subject to change after revising the benchmarking system in future!

Subsequently stakeholders agreed that the financial reporting as it stood in the RS was sufficient for monitoring the performance of the Licensees hence the financial indicators were dropped from the list of indicators drawn up. This cut down the number of indicators to 38 consisting of: 27 for the distribution companies; 9 for the transmission companies; and 2 for the generation companies. These KPI's would be worked out from the data that the companies presented in the RS's and published in an agreed format.

These KPI's together with the RS's that provide the data for working them out have now constituted ERA's benchmarking system⁴ .

Lessons Learned

The entire process of developing first, the Reporting Schedules, then the Performance Indicators has provided some important learning points that are worth sharing.

1. One does not have to go straight into applying KPI's but can progress slowly from an acceptable reporting system. It is always important to have it at the back of our minds that while the KPI's are important and useful they are not an end to themselves. Other means of tracking performance can be equally useful.
2. Operators do not necessarily want internationally accepted indicators to benchmark against. As a matter of fact most would prefer home grown indicators that they can easily identify with.
3. It is important to agree on an acceptable number of indicators with the target stakeholders. There is a benchmark of the acceptable number of indicators but this number can be varied depending on the system being monitored and what the stakeholders find acceptable.
4. The target stakeholders need to be consulted at each and every stage to make the indicators acceptable to them.
5. The RS's were drawn mainly with the major players in the Ugandan electricity sector in mind. There are, however, some smaller players (mini-hydro plants) in existence, and a few others are in the pipeline. These have expressed a concern that the RS's are too big and complex for their small operations. While bearing in mind the negative effect of proliferating RS's, their concerns are being addressed.
6. In introducing KPI's, stakeholders should not be burdened with reporting requirements. "Reporting fatigue" can easily lead to stakeholders furnishing false data hence rendering the entire benchmarking system useless!

III.

¹ The full Act can be found at ERA's website, http://www.era.or.ug .

² Licensees are companies that have been granted licences by ERA to operate in the electricity sector at generation, transmission, and distribution levels of the main grid, or to operate isolated grids.

³ These were selected from key companies, institutions and individuals that have a role to play in the electricity sector, or have keen interest in the business of the sector.

⁴This kind of neatly solved the vexing problem of how to gather useful data consistently for the benchmarking system!

Copyright 2006© Institute for Public-Private Partnerships, Inc. All rights reserved