New paper: Measuring IT performance - what really matters
Category: current-news | Author: Craig Errey | Date: 14/10/2005
The problem of measurement
It’s easy to get caught up in the hype of easily quantifiable metrics — they sound impressive and certainly woo the paymasters with the giddy heights of teraflops, terabytes and pterodactyls — oh, wait, that’s a dinosaur — but, similarly, bigger isn’t better.
While IT is not a dinosaur - it is and will remain to be an integral part of business — IT managers who do not appreciate the organisational environment may certainly go that way. With billions of dollars spent globally on IT, we must quantify the return on investment.
IT performance has long been measured using statistics like transactions per hour, concurrent clients, FLOPS and other raw throughput figures — but it’s no good buying the capacity to handle 50,000 transactions per hour if your end users can only manage 5,000 because they find the new user interface too hard to use or your fulfilment processes are paper based!
IT, like any other expenditure in business, can be measured with the Balanced Scorecard. A simple, yet powerful measurement and management framework, it keeps us honest by specifying measurement in terms of Finances (e.g. profit and loss), Business processes (e.g. throughput, fulfilment, waste), Staff (e.g. retention, skill growth) and Customers (e.g. market share, revenue per customer). For IT, as for business, the old maxim stands: ‘if you can’t measure it, you can’t manage it’.
Using the Balanced Scorecard to create a measurement framework IT
Fortunately, the process is quite straightforward, and I’m going to cover the four simple steps briefly. You can refer to the further reading for more details.
Creating a Balanced Scorecard begins with assessing the organisation’s foundations in terms of market opportunities, competitors, the financial position and long and short term goals — that is, the organisation, not the specific IT project itself. These are important as funds need to be spent wisely in the context of real business needs, rather than ‘must haves’ — bankruptcy by technology is not a desirable option.
Second, you need to create the overall business strategy — here you specify what you expect will work and be successful in meeting the organisation’s performance indicators. This is followed by decomposing the strategy into a series of objectives that will contribute to their achievement.
So far there has been no discussion of any technology. Success in an organisation is a strategically aligned set of activities, spanning operations, marketing, finance, HR and other core areas. IT should be treated as a means to an end, providing the foundations to drive these various functions. This way, the deployment of IT is directly aligned with business activities and ultimately business performance. Rejecting the idea that IT is a panacea is a critical change in thinking for many IT managers.
The third step is to create a strategic map of the business strategy to causally connect the objectives and their specific activities. For example, a strong workforce contributes to better customer service, which in turn contributes to more referrals, which contributes to more sales at a cheaper cost, which contributes to greater profitability.
The final step is defining the set of performance measures based on the activities and the strategic map.
Relating IT to the business
The business activities ultimately decompose to people’s daily activities in the organisation, such as handling a call, processing a claim, or performing a sales transaction. On the customer side it’s finding information or transacting. It is the user interface providing these activities that has a direct line of sight to the top performance indicators, such as revenue, profit, market share, staff and customer satisfaction.
With business increasingly being defined in terms of services, people’s activities and their efficiency and effectiveness are primary drivers of organisational success — as is the user interface supporting these activities. So when you set up your measures, make sure they reflect performance at the user interface level — not just the technology level. After all, no one sees the server, they only ever see the user interface.
Pitfalls to avoid when setting up a Balanced Scorecard
These are the same with any measurement system:
- Too much time taken to produce the measures
- Too many performance measures
- Too many initiatives
- Failure to link initiatives with performance measures
- Performance measures that are not in direct control
- Performance measures that are unrelated to the business, and
- Performance measurements that are lag indicators (e.g. market share), rather than lead indicators (e.g. referrals)
Further reading:
Kaplan, Robert S. and David P. Norton. The Balanced Scorecard: Translating Strategy into Action.Harvard Business School Press, 1996.
