IT Outsourcing Metrics: A Good Example of Management Controls
Nick Krym of Pragmatic Outsourcing Provides a Great Example of IT Service Levels
In the world of Service Level Agreements and IT outsourcing, contractual metrics are difficult to conjure because the work is intangible, with the exception of uptime, help desk/desk side services, and application development milestones. However, metrics are the dashboard by which effective vendor managers can effectively manage outsourcing vendors. Today, Nick Krym posted a great blog article on metrics that’s worth a look.
What makes his approach to IT outsourcing metrics noteworthy?
It begins with the clear segregation of the outsourcing vendors’ maintenance service levels from development service levels. He recognizes up front that the internal stakeholders have different objectives and the contractual pricing is based on different outcomes. Hence, he has two categories of service levels, not one lengthy set of service levels that measure everything or a short set of service levels that apply the same vendor management approach to each service type. BPO vendor managers can use the same approach, as different transaction types require different service levels, but, by breaking the into clear categories, vendor performance of each transaction type can be evaluated. For example, call center inbound customer service and call center inbound customer written correspondence.
The second sound aspect of Nick’s approach is his approach of leveraging a simple few metrics to measure the outsourcing vendors’ effectiveness. Between 4-8 contract service levels is all that most organizations need, and he uses 4-5 in each category. I know of one situation where the client measured 350+ metrics for an IT outsourcing engagement. Not only was it incredibly time consuming to manage and calculate (and rebut!), but the effect of missing a single service level was essentially meaningless to the vendor. Think about it: if you have 10% of your monthly contract value at risk, missing any of the 350 service levels on a $10M/month contract is worth exactly $2,857.14. That’s not exactly going to get people motivated to fix things, right? Having fewer metrics makes each metric more meaningful.
Finally, Nick uses control limits to communicate acceptable service level performance. In each chart, he uses high and low limits. The majority of vendor managers I’ve met use a single line – the service level. However, what most folks don’t understand is that by using high and low control limits, you can:
- Clearly communicate a range of acceptable performance.
- Understand if the vendor is over performing in some areas to offset performance in other areas.
- Display trends within the acceptable performance range to proactively manage impending failure before the vendor misses the SLA.
- Moderate expensive overperformance that the vendor need not do. Remember – you pay for the extra quality, which may not be tangible.
Of course, I will note that Nick’s vendor is performing pretty horribly, but, as he says, he inherited the vendor through M&A activity. This means the vendor needs an opportunity to meet expectations before terminating the relationship. Besides this, here are some opportunities for improvement:
- Never average daily turnaround time metrics to create a monthly metric (top left graph). Averages distort performance. Rather, create a monthly turnaround time based on all transactions and show daily/weekly trending using a month-to-date trend. Then, bin items based on TAT and do root cause analysis on items that fall below TAT. And then histogram based on counts of root cause analysis.
- There are a few glitches. First, the top left graph uses decimal places in the Y-axis, but the others don’t. Second, the top right graph indicates a NEGATIVE response time between Oct 14 and Oct 21, probably because the line is formatted to be curved, a no-no in the data visualization world. Third, consider rounding to a single decimal place, unless contract metrics require a 2nd decimal place. Lastly, it appears that there are lots of zeros, which is probably due to volumes, so it may be important to indicate volumes using a secondary Y-axis to make this clearer.
Great job, Nick!
If you’re interested in 360° Vendor Management’s perspective on metrics, please visit our articles on Top Ten Service Level Agreement Considerations, Outsourcing Vendor Metrics, Operational Metrics, Key Performance Indicators, and Transformation Metrics.
A quick reminder: if you find this content interesting, please subscribe via Google’s automated delivery service to receive updates using the “Subscribe” field on the right side of the page or by clicking here.
Related posts:
That’s interesting that BPO vendor managers can use the same approach, as different transaction types require different service levels, but, by breaking the into clear categories, vendor performance of each transaction type can be evaluated. Nice blog.