Meaningful Metrics

I enjoyed “Moneyball” when we went to see it last fall and was pleased to see it get a nomination from the Academy of Motion Pictures a few months ago. I didn’t believe it was worthy of a “best picture” Oscar, but I didn’t mind the extra attention it should get from being placed on the list.

I’m speaking about this on a technology blog because I believe that we in IT could benefit from a little bit of “Moneyball” thinking: what are the most meaningful statistics we should be monitoring to see how well we are doing for the business?

I know of a lot of technology departments that report an “uptime percentage” of 99% or better and I’ve often wondered how many executives’ eyes glaze over when another report is presented with this boring statistic. There are no standards of how to calculate this number, and many tech execs (not you, of course) exclude “scheduled maintenance” windows and other pre-determined outages in order to reflect how well they do at keeping systems up when they said they would keep them up. This is not a bad thing.

But if you had to tell the business a pure number – the percent that the systems were up vs. the percent they were down (for ANY reason) – maybe you would be more productive during scheduled outages or not have them as often. Maybe you should reflect a trend over the last few quarters of increasing the amount of hours each month that the systems were available. Maybe it’s not even “uptime” that’s important. Maybe “percent of time the systems ran at 80% of capacity during the week” or some other more meaningful gauge.

Really what I’m challenging you to consider is putting stats front and center that will encourage changes in IT behavior that benefit the business. Are you comfortable enough with a transparent department to partner with accounting or internal audit to monitor key performance indicators? Then pull together some thoughtful people for informal discussions on how to improve IT performance by tracking and reporting on stuff you haven’t thought of tracking before.

Two things come out of this exercise that will improve your organization. One is the comfort you give to everyone that you are willing to be transparent. The other is that peer departments will follow your lead and (hopefully) begin looking at themselves in new ways.

After Moneyball came out, I read an article in Sports Illustrated musing about which stat was better to judge Quarterback success: QB ratings or wins. And not just “wins” but wins in big games. I know some of you track all KINDS of stuff: helpdesk closed tickets; percent of code defects; hardware repairs, etc. But we need to take a hint from sports – how well are we doing when the pressure is on? Tracking that and changing our processes and behaviors based upon those signals is much better than boring reporting of uptime.