On Benchmarking and Elvis

Unemployment numbers notwithstanding, the US economy has picked up ever so slightly in the past few months based on the anecdotal “evidence” from people I talk to. With that there has come a bit of a resurgence of the concept of benchmarking.  Notice no one benchmarks in a down economy.

The problem as I see it, is that benchmarking promotes what Jonas Ridderstróale and Kjell A. Nordstróm call Karaoke Capitalism – a propensity to imitate rather than innovate. Their message is essentially, “Look, no matter how drunk the audience (prospects and customers) gets, you are still not Elvis (Apple).” Benchmarking kills innovation and this is a death knell for most companies. Do think Apple benchmarks?

In the past, I was a disciple of this kind of thinking and attended dozens of benchmarking sessions at business conferences until I noticed one of two universal reactions in both myself and the other participants.

  1. If my company outperformed the average, I thought, “Well, I guess I am OK there. There is no reason to make any serious adjustments. After all, we are better than average.” The result – inaction on my part.
  2. If my company underachieved when compared to the mean, I thought, “Hmmm, I think there might be something wrong about this data.” I would begin to ask questions, not about how I could improve, but rather about the data set. “How many companies including were under X number of people? How many companies also did software development? How many did not have a training center?” The result – inaction on my part.

Now that I have seen the light (thanks mostly to Ron Baker) I have been a constant bane to the existence of those that attend these session. I share my story and ask the participants as they exit the room, “What specifically are you going to change about your company now that you know this information?” The usual response is stuttering followed by an expression of either or both of my observations noted above.

I want to be clear, I do not think action should be taken based on this data in the first place. Why? Because ultimately we (myself and those attending the sessions) were right about this data – it is not scientific! In many cases the data is not even a valid statistical sample set from the pool of companies that is attempting to be benchmarked. Comparing ourselves with a mean means nothing. On average every person on the planet has one developed set of mammary glands and one testicle. So what! Who cares?

Lastly, also almost all benchmarking data is based on numbers available from the financial system, which by definition are accounting for what happened in the past. None of these numbers is a indicator or predictor of future success.

Instead of filling out forms and attending benchmarking sessions, I suggest thinking about how to innovate and create value for your customers today and in the future!


Thoughts on ITA Breakout Session – Professional Service Firm KPIs

This is the second in a series of postings about my thoughts from sessions that I attended at the Information Technology Alliance’s Fall Collaborative (<—I love that word) held in Palm Springs.

Presented by Jeanne Urich of SPI Research. The highlight of my ITA experience occurred during this session when Jeanne acknowledged that a conversation that the two of us had at the spring ITA meeting has influenced her thinking and that she now believes that fixed price agreements are better for customers and consultants. Now to convince her to give up on these dozens of benchmarks! I think this will be a much harder task.

The study consists of 175 metrics, 20 of them deemed key performance indicators. The problem in my mind is that all, save one of the 20 are inwardly focused on the firm, even those in the “Service execution” and “Client relationship” areas are firm-based.

The problem as I see it with most benchmarking is that it focuses us on all the wrong things: the past rather than the future; internal rather than the customer; efficiency rather than effectiveness.

For example, one “customer” metric was about project completion success. The question was asked of the providers, “What percentage of your projects are completed on time and on budget?” The average answer was 74 percent. This more than doubles the number (35 percent) according to a study done by The Standish Group who asked the same question of customers. Needless to say, I side with customers on this one. Again, to her credit Jeanne acknowledged this flaw.

Many of the other metrics are based on the false premise that value delivered is equal to rate times hours, aka, the labor theory of value. This theory is demonstrably false and belief in it has been proven to cause harm.

I have sat in on countless benchmarking session and the reactions of the attendees is always the same: a) if they are doing better than the benchmark, they think they are OK and do nothing, and b) if they are worse than the benchmark, they dispute the data and still do nothing. Path (b) actually happened twice during the session!

My beef with all benchmarking in business is that while it attempts to appear scientific, it is not even close. To her great credit Jeanne is very careful about saying that these metrics are correlative not causal. Unfortunately, most people do not understand this distinction and are lulled into the illusion of control via data.

The findings are always similar and in many cases are nothing more than a bunch of truisms:

  • Firms who market well have higher revenue. (Yet, marketing spend, even among top firms, is less than average across all industries.)
  • Firms who close more business (win to bid ratio) are more successful. (The question is, what do they do differently that allows them to have a higher win to bid ratio? Win more or bid less?)
  • Few firms grew revenue in 2009.
  • Clear vision and strategy and taking care of your people are important in professional firms. Firms that focus on culture are rated as better places to work. (What kind of culture?)

Conclusions are almost always the same – “Increase revenue, lower discretionary spending.” Always a good idea.