Organizational Diagnostics

Benchmarking Survey Results

Return to Articles listing

At some point you will be looking at a score of 5.2 on a 1-to-7 scale and wondering "Is that a good score?". This is the essential dilemma of benchmarking.  Establishing the appropriate reference point for a survey question is a complex issue. A given score can only be interpreted compared to some reference point.  For example, after 15 years of experience, we know that questions about loyalty to the company seldom produce more than mid-level averages.  But questions about the quality of supervision can average as high as 6 out of 7 or as low as 3 out of 7.  So an average score of 5 on loyalty would be fantastic!  A 5 on quality of supervision would be nice, but nothing to write home to Mom about.

One of the more popular strategies for handling the problem is to secure "national norms", but we believe it suffers from several serious defects:

It is usually not a representative sample of an industry; most typically it is only the previous clients of the survey consultant.  It is often quite diverse.  An "Electronics Industry" database could include manufacturers of semiconductors as well as consumer products, and even representatives of the service industry.  More important, it does not constitute a definition of what is normative, typical, or expected.  It is only the average for the companies in the sample.

On issues such as development opportunities, quality of supervision, benefits, morale, or recognition, your major competitor is local companies in any industry who can attract away key employees who are not tied to any industry group.  This includes almost all employees except highly-focused engineering staff.

On issues such as cycle time, inventory turns, or customer response time it is perfectly appropriate to compare your company to similar firms.  These numbers directly reflect your performance in the market.  But they are also not employee survey items!  They require data from actual system performance.

Generic survey questions feel...well, generic.  They have to be written at a high level of generality so they can apply across industry groups and across organizational levels.

A customized survey allows you to examine the specific and unique issues for your company.  It allows tailoring the language to fit the terminology most comfortable for the respondent. The survey feels like it was written by "someone who understands us" rather than by an academic or some stranger.  That means more precise and more useful information. 

Suppose you discover that morale is low.  Suppose you even find out it is lower than morale scores for comparable companies.  Now what?  What would you do differently to improve the situation?  With more sophisticated analysis techniques (along with a customized instrument), it is possible to unravel what drives morale.  It is then possible to identify the strongest leverage points for improving morale.  In some companies we have worked with, the drivers of morale were quite local, like supervisor behavior or working conditions.  In other companies, the more critical drivers of morale were teamwork skills, especially across departments.  In still others, the key drivers were executive behavior. These drivers cannot be identified from looking only at the levels of survey items, even with comparisons to national norms.

Having benchmarks for the LEVEL of survey items draws the users of the data into a search for their lowest scores.  Some scores are low, but still unimportant.  We routinely find that scores about satisfaction with pay are among the lowest in a survey.   But in good companies, those low scores are also unrelated to morale, loyalty, optimism, or executive credibility.  People just want to make more money, but their dissatisfaction is not detracting from their enthusiasm or desire to stay with the company.  Often in the same survey, we will see scores on teamwork or cooperation between departments that are at a higher level, but their linkage to morale, loyalty, etc. is strong.  It would be a waste of resources to hold seminars explaining the pay policies; the real need is for teaching cooperative strategies and reviewing the reinforcement for greater coordination between groups.

The best benchmark is self-referencing to previous performance.  You might be better than most, but what if you are worse than you were six months ago...and slipping downward?  The second best benchmark is your own "best of class".  Internal examples of excellence give people realistic models of what is achievable in your company.

The net result is that there are few, if any, valid decisions that can be made with "national industry norms".  They satisfy a normal curiosity to know where we stand, but they can short circuit a more important set of questions about the appropriate next steps toward a desired future.

We believe there is an alternative benchmarking procedure.  There are criteria other than industry trends which can be used to establish benchmarks for key survey items: 

• Linkage to important strategic initiatives 
• Reflects company values 
• Likely to have strong positive impact on employee experience (morale, loyalty)

Items that fail to satisfy any of these criteria should probably not be included in the survey; there is no compelling reason to track that data!

Determining an appropriate minimum median score is assisted by some empirical reference points.  The most valuable reference points are:

• Self-referencing to previous performance 
• Self-referencing to current internal "best of class" performance
• External referencing to key market competitors 
• External and referencing to regional employment competitors

Anecdotal information on comparisons to external competitors can often be secured from current employees who moved from those firms or through other industry contacts.  Information about local companies who are not direct industry competitors can often be obtained by simply asking.

Using these reference points, decision-makers can determine a benchmark level for each section of the survey.  This does not require benchmarking every question.  Typically each section will have a summary question.  For example, a section on quality of supervision might contain a question such as "My manager treats me in a way that motivates me to give my best effort for the company".  The respondent's degree of agreement with that single question should be benchmarked.  Individual items about work organization, leading effective meetings, cultivating teamwork, or other elements of supervision only need to be benchmarked if they are clearly linked to a particular strategic initiative or value.

The process defined here not only establishes benchmarks for survey data, it brings decision makers to a sharper awareness of their own priorities and trade-offs.  That understanding greatly facilitates later decision making.