These Three Criteria Determine the Value of a Safety Culture Survey

Most leading organizations today use a culture survey of one kind or another. Some are directly related to safety and some indirectly. Often they are given annually. How worthwhile are these instruments and how should they be used?

Whether your survey is designed to measure organizational culture in general, safety culture in particular, or both, three criteria determine the value you get from the effort:

(1) It is properly validated and demonstrated to predict the outcomes you want.
(2) It can be given regularly over time and tells leaders where they are succeeding and where they need to focus attention.
(3) Leaders give attention to the results, provide helpful perspectives, and take direct action.

Unfortunately the surveys used by most organizations don’t satisfy any of the criteria above. Typically surveys are not validated at all or not properly; they measure variables that haven’t been demonstrated to make a difference, and they are given intermittently and not acted on. Leaders often look at results superficially and consultants often mislead leadership by encouraging them to focus on normative rather than raw data. Let’s dig into all this and I’ll explain why it matters.

First, don’t do another culture survey until you have properly digested the last one and are prepared to take action based on the findings. Each time you give a survey and then let it sit in your computer you lose important credibility. It is not uncommon to find that the great majority of actionable information is already available to you; adding more won’t add that much. Understanding what you have already, and taking action can matter, a lot.

Why do the criteria above matter?

Validation

Having your survey properly validated is crucial, because otherwise you have no idea what the information actually means. Survey questions can be tricky and you want to know that the questions actually predict outcomes that are important to your organization. Otherwise you are engaged in an expensive guessing game.

A proper validation study finds that the answers  employees give are correlated with some important aspect of organizational performance. In the case of safety that should mean incident rates.

There is a wrinkle here unfortunately. Every validated culture survey I know of uses some version of recordable incident rates as the validation measure. For sure this is better than nothing. But recent findings in serious injuries and fatalities (SIFs) have shown that recordable injuries are declining while serious injuries and fatalities are staying level or increasing. So the safety culture survey you are using may well not tell you whether or not your culture is strong or weak with respect to SIFs.

Finding out about this is crucial to your efforts to prevent SIFs, the most important thing you can do! You want to know which variables are most strongly associated with excellence in SIFs.   

Regular administration

You want to know two things: where are we at this point on each variable, and are we improving or not? Both questions are important but the second one is crucial. You should be improving. People may say you are but the data will tell you if you really are.

Raw numbers as well as norms

Consultants typically focus your attention on the norms. How do we compare to other organizations? What percentile do we fall in? These are good questions in one way, but they are not the most important ones. If your industry is failing miserably at employee engagement, you shouldn’t feel good about being in the 75^th percentile unless you are willing to accept objectively poor performance. Good performance is not doing better than others who are doing badly.

Let me elaborate on this. Say the survey question is “I trust my supervisor”. The employee can answer “always” to “never” or some place in between. What average answer tells you that you have good performance? Often the “top two boxes” are taken as positive, which is reasonable. If the majority of your employees don’t endorse the top two boxes, you have a real problem, irrespective of the percentile score. In actuality it is not uncommon in some industries to see the 50^th percentile be less than 50% positive on a question like this. The 75^th percentile could well be around 60% positive.

You should decide, irrespective of norms, what is an acceptable percentage. You can’t expect 100% positive because some employees will say no out of general negativity. This group is usually less than 10% of your employees. So it would be reasonable to set a target of 90% positive. If that sounds daunting to you please reconsider. Do you really want to be satisfied with an organization whose supervisors aren’t almost always trusted? What would it mean to your performance, in safety, and everything else, if you employee trust scale on the survey showed 90% positive scores vs. 50% positive?

To sum up, safety culture surveys can be very useful tools for improving safety performance, if and only if they are properly designed, validated, administered, responded to, and communicated.

For more information on improving your culture, visit our safety culture page or read Tom‘s newest book about organizational culture, If Your Culture Could Talk: A Story About Culture Change.