There are not many safety professionals who have been audited that do not already know exactly what the title means! I do not know where this numbers game originated from, but I can say this with 100% certainty… it is a MAJOR PROBLEM in our profession and does more damage than any solid audit could ever hope to accomplish. For those wondering what the “numbers game” is, let me explain, as well as provide examples of how silly this game is within our profession. Auditing is a CRITICAL FUNCTION within any management system, and I believe that in a safety management system, it is in the Top 3 of critical functions. So when SAFTENG LLC does an audit our goal is to leave no rock unturned, leave no system unchallenged, and to leave the business with a comfort level that if OSHA was to arrive the next morning their would be little they could find that has not already been identified and an action plan in place to correct! When audits are done this way and the facility responds to the audit results, the impact can be HUGE for the business and the workers; however, let either the audit process or the response to the audit be taken lightly, many bad things can come from this for the business, as well as the workers. But I digress from my “numbers game”…
The “numbers game” is basically when senior managers look solely at the number of findings from a safety/PSM audit. They do not look at the severity of the finding, only how many findings were received from the audit. I have seen it so bad, that some organizations plot their audit results based on the number of findings and those facilities in the top 25% get the !@#$% beat out of them, and in some cases, people lose their jobs.
Why is this just so wrong? Take a Process Safety Management 3-year audit and let us compare Plant 1 and Plant 2’s performance. Now, looking at the data (i.e., number of findings), Plant 1 looks much better than Plant 2. Plant 1 had only 14 findings from its PSM audit, and Plant 2 had 45 findings. With just this data, who would you say is doing a better job at process safety, and which plant would have the lower risk profile for a catastrophic accident? Those of you who have played this game know this is a TRICK QUESTION, but for those who have been fortunate NOT to play this game, let me explain.
When a facility has a mature process safety management system, especially when they are stretching to achieve world-class process safety, the facility will have very detailed programs/practices/permits. Thus, there is MUCH MORE to audit and thus a larger opportunity for a finding. Now, a sound audit system will distinguish between internal requirements and OSHA/EPA requirements. However, many audits do NOT make this distinction; some make the distinction, but this distinction NEVER makes up to the top managers who review the reports. Plant 2 is this plant; they are indeed at an OSHA VPP STAR level and have many best practices in place. There is, of course, room for improvement in several areas, but all in all, they are doing quite well and have their risk profile well defined and working to lower it.
When a facility is NOT living or even doing PSM/RMP (i.e., they have no written programs and are not doing PSM activities), then there is NOTHING to audit. In this situation, many auditors will write 14 findings based on the fact that the facility needs to implement the 14 elements of PSM. The facility has no written programs, no training, no activities, etc., to control the use, storage, and handling of their HHC/EHS, and thus they end up with only 14 findings. And to put the cherry on top of this crazy scenario (which is 100% factual), Plant 1 won a corporate safety award for their safety performance in the presiding year (of course, based on OSHA rate and not PSM performance).
I have been working on a scoring system for my PSM/RMP audits where each requirement is weighted and assigned points. I use the OSHA PQV checklist and the EPA RMP Checklist(s) to break down the specific requirements. So basically, each element will have a point total possible. Based on how the facility measures up to each specific requirement, they will get a percentage grade for each element and an overall score. For example, let’s say that Hot Work has 100 points and Plant 1 has nothing in place regarding a program or permit that complies with 1910.252(a), and thus they get 0%. Plant 2 has an extensive Hot Work Permit system and field auditing system for hot work. Still, we found several permits not being followed 100% in the field, and we found some permits not being closed out properly, so the facility ended up with two (2) findings for a score of 90%. Remember I said the requirements were “weighted” by severity, and in my eye, failure to execute a program/permit in the field has a higher concern. Now if we are playing the numbers game, Plant 1 looks like they are PERFECT when, in fact, they have NOTHING, and Plant 2 looks like they are doing “OK” when, in fact, they are doing extremely well. All 14 elements would be audited, and an overall score would be provided. As long as the same auditors conducted all the audits within an organization, the facilities could be ranked by audit results, and this would provide an ACCURATE reflection of which facilities need help and which ones should be providing the help.
So, the next time you arrive at an audit wrap-up meeting, sit back and observe what some people at the table are doing. Look for those who are READING the findings to understand the actual issue, and look for those who are NOT reading the findings and are just tallying up the numbers. The ONLY way out of this vicious numbers game is the EDUCATION of management about how safety numbers can be so easily MANIPULATED and that the substance is far more valuable than the numbers.
And by the way, this is a PERFECT LEAD INTO utilizing LEADING INDICATORS in safety and process safety, but I will save that discussion for another snow day.