In today’s safety arena, metrics still rage on. I believe in metrics; everything should be DEFINED and QUANTIFIED in safety, and “performance metrics” are no different. Advanced SMSs utilize “leading indicators” rather than “lagging indicators” such as injury rates, WC $s, etc. This is a significant advancement in safety – measuring the activities (and their quality) that drive safety performance. But in today’s workplace, even leading indicator metrics can become a “numbers game”. Management and workers can figure out how to manipulate the “leading indicators” so that goals are met/exceeded while risk runs rampant.
As we have discussed many times, our SMS must be built upon the PLAN-DO-CHECK-ACT model, and the CHECK element plays a significant role in ensuring that our leading indicators are not being manipulated. A solid CHECK function will validate the data we receive from our activities and leading indicators. Without it, and with a lot of pressure from management to “hit the leading indicators numbers,” safety activities can be “pencil-whipped” to achieve those goals. And this manipulation can lead to management believing risks are under control.
Take our Permit to Work (PTW) process; it plays a major role in our administrative controls within the SMS. Thus we should be verifying/validating its effectiveness to control risks. We do this by…
Auditing the heck out of our safe work permits. This includes the issuing process, the approval process, the working under the permit controls, and the closure of the permits. We do “desktop” and “field audits” to get a complete picture of this safety process.
Most of these audits are performed by non-safety team members. Goals are set for the number/% of permits to be audited (both desktop and field audits), and in some organizations, “grade” goals may be set. I have personally experienced both goals being hit and believing the process was functioning at the level I had hoped for. Until such time, I learned of this “data validation” process in my Statistical Process Control (SPC) training (this was before my Six Sigma days).
After training, I returned to my plant and had to conduct a “project” to complete my certification process. My plant manager and I decided (as he knew our permitting process was not as good as the numbers said) to use the PTW process as my project. Man, I was shocked at the “Gap” we identified.
This project entailed bringing in other safety engineers from around the company for our annual turn-around/shutdown. They would audit 100% of the permits issued during this 12-day period. Their score for these 300+ audited permits was less than 70%; our score before this project was over 90%. Coming out of the turn-around, the internal safety team began a focused audit process where we, too, had a score of less than 70%.
Some of you have heard me mention “managing the gap”; this is what I mean. We performed a causal analysis to establish how we had such a gap and what we needed to do to close the gap. The vast majority of opportunity came from “pen-whipping” the permits; the other part was better training for permit issuers and approvers.
But for over a year, the management team believed we had a solid PTW process based on our internal auditing numbers (e.g., leading indicator). Nothing could have been further from the truth.
So VALIDATE YOUR LEADING INDICATORS, as they too can be manipulated just like many will manipulate their injury numbers (e.g. lagging indicators).
