If you don’t know me and are confused as to what makes me tick as a safety pro… it’s data! And not just any data – VERFIED AND VALIDATED DATA!
And this leads me to “on-line reviews”. I hear a lot of people refer to these on-line reviews about vacation rentals, restaurants, and products. They are serious about convincing themselves “it must be ok, just look at all these 5 star reviews”. I’m sorry, but I give these “reviews” very little credence as they are not validated and verified in any meaningful manner.
Yes, I do read them, but I read them to identify “trends” and I always start with the 5 and 1 star reviews. But some big orgs have come out in recent years stating they up to 10% of the reviews on their website are bots attempting to influence buyers. If we can rememeber backnin the day when Amazon was having troubles with sellers buying reviews and uploading them to influence buyers.
In safety, METRICS can be very helpful in gauging progress. But these METRICS MUST BE VERIFIED/ VALIDATED before we make decision based on it. One thing I learned being a Six Sigma Black Belt was that “making decisions on bad data will result in making bad decisions”. Or as some like to say… “Trust but Verify”.
A big part of this is simply done by auditing our safeguards/barriers/controls on a routine basis. These audits can be 1st Party (the unit/department audits itself), 2nd Party (someone within the company, but detached from the unit/department audits), or 3rd Party (an SME outside the org audits). All have valid needs within a mature SMS. My approach was that the unit/department does the bulk of auditing itself and the safety team would audit a much smaller sample to VALIDATE the unit/department results.
For example: the unit has done 300 behavioral observations and claim that 99% were SAFE. Using the exact same observation card and observing the same sample set of workers and contractors, the safety team finds the rate at 79%. A 3rd party is brought in BLINDLY and spends a week in the unit and completes 100 observations (under the same operational conditions) and finds it to be 85% SAFE. Is 14% variation “good enough” for you to be making decisions on?
If we believe the unit data we would believe “all is well, continue on team and keep up the good work”. But the 3rd Party, as well as 2nd Party Observations, identified 2 trends, in their sampling that the unit failed to identify: 1) PPE errors and 2) vehicle access into HAZLOCs. Both are serious deviations from established hazard control efforts and need to be addressed. However, had we not attempted to VALDATE the observation data, we would be MISSING these opportunities.
Moral of this safety story… collect data! Collect as much data as we can! But the bulk of this data should be LEADING INDICATORS.
But have little faith in the data UNTIL it has been VERIFIED AND VALIDATED via some recognized methodology such as a Gauge R&R exercise. And yes, this means ALL DATA that is being used to assess risks and make decisions on how the safety process (SMS) needs to be to be modified to meet these risk challenges.
