I found this study to be VERY INTERESTING in that I have always said you line 20 of us and show us all the same photo and ask us, one ata time, to identify the hazards in the photo. We all know how big a spread we would lie in!!! In this study, 31 qualitative tools used for estimation risks associated with industrial machines and which follow the ISO 14121-1: 2007 guidelines were analyzed by (i) comparing their risk estimation parameters and (ii) applying the different tools to estimate risks associated with 20 hazardous situations (scenarios).
The objective of this study was to theoretically compare the performances of tools in estimating risks and to evaluate whether tools estimate risks uniformly. Ideally, the risk levels obtained by different users when applying the different tools to the same risk scenarios should be more or less similar. The risk levels obtained by the same users when applying the different tools to the same risk scenarios at different times should also show certain similarities. As such, any important variability in risk estimations can be attributed to flaws or biases in the tools and these can be based on parameters as well as the architectures of the tools. In order to compare tools which have different parameters, as well as different number of levels or thresholds for those parameters, it was required to set up equivalence scales for the different risk estimation parameters. By using common benchmarks, it is showed that this comparison of different risk estimation tools is possible. Therefore, in this report, the setting up of those equivalence scales is described and some analysis of the different parameters used in the tools is presented. Some guidelines on how to define parameters in risk estimation tools in order to make risk estimations easier as well as independent of tools and users (i.e. repeatability of risk estimation results), are also provided. Moreover, the differences obtained in the risk estimation results when applying different risk estimation tools to the same hazardous situations involving dangerous machines are studied by investigating (i) the influence of the types of risk estimation parameters and methods of construction of the tools, (ii) the influence of the number of levels for each parameter and (iii) the influence of the number of risk levels on the results. As such, the 31 risk estimation tools are compared by applying them to 20 hazardous situations. The results show significant differences among the tools in estimating risks associated with the same hazardous situations, i.e. risk is tool dependent. The scope of the tool and its construction or architecture seem to be one of the contributing factors in this variability of the results. Tools that follow the 2 configurations proposed in ISO 14121-1:2007 produce similar average risk levels but both configurations have tools that will underestimate or overestimate risk associated with hazardous situations. This leads to conclude that simple tools, which have 2 parameters, can be as effective as more detailed tools, which have 4 parameters. It was also observed that the 31 tools could be grouped as 9 low risk estimating tools, 8 intermediate risk estimating tools and 14 high risk estimating tools. Moreover, there are tools which are not appropriate for machinery risk assessment even if their scopes often state the opposite. Finally, the observations in the behaviors of the different tools have guided the authors in proposing a series of construction rules for the tools in order to alleviate most of the problems associated with the variability in the risk estimations. Those recommendations can potentially guide users of risk estimation tools when choosing, designing or using a risk estimation tool.
