Hazard log Information System, subsystem, unit Description Cause s Possible effects, effect on system Category(hazard level --probability and severity) Design constraints Corrective or preventative measures, possible safeguards, recommended action Operational phase when hazardous Responsible group or person for ensuring safeguards provided Tests(verification to be undertaken to demonstrate safety Other proposed and necessary actions Status of hazard resolution process Risk and hazard level measurement Risk= f(likelihood, severity) Impossible to measure risk accurately Instead, use risk assessment Accuracy of such assessments is controversial o avoid paralysis resulting from waiting for definitive data, we assume we have greater knowledge than scientists actually possess and make decisions based on those assumptions William Ruckleshaus Cannot evaluate probability of very rare events directly So use models of the interaction of events that can lead to an accident
c ������������������ ������������������������ Hazard Log Information System, subsystem, unit Description Cause(s) Possible effects, effect on system Category (hazard level −− probability and severity) Design constraints Corrective or preventative measures, possible safeguards, recommended action Operational phase when hazardous Responsible group or person for ensuring safeguards provided. Tests (verification) to be undertaken to demonstrate safety. Other proposed and necessary actions Status of hazard resolution process. ����������������� ����������������������������������� Risk and Hazard Level Measurement Risk = f (likelihood, severity) Impossible to measure risk accurately. Instead, use risk assessment: Accuracy of such assessments is controversial. ‘‘To avoid paralysis resulting from waiting for definitive data, we assume we have greater knowledge than scientists actually possess and make decisions based on those assumptions.’’ William Ruckleshaus Cannot evaluate probability of very rare events directly. c � So use models of the interaction of events that can lead to an accident
eyeson-63 Risk Modeling In practice, models only include events that can be measured Most causal factors involved in major accidents are unmeasurable Unmeasurable factors tend to be ignored or forgotten Can we measure software?(what does it mean to measure design?) Human error? Risk assessment data can be like the captured spy, if you torture it long enough, it will tell you anything you want to know William ruckelshaus Risk in a Free Society Misinterpreting Risk Risk assessments can easily be misinterpreted 10 Extended system boundary 10 System Boundary 10·10
c ������������������ Risk Modeling In practice, models only include events that can be measured. Most causal factors involved in major accidents are unmeasurable. Unmeasurable factors tend to be ignored or forgotten. Can we measure software? (what does it mean to measure design?) Human error? Risk assessment data can be like the captured spy; if you torture it long enough, it will tell you anything you want to know. William Ruckelshaus Risk in a Free Society c ������������������ Misinterpreting Risk Risk assessments can easily be misinterpreted: 10 −4 −3 System Boundary Extended system boundary 10 10 −3 −3 −6 . 10 = 10
EXample of unrealistic risk assessment contributing to an accident Design: System design included a relief valve opened by an operator to protect against overpressurization. A secondary valve was installed as backup in case the primary valve failed. The operator must know if the first valve did not open so the second valve could be activated Events The operator commanded the relief valve to open. The open position indicator light and open indicator light both illuminated. The operator, thinking the primary relief valve had opened, did not activate the secondary relief valve. However, the primary valve was NoT open and the system exploded Causal Factors: Post-accident examination discovered the indicator light circuit wa wired to indicate presence of power at the valve, but it did not indicate valve position. Thus, the indicator showed only that the activation button had been pushed not that the valve had opened An extensive quantitative safety analysis of this design had assumed a low probability of simultaneous failure for the two relief valves, but ignored the possibility of design error in the electrical wiring; the probability of design error was not quantifiable. No safety evaluation of the electrical wiring was made; instead confidence was established on the basis of the low probability of coincident failure of the two relief valves The Therac-25 is another example where unrealistic risk assessment contributed to the losses
c ������������������������ Example of unrealistic risk assessment contributing to an accident Design: System design included a relief valve opened by an operator to protect against overpressurization. A secondary valve was installed as backup in case the primary valve failed. The operator must know if the first valve did not open so the second valve could be activated. Events: The operator commanded the relief valve to open. The open position indicator light and open indicator light both illuminated. The operator, thinking the primary relief valve had opened, did not activate the secondary relief valve. However, the primary valve was NOT open and the system. exploded. Causal Factors: Post−accident examination discovered the indicator light circuit was wired to indicate presence of power at the valve, but it did not indicate valve position. Thus, the indicator showed only that the activation button had been pushed, not that the valve had opened. An extensive quantitative safety analysis of this design had assumed a low probability of simultaneous failure for the two relief valves, but ignored the possibility of design error in the electrical wiring; the probability of design error was not quantifiable. No safety evaluation of the electrical wiring was made; instead confidence was established on the basis of the low probability of coincident failure of the two relief valves. The Therac−25 is another example where unrealistic risk assessment contributed to the losses
Classic Hazard Level Matrix SEVERITY ll Catastrophic Critical Marginal Negligible A Frequent 1-A 1-A III-A Ⅳ∨-A B Moderate 1-B II-B ∨- C Occasional l-C lI-C I-t LIKELIHOOD D Remote 1-D lI-D l-D ∨-D E Unlikely I-E E III-E E F Impossible I-F l-FⅢ-F Another Example Hazard Level Matrix A B D E F Frequent Probable Occasional Remote Improbable Impossible Design action Design action Design action Hazard must Catastrophic required to required to required to or hazard I control hazard control hazard control hazard probability 3 reduced 12 Design action Design action/Hazard must Hazard contr Critical eliminate or eliminate or lazaro"desirable quired to required to be control control hazard control hazard probability cost effective Assume willImpossible 4 reduced 6 Marg niy/ design action Hazard must Hazard control Normally not uired to be controlled desirable if eliminate or or hazard cost effective control hazard probability 5 reduced 6 Negligible Negligible hazard -H 12
c ������������������ Classic Hazard Level Matrix ������������������������ SEVERITY A B C LIKELIHOOD D E F I II III IV Catastrophic Critical Marginal Negligible Frequent Moderate Occasional Remote Unlikely Impossible c � I−A II−A III−A IV−A I−B II−B III−B IV−B I−C II−C III−C IV−C I−D II−D III−D IV−D I−E II−E III−E IV−E I−F II−F III−F IV−F ����������������� ������������������������ Another Example Hazard Level Matrix A B C D E F Frequent Probable Occasional Remote Improbable Impossible 10 11 12 12 12 12 ������������������������� ������� ���������� � ������������������������ ������������������������ ������� ���������� � ������������� ������������������������� ������������ � � ��� ��������������� ����������������� ���� ������������������� ������������������������� ��� ��������������� ��������� ��������� ��������������������� ������������� ������������ � � ��� 12 II Marginal III Negligible IV 1 2 3 4 9 12 3 4 6 7 12 12 5 6 8 10 12 ����������������� ���� ��������������� ������������ � � ��� ������������� ������� �������������������� ������������������� ��������������� � � �������������� ��������������� ����������� �������� ��� ����������������� ���� ��������������� ������������������� ��������������������� ��������� ��������� ��� ��������������� ������������������������� ��������������������� ��������� ��������� ��� ��������������� ������������������������� ��������������������� ��������� ��������� ��� ��������������� ������������������������� ��������������������� ��������� ��������� ��� ��������������� ������������������������� ��������������������� ��������� ��������� ��� ��������������� ������������������������� ������������������� Critical Catastrophic I
Hazard level Assessment Not feasible for complex human/computer-controlled systems No way to determine likelihood Almost always involves new designs and new technology Severity is often adequate(and can be determined) to plan effort to spend on eliminating or mitigating hazard May be possible to establish qualitative criteria to evaluate potential hazard level to make deployment or technology decisions, but will depend on system Example of qualitative criteria AATT Safety Criterion The introduction of aatt tools will not degrade safety from the current level Hazard level assessment based on Severity of worst possible loss associated with tool Likelihood that introduction of tool will reduce current safety level of ATC system
c ������������������ ������������������������ Hazard Level Assessment Not feasible for complex human/computer−controlled systems No way to determine likelihood Almost always involves new designs and new technology Severity is often adequate (and can be determined) to plan effort to spend on eliminating or mitigating hazard. May be possible to establish qualitative criteria to evaluate potential hazard level to make deployment or technology decisions, but will depend on system. c ������������������ ������������������������ Example of Qualitative Criteria AATT Safety Criterion: The introduction of AATT tools will not degrade safety from the current level. Hazard level assessment based on: Severity of worst possible loss associated with tool Likelihood that introduction of tool will reduce current safety level of ATC system