Design for Safety Unfortunately, everyone had forgotten why the branch came off the top of the main and nobody realized that this was important. Trevor Kletz What Went Wrong? Before a wise man ventures into a pit, he lowers a ladder so he can climb out Rabbi Samuel Ha-Levi Ben Joseph ibm Nagrela
c ��������������������� ���������� Design for Safety Unfortunately, everyone had forgotten why the branch came off the top of the main and nobody realized that this was important. Trevor Kletz What Went Wrong? Before a wise man ventures into a pit, he lowers a ladder so he can climb out. Rabbi Samuel Ha−Levi Ben Joseph Ibm Nagrela
Design for Safety Software design must enforce safety constraints Should be able to trace from requirements to code(vice versa) Design should incorporate basic safety design principles Safe Design Precedence HAZARD ELIMINATION Substitution Simplification Decoupling Elimination of human errors Reduction of hazardous materials or conditions Decreasing cost HAZARD REDUCTION Increasing effectiveness Design for controllability Barriers Lockins. Lockouts, Interlocks Failure Minimization Safety Factors and Margins unfancy HAZARD CONTROL Reducing exposure Isolation and containment Protection systems and fail-safe design DAMAGE REDUCTION
c ��������������������� ���������� Design for Safety Software design must enforce safety constraints Should be able to trace from requirements to code (vice versa) Design should incorporate basic safety design principles c ��������������������� ���������� Safe Design Precedence HAZARD ELIMINATION Reduction of hazardous materials or conditions Elimination of human errors Substitution Simplification Decoupling HAZARD REDUCTION Design for controllability Barriers Lockins, Lockouts, Interlocks Failure Minimization Safety Factors and Margins Redundancy HAZARD CONTROL Reducing exposure Isolation and containment Protection systems and fail−safe design DAMAGE REDUCTION Decreasing cost Increasing effectiveness
Hazard elimination a SUBSTITUTION e Use safe or safer materials Simple hardware devices may be safer than using a computer. No technological imperative that says we MUST use computers to control dangerous devices Introducing new technology introduces unknowns and even unk-unks Design a SIMPLIFICATION Criteria for a simple software design 1. Testable: Number of states limited determinism vs nondeterminism single tasking vs. multitasking polling over interrupts 2. Easily understood and readable 3. Interactions between components are limited and straightforward 4. Code includes only minimum features and capability required by system Should not contain unnecessary or undocumented features or unused executable code 5. Worst case timing is determinable by looking at code
c ��������������������� ���������� Hazard Elimination SUBSTITUTION Use safe or safer materials. Simple hardware devices may be safer than using a computer. No technological imperative that says we MUST use computers to control dangerous devices. Introducing new technology introduces unknowns and even unk−unks. c ��������������������� ���������� SIMPLIFICATION Criteria for a simple software design: 1. Testable: Number of states limited determinism vs. nondeterminism single tasking vs. multitasking polling over interrupts 2. Easily understood and readable 3. Interactions between components are limited and straightforward. 4. Code includes only minimum features and capability required by system. Should not contain unnecessary or undocumented features or unused executable code. 5. Worst case timing is determinable by looking at code
SIMPLIFICATION (cont) Reducing and simplifying interfaces will eliminate errors and make designs more testable Easy to add functions to software, hard to practice restraint Constructing a simple design requires discipline, creativity, restraint and time Design so that structural decomposition matches functional decomposition a DECOUPLING Tightly coupled system is one that is highly interdependent Each part linked to many other parts Failure or unplanned behavior in one can rapidly affect status of others Processes are time-dependent and cannot wait Little slack in system Sequences are invariant Only one way to reach a goal System accidents caused by unplanned interactions Coupling creates increased number of interfaces and potential interactions
c ��������������������� ���������� SIMPLIFICATION (con’t) Reducing and simplifying interfaces will eliminate errors and make designs more testable. Easy to add functions to software, hard to practice restraint. Constructing a simple design requires discipline, creativity, restraint, and time. Design so that structural decomposition matches functional decomposition. . c ��������������������� ���������� DECOUPLING Tightly coupled system is one that is highly interdependent: Each part linked to many other parts. Failure or unplanned behavior in one can rapidly affect status of others. Processes are time−dependent and cannot wait. Little slack in system Sequences are invariant. Only one way to reach a goal. System accidents caused by unplanned interactions. Coupling creates increased number of interfaces and potential interactions
DECOUPLING (con't) Computers tend to increase system coupling unless very careful Applying principles of decoupling to software design Modularization How split up is crucial to determining effects Firewalls Read-only or restricted write memories Eliminate hazardous effects of common hardware failures Design ELIMINATION OF HUMAN ERRORS Design so few opportunities for errors Make impossible or possible to detect immediately Lots of ways to increase safety of human-machine interaction Making status of component clear. Designing software to be error tolerant etc.(will cover separately) Programming language design Not only simple itself (masterable), but should encourage the production of simple and understandable programs Some language features have been found to be particularly error prone
c ��������������������� ���������� DECOUPLING (con’t) Computers tend to increase system coupling unless very careful. Applying principles of decoupling to software design: Modularization: How split up is crucial to determining effects. Firewalls Read−only or restricted write memories Eliminate hazardous effects of common hardware failures c ��������������������� ���������� ELIMINATION OF HUMAN ERRORS Design so few opportunities for errors. Make impossible or possible to detect immediately. Lots of ways to increase safety of human−machine interaction. Making status of component clear. Designing software to be error tolerant etc. (will cover separately) Programming language design: Not only simple itself (masterable), but should encourage the production of simple and understandable programs. Some language features have been found to be particularly error prone