Re: [SystemSafety] The bomb again

From: Nancy Leveson < >
Date: Thu, 10 Oct 2013 20:55:42 -0400


I don't know what prompted Peter's critique of STAMP. But I feel compelled to respond. [Peter's comments are in italics]

*Two hundred fifty years ago, David Hume proposed two characterisations of cause which have persisted ... Ten years ago, Nancy announced that she had a new conception of causality, which was embodied in STAMP. I saw, and continue to see, a problem in redefining a concept which has had a good and productive run in science for three centuries (if not two and a half millenia). It could well be that this new concept is a very useful concept; it could be very helpful in identifying areas of interest in accident investigation; indeed, judging by the interest in STAMP I imagine many people think it is. But why not choose a different word for it?* *
*
The world has changed a lot in the past 250 years, particularly the world of engineering. Whole new fields have been created, some to explain the behavior of new machines. An example is the development of thermodynamics to explain the behavior of steam engines and their tendency to explode and cause accidents. After WWII, the growth of new technology exploded, with the notable introduction of the computer, which has revolutionized the world of engineering. New types of math and engineering concepts were invented (or resurrected, such as Boolean algebra) to keep up. To say that a philosophical concept was good enough 250 years ago so it should still be good enough today reminds me of Kelvin's [alleged] statement in 1900 that "There
is nothing new to be discovered in physics now; All that remains is more and more precise measurement."

As we build more complex systems, new engineering techniques (and sometimes new math) is needed to keep up. Accident causes are changing as the design of our systems changing, or at least increasing, as the design of our systems change. The argument in my new book is that what was adequate in the past is no longer adequate. STAMP is a new accident causality model that extends the older models to include new accident causes in our increasingly complex world (it still includes the old ones, i.e., the old models as a subset). In reality, STAMP is not really "new" in that it is simply an application of systems theory to the specific concept of safety. Actually, much of what is in STAMP is implied in the writings of the aeronautical engineers who invented System Safety in the late 1940s and 1950s (such as C.O. Miller who claimed to have come up with the term System Safety). Systems theory itself was a reaction to exactly the kind of philosophical thinking that Peter's alludes to because the traditional concepts did not fit the world of engineering that was beginning to appear in the 1940s and 1950s. ISystems theory was a reaction to analytic reductionism (by those like Descartes) which assumes that a system is the sum of its parts. Systems theory, which underlies System Engineering (which arose at the same time in the 1950s) instead assumes that a system can be more than the sum of its parts. System Safety and System Engineering were closely allied in 1950s missile defense systems for which they were originally developed. Analytic reduction was an important concept the development of modern physics, but after 200 years, engineers started to build systems for which something new was needed.

*People who like STAMP could *obviously* use WBA for parts of what they do - the two methods are compatible. And they would see the same advantage as other WBA users. The only hindrance to such practice is use of the word "causal" for two different concepts.*

*
*
*

The other thing about using STAMP is you have to buy the model. Now, I'm sure it is helpful, because the people developing STAMP are very smart and very dedicated and have been at it for a decade. But is the model right? One might well be able to persuade engineers that the STAMP social/organisational model is the bee's knees, but it is a quite different matter to persuade the experts in those things, the organisational scientists.

Constance Perrin wrote a book in which she investigated some incidents at nuclear power stations and came to the conclusion that there was a tension between the way the plant was conceived to work organisationally and the architecture of plant operations impregnated in the minds of the operators, who came mostly from the "nuclear navy", which had/has a modus operandi completely different from the intended plant-operations architecture. A crucial insight. It is not obvious to me how a STAMP analysis would lead you to the same conclusion. (Maybe a good project to try?) That is why I prefer to leave these matters to the organisational theorists (despite their insistence upon using a language whose syntax and vocabulary is identical with those of English but whose semantics appears to come from Alpha Centauri).

Ten years ago, some colleagues in Braunschweig compared analyses of the same accident (the Brühl derailment) using WBA and using STAMP. STAMP identified a lot of organisational features of the Deutsch Bahn (German railways, as it then was; now it's DB). STAMP likes to see feedback, but the DB, like many German organisations, is hierarchical and STAMP wanted to see cycles where things were acyclic. It wouldn't have been helpful, because, well, I guess you could try to tell DB to change things, but they would say "we have been doing it like this for over a century; here are the reasons we do it this way (giving you the very thick history book); it has evolved so and it more or less works; and if we change it to something new we are likely to introduce weaknesses which we won't know about until we start having accidents because of them." And, you know, that's not a bad set of reasons: you don't change things that aren't really broken, even when a major scientist redefines "broken" for you. (In contrast, they are happily adopting WBA through third-party recommendation and training.)*

On Wed, Oct 9, 2013 at 7:52 AM, Peter Bernard Ladkin < ladkin_at_xxxxxx

> On 10/8/13 7:43 AM, Matthew Squair wrote:
>
>> Isn't the question of whether you trust their efforts really a variant of
>> the agency dilemma? And
>> isn't that what 'design' of the socio-technical system should address,
>> and what a methodology such
>> as STAMP can assist you in doing?
>>
>
> Well, I bet some System Safety List old hands are chuckling to themselves
> at this one. I'd hate to disappoint........
>
> Let's stick with accident analysis. We have our own method for causally
> analysing accidents, called WBA. It is used in certain large German
> companies, and of course Causalis uses it for clients who wish for causal
> analyses.
>
> WBA does not have an inbuilt method for classifying
> operator/organisational/**society/legal/governmental (OOSLG) factors as
> such. We use the PARDIA classification for pointy-end activities and use
> decision-theoretic analyses (Rational Cognitive Modelling) for multi-agent
> interactions. WBA does determine where any OOSLG factors are present and
> those that are not addressed by PARDIA and RCM I like to leave for the
> organisational scientists such as Downer and Perrow.
>
> Two hundred fifty years ago, David Hume proposed two characterisations of
> cause which have persisted. One is the constant-conjunction criterion
> (CCC), beloved of those who collect statistics on repeatable events, for
> example the superb recent work of Judea Pearl and colleagues. The other is
> the counterfactual criterion, which WBA uses in a form called the
> Counterfactual Test (CT). Almost all conceptions of causality in the
> scientific, engineering and philosophical literature are one or the other
> of these. CT is used in an intuitive fashion by more or less all aviation
> accident investigations, and it occurs explicitly in the USAF guidance for
> aviation accident investigation. It should be more or less obvious why this
> is so - commercial-aviation accidents are not events which repeat in a
> manner in which CCC can address. (In contrast, CCC *obviously* helps with
> road safety, because road accidents happen with appropriate frequencies for
> CCC techniques to be useful.)
>
> Ten years ago, Nancy announced that she had a new conception of causality,
> which was embodied in STAMP. I saw, and continue to see, a problem in
> redefining a concept which has had a good and productive run in science for
> three centuries (if not two and a half millenia). It could well be that
> this new concept is a very useful concept; it could be very helpful in
> identifying areas of interest in accident investigation; indeed, judging by
> the interest in STAMP I imagine many people think it is. But why not choose
> a different word for it?
>
> We had a discussion on the York list. It wasn't scientifically very
> fruitful (but I do remember fondly - and repeat - a particular piece of
> repartee).
>
> People who like STAMP could *obviously* use WBA for parts of what they do
> - the two methods are compatible. And they would see the same advantage as
> other WBA users. The only hindrance to such practice is use of the word
> "causal" for two different concepts.
>
> The other thing about using STAMP is you have to buy the model. Now, I'm
> sure it is helpful, because the people developing STAMP are very smart and
> very dedicated and have been at it for a decade. But is the model right?
> One might well be able to persuade engineers that the STAMP
> social/organisational model is the bee's knees, but it is a quite different
> matter to persuade the experts in those things, the organisational
> scientists.
>
> Constance Perrin wrote a book in which she investigated some incidents at
> nuclear power stations and came to the conclusion that there was a tension
> between the way the plant was conceived to work organisationally and the
> architecture of plant operations impregnated in the minds of the operators,
> who came mostly from the "nuclear navy", which had/has a modus operandi
> completely different from the intended plant-operations architecture. A
> crucial insight. It is not obvious to me how a STAMP analysis would lead
> you to the same conclusion. (Maybe a good project to try?) That is why I
> prefer to leave these matters to the organisational theorists (despite
> their insistence upon using a language whose syntax and vocabulary is
> identical with those of English but whose semantics appears to come from
> Alpha Centauri).
>
> Ten years ago, some colleagues in Braunschweig compared analyses of the
> same accident (the Brühl derailment) using WBA and using STAMP. STAMP
> identified a lot of organisational features of the Deutsch Bahn (German
> railways, as it then was; now it's DB). STAMP likes to see feedback, but
> the DB, like many German organisations, is hierarchical and STAMP wanted to
> see cycles where things were acyclic. It wouldn't have been helpful,
> because, well, I guess you could try to tell DB to change things, but they
> would say "we have been doing it like this for over a century; here are the
> reasons we do it this way (giving you the very thick history book); it has
> evolved so and it more or less works; and if we change it to something new
> we are likely to introduce weaknesses which we won't know about until we
> start having accidents because of them." And, you know, that's not a bad
> set of reasons: you don't change things that aren't really broken, even
> when a major scientist redefines "broken" for you. (In contrast, they are
> happily adopting WBA through third-party recommendation and training.)
>
> All of which is not to say that we indulge heavily in NIH round here.
> Indeed, there was a major STAMP workshop recently put on by colleague
> Schnieder in Braunschweig, which generated a lot of interest. That's very
> welcome - as Nancy says, the important thing is thinking hard about hazards
> and accidents and using whatever help you can get.
>
> PBL
>
> Prof. Peter Bernard Ladkin, Faculty of Technology, University of
> Bielefeld, 33594 Bielefeld, Germany
> Tel+msg +49 (0)521 880 7319 www.rvs.uni-bielefeld.de
>
>
>
>
>
> ______________________________**_________________
> The System Safety Mailing List
> systemsafety_at_xxxxxx >

-- 
Prof. Nancy Leveson
Aeronautics and Astronautics and Engineering Systems
MIT, Room 33-334
77 Massachusetts Ave.
Cambridge, MA 02142

Telephone: 617-258-0505
Email: leveson_at_xxxxxx
URL: http://sunnyday.mit.edu



_______________________________________________ The System Safety Mailing List systemsafety_at_xxxxxx
Received on Fri Oct 11 2013 - 02:55:52 CEST

This archive was generated by hypermail 2.3.0 : Tue Jun 04 2019 - 21:17:06 CEST