Re: [SystemSafety] a discursion stimulated by recent discussions of alleged safety-critical software faults in automobile software

From: Nancy Leveson < >
Date: Tue, 12 Nov 2013 03:49:38 -0500

I'd like to suggest that mixing up engineering and law is a mistake. There are, or at least should be, different goals. When I am involved in the investigation of major accidents, I often find that the lawyers prevailed over the engineers in making engineering decisions. In other cases, I have gone in and told the engineers what they need to do to make their systems safe. They agree but tell me that their lawyers will not let them do what I am suggesting. I wash my hands of the company then and try to make sure that I sell any stock I own in them.

In addition, the legal definition of "cause" is not necessarily the same as the engineering definition of "cause." Nor need it be the same as, again, the goals are different. Engineers are not trying to find one person or organization with deep pockets or to put someone in jail.


On Tue, Nov 12, 2013 at 1:52 AM, Peter Bernard Ladkin < ladkin_at_xxxxxx

> It seems worth while making again a point I have made before.
> It is not about blame. Which, by the way, I wouldn't necessarily call an
> emotion (Wikipedia, for example, thinks it is an act). It is about
> assignment of responsibility for a deleterious event with a view to
> dispensing compensation. This is a general principle of human behavior and
> lawmaking for thousands of years and occurs in many if not all human
> societies. I won't argue here the case for compensating people for harm you
> have caused them. I'm glad we adhere to it and that I don't live 1600 years
> ago.
> So, if you are a 1970's hotel owner and a rock group trashes some of your
> rooms, you are entitled to a determination of responsibility, and adequate
> compensation from those deemed responsible. Since that will often be
> disputed (likely not by a 1970's rock group, for which it was a source of
> pride), it needs to be decided by the appropriate means, which for us is a
> court of law.
> It used to be the case in GB that hordes of foreigners came ashore from
> boats, took what they wanted, trashed the restaurants as if they were
> Bullingdon boys, and took women into slavery. They had to be fought off.
> When this started being successful, they quit (apart from those who stayed,
> which ruined their business model another way). Every three-year old who
> has played in a sandbox knows this phenomenon, which manifestly does not
> stop when one is older: John Kenneth Galbraith wrote about the power of
> large corporations and the consequences for human society between 40 and 55
> years ago. So there is also another point to this kind of action:
> resistance stops other people doing stuff.
> Toyota knew they had spaghetti code in this acceleration-control kit. They
> wrote so themselves, which you can see in the evidence. They also knew and
> know the consequences of such complexity, namely a lack of control over the
> behavioral properties of the program. That is also in the evidence. It
> didn't stop them using the code again and again (it was still in the 2010
> model year, apparently). That won't continue. For example, they have
> recently signed a contract with Altran UK to develop examples of useful
> code which is free of run-time error.
> If you don't like principles of fairness and responsibility, and developed
> organisations (the courts) with the power to set those principles of
> fairness for everyone and every organisation without exception, just try
> doing without it..........
> Prof. Peter Bernard Ladkin, University of Bielefeld and Causalis Limited
> On 12 Nov 2013, at 03:07, "Les Chambers" <les_at_xxxxxx >
> What bothers me is the alarming repeat performances we have of these
> disasters. And the eye-watering sums of money spent on forensics and
> retribution. These events are typically passed over to the legal profession
> who proceed to dine out on the assignation of blame.......
> _______________________________________________
> The System Safety Mailing List
> systemsafety_at_xxxxxx >

Prof. Nancy Leveson
Aeronautics and Astronautics and Engineering Systems
MIT, Room 33-334
77 Massachusetts Ave.
Cambridge, MA 02142

Telephone: 617-258-0505
Email: leveson_at_xxxxxx

_______________________________________________ The System Safety Mailing List systemsafety_at_xxxxxx
Received on Tue Nov 12 2013 - 09:49:49 CET

This archive was generated by hypermail 2.3.0 : Tue Jun 04 2019 - 21:17:06 CEST