Re: [SystemSafety] a discursion stimulated by recent discussions of alleged safety-critical software faults in automobile software

From: Les Chambers < >
Date: Wed, 13 Nov 2013 10:23:24 +1000


I made a brief reference to a very long story. The full report can be found here:

It turns out that, in the interests of getting the problem fixed quickly, the then Premier of Queensland signed a legal instrument that prevented the Queensland government from suing the prime contractor in question. I am in furious agreement that the enquiry would have been justified if there was some possibility of recovering the hundreds of millions of dollars of wasted public funds, however that was not to be. Instead the ensuing enquiry was driven by the political need to placate the anger of the 78,000 employees of Queensland Health who were adversely affected in some way.

In general I believe that, in the context of complex systems development, the punative: "punish them so they don't repeat offend" or the serial killer approach: "take em off the streets so they can't reoffend" are thirty meter metaphors - they look good from a distance but when you engage with them close-up, as I have, they don't work. I have no doubt that these very large organisations will all reoffend no matter how much you shame them, fine them or jail them, not because they are evil empires but because they employ fallible human beings. There are many repeating failure modes. One of the most common is the sales function getting too much control over a bid package resulting in a flight of fancy that gets turned into a fixed price for a complex system that cannot be delivered for the peanuts offered - with the resulting crashing of schedules, reduction in functionality and cutting of corners especially in testing ... followed by very expensive failure in use. The bad actors in these scenarios do not come ashore with battle axes like Vikings ready to rape and pillage. They while way their incompetent hours wreaking havoc, making poor decisions or no decisions and when things turn pear shaped, take their redundancy payouts, get another job and start again. This just keeps happening and quite frankly IT WILL NOT DO!

I can offer several solutions that one day will happen because they must happen. In organisations like NASA they have already happened because of the obvious link between bad technology and death (astronauts have a marvellous think-ahead "what can kill me next" attitude). Firstly our profession must develop the equivalent of NASA's flight rules. Rules for complex system development and operation that must NEVER be broken. Graduates must leave university with these rules ingrained to the point where they would rather stick a fork in their eye than put 10,000 globals in a real-time application. I hope you are doing your part here Peter. Secondly, systems and software engineering management must gain and retain power over technology implementation. We already have the concept of separation of concerns in architectural design. Society has had the concept of the executive, the legislature and the judiciary for centuries. The executive sets policy and gives leadership, the legislature debates the details and creates the laws and the judiciary enforces same. Corruption and societal degradation usually occurs when one personality has power over all three. This is a frequent feature of failed systems projects.

Lastly I offer a dangerous idea: that some day, those with the knowledge to create and maturity to manage complex systems may break free of their chains of servitude and form a fifth estate. And in so doing save the planet.

This is my contribution for the year. On Nov 24 I'm scheduled to set sail on the sloop Northern Child from Las Palmas in the Canary islands bound for Saint Lucia in the Bahamas. Northern Child's progress can be tracked at:

I sincerely hope this technology works.



From: Peter Bernard Ladkin [mailto:ladkin_at_xxxxxx Sent: Tuesday, November 12, 2013 4:53 PM To: Les Chambers
Cc: Matthew Squair; Steve Tockey;
systemsafety_at_xxxxxx Subject: Re: [SystemSafety] a discursion stimulated by recent discussions of alleged safety-critical software faults in automobile software  

It seems worth while making again a point I have made before.  

It is not about blame. Which, by the way, I wouldn't necessarily call an emotion (Wikipedia, for example, thinks it is an act). It is about assignment of responsibility for a deleterious event with a view to dispensing compensation. This is a general principle of human behavior and lawmaking for thousands of years and occurs in many if not all human societies. I won't argue here the case for compensating people for harm you have caused them. I'm glad we adhere to it and that I don't live 1600 years ago.  

So, if you are a 1970's hotel owner and a rock group trashes some of your rooms, you are entitled to a determination of responsibility, and adequate compensation from those deemed responsible. Since that will often be disputed (likely not by a 1970's rock group, for which it was a source of pride), it needs to be decided by the appropriate means, which for us is a court of law.  

It used to be the case in GB that hordes of foreigners came ashore from boats, took what they wanted, trashed the restaurants as if they were Bullingdon boys, and took women into slavery. They had to be fought off. When this started being successful, they quit (apart from those who stayed, which ruined their business model another way). Every three-year old who has played in a sandbox knows this phenomenon, which manifestly does not stop when one is older: John Kenneth Galbraith wrote about the power of large corporations and the consequences for human society between 40 and 55 years ago. So there is also another point to this kind of action: resistance stops other people doing stuff.  

Toyota knew they had spaghetti code in this acceleration-control kit. They wrote so themselves, which you can see in the evidence. They also knew and know the consequences of such complexity, namely a lack of control over the behavioral properties of the program. That is also in the evidence. It didn't stop them using the code again and again (it was still in the 2010 model year, apparently). That won't continue. For example, they have recently signed a contract with Altran UK to develop examples of useful code which is free of run-time error.  

If you don't like principles of fairness and responsibility, and developed organisations (the courts) with the power to set those principles of fairness for everyone and every organisation without exception, just try doing without it..........  

PBL Prof. Peter Bernard Ladkin, University of Bielefeld and Causalis Limited

On 12 Nov 2013, at 03:07, "Les Chambers" <les_at_xxxxxx

What bothers me is the alarming repeat performances we have of these disasters. And the eye-watering sums of money spent on forensics and retribution. These events are typically passed over to the legal profession who proceed to dine out on the assignation of blame.......

The System Safety Mailing List
systemsafety_at_xxxxxx Received on Wed Nov 13 2013 - 01:23:44 CET

This archive was generated by hypermail 2.3.0 : Tue Jun 04 2019 - 21:17:06 CEST