Re: [SystemSafety] Automobile Safety-Critical Kit (Bookout v. Toyota Motor transcript)

From: Heath Raftery < >
Date: Fri, 01 Nov 2013 07:38:04 +1100

I read the transcript as best I could yesterday. To read it word for word would likely take a couple of days, so I skimmed. Interestingly, the excerpts in the EE Times article I did read verbatim - I think that section was not only of greatest interest to engineers, but probably ended up having the greatest impact on the court.

Mr Barr seems, based on the transcript, to be an excellent code reviewer. He's also an excellent communicator. He put together a very convincing case that the code was unsatisfactory. Not only was the quality of the code poor, their review and tracking procedures were sorely lacking. This is beneath what we would want to believe a car manufacturer adheres to.

On the other hand, I remain unconvinced the software had anything to do with the crash. The driver was 76 years old at the time. This crash was subject to an NTSB investigation, and investigators found no evidence that it was a software fault or a hardware fault. The crash recorder says the driver pushed the accelerator and was not pushing the brakes, and then the car was hit. Even Mr Barr's demonstration of a potential fault scenario (inject a bit flip, task dies, cruise control is resumed while the brake is lightly applied - acceleration continues past setpoint), while clearly a bug that should be fixed, is very hard to imagine as relevant to the crash.

I believe Mr Barr rightly convinced the court that the software development project lacked quality, and then, by virtue of some terrible cross-examination, he leads the jury to assume that the software bugs could have caused the accident.

I suspect, with no proof, that the code reached that state iteratively - I could well imagine it started with the best intentions and best practices. Then, as budgets dwindled, management got jumpy, and feature request/bug fixes got thrown in at the last minute, the code got messy and the review process fell away. The team probably consoled themselves by enormous system tests - in 1000's of km of testing, not one issue, so the crappy code works. It looks like rubbish, but it works.

Then, an old lady hits the accelerator instead of the brake, the code gets presented in court, a jury of laypeople looking for someone to blame are shocked at what actually runs the cars we drive and boom - Toyota is at fault in an accident causing death.

In the end I think people will have to start to expect software to cost more - we need these laborious (and rather dull, frankly) safety practices for two reasons: the first is that they can help build in quality (cue endless argument about whether they are the best way or even an effective way) and second because as engineers we're responsible for people's lives and sooner or later we need *evidence* that we've taken that into consideration.

It's an incredibly timely episode for our workplace, where we develop hardware and firmware for safety critical applications in another field entirely (mining) - there's this uneducated push to "go SIL" but no one appreciates the learning curve and the cost, and when push comes to shove, we just resort back to getting the job done as quickly as possible. That's starting to change.


On 1/11/2013 5:30 AM, Chris Hills wrote:
> The complete transcript.
> *From:*systemsafety-bounces_at_xxxxxx > [mailto:systemsafety-bounces_at_xxxxxx > Of *Chuck_Petras_at_xxxxxx > *Sent:* 31 October 2013 17:57
> *To:* systemsafety_at_xxxxxx > *Subject:* Re: [SystemSafety] Automobile Safety-Critical Kit (Bookout v.
> Toyota Motor transcript)
> Some excerpts from the court transcript...
> Toyota Case: Inside Camry’s Electronic Control Module,
> <>
> Chuck Petras, PE**

The System Safety Mailing List
systemsafety_at_xxxxxx Received on Thu Oct 31 2013 - 21:38:21 CET

This archive was generated by hypermail 2.3.0 : Tue Jun 04 2019 - 21:17:06 CEST