Re: [SystemSafety] Therac-25 redux

From: Les Chambers < >
Date: Mon, 18 Aug 2014 11:30:43 +1000

Thanks Peter

Vita brevis, ars longa.

I guess Hippocrates was right. In a single lifetime no one person will ever accumulate the knowledge of what could go wrong in any domain; a single life is too short. Collective eternal memory requires collaboration (across generations, across millennia) and an accessible body of knowledge. Don't say it can't be done, if Hippocrates (460 – 370 BCE) can still be quoted today so can modern wisdom.
For the first time ever the web has made eternal collaborative memory possible. When I published this book on Amazon: ... it occurred to me that it will never go "out of print"; the author, Rex Kimlin is in his nineties and won't be with us for much longer. But the thoughts of a man who risked his life 35 times with a 50 percent probability of death will be with us for as long as Amazon exists (is Bezos immortal?). The same is true for Harold Thimbleby's book (Press On). It looks great by the way, anyone extolling the virtues of state engines for interactive design has got to be a righteous dude.
Well written books on system safety are a great thing. But I think we need to take publishing-what-could-go-wrong a step further. And that is to make the kernels of wisdom present in all these books more accessible in an abbreviated form, expressed as an ontology. For example, if pressed, I could probably reduce this tome: to and A4 page of managerial bad behaviour patterns that lead to disaster; how to recognise them; how to overcome.
Published on the Web this kind of thing would be invaluable to anyone doing a hazard analysis. Ph.D. thesis material perhaps? How would you structure such a thing? A wiki for disaster. How would you triumph over the guardians of polite capitalism and move beyond the pettiness of "I won't divulge my hazards in case someone steals my stuff." It seems a shame that the outcome of collective memory loss and ignorance should be the death of innocents. By the way, Caesar Augustus died 2000 years ago tomorrow (Tue 19 August). We have him to thank for the month of August. Let us not forget.

-----Original Message-----
From: systemsafety-bounces_at_xxxxxx [mailto:systemsafety-bounces_at_xxxxxx Peter Bernard Ladkin
Sent: Saturday, August 16, 2014 5:13 PM
To: systemsafety_at_xxxxxx Subject: Re: [SystemSafety] Therac-25 redux

On 2014-08-16 06:10 , Les Chambers wrote:
> A quick search of the Internet did not reveal any publication of
drug-infusion pump hazards. Is
> anyone aware of same?

Yes we are (I much less than some others here). Vulnerabilities with medical devices, especially
implantable medical devices, are a big thing. Harold Thimbleby at Swansea has been working on it for
a couple decades. He's primarily an HMI guy but we wrote a couple papers on security and safety. He
has an award-winning book on interface design with MIT Press called Press On.

Ross Anderson at Cambridge is aware of the issues with medical device security, but works primarily
in other areas. He was at Black Hat this year, where the IOActive stuff on the Cobham kit was
presented. He does have some strong opinions on the state of the practice in medical-device security

Barnaby Jack was one of the best known (that is, notorious) security thespians. I understand he was
about to demo defibrillator and infusion-pump vulnerabilites at Black Hat last year when he
overdosed himself on recreational and other drugs a week before. He has a Wikipedia page, which one
can be sure was not written by him :-)

You have to be somewhat careful of the "security theatre" surrounding medical-device
vulnerabilities. I am told that patient welfare is not being well served by the current addiction to
media exposure. Indeed, it is one of the three topics in the paper I wrote a week ago for the
upcoming SSS in February in Bristol . (One of the others is, by
request, MH 17. It is about how one might do security risk assessment. It turns out to be different
in some crucial ways from safety risk assessment.)

There are indeed stories to be told, and recently I have been reading some. Neither Harold nor Ross
is on this list, but one of our lurkers is a renowned expert on medical-device safety and security.

One of the big problems, not well served by security theatre, is that some of the implantable kit
was designed and implanted quite a while ago, before people paid that much attention to the kind of
antics security thespians can get up to nowadays. But fixing it, that is, updating a device,
requires more surgery, which is not without risk and of course considerable inconvenience to the
patient. That has to be balanced against the chances that some jerk behind you in the line at
Starbucks will reprogramme your defibrillator with a phone.

I would imagine that one of the reasons this topic is hitting the press now is, as The Economist
hinted, the US FDA appears to be embarking on a push to get this all sorted.

> This brings me to my point: wouldn't it be great if we had a readily
accessible ontology of hazards
> for various application domains. It's an obvious idea. Is anyone aware of
discussions along these
> lines?

I think, from what I understand, that that's part of the FDA plan. It's certainly part of general EU
planning: Ross Anderson, Rainer Böhme, Richard Clayton and Tyler Moore
have a report "Security Economics and the Internal Market" for ENISA, which says as one of its first
recommendations that there should be vulnerability databases with compulsory notification requirements.

> "Open source" hazard ontologies would solve the problem of corporate
memory loss, amnesia and
> denial.

Yes, but I doubt there is any chance. Too much proprietary information is involved for any effective
vulnerability catalogue to be public.

PBL Prof. Peter Bernard Ladkin, Faculty of Technology, University of Bielefeld, 33594 Bielefeld, Germany
Tel+msg +49 (0)521 880 7319

The System Safety Mailing List

The System Safety Mailing List
systemsafety_at_xxxxxx Received on Mon Aug 18 2014 - 03:31:23 CEST

This archive was generated by hypermail 2.3.0 : Wed Feb 20 2019 - 02:17:07 CET