Re: [SystemSafety] How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?

From: Martyn Thomas < >
Date: Sat, 23 Apr 2016 16:43:35 +0100

On 22/04/2016 12:10, Mike Ellims wrote:

... ...
> And Hi Martyn
> > Recertification after software change. Or do we just accept the huge
> attack surface that a fleet of AVs presents?
> For “recertification” Goggle’s approach to date seems to be to rerun
> all the driving done so far via simulation… I’m not sure what your
> implying with the comment on attack surfaces. Some far, as far as I
> can tell aside from updates there is not vehicle to vehicle
> communications. GPS is probably vulnerable to spoofing and jamming
> which could be an issue but one would hope that had been accounted for
> as it would count as a sensor failure…

The AVs depend on software that is occasionally updated. They depend on data that is occasionally updated. They depend on sensors that could be jammed, flooded or spoofed. Then (as has already been mentioned) car manufacturers connect other networked systems (bluetooth, phone, radio, TV ...) to internal networks that are also connected to safety-related subsystems. Everything that I have mentioned is a possible channel for cyberattack. When we have a fleet of AVs, that's a huge set of possible vectors for cyberattack (which I referred to as the "attack surface").

Now, let's imagine that Google has carried out exhaustive penetration testing (I know this is impossible - which makes the following argument even stronger) and that we agree that their AV is secure against all possible attacks. Then they release a software change. Re-running all the driving, through simulation, isn't enough. They have to rerun exhaustive pen testing too (which could involve all possible attacks under all possible driving conditions). Recertification feels to me like an important issue and I haven't heard anything that gives me confidence that anyone yet has a feasible approach to a solution.

> > The way in which AVs could change the safety of the total road
> transport system. Is anyone studying total accidents rather than AV
> accidents?
> Yes, lots and lots of people mostly government bodies that that
> collect the accident data in the first place and they tend to
> commission detailed studies from outside organization (that don’t
> quite answer the question your interested in). In addition to that
> there are a few manufacture/academic partnerships that study major
> road accidents in forensic detail alongside police (I know of one in
> Germany and one in the UK) which is intended to address many of the
> limitations to police investigations. In addition some of the big auto
> manufactures have their own departments e.g. VW have their own
> statistics department looking at this. In addition there is a large
> academic community concerned examining traffic accidents.

You misunderstand me - probably because I was not clear enough. I meant to ask whether anyone is currently studying the impact that AVs are having (and will have) on the overall safety of the total road transport system. For example, will the knowledge (by drivers, cyclists, pedestrians ...) that many vehicles are AVs change the behaviour of these other road users in a way that changes the frequency of accidents in which an AV is not deemed to have been at fault (and in which it may not even have been involved)?

To illustrate what I mean with just one, very small, example, cyclists might get used to AVs passing them with a wider clearance than is the normal behaviour of human drivers. (This /should/ happen because the code of acceptable driving - called the /Highway Code/ in the UK, for instance - sets a standard that many drivers currently forget or ignore). This could change cyclists' behaviour, after some time, in a way that leads them to have more accidents with cars that have human drivers. It's possible even that the overall rate of accidents between cars and cyclists would rise as a consequence of introducing AVs, even though the AVs had may fewer accidents with cyclists than the average for non-AVs before their introduction.

Should there be a safety argument that the introduction of AVs will not reduce the safety of the road transport system, rather than a safety argument that AVs are as safe or safer than cars driven by humans?



The System Safety Mailing List
systemsafety_at_xxxxxx Received on Sat Apr 23 2016 - 17:45:40 CEST

This archive was generated by hypermail 2.3.0 : Tue Jun 04 2019 - 21:17:08 CEST