Re: [SystemSafety] Practical Statistical Evaluation of Critical Software

From: Derek M Jones < >
Date: Sun, 01 Mar 2015 23:30:17 +0000


Matthew,

> Isn't the model implicitly normative? After all we're talking about testing
> a designed thing over which we have control, rather than the world.

We are testing a thing over which we don't have full control, it contains faults.

The World is where the input that has not been used during testing and which will trigger one or more remaining faults will come from.

> So from that perspective the approach is to tell engineers that if you do
> A,B,C then your software's behaviour is going to be close enough to a
> memoryless process such that the statistical techniques of the annex are
> useful.

I could propose a fault model that was very similar to phone calls being received at a telephone exchange; I'm sure it could be made plausible. The maths has already been worked out and it involves the Erlang distribution (which is the sum of k independent, memory-less distributions).

Where is the data to show that use of Bernoulli/Poisson gives more accurate predictions than Erlang? It might in some cases and not in others.

We are a long way from having a good theory of software faults. It is too soon to recommend a single model for fault analysis.

It would be wise to provide a few pointers and not force people down specific paths of analysis.

> Of course A, B, C might not be very well articulated, and the intent
> implicit rather than explicitly stated. I don't know as I haven't read the
> Annex (apologies).
>
>
> On Mon, Mar 2, 2015 at 5:21 AM, Derek M Jones <derek_at_xxxxxx >
>> Peter,
>>
>> Lets apply a well known adage:
>> "All models are wrong but some are useful".
>>
>> Lets put to one side the extent to which the proposed model is wrong.
>>
>> I don't see how the proposed model is useful.
>>
>> The use of Bernoulli/Poisson mathematics is proposed and you correctly
>> point out that this only works if the data has the desired properties.
>>
>> I would have proposed any combination of distributions, said the same
>> thing and been just as correct as you.
>>
>> Surely the approach should be to tell engineers to find out what
>> distribution(s) their data has and then apply the probability
>> analysis appropriate to that distribution(s)?
>>
>>
>>
>> As I mentioned last week, Bev Littlewood and I have been writing a short
>>> practical guide to
>>> statistical evaluation of software with high-reliability requirements. It
>>> has been apparent to us
>>> for a while that IEC 61508-7 Annex D is an insufficient and in some
>>> respects misleading guide to the
>>> statistical evaluation of critical software, and it's been there in the
>>> standard by now for 18
>>> years. Time to fix that. This is the material we think should go into a
>>> revision.
>>>
>>> It's available at http://www.rvs.uni-bielefeld.de/publications/Papers/
>>> LadLitt20150301.pdf and has
>>> also been submitted for publication.
>>>
>>> PBL
>>>
>>> Prof. Peter Bernard Ladkin, Faculty of Technology, University of
>>> Bielefeld, 33594 Bielefeld, Germany
>>> Je suis Charlie
>>> Tel+msg +49 (0)521 880 7319 www.rvs.uni-bielefeld.de
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> The System Safety Mailing List
>>> systemsafety_at_xxxxxx >>>
>>>
>> --
>> Derek M. Jones Software analysis
>> tel: +44 (0)1252 520667 blog:shape-of-code.coding-guidelines.com
>>
>> _______________________________________________
>> The System Safety Mailing List
>> systemsafety_at_xxxxxx >>
>
>
>

-- 
Derek M. Jones           Software analysis
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com
_______________________________________________
The System Safety Mailing List
systemsafety_at_xxxxxx
Received on Mon Mar 02 2015 - 00:30:18 CET

This archive was generated by hypermail 2.3.0 : Thu Apr 25 2019 - 17:17:07 CEST