Re: [SystemSafety] Software reliability (or whatever you would prefer to call it)

From: Nick Tudor < >
Date: Tue, 10 Mar 2015 08:24:02 +0000


Hi Steve

To put another view, Peter said (re the notion of software reliability "One issue is whether (a) the notion makes sense. Another issue is, (b) if it makes sense, whether
some of its characteristics are isomorphic to those of Bernoulli's Urn Model. The answer to both
those questions is yes."

The other view is that the answer to question a) is emphatically no, which makes question b) somewhat redundant, especially as it is trying use some maths incorrectly to justify an observed behaviour of a system not solely the software.

Nick Tudor
Tudor Associates Ltd
Mobile: +44(0)7412 074654
www.tudorassoc.com

*77 Barnards Green Road*
*Malvern*
*Worcestershire*
*WR14 3LR*
*Company No. 07642673*
*VAT No:116495996*

*www.aeronautique-associates.com <http://www.aeronautique-associates.com>*

On 10 March 2015 at 08:12, Peter Bernard Ladkin <ladkin_at_xxxxxx
> wrote:

> On 2015-03-09 18:48 , Steve Tockey wrote:
> > *) Software itself is inherently deterministic.
>
> This is a point repeatedly emphasised by Nick Tudor. There is a simpler
> way to say it: SW behaves in
> this case as a mathematical function.
>
> > *) The underlying hardware should also be—for the most
> part—deterministic.
>
> Dear me. We just heard yesterday Kevin Driscoll's talk "Murphy was an
> Optimist". I first heard it in
> 2010 at SAFECOMP in Vienna. Kevin now has a nine-hour tutorial on it. It's
> about all the ways in
> which systems could go wrong in ways not anticipated by the designers.
>
> Byzantine failures are just the start. In the 1980's, when they first
> appeared in the computer
> science literature, they were judged to be theoretical constructs of no
> practical interest. In the
> later 1990's, people were beginning to have second thoughts, and in
> SAFECOMP 2003 Kevin described
> how a major commercial aircraft type almost lost its airworthiness
> certificate because of a burst of
> Byzantine failures.
>
> Kevin makes the point that (if I may phrase it so) "digital is just an
> approximation of analogue
> processes".
>
> So I think whether one accepts your statement depends upon what one takes
> to be covered by the
> phrase "for the most part".
>
> > So the issue really isn't "reliability of the software" per se,
>
> One issue is whether (a) the notion makes sense. Another issue is, (b) if
> it makes sense, whether
> some of its characteristics are isomorphic to those of Bernoulli's Urn
> Model. The answer to both
> those questions is yes.
>
> Indeed, as I noted yesterday, there is a well-defined notion of function
> reliability (but I didn't
> call it that), in which you compare a candidate function with a norm
> function. There is no mention
> of SW in the definition of function reliability. But if you take SW to be
> rigorously deterministic,
> the definition of function reliability applies to that software.
>
> Suppose your SW is indeed deterministic. Bertrand pointed out yesterday
> that that the phenomenon
> captured in function reliability is only one of three ways in which the
> output you get from your SW
> is or is not what you had hoped for. Depending on your depth of analysis,
> it may also include
> specification reliability (how often the software gives wrong results when
> its result conforms with
> its specification, or when the specification does not determine what the
> result should be) and mixed
> reliability (in which a HW anomaly may also be causally involved).
>
> And, of course, your SW may in fact not be deterministic. The
> non-determinism might well contribute
> to the success or failure of your SW on particular inputs.
>
> > the issue is "reliability of the software under input space S".
>
> I think you are missing a crucial distinction. There are two parameters,
> as I noted in previous. The
> domain of input is important, but so is - crucially - the distribution of
> those inputs in your sample.
>
> > If one had the ability to characterize input space S for some software,
> and if someone were able to
> > comprehensively cover all possible inputs (in all possible sequences…)
> then one could truly measure
> > the "reliability" of the code with respect to S—"What percent of the
> members of input space S lead
> > to failures?", or, "On average, what time period passes between members
> of input space S that lead
> > to failures?"
>
> Not as I interpret your one parameter "input space". How often the
> software succeeds or fails is
> dependent crucially on the input sample. As I have pointed out, if there
> is at least one input on
> which the software succeeds and one input on which the software fails, you
> can make the function
> reliability anything between 0% and 100% by varying the input distribution.
>
> > The problem is that the input space S for any non-trivial program is
> massive, so ...
>
> I think for this entire paragraph to make much sense as an argument, you
> first need to distinguish
> between input domain and the distribution of inputs to the SW.
>
> > a) some are arguing that software is deterministic so reliability is
> moot. True to an extent, but it
> > depends on the chosen input space S
>
> No. Not true to any extent. The notion of function reliability is well
> defined.
>
> > b) some are arguing that if input space S' is a fair and accurate
> representation of input space S
> > then conclusions about reliability of software under S can be inferred
> from reliability observed
> > under S'
>
> To judge this, I think I would need to know what you are meaning by "input
> space".
>
> > c) some are arguing that there's no way (today) to ever know if input
> space S' is a fair and
> > accurate representation of input space S, so any conclusions about
> reliability of software under S'
> > aren't necessarily transferable to the same software under input space S
>
> I think that it would be wrong to suggest that there aren't ways of
> knowing.
>
> PBL
>
> Prof. Peter Bernard Ladkin, Faculty of Technology, University of
> Bielefeld, 33594 Bielefeld, Germany
> Je suis Charlie
> Tel+msg +49 (0)521 880 7319 www.rvs.uni-bielefeld.de
>
>
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety_at_xxxxxx >



The System Safety Mailing List
systemsafety_at_xxxxxx Received on Tue Mar 10 2015 - 09:24:10 CET

This archive was generated by hypermail 2.3.0 : Tue Jun 04 2019 - 21:17:07 CEST