Re: [SystemSafety] Software reliability (or whatever you would prefer to call it)

From: David Ward < >
Date: Fri, 6 Mar 2015 11:02:37 +0000

Michael's proposal is very similar to the approach contained in ISO 26262 Part 8 Clause 12 "Qualification of software components".

To grossly simplify, it requires generation of confidence through black-box testing (including considering abnormal cases as well as testing against requirements) but for ASIL D evidence of structural coverage is also required.

David Ward (Dr)
Head of Functional Safety

-----Original Message-----
Sent: 06 March 2015 10:56
To: systemsafety_at_xxxxxx Subject: Re: [SystemSafety] Software reliability (or whatever you would prefer to call it)

Martyn suggests that we put the language to one side.

My take on the core problem.

IEC 61508-7 [2010] Annex D "provides initial guidelines on the use of a probabilistic approach to determining software safety integrity for pre-developed software based on operational experience. This approach is considered particularly appropriate as part of the qualification of operating systems, library modules, compilers and other system software."

In effect, I select an appropriate set of test data, run my system for a long time (or run lots of systems for a short time) and conclude - if no failures are detected - that the system is safe.

The longer that I test for, the higher the SIL level that can be assigned to the component that is being evaluated.

In my book, this is Black-Box testing.

If we revise this appendix as Peter proposes, then we may be able to help people to select more appropriate test data (and this may be an improvement) - but this will still be Black Box testing.

If we can't avoid this appendix altogether (and I'm sure that Bertrand is right about this), then we should - surely - be able to require some additional "White Box" assessments, such as code reviews, design reviews, etc (in line with the rest of the standard).

If we can achieve this, I would sleep more easily.


-----Original Message-----
From: systemsafety-bounces_at_xxxxxx [mailto:systemsafety-bounces_at_xxxxxx Sent: 06 March 2015 09:56
To: systemsafety_at_xxxxxx Subject: [SystemSafety] Software reliability (or whatever you would prefer to call it)

I'm puzzled by much of this discussion. Consider this common example:

A company creates a software package and submits it for beta testing by a group of users. Assume that the package reports how often it is used and for how long, and the users report all errors they encounter. Assume there is a single instance of the software on a server that all the users use.

The company corrects some of the errors that are reported.

The company calculates some measure of the amount of usage before failure. Call it MTBF.

The MTBF is observed to increase.

What word shall we use to describe the property of the software that is increasing?

I'd call it "reliability". If you would, too, then how can software reliability not exist?

I don't mind if you want to use a different word to describe the property. Let's just agree one, do a global replace in the offending standards and move on ...

... to discussing a practical upper bound on the "reliability" that can be assessed in this way - and on the assumptions that should be made explicit before using any such assessment as a prediction of future performance.


The System Safety Mailing List

The System Safety Mailing List

Watling Street, Nuneaton, Warwickshire, CV10 0TU, England
Registered in England and Wales No. 402570
VAT Registration  GB 100 1464 84

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

The System Safety Mailing List systemsafety_at_xxxxxx
Received on Fri Mar 06 2015 - 12:02:50 CET

This archive was generated by hypermail 2.3.0 : Tue Jun 04 2019 - 21:17:07 CEST