ref: Bishop, P.G. “Rescaling Reliability Bounds for a New Operational Profile”, International Symposium on Software Testing and Analysis (ISSTA 2002), vol. 27 (4), pp. 180-190,2002 http://www.adelard.com/papers/issta2002_bishop_rescale.pdf
If the assumptions made are valid, you can get a massive increase in failure rate if you switch to a new, radically different, profile. On the other hand, if you deliberately test to get "fair" coverage, the rescaled failure rate can be pretty insensitive to changes in profile (even drastic ones).
Obviously, such tests is not possible from operating experience, but use in different profiles gets a bit closer to that ideal
Peter Bishop
Adelard LLP
Peter Bernard Ladkin wrote:
>
> On 6/27/13 4:23 PM, Nancy Leveson wrote:
>> Someone [Metthew Squair] wrote: >> > I've been thinking about Peter's example a good deal, the developer >> seems to me to have made an >> > implicit assumption that one can use a statistical argument based on >> successful hours run to justify >> > the safety of the software. >> And Peter responded: >> > It is not an assumption. It is a well-rehearsed statistical argument >> with a few decades of >> > universal acceptance, as well as various successful applications in >> the assessment of emergency >> > systems in certain English nuclear power plants. >> >> "Well-rehearsed statistical arguments with a few decades of universal >> acceptance" are not proof. >> They are only well-rehearsed arguments. Saying something multiple >> times is not a proof.
>> I agree with the original commenter about the implicit assumption, >> which the Ariane 5 case disproves >> (as well as dozens of others).
>> Perhaps the reason why software reliability modeling still has pretty >> poor performance after at >> least 40 years of very bright people trying to get it to work is that >> the assumptions underlying it >> are not true.
>> When someone wrote: >> > I don't think that's true, >> Peter Ladkin wrote: >> >>You might like to take that up with, for example, the editorial >> board of IEEE TSE. >> >> [As a past Editor-in-Chief of IEEE TSE, I can assure you that the >> entire editorial board does not >> read and vet the papers, in fact, I was lucky if one editor actually >> read the paper. Are you >> suggesting that anything that is published should automatically be >> accepted as truth? That nothing >> incorrect is ever published?]
This archive was generated by hypermail 2.3.0 : Sun Feb 17 2019 - 09:17:06 CET