Re: [SystemSafety] Practical Statistical Evaluation of Critical Software

From: Peter Bernard Ladkin < >
Date: Mon, 02 Mar 2015 08:44:22 +0100


to my mind, your contentions show that you don't really understand the difference between abstraction (or description) and modelling. I wrote about this some twenty years ago. It keeps coming up, indeed the last time was four weeks ago in Bristol.

People who build models try to make something which "looks like" the world as they conceive it. Some bits fit, some bits may not - usually there are some such bits. They may do better or they may do worse.

But how do they tell if their models are any good? Or where they do well or worse? The answer is that models are compared with "reality". Now that's interesting - how does one "compare" with "reality"?

That is exactly the problem which John Locke had with his theory of perception, and of which Berkeley offered what is still taken to be the definitive criticism. Refined in the meantime by people such as George Moore and John Austin.

To "compare" with "reality", you can't build yet another model, for the same question arises and remains thereby unanswered. "Comparing with reality" is an operation which is not modelling, but which those who claim all-is-modelling do not describe.

If they try to do so, they invariably encounter description or abstraction. Abstraction is a matter of selecting out common features between the "reality" you are attempting to grasp and some construct you can manipulate, such that the manipulations of the construct correspond to actions in the "reality" in some understandable way. (This is an attempt to put in one sentence something that takes rather longer to set out carefully. I know it sounds circular in this sentence.)

I prefer to call it "description", because people generally know what a description is and what its characteristics are, and that the point of a description is to be true of whatever one is describing. That is, roughly speaking, the problem which Locke had - he couldn't account for describing. When you correctly describe some object as "just blue, and not another color", you can infer that object is not "black and not another color" and your inference is guaranteed to be true:
(all-)blue and (all-)black are contraries.

So if you have a pile of oranges and apples on the table, and you want to divide them as equally as possible amongst the five kids at the birthday party, then arithmetic and arithmetic operations might occur to you as a useful abstraction - as a useful description for what you want to do.

Abstraction is different from modelling, at least in that, when you draw a conclusion from calculating with an abstraction, that conclusion is true (providing the abstraction is valid). Whereas when you are working with a model, you don't generally know whether it is or not. You just hope it is, just like most of the rest of your model.

So, when you draw conclusions from your arithmetic about how the apples and oranges are to be divided most equally amongst the five children, you know that that is the correct answer - it is guaranteed to be true (provided you did your arithmetic correctly). So you can just go ahead and do it.

If there is a car accident outside in the road - somebody braked on the curve, went off the road and hit a wall, and you want to try to estimate how fast the car was going, then arithmetic is not going to help you much. It's going to "apply" just as much as it does to apples and oranges, but it's not going to get you anywhere near an answer to your question. Newtonian mechanics is going to do a lot better. Ah, someone says, but Newtonian mechanics is a *model* - it makes assertions that we know are *not true*. Well, I would reply, in this case not - it's really being used as an abstraction. Yes, there are errors in Newtonian calculations which relativistic mathematics can highlight, but those errors are bounded, and those error bounds can be propagated mathematically through the calculations, enabling you to state your conclusion, which is your Newtonian-calculated figure within bounds which your error calculation says pertain. That conclusion is as much guaranteed-true as in the apples and oranges case with arithmetic.

Despite this issue being, essentially, over three hundred years old, and successfully (partly) understood then, it recurs nowadays. Brian Cantwell Smith wrote an essay in the 1990's in which he claimed you can't prove software correct, in which he made the Lockean veil-of-perception mistake. James Fetzer did something similar. He talked about software being "correct" or not, without saying correct with respect to what, and drew political conclusions which were politically unacceptable to many computer scientists. Nobody pointed out in print the fallacy in his argument. He claimed (I paraphrase) you couldn't mathematically guarantee anything to be true about real-world things. That's nonsense. Here's my fridge. I'm going to prove to you that my fridge is correct. In order to do that, I've got to try to tell you what it is for my fridge to be correct (or not). So here goes: to be correct, my fridge is to fulfil the condition P(x) OR NOT-P(x). (Take P to be any decidable property of my fridge.) It is then a trivial exercise to *prove mathematically* the correctness of my fridge.

(The point of that exercise, obviously, is not to show anything useful about my fridge. It is to
show that, whatever his reasoning might be, Fetzer's conclusion is mistaken.)

Nevertheless, despite regular attempts to highlight the essential difference between description (or abstraction) and modelling, people still don't get it.

For example, Nancy Leveson. It came up again in the question session after my talk in Bristol. I had briefly indicated how Causal Fault Analysis can be used along with a judgement of the values of quasi-Boolean variables in order to get a pruned - and I would claim helpful - decision graph about the risks where there is much uncertainty (which applies to security risks in particular). Simon Whiteley asked me if I had considering using STAMP, and if not why not. I said that STAMP is a model, and CFA is a description technique, and (truth be told, I don't know whether I actually stated this next - maybe Simon or someone else remembers) my aim is generally to describe causal situations so that conclusions were guaranteed to be true given the truth of necessary assumptions, whereas if one uses a model such as STAMP, one has to "buy into" the model (in particular the "feedback model" of causality, which in contrast with the counterfactual interpretation used in WBA and CFA has not yet withstood four decades of being hacked on by some of the smartest logical minds), and thereby are the conclusions not guaranteed to represent "the world" - as one says, they are only as good as the model might be. Nancy didn't like that. She is of the camp in which everything is "modelling". I deferred a discussion on the issue of modelling versus description (and if I hadn't, I'm sure Mike would have :-) ). I think if people can't acknowledge the difference over twenty years, they are not going to do so in five minutes of repartee after a talk on something else.

She asked: how can I deal with human factors? (With WBA or CFA. Actually, she didn't ask, she simply claimed I couldn't. I'm pretending she phrased it as a question.) Answer: like WBA/CFA deals with anything else - put some in if you need them. (As Karsten Loer did in his WBA Diploma thesis from seventeen years ago, which is Section 2 of the Causal Systems Analysis book which has been available on the WWW for fourteen years.) Nancy maintains, however (at least she did afterwards in Bristol) that I "can't deal" with human factors.

I think it would help system engineering a lot to get clear on the difference between modelling and description.

I also think this is what Matthew was getting at in his note. The Urn Model has that unfortunate word "Model" in its name, but in fact it is easy to extract the descriptive elements from the story: "balls" don't need to be balls; they don't need to be "painted white (or black)"; there just need to be selectable entities of two sorts, and exactly the same entities must be available for selection in exactly the same way, repeatedly, and the selection action must satisfy certain conditions. Certain kinds of software fulfils that description, just as the apples and oranges on the table fulfil the arithmetic description. And for the software which fulfils that description, the conclusions about time to first failure and MTTF and so on, derived from the mathematics, are valid.

You want to say "not good enough for me". OK. Maybe you want to use Erlang distributions or whatever they are called. Fine. Go ahead. Nothing wrong with that if you can do it. If you can show how those distributions describe the execution of software in some fundamental way, and the math gives you useful conclusions.

But there is nothing in that observation at all which justifies any criticism of using the Urn Model where it can be used, just as there is nothing in the insufficiency of arithmetic to describe the dynamics of an accident to stop us using it successfully in dividing up apples and oranges equally amongst five kids.

PBL Prof. Peter Bernard Ladkin, Faculty of Technology, University of Bielefeld, 33594 Bielefeld, Germany Je suis Charlie
Tel+msg +49 (0)521 880 7319

The System Safety Mailing List
systemsafety_at_xxxxxx Received on Mon Mar 02 2015 - 08:44:29 CET

This archive was generated by hypermail 2.3.0 : Tue Jun 04 2019 - 21:17:07 CEST