Re: [SystemSafety] Qualifying SW as "proven in use" [Measuring Software]

From: Derek M Jones < >
Date: Fri, 21 Jun 2013 14:29:05 +0100


> Getting this evidence is pretty tricky, as parallel developments for the
> same project won't happen.

NASA spends hundreds of millions to take a few snaps of a distant planet, we need software engineering researchers with some vision and drive (and connections).

Lets say $1million for a project, run this 20 times in parallel to get some statistical significance and add in say $5 million to cover the research side. $25 million, peanuts ;-)

Of course NASA funding is really a way of channeling money to political constituencies. It might just happen that the random selection of those 20 project locations appear to follow the NASA pattern, not a problem.

For those who currently don't have $25 million in funding... The SQALE document, along with many other documents of this kind, list rules of the form "don't use XYZ".
The XYZ listed by SQALE are among the usual suspects.

If developers cannot use XYZ then they will have to use some other construct. I have yet to see any analysis that compares a XYZ against the alternatives (I work in source code analysis and keep my eyes open for such work).

For a lot less than $25 million an experiment comparing developers allowed to use a particular XYZ vs. those not allowed to use XYZ could be run.

It might be the case that while XYZ is bad it is actually better than the alternatives.

> But you might be able to infer something on average over multiple projects.
> Derek M Jones wrote:
>> Thierry,
>>> To answer your questions:
>>> 1) Yes, there is some objective evidence that there is a correlation
>>> between a low SQALE index and quality code.
>> How is the quality of code measured?
>> Below you say that SQALE DEFINES what is "good quality" code.
>> In this case it is to be expected that a strong correlation will exist
>> between a low SQALE index and its own definition of quality.
>>> For example ITRIS has conducted a study where the "good quality" code
>>> is statistically linked to a lower SQALE index, for industrial
>>> software actually used in operations.
>> Again how is quality measured?
>>> No, there is not enough evidence, we wish there would be more people
>>> working on getting the evidence.
>> Is there any evidence apart from SQALE correlating with its own
>> measures?
>> This is a general problem, lots of researchers create their own
>> definition of quality and don't show a causal connection to external
>> attributes such as faults or subsequent costs.
>> Without running parallel development efforts that
>> follow/don't follow the guidelines it is difficult to see how
>> reliable data can be obtained.

Derek M. Jones                  tel: +44 (0) 1252 520 667
Knowledge Software Ltd
Software analysis     
The System Safety Mailing List
Received on Fri Jun 21 2013 - 15:29:11 CEST

This archive was generated by hypermail 2.3.0 : Sun Feb 17 2019 - 17:17:05 CET