曝光台 注意防骗
网曝天猫店富美金盛家居专营店坑蒙拐骗欺诈消费者
programmers were asked also to estimate the reliability of their programs. Those who did gave estimates of
0.75, 0.8, 0.85, 0.85, 0.87, 0.9, 0.9, 0.9, 0.95, 0.95, 0.95, 0.97, 0.975, 0.98, 0.98, 0.995, 0.998, 0.998, 0.999,
0.999, and 1.0. Most of the programs were more reliable than the programmers estimated.
The effort-level estimates obtained from the work logs are necessarily approximate since the
programers maintained the logs themselves. They were asked to record time spent in reading the
requirements specification, designing and implementing the program, and in debugging and testing the
program. The reading time varied from one to 35 hours with an average of 5.4; the design time from four
to 50 hours with an average of 15.7; and the debugging time from four to 70 hours with an average of 26.6.
- 9 -
4. EXPERIMENTAL RESULTS
For each test case executed, each program produces a 15 by 15 Boolean array, a 15 element Boolean
vector, and a single Boolean launch decision, for a total of 241 results. The program calculates these results
from the simulated radar tracking data and various parameters, all of which are randomly generated for
each test case. The launch condition is the only true output in this application. The other results are really
intermediate although they must be produced since the specifications require them as part of the
determination of the launch condition. For the programs written for this experiment, all these results must
be supplied to the driver program during testing to allow for error detection. We record failure for a
particular version on a particular test case if there is any discrepancy between the 241 results produced by
that version and those produced by the gold program, or the version causes some form of exception (such as
negative square root) to be raised during execution of that test case.
The quality of the programs written for this experiment is remarkably high. Table 1 shows the
observed failure rates of the twenty seven versions. Of the twenty seven, no failures were recorded by six
versions and the remainder were successful on more than 99% of the tests. Twenty three of the twenty
seven were successful on more than 99.9% of the tests.
Table 2 shows the number of test cases in which more than one version failed on the same input. We
find it surprising that test cases occurred in which eight of the twenty seven versions failed.
Where multiple failure occurred on the same input, it is natural to suspect that the failures occurred in
the versions supplied by only one of the universities involved. It might be argued that students at the same
university have a similar background and that this would tend to cause dependencies. However, the exact
opposite has been found. Table 3 shows a correlation matrix of common failures between the versions
supplied by the two universities. For table 3, and for table 1, versions numbered 1 through 9 came from
UVA and versions numbered 10 through 27 came from UCI. A table 3 entry at location i, j shows the
- 10 -
Table 1 - Version Failure Data
Version Failures Pr(Success) Version Failures Pr(Success)
1 2 0.999998 15 0 1.000000
2 0 1.000000 16 62 0.999938
3 2297 0.997703 17 269 0.999731
4 0 1.000000 18 115 0.999885
5 0 1.000000 19 264 0.999736
6 1149 0.998851 20 936 0.999064
7 71 0.999929 21 92 0.999908
8 323 0.999677 22 9656 0.990344
9 53 0.999947 23 80 0.999920
10 0 1.000000 24 260 0.999740
11 554 0.999446 25 97 0.999903
12 427 0.999573 26 883 0.999117
13 4 0.999996 27 0 1.000000
14 1368 0.998632
Table 2 - Occurrences of Multiple Failures
Number Probability Occurrences
2 0.00055100 551
3 0.00034300 343
4 0.00024200 242
5 0.00007300 73
6 0.00003200 32
7 0.00001200 12
8 0.00000200 2
- 11 -
Table 3 - Correlated Failures Between UVA And UCI
UVA Versions
1 2 3 4 5 6 7 8 9
10 0 0 0 0 0 0 0 0 0
11 0 0 58 0 0 2 1 58 0
12 0 0 1 0 0 0 71 1 0
13 0 0 0 0 0 0 0 0 0
14 0 0 28 0 0 3 71 26 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 1 0 0 0
17 2 0 95 0 0 0 1 29 0
UCI 18 0 0 2 0 0 1 0 0 0
Versions 19 0 0 1 0 0 0 0 1 0
20 0 0 325 0 0 3 2 323 0
21 0 0 0 0 0 0 0 0 0
22 0 0 52 0 0 15 0 36 2
23 0 0 72 0 0 0 0 71 0
24 0 0 0 0 0 0 0 0 0
25 0 0 94 0 0 0 1 94 0
26 0 0 115 0 0 5 0 110 0
27 0 0 0 0 0 0 0 0 0
number of times versions i and j failed on the same input. In table 3, the rows are labeled with UCI version
numbers and the columns with UVA version numbers. Thus, a non-zero table entry show the number of
common failures experienced by a UVA version and a UCI version. In the preliminary analysis of common
faults, all were found to involve versions from both schools.
中国航空网 www.aero.cn
航空翻译 www.aviation.cn
本文链接地址：
航空资料35(189)
