Satisfaction and Perceived Usefulness make up Hypothesis 4 and 5.
H4: Use of a software cost estimation tool for software development projects increases the users’ satisfaction with the estimation technique.
H4a: The PSEstimate group will have higher satisfaction than COCOMO II group and the COCOMO II group will have higher satisfaction than the Manual group.
H4b: The COCOMO II and PSEstimate group together will have higher satisfaction than the Manual group.
H5: Use of a software cost estimation tool for software development projects increases the user’s perceived usefulness with the estimation technique.
H5a: The PSEstimate group will have higher perceived usefulness than COCOMO II group and the COCOMO II group will have higher perceived usefulness than the Manual group.
H5b: The COCOMO II and PSEstimate group together will have higher perceived usefulness with the estimation technique than the Manual group.
The constructs for satisfaction and perceived usefulness will be analyzed with identical analysis techniques. The first step in checking for differences among the three treatment group for satisfaction and perceived usefulness is to conduct two psychometric tests on the items that measure these two constructs.
An item-total along with Cronbach’s alpha is a standard technique to test reliability. The results of these tests are shown in Table 8-16 and Table 8-17:
Item Wording Scale Item-Total Cronbach’s Alpha
Very Dissatisfied---Very Satisfied 1-7 .81
Very displeased---Very Pleased 1-7 .92
Very frustrated--- Very contented 1-7 .85 Absolutely terrible—Absolutely
delighted 1-7 .75
.93
Table 8-16 Item-Total for Satisfaction
Item Wording Scale Item-Total Cronbach’s Alpha Using the software estimation technique in
this experiment improves my performance in conducting software cost estimation.
1-7 .94 Using the software estimation technique in
this experiment improves my productivity in conducting software cost estimation.
1-7 .94 Using the software estimation technique in
this experiment improves my effectiveness in conducting software cost estimation.
1-7 .95 Overall, the software technique used in this
experiment was useful in conducting software cost estimation.
1-7 .88 .97
Table 8-17 Item-Total and Cronbach’s Alpha for Perceived Usefulness The results show strong item-total correlation for the items with the constructs.
Cronbach’s alpha is excellent for both constructs.
Having very reliable measures, a new measure called TotalSat was created that is a summation of the four satisfaction items. Also another measure TotalUse was created that was also a summation of the four perceived usefulness items. These variables are used as dependent variables in the analysis.
A bootstrapping technique was used to test the two hypotheses with the newly created dependent variables TotalSat and TotalUse. The results follow:
Treatment Satisfaction (TotalSat) Perceived Usefulness (TotalUse) Manual Mean = 12.5
Std. Dev = 4.14
Mean = 12 Std. Dev = 6.6 COCOMOII Mean = 17.7
Std. Dev = 4.1
Mean = 20 Std. Dev = 5.2 PSEstimate Mean = 16.5
Std. Dev = 4.3
Mean = 16 Std. Dev = 5.0
Table 8-18 Satisfaction and Treatment Means
Based on Table 8-18 hypothesis H4a and H5a are not supported. COCOMO II had a higher satisfaction and perceived usefulness than both the PSEstimate and manual group. By combining the COCOMO II and PSEstimate group together, a tool versus no tool analysis can be conducted. This test can be used to test hypothesis H4b and H5b.
Contrast Satisfaction Perceived Usefulness
Tool vs. No Tool .014 .012
Table 8-19 Bootstrap p-vals for Satisfaction and Perceived Usefulness From Table 8-19 there is support for Hypothesis H4b and H5b. The satisfaction and perceived usefulness of using an estimation tool was significantly better than not using an estimating tool or conducting the estimation manually.
The results tend to show two different underpinnings when analyzing satisfaction and perceived usefulness. The first thing is individuals do not like to do estimation manually. It is rather frustrating for some and many people do not think it is an effective use of their time. This can result in a low rating for satisfaction and perceived usefulness.
carried over to the main experiment. The second result is people really liked using COCOMO II to do estimating. Maybe because it is the state-of-the-practice tool, but whatever the reason, people report high satisfaction and perceived usefulness with COCOMO II. The PSEstimate group was inconclusive. It was not significantly different from the manual group or the COCOMO II group. The PSEstimate group was almost significantly different from the Manual group. The raw p-values were .03 and .07 when comparing the Manual group versus the PSEstimate group. The bootstrapping reduced the p-values to non-significant results. Some additional work needs to be conducted to see what is causing satisfaction and perceived usefulness to lag slightly below COCOMO II.
From the results is it clear that PSEstimate needs more development, most likely in the interface. PSEstimate is addressing a much more complex task in the explict modeling of team structure versus the COCOMO II version. A better way of inputting team information can affect the perceived usefulness scores.