Perforce Defect Tracking Integration Alpha Programme Report

1. Introduction

This is a summary of the most important results of the alpha programme of the Perforce Defect Tracking Integration project.

The readership of this document is the project staff.

This document is not confidential.

2. Executive Summary

The alpha programme was very successful. It confirmed and clarified our most important requirements. It reduced the importance of some marginal features, so that we can now concentrate on meeting those requirements. It taught us a great deal about the variety of configurations we can expect to see in use.

There are no major changes of direction or planning as a result of the alpha programme.

3. Alpha Programme Goals

The goal of the alpha programme was to gather feedback on the project requirements. This document suggests changes to the requirements, and therefore suggests a plan of development.

The alpha programme also revealed defects in the software -- mainly in the difficulty of installation and configuration (failure to meet requirement 63 and requirement 75). These are not discussed in this document.

4. Alpha Programme Planning

We sent out an invitation to participate in alpha testing on 2000-09-28 [RB 2000-09-28] and received quite a few responses. We selected a few sites to which we could reasonably travel, and asked them for details of their configuration. The results are tabulated below. Note that Perforce and the Professional Services Group of TeamShare also volunteered to go through alpha testing.

We selected Mahi Networks and Quokka Sports to visit. We also arranged for Derek George of TeamShare PSG to accompany us to Mahi Networks.

5. Summary of Results

A full list of about 150 issues discovered during the alpha programme can be found in the alpha test reports [GDR 2000-10-23], [RB 2000-10-24], [GDR 2000-10-26], [GDR 2000-10-27], [GDR 2000-10-30], [GDR 2000-10-31], [GDR 2000-11-01], and [GDR 2000-11-03].

5.1. Requirements Feedback

1. Nobody is really interested in making changes in Perforce from the defect tracker interface, even though we offered (requirements 38-40, requirement 16). Phew. This would've been hard to implement and reduce the quality of the rest of the integration.

2. Everybody is interested in being able to see more Perforce information from the defect tracker interface: mainly in being able to get details about a changelist. The nearest requirement we currently have for this is requirement 73. For defect trackers with a web interface it may be best to solve this by running a P4WEB in "browse only" mode and linking to it from the defect tracker client. A similar solution may work for defect trackers with a native interface, given that most operating systems have an easy way to launch a browser from a link in a GUI these days.

3. We must be sure to enforce the DT's workflow policy in Perforce, rather than allowing people to get around it using Perforce lax interface [GDR 2000-10-27, 15]. Workflow enforcement is a key requirement we haven't specified, except with requirements 56-59.

4. Both managers and developers wanted to be able to prevent Perforce submissions to some branches, except when assigned something to fix [GDR 2000-10-31, 18 and 22] (requirements 56-58). This also came up in conversation with managers at Quokka Sports, and in e-mail discussions with others, and in discussion on the perforce-user mailing list. This may be possible with triggers. We need to at least document how to do it, and probably provide a solution.

5. Reports for SQA [GDR 2000-10-27, item 9].

6. We need to specify the administration and maintenance overhead requirements [GDR 2000-10-30, 12].

7. Nobody was interested in the idea of associating filespecs in any special way (requirement 53-55). Perhaps we should remove the special support for filespecs and let people set up their own fields for them, replicated like any other, if they want them. This would simplify the replicator.

8. Perforce needs to provide a better interface to fixes in P4WIN. People want to be able to see them more easily, and to set the effect of a fix to something other than "closed" [RB 2000-10-24, 12].

5.2. Major Defects

The software mostly met the specification for version 0.3, so in that sense there were few defects. The main problem, a we expected, was installation and configuration. The alpha testing schedules left a whole afternoon for installation and configuration, and it was barely enough, even though we did it ourselves. It's doubtful that we were meeting requirement 63 even at the critical level (40 hours installation time).

Most of the configuration problems were caused by unexpected twists in the TeamTrack and Perforce setups. We have two major ideas about how to improve configurability: improving the abstractions in the replicator so that it makes fewer assumptions, and generating the Perforce jobspec rather than asking the administrator to create translations to and from it.

6. Conclusions

The alpha programme was a great success and very worthwhile. It exposed us to real world situations early enough to avoid major changes of direction late in the schedule, and will help us to focus on the most important requirements.

The next few steps are to adjust the project requirements, plan the next few versions, specify version 1.0, and develop solutions for the outstanding requirements.

A. References

B. Document History

Copyright © 2000 Ravenbrook Limited. This document is provided "as is", without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this document. You may make and distribute verbatim copies of this document provided that you do not charge a fee for this document or for its distribution.