Ravenbrook / Projects / Perforce Defect Tracking Integration / Project Documents
Perforce Defect Tracking Integration Project
This document reports the requirements for Perforce Software's project to integrate Perforce with third party defect tracking systems. The requirements will change, so it's important to realize that this document is a snapshot of the current understanding. In particular, [14] lists requirements questions which are unanswered.
The main purpose of this document is to allow the project architecture to be decided, by developing and evaluating alternative proposals against these requirements. The project manager can then plan and track implementation.
The document will also provide a baseline for a project requirements document to be maintained throughout the life of the project. The project requirements define the quality of the project's results and are the source material for project tests.
Last, but not least, this document justifies project decisions so that in future, when the requirements change, Perforce can intelligently understand the consequences. For example, they can work out which parts of the implementation are dependent on which requirements.
The intended readership of this document is Perforce Software's senior technical staff, and the manager of the defect tracking integration project. It could be made available to all Perforce staff.
This document is not confidential.
Perforce might want to make a similar document available to some Perforce customers who are interested in defect tracking integration, and to some defect tracking system vendors, so that they can review the requirements and correct misconceptions or errors as early as possible in the project, when they are least costly to correct.
Each requirement is a single specific measurable testable statement. Requirements can be taken out of context and still be meaningful. (This is important for traceability and preventing misunderstanding when reading requirements later.) This means that they are somewhat repetitive to read. The requirements won't often be read from start to finish.
When this document is used to create the maintained requirements, each requirement should be tagged with a unique and permanent identifier which can be used in other documents and in code and tests.
In this document the requirements are simply written out under headings which help to explain them and provide context. The requirements are organized by the role of the user who cares most about them. This is only one way to view requirements [Gilb 1988].
I have used four levels of requirement. I use some words in a precise way to express the level of requirement.
I say "must" when the requirement is critical to the project. This means that the project would fail if the requirement was not met.
I say "should" when the requirement is essential to the project. This means that the requirement could be omitted after renegotiation with the customer. Most requirements fall into this category.
I say "may" when the requirement is optional to the project. This means that the requirement could be omitted without renegotiation with the customer. In general, the project will try to meet these requirements, but may elect not to in order to deliver or because of constraints on resources.
I say "might" when the requirement is just nice to have. These requirements should only be met if they can be done cheaply, for example, as a side-effect of meeting other requirements. These requirements can contribute a great deal to customer satisfaction and goodwill.
I have used several terms in this document which have specific meanings. This section could usefully become a project glossary.
A "task" is an entity in the defect tracking system which represents a piece of work to be done that could result in changes to the product materials stored in Perforce. Most defect tracking systems I have encountered have a record called an "issue" which represents a defect or enhancement request. Mostly this is also used to represent different tasks as it moves through the workflow.
The "integration" is the software (documentation and code) which will be delivered by this project to satisfy the requirements.
The "kit" or "integration kit" is the software (documentation and code) which will be delivered by this project to allow people to integrate Perforce with defect tracking systems themselves. (See also [5].)
This section summarizes the high level user requirements described in "User Requirements for Defect Tracking" [RB 2000-04-11]. All the other user requirements in this document can be seen as being derived from these.
The defect tracker state must be consistent with the state of the product sources [RB 2000-04-11, 3.1].
The defect tracking integration must make the jobs of the developers and managers easier (i.e. make it easier for them to produce a quality product etc.) [RB 2000-04-11, 3.2].
It must be easy to discover why the product sources are the way they are, and why they have changed, in terms of the customer requirements [RB 2000-04-11, 3.3].
The interface that allows Perforce to be integrated with defect tracking systems must be public, documented, and maintained [RB 2000-04-11, 3.4].
The integration must provide the ability to ask questions involving both the defect tracking system and the SCM system [RB 2000-04-11, 3.5].
The first and second parties are Perforce Software and their customers. The third party is the defect tracking system vendor.
Perforce should be integrated with TeamTrack by TeamShare.
The integration should support all of the operations currently available in the TeamTrack integration with other SCM systems.
The integration should use the TeamTrack C++ interface to the TeamTrack database.
Perforce should be integrated with DevTrack by TechExcel.
The integration should use the DevTrack COM interface to the DevTrack database.
The integration should support all the operations currently available in the DevTrack integration with Visual SourceSafe (Attach, Detach, Check In, Check Out, Undo Check Out, Get, View Detail, Label, Directory).
The integration might support versioning of documents in DevTrack's document management interface.
Perforce may be integrated with Remedy ARS by Remedy.
The integration should use the Remedy API.
Perforce may be integrated with TRACK Defects by Soffront.
The integration should support all of the operations currently available in the TRACK Defects integration with other SCM systems.
The integration should use the TRACK COM interface to the TRACK database.
Perforce might be integrated with Bugzilla.
Perforce might be integrated with Peregrine Systems' forthcoming defect tracking system.
The fourth party is someone (not Perforce) who wants to integrate Perforce with a defect tracking system.
Non-Perforce people must be able to integrate Perforce SCM with defect tracking systems.
The amount of effort required to extend the integration to work with a new defect tracking system must be less than 6 person-months, should be less than 2 person-months, may be less than 2 person-weeks, and might be one person-week.
The integration design must be documented.
The integration must be open.
The integration must be implemented using readily available tools.
The integration design must itself be modifiable at reasonable cost.
The amount of effort required to port the integration to a new operating system must be less than 6 person-months, should be less than 2 person-months, may be less than 2 person-weeks, and might be one person-week, provided the tools on which the integration is based are available and stable on that operating system.
The integration must be stable (but, see [14.1]).
Perforce must avoid making changes which break integrations by third and fourth parties.
Perforce must keep the cost of changes to the integration small for integrations by third and fourth parties.
The integration must be maintained.
Perforce must change the integration so that it continues to work with new releases of other Perforce software.
Perforce must change the integration so that it continues to work with new releases of supported defect tracking systems.
The integration must be supported.
Perforce support must be able to answer questions about integrating Perforce with defect tracking systems.
The integration must be supportable at reasonable cost (but, see [14.2]).
The integration project may deliver the ability to drive Perforce from the defect tracking system early.
The integration should allow routine defect resolution tasks from the defect tracking system alone without the converse requirement (see [6.1] and [6.2]).
This is the type of integration that the defect tracking vendors expected. They also may supply resources to do it. It allows a marketing claim of integration without full integration.
Developers must be able to carry out routine defect resolution tasks using only the defect tracker's interface [GDR 2000-05-03, 5.1].
Developers must be able to get copies of, check out, etc. files associated with a task [GDR 2000-05-03, 6.3].
The defect tracker must display the list of files associated with a task and allow actions on (groups of) those files.
See also [6.3.1] on associating changes with tasks.
Developers must be able to carry out routine defect resolution tasks using only the Perforce interface [GDR 2000-05-03, 5.1].
Developers must be able to see the tasks assigned to them through the Perforce interface.
Developers may be able to check out the files associated with a task easily [GDR 2000-05-03, 6.3].
See also [6.3.1] on associating changes with tasks.
See also [12.2] on finding associated files.
Developers must be able to associate the changes they made in order to complete a task with that task [GDR 2000-05-03, 6.5 and 6.6].
The integration must allow association of changes with tasks at submit time [GDR 2000-05-03, 6.8].
The integration must allow association of changes with tasks after submit time [GDR 2000-05-03, 6.7].
The integration must allow changes to be associated with more than one task.
The integration must allow tasks to be associated with more than one change.
The integration may allow recording of the impact of the change on the task (e.g. whether it "fixes" it or what). A changelist might only be an analysis, or a back out, or some other thing which doesn't resolve the issue [GDR 2000-05-05b, 2.2].
Developers should be able to identify the tasks that they are working on [GDR 2000-05-03, 6.9].
Developers should be able to find out why (in terms of tasks) they have files checked out [GDR 2000-05-03, 6.9].
Developers may be able to find out where the partially completed work on a task is [GDR 2000-05-03, 6.10].
Developers might be able to branch and merge the files associated with a task [GDR 2000-05-03, 6.4].
The analyst must be able to associate (groups of) files with the task [GDR 2000-05-03, 6.2].
The analyst might be able to record the nature of the association.
Note that this isn't the same as associating changes with a task [6.3.1].
The manager may prevent unapproved changes to (groups of) files.
The manager may limit users' ability to change files to those associated with issues assigned to them.
The manager may limit users' ability to merge changes into a codeline until the change is approved.
The manager might be able to limit users' ability to access (at all) files to those associated with issues assigned to them.
See [14.3] on tester requirements.
The SQA group must be able to produce (often for the customer) a report showing that the defect tracker's idea of the status of issues affecting a release matches Perforce's idea of the changes that have been made for that release, highlighting any discrepancy [GDR 2000-05-03, 6.12].
The integration must not require administrators to be more than, say, a basic Perl or Python hacker and an experienced Perforce administrator, should not require more than one year of Perforce administration experience, may not require them to be more than ordinary Perforce administrators, and might not require them to be more than people who've just downloaded Perforce and started it.
The integration must be installable by the administrator.
Installing the integration for a supported defect tracking system must take less than 1 person-week, should be less than 2 person-day, may be less than 1 person-day, and might be less than 3 person-hours.
Removing the integration from a supported defect tracking system must take less than 1 person-week, should be less than 2 person-day, may be less than 1 person-day, and might be less than 3 person-hours.
The integration should only make additions to the Perforce state and the defect tracker state (i.e. it should not delete information).
The changes that are made by installing the integration must be clearly documented.
The integration must log changes that it makes so that they can be undone in an emergency.
Administrators should be able to develop new queries of the defect tracker's database and the relationship between issues, tasks, changelists, revisions, and files.
The integration must not require users of the defect tracker's interface [6.1] to Perforce to be more than reasonably competent with Perforce, should not require more than Perforce basics, may not require than basic familiarity with SCM concepts (e.g. a VSS user), and might not require any previous SCM experience.
The integration must not require users of the Perforce interface [6.2] to the defect tracker to be more than reasonably competent with jobs, shouldn't require more than basic familiarity with jobs, may not require any more than reasonable competence with Perforce, and might not require more than Perforce basics (for companies introducing both Perforce and defect tracking at once).
Users must be able to find out which tasks have affected a file's or group of files' development using either the defect tracker's interface or the Perforce interface [GDR 2000-05-03, 6.11].
Users must be able to find out why a changelist was made in terms of tasks from either the defect tracker's interface or the Perforce interface [GDR 2000-05-03, 6.13].
Users must be able to find out which changelists were made for a task from either the defect tracker's interface or the Perforce interface [GDR 2000-05-03, 6.16].
Users must be able to find out which files are associated with an issue from either the defect tracker's interface or the Perforce interface [GDR 2000-05-03, 6.14].
Users must like the integration as much as a slap round the face with a wet fish, should like it as much as the defect tracking systems, may like the integration as much as Perforce, and might like it as much as a cup of really good coffee.
There are no firm delivery requirements from anyone.
The project must require less than six weeks of Christopher Seiwald's time, should require less than four weeks, may require less than two weeks, and might require only a few days.
See also [14.12] on management time.
The project must require less than 90% of Gareth Rees' time, should require less than 75%, may require less than 50%, and might require none. Gareth has other duties for Ravenbrook requiring at least 10%, would like to be able to broaden his client base.
The project must require less than 60% of Richard Brooksby's time, should require less than 30%, may require less than 25%, and might require 10%. Richard has other clients, and would like to put time into developing Ravenbrook projects and relationships, but will definitely at least need to review work done for the project.
The project must cost less than some amount to be decided by Perforce [14.3], should cost less than $300K, may cost less than $150K, might cost as little as $100K.
The integration project should use Perforce as its SCM tool.
The integration project should use open languages and tools. For example, we shouldn't require people to buy Visual Basic in order to integrate, but we might ask them to have Python or something similar.
The integration project may use Python as its implementation language. This appears to be a generally favoured language of Perforce customers. I gather this from Perforce's own use of Python and discussions at the Perforce User Conference 1999.
The integration project might use Perl as its implementation language. It's less favoured than Python. See above for justification.
The project must increase goodwill toward Perforce from their customers [RB 2000-02-02, section 3, goal 2]
The project progress and status should be open to Perforce customers.
The project should be responsive to customer queries.
The project must support customers trying to integrate with supported defect trackers.
The project should support customers trying to integrate with other defect trackers using the kit.
The project should develop goodwill from defect tracking vendors toward Perforce.
The project should develop goodwill from supported defect tracking system vendors.
The project may develop goodwill from other defect tracking vendors.
How stable [5.3] must the integration be? Stability can be stated in terms of the frequency of change that requires effort from the customer. For example, the project could promise not to require user effort more than once per year.
How much should it cost to support the integration [5.5]? This could be stated in terms of number of calls relating to the integration, if there is no better measure of support costs.
I haven't had much input from testers or test groups about their requirements. This needs to be investigated, since they are important customers of defect tracking systems. The kinds of requirements I anticipate are:
find out what exactly has changed for a release or because of an issue for white box testing;
manage test suite stuff and changes to that;
associate regression test sources and stuff with issues.
Customers may wish to migrate from their existing defect tracking systems to a Perforce integrated defect tracker.
How much can it cost each time there's a new release of a defect tracker, given the requirement to maintain the integration [5.4]?
What frequency of defect tracker releases must the project cope with?
What frequency of Perforce releases must the project cope with?
What period of integration releases must the project exceed?
What rate of events must the defect tracker cope with? This requirement would have to be described in terms of one or more reference platforms.
How much bigger will the integration make the defect tracker's database?
How much bigger will the integration make Perforce's database?
How big is the integration implementation?
How much storage does it require?
What's the delay between events in Perforce and visibility in the defect tracker's interface?
What's the delay between events in the defect tracker and visibility in the Perforce interface?
What overhead can the integration impose on actions in the Perforce interface?
What overhead can the integration impose on actions in the defect tracker's interface?
How often can the databases become inconsistent? (And be detected at release time by a consistency check?)
How often can the integration stop working and need intervention?
To what extent can users tolerate inconsistent answers from the defect tracking system and Perforce? This might happen, for example, if there are simultaneous conflicting changes to task status on both sides. The integration may notice this and notify the administrator but from then on there may inconsistent answers until the situation is resolved. How much of this is OK? The integration could monitor the rate of this kind of thing and report when it gets too high or refuse to continue without intervention, in order to ensure integrity.
How much overhead does using the defect tracking integration impose on routine defect resolution work? This must be negative over defect tracking without the integration, but it will be positive if the customer is introducing defect tracking.
How little of Leslie Smith's management time must the project require?
What's the maximum budget?
| [GDR 2000-05-03] | "Requirements and Use Cases for Perforce/Defect Tracking Integration"; Gareth Rees; Ravenbrook Limited; 2000-05-03. |
| [GDR 2000-05-05a] | "Meeting with Manu Das (Soffront), 2000-05-04"; Gareth Rees; Ravenbrook Limited; 2000-05-05. |
| [GDR 2000-05-05b] | "Meeting with Richard Geiger (NetApp), 2000-05-04"; Gareth Rees; Ravenbrook Limited; 2000-05-05. |
| [GDR 2000-05-05c] | "Meeting with Pixo, 2000-05-04" (e-mail message); Gareth Rees; Ravenbrook Limited; 2000-05-05 22:52:17 GMT. |
| [GDR 2000-05-11] | "Defect Tracking Project Meeting, 2000-05-11"; Gareth Rees; Ravenbrook Limited; 2000-05-11. |
| [Gilb 1988] | "Principles of Software Engineering Management"; Tom Gilb; Addison-Wesley; 1988; ISBN 0-201-19246-2. |
| [RB 1999-05-20] | "Product Quality through Change Management"; Richard Brooksby; Geodesic Systems; 1999-05-20; <URL: http://richard.brooksby.org/1999/05/20/pqtcm/>. |
| [RB 2000-02-02] | "Steps to Defect Tracking"; Richard Brooksby; Ravenbrook Limited; 2000-02-02. |
| [RB 2000-04-11] | "User Requirements for Defect Tracking"; Richard Brooksby; Ravenbrook Limited; 2000-04-11. |
| 2000-05-05 | RB | Initial outline drafted from "User Requirements for Defect Tracking" [RB 2000-04-11] and "Requirements and Use Cases for Perforce/Defect Tracking Integration" [GDR 2000-05-03]. |
| 2000-05-06 | RB | Refined and completed import of requirements from use cases. Added attribute requirements from "Principles of Software Engineering Management" [Gilb 1988]. |
| 2000-05-08 | RB | Refined important attribute requirements. Reorganized outline. Drafted text report. |
| 2000-05-25 | RB | Marked as "not confidential" as agreed with Perforce [GDR 2000-05-11, 5]. |
| 2000-05-31 | GDR | Converted to XHTML. |
| 2000-06-30 | GDR | Made references link directly to target, rather than the references section. |
Copyright © 2000 Ravenbrook Limited. This document is provided "as is", without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this document. You may make and distribute verbatim copies of this document provided that you do not charge a fee for this document or for its distribution.
$Id: //info.ravenbrook.com/project/p4dti/template/default.html#4 $
Ravenbrook / Projects / Perforce Defect Tracking Integration / Project Documents