Support report for P4DTI version 1.0

1. Introduction

This is a report describing the serious outstanding defects in P4DTI version 1.0, together with some advice on how to work around them.

The intended readership is Perforce support.

This document is not confidential.

2. Defects

job000022: No migration from Perforce jobs

Migration is currently only supported for TeamTrack.

How to go about migration depends on an organization's requirements.

1. If the organization doesn't mind job fields and values values changing, then migration is straightforward. Section 6 of the P4DTI Administrator's Guide explains how to migrate from Perforce jobs by editing a migration script, converting jobs to defect tracking issues, deleting jobs and then replicating back. (I suggest that support try out this procedure a few times to gain experience with it and to understand the pitfalls.)

2. If the organization require that job fields and values stay the same (perhaps because they have documents and scripts that depend on the way their jobs are set up), then the procedure in Section 6 of the P4DTI Administrator's Guide won't work. Instead, the organization is going to have to develop their own extension to the P4DTI. Section 7 of the P4DTI Integrator's Guide provides some clues, but doesn't give enough information for organizations to develop this extension. Two possibilities here: (1) wait for the integration kit, expected towards the end of 2001-03 and use that; (2) hire Ravenbrook or some other consulting firm to carry out the extension.

If someone needs to migrate jobs from Perforce to Bugzilla, please contact Ravenbrook. We can develop a migrate_bugzilla.py script and method on request.

job000042: Rapid changes in the DT cause conflicts

Symptom. The owner of a job receives an automatically generated e-mail message from the replicator looking like this:

Defect tracker issue 'BUG12345' and Perforce job 'BUG12345' have both changed since the last time the replicator polled the databases. The replicator's conflict resolution policy decided to overwrite the job with the issue.

The replicator has therefore overwritten Perforce job 'BUG12345' with defect tracker issue 'BUG12345'. See section 2.2 of the P4DTI User Guide for more information.

Cause 1 and solution. This e-mail can be caused by users simultaneously changing a defect tracking issue and the corresponding Perforce job, as described in section 2.2 of the P4DTI User's Guide. This is correct behaviour. In this case the user who changed the job should check to see if their changes have been lost by the overwrite. If so, they should edit the job again and make their changes again. Their changes to the job are included in the e-mail, so nothing has been lost.

Cause 2 and workaround. If someone makes two changes to an issue in the defect tracker in quick succession, the first change may be replicated successfully to Perforce. But when the replicator next polls the servers, it sees that the issue has changed again, but so has the corresponding job (it sees the change that it made itself on the previous replication). So it thinks both have changed and it overwrites the job with the issue. No harm is done by this feature of the P4DTI; it's just annoying to be sent e-mail needlessly. Users should just ignore these e-mails.

The underlying cause is that Perforce doesn't update the "always" fields in a job when someone affects that job by making or deleting a fix. This means that the replicator can't accurately tell who last changed a job, and so it has to consider changes that it made on previous replications as being candidates for replication. This is Perforce's own job004308.

job000077: UG contains no version or release related documentation

When an organization has the P4DTI running successfully and developers are conscientiously creating fix records in Perforce, then it should be possible to automatically generate lists of fixed and open issues for a release.

For fixed issues in a release, issue the command:

p4 fixes -i files-contributing-to-the-release

The filespec will be something like //depot/project/foo/release/2.7/... if each release has its own branch, or something like //depot/project/foo/...@1234 if releases are identified by a changelevel on the master sources, or something like //depot/project/foo/...@release-2.7 if releases are identified by labels.

This generates a list of fixes contributing to the release. This can then be processed to get a list of jobs fixed in the release. By subtracting these issues from the total set of issues for that project, you get the list of issues open in that release. With a little more sophistication you can ignore issues introduced subsequently to that release: for example, by having an "Introduced-in" field in your issue which gives the earliest release that can possibly have the defect.

For issues fixed between release 2.6 and release 2.7, you should subtract the issues fixed in release 2.6 from the issues fixed in 2.7. If the two releases come from the same branch, you can also generate this list with a command like:

p4 fixes -i //depot/project/foo/...@1234,2345

Here release 2.6 was made at changelevel 1234 and release 2.7 at changelevel 2345.

You can see how Ravenbrook generate lists of known and fixed issues by looking at the script known_issues.py.

Ravenbrook can provide consultancy to organizations wishing to develop this kind of report. See Ravenbrook's Perforce services page.

job000135: Replicator makes no more progress if replication to Perforce fails

Symptom. The integration administrator receives a sequence of automatically generated e-mails from the replicator that look like this:

The replicator failed to poll successfully, because of the following problem:

Exception:
TeamShare API error: SOCKET_CONNECT_FAILED: Socket Connect failed.

A variety of errors may appear here.

Cause. The replicator has become stuck and can make no more progress. Possibly it can't connect to one or both of the servers; possibly it can't replicate issues because it has been misconfigured. The replicator backs off, doubling its waiting time each time it encounters the problem. When the problem is fixed it will continue normally.

Solution. The replicator can't continue until the problem is fixed. The administrator must identify and fix the problem. Section 11.2 of the P4DTI Administrator's Guide is a good place to start.

job000148: Too long to replicate because of reading USERS, STATES, etc. many times

(This note talks about users only, since that's the only case where we've actually seen the problem in operation. If you see this for other entities such as projects or states please let Ravenbrook know.)

Symptom. This occurs in the TeamTrack integration. The replicator takes a long time to replicate issues. The P4DTI log contains many messages like "Reading USERS table."

Cause 1. The replicator keeps a cache of TeamTrack users. If it tries to replicate a user and it fails to find it in its cache, then it will re-read the USERS table from TeamTrack. This is so that when you add a new user to TeamTrack the replicator will carry on working properly (that is, it will notice the new user and replicate them appropriately). But note that we recommend people to stop and start the replicator when they add new users; see Section 9.1 of the P4DTI Administrator's Guide.

However, this has the consequence that if you have a Perforce user who doesn't have a corresponding userid in TeamTrack, then whenever the replicator attempts to replicate anything to do with that user (a job owned by that user, a changelist made by that user, a fix made by that user) then it will re-read the USERS table just in case the user has been newly added.

Solution. The best solution is to make sure that all Perforce users have corresponding TeamTrack userids. Check that e-mail addresses match between the two systems. Note that the replicator sends an e-mail to the P4DTI administrator each time it starts up reporting which users of each system have no corresponding user in the other system. This report can be used to identify problematic users.

If it is impractical to provide TeamTrack licences for all Perforce users, then it is possible to patch the replicator so that it doesn't re-read the users table when it fails to find a user in the cache. Ravenbrook can provide such a patch on request.

Cause 2 and solution. Slow operation of the P4DTI may also be due to a slow connection to the TeamTrack server. In this case there's little that can be done except to improve the network connection. Try running the replicator on the same machine as the TeamTrack server.

job000159: Fixes which the replicator rejects still get replicated

Symptom. A user fixes a job using p4 fix or its equivalent in the GUI. However, that user doesn't have permission to change the job, so the replicator sets the job state back to its old value. But p4 fixes still shows the fix.

Cause. The replicator doesn't revert fixes when it reverts the job. The reason for this is that it can't be sure that the fix caused the change in the job; the fix might have been perfectly legitimate even though the change to the job was wrong.

Workaround. If the fix was correct, get someone who has permission to update the job state in Perforce, or transition the corresponding issue in the defect tracker. If the fix was incorrect, delete it from Perforce with p4 fix -d.

job000197: P4DTI doesn't do ESMTP authentication

Symptom. The replicator fails to start, reporting an authentication failure from its SMTP server. (It would be nice to have the actual error message here, but I didn't record it at the time.)

Cause. The mail server is using the ESMTP protocol and requires clients to authenticate themselves. The P4DTI doesn't support ESMTP authentication.

Workarounds. You can stop the replicator from attempting to send e-mail by setting the smtp_server configuration parameter to None. But this is a bad idea because users won't be informed when they make illegal changes in Perforce, and the administrator won't be informed when the replicator can't replicate. A better alternative would be to run an ordinary SMTP server especially for the replicator to use.

job000201: Empty assigned_to field can't be replicated

Symptom. This happens only in the Bugzilla integration. The integration administrator receives a sequence of automatically generated e-mails from the replicator that look like this:

The replicator failed to poll successfully, because of the following problem:

Exception:
Perforce error: Error in job specification. Missing required field 'Assigned_To'.

Cause. The "Assigned_To" field has become empty in Bugzilla; we believe that this can't happen in Bugzilla 2.10, so that's why it causes an error in the replicator. So this can only happen at a site which is not running Bugzilla 2.10, or which has changed the way Bugzilla 2.10 works, or where someone has edited the Bugzilla database directly, bypassing Bugzilla.

Workaround. Assign the bug to someone in Bugzilla.

If you're sure that you're running a vanilla Bugzilla 2.10, then please try to work out how the "Assigned_To" field became empty and report this to Ravenbrook so that we can fix the problem.

job000258: Perforce users need a TeamTrack licence to use TeamTrack features

Symptom. In the TeamTrack integration, if you try to assign a job in Perforce to a Perforce user who doesn't have a TeamTrack licence (by editing the Owner field), then the replicator refuses to replicate it; it resets the Owner field and sends e-mail.

Cause. To get the benefit of TeamTrack features, TeamShare insist that you buy TeamTrack licences. We agreed with TeamShare that the P4DTI would respect this policy; see [GDR 2000-08-18, 3.3.1].

Solution. Buy new TeamTrack licences for all users who need to use TeamTrack features (whether directly or via the P4DTI).

A. References

B. Document History

Copyright © 2001 Ravenbrook Limited. This document is provided "as is", without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this document. You may make and distribute verbatim copies of this document provided that you do not charge a fee for this document or for its distribution.