Test design

1. Introduction

This document describes the automated test cases for the P4DTI, how they are designed, and how to run them. The purpose of the document is to allow P4DTI developers to collaborate on test case development and so improve the quality of the P4DTI.

The intended readership is P4DTI developers and people developing new integrations using the integration kit.

This document is not confidential.

2. Test framework

The automated tests use the Python unit test framework [PyUnit]. To use the framework with these tests, put the module unittest.py from the PyUnit distribution on Python's library path. New unit test scripts should be called test_module.py, where module.py is the module being tested. Each test script should define a function called tests, which returns an object belonging to the unittest.TestSuite class. The run_tests.py script finds all tests in the current directory and runs them.

To run the unit test for the message module, run the command python test_message.py.

To run the "wrapping" test case (only) from the unit test for the message module, run the command python test_message.py wrapping.

To run all the tests in this directory, run the command python run_tests.py.

3. Test database

This section specifies test databases. The test cases rely on specific features of the test databases (e.g., presence of certain fields) in order to test features of the P4DTI.

3.1. Bugzilla test database

The Bugzilla test databases are stored as bugzilla-VERSION-mysqldump, where VERSION is the Bugzilla version (2.12, 2.14, etc). Each is produced by running Bugzilla, building up a bugs database, and using mysqldump to generate the file.

See [NB 2001-10-17] for the design, noting the following additions:

1. A user with spaces in their login name, to whom at least one bug is assigned (regression test for job000289).

4. Testing the TeamTrack integration

TeamTrack test procedure

Gareth Rees, Ravenbrook Limited, 2001-02-26


1. Introduction

This document explains how to carry out a test procedure for the
Perforce/TeamTrack integration on Windows NT or Windows 2000.

The intended audience is P4DTI project staff.


2. Setting up a testing session

Use this procedure to start a fresh testing session with a new TeamTrack
server, Perforce server and P4DTI (but don't run the P4DTI yet).

  1. Stop any running Perforce servers and TeamTrack servers.

  2. Make a new dated directory like d:\p4dti-test\YYYY-MM-DD\p4d.

  3. Start a Perforce server in that directory (p4d -p 0.0.0.0:1666 -r
     d:\p4dti-test\YYYY-MM-DD\p4d).  Add a license file.

  4. Start P4Web in browse mode (p4web -p 127.0.0.1:1666 -b).

  5. Copy teamtrack-test.mdb into d:\p4dti-test\YYYY-MM-DD, using the
     TeamTrack Administrator.  Call the new ODBC database called
     something like "P4DTI YYYY-MM-DD"; connect to that database; start
     the TeamTrack web server.

  6. Configure and run the perforce_sample_data.py script.

  7. Unpack the P4DTI release into d:\p4dti-test\YYYY-MM-DD\p4dti.

  8. Copy the appropriate configuration file from this directory to
     d:\p4dti-test\YYYY-MM-DD\p4dti\config.py.


3. Initial installation tests

  1. Normal installation.  Follow the setup procedure; start the
     replicator.  It should replicate all issues successfully.

  2. Recent start date.  Follow the setup procedure; set the start_date
     parameter to the current time; start the replicator.  The
     replicator should replicate no issues.

     Set the start_date parameter to an old date; restart the
     replicator.  The replicator should replicate the old issues.

  job000112. Can't easily replicate by project

     Set replicate_p to "return self['PROJECTID'] in [3,4]".  Check
     that only the programming projects are replicated.


4. Use cases and regression tests (users)

  1. Follow the setup procedure; start the replicator.  Check the
     initial e-mail message to the administrator.

  2. Run perforce_sample_data.py to generate some activity in Perforce.

  3. Follow use cases in the User Guide.

  job000006. TeamShare API error reporting is inadequate

     Make an edit you don't have permission to.  Make a transition you
     don't have permission to.

  job000024. "\012" in fixes table in TeamTrack

  job000053. Implicit changes made by the DT don't get replicated back

  job000063. Dates aren't replicated with fixes

  job000087. Replicator doesn't enforce licenses in TeamTrack

     Make a edit in Perforce as a user who has no corresponding user
     in TeamTrack.

  job000107. You can't replicate an issue's project

  job000109. Can't get at Perforce information from TeamTrack

  job000133. You can't close a job by fixing it

  job000146. Empty date fields in TeamTrack give the replicator a headache

     Add "ESTTIMETOFIX" and "ACTTIMETOFIX" to the replicated_fields.

  job000156. Pending changelists aren't clearly indicated as such

  job000169. Change numbers are links in TeamTrack even when no
  changelist URL has been specified

  job000182. Elapsed time fields aren't replicated properly

  job000188. tTrack integration fails with non-ASCII characters

  job000223. Quote in change comment terminates display in TeamTrack

  job000226. Newlines don't show up in change descriptions in TeamTrack

  job000233. When you submit a new issue to TeamTrack it overwrites the issue


5. Use cases and regression tests (administrator)

  job000108. Userids can't differ between TeamTrack and Perforce

  job000140. Logical field name "code" not allowed in TeamTrack

  job000141. Can't add to replicated_fields list

A. References

B. Document History

Copyright © 2001 Ravenbrook Limited. This document is provided "as is", without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this document. You may make and distribute verbatim copies of this document provided that you do not charge a fee for this document or for its distribution.