TEST REPORT FOR RELEASE 0.3.1 Gareth Rees, Ravenbrook Limited, 2000-10-17 1. INTRODUCTION 2. INSTALLATION REPORT Defects in the install procedure: 1. You can't read the SAG until you've installed the software, but you can't install the software without reading the AG. We need a README distributed with the .exe. This could point at the copy of the AG on our web site. 2. We still call it SAG even though it's really the AG. 3. When you've downloaded p4gdr, what do you do with it? Give reference. 4. Use c:\propgram files\p4dti\test-repository for the location of the test repository. (Keep all the cruft in one place.) 5. We need uninstall instructions. 6. Think about phrasing of instructions. Currently they are phrased as motivation; instructions. But the motivation looks like an instruction so the installer may try to carry it out without reading on to find out how. Improve this somehow. 7. Give suggested names for things like the ODBC data source. 8. Missing instruction: select new data source in File/Connect. 9. Missing instruction: stop and start the broker service (Options -> Manage Services). 10. Say that P4DTI-case is the default TeamTrack user for the replicator. 11. The TeamTrack group that the replicator user should be in is called "Administrator", not "Administrators". 12. In the jobspec section, give command for editing the jobspec: "p4 -p 127.0.0.1:1667 jobspec". 13. Use 
 in the Perforce examples, check that copy and paste work
from Internet Explorer.

14. Organize the instructions so that it's clear when you're done with a
dialog or tool.  There are two separate operations to be performed
in the TeamTrack Administrator; three things to do to the jobspec.  It's
easy to miss one.

15. Don't have a "run" instruction at the end of section 4, because it 
won't work until you've configured it.

16. Need configuration instructions in section 5.

17. We don't have any recommended approach for getting back to a sensible
state.  So recommend that user does installation on a new machine.

18. We need to advise people to back up all their stuff (and how).

19. We need to say how to add the registry values and what type they must
be (REG_SZ).

20. Need to mention in troubleshooting section: keep/discard will only 
try the replication again: if a persists it will just fail again. 
You have to understand and fix the problem.


3. DEFECTS IN INSTALLED SOFTWARE

21. The check consistency script doesn't report missing stuff in the 
jobspec and the CASES table etc.

22. The check consistency script doesn't say "OK" when things are.

23. Wrong sample database in release (the one distributed with 4401 
also contains random Perforce stuff).

24. The replicator's message about a "previous installation" is 
misleading.  Just report which fields it found present and which it 
added.

25. Design problem: it's tempting to set the state to "replicate" 
rather than "keep".

26. Design defect: since TeamTrack case IDs don't correspond to the 
displayed case ids, then our mapping to Perforce names will be 
infinitely confusing.  So why not use the human-readable string 
instead?

27. Replicator stops when trying to do change -o because it doesn't 
have a client.  (Why do you need a client to do change -o?)

28. Description field in the Perforce Fixes section of the VC part of a 
case description in TeamTrack doesn't do the right thing with escapes 
like \012.

29. Perforce "p4 fix -c 1 jobname -s status" does the fix and then 
complains about stuff after the command.  This is a bug.  Perforce 
should verify the whole command line.

30. Failing to find a corresponding state should look more like an 
expected failure rather than a fatal error (which is what it looks 
like now).

31. Assertions should be fatal: need to change the set of errors which 
are captured by the replicator.

32. Suppose that you make a fix whose effect on the job is illegal; you 
get a conflict and then you discard on the Perforce side.  (a) why 
did the fix get replicated? because you have to do something first. 
(b) the fix is now incorrect in the sense that it didn't cause the 
effect that is claimed.  So should we delete the fix?  What is the 
requirement here?

33. Need to tell people how to stop the replicator.

34. The check script could print progress messages: what is it checking 
exactly?  It's purpose is to reassure.


4. OBSERVATIONS

35. Worked correctly when a pending changelist with associated fix was 
submitted (that is, it closed the job and replicated the closure to 
TeamTrack -- although it found a strange transition in doing so).


A. REFERENCES


B. DOCUMENT HISTORY

2000-11-21  RB  Numbered items so that they could be referenced. 
Reformatted as document rather than e-mail.

---

Copyright 2000 Ravenbrook Limited.  This document is provided "as is",
without any express or implied warranty.  In no event will the authors
be held liable for any damages arising from the use of this document. 
You may make and distribute verbatim copies of this document provided
that you do not charge a fee for this document or for its distribution.

$Id: //info.ravenbrook.com/project/p4dti/doc/2000-10-17/release-0.3.1-test-report/index.txt#3 $