Received: from martin.ravenbrook.com (martin.ravenbrook.com [193.112.141.241]) by raven.ravenbrook.com (8.9.3/8.9.3) with ESMTP id SAA10291 for ; Fri, 23 Mar 2001 18:03:47 GMT Received: from [193.237.68.82] (grouse.ravenbrook.com [193.112.141.254]) by martin.ravenbrook.com (8.8.8/8.8.7) with ESMTP id RAA04292 for ; Fri, 23 Mar 2001 17:59:13 GMT (envelope-from gdr@ravenbrook.com) Mime-Version: 1.0 X-Sender: gdr@pop3 Message-Id: Date: Fri, 23 Mar 2001 18:03:19 +0000 To: p4dti-staff@ravenbrook.com From: Gareth Rees Subject: Test report for release 1.0.5: migrating jobs to TeamTrack Content-Type: text/plain; charset="us-ascii" ; format="flowed" NDL and I went through the procedure in and found the following problems: 1. Item 2 in migrate_teamtrack.py claims that the selections for the ISSUETYPE field have TS_FLDID = 4 in the TS_SELECTIONS table. This isn't true. Someone might have reordered the fields using the TeamTrack Administrator, in which case the field id of the ISSUETYPE field will be different. In fact, the ISSUETYPE selections can be found using this SQL query: SELECT * FROM TS_SELECTION WHERE TS_FLDID=(SELECT TS_ID FROM TS_FIELDS WHERE TS_DBNAME='ISSUETYPE') 2. Item 4 in migrate_teamtrack.py has the same problem (but for severity, not issue type). 3. Item 4 in migrate_teamtrack.py claims that the Severity field is required by TeamTrack. This is true by default, but it may well have been changed. 4. Other fields may have been required by the TeamTrack Administrator, but we don't mention the possibility. 5. Someone might put some effort into creating a migration script and then expect to be able to run it again later. This isn't true. The values for the various fields may change and therefore the migration script may go out of date and no longer be usable. 6. Writing a robust migration is very hard -- about as hard as configuration used to be in the alpha P4DTI. You can't expect ordinary sysadmins to be able to do it. There needs to be much more support and error checking from the program. 7. You can't both keep your job numbers the same and have them match the bug ids in TeamTrack. You can have either one, but not both. (This is because TeamTrack has its own numbering.) 8. "import replicator" is missing. 9. The query that gets the issue again needs to change if you change the padding of issue numbers! (It uses '%05d' for the ISSUEID field: maybe this could be a LIKE comparison.) 10. The script didn't replicate changelists. 11. When I added replication of changelists, it didn't work -- because you can't run "p4 change -o" without a client to do it from. Now the replicator creates a client in its init() method -- but we can't call r.init() because that method checks for P4DTI fields and there aren't any. Clearly the replicator class is not suitable for using before Perforce has been set up. 12. Similarly, the consistency checking script ought to be able to help us check that things are going well (even if it produces lots of errors, one could grep -v them away and concentrate on fixing the real errors). 13. Conclusion: I don't think that the migrate script is close to being ready for P4DTI administrators to use. * For the record, here's the migrate_teamtrack.py that we ended up with: # migrate_teamtrack.py -- Migrate Perforce jobs to TeamTrack. # # Gareth Rees, Ravenbrook Limited, 2000-10-19. # # $Id: //info.ravenbrook.com/project/p4dti/version/1.0/code/replicator/migrate_teamtrack.py#7 $ # # 1. Introduction # # This file contains the outline of a program that can convert Perforce jobs # into TeamTrack cases and submit them to TeamTrack. Since I don't know what # fields you have in your Perforce jobs or how you plan to convert them to # fields in TeamTrack, I can't prepare a complete solution for you. Instead, # you'll have to go through this file, read the comments and fill in the # details. # # Each instance where you need to think and possibly edit the code is marked # with three asterisks (***). # # Run this script from the command line by switching to the directory # containing this file, then run the command "python migrate_teamtrack.py". # # You should also read section 6.2 of the P4DTI Administrator's Guide. # # --- # # This file is copyright (c) 2001 Perforce Software, Inc. All rights # reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, # BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS # OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND # ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR # TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE # USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH # DAMAGE. from config import * import configure_teamtrack import dt_teamtrack import p4 import replicator import string import teamtrack # Generate the configuration for the dt_teamtrack module. teamtrack_config, p4_config, replicator_config, _ = \ configure_teamtrack.configuration \ ( administrator_address = administrator_address, changelist_url = changelist_url, closed_state = closed_state, log_file = log_file, p4_client_executable = p4_client_executable, p4_password = p4_password, p4_port = p4_port, p4_server_description = p4_server_description, p4_user = p4_user, poll_period = poll_period, replicate_p = replicate_p, replicator_address = replicator_address, replicated_fields = replicated_fields, smtp_server = smtp_server, start_date = start_date, teamtrack_password = teamtrack_password, teamtrack_server = teamtrack_server, teamtrack_user = teamtrack_user, verbose = verbose, ) # Construct TeamTrack, Perforce, Replicator instances. dt = dt_teamtrack.dt_teamtrack(rid, sid, teamtrack_config) p4i = p4.p4(p4_config) r = replicator.replicator(rid, dt, p4i, replicator_config) tt = dt.server p4i.run('client -i', p4i.run('client -o')) # Setup of TeamTrack for the P4DTI by adding fields to the CASES table. dt.init() # convert_newlines: string -> string. Convert a multi-line string from # Perforce's format (\n only) to TeamTrack's format (\r\n). def convert_newlines(s): # Remove final newline (if any). if s and s[-1] == '\n': s = s[:-1] # Replace \n with \r\n. s = string.replace(s, '\n', '\r\n') return s # *** Fetch Perforce jobs for migrating. To start with, I suggest only # migrating a single job, to check that the script is working correctly. # Specify the job you want to migrate in the -e option to the jobs command, # like this: # jobs = p4i.run('jobs -e "job=job000002"') # *** When you are happy that the migration is working correctly, fetch all the #jobs, like this: jobs = p4i.run('jobs -e "job>job000082"') # Now submit each Perforce job as a TeamTrack case. for job in jobs: # Make a TeamTrack case that will correspond to the job. case = tt.new_record(teamtrack.table['CASES']) # Now fill in each of the case's fields with appropriate values from the # job. Five fields are normally required by TeamTrack: TITLE, DESCRIPTION, # ISSUETYPE, PROJECTID and SEVERITY. # 1. *** The TeamTrack TITLE should be a one-sentence summary and the # DESCRIPTION a detailed explanation. If you've been using the default # Perforce jobspec, then you only have one field, the Description. So I # suggest making the TITLE be the first line of the description, and the # DESCRIPTION be the full description. You may need to go into TeamTrack # and fix things up later by hand. case['TITLE'] = job['Title'] case['DESCRIPTION'] = convert_newlines(job['Description']) # 2. *** The ISSUETYPE indicates whether the case is a bug or change # request or whatever. The legal values for this field are found in the # TS_ID field in the TS_SELECTIONS table where TS_FLDID = 4. I have chosen # the value 1, which means "Bug Report" in my TeamTrack database (you must # check yours to find an appropriate value). case['ISSUETYPE'] = 143 # 3. *** The legal values for the PROJECTID field are found in the TS_ID # field in the TS_PROJECTS table. Project 2 is the first project in my # TeamTrack database (you must check yours to find an appropriate value). # Or you could compute the TeamTrack project based on the Perforce project # if your Perforce jobs have a project field. project_table = { 'p4dti/p4dti': 49, 'p4dti/project': 50, 'hdasi': 51, 'www.ravenbrook.com': 52, 'p4sr': 53, 'p4report': 55, 'test': 57, } p4_project = job['Project'] if p4_project == 'p4dti': p4_project = p4_project + '/' + job['Product'] case['PROJECTID'] = project_table[p4_project] # 4. *** The legal values for the SEVERITY field are found in in the TS_ID # field in the TS_SELECTIONS table where TS_FLDID = 14. I have chosen the # value 30, which means "Medium" in my TeamTrack database (you must check # yours to find an appropriate value). Better still would be to compute # the TeamTrack severity based on the Perforce severity (if your Perforce # jobs have a severity field). # 5. *** You don't have to fill in any more fields in the case if you don't # want to. But if you have more fields in Perforce, you'll probably want # to fill in some more fields in the TeamTrack case. You will need to read # the TeamTrack database schema [TeamShare 2000-01-20] to understand the # legal values for fields in a TeamTrack case. The two fields you can't # set at this point are the STATE field (because when the case is submitted # it ends up in state New or whatever you've defined to be the first state # in the workflow) and the OWNER field (because TeamTrack automatically # sets the OWNER field when you submit an issue); see steps 11 and 12 # below. priority_table = { 'critical': 145, 'essential': 146, 'optional': 147, 'nice': 148, } case['PRIORITY'] = priority_table[job['Priority']] case['HISTORY'] = convert_newlines(job['History']) case['ANALYSIS'] = convert_newlines(job['Analysis']) case['EVIDENCE'] = convert_newlines(job['Evidence']) for f in [ 'Introduced-in', 'Observed-in', 'Test' ]: ttf = string.upper(string.replace(f,'-','_')) if job.has_key(f): case[ttf] = job[f] org_table = { 'Ravenbrook': 144, 'TeamShare': 149, 'Perforce': 150, } case['ORGANISATION'] = org_table[job['Organization']] # 6. Set P4DTI fields based on the configuration. case['P4DTI_RID'] = rid case['P4DTI_SID'] = sid case['P4DTI_ACTION'] = 'replicate' # 7. *** Set the jobname that the case will be replicated to. If you want # to keep jobnames the same in Perforce, then use the jobname here. If you # would prefer the Perforce jobname to match the case id, comment out the # next line. case['P4DTI_JOBNAME'] = job['Job'] # 8. *** Pick a user on whose behalf to submit the case. If your Perforce # userids are the same as the TeamTrack userids, then it should be safe to # use the User field of the job. Otherwise, you'll have to convert the # Perforce user somehow. user = job['User'] # 9. Submit the case. issueid = case.submit(user) # 10. Fetch the case from the database so that we can set the STATE and # OWNER fields. case = tt.query(teamtrack.table['CASES'], "TS_ISSUETYPE = %d AND TS_ISSUEID LIKE '%06d'" % (case['ISSUETYPE'], issueid))[0] # 11. *** Set the STATE based on the Perforce job status. The state_table # is a map from Perforce job status to TeamTrack state. The legal values # for TeamTrack states are given by the TS_ID field in the TS_STATES table; # you must look in your TeamTrack database for the right values. state_table = { 'open': 33, 'suspended': 34, 'closed': 35, 'duplicate': 36, } case['STATE'] = state_table[job['Status']] # 12. Set the OWNER field to be the user who submitted the case (who came # from the job's user field in step 6 above). case['OWNER'] = case['SUBMITTER'] # 13. Update the case in the database (note that by using 'update' rather # than 'transition' we bypass TeamTrack's checks on the workflow). case.update() # 14. Report on what's been done. print ("Submitted a case with issueid %d corresponding to job %s." % (issueid, job['Job'])) print '-' * 79 print case print '-' * 79 print job print '-' * 79 # 15. Replicate fixes. issue = dt.issue(str(case['ID'])) fixes = r.job_fixes(job) for f in fixes: change, client, date, status, user = r.translate_fix_p4_to_dt(f) issue.add_fix(change, client, date, status, user) changelist = p4i.run('change -o %d' % change)[0] r.replicate_changelist_p4_to_dt(changelist) print "Fix for change", f['Change'], "with status", f['Status'], "." print '-' * 79 # A. References # # [TeamShare 2000-01-20] "TeamTrack Database Schema (Database Version: 21)"; # TeamShare; 2000-01-20; . # # # B. Document History # # 2000-10-19 GDR Created. # # 2000-12-08 RB Moved config_teamtrack to init_teamtrack. # # 2000-12-18 NB Moved init_teamtrack to init. # # 2001-02-22 GDR Rewritten so it can run before you've installed the P4DTI. # # 2001-02-23 GDR Added STATE and OWNER. Set the P4DTI fields, including # P4DTI_JOBNAME. # # 2001-03-02 RB Transferred copyright to Perforce under their license. # # 2001-03-23 GDR Replicate fixes.