| Title | Replicator is slow due to reading auxiliary tables many times |
| Status | closed |
| Priority | essential |
| Assigned user | Gareth Rees |
| Organization | Ravenbrook |
| Description | The replicator may read auxiliary tables in the defect tracker (like the TeamTrack USERS table) many times in the course of replicating a single job or issue. This slows replication down and increases the possibility of conflicts. |
| Analysis | We have only seen this problem at a beta test site, where it resulted from two conditions: 1. The P4DTI was running on a different machine from the TeamTrack server, thus requiring users, states and so on to be accessed across the network. 2. The internal network was very slow (even Perforce was badly affected). There are two design problems that underlie this: 1. The configuration generator creates one user_translator for each user field, and another for use in translating changes and fixes. Each user_translator has its own idea of what the user map is. This has the potential for inconsistency. There should be one user map per dt_teamtrack object. 2. Every time it misses a user, it re-reads the USERS table in case the user has been added. But missing a user can happen every time something happens, whereas adding a user is rare. Better strategies can be applied to both problems: 1. Only create one user_translator object in configure_teamtrack.py and to use that in all places where one is needed (this is safe). Similar remarks apply to the other translators, but user_translator is the one where this problem is most obvious. 2. Don't read the USERS table each time a user isn't found, but less frequently. Once per poll would be good. 3. Even once per poll is too much when there are many users and a slow network connection to the TeamTrack server. Since we tell people to restart the replicator when they add a user, it may be best never to refresh the cache. See also < http://www.ravenbrook.com/project/p4dt...2001-02-22/version-1.0-support/#job-148>. |
| How found | manual_test |
| Evidence | <http://www.ravenbrook.com/project/p4dt...2000-12-08/release-0.4.2-test-report-2/>, item 4. |
| Observed in | 0.4.2 |
| Introduced in | 0.4.0 |
| Created by | Gareth Rees |
| Created on | 2000-12-08 19:31:32 |
| Last modified by | Gareth Rees |
| Last modified on | 2001-12-10 19:13:46 |
| History | 2000-12-08 GDR Created. 2001-02-27 NB Made description more generic, as this also applies to Bugzilla. |
| Change | Effect | Date | User | Description |
|---|---|---|---|---|
| 13708 | closed | 2001-06-30 17:22:08 | Gareth Rees | Added flags which indicate whether TeamTrack data has been read and cached this poll, so that we don't read these tables more than once per poll. |
| 10155 | open | 2001-03-17 14:21:56 | Gareth Rees | Formatted configure_teamtrack.py as a document. Added links to design. Make only a single user translator to fix job000148. |