| Title | No automatic feedback about overheads of "hot" versus "rash" MPS |
| Status | closed |
| Priority | essential |
| Assigned user | Richard Brooksby |
| Organization | Ravenbrook |
| Description | The "hot" variety of the MPS is intended for production, but still includes most assertions, as long as they are not on critical paths. The intention is that the "hot" variety perform almost as well as a "rash" variety, which has all all assertions disabled. We should have automatic feedback to the development team about whether this requirement is being met. |
| Analysis | Continuous integration should run regular comparisons of hot and rash builds, at the very least using our own gcbench and djbench programs, but ideally with real clients' benchmarks (see also job003834). If possible, we should collect, tabulate, and plot this data to see the impact of our changes. |
| How found | unknown |
| Evidence | At changelist 189674 on Silverbird (OS X): make -f lii6ll.gmk VARIETY="hot" gcbench djbench make -f lii6ll.gmk VARIETY="rash" gcbench djbench xci6ll/hot/gcbench -x 1564912146 amc -> 14.1s xci6ll/rash/gcbench -x 1564912146 amc -> 13.4s xci6ll/hot/djbench -x 861801839 mvff -> 23.9s xci6ll/rash/djbench -x 861801839 mvff -> 16.2s This quick test shows we're doing OK for GC, but not so great for the manual MVFF pool. |
| Created by | Richard Brooksby |
| Created on | 2016-03-07 10:50:29 |
| Last modified by | Gareth Rees |
| Last modified on | 2016-04-20 13:16:17 |
| History | 2016-03-07 RB Created. |
| Change | Effect | Date | User | Description |
|---|---|---|---|---|
| 191339 | closed | 2016-04-20 13:15:18 | Gareth Rees | Add a test checking that the hot variety is not too much slower than the rash variety. Turn this on for continuous integration testing via "make test". |