| Title | Too hard to maintain the test suite |
| Status | closed |
| Priority | optional |
| Assigned user | Gareth Rees |
| Organization | Ravenbrook |
| Description | Multiple interacting problems: * It's too hard to add new test cases and maintain the existing test cases. * The easiest way to add a new test is to copy and paste an existing test case, but as a result there's a lot of duplicated code in the test suite. * Too much book-keeping to add a new test case to the automated test suite (have to edit comm.gmk, commpre.nmk and fiddle with the Xcode project). * The test script is embedded in the Xcode project as just one line, so merges are difficult. |
| Analysis | Solve the book-keeping problems by: * Putting list of tests in one place * Moving Xcode test script into tool subdirectory |
| How found | inspection |
| Evidence | None as yet. |
| Created by | Gareth Rees |
| Created on | 2014-01-09 14:51:37 |
| Last modified by | Gareth Rees |
| Last modified on | 2014-03-27 17:21:39 |
| History | 2014-01-09 GDR Created. |
| Change | Effect | Date | User | Description |
|---|---|---|---|---|
| 185050 | closed | 2014-03-27 15:19:42 | Gareth Rees | Test case database now notes which test cases use threads. New test suite "testansi" consists of test cases that run on the generic ("ANSI") platform. New target "ansi" builds the MPS with the CONFIG_PF_ANSI CONFIG_THREAD_SINGLE and CONFIG_PROTECTION_NONE settings. Build and test the "ansi" target as part of "make test" for the benefit of the buildbots (just Linux and FreeBSD for the moment). |
| 184971 | open | 2014-03-24 12:12:24 | Gareth Rees | Don't list the test cases to be run in commpre.nmk: list them instead in testrun.bat for consistency with other platforms. Format lists of files and test cases with one per line so that merges are easier. Improve testrun.bat: better success/failure output; fewer gotos. |
| 184965 | open | 2014-03-24 11:41:01 | Gareth Rees | Ensure that all test cases call testlib_init so that the testlib assertion handler is installed: this ensures that on Windows you can set MPS_TESTLIB_NOABORT to avoid the dialog box from abort(). |
| 184960 | open | 2014-03-22 18:25:33 | Gareth Rees | Better to use /bin/sh and not have such fancy syntax. |
| 184959 | open | 2014-03-22 18:15:06 | Gareth Rees | testrun.sh needs to be run by bash in order to use a list. |
| 184958 | open | 2014-03-22 18:04:56 | Gareth Rees | Don't list the test cases to be run in the Xcode project: list them in testrun.sh instead so that this is shared with the FreeBSD and Linux platforms. Format the list with one test case per line so that merges are easier. |