Use CircleCI's "parallelism" feature to speed up runs of the integration test suite #4054

New Issue

exarkun · 2023-08-01T14:02:05Z

exarkun commented

2023-08-01 14:02:05 +00:00

CircleCI has a feature where you can tell it to run several executors concurrently for a single job. It tells each executor how many concurrent executors it is running and what its unique index in that sequence is. Then it is up to the job to pick a slice of work which, when combined with the choice made on all the other jobs, combine to represent the full test suite run.

For a test suite (as opposed to something like the compilation of a large C program), the "slice of work" can be some subset of the tests in the test suite.

There are a few possible approaches for convincing pytest to run such a subset. If we combine one of these with the CircleCI feature then we can leverage more CPU power to (possibly) complete the integration tests more quickly.

CircleCI has a feature where you can tell it to run several executors concurrently for a single job. It tells each executor how many concurrent executors it is running and what its unique index in that sequence is. Then it is up to the job to pick a slice of work which, when combined with the choice made on all the other jobs, combine to represent the full test suite run. For a test suite (as opposed to something like the compilation of a large C program), the "slice of work" can be some subset of the tests in the test suite. There are a few possible approaches for convincing pytest to run such a subset. If we combine one of these with the CircleCI feature then we can leverage more CPU power to (possibly) complete the integration tests more quickly.