Issue Task Under Review

#111428Run Playwright e2e tests with multiple workers in CI

Benni Mack 11/12/26, 20:16 updated 11/12/26, 20:51

Description

The Playwright e2e shards are the longest jobs of the pre-merge pipeline. In CI every shard runs its tests with a single worker (workers: process.env.CI ? 1 : worker in Build/playwright.config.ts), so about 90 tests per shard are executed strictly one after another. Setting up the instance takes about 15 seconds, the rest of the job is test execution.

Raising the worker count alone does not work, because the specs are not isolated from each other:

  • All workers share one backend user and one session. Flash messages, module state, user settings, user TSconfig and the selected workspace leak from one worker into the tests of another.
  • Several specs save the same styleguide record. FormEngine submits every field of a record, so a form that was opened before another worker saved reverts that change.
  • Some specs change state every other spec observes: install tool and system settings, the site configuration, pages other specs navigate to, workspaces.
  • Playwright distributes specs to shards by test count, not by duration, which leaves some shards with far more test time than others.

Goal: run the e2e specs with several workers per shard in CI.

  • one backend administrator per worker
  • specs that collide only with each other run one at a time, next to the parallel ones
  • specs that change global state run on their own, after the parallel ones
  • spec files are distributed to the shards by their measured duration

Measured locally (composer mode, SQLite), the slowest of the four shards drops from 276 to about 138 seconds with three workers. This has not been measured on the CI runners yet.

Follows #111424, which made the two slowest FormEngine specs cheaper.

Reviews 1

History 02