Review main Merged

95313[TASK] Stabilize browser based test suites

Stefan Bürk CR +2 V +2 Normal +92 −25 updated 08/13/26, 14:10

Commit message

Browser based test suites fail sporadically on their very first
expectation, although the HTTP request itself succeeds. Two independent
causes were found, plus a diagnostics gap that hid both of them.

1) The web server cannot reach its backend

The page the browser received is an Apache error page instead of the
application:

  Proxy Error
  The proxy server could not handle the request
  Reason: DNS lookup failure for: phpfpm

runTests.sh starts the php-fpm container and the Apache container
directly after each other and then waits for Apache only. Apache
resolves the "phpfpm" network alias lazily, on the first proxied
request. When that request arrives before the container DNS has
registered the alias, mod_proxy answers with its own error page. Once a
mod_proxy worker is put into error state, Apache does not retry it for
60 seconds by default, so a run cannot recover within the test timeout.

"waitFor web 80" only probes the Apache listener, neither the php-fpm
backend nor the resolvability of its network alias is ever awaited. Wait
for the backend before any request can reach Apache. waitFor() now takes
several "host port" pairs and probes them from a single container, so
the added check does not attach another container to the network: every
container attached or detached reconfigures it, which browsers in other
containers observe as a network change.

The failure is not specific to a database backend: in a CI like local
sequence the mariadb and mysql runs failed with the same proxy error.
Verified with podman, PHP 8.2 and postgres 10, using an additional
proxied request right after the readiness probe to amplify the race.
Unmodified 3 of 6 runs failed, patched 0 of 6.

2) The installer never renders after an aborted asset load

The installer entry point ships an empty body and renders the first step
client side, so the headline the specs wait for exists only once the ES
module graph has been evaluated. A network reconfiguration aborts
in-flight requests with ERR_NETWORK_CHANGED, and the browser does not
retry a module request that failed that way. The module graph never
resolves, the body stays empty, and the whole installation is
unreachable. Observed in job 5788444, where the web server logged 22
requests, all with status 200, and no error at all, while the browser
console reported six aborted module requests within one millisecond.

Open the installer through a helper that reloads once when the first
step did not render. The step is idempotent, the installer keeps its
FIRST_INSTALL marker until the last step. Verified by aborting one
module request on the first load only: the run recovers and passes.

3) A failed run left no evidence

Three things are collected now:

* Playwright writes its page snapshot into the output directory
  typo3temp/var/tests/playwright-results, which the CI artifact paths
  did not collect. Collect that directory in the browser test jobs.
* The e2e-install project disables retries, so the shared
  "on-first-retry" trace never triggers. Keep trace and screenshot of
  the single attempt instead.
* The web and php-fpm container logs are written into the log directory
  of the test instance when a run fails. An error raised by the web
  server itself, such as the proxy error above, appears only there and
  was lost with the container.

Differences in the backports:

* main and 14.3 patch the two call sites in runPlaywright() and
  runPlaywrightInstall(). 13.4 has no installer suite, but starts the
  same container pair in three further places, so it patches four call
  sites: runPlaywright() and the acceptance, acceptanceComposer and
  acceptanceInstall suites.
* The installer helper and the trace and screenshot settings apply to
  main and 14.3 only, 13.4 has no installer suite and no such project.
* Container logs are collected for the playwright suites on all
  branches. On 13.4 that is runPlaywright(); the codeception suites keep
  their own report handling.
* The artifact path is added to the e2e and e2e-install jobs on main and
  14.3, and to the e2e and accessibility jobs on 13.4.

The installer JavaScript has no rejection handling on its request chain,
so a request failing after the module graph loaded also leaves an empty
page. That is user facing behaviour and not what caused these failures,
it is left to a dedicated change.

Review

  • Code review CR +2 Done
    • Stefan Bürk +2
    • Benni Mack +1
  • Verified V +2 Done
    • Stefan Bürk +2
    • Benni Mack +1
    • core-ci +1 CI

Issues 2

Files 11

Review log 710

  1. Stefan Bürk uploaded patch set 1 08/13/26, 11:05

  2. core-ci voted on patch set 1 V +1 08/13/26, 11:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/106640

  3. Stefan Bürk uploaded patch set 2 V +1 08/13/26, 11:42

  4. core-ci voted on patch set 2 V +1 08/13/26, 11:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/106644

  5. Stefan Bürk voted on patch set 2 CR -1 08/13/26, 12:39

  6. Stefan Bürk uploaded patch set 3 CR -1 V +1 08/13/26, 12:46

  7. core-ci voted on patch set 3 V +1 08/13/26, 12:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/106649

  8. Benni Mack voted on patch set 3 CR +1 V +1 08/13/26, 13:09

  9. Stefan Bürk voted on patch set 3 CR +2 V +2 08/13/26, 14:10

  10. Stefan Bürk merged patch set 4 08/13/26, 14:10