Review main Abandoned

94925[WIP][TASK] Share nightly composer install via artifact

Stefan Bürk CR 0 V 0 Normal +91 −24 updated 08/01/26, 15:17

Commit message

The nightly pipeline resolves only two distinct composer dependency
sets, "max" (composer update) and "min" (composer update
--prefer-lowest), yet every functional and unit consumer job ran the
identical resolution independently - around two dozen full "composer
update" runs per nightly.

Resolve each set once in a dedicated "prepare" stage and publish the
installed instance as an artifact: vendor/, bin/ and the git-ignored
web-dir scaffolding that composer generates (index.php, .htaccess,
typo3conf/, fileadmin/, typo3temp/) - without the latter the testing
framework aborts with "Unable to determine path to entry script". The
functional (8) and unit (4) consumer jobs now depend on the matching
prepare job, unpack the artifact and run the tests, instead of
resolving dependencies themselves. The installed tree is relocatable
- it contains no symlinks, and composer autoload and binstubs resolve
paths at runtime relative to their own location - so it can be built
in one job and reused in another workspace.

This trades roughly two dozen redundant dependency resolutions per set
for a single one, reducing aggregate runner load on the shared
infrastructure; per-pipeline wall-clock is expected to stay roughly
neutral. The pack/unpack is kept inline in the CI definition for now;
if an infrastructure measurement confirms the benefit, it should move
into runTests.sh suites and extend to the remaining composerInstall
consumers.

Unlike everything else in the pipeline, the pack step does not run in
a container but directly on the runner, whose shell provides neither
"git" nor a guaranteed set of external tools. It therefore selects
the paths to archive with shell builtins, skipping the ones composer
did not create - none of them holds a tracked file, so archiving them
as a whole is equivalent to selecting the generated files.

Review

  • Code review CR 0
  • Verified V 0
    • core-ci +1 CI

Files 4

Review log 613

  1. Stefan Bürk uploaded patch set 1 07/24/26, 13:14

  2. Stefan Bürk wrote 07/24/26, 13:22

    Topic set to core-ci
  3. core-ci voted on patch set 1 V +1 07/24/26, 13:25 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105314

  4. core-ci voted on patch set 1 V -1 07/24/26, 13:56 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105317

  5. Stefan Bürk uploaded patch set 2 V -1 07/24/26, 14:05

  6. core-ci voted on patch set 2 V +1 07/24/26, 14:22 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105320

  7. Stefan Bürk uploaded patch set 3 V +1 08/01/26, 13:47

  8. core-ci voted on patch set 3 V +1 08/01/26, 13:57 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105800

  9. core-ci voted on patch set 3 V -1 08/01/26, 14:35 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105803

  10. Stefan Bürk uploaded patch set 4 V -1 08/01/26, 14:36

  11. core-ci voted on patch set 4 V +1 08/01/26, 14:44 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105804

  12. core-ci commented on patch set 4 08/01/26, 14:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105805

  13. Stefan Bürk abandoned the change 08/01/26, 15:17

    Measured on the nightly this change was meant to speed up (pipeline 105805 against 105802, which differs only by this change): it does not pay off. Normalised against the jobs it does not touch, the 48 consumer jobs end up roughly 250s slower in aggregate, plus 47s for the two prepare jobs, and the new prepare stage delays the first functional job from +3s to +22s. The premise does not hold on this infrastructure. A full composerInstallMax in a consumer job takes 4-5s there (warm .cache, 129 packages), while the 42.8 MB artifact costs 4-5s to download plus 1s to unpack - so per job it is a wash and the prepare jobs are pure additional cost. The number of resolutions (24 to 1) was right, the cost per resolution was off by an order of magnitude. Abandoning rather than reworking: the functional stage is 89 percent of the nightly job seconds and the pipeline duration equals its slowest shard, so the shard balance is where the time is. Sharing the composer install is not a prerequisite for that work. Follow-up patch rebalances the parallel counts.