Review main Merged

96810[TASK] Reduce resource usage of FAL storage index task

Benni Mack CR +2 V +2 Normal +189 −12 updated 10/23/26, 05:06

Commit message

When detecting missing files, the indexer loaded every sys_file
record of the storage into memory at once and looked up each uid
with in_array() in the list of uids seen on the file system, which
scales with the number of files times the number of records.

The records are now streamed, only uid and identifier are fetched,
and the seen uids are kept as a map, so the lookup is constant.

Functional tests now cover the behaviour of the "Update storage
index" task: indexing new files, updating changed files, marking
deleted files as missing and back, keeping the record of a moved
file, and leaving records of other storages untouched.

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Georg Ringer +1
    • Oliver Klee +1
    • Wouter Wolters +1
    • Alexander Schnitzler +1
  • Verified V +2 Done
    • Benni Mack +2
    • Georg Ringer +1
    • Oliver Klee +1
    • Wouter Wolters +1
    • Alexander Schnitzler +1
    • core-ci +1 CI

Issues 1

Files 3

Review log 1214

  1. Benni Mack uploaded patch set 1 10/18/26, 07:32

  2. Benni Mack marked it ready for review 10/18/26, 07:32

  3. core-ci voted on patch set 1 V +1 10/18/26, 07:41 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/111508

  4. Guido Schmechel voted on patch set 1 CR +1 V +1 10/18/26, 09:02

  5. Oliver Klee voted on patch set 1 CR +1 V +1 10/18/26, 09:25

  6. Wouter Wolters voted on patch set 1 CR +1 V +1 10/18/26, 10:24

  7. Benni Mack rebased onto patch set 2 CR +1 V +1 10/18/26, 13:33

  8. Alexander Schnitzler voted on patch set 2 CR +1 V +1 10/18/26, 13:35

  9. core-ci voted on patch set 2 V +1 10/18/26, 13:43 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/111521

  10. Wouter Wolters voted on patch set 2 CR +1 V +1 10/18/26, 13:46

  11. Oliver Klee voted on patch set 2 CR +1 V +1 10/18/26, 14:17

  12. Georg Ringer voted on patch set 2 CR +1 V +1 10/22/26, 20:49

  13. Benni Mack voted on patch set 2 CR +2 V +2 10/23/26, 05:06

  14. Benni Mack merged patch set 3 10/23/26, 05:06