Review 14.3 Merged

97040[TASK] Reduce resource usage of FAL storage index task

Benni Mack CR +2 V +2 Normal +187 −12 updated 10/28/26, 07:42

Commit message

When detecting missing files, the indexer loaded every sys_file
record of the storage into memory at once and looked up each uid
with in_array() in the list of uids seen on the file system, which
scales with the number of files times the number of records.

The records are now streamed, only uid and identifier are fetched,
and the seen uids are kept as a map, so the lookup is constant.

Functional tests now cover the behaviour of the "Update storage
index" task: indexing new files, updating changed files, marking
deleted files as missing and back, keeping the record of a moved
file, and leaving records of other storages untouched.

Review

  • Code review CR +2 Done
    • Benni Mack +2
  • Verified V +2 Done
    • Benni Mack +2
    • core-ci +1 CI

Issues 1

Files 3

Review log 58

  1. Benni Mack commented on patch set 1 10/23/26, 05:06

    Cherry Picked from branch main.
  2. core-ci voted on patch set 1 V -1 10/23/26, 05:15 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/112207

  3. Benni Mack commented on patch set 2 V -1 10/23/26, 05:15

    Published edit on patch set 1.
  4. core-ci voted on patch set 2 V -1 10/23/26, 05:31 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/112208

  5. Garvin Hicking uploaded patch set 3 V -1 10/27/26, 11:44

    backport fix
  6. core-ci voted on patch set 3 V +1 10/27/26, 11:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/112645

  7. Benni Mack voted on patch set 3 CR +2 V +2 10/28/26, 07:42

  8. Benni Mack merged patch set 4 10/28/26, 07:42