93324[BUGFIX] Chunk record processing in ReferenceIndex to prevent OOM
Commit message
Review
-
Code review CR +2 Done
- Benni Mack +2
- Stefan Bürk +1
-
Verified V +2 Done
- Benni Mack +2
- Stefan Bürk +1
- core-ci +1 CI
Issues 1
Files 1
Review log 1015
-
Mikel Wohlschlegel uploaded patch set 1 03/24/26, 08:35
-
Mathias Brodala commented on patch set 1 03/24/26, 08:39
How about using the QueryBuilderPaginator instead? -
core-ci voted on patch set 1 V -1 03/24/26, 08:49 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98546
-
Mikel Wohlschlegel rebased onto patch set 2
V -103/24/26, 09:13 -
core-ci voted on patch set 2 V +1 03/24/26, 09:52 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98551
-
Benni Mack updated the commit message in patch set 3
V +104/22/26, 06:44 -
core-ci voted on patch set 3 V +1 04/22/26, 11:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/100777
-
Benni Mack rebased onto patch set 4
V +108/15/26, 09:32 -
core-ci voted on patch set 4 V +1 08/15/26, 09:39 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/106755
-
Stefan Bürk uploaded patch set 5
V +108/31/26, 11:19 -
Stefan Bürk commented on patch set 5 08/31/26, 11:20
Reply
Thanks for the pointer, but `QueryBuilderPaginator` is not a good fit here and I would like to keep the keyset approach. * It paginates with `setFirstResult()` (OFFSET), which is precisely the degradation this change avoids — on a 90k-row `tt_content` the late pages make the DB walk and discard almost the whole table. * `getTotalItems()` adds a CTE-wrapped `COUNT` per table, while `updateIndex()` already counted the table for the progress listener. * `AbstractPaginator::updateInternalState()` clamps an out-of-range page back to the last page instead of returning an empty result, so it cannot express "iterate until exhausted"; driving it needs a page count frozen before the run, which silently skips records when the table changes underneath. * Its docblock scopes it to presentation-layer pagination (language overlay caveats, "item count jumps on pages") — not to a low-level index rebuild. Keyset over the primary key is deterministic, index-backed and needs no count. -
core-ci voted on patch set 5 V +1 08/31/26, 11:26 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/107419
-
Stefan Bürk voted on patch set 5 CR +1 V +1 08/31/26, 11:27
-
Benni Mack voted on patch set 5 CR +2 V +2 08/31/26, 12:58
-
Benni Mack merged patch set 6 08/31/26, 12:58