Review 14.3 Merged

96190[BUGFIX] Remove garbage collected rows in batches

Georg Ringer CR +2 V +2 Hard +566 −25 updated 09/30/26, 05:47

Commit message

The table garbage collection task removes all expired rows of a table
with a single DELETE statement. On a table that has grown to millions
of rows, sys_log and sys_history being the usual candidates, that
statement is one huge transaction: it holds locks for its whole
runtime and can exceed transaction size limits, which fails the task
with "Transaction size exceed set threshold" on a Galera cluster.

The expired rows are now counted first and then removed in batches of
at most 10000 rows. Every round determines the primary key value of
the last row of the next batch with an ordered and limited select,
and removes the expired rows up to that boundary, which bounds a
single statement to the size of a batch. The round that finds less
than a full batch left removes the remainder. Both statements are
prepared once per table and re-executed per round.

The number of rounds is derived from the counted rows, with one
additional round for rows expiring while the task is running. Rows
left over after that are removed by the next run of the task.

The batches are deliberately not wrapped into a transaction: every
batch is committed on its own, so concurrent operations on the table
are not blocked for the whole runtime of the task.

A table without a primary key, or with a primary key spanning several
columns, has no boundary the batches could be limited with. Such a
table, and any table with less expired rows than the size of a batch,
is still cleaned up with a single statement.

The error of a failed cleanup now names the table that was handled
instead of the selected one, which is empty when all tables are
cleaned up at once.

Review

  • Code review CR +2 Done
    • Georg Ringer +2
  • Verified V +2 Done
    • Georg Ringer +2
    • core-ci +1 CI

Issues 1

Files 6

Review log 34

  1. Georg Ringer commented on patch set 1 09/30/26, 05:32

    Cherry Picked from branch main.
  2. core-ci voted on patch set 1 V +1 09/30/26, 05:43 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/109564

  3. Georg Ringer voted on patch set 1 CR +2 V +2 09/30/26, 05:47

  4. Georg Ringer merged patch set 2 09/30/26, 05:47