Review 13.4 Merged

88259[TASK] Drastically speed up `typo3 cleanup:orphanrecords`

Christian Kuhn CR +2 V +2 Hard +198 −199 updated 02/18/25, 11:44

Commit message

The CLI command implementation of cleanup:orphanrecords is truly awful:
It is extremely expensive and can take minutes if not hours with only
mid-sized TYPO3 instances.

Its main goal is to find and delete pages and attached records that
have no proper connection to the tree root - page rows that are not
part of a hierarchical structure, plus TCA records attached to such
invalid pages.

The code has been moved around since early TYPO3 3.x, later
implementing doctrine and symfony commands, while the algorithms has
never been refactored.

Since TYPO3 v12/v13 - even considering workspace overlays - the page
tree is strict: Each page *must* point to a pid that is either zero,
or a pages row that points to a page having pid zero, recursive. Any
pages not following this are "orphan".

The same goes for all TCA records: Their pid is either zero, or they
point to a list of pages that point to zero.

The patch rewrites the algorithm of `typo3 cleanup:orphanrecords`
leading to dramatic performance improvements.

Stats with a local test instance *before patch*:
Queries: 10.7 Mio
Runtime: 10 minutes 21 seconds

Stats with a local test instance *with patch*:
Queries: 116
Runtime: 0.2 seconds

Stats of course vary depending on instance and network capabilities.
However, casual TYPO3 instances should go down from minutes or hours
to a few seconds.

The change also removes dependency to DataHandler: Orphan pages and
records are removed from database directly.

Review

  • Code review CR +2 Done
    • Christian Kuhn +2
  • Verified V +2 Done
    • Christian Kuhn +2
    • core-ci +1 CI

Issues 1

Files 4

Review log 34

  1. Christian Kuhn uploaded patch set 1 02/18/25, 11:27

  2. core-ci voted on patch set 1 V +1 02/18/25, 11:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/74306

  3. Christian Kuhn voted on patch set 1 CR +2 V +2 02/18/25, 11:44

  4. Christian Kuhn merged patch set 2 02/18/25, 11:44