Issue Bug New Stale

#110572DatabaseRowsUpdateWizard loads all rows of a table in one unbounded query

Stefan Bürk 08/31/26, 11:15 updated 08/31/26, 11:15

Description

Problem

DatabaseRowsUpdateWizard::executeUpdate() iterates over every row of each table that has a row updater attached. The rows are read with a single unbounded query, essentially a select * from the table ordered by uid, and are then streamed with fetchAssociative().

Streaming on the PHP side does not help here. Both database drivers TYPO3 supports for MySQL and MariaDB buffer the complete result set on the client before the first row is handed back, mysqli through store_result and pdo_mysql through the default of PDO::MYSQL_ATTR_USE_BUFFERED_QUERY. pdo_pgsql behaves the same way. The full table is therefore materialized in memory and that allocation counts against the PHP memory limit.

For wide tables with many rows, for example tt_content with roughly 132 columns and 90.000 records, this exhausts even generous memory limits and the upgrade wizard dies with a fatal out of memory error. As this runs during an upgrade, and on exactly the kind of installation that is largest, the practical impact is arguably worse than the same defect in the reference index.

The uid constraint that is already applied to the query is not a fix for this. It is only used to resume after an interrupted run, it comes from the stored start position and it never bounds an individual query.

Solution

Use keyset pagination on the primary key, the same approach that issue #109334 applies to ReferenceIndex::updateIndex(): read a bounded number of rows per query, remember the last handled uid, continue until a chunk comes back short, and release the result between chunks.

Two things to keep in mind for the implementation:

  • Switching to an unbuffered query is no option, because the loop body issues further queries on the same connection, including a transaction around the row update and the sys_registry bookkeeping.
  • In contrast to the reference index, this loop modifies and may delete rows of the very table it traverses. Keyset pagination stays correct, since the primary key of a handled row is never changed and only rows at or below the cursor are touched, but the interaction with the resume logic that persists the start position needs to be verified.

Related

  • #109334 for the same defect in ReferenceIndex::updateIndex(), addressed by change 93324 on review.typo3.org

Reviews 0

No change resolves or names this issue in its commit message, and no comment links one.

Fix this issue

Start the commit message of your patch with this. Add what the patch does, and the versions it goes to in "Releases:". How to create a patch

Commit message
[BUGFIX] DatabaseRowsUpdateWizard loads all rows of a table in one unbounded query Resolves: #110572 Releases: main