#110573Reference index silently discards a whole batch of rows on a hash collision
Description
Problem
ReferenceIndex::updateRefIndexTable collects all new reference index rows of a
record and writes them with a single bulkInsert call. That call is wrapped in a
try/catch which swallows every exception. It was added with #107139 to keep a
workspace publish from being aborted.
The sys_refindex table has a primary key on the hash column, and a bulk insert
is all-or-nothing. When two processes update the reference index of the same
record at the same time, the second insert hits a duplicate key for at least one
hash and the complete batch is discarded. Nothing is logged, the caller sees no
error, and sys_refindex stays incomplete until the next update of that record or
until a full referenceindex:update run.
Impact
The reference index backs the referenced elements information in the backend,
the delete protection for files and records, and workspace publishing. Missing
rows are not visible to an editor and are hard to notice.
Suggested direction
Make the write idempotent instead of silencing the failure, so that no exception
can occur in the first place. Two options:
- write the rows one by one with an insert-or-ignore, or an upsert on the hash
column - or keep the bulk write and retry only the rows that collided
Both work across several web nodes and would allow the catch-all to be removed
again.
Background
Found while reviewing https://review.typo3.org/c/Packages/TYPO3.CMS/+/93035 for
issue #109089. That change proposed an exclusive lock around
ReferenceIndexUpdater to avoid the duplicate key exception. A lock serializes
every backend save of the whole installation and was therefore dropped from that
change. The author of the change reported in issue #109089 that they removed the
same lock from their own production instance again after complaints about
blocked saves.
Reviews 0
No change resolves or names this issue in its commit message, and no comment links one.
Fix this issue
Start the commit message of your patch with this. Add what the patch does, and the versions it goes to in "Releases:". How to create a patch
- Commit message
-
[BUGFIX] Reference index silently discards a whole batch of rows on a hash collision Resolves: #110573 Releases: main