#106037EXT:form Backend module can run into timeout on large sites
Description
On a large setup with many sites (hundreds) and similar numbers of forms (483 in my example) and lots of references (+2000) the form backend module often runs into timeouts when simply loading the list of forms.
The culprit actually is the wayTYPO3\CMS\Form\Service\DatabaseService->getAllReferencesForFileUid()and consequently
TYPO3\CMS\Form\Service\DatabaseService->getAllReferences()tries to load these from sys_refindex
Here essentially a
SELECT ref_uid FROM sys_refindex WHERE softref_key='formPersistenceIdentifier' and ref_uid > 0;
is executed, which fails to match any index, resulting in a full table scan, grinding the task to a halt on large setups (sys_refindex has a count of 3 Million+ entries)
I added the following to my Sitepackage, solving the Problem:
CREATE TABLE sys_refindex (
KEY idx_softref_key (softref_key,ref_uid)
);
I am filing this for EXT:form / Form Framework as it is the only example I found where the sys_refindex is queried like that, but it could be something for the core as well, if the sys_refindex is intended to be used in this way.
Reviews 3
History 612
-
Garvin Hicking commented <span data-processor="relativetime" data-value="2025-01-30T19:22:45+00:00" > 01/30/25, 19:22 </span>
-
Frank Berger commented <span data-processor="relativetime" data-value="2025-01-31T10:56:18+00:00" > 01/31/25, 10:56 </span>
will do!
-
Christian Kuhn commented <span data-processor="relativetime" data-value="2025-01-31T13:41:59+00:00" > 01/31/25, 13:41 </span>
please add it to ext:form ext_tables.sql, not to ext:core. There is an index for a similar query for ext:form already.
-
Gerrit Code Review pushed patch set 1 for main to change 88262 status
NewUnder Review <span data-processor="relativetime" data-value="2025-02-18T13:00:05+00:00" > 02/18/25, 13:00 </span> -
Gerrit Code Review pushed patch set 1 for 13.4 to change 88284 <span data-processor="relativetime" data-value="2025-02-19T20:41:41+00:00" > 02/19/25, 20:41 </span>
-
Gerrit Code Review pushed patch set 1 for 12.4 to change 88285 <span data-processor="relativetime" data-value="2025-02-19T20:41:56+00:00" > 02/19/25, 20:41 </span>
-
Gerrit Code Review pushed patch set 2 for 12.4 to change 88285 <span data-processor="relativetime" data-value="2025-02-19T20:42:46+00:00" > 02/19/25, 20:42 </span>
-
Gerrit Code Review pushed patch set 2 for main to change 88262 <span data-processor="relativetime" data-value="2025-02-20T11:22:21+00:00" > 02/20/25, 11:22 </span>
-
Gerrit Code Review pushed patch set 2 for 13.4 to change 88284 <span data-processor="relativetime" data-value="2025-02-20T11:23:19+00:00" > 02/20/25, 11:23 </span>
-
Garvin Hicking commented <span data-processor="relativetime" data-value="2025-02-20T11:24:38+00:00" > 02/20/25, 11:24 </span>
We will merge this, however not down to TYPO3 v12 due to changes in DataHandler and sys_refindex where we do not want to have cross-concerns.
If this is needed in a v12 instance, it would be best to add the index in a sitepackage or something similar.
-
Garvin Hicking commented and changed status
Under ReviewResolved done0 %100 % <span data-processor="relativetime" data-value="2025-02-20T12:45:09+00:00" > 02/20/25, 12:45 </span>Applied in changeset commit:966c6460e24623f585f35014bc1b50e85309507a.
-
Benni Mack changed status
ResolvedClosed <span data-processor="relativetime" data-value="2025-12-02T14:26:10+00:00" > 12/02/25, 14:26 </span>
That sounds effective to use a key here. Would you like to propose a patch yourself maybe? Should be doable with an ext_tables.sql change only (in
typo3/sysext/core/ext_tables.sqlline 235)?