Issue Bug Closed

#106037EXT:form Backend module can run into timeout on large sites

Frank Berger 01/30/25, 18:18 updated 12/02/25, 14:26

Description

On a large setup with many sites (hundreds) and similar numbers of forms (483 in my example) and lots of references (+2000) the form backend module often runs into timeouts when simply loading the list of forms.

The culprit actually is the way
TYPO3\CMS\Form\Service\DatabaseService->getAllReferencesForFileUid()
and consequently
TYPO3\CMS\Form\Service\DatabaseService->getAllReferences()
tries to load these from sys_refindex

Here essentially a

SELECT ref_uid FROM sys_refindex WHERE softref_key='formPersistenceIdentifier' and ref_uid > 0;

is executed, which fails to match any index, resulting in a full table scan, grinding the task to a halt on large setups (sys_refindex has a count of 3 Million+ entries)

I added the following to my Sitepackage, solving the Problem:

CREATE TABLE sys_refindex (
    KEY idx_softref_key (softref_key,ref_uid)
);

I am filing this for EXT:form / Form Framework as it is the only example I found where the sys_refindex is queried like that, but it could be something for the core as well, if the sys_refindex is intended to be used in this way.

Reviews 3

History 612

  1. Garvin Hicking commented <span data-processor="relativetime" data-value="2025-01-30T19:22:45+00:00" > 01/30/25, 19:22 </span>

    That sounds effective to use a key here. Would you like to propose a patch yourself maybe? Should be doable with an ext_tables.sql change only (in typo3/sysext/core/ext_tables.sql line 235)?

  2. Christian Kuhn commented <span data-processor="relativetime" data-value="2025-01-31T13:41:59+00:00" > 01/31/25, 13:41 </span>

    please add it to ext:form ext_tables.sql, not to ext:core. There is an index for a similar query for ext:form already.

  3. Garvin Hicking commented <span data-processor="relativetime" data-value="2025-02-20T11:24:38+00:00" > 02/20/25, 11:24 </span>

    We will merge this, however not down to TYPO3 v12 due to changes in DataHandler and sys_refindex where we do not want to have cross-concerns.

    If this is needed in a v12 instance, it would be best to add the index in a sitepackage or something similar.

  4. Garvin Hicking commented and changed status Under Review Resolved done 0 % 100 % <span data-processor="relativetime" data-value="2025-02-20T12:45:09+00:00" > 02/20/25, 12:45 </span>

    Applied in changeset commit:966c6460e24623f585f35014bc1b50e85309507a.