Issue Bug Closed

#104407OrphanRecordsCommand fails if list of ids is too long

Roman Teitge 07/17/24, 08:50 updated 11/01/24, 22:15

Description

When running typo3 cleanup:orphanrecords -v via CLI in a big instance mysql throws error because of exceeding the query size limit.
At https://github.com/TYPO3-CMS/lowlevel/blob/main/Classes/Command/OrphanRecordsCommand.php#L102 it searches for all records that are NOT connected. But the $idList array parameter for the notIn() condition is very long in a big instance. And it grows if new entries are added, so raising the query size limit would do it only for the moment. Not to mention the consumption of resources (RAM, CPU).

Reviews 0

No change resolves or names this issue in its commit message, and no comment links one.

History 4

  1. Garvin Hicking changed category 07/31/24, 08:56

  2. Benni Mack commented and changed status New Needs Feedback 10/07/24, 13:19

    Hmm, that might also be an issue in other places then.

    Can you give us some more context:

    a) how many pages do you have in your installation?
    b) does this also occur in v12?
    c) what option did you tune MySQL to make the query size bigger?

  3. Roman Teitge commented 10/22/24, 12:01

    Benni Mack wrote in #note-2:
    > Hmm, that might also be an issue in other places then.
    >
    > Can you give us some more context:
    >
    > a) how many pages do you have in your installation?
    We’ve approx. 86k pages over 700 root pages.
    > b) does this also occur in v12?
    With our update to v12 which is in test stage now there is no error anymore like this. We only need to run this process with a php memory_limit 2G flag.
    > c) what option did you tune MySQL to make the query size bigger?
    You can raise the max_allowed_packet option in MySQL (https://dev.mysql.com/doc/refman/8.4/en/packet-too-large.html) or MariaDB (https://mariadb.com/docs/server/ref/mdb/system-variables/max_allowed_packet/)

    So it seems to be a v11 issue and not v12.

  4. Benni Mack commented and changed status Needs Feedback Closed 11/01/24, 22:15

    Hey Roman,

    thanks for your feedback. I will close this issue now as TYPO3 v11 is now in ELTS, thus in the hands of TYPO3 GmbH and won’t receive any updates anymore. I’m however happy to hear that v12 solves your issues.