Issue Bug Under Review Orphaned

#104816Duplicate referenced inline items when triggering translation for multiple languages at the same time

Jennifer Hauß 09/04/24, 13:23 updated 04/24/26, 06:55

Description

Happend in the list module. We have objects with multiple inline references. Translations were triggered in the list module over the ‘localize to’ option. If only one translation is triggered at once, everything translates like it should. However, as the buttons for the different language are right beside each other, some users will click multiple flag-buttons at once and thus trigger multiple translations at once. If that happends the inline references inside the triggered translation will be messed up. There will be multiple duplicated translated inline references and the l10n_parent will be faulty too. In our instance it happend that they all referenced each other and were hard to clean up that way.
E.g. Main object A has inline object B. Translation for A is triggered for multiple languages at once. The translation will look like: translated Object A has now multiple translations of object B (we call them B1, B2, B3). B1 will point at B in the original language for l10n_parent, while B2 will point at B1, and B3 will point at B2 as l10n_parent. This makes it hard to clean them up via the database, but wouldn’t be nessesary if there weren’t any duplicates to begin with.

The easiest solution would be to maybe disable the other language buttons once one translation is triggered or check if there is already a translation process running and prevent the system from doing multiple translations at once. After a few seconds the translation dialouge will open, but by then an impatient user will already have clicked multiple translation buttons.

Reviews 1

History 811

  1. Tim Lochmüller commented <span data-processor="relativetime" data-value="2025-03-24T20:03:07+00:00" > 03/24/25, 20:03 </span>

    Short note: Hey everyone, we have the same problem in one v12 instance. I try to reproduce the problem in TYPO3 v13, but it works well in v13. More updates shortly…

  2. Tim Lochmüller commented <span data-processor="relativetime" data-value="2025-03-24T20:38:19+00:00" > 03/24/25, 20:38 </span>

    I tought I found the problem. Do you use EXT:solr? Found out, that this hook
    ´\ApacheSolrForTypo3\Solr\IndexQueue\RecordMonitor->processDatamap_afterDatabaseOperation` and the Event behind it `RecordUpdatedEvent` force the problem. But in fact, the problem is just the “long runtime” of one request (especially of the hook). The method or the handling of RecordUpdatedEvent seams very slow, why the problem could exist. If I drop the event call and add a `sleep(2)` the problem occur again.

    So if the process is very fast, the “critical path” is not executed in parallel and the problem is gone. If there are too many DataHandler Hooks and long processing times (like the solr hooks), the problem exists and there are mutiple records in the translations. I think it will be hard, to find the problem :-/ Hard to debug…

  3. Jennifer Hauß commented <span data-processor="relativetime" data-value="2025-03-25T08:58:08+00:00" > 03/25/25, 08:58 </span>

    Yes we do in fact use EXT:solr. It’s also a pretty big page with lots of data and lots of relations to translate so it takes quite a while till the translation is done. so that explains why it happens here.

    Maybe the easiest solution would be to disable the translation buttons until one translation is done to prevent users from triggering multiple translations at the same time?

  4. Tim Lochmüller commented <span data-processor="relativetime" data-value="2025-03-25T10:10:41+00:00" > 03/25/25, 10:10 </span>

    hey jennifer,

    we build a test case for this. The problem also occurs, if two different editors trigger two translations at the same time. So just disable the flags for the one user do not solve the problem. I think the problem have to solve in the core, to avoid such problems.

    If anyone has a hint, which resource has to be locked and cause this race condition, feel free to send me a note ;)

    Regards,
    Tim

  5. Tim Lochmüller commented <span data-processor="relativetime" data-value="2025-03-25T11:30:33+00:00" > 03/25/25, 11:30 </span>

    Found the problem:

    The method “copyRecord” call “copyRecord_procBasedOnFieldType” (to copy child elements) BEFORE “$copyTCE->process_datamap();” is called (to create the parent record). The “copyRecord_procBasedOnFieldType” use the “RelationHandler” to create the target record directly in the right language, but the record is related to the orginal record (perhaps: because the record do not exist in the translation process of the children, because the parent is created a few ms later).
    But the “RelationHandler” only check the foreign_field and not for the languages field. As result… if the copy process takes long and a structure like “page A -> child file B” is copied, the second process see “the copy of file B” in the own translation process (via RelationHandler) and result in two files in the own translation process.

    Test to reproduce:
    - Create a news
    - Assign file to the news
    - create a webpage with many languages
    - translate the news folder into all languages
    - switch to list view
    - “Fast click to all flags”

    Result: If the translation process is “slow enough” the first translation has 1 file assigned to the news, the second translation 2 files, the third 3 (and so on…)…

    I have no smart solution to “quick fix” this problem.

    Regards,
    Tim

  6. Thomas Völkel commented <span data-processor="relativetime" data-value="2025-05-07T14:13:56+00:00" > 05/07/25, 14:13 </span>

    The RelationHandler only checks for the foreign field, and the problem is that a copied record (in the first step) contains the same `uid_foreign` as the original record (before it is updated). So if multiple copies are created at the same time, the RelationHandler selects both the original and the processing record.

    Since the selection is based on `uid_foreign`, we will try to modify a record copy before it is stored in the DB for the first time, in our case only for file records.

    class DataHandler extends \TYPO3\CMS\Core\DataHandling\DataHandler
    {
        public function copyRecord($table, $uid, $destPid, $first = false, $overrideValues = [], $excludeFields = '', $language = 0, $ignoreLocalization = false)
        {
            if ($table === 'sys_file_reference') {
                $row = $this->recordInfoWithPermissionCheck($table, (int) $uid, Permission::PAGE_SHOW);
    
                if (!isset($overrideValues['uid_foreign'])) {
                    $overrideValues['uid_foreign'] = -1 * $row['uid_foreign'];
                }
            }
    
            return parent::copyRecord($table, $uid, $destPid, $first, $overrideValues, $excludeFields, $language, $ignoreLocalization);
        }
    }
  7. Tim Lochmüller commented and changed attached example.mov TYPO3 Version 12 14 PHP Version 8.2 8.4 <span data-processor="relativetime" data-value="2026-01-09T08:58:45+00:00" > 01/09/26, 08:58 </span>

    Update: This is a general problem of the DataHandler and is part of all current TYPO3 versions.

    You can easily reproduce this problem, if you add a “sleep(2);” in DataHandler->hook_processDatamap_afterDatabaseOperations to simulate “longer running hooks” in your TYPO3 instance. Attached a video that show the example: It is a text+image element with one image. I added the 2 seconds delay and translate into different languages in parallel. The last CE that will be opened got 5 images in the translation process (because there are 5 translations processes in parallel).

  8. Manuel Munz commented <span data-processor="relativetime" data-value="2026-01-09T09:07:00+00:00" > 01/09/26, 09:07 </span>

    Thanks for the update and the tip for reproduction @spyker

    We also see the issue in TYPO3 v12.4.38 when doing EXT:l10nmgr imports.

    The patch in #note-6 fixes the duplicated sys_file_references problem for us.