Issue Task Closed

#108352Improve performance of Localization\LocalizationFactory::getMessageCatalogue to make cache:warmup faster

Ayke Halder 11/27/25, 16:17 updated 11/09/26, 11:14

Description

Calls to TYPO3\CMS\Core\Localization\LocalizationFactory::getMessageCatalogue() accumulates a lot of time as this is run more than 400 times on running typo3 cache:warmup.

    protected function getMessageCatalogue(string $fileReference, string $locale, string $domainName, bool $useDefault = true): MessageCatalogueInterface
    {
        $actualSourcePath = $this->labelFileResolver->resolveFileReference($fileReference, $locale, $useDefault);
        // Add the resource to Symfony Translator, if not added yet.
        $cacheIdentifier = 'symfony-translator-localization-factory-' . md5($actualSourcePath . '-' . $locale . '-' . $domainName);
        if (!$this->runtimeCache->has($cacheIdentifier)) {
            // @todo: we need to be more flexible with the file ending here.
            $fileExtension = (string)pathinfo($actualSourcePath, PATHINFO_EXTENSION);
            $this->translator->addResource($fileExtension ?: 'xlf', $actualSourcePath, $locale, $domainName);
            $this->runtimeCache->set($cacheIdentifier, true);
        }
        return $this->translator->getCatalogue($locale);
    }

The following call takes around 0.3 seconds to run:
With $this->translator being Symfony\Component\Translation\Translator.

    $this->translator->addResource($fileExtension ?: 'xlf', $actualSourcePath, $locale, $domainName);
    $this->translator->getCatalogue($locale); // <-- 0.3 seconds on each run

——

TYPO3\CMS\Core\Localization\CacheWarmer::warmupCaches() takes around 80 seconds on running typo3 cache:warmup.

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Core\ExpressionLanguage\ProviderConfigurationLoader':
0.00070405006408691

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Core\Localization\CacheWarmer':
81.924473047256

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Core\Page\ImportMapCacheWarmer':
0.061582088470459

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Fluid\EventListener\CacheWarmupEventListener':
1.9844920635223

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Core\Http\MiddlewareStackResolver':
0.0024302005767822

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Core\Site\Set\SetRegistry':
0.011202096939087

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Core\Configuration\SiteConfiguration':
0.0040719509124756

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Core\Schema\TcaSchemaFactory':
0.097172975540161

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Core\Imaging\IconRegistry':
0.0052101612091064

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Core\Package\PackageManager':
0.0011699199676514

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Dashboard\ServiceProvider':
0.0015008449554443

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Backend\ServiceProvider':
0.00077986717224121

TIME for 'TYPO3\CMS\Core\Cache\Event\CacheWarmupEvent' in 'TYPO3\CMS\Backend\ServiceProvider':
0.00087785720825195

Steps to reproduce:

  1. Apply patch for time measurement
  2. ddev exec time bin/typo3 cache:flush
  3. ddev exec time bin/typo3 cache:warmup --group system
Subject: [PATCH] TIME cache-warmup
---
Index: typo3/sysext/core/Classes/Cache/Event/CacheWarmupEvent.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/typo3/sysext/core/Classes/Cache/Event/CacheWarmupEvent.php b/typo3/sysext/core/Classes/Cache/Event/CacheWarmupEvent.php
--- a/typo3/sysext/core/Classes/Cache/Event/CacheWarmupEvent.php    (revision 0c62b2dd24422d4dbc869e1bdbe84bced4383fca)
+++ b/typo3/sysext/core/Classes/Cache/Event/CacheWarmupEvent.php    (date 1764259566902)
@@ -24,6 +24,10 @@
 {
     private array $errors = [];

+    private ?float $startTime = null;
+
+    private ?string $lastCaller = null;
+
     public function __construct(private readonly array $groups) {}

     public function getGroups(): array
@@ -33,11 +37,20 @@

     public function hasGroup(string $group): bool
     {
+        if($this->lastCaller) {
+            echo "\nTIME for '" . __CLASS__ . "' in '" . $this->lastCaller . "':\n" . (microtime(true) - $this->startTime) . "\n";
+        }
+        $this->lastCaller = debug_backtrace()[1]['class'] ?? 'UNKNOWN';
+        $this->startTime = microtime(true);
         return in_array($group, $this->groups, true);
     }

     public function getErrors(): array
     {
+        if($this->lastCaller) {
+            echo "\nTIME for '" . __CLASS__ . "' in '" . $this->lastCaller . "':\n" . (microtime(true) - $this->startTime) . "\n";
+            $this->lastCaller = null;
+        }
         return $this->errors;
     }

Reviews 3

History 915

  1. Ayke Halder changed subject Improve performance of Localization\CacheWarmer to make cache:warmup faster Improve performance of Localization\LocalizationFactory::getMessageCatalogue to make cache:warmup faster description <span data-processor="relativetime" data-value="2025-11-27T17:15:21+00:00" > 11/27/25, 17:15 </span>

  2. Ayke Halder commented <span data-processor="relativetime" data-value="2025-11-27T17:26:59+00:00" > 11/27/25, 17:26 </span>

    As stated in https://review.typo3.org/c/Packages/TYPO3.CMS/+/91135

    > This has been achieved via the runtime cache, even though this will be removed once TYPO3 adds direct caching to the symfony translation component.

    Maybe adding this direct caching to the symfony translation component will improve the performance.

  3. Ayke Halder commented <span data-processor="relativetime" data-value="2025-11-27T19:29:02+00:00" > 11/27/25, 19:29 </span>

    In TYPO3\CMS\Core\Localization\LocalizationFactory::getMessageCatalogue():

        // This one completely clears the catalog for given $locale
        $this->translator->addResource($fileExtension ?: 'xlf', $actualSourcePath, $locale, $domainName);
    
        // This one recreates the new catalog for $locale for **all known domains** and using **all and every resources/files added in the past**!
        $this->translator->getCatalogue($locale); // <-- 0.3 seconds on each run

    Then the following loop becomes slower and slower with each iteration as more and more resources/files gets added:
    https://github.com/symfony/translation/blob/c586b151e8e06987d905679a11f1dd5cc5bc562b/Translator.php#L372-L381

                foreach ($this->resources[$locale] as $resource) {
                    if (!isset($this->loaders[$resource[0]])) {
                        if (\is_string($resource[1])) {
                            throw new RuntimeException(\sprintf('No loader is registered for the "%s" format when loading the "%s" resource.', $resource[0], $resource[1]));
                        }
    
                        throw new RuntimeException(\sprintf('No loader is registered for the "%s" format.', $resource[0]));
                    }
                    $this->catalogues[$locale]->addCatalogue($this->loaders[$resource[0]]->load($resource[1], $locale, $resource[2]));
                }

    ——

    Side note about the following change:
    https://review.typo3.org/c/Packages/TYPO3.CMS/+/91066/28/typo3/sysext/core/Classes/Localization/CacheWarmer.php

    $domainName is set like $actualSourcePath = /var/www/html/typo3/sysext/backend/Resources/Private/Language/locallang_siteconfiguration.xlf
    Instead of e.g. $domainName = backend.siteconfiguration as before.

    Thereby the array keys in the Translator-loop are becoming very long… performance

  4. Benni Mack commented and changed status Under Review Closed <span data-processor="relativetime" data-value="2026-11-09T11:14:26+00:00" > 11/09/26, 11:14 </span>

    Solved via #108374