Issue Task Under Review Quiet

#90143Redirects: Poor performance of redirect matching for large redirects table

Mario Lubenka 01/17/20, 14:26 updated 10/30/26, 21:54

Description

Problem description

The current redirect matching retrieves all redirects from the database table and stores it in an internal cache.
This is then fetched and filtered for the route that matches the request.

Related files:

With a large redirects table this takes a pretty long time.
I created 350.000 unique redirects and it took the page about 5 seconds to redirect to the target page.
Before you say “350.000 will never be reached”: On a long term a heavily edited website might reach that number at some point in time (assuming this feature will continue to exist with further TYPO3 versions).

Suggested solution

I suggest that the redirect matching is mainly done via the database query.
Most of the logic can be moved into a WHERE clause. Operations which are not available via database operations would then be applied to the remaining records.
With that the above 5 seconds of loading time could be reduced to merely 1 second.

This might also imply that the redirect cache via RedirectCacheService is not needed anymore.
Databases already come with caching mechanisms, as to such I fail to understand why a PHP cache is also needed.

This is a perfomance improvement which should be applied to TYPO3 9.5 as well.

Reviews 2

Files 1

History 729

Show 14 earlier messages
  1. Gerrit Code Review pushed patch set 1 for master to change 62956 status New Under Review <span data-processor="relativetime" data-value="2020-01-17T16:49:10+00:00" > 01/17/20, 16:49 </span>

  2. Mario Lubenka commented and changed attached redirect-measure-time.ods <span data-processor="relativetime" data-value="2020-01-20T15:36:57+00:00" > 01/20/20, 15:36 </span>

    Adding some performance numbers. ;)

    Test Setup

    I installed a TYPO3 10.3.0-dev with the Introduction package and created multiple redirects by adding the following code after Bootstrap::init in index.php and reloaded the page ONCE. After that I removed the code.

        /** @var \TYPO3\CMS\Core\Database\Connection $conn */
        $conn = GeneralUtility::makeInstance(ConnectionPool::class)
            ->getConnectionForTable('sys_redirect');
        $n = 400000; // 400k redirects
        for ($i = 1; $i <= $n; $i++) {
            $conn->insert('sys_redirect', [
                'source_host' => '*',
                'source_path' => '/test' . $i,
                'target' => 't3://page?uid=80'
            ]);
        }

    I did 6 sets with a different number of redirect records in the database (1k up to 400k).
    Before each set, the typo3temp folder was removed to make sure all code caches are regenerated.
    During each set, the page load time until redirect header was taken 6 times.
    The first value was discarded because Cache had to regenerate, leaving us with 5 values for the page load time.

    The time was measured using the following command, returning the time in seconds.

    curl -w %{time_total} --url https://t3-contribution.ddev.site/test10

    Results

    The full data is in the attached redirect-measure-time.ods file.
    The table below shows the average response time until the redirect happens in miliseconds.

    _.# redirects _.Current implementation _.Proposed implementation
    1000 61ms 53ms
    10000 130ms 68ms
    50000 426ms 105ms
    100000 701ms 141ms
    250000 1560ms 263ms
    400000 2384ms 346ms

    As you can see, the proposed implementation is a lot faster when many redirect records in the database.

    With large TYPO3 websites, the current implementation may cause page load times up to 5 seconds, which is what we discovered in our real project.