#104038Typoscript cache can become quite large
Description
In our setup the typoscript cache gets quite large. We’re seeing ~10MB cache size.
This is not a huge issue if you’re running a few sites, but we’re running a setup with ~3000 TYPO3 sites, having 10MB data that needs to be read for each site affects disk caching rates, which is untenable for us.
I’ve been examining the typoscript cache and it’s not that we’re really storing that much, it’s mostly that the design of the typoscript parser combined with the ineffeciencies of serialize, causes the file to explode in size. I’ve tried replacing it with igbinary_serialize and that cuts the size to ~1/8.
Ofcourse, adding a dependency on igbinary to TYPO3 is probably not a good decision, so that’s not the right fix.
However, Looking through the typoscript cache, I don’t really understand why it’s a code cache, it just defines a PHP file that returns the contents of the serialized objects, it doesn’t use any capability that shouldn’t work with a plain data cache.
I’ve created a patch for this and hope for some feedback.
Reviews 1
History 410
-
Gerrit Code Review pushed patch set 1 for main to change 84603 status
NewUnder Review <span data-processor="relativetime" data-value="2024-06-11T08:31:50+00:00" > 06/11/24, 08:31 </span> -
Gerrit Code Review pushed patch set 2 for main to change 84603 <span data-processor="relativetime" data-value="2024-06-11T08:44:12+00:00" > 06/11/24, 08:44 </span>
-
Gerrit Code Review pushed patch set 3 for main to change 84603 <span data-processor="relativetime" data-value="2024-06-11T08:47:56+00:00" > 06/11/24, 08:47 </span>
-
Gerrit Code Review pushed patch set 4 for main to change 84603 <span data-processor="relativetime" data-value="2024-06-11T09:37:27+00:00" > 06/11/24, 09:37 </span>
-
Gerrit Code Review pushed patch set 5 for main to change 84603 <span data-processor="relativetime" data-value="2024-06-11T09:57:03+00:00" > 06/11/24, 09:57 </span>
-
Gerrit Code Review pushed patch set 6 for main to change 84603 <span data-processor="relativetime" data-value="2024-06-11T10:23:20+00:00" > 06/11/24, 10:23 </span>
-
Rasmus Sallling commented <span data-processor="relativetime" data-value="2024-06-11T10:27:21+00:00" > 06/11/24, 10:27 </span>
Patch is up now.
The patch just contains the raw changes to convert it from a code cache to a data cache.
But I believe some compatibility changes are probably also needed before this could potentially be merged. I guess that’s a premature discussion, but I figure either some code to override the class to VariableFrontend if it’s defined as a PhpFrontend in user configuration. Alternatively the cache could be renamed to avoid that issue completely.
-
Christian Kuhn commented <span data-processor="relativetime" data-value="2024-06-11T13:10:57+00:00" > 06/11/24, 13:10 </span>
We’ll probably switch away from serialize/unserialize towards a direct creation of data objects using new() in cache files.
This promises higher performance since opcache can kick in. It will require not switching away from PhpFrontend.
I’m currently working within the parser code to allow this (among other things), so we can see if that works out.
We’ll end up with smaller files along the way. -
Rasmus Sallling commented <span data-processor="relativetime" data-value="2024-06-11T15:10:49+00:00" > 06/11/24, 15:10 </span>
Makes sense, not sure I completely share the your apparent comfort with dynamically creating php code, but I can certainly see it performing better than the current serialize/unserialize approach.
Will track your progress instead then. Thanks for the review.
thanks for your issue, just a FYI: no patch yet added