64407[BUGFIX] Fix handling of deprecated cache configuration
Commit message
Review
-
Code review CR +2 Done
- Benjamin Franzke +2
- Georg Ringer +1
- Oli Bartsch +1
- Johannes Kasberger +1
-
Verified V +2 Done
- Benjamin Franzke +2
- Georg Ringer +1
- Johannes Kasberger +1
- TYPO3com +1 CI
Issues 1
Files 2
Review log 2027
Show 12 earlier messages
-
Markus Klein uploaded patch set 1 05/05/20, 11:33
-
TYPO3com voted on patch set 1 V +1 05/05/20, 11:44 Completed build in 11m 17s on Tue, 5 May, 11:44 AM Test Summary: 73663 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3202
-
Markus Klein updated the commit message in patch set 2 05/05/20, 11:53
-
Johannes Kasberger voted on patch set 2 V +1 CR +1 05/05/20, 12:02
-
TYPO3com voted on patch set 2 V +1 05/05/20, 12:03 Completed build in 10m 31s on Tue, 5 May, 12:03 PM Test Summary: 73663 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3203
-
Georg Ringer voted on patch set 2 CR +1 05/05/20, 12:27
-
Guido Schmechel commented on patch set 2 05/05/20, 12:32
-
Markus Klein uploaded patch set 3 05/05/20, 12:44
-
Markus Klein commented on patch set 3 05/05/20, 12:45
CacheManager.php:97ReplyDone -
Johannes Kasberger voted on patch set 3 V +1 CR +1 05/05/20, 12:45
-
Guido Schmechel commented on patch set 3 CR +1 05/05/20, 12:51
Sorry :-) -
TYPO3com voted on patch set 3 V -1 05/05/20, 12:56 Completed build in 11m 32s on Tue, 5 May, 12:56 PM Test Summary: 1 of 74148 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3204
-
Markus Klein uploaded patch set 4 05/05/20, 13:08
-
TYPO3com voted on patch set 3 V +1 05/05/20, 13:09 Completed build in 36m 07s on Tue, 5 May, 01:09 PM Test Summary: 73663 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3204
-
Markus Klein uploaded patch set 5 05/05/20, 13:13
-
TYPO3com voted on patch set 4 V -1 05/05/20, 13:19 Completed build in 11m 45s on Tue, 5 May, 01:19 PM Test Summary: 6 of 74153 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3205
-
TYPO3com voted on patch set 5 V -1 05/05/20, 13:25 Completed build in 12m 09s on Tue, 5 May, 01:25 PM Test Summary: 2 of 74153 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3206
-
TYPO3com voted on patch set 5 V +1 05/05/20, 13:38 Completed build in 37m 40s on Tue, 5 May, 01:38 PM Test Summary: 73669 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3206
-
Johannes Kasberger voted on patch set 5 V +1 CR +1 05/06/20, 08:30
-
Georg Ringer voted on patch set 5 V +1 CR +1 05/06/20, 14:53
-
Oli Bartsch voted on patch set 5 CR +1 05/07/20, 09:59
-
Benjamin Franzke commented on patch set 5 05/08/20, 13:07
In the issue you wrote: > $GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['cache_pages']['backend'] = \TYPO3\CMS\Core\Cache\Backend\RedisBackend::class; > According to documentation this should still work, but it does not. May you post a link to the documentation that says that this is still supported? I'm unable to find it. We have to differentiate between these two cases: * Support for deprecated cache configuration using cache_ prefix (which is still supported) * Support for overwriting new cache identifier configurations with deprecated identifier names (which this BUGFIX tries to fix) This change has the drawback that a configuration for a cache using the *new* naming scheme would be overwritten by an *earlier* loaded extension that uses the deprecated naming scheme. To be fair: I'm not sure, maybe the impact of this bugfix is not that problematic because my scenario is quite unlikely. Still I'd like to know where documentation says that this is still supported. -
Markus Klein commented on patch set 5 05/08/20, 13:50
> Patch Set 5: > > In the issue you wrote: > > > $GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['cache_pages']['backend'] = \TYPO3\CMS\Core\Cache\Backend\RedisBackend::class; > > According to documentation this should still work, but it does not. > > May you post a link to the documentation that says that this is still supported? I'm unable to find it. > > > We have to differentiate between these two cases: > > * Support for deprecated cache configuration using cache_ prefix (which is still supported) > * Support for overwriting new cache identifier configurations with deprecated identifier names (which this BUGFIX tries to fix) > > > This change has the drawback that a configuration for a cache using the *new* naming scheme would be overwritten by an *earlier* loaded extension that uses the deprecated naming scheme. > > To be fair: I'm not sure, maybe the impact of this bugfix is not that problematic because my scenario is quite unlikely. Still I'd like to know where documentation says that this is still supported. Without even looking into documentation: As long as the cache_* names are not marked as breaking change, this simply MUST be handled. As simple as that. typo3/sysext/core/Documentation/Changelog/10.0/Deprecation-88366-DefaultCachingFrameworkCacheNamesChanged.rst The big problem here is not that things fall apart because of the bug (well bugs happen and can be fixed) the problem is that there is no obvious error message or the like. I would rather call it "it fails silently", which is the worst case. (In our case we used Redis only on the live system, and things only went south there.) Having the cache groups being removed, like it happens with the example, causes the cache flush commands to to NOTHING. -
Benjamin Franzke commented on patch set 5 05/08/20, 14:52
Actually Benni added a silent configuration upgrade for your usecase: https://review.typo3.org/c/Packages/TYPO3.CMS/+/59661/7/typo3/sysext/install/Classes/Service/SilentConfigurationUpgradeService.php The actual problem (which we should generally tackle) is that those silent upgrades dont catch settings in AdditionalConfiguration.php and ext_localconf.php On the other hand: cache configuration *overwrites* (not new definitions) in ext_localconf.php were never supported in TYPO3 v7/v8/v9 (yes, they worked, sometimes…). Why? Not because someone documented it, but for technical reason: It was possible (it is not longer in v10) that the cache, that you are trying to configure (in ext_localconf.php), was already initialized at the time that you are trying to add configuration for that cache. That would happen when a class is instantiated in ext_localconf that calls `getCache('cache_pages')` in the constructor. Such a change could even have been introduced within a BUGFIX release. That would have broken your scenario of not testing the production code path on your staging/testing server as well. Therefore you could never safely rely on cache configuration overwrites via ext_localconf.php Note: To prevent such unforeseeable problems the patches https://review.typo3.org/59231 and https://review.typo3.org/c/Packages/TYPO3.CMS/+/58255/71/typo3/sysext/core/Classes/ServiceProvider.php#45 clarified the context where CacheManager is allowed to be used in v10. -
Markus Klein commented on patch set 5 05/08/20, 15:19
> Patch Set 5: > > Actually Benni added a silent configuration upgrade for your usecase: https://review.typo3.org/c/Packages/TYPO3.CMS/+/59661/7/typo3/sysext/install/Classes/Service/SilentConfigurationUpgradeService.php > > The actual problem (which we should generally tackle) is that those silent upgrades dont catch settings in AdditionalConfiguration.php and ext_localconf.php Exactly! > > On the other hand: cache configuration *overwrites* (not new definitions) in ext_localconf.php were never supported in TYPO3 v7/v8/v9 (yes, they worked, sometimes…). > nope, not sometimes, always!! Configuration in TYPO3_CONF_VARS has always been adjustable in ext_localconf.php > Why? Not because someone documented it, but for technical reason: > It was possible (it is not longer in v10) that the cache, that you are trying to configure (in ext_localconf.php), was already initialized at the time that you are trying to add configuration for that cache. And here you go: undocumented breaking change. That means that cache configuration would need to be moved away from TYPO3_CONF_VARS. > That would happen when a class is instantiated in ext_localconf that calls `getCache('cache_pages')` in the constructor. [ref1] And we would have noticed before if some extension did so, I presume. This has nothing to do with the current issue, IMO. > Such a change could even have been introduced within a BUGFIX release. That would have broken your scenario of not testing the production code path on your staging/testing server as well. > Therefore you could never safely rely on cache configuration overwrites via ext_localconf.php > ext_localconf.php has the sole purpose of configuring the instance. ;-) > Note: To prevent such unforeseeable problems the patches https://review.typo3.org/59231 and https://review.typo3.org/c/Packages/TYPO3.CMS/+/58255/71/typo3/sysext/core/Classes/ServiceProvider.php#45 clarified the context where CacheManager is allowed to be used in v10. So this is great, because it invalidates [ref1] above and makes the cache config via ext_localconf possible again. For me there are these things to consider: 1) never fail without any log or error message. clearly violated in this case 2) if something was "only" deprecated, it has to work as before. Not a single step to the left or right. 3) this patch makes an existing use case working again 4) if cache configuration is not "allowed" in ext_localconf anymore, file a breaking change rst! communiation! 5) if cache configuration is "static" and must not be modified during runtime, move it away from TYPO3_CONF_VARS. My sole intention with this patch for other integrators of TYPO3 is to safe them the time for searching their ass why FE behaves weird, just because they overlooked adjusting their cache configuration. If you don't agree on the approach, propose a better one, but 2) still counts. Of course 4) is a "solution" as well, but I would not call this clever in terms of outside perception of the project as we strive to have most breaking changes early and not on a released LTS. -
Benjamin Franzke commented on patch set 5 V +2 CR +2 05/20/20, 22:47
I gave this some more thoughts… I still think this is not a good solution (because deprecated configuration overwrites non deprecated settings), but it seems its the least worst of the solutions we can come up with. The issue really was/is that https://review.typo3.org/c/Packages/TYPO3.CMS/+/60892 introduced a buggy handling of deprecated cache definitions. That patch should rather have dropped/ignored deprecated keys, once there is a configuration available using the *new* naming scheme. Why? It was the intention of that patch to support the old school way of *defining* caches, not the oldschool way of *overwriting* cache entries using deprecated identifier that have already be migrated (as that was already handled within the silent migration wizard). Anyway, why do I still think this solution is better (less ugly) than the proper fix? Well, we now have the situation that deprecated cache *overwrites* became somehow supported (due to that patch) and we can't simply revert that (would be another breaking cache). Now, as said in earlier comments, this patch is only problematic for the – in v9 – unsupported case of *overwriting* cache configuration via ext_localconf.php (where a usage of the deprecated naming scheme in an early extension will now overwrite non-deprecated settings for a later extension) – but that doesn't break existing cache overwrites, as that flaw was already intdroduced by 60892. On the positive side this patch is fixing existing (speaking of v9) configurations via AdditionalConfiguraton.php (which the silent configuration wizard is unable to handle). For these reasons I'm now going to merge this. Tested against current master, still works. Thanks for your patience. -
Benjamin Franzke merged patch set 6 05/20/20, 22:47
CacheManager.php:97