84016[BUGFIX] Circumvent possible race condition in cache retrieval
Commit message
Review
-
Code review CR +2 Done
- Benjamin Franzke +2
- Oliver Klee +1
- Stefan Bürk +1
- Nikita Hovratov +1
-
Verified V +2 Done
- Benjamin Franzke +2
- Stefan Bürk +1
- core-ci +1 CI
Issues 1
Files 4
Review log 2732
Show 57 earlier messages
-
Patrick Schriner uploaded patch set 1 04/24/24, 11:40
-
core-ci voted on patch set 1 V +1 04/24/24, 11:45 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56656
-
Patrick Schriner updated the commit message in patch set 2
V +104/24/24, 11:46 -
Andreas Kienast commented on patch set 1 04/24/24, 11:46
Do you stumble across an actual bug? To me, the scenario where a TypeError is issued is highly hypothetical and doesn't happen in any way, ever. -
core-ci voted on patch set 2 V +1 04/24/24, 11:53 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56657
-
Christian Kuhn commented on patch set 2 04/24/24, 11:56
Reply
Same question: This sounds like an edge case: * may happen when a core cache-clear happens just between has() and require(). This would be odd since core-cache should usually not be cleared in production at all, and there is very very little cpu time between has() and require(). * If a (core) cache->set() wrote broken data, for instance due to 'disk full'. Silently ignoring this with the patch would then start hammering the site with write-tries. Unsure on how to act on this, at first thought. -
Patrick Schriner commented on patch set 2 04/24/24, 11:56
Reply
I stumbled upon it because in a clustered, slow environment I was writing a variant of the SimpleFileBackend that catches a possible failed require; in which case it returns false. This place is the only one where this is not captured. But tbh in any high traffic-env you could have a race condition between the ->has and the ->require call.
-
Oliver Klee updated the commit message in patch set 3
V +104/24/24, 12:09 -
core-ci voted on patch set 3 V +1 04/24/24, 12:14 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56660
-
Andreas Kienast voted on patch set 3 CR +1 04/24/24, 12:27
-
Willi Wehmeier commented on patch set 3 04/25/24, 06:38
Reply
@lolli@schwarzbu.ch In general I share your point, that core-cache should not be cleared in production. But what about the site configuration? The configuration is editable in the backend, needs a cache flush to become operative and the cache is bound to cache.core without the ability to switch to another cache backend. -
Andreas Kienast voted on patch set 3 V +1 04/25/24, 07:29
-
Benjamin Franzke commented on patch set 3 CR -1 04/25/24, 07:30
on it -
Benjamin Franzke uploaded patch set 4
CR +1V +104/25/24, 08:35 -
Benjamin Franzke commented on patch set 4 04/25/24, 08:35
Reply
A cache is volatile and can always be invalidated, therefore it's correct to always assume it may have been removed, even if we checked `has` before. Also has() can suffer from file system caches (think of nfs setup that are used by some installations) I think full disk is full disk and cache recreation doesn't harm with that, it's better to create a proper system in this case, as it's more likely to produce a usable error then. I think an invalid cache must be ignored. I thought about an exception, but came to the conclusion that we should ignore it. Logging would be fine, but then again, that's worse in case of full-disk scenario. I've also adapted the code to fix it in all placed where caches are used and not checked after retrieval. -
core-ci voted on patch set 4 V +1 04/25/24, 08:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56691
-
Oliver Klee commented on patch set 4 04/25/24, 08:48
MiddlewareStackResolver.php:84If we have this line, we can drop the `=== false` check above, I think. We might need to adapt the comment: ``` Either the cache entry has been removed in the meantime, or the result is invalid. ```PackageStatesPackageCache.php:47The first condition is redundant if we have the second condition.ImportMap.php:161Ditto.SetRegistry.php:93Ditto. -
Christian Kuhn commented on patch set 4 CR -1 04/25/24, 10:58
Reply
Yeah. I wanted to raise and discuss the question, I was aware a solution could be relatively simple. I hope it's ok to still -1 on this for now, to give me a chance for a good review.temp -1. will review. -
Georg Ringer commented on patch set 4 04/25/24, 11:57
can't we just use get and require than to avoid duplicate calls? -
Benjamin Franzke commented on patch set 4 04/25/24, 12:17
Reply
No, that would be a fail-first approach instead of handling a definitive failure upfront. If we know they action will fail, we shouldn't try it to save one call. Less code doesn't mean less work todo, the explicit `has()` prevents an unneeded `require()` invocation.MiddlewareStackResolver.php:84ReplyI'm aware that `!is_array()` would also handle `false`, but it is important to define business logic here and to distinct between two cases: * no result * invalid result Despite the reaction to both cases is (currently) equal, it is important for us as programmers to react to both cases that are known, and not apply our our fastpaths that may fail badly once someone says: "well, we also write an array, so it's fine to remove the `is_array()` check". This is the reason why I decided to (hard)code both cases explicitly.PackageStatesPackageCache.php:47ReplySee above.ImportMap.php:161ReplySee above.SetRegistry.php:93ReplySee above. -
Oliver Klee commented on patch set 4 04/25/24, 18:00
MiddlewareStackResolver.php:84ReplyAck. -
Chris Müller commented on patch set 4 05/03/24, 15:50
MiddlewareStackResolver.php:91There is a typo in the method name, the "e" is missing in "Middlware" -
Willi Wehmeier uploaded patch set 5
CR +1V +105/07/24, 05:51 -
Willi Wehmeier commented on patch set 5 05/07/24, 05:52
MiddlewareStackResolver.php:91Replydone -
core-ci voted on patch set 5 V +1 05/07/24, 05:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57317
-
Benjamin Franzke voted on patch set 5 CR +1 V +1 05/07/24, 07:41
-
Oliver Klee voted on patch set 5 CR +1 05/07/24, 09:38
-
Stefan Bürk voted on patch set 5 CR +1 V +1 05/10/24, 17:45
-
Nikita Hovratov voted on patch set 5 CR +1 05/10/24, 18:36
-
Benjamin Franzke voted on patch set 5 CR +2 V +2 05/13/24, 06:42
-
Benjamin Franzke commented on patch set 5 05/13/24, 07:01
Reply
Hey lolli, since there is a 12.x release tomorrow I'm going to merge this now as I'm very sure that this is the correct fix. I agree this is a rare case and is likely caused by using network attached storay or alike. Still the fix to check for proper return values is valid in any case and will properly handle caces where statcache vs real-php-evaluation return different results.. Hope that's OK for you. -
Benjamin Franzke merged patch set 6 05/13/24, 07:01