Review main Merged

84016[BUGFIX] Circumvent possible race condition in cache retrieval

Patrick Schriner CR +2 V +2 Normal +46 −8 updated 05/13/24, 07:01

Commit message

To circumvent race conditions where the cache is removed between
`has()` and `require()` invocations, the return value of the
cache result needs to be verified, as `false` is returned if
the cache entry is not available.

Review

  • Code review CR +2 Done
    • Benjamin Franzke +2
    • Oliver Klee +1
    • Stefan Bürk +1
    • Nikita Hovratov +1
  • Verified V +2 Done
    • Benjamin Franzke +2
    • Stefan Bürk +1
    • core-ci +1 CI

Issues 1

Files 4

Review log 2732

Show 57 earlier messages
  1. Patrick Schriner uploaded patch set 1 04/24/24, 11:40

  2. core-ci voted on patch set 1 V +1 04/24/24, 11:45 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56656

  3. Patrick Schriner updated the commit message in patch set 2 V +1 04/24/24, 11:46

  4. Andreas Kienast commented on patch set 1 04/24/24, 11:46

    Do you stumble across an actual bug? To me, the scenario where a TypeError is issued is highly hypothetical and doesn't happen in any way, ever.
  5. core-ci voted on patch set 2 V +1 04/24/24, 11:53 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56657

  6. Christian Kuhn commented on patch set 2 04/24/24, 11:56

    Reply

    Same question: This sounds like an edge case: * may happen when a core cache-clear happens just between has() and require(). This would be odd since core-cache should usually not be cleared in production at all, and there is very very little cpu time between has() and require(). * If a (core) cache->set() wrote broken data, for instance due to 'disk full'. Silently ignoring this with the patch would then start hammering the site with write-tries. Unsure on how to act on this, at first thought.
  7. Patrick Schriner commented on patch set 2 04/24/24, 11:56

    Reply

    I stumbled upon it because in a clustered, slow environment I was writing a variant of the SimpleFileBackend that catches a possible failed require; in which case it returns false. This place is the only one where this is not captured. But tbh in any high traffic-env you could have a race condition between the ->has and the ->require call.
  1. Oliver Klee updated the commit message in patch set 3 V +1 04/24/24, 12:09

  2. core-ci voted on patch set 3 V +1 04/24/24, 12:14 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56660

  3. Andreas Kienast voted on patch set 3 CR +1 04/24/24, 12:27

  4. Willi Wehmeier commented on patch set 3 04/25/24, 06:38

    Reply

    @lolli@schwarzbu.ch In general I share your point, that core-cache should not be cleared in production. But what about the site configuration? The configuration is editable in the backend, needs a cache flush to become operative and the cache is bound to cache.core without the ability to switch to another cache backend.
  5. Andreas Kienast voted on patch set 3 V +1 04/25/24, 07:29

  6. Benjamin Franzke commented on patch set 3 CR -1 04/25/24, 07:30

    on it
  7. Benjamin Franzke uploaded patch set 4 CR +1 V +1 04/25/24, 08:35

  8. Benjamin Franzke commented on patch set 4 04/25/24, 08:35

    Reply

    A cache is volatile and can always be invalidated, therefore it's correct to always assume it may have been removed, even if we checked `has` before. Also has() can suffer from file system caches (think of nfs setup that are used by some installations) I think full disk is full disk and cache recreation doesn't harm with that, it's better to create a proper system in this case, as it's more likely to produce a usable error then. I think an invalid cache must be ignored. I thought about an exception, but came to the conclusion that we should ignore it. Logging would be fine, but then again, that's worse in case of full-disk scenario. I've also adapted the code to fix it in all placed where caches are used and not checked after retrieval.
  9. core-ci voted on patch set 4 V +1 04/25/24, 08:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56691

  10. Oliver Klee commented on patch set 4 04/25/24, 08:48

    MiddlewareStackResolver.php:84

    If we have this line, we can drop the `=== false` check above, I think. We might need to adapt the comment: ``` Either the cache entry has been removed in the meantime, or the result is invalid. ```

    PackageStatesPackageCache.php:47

    The first condition is redundant if we have the second condition.

    ImportMap.php:161

    Ditto.

    SetRegistry.php:93

    Ditto.
  11. Christian Kuhn commented on patch set 4 CR -1 04/25/24, 10:58

    Reply

    Yeah. I wanted to raise and discuss the question, I was aware a solution could be relatively simple. I hope it's ok to still -1 on this for now, to give me a chance for a good review.
    temp -1. will review.
  12. Georg Ringer commented on patch set 4 04/25/24, 11:57

    can't we just use get and require than to avoid duplicate calls?
  13. Benjamin Franzke commented on patch set 4 04/25/24, 12:17

    Reply

    No, that would be a fail-first approach instead of handling a definitive failure upfront. If we know they action will fail, we shouldn't try it to save one call. Less code doesn't mean less work todo, the explicit `has()` prevents an unneeded `require()` invocation.

    MiddlewareStackResolver.php:84 Reply

    I'm aware that `!is_array()` would also handle `false`, but it is important to define business logic here and to distinct between two cases: * no result * invalid result Despite the reaction to both cases is (currently) equal, it is important for us as programmers to react to both cases that are known, and not apply our our fastpaths that may fail badly once someone says: "well, we also write an array, so it's fine to remove the `is_array()` check". This is the reason why I decided to (hard)code both cases explicitly.

    PackageStatesPackageCache.php:47 Reply

    See above.

    ImportMap.php:161 Reply

    See above.

    SetRegistry.php:93 Reply

    See above.
  14. Oliver Klee commented on patch set 4 04/25/24, 18:00

    MiddlewareStackResolver.php:84 Reply

    Ack.
  15. Chris Müller commented on patch set 4 05/03/24, 15:50

    MiddlewareStackResolver.php:91

    There is a typo in the method name, the "e" is missing in "Middlware"
  16. Willi Wehmeier uploaded patch set 5 CR +1 V +1 05/07/24, 05:51

  17. Willi Wehmeier commented on patch set 5 05/07/24, 05:52

    MiddlewareStackResolver.php:91 Reply

    done
  18. core-ci voted on patch set 5 V +1 05/07/24, 05:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57317

  19. Benjamin Franzke voted on patch set 5 CR +1 V +1 05/07/24, 07:41

  20. Oliver Klee voted on patch set 5 CR +1 05/07/24, 09:38

  21. Stefan Bürk voted on patch set 5 CR +1 V +1 05/10/24, 17:45

  22. Nikita Hovratov voted on patch set 5 CR +1 05/10/24, 18:36

  23. Benjamin Franzke voted on patch set 5 CR +2 V +2 05/13/24, 06:42

  24. Benjamin Franzke commented on patch set 5 05/13/24, 07:01

    Reply

    Hey lolli, since there is a 12.x release tomorrow I'm going to merge this now as I'm very sure that this is the correct fix. I agree this is a rare case and is likely caused by using network attached storay or alike. Still the fix to check for proper return values is valid in any case and will properly handle caces where statcache vs real-php-evaluation return different results.. Hope that's OK for you.
  25. Benjamin Franzke merged patch set 6 05/13/24, 07:01