Review main Merged

85139[BUGFIX] Respect condition state for PageTSConfig runtime cache

Friedemann Altrock CR +2 V +2 Normal +115 −36 updated 04/10/26, 18:48

Commit message

The method `BackendUtility::getPagesTSconfig()` has been using
a two level cache since #100701 to reduce the memory footprint.

Technically, a cache entry `pageTsConfig-pid-to-hash-<uid>` is
written pointing to a concrete PageTsConfig instance based on
relevant cache data. The goal was to reuse a `PageTsConfig`
instance when the config does not differ because it is inherited
from the rootline.

However, this approach missed the fact that even with the same
basic content, conditions can be evaluated differently for
distinct pages which can result in a different PageTsConfigArray.

This change adds a hash of the matched conditions to the second
cache level. As a consequence, the TS configuration has to be
evaluated at least once for each distinct page UID.

In other words, this change trades off runtime performance for
correctness.

It may be possible to investigate and find a broader approach to
improve performance by doing the condition evaluation independent
of the TS configuration parsing, but this is out of scope and needs
to be done in dedicated change.

Review

  • Code review CR +2 Done
    • Stefan Bürk +2
    • Christian Kuhn +1
  • Verified V +2 Done
    • Stefan Bürk +2
    • Christian Kuhn +1
    • core-ci +1 CI

Issues 2

Files 6

Review log 3546

Show 1621 earlier messages
  1. Friedemann Altrock uploaded patch set 1 07/06/24, 13:27

  2. core-ci voted on patch set 1 V +1 07/06/24, 13:32 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61098

  3. Stefan Bürk commented on patch set 1 CR -1 07/06/24, 19:29

    -1 see other comment

    BackendUtility.php:782

    This literally renders the cache completly write only - because the cache gets does not use the getConditionHash() (and cannot) - at least from reading, did not checked that. This needs correct analysis and debugging, thus -1 for now.
  4. Friedemann Altrock commented on patch set 1 07/07/24, 06:09

    l

    BackendUtility.php:782 Reply

    Yes! And no. The point of this fix is precisely that the cache cannot be automatically the same for all pages just because the TSconfig from the rootLine is the same. Yes, when called with a new page UID that the cache has not seen yet, if (and only if) the rootLine config AND the conditions as evaluated for this specific page UID hash to the same value as a previously cached page, we are writing the same TS config to the cache again. No, this is not write-only! This still caches pageTSconfig objects of all page UIDs this function is called with (the pid-to-hash level). We do save memory, but we may have to repeat a bunch of computation. As soon as we allow conditional expressions, every page TSconfig in the system could be different, so we have to evaluate the configuration for each page UID at least once!
  5. Friedemann Altrock uploaded patch set 2 CR -1 V +1 07/07/24, 10:55

  6. core-ci voted on patch set 2 V +1 07/07/24, 11:00 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61136

  7. Friedemann Altrock commented on patch set 2 07/07/24, 11:06

    BackendUtility.php:782 Reply

    Right, I see how you came to the conclusion that it was write-only. I removed the cache-get with $cacheRelevantDataHash as it indeed never succeeds.
  8. Friedemann Altrock uploaded patch set 3 V +1 08/02/24, 09:53

  9. core-ci voted on patch set 3 V +1 08/02/24, 09:58 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62712

  10. Benni Mack updated the commit message in patch set 4 V +1 10/18/24, 13:41

  11. Stefan Bürk uploaded patch set 5 03/27/25, 13:05

  12. Stefan Bürk uploaded patch set 6 03/27/25, 13:07

  13. Stefan Bürk commented on patch set 6 03/27/25, 13:10

    Minor modifications: * Rebased change on current main branch. * Modified the conditional hash calculation to use json_encode() ahd `xxh3` algo instead of md5()/serialize(). * Modified the cache key generation within the BU::getPagesTSconfig() method and moved it to one place having it one place. * Changed the explanation / commit message.
  14. core-ci voted on patch set 5 V +1 03/27/25, 13:14 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76454

  15. core-ci voted on patch set 6 V +1 03/27/25, 13:15 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76455

  16. Benni Mack commented on patch set 6 CR -2 03/27/25, 13:19

    we'll wait for Feedback from @lolli@schwarzbu.ch
  17. Stefan Bürk commented on patch set 6 CR +1 V +1 03/27/25, 13:19

    @lolli@schwarzbu.ch should review this because of the partial revert and addition in the PageTsConfig object. From my side okay.
  18. Friedemann Altrock commented on patch set 6 V +1 03/27/25, 13:40

    Sorry, this is wrong. The cache is still present (not removed), is still 2-level, and still makes sense. What is true: for all _distinct_ page UIDs, we need to run the TS compilation at least once so that the conditions are evaluated correctly. The function `getPagesTSconfig` has only 1 argument: the page UID. That is the reason for the two levels of the cache! The first level of the cache answers the question: have we calculated the TS config for this page before? The second level then maps the result of that calculation, depending on the matched TS conditions! So, this change trades a runtime performance impact (potentially duplicated work) for correctness.
  19. Friedemann Altrock updated the commit message in patch set 7 CR +1 V +1 03/27/25, 14:24

    Copied Votes: * Code-Review-2 (copy condition: "changekind:NO_CHANGE OR **is:MIN**")
  20. Friedemann Altrock commented on patch set 7 03/27/25, 14:26

    Changed the commit message to explain how the cache use changed.

    Reply

    Done
  21. Friedemann Altrock commented on patch set 7 03/27/25, 14:26

    Reply

    Done
  1. core-ci voted on patch set 7 V +1 03/27/25, 14:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76471

  2. Stefan Bürk commented on patch set 7 03/27/25, 16:07

    Reply

    > Sorry, this is wrong. Nope, sorry to correct you here, but the second level cache is gone now. It's not about blaming, but my statement was correct. Don't be confused that two keys are used (one pointing to another. To be correcet, we could simply use one key now. For a new pid the PageTsConfig object is build and evualuated, it still may be generating the same hash for multiple sites and set it for the first level. The next page will reubild it again, not using a "second level key" to check if it really needs to rebuild the PageTsConfig object - which is not possible. We could use only a `pid-<pid>`hash removing the dedicated hash cache key and have 100% the same working state as with the current two keys and thus only a one level cache (currently composed with two keys as lookup reference) - but still, only cache fetch on the first level, not the second. Going to delay the key merging into one key until loli had time to review the changed on the PageTsConfig object. Changed commit message still fine however, so keeping it also for now.
  3. Friedemann Altrock uploaded patch set 8 V +1 03/27/25, 16:22

    Copied Votes: * Code-Review-2 (copy condition: "changekind:NO_CHANGE OR **is:MIN**")
  4. Friedemann Altrock commented on patch set 8 03/27/25, 16:26

    Reply

    Right, I added 4 lines of code to the method to make it so the 2nd level of the cache is actually used explicitly, not only implicitly through the pageTsConfigHash == PageTsConfig value relationship. Object identities might be important after all.
  5. Friedemann Altrock uploaded patch set 9 03/27/25, 16:30

    Copied Votes: * Code-Review-2 (copy condition: "changekind:NO_CHANGE OR **is:MIN**")
  6. core-ci voted on patch set 8 V -1 03/27/25, 16:32 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76482

  7. core-ci voted on patch set 9 V +1 03/27/25, 16:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76483

  8. Stefan Bürk commented on patch set 9 03/27/25, 17:30

    Reply

    Totally unneeded and brain melting now, makes no sense. Going to push a last time and than wait until loli has time to look onto it.
  9. Stefan Bürk commented on patch set 9 03/27/25, 17:33

    Reply

    Let's keep it in the current state. Will discuss this with lolly. I'd say we could drop the condition hash again from the PageTsConig as it is not needed touse it. We simply can use a "page id" page based cache entry. For each page the PageTsConfig has to be created anyway (and is done currently). Ending up in simply reverting the second cache level. Will coop on this with lolli.
  10. Benni Mack wrote 03/27/25, 17:38

    Hashtag added: lolli-feedback
  11. Christian Kuhn commented on patch set 9 04/07/25, 13:41

    PageTsConfigFactory.php:143

    hmm. maybe we should just add getConditionListWithVerdicts without obfuscating it more? this shouldn't increase size of the object too much (given a limited list of conditions), and we could suppress hashing the hash again in the BU consumer if doing so. what do you think?
  12. Christian Kuhn uploaded patch set 10 V +1 04/07/25, 13:59

    Copied Votes: * Code-Review-2 (copy condition: "changekind:NO_CHANGE OR **is:MIN**")
  13. core-ci voted on patch set 10 V +1 04/07/25, 14:06 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77142

  14. Christian Kuhn uploaded patch set 11 V +1 04/07/25, 14:10

    Copied Votes: * Code-Review-2 (copy condition: "changekind:NO_CHANGE OR **is:MIN**")
  15. Christian Kuhn commented on patch set 11 04/07/25, 14:10

    Reply

    Done

    PageTsConfigFactory.php:143 Reply

    Done
  16. Christian Kuhn uploaded patch set 12 04/07/25, 14:13

    Copied Votes: * Code-Review-2 (copy condition: "changekind:NO_CHANGE OR **is:MIN**")
  17. Christian Kuhn commented on patch set 12 04/07/25, 14:15

    i don't like it *that* too much to expose the conditions with verdicts in the pageTsConfig object, but the object is internal and there is no better solution for now. mid-term, BU::getPagesTSconfig() should somehow vanish at some point, but that has much more impact, so we'll go with this bug fix for now.
  18. Stefan Bürk voted on patch set 12 CR +1 V +1 04/07/25, 14:17

  19. core-ci voted on patch set 12 V +1 04/07/25, 14:20 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77147

  20. core-ci voted on patch set 11 V -1 04/07/25, 14:20 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77143

  21. Christian Kuhn voted on patch set 12 CR +1 V +1 04/07/25, 14:21

  22. Benni Mack voted on patch set 12 CR 0 04/07/25, 14:46

  23. Stefan Bürk voted on patch set 12 CR +2 V +2 04/07/25, 23:00

  24. Stefan Bürk merged patch set 13 04/07/25, 23:11

  25. Christian Kuhn wrote 04/10/26, 18:48

    Hashtag removed: lolli-feedback