Review main Merged

81801[FEATURE] Introduce CacheDataCollector API

Sascha Nowak CR +2 V +2 Expert +975 −85 updated 09/30/24, 08:43

Commit message

A new API has been introduced to collect cache tags and their
lifetime during frontend rendering. This API is used in the
core to collect cache tags from page cache and content object
cache.

The API is implemented as a new PSR-7 request attribute
`frontend.cache.collector` to remove the dependency from TSFE.

Every cache tag has a lifetime. The minimum lifetime is
calculated from all given cache tags. API users don't have
to deal with it individually. The default lifetime for a
cache tag is 86400 seconds (24 hours).

The current TSFE api is deprecated in favor of the new API
as the current API implementation does not allow to set a
lifetime and extension authors have to workaround it. The
TSFE api will be removed with the next major version.

The frontend employs the following strategy: A relatively
early middleware adds an empty new cacheDataCollector
instance as attribute to request. Extensions rendering
code based on database records can then add cache tags
to this attribute. The FE rendering adds the final cache
content, the middleware then compiles the final cache
entries and triggers their persistence. There is an
additional event `AddCacheTagEvent` the middleware listens
to. It allows adding cache tags at places where the request
is not available. This is a tribute to technical debt that
can not be avoided at the moment. That event is however
marked `@internal` and may vanish later.

Review

  • Code review CR +2 Done
    • Benjamin Franzke +2
    • Guido Schmechel +1
    • Torben Hansen +1
    • Stefan Bürk +1
    • Christian Kuhn +1
    • Sascha Nowak +1
    • Benni Mack +1
  • Verified V +2 Done
    • Benjamin Franzke +2
    • Guido Schmechel +1
    • Torben Hansen +1
    • Stefan Bürk +1
    • Christian Kuhn +1
    • Sascha Nowak +1
    • core-ci +1 CI

Issues 1

Files 21

Show 1 more file

Review log 77114

Show 5689 earlier messages
  1. Sascha Nowak uploaded patch set 1 11/20/23, 14:58

  2. core-ci voted on patch set 1 V -1 11/20/23, 15:09 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49137

  3. Sascha Nowak uploaded patch set 2 V -1 11/20/23, 16:43

  4. core-ci voted on patch set 2 V -1 11/20/23, 16:53 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49161

  5. Sascha Nowak uploaded patch set 3 V -1 11/20/23, 17:13

  6. core-ci voted on patch set 3 V -1 11/20/23, 17:22 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49167

  7. Sascha Nowak marked it work in progress 11/20/23, 17:30

  8. Sascha Nowak uploaded patch set 4 V -1 11/20/23, 17:40

  9. core-ci voted on patch set 4 V -1 11/20/23, 17:46 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49175

  10. Sascha Nowak uploaded patch set 5 V -1 11/20/23, 18:28

  11. core-ci voted on patch set 5 V +1 11/20/23, 18:34 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49181

  12. Sascha Nowak uploaded patch set 6 V +1 11/21/23, 09:34

  13. Sascha Nowak marked it ready for review 11/21/23, 09:36

  14. core-ci voted on patch set 6 V +1 11/21/23, 09:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49205

  15. Oliver Klee updated the commit message in patch set 7 V +1 11/21/23, 10:09

  16. core-ci voted on patch set 7 V +1 11/21/23, 10:14 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49208

  17. Benjamin Franzke commented on patch set 7 11/21/23, 10:26

    CacheTagCollectorInterface.php:28

    Maybe use `resolveLifetime()` to denote that this is a calculated getter?

    CacheTagsAttribute.php:28

    let's drop this comment, otherwise we'd move it into EXT:frontend, but EXT:core is fine IMO

    Feature-102422-CacheTagCollectorApi.rst:14

    their
  18. Sascha Nowak uploaded patch set 8 V +1 11/21/23, 11:39

  19. Sascha Nowak commented on patch set 8 11/21/23, 11:39

    CacheTagCollectorInterface.php:28 Reply

    Done

    CacheTagsAttribute.php:28 Reply

    Done

    Feature-102422-CacheTagCollectorApi.rst:14 Reply

    Done
  20. core-ci voted on patch set 8 V -1 11/21/23, 11:44 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49214

  21. Sascha Nowak uploaded patch set 9 V -1 11/21/23, 14:48

  22. core-ci voted on patch set 9 V +1 11/21/23, 14:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49241

  23. Sascha Nowak rebased onto patch set 10 V +1 11/21/23, 18:22

  24. core-ci voted on patch set 10 V +1 11/21/23, 18:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49280

  25. Benjamin Franzke commented on patch set 10 11/23/23, 15:04

    CacheTagsAttribute.php:40

    maybe delay writing to the cache record from TSFE to this middleware, so that cacheTags definition and final usage are in a central place. Allows other middlewares to process/prevent caching etc..
  26. Andreas Wolf commented on patch set 10 11/23/23, 18:49

    Only wording issues apart from one thing about a mock in the tests.

    Feature-102422-CacheTagCollectorApi.rst:19

    … dependency to TSFE. … makes using …

    Feature-102422-CacheTagCollectorApi.rst:28

    … work around it. … TSFE API …

    Feature-102422-CacheTagCollectorApi.rst:45

    … Add multiple … different lifetimes

    Feature-102422-CacheTagCollectorApi.rst:56

    … Add multiple … different lifetimes

    Feature-102422-CacheTagCollectorApi.rst:88

    … lifetime, calculated from …

    CacheTagsAttributeTest.php:38

    I'm not sure if mocking the request does make sense here, since we have no security that it really works with an actual request. This is especially true since the request is not mocked in ContentObjectRendererTest.

    TypoScriptFrontendController.php:1439

    … add them to …
  27. Sascha Nowak rebased onto patch set 11 V +1 11/24/23, 08:33

  28. Sascha Nowak rebased onto patch set 12 11/24/23, 09:19

  29. Sascha Nowak marked it work in progress 11/24/23, 09:21

  30. Sascha Nowak uploaded patch set 13 11/24/23, 09:25

  31. core-ci voted on patch set 11 V -1 11/24/23, 09:40 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49841

  32. core-ci voted on patch set 12 V +1 11/24/23, 09:44 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49851

  33. Sascha Nowak uploaded patch set 14 11/24/23, 09:51

  34. Sascha Nowak marked it ready for review 11/24/23, 09:51

  35. core-ci voted on patch set 13 V -1 11/24/23, 10:19 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49855

  36. Christian Kuhn commented on patch set 14 12/04/23, 12:51

    minor code remarks. structurally, we should chat on this since i'm working in a related area currently and we should align on how we solve things in FE.

    CacheTagTest.php:26

    /** @test */ please, see other comment below.

    CacheTagsAttributeTest.php:30

    please avoid the attribute at least for now. doing so will confuse the test-splitter for functional scripts and i think it would be good to not erode this into the codebase for now.

    Repository.php:39

    not optional, remove ? here and in handling below.
  37. Sascha Nowak uploaded patch set 15 08/01/24, 20:27

  38. Sascha Nowak uploaded patch set 16 08/01/24, 20:32

  39. Sascha Nowak commented on patch set 16 08/01/24, 20:32

    CacheTagTest.php:26 Reply

    Done

    CacheTagsAttributeTest.php:30 Reply

    Done

    Repository.php:39 Reply

    Done

    TypoScriptFrontendController.php:1439 Reply

    Done
  40. Sascha Nowak uploaded patch set 17 08/01/24, 20:40

  41. Sascha Nowak commented on patch set 17 08/01/24, 20:40

    Feature-102422-CacheTagCollectorApi.rst:19 Reply

    Done

    Feature-102422-CacheTagCollectorApi.rst:28 Reply

    Done

    Feature-102422-CacheTagCollectorApi.rst:45 Reply

    Done

    Feature-102422-CacheTagCollectorApi.rst:56 Reply

    Done

    Feature-102422-CacheTagCollectorApi.rst:88 Reply

    Done
  42. Sascha Nowak commented on patch set 17 08/01/24, 20:44

    Reply

    Done

    CacheTagsAttributeTest.php:38 Reply

    Done
  43. Sascha Nowak commented on patch set 17 08/01/24, 20:45

    CacheTagsAttribute.php:40 Reply

    Done
  44. core-ci voted on patch set 15 V -1 08/01/24, 20:47 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62646

  45. core-ci voted on patch set 16 V -1 08/01/24, 21:00 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62649

  46. core-ci voted on patch set 17 V -1 08/01/24, 21:12 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62653

  47. Sascha Nowak rebased onto patch set 18 V -1 08/04/24, 16:23

  48. core-ci voted on patch set 18 V -1 08/04/24, 16:35 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62833

  49. Sascha Nowak uploaded patch set 19 V -1 08/04/24, 17:16

  50. core-ci voted on patch set 19 V -1 08/04/24, 17:25 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62834

  51. Sascha Nowak uploaded patch set 20 V -1 08/04/24, 17:38

  52. core-ci voted on patch set 20 V -1 08/04/24, 17:47 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62835

  53. Sascha Nowak uploaded patch set 21 V -1 08/04/24, 18:04

  54. core-ci voted on patch set 21 V -1 08/04/24, 18:13 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62836

  55. Sascha Nowak uploaded patch set 22 V -1 08/04/24, 18:21

  56. core-ci voted on patch set 22 V -1 08/04/24, 18:27 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62837

  57. Sascha Nowak uploaded patch set 23 V -1 08/04/24, 18:38

  58. core-ci voted on patch set 23 V -1 08/04/24, 18:45 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62838

  59. Benjamin Franzke commented on patch set 23 08/23/24, 06:57

    CacheDataCollectorAttribute.php:46

    revisiting this one year later I noticed: let's be explicit here and name it `cacheDataCollector`, `cacheData` is not really self-explanatory
  60. Sascha Nowak uploaded patch set 24 V -1 08/26/24, 05:59

  61. core-ci voted on patch set 24 V -1 08/26/24, 06:06 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64175

  62. Torben Hansen commented on patch set 24 08/26/24, 06:26

    cacheDataCollector

    Deprecation-102422-CacheDataCollectorApi.rst:1

    Path for this file must be `typo3/sysext/core/Documentation/Changelog/13.3/`

    Feature-102422-CacheDataCollectorApi.rst:1

    Path for this file must be `typo3/sysext/core/Documentation/Changelog/13.3/`
  63. Sascha Nowak uploaded patch set 25 V -1 08/26/24, 06:43

  64. Sascha Nowak commented on patch set 24 08/26/24, 06:43

    Reply

    Done

    CacheDataCollectorAttribute.php:46 Reply

    Done

    Deprecation-102422-CacheDataCollectorApi.rst:1 Reply

    Done

    Feature-102422-CacheDataCollectorApi.rst:1 Reply

    Done
  65. core-ci voted on patch set 25 V +1 08/26/24, 06:50 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64178

  66. Christian Kuhn commented on patch set 25 08/26/24, 12:51

    note i'm involved here as well and will review within the next days.
  67. Benjamin Franzke commented on patch set 25 08/27/24, 06:21

    AddCacheTagEvent.php:22

    In a call with lolli last week I figured we should stress that * this event is a workaround for code that has no access to the request(attribute) cacheDataCollector. * It's not an API that shall be used if the request is available * It's not an API that may be relied on to hook into/intercept cache-data generation => that would be sth. different, to be provided by the collector itself => It's really just there to allow passive cache-data signaling, without exactly knowing the current context (which we added for repositories that have no request knowledge).

    Deprecation-102422-CacheDataCollectorApi.rst:6

    The title of this RST should be sth like "TypoScriptFrontendController->addCacheTags" (otherwise it reads as if we introduce and deprecate the collector in one go)
  68. Sascha Nowak uploaded patch set 26 V +1 08/27/24, 12:11

  69. Sascha Nowak commented on patch set 26 08/27/24, 12:13

    AddCacheTagEvent.php:22 Reply

    Done

    Deprecation-102422-CacheDataCollectorApi.rst:6 Reply

    Done
  70. core-ci voted on patch set 26 V -1 08/27/24, 12:24 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64301

  71. Sascha Nowak rebased onto patch set 27 V -1 08/27/24, 17:56

  72. core-ci voted on patch set 27 V +1 08/27/24, 18:01 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64345

  73. Benjamin Franzke uploaded patch set 28 V +1 08/28/24, 09:33

  74. Benjamin Franzke uploaded patch set 29 08/28/24, 09:33

  75. core-ci voted on patch set 29 V -1 08/28/24, 09:43 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64400

  76. core-ci voted on patch set 29 V +1 08/28/24, 09:50 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64400

  77. Christian Kuhn uploaded patch set 30 V +1 09/13/24, 12:50

  78. core-ci voted on patch set 30 V +1 09/13/24, 12:57 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65676

  79. Christian Kuhn uploaded patch set 31 V +1 09/13/24, 14:25

  80. core-ci voted on patch set 31 V +1 09/13/24, 14:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65689

  81. Christian Kuhn uploaded patch set 32 V +1 09/13/24, 14:49

  82. Christian Kuhn uploaded patch set 33 09/13/24, 14:50

  83. Chris Müller commented on patch set 33 09/13/24, 14:56

    Feature-102422-CacheDataCollectorApi.rst:18

    I assume, this should be a "PSR-7 request attribute" instead a "PSR-15 request attribute" (PSR-15 is request handler aka middleware)?

    Feature-102422-CacheDataCollectorApi.rst:53

    This example is the same as above.

    Feature-102422-CacheDataCollectorApi.rst:99

    It is designed to allow ...

    Feature-102422-CacheDataCollectorApi.rst:105

    Please add an empty line between caption and code.
  84. core-ci voted on patch set 32 V -1 09/13/24, 15:01 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65691

  85. Christian Kuhn uploaded patch set 34 09/13/24, 15:02

  86. core-ci voted on patch set 33 V -1 09/13/24, 15:02 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65692

  87. Christian Kuhn commented on patch set 34 09/13/24, 15:05

    Feature-102422-CacheDataCollectorApi.rst:18 Reply

    Done

    Feature-102422-CacheDataCollectorApi.rst:53 Reply

    Done

    Feature-102422-CacheDataCollectorApi.rst:99 Reply

    Done

    Feature-102422-CacheDataCollectorApi.rst:105 Reply

    Done
  88. Christian Kuhn uploaded patch set 35 09/13/24, 15:05

  89. core-ci voted on patch set 34 V +1 09/13/24, 15:08 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65693

  1. core-ci voted on patch set 35 V +1 09/13/24, 15:12 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65694

  2. Christian Kuhn voted on patch set 35 CR +1 V +1 09/13/24, 15:13

  3. Sascha Nowak voted on patch set 35 CR +1 V +1 09/13/24, 15:15

  4. Benjamin Franzke voted on patch set 35 CR +1 V +1 09/13/24, 15:55

  5. Torben Hansen voted on patch set 35 CR +1 V +1 09/13/24, 17:53

  6. Christian Kuhn uploaded patch set 36 CR +1 V +1 09/13/24, 18:56

  7. Benni Mack voted on patch set 36 CR +1 09/13/24, 18:58

  8. Christian Kuhn updated the commit message in patch set 37 CR +1 09/13/24, 18:58

  9. Benni Mack voted on patch set 37 CR +1 09/13/24, 19:00

  10. core-ci voted on patch set 36 V -1 09/13/24, 19:04 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65704

  11. core-ci voted on patch set 37 V -1 09/13/24, 19:05 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65705

  12. Christian Kuhn uploaded patch set 38 CR +1 V -1 09/13/24, 19:07

  13. core-ci voted on patch set 38 V +1 09/13/24, 19:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65706

  14. Christian Kuhn voted on patch set 38 CR +1 V +1 09/13/24, 19:18

  15. Torben Hansen voted on patch set 38 CR +1 V +1 09/13/24, 19:50

  16. Stefan Bürk voted on patch set 38 CR +1 09/13/24, 20:16

  17. Benni Mack voted on patch set 38 CR +1 09/13/24, 20:18

  18. Stefan Bürk voted on patch set 38 V +1 09/13/24, 20:49

  19. Benjamin Franzke voted on patch set 38 CR +1 V +1 09/14/24, 03:17

  20. Guido Schmechel commented on patch set 38 CR +1 V +1 09/14/24, 05:49

    Love it!
  21. Sascha Nowak voted on patch set 38 CR +1 V +1 09/14/24, 06:12

  22. Benjamin Franzke voted on patch set 38 CR +2 V +2 09/14/24, 07:16

  23. Benjamin Franzke merged patch set 39 09/14/24, 07:17

  24. Andreas Kienast commented on patch set 39 09/17/24, 12:19

    CacheTimeout.php:41

    Regression: `$cacheDataCollector->resolveLifetime()` may return PHP_INT_MAX. Adding EXEC_TIME leads to an overflow and PHP will return a float value: `9.2233720385814E+18`. Forcing an int cast however leads to `-9223372035128201216`.
  25. Christian Kuhn commented on patch set 39 09/30/24, 08:43