Review main Abandoned

96351[BUGFIX] Persist Extbase class schemata when built

Stefan Bürk CR 0 V -1 Normal +180 −33 updated 10/21/26, 06:46

Commit message

ReflectionService wrote all class schemata into the extbase cache from
its destructor. When the garbage collector runs that destructor while
another serialize() call is in progress, the nested serialize()
continues the reference table of the outer call. The back references
of the cache entry are then numbered in the outer table, the entry is
stored with a valid HMAC, and the next ReflectionService fails to
unserialize it with "Error at offset N of M bytes". The outer string
is damaged as well, since the nested values are counted in its table.

Since TYPO3 v14 the class schemata hold shared TypeInfo objects, which
serialize as back references and make this far more likely. It shows
up deterministically in functional test suites and can happen in long
running processes.

Each class schema is now written to a cache entry of its own as soon
as it has been built, and read from there on first use, the way
DataMapFactory handles its data maps in the same cache. No cache write
is left in a destructor, which also drops the dependency on the global
configuration still being set at that point.

Writing the whole set again for every new schema would keep a single
entry, but serialize it once per class. Persisting at the end of the
request needs a lifecycle hook Extbase does not have for CLI and long
running processes. Serializing into a string up front does not help,
any serialize() inside a destructor still shifts the numbering of the
outer call.

Review

  • Code review CR 0
  • Verified V -1
    • core-ci -1 CI

Issues 1

Files 5

Review log 1416

  1. Stefan Bürk uploaded patch set 1 10/04/26, 18:58

  2. core-ci voted on patch set 1 V +1 10/04/26, 19:06 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/110050

  3. Stefan Bürk marked it ready for review 10/04/26, 20:28

  4. Oliver Klee voted on patch set 1 V +1 10/05/26, 09:32

  5. Alexander Schnitzler voted on patch set 1 CR +1 V +1 10/17/26, 06:23

  6. Oliver Klee voted on patch set 1 CR +1 10/17/26, 17:07

  7. Garvin Hicking commented on patch set 1 CR -1 10/19/26, 18:00

    Unresolved

    I would like to talk about two things I found in review and specifically the 13.4 backport (and probably 14.3): 1: Removing the protected property and public method is basically a breaking change, because this class is neither final nor marked `internal` and consumers might have subclassed/xclassd it (though not advisable). So I'm not sure enough this is worth a breaking change, and would like your feedback on it. In a backport at least we could keep the property and destructor and just let it be empty/unused? 2: Currently, closures are possible as closures for attributes (in PHP 8.5), like: ``` class ModelWithClosureDefaultValue extends AbstractEntity { protected string $foo = ''; public function getFoo(\Closure $formatter = static function (string $value): string { return $value; }): string { return $formatter($this->foo); } } ``` If this is utilized and cached, this fails mid-request now with an exception. Previously, this was "silently" swallowed. We need to decide: a.) Closures are not supported to be cached, and would mean application state failure. Then we need to throw a more meaningful error here, and the patch is fine. b.) We want to carry closures forward and just don't cache/serialize them fully. In that case we'd need to swallow the exception (or log it?) Bonus: I did a small performance benchmark. With limited usage of class schematas, this patch greatly improves performance. Before it ran only once, and incurring a 7.7ms "penality". Now it's 25µs per class. So the break-even until this new variant gets slower is at around ~300 classes. I'd argue this is a neglectible impact and it's fine 😊 Bonus 2: 5. The current test doesn't really exercise serialisation. It uses TransientMemoryBackend, which VariableFrontend never serialises for. It returns the same object instance, which is why assertSame passes. So it can't catch any of the problems above. Maybe we'd need a functional test setup for this?!
  8. Alexander Schnitzler commented on patch set 1 V 0 10/19/26, 18:12

    Resetting my verified vote since the test actually doesn't really test. I have never really seen anyone XClass the ReflectionService but if we are strict, this is possibly breaking, at least when backported. I don't see how this is a class people should use publicly no matter if it's marked internal but I let others decide here. A real test, one that also checks the Closure, would be great here.
  9. Garvin Hicking uploaded patch set 2 CR +1 V +1 10/19/26, 18:55

    add currently failing closure test
  10. Garvin Hicking commented on patch set 2 10/19/26, 18:56

    Reply Unresolved

    Yeah I also agree that this is a very unlikely class anyone should touch. I'd be fine with a "breaking bugfix" on this, fwiw. I've added the test I came up with during review, which is red however until we decide how (if) a closure should be treated.
  11. core-ci voted on patch set 2 V -1 10/19/26, 19:11 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/111699

  12. Stefan Bürk abandoned the change 10/19/26, 20:38

  13. Alexander Schnitzler commented on patch set 2 10/20/26, 07:54

    I had a closer look at the closure question and the backports, since I'd like to pick this up again. Closures The current behaviour on main is not "silently swallowed", only the test setup hides it. With the real `ReflectionService` and `VariableFrontend` on a serializing backend (PHP 8.5), a model with a closure in an attribute or a parameter default value ends in PHP Fatal error: Uncaught Exception: Serialization of 'Closure' is not allowed ... ReflectionService->__destruct() at shutdown, after the response was sent. And since all schemata share one cache entry, a single such class means no schema is cached at all anymore. 14.3 and 13.4 behave the same on PHP 8.5. As PHP 8.5 explicitly allows closures in constant expressions (e.g. `Assert\Callback`), I'd go for option b: keep such a schema in the runtime cache only. - $this->dataCache->set($this->getClassSchemaCacheIdentifier($className), $classSchema); + try { + $this->dataCache->set($this->getClassSchemaCacheIdentifier($className), $classSchema); + } catch (\Exception) { + // A closure in an attribute argument or a parameter default value (PHP 8.5) + // cannot be serialized, such a schema is built again in every process. + } With that, the unit tests and Garvin's functional tests are green locally; the unit tests fail against main's `ReflectionService` as they should. Backports The change does not cherry-pick cleanly to 14.3/13.4 (constructor and unit test differ). I'd keep `$dataCacheNeedsUpdate` and an empty `__destruct()` there, so subclasses calling `parent::__destruct()` don't break. The closure fixtures are PHP 8.5 syntax, so on those branches `lintPhp` and `phpstan` with `-p 8.2` and the functional test fail with "Constant expression contains invalid operations". I'd backport the production change (including the `try`/`catch`, as both branches also run on PHP 8.5) plus the unit tests only, and keep the closure tests on main. On 13.4 the unit test needs `new TransientMemoryBackend('Testing')`. With that, the new unit tests fail on the unpatched branches and pass with the backport on PHP 8.2 and 8.5, and the extbase unit and functional suites (sqlite, PHP 8.5) are green on both branches. @Stefan: the fix itself is good and I think it's worth landing. Would you like to restore the change and carry on with it, or would it be fine with you if I take it over and push the next patch set?
  14. Garvin Hicking commented on patch set 2 10/20/26, 08:02

    Reply

    +1 on Alexanders assessment, I also don't think we should abandon this. I am sorry if my comments suggested this was what I was going for (I was not; I merely wanted to discuss/point out).
  15. Stefan Bürk commented on patch set 2 10/21/26, 03:12

    Reply

    I abandoned the change because the closure thing is a completly dedicated issue and **should** be handled within it's own issue and change. Try to deal here with it mixes the concern of the change and mixing stuff up. Using the try/catch simply hides the "non cachable" class schema away, and a developer does not get aware of it that something in a model prevents caching the schema of it. In my eyes, we should not hide that away by catching the exception, and loging is not "hit the developers finger" enough. In my eyes the extra stuff added to **my** change should be extracted into a dedicated change (the test) and instead looked into the ClassSchema building around the "callable" stuff and to proberly handle that - or deal with questions like * Do we need the callable method PHP8.5 syntax stuff or can we throw it away? * **OR** Can it be made serialiuze/unserialze safe ? But that's out of the scope of this change. Combined with this change at least other model schemas are "persistested", but given that soem developer may use that in most of there models in some way would mean that the persisted cache gets pretty obsolete. I leave it to you both how to contine -- reenable the change and extract the stuff into a own handling or not. Maybe "class" schema should evaluate if it can be serialized (cached) or not - but that would also rule out backends not serializing/unserializing. But again, thats schema realated and should be it's own handling branch. Note: Adding the try-catch here leads that noone is going to fix that because it is hidden away (until somewone removes the try-catch here). That can and should be not the solution for the PHP8.5 callable stuff. The scope of my change was the service and cache handling within the __destruct(), and not fixing PHP8.5 syntax suger issues within the schema building/handling. From my side -1 on the try-catch and -1 on addign the unrelated test-case for another issue to this change, thus the abandoning of it. I do not give my name for a change adding the try-catch, because I personally do not take that as a solution for the unrelated throwen in issue here. Garvin should be able to un-abandon the change, but if the the conses is to use try-catch than please go ahead, create a additional issue for it, push the change as a new change with 2 resolve lines - okay ... that does not resolve the php8.5 callback issue anyway. If try-catch than only to throw a concrete exception with a TYPO3 Exceoption Code - but not swallowing the exception.
  16. Alexander Schnitzler commented on patch set 2 10/21/26, 06:46

    Reply Unresolved

    Thanks Stefan, understood. Splitting the closure topic into its own issue makes sense to me. For the record, the behaviour with and without the `try`/`catch`, on PHP 8.5 with a serializing cache backend and a model that has a closure in an attribute argument or a parameter default value: * main today: the schema is built, the response is sent, then `ReflectionService->__destruct()` ends in "Serialization of 'Closure' is not allowed" at shutdown. Since all schemata share one cache entry, no schema is cached at all anymore. * PS1 (no `try`/`catch`): the exception is thrown during the request, when the schema is built, so the request fails. Other schemata are cached. * with `try`/`catch`: the request succeeds, the schema is only kept in the runtime cache and rebuilt in every process, without any notice. 14.3 and 13.4 behave like main on PHP 8.5, so a backport of PS1 turns the fatal at shutdown into a failing request there. Hence the `try`/`catch` idea. It's safe to backport; on main we could instantly move forward into a different direction. On your question whether it can be made serialize/unserialize safe: I don't see a way from the outside, only within `ClassSchema` itself via `__serialize()`/`__unserialize()`. Closures end up in the schema through `Validate` and `FileUpload` attribute instances (`newInstance()`) and parameter default values (`getDefaultValue()`). Simply filtering them out would make a cached schema differ from a freshly built one (e.g. a validator losing its callback). But since closures in constant expressions are static, `__serialize()` could store where they come from (class, method/property, parameter, attribute) and `__unserialize()` could restore them via reflection. That would also cover first-class callables like `strlen(...)`.