Review main Merged

89405[BUGFIX] Resolve FlexForm property groups across sheets

Oli Bartsch CR +2 V +2 Hard +327 −44 updated 10/20/26, 08:04

Commit message

Dotted FlexForm field names like "settings.storagePid" are
grouped by the Record API. If such a group is spread over
multiple sheets, accessing it without a sheet name, as Fluid
does for {flex.settings.storagePid}, threw an "ambiguous"
exception.

Accessing a property path without a sheet name now merges the
groups of all sheets recursively. Any other value existing in
more than one sheet is still ambiguous and throws an exception.

Additionally, resolved RecordPropertyClosures are now cached,
since they were instantiated again on every access.

Review

  • Code review CR +2 Done
    • Oli Bartsch +2
    • Garvin Hicking +1
    • Simon Schaufelberger +1
  • Verified V +2 Done
    • Oli Bartsch +2
    • Garvin Hicking +1
    • Simon Schaufelberger +1
    • core-ci +1 CI

Issues 1

Files 4

Review log 3852

Show 1627 earlier messages
  1. Oli Bartsch uploaded patch set 1 05/15/25, 11:22

  2. core-ci voted on patch set 1 V -1 05/15/25, 11:31 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/78965

  3. Nikita Hovratov updated the commit message in patch set 2 V -1 06/06/25, 17:09

  4. core-ci voted on patch set 2 V -1 06/06/25, 17:19 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80005

  5. Nikita Hovratov uploaded patch set 3 V -1 06/06/25, 17:59

  6. core-ci voted on patch set 3 V -1 06/06/25, 18:07 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80006

  7. Nikita Hovratov uploaded patch set 4 V -1 06/06/25, 18:35

  8. core-ci voted on patch set 4 V -1 06/06/25, 18:41 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80007

  9. Nikita Hovratov uploaded patch set 5 V -1 06/06/25, 18:47

  10. core-ci voted on patch set 5 V +1 06/06/25, 18:52 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80008

  11. Nikita Hovratov uploaded patch set 6 V +1 06/06/25, 19:30

  12. core-ci voted on patch set 6 V +1 06/06/25, 19:35 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80009

  13. Nikita Hovratov uploaded patch set 7 V +1 06/06/25, 20:32

  14. core-ci voted on patch set 7 V +1 06/06/25, 20:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80014

  15. Nikita Hovratov uploaded patch set 8 V +1 06/06/25, 20:54

  16. core-ci voted on patch set 8 V +1 06/06/25, 21:00 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80016

  17. Nikita Hovratov uploaded patch set 9 V +1 06/06/25, 21:11

  18. core-ci voted on patch set 9 V +1 06/06/25, 21:21 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80018

  19. Nikita Hovratov uploaded patch set 10 V +1 06/06/25, 21:22

  20. core-ci voted on patch set 10 V +1 06/06/25, 21:34 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80023

  21. Nikita Hovratov uploaded patch set 11 V +1 06/06/25, 22:07

  22. core-ci voted on patch set 11 V +1 06/06/25, 22:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80024

  23. Christian Kuhn commented on patch set 11 CR -1 06/10/25, 09:39

    i'm not convinced this is a great approach to 'solve' the forge issue with fluid: this essentially implements a 'sub-language' for dotted notations by inventing a 'group' and adding this to a low level part of the system. this feature may add additional issues. what if a sheet is named settings? other places solved the ambiguous 'dot' by forcing quotation as \. if the special meaning of the dot should be taken away. TS is an example here and i think flex stuff does it as well at some places already?
  24. Christian Kuhn commented on patch set 11 06/10/25, 10:38

    Reply

    > other places solved the ambiguous 'dot' by forcing quotation as \. quotation as backslash-dot ...
  25. Nikita Hovratov commented on patch set 11 06/10/25, 10:51

    Reply

    This is not about Fluid anymore. It's an alignment to the already existing behavior in Extbase, where settings.* are grouped in sub-arrays. This functionality is already there in FlexFormService->convertFlexFormContentToArray
  26. Nikita Hovratov commented on patch set 11 06/10/25, 10:55

    Reply

    So we are not inventing a new dot syntax, as it's already there. This patch "merely" solves the issue having the group identifier in two different sheets. Having a sheet named settings is no problem at all, too.
  27. Christian Kuhn commented on patch set 11 06/10/25, 11:31

    Reply

    you are adding a new flex language construct at this point by introducing 'flex form property groups' on a low level. i don't think this is where it belongs and sub applications that rely on the (bad) idea of extbase to have 'settings' "grouped" should rather implement this on their own. you're now trying to follow https://review.typo3.org/c/Packages/TYPO3.CMS/+/89096 which (recently!) took a similar decision, cementing the magic introduced with unquoted 'dotted' properties. from this point of view i don't agree on https://review.typo3.org/c/Packages/TYPO3.CMS/+/89096 either, which was a regression fix from https://forge.typo3.org/issues/105621 ? it looks to me as if a series of questionable decisions has been taken in this area without the consequences being thought through. it's a pity this only comes up now and i think it would at least be a good idea to take a step back and look at the entire picture again to see where it leads. what do you think?
  1. Christian Kuhn commented on patch set 11 06/10/25, 11:51

    RecordFieldTransformerTest.php:1824

    this shows the strategy is not a great idea: imagine there is another field called "some". what is returned? the array ob 'sub groups' or the 'group' value or an exception stating this is not allowed? or a merge of both? a default? imho: do not add magic to properties when they have dots. a property is a property name and not a magic group or sub path. this should be done by 'application' that needs it as special code (like extbase does to prepare the 'settings' array) and only for prefixes that need it for historically bad decisions. any code that implemented this on a lower level should vanish again. what happens if such a 'grouped' field is a relation where the path is within an inline child column? would this have quotes? would you which to have the same parser magic at this point? what happens with ambiguity at this point when there is a relation attached to a property called 'settings' and another one to a property called 'settings.foo'? what is returned if i want relations for 'settings'?
  2. Nikita Hovratov commented on patch set 11 06/10/25, 12:06

    RecordFieldTransformerTest.php:1824 Reply

    Agree on the problem with `some` and `some.thing`. Had the same problem in mind.
  3. Nikita Hovratov commented on patch set 11 06/10/25, 12:23

    Reply

    This API is designed to replace the whole DataProcessor overhead. If FlexForm settings aren't grouped anymore we can't have a drop-in replacement for `FlexFormProcessor` anymore. We should think about the users first. They are used to grouped FlexForm properties. I would rather find out the missing pieces (which you already pointed out) and cover them with tests. As this API is brand new, we can basically do whatever we want. Right now this is first and foremost used by Content Blocks. Normally, there is no other "application level" in between.
  4. Christian Kuhn commented on patch set 11 06/10/25, 12:47

    Reply

    an alternative could be to revert the entire thing and hand it over to content blocks as incubator since too many details have not been thought through, or turn out to be not stable enough for core usage. this could buy us time since content blocks is the only consumer right now. it looks to me (after scratching the surface today only) as if all this multiplies complexity so much that we tend to fall into one issue after the other with it. also, there is the flex datastructure simplification approach by bartsch that would allow proper compile-time evaluation of flex form, and i wonder if we shouldn't get this done before looking at questions like 'handle property names as groups' and 'having sheet names in paths' and 'having some property name in two different sheets' questions a second time.
  5. Nikita Hovratov commented on patch set 11 06/10/25, 15:28

    Reply

    Alright, let's discuss the details in our private Content Blocks channel (you should be a member).
  6. Benjamin Franzke commented on patch set 11 06/11/25, 07:16

    Reply

    I/We love public discussions, why not continue here? That way anyone can follow decision making. Whatever we decide here: We want Content Blocks to be a core principle, so we shouldn't handle it as a consumer, but think where concepts can be incorporated in our environment in a proper way. And it's actually not just a content blocks issue, I thought about similar workarounds for access to settings ¯\_(ツ)_/¯ I really understand @lolli@schwarzbu.ch argument, maybe it's not the data provider that should implement the sub-grouping, but fluid, maybe with a new/nicer/explicit quoting syntax? `{foo.[bar.baz]}` to access `bar.baz` on `foo`. I'm all in for thinking for our users, but quoting properties with backslashes just doesn't feel right…
  7. Oli Bartsch commented on patch set 11 06/11/25, 07:53

    Reply

    Oke, so my understanding is: In the past, we consistently used `convertFlexFormContentToArray`. Regardless of the Record API, this has always led to a known issue: when a property path — which may contain dots as part of the name (without implying hierarchy) — appears in multiple sheets, the value from the last occurrence would override the others. This behavior is corrected by https://review.typo3.org/c/Packages/TYPO3.CMS/+/89096, which now treats sheets as part of the data hierarchy. From my perspective, this is the correct approach and fully independent of both Fluid notation and the Record API. Otherwise, sheets would only serve a visual purpose — just for grouping fields in FormEngine — and we'd have to throw an exception for duplicate property paths. As mentioned, the bug where the last property path 'wins' has been around for quite some time and is now finally addressed. I’ve implemented necessary fallbacks, both technically and logically, so that when used correctly, there should be no noticeable change in behavior. The only conceptual shift would be to split property paths by dots and treat them as nested arrays. I also think that this introduces a new and questionable behavior. My initial thought when encountering the issue was: why can’t Fluid support this natively? However, Simon confirmed that sth like `{data.pi_flexform.settings\.something}` does not work.
  8. Simon Praetorius commented on patch set 11 06/11/25, 08:59

    Reply

    Yes, currently it doesn't work, see this rather old Fluid issue: https://github.com/TYPO3/Fluid/issues/391 We might look into it though, I also think that this is a missing feature in Fluid. The implementation might get ugly because of the rather complex regular expressions, but it might be doable. So if the only point of this change is to work around Fluid, it might be worthwhile to at least investigate this first. However, Nikita mentioned that it's not just about Fluid (contrary to the referenced issue of this patch).
  9. Nikita Hovratov commented on patch set 11 06/11/25, 10:02

    Reply

    The API is not widely used on PHP-level so far, that's why it's currently mainly an issue in Fluid. As Oli said in his last sentence, the only conceptual question to answer is, whether we should split identifiers with dots into sub-groups. This is the status quo with the new API. It only doesn't work with multiple sheets (which this patch tries to solve). For the users, it would of course be the best to not introduce another Fluid syntax and have b/w compatibility for existing Fluid code.
  10. Nikita Hovratov commented on patch set 11 06/11/25, 10:10

    Reply

    Another idea could be to restrict this behavior with sub-arrays to `settings`. This is the only commonly used group identifier, which is used in Extbase and in other context like FluidTemplateContentObject. People expect it to be a sub-array and it's very unlikely they want `settings` directly as a value.
  11. Benni Mack updated the commit message in patch set 12 CR -1 V +1 04/22/26, 06:33

  12. core-ci voted on patch set 12 V -1 04/22/26, 07:22 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/100735

  13. Christian Kuhn wrote 06/18/26, 23:08

    Removed reviewer Christian Kuhn.
  14. Oli Bartsch uploaded patch set 13 V -1 10/10/26, 09:21

  15. Oli Bartsch commented on patch set 13 10/10/26, 09:28

    Reply

    Reworked from scratch. The dotted grouping isn't new here: processFlexForm() already does it, and a single-sheet get('settings') already returns the group. The patch only stops a lookup without a sheet name from rejecting that group when it spans several sheets. Groups are now merged recursively, and any other collision still throws 1731962638. That covers all cases: - some together with some.thing: some without a sheet name throws. - A relation on settings together with settings.foo: settings without a sheet name throws. - A sheet named settings is unaffected, since sheets are only addressed with /. Fluid can't do \. (TYPO3/Fluid#391), so quoting isn't an option. Also fixed: resolved closures are now cached instead of being instantiated on every get().
  16. Oli Bartsch commented on patch set 13 10/10/26, 09:33

    RecordFieldTransformerTest.php:1824 Reply

    Done
  17. core-ci voted on patch set 13 V +1 10/10/26, 09:37 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/110782

  18. Simon Schaufelberger voted on patch set 13 CR +1 V +1 10/10/26, 18:24

  19. Oliver Klee voted on patch set 13 V +1 10/11/26, 07:29

  20. Oli Bartsch rebased onto patch set 14 CR +1 V +1 10/19/26, 09:30

  21. core-ci voted on patch set 14 V +1 10/19/26, 09:57 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/111614

  22. Garvin Hicking voted on patch set 14 CR +1 V +1 10/20/26, 07:42

  23. Simon Schaufelberger voted on patch set 14 CR +1 V +1 10/20/26, 08:01

  24. Oli Bartsch voted on patch set 14 CR +2 V +2 10/20/26, 08:03

  25. Oli Bartsch merged patch set 15 10/20/26, 08:04