Review main Merged

85046[FEATURE] Automatically resolve relations for records

Benni Mack CR +2 V +2 Nightmare +3837 −204 updated 08/27/24, 06:27

Commit message

What does it do? When creating a record
object out of a database row (via the RecordFactory),
the Record object now holds enriched values
for fields where the content is "known" based
on TCA / Schema API.

This is especially relevant for records with
possible relations, for field types such as
"type=group", "type=select" with foreign_table set
and "type=inline" as an example.

The main purpose for this relation enriching
is rendering of the backend page module and
content in frontend, but it can be used
for any kind of TCA-based record.

It also works recursively and with circular
dependencies thanks to the RecordIdentityMap.

In order to avoid any problems with large amounts
of data, an approach of "Lazy+Greedy Fetching"
was chosen.

How does this approach work under the hood?

As an example, we load 10 content elements on a
page (1 DB query) so we can render them.

Step 1: Lazy Collections / RecordPropertyClosure

RecordFactory filters out only the relevant
fields and their values from a record's type.
Now, the RecordFactory also checks for fields
with their meaning and uses a different value for
a field. Example: For a type=inline value, there
was the number "5" as value (= 5 relations) available,
now we know we need the relation records (as a collection)
properly sorted resolved there as well. For this to
work, the new RecordFieldTransformer creates
LazyCollection objects or RecordPropertyClosure objects
(for a 1:1/n:1 relation) which means that the DB query is
not made (yet) but only called when the value is accessed
for the first time ("lazy loading").

Step 2: Getting the related UID/Table Pairs

The RecordIdentityMap now knows about the
10 Records from tt_content, as they have been created
completely before handing it to the output rendering.
There comes the fun part. As soon as the value (with a
lazy closure) is accessed for the first time,
the RelationResolver checks the RelationHandler
to find the table / uids that we should resolve.

In our case, we now know that our first content element
has 5 relations to a DB table e.g. "tx_mycarousel_item"
with UIDs 12,13,14,15,16. Thanks to the RelationHandler,
we also have the proper sorting of these items.

Step 3: A greedy database query to get the full DB rows

So, for the first content element, we want the 5
complete, related DB rows. The RelationResolver
now sends this query to the "GreedyDatabaseBackend"
which uses a subquery to not only fetch
the 5 DB rows, but ALL rows of this DB table
that are on the same PID with 1 DB query (using subselects).

It however only returns the 5 items, and keeps the
other items in a runtime cache.

At this point we have made 3 DB queries.

Currently, we then do the language + workspace overlays.

Step 4: The long way back

The RelationResolver now has the full DB rows and sorts
them. The RecordFieldTransformer builds Record / Collection
objects out of it, checks if an object has been created
already (via the IdentityMap) or creates new ones,
utilizing again the Lazy approach from step 1 to ensure
we only resolve the records when we need them.

Responsibilities:

- RecordFieldTransformer
  - knows what to do based on the Field Type
  - returns objects, never raw DB records
  - initializes the lazy collections / closure objects
- RelationResolver
  - uses RelationHandler to resolve uids
  - knows and applies the sorting
- GreedyDatabaseBackend
  - does overlays / enableFields

The design decisions behind this approach:
- We only build Record objects when they are requested explicitly
- We distinguish the cardinality (1:1 / n:1 vs. 1:n)
- We do overlays on a very end of the chain

Kudos to Nikita Hovratov for creating the first draft
of this approach here: https://review.typo3.org/c/Packages/TYPO3.CMS/+/83725
along with comprehensive tests.

PS: In the mid-term, the RelationResolver could be
based on the sys_refindex and minimize queries.

Review

  • Code review CR +2 Done
    • Oli Bartsch +2
    • Benjamin Franzke +1
    • Nikita Hovratov +1
  • Verified V +2 Done
    • Oli Bartsch +2
    • Benjamin Franzke +1
    • Nikita Hovratov +1
    • core-ci +1 CI

Issues 3

Files 97

Show 77 more files

Review log 131195

Show 113170 earlier messages
  1. Benni Mack uploaded patch set 1 07/01/24, 13:07

  2. core-ci voted on patch set 1 V -1 07/01/24, 13:12 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60773

  3. Benni Mack uploaded patch set 2 V -1 07/11/24, 18:25

  4. core-ci voted on patch set 2 V -1 07/11/24, 18:31 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61406

  5. Benni Mack uploaded patch set 3 V -1 07/11/24, 20:44

  6. core-ci voted on patch set 3 V -1 07/11/24, 20:52 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61415

  7. Benni Mack rebased onto patch set 4 V -1 07/11/24, 22:26

  8. core-ci voted on patch set 4 V -1 07/11/24, 22:32 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61419

  9. Benni Mack uploaded patch set 5 V -1 07/12/24, 06:33

  10. core-ci voted on patch set 5 V -1 07/12/24, 06:41 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61452

  11. Benni Mack uploaded patch set 6 V -1 07/12/24, 23:24

  12. core-ci voted on patch set 6 V -1 07/12/24, 23:32 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61516

  13. Benni Mack uploaded patch set 7 V -1 07/12/24, 23:45

  14. core-ci voted on patch set 7 V +1 07/12/24, 23:50 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61517

  15. Benni Mack uploaded patch set 8 V +1 07/12/24, 23:51

  16. core-ci voted on patch set 8 V -1 07/12/24, 23:59 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61518

  17. Benni Mack uploaded patch set 9 V -1 07/17/24, 14:16

  18. core-ci voted on patch set 9 V -1 07/17/24, 14:26 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61695

  19. Benni Mack uploaded patch set 10 V -1 07/17/24, 14:31

  20. core-ci voted on patch set 10 V -1 07/17/24, 14:39 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61697

  21. Oli Bartsch uploaded patch set 11 V -1 07/18/24, 10:54

  22. core-ci voted on patch set 11 V -1 07/18/24, 11:03 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61739

  23. Oli Bartsch uploaded patch set 12 V -1 07/29/24, 07:34

  24. core-ci voted on patch set 12 V -1 07/29/24, 07:41 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62362

  25. Benni Mack uploaded patch set 13 V -1 08/07/24, 16:20

  26. core-ci voted on patch set 13 V -1 08/07/24, 16:28 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63005

  27. Benni Mack uploaded patch set 14 V -1 08/08/24, 23:16

  28. core-ci voted on patch set 14 V -1 08/08/24, 23:23 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63096

  29. Benni Mack uploaded patch set 15 V -1 08/09/24, 09:00

  30. core-ci voted on patch set 15 V -1 08/09/24, 09:07 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63122

  31. Benni Mack uploaded patch set 16 V -1 08/09/24, 10:43

  32. core-ci voted on patch set 16 V +1 08/09/24, 10:48 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63128

  33. Benni Mack uploaded patch set 17 V +1 08/09/24, 12:54

  34. core-ci voted on patch set 17 V +1 08/09/24, 13:01 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63139

  35. Benni Mack uploaded patch set 18 V +1 08/09/24, 13:25

  36. core-ci voted on patch set 18 V +1 08/09/24, 13:31 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63141

  37. Nikita Hovratov uploaded patch set 19 V +1 08/10/24, 21:37

  38. core-ci voted on patch set 19 V +1 08/10/24, 21:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63172

  39. Nikita Hovratov commented on patch set 19 CR -1 08/10/24, 23:05

    Testing with Content Blocks currently.
  40. Nikita Hovratov uploaded patch set 20 CR -1 V +1 08/11/24, 10:46

  41. Nikita Hovratov commented on patch set 20 08/11/24, 10:48

    LazyRecordCollection.php:83

    I changed this method to allow to modify the entries with another RecordInterface. This is needed for Content Blocks, as it decorates "Record" with "ContentBlockData" and overrides it.
  42. core-ci voted on patch set 20 V +1 08/11/24, 10:54 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63189

  43. Nikita Hovratov commented on patch set 20 08/11/24, 12:10

    RecordFieldTransformer.php:72

    FieldTypeFactory->createFieldType does not apply `columnsOverrides`. This leads to `$fieldInformation->getConfiguration()` not containing possible maxitems overrides and thus inconsistent data handling. I suppose we need to manually get the merged columns overrides array based on the context of the sub-schema.
  44. Nikita Hovratov commented on patch set 20 08/11/24, 12:22

    RecordFieldTransformer.php:103

    This is a problem. If the record data of a 1:1 / n:1 relation is empty (=null), we don't have any RawRecord for the initialized Ghost object. This means every call will result in some uninitialized property error. Do we need a NullRecord perhaps?
  45. Nikita Hovratov commented on patch set 20 08/11/24, 12:24

    RecordFieldTransformer.php:162

    As discussed in Slack: The array of parameters is pretty useless in frontend context when trying to use TypoLink VH. A parameter object would be cool, which the VH also understands.
  46. Nikita Hovratov commented on patch set 20 08/11/24, 12:44

    phpstan-baseline.neon:274

    Can we get rid of new stan errors?
  47. Oli Bartsch uploaded patch set 21 V +1 08/13/24, 10:22

  48. core-ci voted on patch set 21 V +1 08/13/24, 10:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63293

  49. Oli Bartsch uploaded patch set 22 V +1 08/13/24, 14:57

  50. core-ci voted on patch set 22 V +1 08/13/24, 15:02 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63325

  51. Oli Bartsch uploaded patch set 23 V +1 08/14/24, 11:50

  52. core-ci voted on patch set 23 V -1 08/14/24, 11:59 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63372

  53. Oli Bartsch rebased onto patch set 24 V -1 08/16/24, 08:23

  54. core-ci voted on patch set 24 V -1 08/16/24, 08:32 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63561

  55. Oli Bartsch uploaded patch set 25 V -1 08/16/24, 08:47

  56. core-ci voted on patch set 25 V -1 08/16/24, 09:04 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63562

  57. Oli Bartsch uploaded patch set 26 V -1 08/16/24, 11:27

  58. core-ci voted on patch set 26 V -1 08/16/24, 11:34 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63576

  59. Oliver Klee commented on patch set 26 08/16/24, 13:17

    LazyRecordCollection.php:96

    typo: Remvoing -> Removing
  60. Oli Bartsch uploaded patch set 27 V -1 08/16/24, 13:36

  61. core-ci voted on patch set 27 V +1 08/16/24, 13:43 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63580

  62. Oli Bartsch commented on patch set 27 08/16/24, 13:44

    phpstan-baseline.neon:274 Reply

    Done

    LazyRecordCollection.php:96 Reply

    Done

    RecordFieldTransformer.php:72 Reply

    Done

    RecordFieldTransformer.php:103 Reply

    Done

    RecordFieldTransformer.php:162 Reply

    Done
  63. Oli Bartsch uploaded patch set 28 V +1 08/16/24, 14:34

  64. core-ci voted on patch set 28 V -1 08/16/24, 14:42 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63587

  65. core-ci voted on patch set 28 V +1 08/16/24, 14:59 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63587

  66. Nikita Hovratov commented on patch set 28 CR +1 V +1 08/16/24, 15:00

    Amazing! Thanks to all!
  67. Oli Bartsch uploaded patch set 29 CR +1 V +1 08/16/24, 23:44

  68. core-ci voted on patch set 29 V +1 08/16/24, 23:49 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63599

  69. Oliver Klee commented on patch set 29 08/17/24, 18:29

    RelationResolver.php:68

    We can make the type even more specific: ``` @return list<array{table: string, row: array<string, mixed>}> ```
  70. Oli Bartsch uploaded patch set 30 V +1 08/19/24, 06:39

  71. Oli Bartsch commented on patch set 30 08/19/24, 06:40

    RelationResolver.php:68 Reply

    Done
  72. core-ci voted on patch set 30 V +1 08/19/24, 06:47 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63749

  73. Oli Bartsch uploaded patch set 31 V +1 08/19/24, 08:51

  74. core-ci voted on patch set 31 V +1 08/19/24, 08:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63756

  75. Oli Bartsch voted on patch set 31 CR +1 V +1 08/19/24, 09:02

  76. Nikita Hovratov commented on patch set 31 08/19/24, 18:28

    RecordFieldTransformer.php:79

    We discussed this in our Content Blocks meeting today. So this differs from our implementation: It doesn't matter what is defined for maxitems: file, select and group are always a Collection no matter what. Only exception we made is selectSingle. Reason: A change in TCA maxitems has a direct influence in how templates need to be build. This behavior is unforseeable and can lead to frustration. I would rather avoid this and have less "beautiful" templates that unexpected behavior.
  77. André Buchmann commented on patch set 31 08/19/24, 19:40

    RecordFieldTransformer.php:79 Reply

    I get your reasoning to remove the maxitems logic. None the less I opt to keep it. By setting maxitems=1 I expect to get only one result, just like with renderType=selectSingle. It also enables a generic way for custom renderType definitions to create the non-collection behaviour for relations.
  78. Nikita Hovratov commented on patch set 31 08/20/24, 06:38

    RecordFieldTransformer.php:79 Reply

    Hi Andre, yes, at first thought this might make more sense and looks better in the template. However, changes in maxitems will always be breaking. 1. Your own templates will need adaption. 2. Your code needs adaption. 3. You can't use partials at all (Think about the Gallery partial, which supports 1 or more images) 4. Changing maxitems in third-party extensions will break 1. and 2. and you can't easily fix it. The only exception I would make is type select with renderType selectSingle. This really means it is only possible to have 1 value logically. From own experience, it happens more than often that requirements change and you have to allow multiple images or something similar. Such a change will cost more time (and money). At last, Content Blocks already has this behavior and it works great. There was nobody complaining not having a non-collection. And upgrades will be smoother for current users when we integrate this.
  79. Oli Bartsch uploaded patch set 32 CR +1 V +1 08/20/24, 07:24

  80. core-ci voted on patch set 32 V +1 08/20/24, 07:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63806

  81. Oli Bartsch commented on patch set 32 08/20/24, 07:32

    RecordFieldTransformer.php:79 Reply

    I have updated the patch. Let's wait for some feedback from @benni@typo3.org
  82. André Buchmann commented on patch set 32 08/20/24, 11:20

    RecordFieldTransformer.php:79 Reply

    Heyho Nikita, totally agree with the cost efficiency point for changing requirements. Changing maxitems to something bigger than 1 would cause a adaption of the templates. The same argument can be provided for the renderType. Requirement changes can lead to changes in the renderType from a single selection to a multi select or the other way round. I actually had this case already. Why not provide relational fields always as collections? That way you can be sure to get always the same data structure event after "small" interface changes for the editor.
  83. Nikita Hovratov commented on patch set 32 08/20/24, 11:37

    RecordFieldTransformer.php:79 Reply

    The select field currently has the problem that it does not define cardinality by its type. Type "select" can be anything 1:1, 1:n or n:1 depending on how the user works with it. So the renderType "selectSingle" is the only indicator for a n:1 relation we have right now. This is sub-optimal of course. My vision is to split type select into "selectOne" and "selectMultiple". Then you can use the type to determine, if you have an array of items or not. RenderTypes can still be there to either have a "selectSingle", "sideBySide", etc for "selectOne" and everything except "selectSingle" for "selectMultiple". Having this in mind, the best option is to use "selectSingle" as indicator for now and later to migrate it to type "selectOne". This means using either type is a hard opt-in and changing it later leads to changes in code. So the clever developer should use "selectMultiple(SideBySide)" with "maxitems: 1", if he thinks requirements might change.
  84. Oli Bartsch commented on patch set 32 08/20/24, 12:23

    RecordFieldTransformer.php:79 Reply

    +100 for finally resolving the renderType=selectSingle. This bothers me since we migrated all those renderTypes and "eval" options of type input to dedicated types. selectSingle might be (not sure at the moment) the last renderType which needs to be checked for data processing. renderType's however were introduced for FormEngine display purposes only. https://github.com/TYPO3/typo3/blob/main/typo3/sysext/core/Classes/Configuration/Tca/TcaPreparation.php#L346
  85. Benni Mack commented on patch set 32 08/20/24, 17:54

    RecordFieldTransformer.php:79 Reply

    I do understand this makes the whole templating part easier, and plus, (at least that's my understanding) we're backwards-compatible with existing CB templates. I however think that resolving a 1:1 relation as a collection is semantically wrong. I mean, even people in Extbase don't make this a ObjectStorage but a relation to a single object. Next to the use-case you mentioned with "select" I also consider a group field to one single record not a collection. If I have a DB field "Responsible BE User" pointing to a be user object, I don't want a collection. at all. So this is two-fold and depending on the use-case. So, if we can agree that a) there is a semantic difference between 1:1 (maxitems = 1) and 1:n (maxitems != 1) b) the topic is about making templating easier / "non-breaking" and adding compat with CB <= 0.7) then we might find a solution.
  86. Nikita Hovratov commented on patch set 32 08/20/24, 18:10

    RecordFieldTransformer.php:79 Reply

    > a) there is a semantic difference between 1:1 (maxitems = 1) and 1:n (maxitems != 1) I wouldn't make it dependent on maxitems. If at all, we need to introduce another TCA option "relationship" like with type "category". Reason: In some cases I want to override other extensions with "maxitems: 1", eventhough it is a Collection. This shall not be breaking in the extensions code and might be unexpected. So maxitems is in my opinion a poor man's solution to this problem that we should avoid at all cost. > b) the topic is about making templating easier / "non-breaking" and adding compat with CB <= 0.7) yes, the non-breaking part is the most important. Having compat to CB is a bonus.
  87. Benni Mack commented on patch set 32 08/20/24, 18:14

    RecordFieldTransformer.php:79 Reply

    Because then I would solve it like this: we _could_ add a custom Iterator to the RecordLazyGhost, which would then return the actual Record object (which would then only be one record object) and we could iterate the same way as iterating over the Collection. I agree that we should add a new TCA option and unify this feature set, let's also consult Olly Hader for this.
  88. Nikita Hovratov commented on patch set 32 08/20/24, 18:22

    RecordFieldTransformer.php:79 Reply

    If I understand you correctly, I then have the option to use it directly or as a Collection, depending on if I want to keep it non-breaking for outside (or own) changes? This could be a good compromise for people who want clean (and semantically correct) code and for extension authors deciding to keep their code compatible for TCA overrides.
  89. Nikita Hovratov commented on patch set 32 08/20/24, 18:35

    RecordFieldTransformer.php:79 Reply

    Ah, now I get it. When I have a collection and then override it to only allow 1 value, then the custom iterator kicks in and it works. When I start out with only 1 value and then remove the restriction for 1 value, then it's breaking. But I could write my code from the beginning as a collection, eventhough I only allow 1 value in case the requirements change. Then no code changes would be necessary.
  90. Oli Bartsch uploaded patch set 33 V +1 08/21/24, 14:25

  91. Oli Bartsch commented on patch set 33 08/21/24, 14:26

    RecordFieldTransformer.php:79 Reply

    Hi @benni@typo3.org @nikita.h@live.de, I've updated the patch, please have a look.
  92. Oli Bartsch uploaded patch set 34 08/21/24, 14:30

  93. core-ci voted on patch set 33 V -1 08/21/24, 14:33 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63915

  94. core-ci voted on patch set 34 V +1 08/21/24, 14:39 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63916

  95. Nikita Hovratov commented on patch set 34 08/21/24, 17:19

    RecordFieldTransformer.php:94

    We need a NullRecord just like for normal Records here.

    RecordFieldTransformer.php:113

    We need a NullRecord just like for normal Records here.

    RecordLazyGhost.php:30

    We need \Countable interface as well.

    FileReferenceLazyGhost.php:25

    We need \Countable interface as well.

    FolderLazyGhost.php:25

    We need \Countable interface as well.
  96. Nikita Hovratov commented on patch set 34 08/21/24, 17:20

    RecordFieldTransformer.php:79 Reply

    This looks alright to me. See my comments for new findings I had.
  97. Nikita Hovratov updated the commit message in patch set 35 V +1 08/21/24, 17:21

  98. core-ci voted on patch set 35 V +1 08/21/24, 17:26 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63927

  99. Oli Bartsch uploaded patch set 36 V +1 08/22/24, 06:37

  100. core-ci voted on patch set 36 V +1 08/22/24, 06:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63957

  101. Oli Bartsch commented on patch set 36 08/22/24, 06:45

    RecordFieldTransformer.php:94 Reply

    This seems to be not as easy as it sounds since the $instance is always a FileReference. For Records, we also return a record while setting the NullRecord only instead of the RawRecord. This is different to the FileReference where we would need to "fake" a reference due to the internal logic e.g. accessing a original file.

    RecordFieldTransformer.php:113 Reply

    Same problem as with FileReference

    RecordLazyGhost.php:30 Reply

    Done

    FileReferenceLazyGhost.php:25 Reply

    Done

    FolderLazyGhost.php:25 Reply

    Done
  102. Nikita Hovratov commented on patch set 36 08/22/24, 07:11

    RecordFieldTransformer.php:94 Reply

    Acknowledged

    RecordFieldTransformer.php:113 Reply

    Acknowledged
  103. Nikita Hovratov voted on patch set 36 V +1 08/22/24, 08:24

  104. Oli Bartsch voted on patch set 36 CR +1 V +1 08/22/24, 16:09

  105. Nikita Hovratov uploaded patch set 37 CR +1 V +1 08/22/24, 17:36

  106. core-ci voted on patch set 37 V +1 08/22/24, 17:43 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64005

  107. Nikita Hovratov commented on patch set 37 08/22/24, 17:47

    RecordFieldTransformer.php:79 Reply

    Unfortunately, the LazyGhost objects still make trouble. There is no way to know, if there is a filled record behind the ghost or nothing (no set). Here an example: ``` <f:if condition="{data.record}"> Record: {data.record.title} </f:if> ``` The condition will always be `true` as it is an object no matter what. The workaround could be: ``` <f:if condition="{data.record.uid}"> Record: {data.record.title} </f:if> ``` But this is not how people are used to it (e.g. in extbase). Same problem of course with FileReference. There one must check for `{image.originalFile}` as image will always be an uninitialized FileReference. In ContentBlocks I can circumvent this in my ContentBlockDataDecorator where I can check for `NullRecord` and `FileReference->getOriginalFile()` in the code. However, this leads of course to the lazy thing not being used at all.
  108. Nikita Hovratov commented on patch set 37 08/22/24, 17:51

    FluidBasedContentPreviewRenderer.php:94

    This could lead in seldom cases that `record` from the `$row` is overridden. But only if someone defined such a custom column. Maybe we can just ignore it, as it is such an unlikely event. What do you think?
  109. Nikita Hovratov commented on patch set 37 08/22/24, 20:29

    FluidBasedContentPreviewRenderer.php:94 Reply

    We ignore this and make it in v14 the only variable as breaking change.
  110. Benjamin Franzke uploaded patch set 38 V +1 08/23/24, 09:43

  111. Benjamin Franzke uploaded patch set 39 08/23/24, 09:50

  112. core-ci voted on patch set 38 V -1 08/23/24, 09:51 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64059

  113. core-ci voted on patch set 39 V +1 08/23/24, 09:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64060

  114. Oliver Klee wrote 08/23/24, 10:06

    Removed cc Oliver Klee.
  115. Benjamin Franzke uploaded patch set 40 V +1 08/23/24, 11:35

  116. core-ci voted on patch set 40 V +1 08/23/24, 11:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64064

  117. Benjamin Franzke uploaded patch set 41 V +1 08/23/24, 12:04

  118. core-ci voted on patch set 41 V +1 08/23/24, 12:10 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64067

  119. Benjamin Franzke uploaded patch set 42 V +1 08/23/24, 14:09

  120. core-ci voted on patch set 42 V -1 08/23/24, 14:16 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64083

  121. Benjamin Franzke uploaded patch set 43 V -1 08/23/24, 14:24

  122. core-ci voted on patch set 43 V +1 08/23/24, 14:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64085

  123. Benjamin Franzke uploaded patch set 44 V +1 08/23/24, 15:25

  124. Benjamin Franzke commented on patch set 44 08/23/24, 15:27

    RecordFieldTransformer.php:125

    @benni@typo3.org I'd like to discuss with you whether `RecordInterface` is fine to be returned here. I added that in https://review.typo3.org/c/Packages/TYPO3.CMS/+/85046/43..44 because non `Record` instances were wrapped into the lazy record proxy. My guess is: That wrapping only happens because the LadyRecordProxy was too inflexible and therefore that was done. Therefore I started to allow `RecordInterface` to be returned here. (Although I think currently no code puts non `Record` objects into the record identity map)
  125. Nikita Hovratov commented on patch set 44 08/23/24, 15:27

    RecordFieldTransformer.php:79 Reply

    Done
  126. Benjamin Franzke commented on patch set 44 08/23/24, 15:32

    RecordFieldTransformer.php:79 Reply

    > Unfortunately, the LazyGhost objects still make trouble Ack, I came to the same conclusion, which is why lazy 1:* relations are now handled by the Record object instead of proxies.

    RecordFieldTransformer.php:125 Reply

    acked by benni via slack
  127. core-ci voted on patch set 44 V -1 08/23/24, 15:34 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64087

  128. Nikita Hovratov commented on patch set 44 08/23/24, 15:35

    RecordFieldTransformer.php:79 Reply

    We did a full cycle with fixing bugs and at the end we are again at the start of this discussion. The workaround with \IteratorInterface now doesn't work anymore. Ok, so fine. As long as we keep in mind that we should mid-term introduce said TCA options for relationships, we can go with that. The ultimate goal should be to allow maxitems: 1 and a "oneToMany" relation like it is possible in extbase. As it is easy to break templates/code, this needs a prominent warning in the rst. I will document this as a breaking change in the next Content Blocks release.
  129. Benjamin Franzke uploaded patch set 45 V -1 08/23/24, 15:38

  130. core-ci voted on patch set 45 V +1 08/23/24, 15:45 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64088

  131. Benjamin Franzke commented on patch set 45 08/23/24, 15:52

    RecordFieldTransformer.php:79 Reply

    Implementing iterators is only half of the story, then you'd also need `ArrayAccess`, because templates may use `foo.image.0`… I'm really against workarounds in a new implementation, sorry. I think the real curlpit is that the relation type is derived from `maxitems=1`. That is way too implicit and should probably be reconsidered. I'm in favor of an explicit configuration option and personally think common things like `.image` and `.assets` should always resolve to collections. …but this is just my opinion.
  132. Nikita Hovratov commented on patch set 45 08/23/24, 16:10

    RecordFieldTransformer.php:79 Reply

    Full ack. I didn't have the direct access in mind, thanks. With .image and .assets always resolve to collection, I agree. I can imagine them having the "oneToMany" relationship option later, so it is derived from this. Overrides of this option should be discouraged. This would be like overriding the TCA type.
  133. Nikita Hovratov uploaded patch set 46 V +1 08/24/24, 00:53

  134. core-ci voted on patch set 46 V +1 08/24/24, 01:00 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64105

  135. Nikita Hovratov uploaded patch set 47 V +1 08/24/24, 01:35

  136. core-ci voted on patch set 47 V +1 08/24/24, 01:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64107

  137. Nikita Hovratov commented on patch set 47 08/24/24, 02:25

    RecordFieldTransformer.php:79 Reply

    Hey, so I just did that. The problem is, if we don't do it now, we will have breaking changes later. Summary of my changes: - Introduce TCA option "relationship" for all "relation" type fields. - Change all "maxitems: 1" occurences to "relationship: 'oneToOne'" - Add TcaPreparation for "relationship: 'oneToOne'" to "maxitems: 1" - Adjust TcaPreparation of "renderType: 'selectSingle'" to "relationship: 'oneToOne'" (but only, if relationship is not set already) - Only use relationship to determine cardinality in FieldTransformer. - Adjust rst accordingly. Now the separation between maxitems and cardinality is clean. Current Content Blocks templates won't break either. Please, if I did forget something hit me up. I also wish this patch to finally go through. I will do the best to my abilities to make everything right.
  138. Nikita Hovratov voted on patch set 47 CR +1 V +1 08/24/24, 02:25

  139. Benni Mack commented on patch set 47 08/24/24, 15:51

    Thanks for your much appreciated effort, Nikita. Fine by me with the changes, we just need to be sure to document them (and I assume we need to evaluate this somewhere in DataHandler and Extbase as well at a later stage)!
  140. Oliver Klee commented on patch set 47 08/24/24, 16:06

    Are we sure that all relations marked as "oneToOne" are actually 1:1 relationships and not n:1 relationships (e.g., where pages are referenced)?
  141. Oliver Klee commented on patch set 47 08/24/24, 16:07

    Reply

    For example, the fe_groups.felogin_redirectPid looks more like an n:1 relationship than a 1:1 relationship.
  142. Christian Kuhn commented on patch set 47 08/25/24, 11:50

    StandardContentPreviewRenderer.php:91

    DI possible?

    typo3tests_contentelementb_collection_recursive.php:67

    we shouldn't need the columns enabled in ctrl anymore.

    ext_tables.sql:3

    we shouldn't need these anymore.
  143. Oli Bartsch uploaded patch set 48 CR +1 V +1 08/25/24, 13:46

  144. Oli Bartsch uploaded patch set 49 08/25/24, 13:53

  145. core-ci voted on patch set 48 V +1 08/25/24, 13:54 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64156

  146. Oli Bartsch commented on patch set 49 08/25/24, 13:55

    StandardContentPreviewRenderer.php:91 Reply

    Unfortunately this class is not internal and actively extended, even by Core (`FormPagePreviewRenderer`).

    typo3tests_contentelementb_collection_recursive.php:67 Reply

    Done

    ext_tables.sql:3 Reply

    Done
  147. core-ci voted on patch set 49 V +1 08/25/24, 14:10 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64157

  148. Oliver Klee commented on patch set 49 08/25/24, 15:46

    Reply

    If we're treating n:1 and 1:1 relations the same as far as the left side of the relation is concerned (as it seems), maybe we can rename this to "toOne" in order to make this correct and less confusing?
  149. Benjamin Franzke commented on patch set 49 08/26/24, 06:42

    RecordFieldTransformer.php:79 Reply

    thanks for the immediate effort Nikita, sounds great to me!
  150. Benjamin Franzke commented on patch set 49 08/26/24, 07:30

    Reply

    ack, we should add `manyToOne` as well, and basically set it most of the times, as the relations that were changed are actually manyToOne, I only found one exception: `sys_file_metadata.metadata => sys_file_metadata` which is a true 1:1 relation. ``` be_users.avatar => file sys_file_metadata.l10n_parent => sys_file_metadata sys_file_reference.l10n_parent => sys_file_reference pages.shortcut => pages pages.content_from_pid => pages pages.mount_pid => pages 1:1! sys_file.metadata => sys_file_metadata sys_file_collection.folder_identifier => folder sys_file_reference.uid_local => sys_file sys_filemounts.identifier => folder fe_groups.felogin_redirectPid => pages fe_users.felogin_redirectPid => pages backend_layout.icon => file index_config.alternative_source_pid => pages reactions.storage_pid => pages reactions.impersonate_user => be_user ``` I'd support both, `manyToOne` and `oneToOne`, and having them explicit will allow for proper introspection via TcaSchema API.
  151. Nikita Hovratov commented on patch set 49 08/26/24, 08:55

    Reply

    TCA type category has "relationship" "oneToOne" eventhough it is technically "manyToOne": https://docs.typo3.org/m/typo3/reference-tca/main/en-us/ColumnsConfig/Type/Category/Index.html#confval-category-relationship I'm not sure if we really need this distinction for TcaSchema. Would this be somehow used in the future? I think this could be a case, where we could keep it simple instead of 100% technically correct. Otherwise, we would need to streamline type category as well or this would lead to confusion.
  152. Oli Bartsch uploaded patch set 50 V +1 08/26/24, 10:02

  153. core-ci voted on patch set 50 V -1 08/26/24, 10:11 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64192

  154. Christian Kuhn commented on patch set 50 08/26/24, 10:24

    StandardContentPreviewRenderer.php:91 Reply

    then use an inject* method and leave a comment this is done because the class is not a leaf class so other classes that extend can use __construct() without calling parent::__construct(). (just as note, you can leave that out in this patch, i don't insist on this since the change is about a different thing and we could quite easily change this details later. if so, closes comment. if you find the time, change it).
  155. Benjamin Franzke uploaded patch set 51 V -1 08/26/24, 10:56

  156. Oliver Klee commented on patch set 51 08/26/24, 11:01

    TcaPreparation.php:334

    2 typos: ``` * Add "'relationship' => 'manyToOne" to all "'type' => 'select'" column fields with "'renderType' => 'selectSingle'". ```
  157. core-ci voted on patch set 51 V +1 08/26/24, 11:04 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64196

  158. Benjamin Franzke uploaded patch set 52 V +1 08/26/24, 11:48

  159. Benjamin Franzke commented on patch set 52 08/26/24, 11:48

    Reply

    Let's go for the clean api for new types right away and treat category as before. Documentation is already pretty good in outlining that the naming is wrong and that can be clarified there, that the relationship properties are badly named.

    StandardContentPreviewRenderer.php:91 Reply

    I of course fully agree lolli, but would resolve because the class already does this "wrong" for `GeneralUtility::makeInstance(IconFactory::class)`, which means we'd have to refactor some non-relevant cases. That makes it better suited for a separate TASK to streamline this class to use DI.

    TcaPreparation.php:334 Reply

    Done
  160. core-ci voted on patch set 52 V +1 08/26/24, 11:53 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64199

  161. Nikita Hovratov commented on patch set 52 08/26/24, 12:14

    TcaPreparation.php:347

    This should be foreign_table I believe.
  162. Benjamin Franzke commented on patch set 52 CR -1 08/26/24, 12:51

    TcaPreparation.php:347 Reply

    upfront: i think we messed up since patchset 47 and somewhere confused manyToOne with oneToMany. The idea to check for `foreign_field` was to check for `many` to apply, but that's `oneToMany` then, but we've now written `manyToOne`. I think we got confused by the `ToOne` debate. It's not about to-one, but `FromOne`. I'm going to rethink all changes since patchset 47 now. Sorry. (And for reference, I wanted to exclude mm cases like typo3/sysext/backend/Tests/Functional/Fixtures/Extensions/test_tca_select_items/Configuration/TCA/tca_select_items.php `mm_field`)
  163. Benjamin Franzke uploaded patch set 53 CR -1 V +1 08/26/24, 14:00

  164. Benjamin Franzke uploaded patch set 54 08/26/24, 14:14

  165. core-ci voted on patch set 53 V -1 08/26/24, 14:16 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64208

  166. Oli Bartsch uploaded patch set 55 08/26/24, 14:17

  167. core-ci voted on patch set 54 V -1 08/26/24, 14:22 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64210

  168. core-ci voted on patch set 55 V -1 08/26/24, 14:29 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64211

  169. Oli Bartsch uploaded patch set 56 V -1 08/26/24, 15:56

  170. core-ci voted on patch set 56 V -1 08/26/24, 16:07 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64224

  1. Oli Bartsch uploaded patch set 57 V -1 08/26/24, 16:20

  2. core-ci voted on patch set 57 V +1 08/26/24, 16:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64229

  3. Nikita Hovratov commented on patch set 57 08/26/24, 16:35

    TcaPreparation.php:347 Reply

    Done
  4. Nikita Hovratov commented on patch set 57 08/26/24, 16:41

    TcaPreparation.php:358

    There is no foreign_field option for select as far as I know. So no true oneToMany is possible, only a list of uids.
  5. Benjamin Franzke uploaded patch set 58 V +1 08/26/24, 17:28

  6. Benjamin Franzke commented on patch set 58 08/26/24, 17:28

    TcaPreparation.php:347 Reply

    Done

    TcaPreparation.php:358 Reply

    thanks, too bad. Removed that for now.
  7. Benjamin Franzke uploaded patch set 59 08/26/24, 17:32

  8. core-ci voted on patch set 58 V +1 08/26/24, 17:35 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64237

  9. core-ci voted on patch set 59 V +1 08/26/24, 17:37 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64238

  10. Benjamin Franzke uploaded patch set 60 V +1 08/26/24, 17:51

  11. Benjamin Franzke commented on patch set 60 08/26/24, 17:51

    RecordFieldTransformer.php:175

    if `nullable=true` is set in configuration, `0` needs to be interpreted as `01.01.1970`, not as `null`
  12. core-ci voted on patch set 60 V +1 08/26/24, 17:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64239

  13. Benjamin Franzke uploaded patch set 61 V +1 08/26/24, 18:52

  14. Benjamin Franzke commented on patch set 61 08/26/24, 18:52

    RecordFieldTransformer.php:175 Reply

    Done
  15. core-ci voted on patch set 61 V +1 08/26/24, 18:57 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64248

  16. Nikita Hovratov uploaded patch set 62 CR +1 V +1 08/26/24, 19:46

  17. core-ci voted on patch set 62 V +1 08/26/24, 19:54 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64249

  18. Nikita Hovratov voted on patch set 62 CR +1 V +1 08/26/24, 19:55

  19. Oli Bartsch uploaded patch set 63 CR +1 V +1 08/26/24, 21:17

  20. core-ci voted on patch set 63 V +1 08/26/24, 21:22 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64255

  21. Oli Bartsch voted on patch set 63 CR +1 V +1 08/26/24, 21:25

  22. Benjamin Franzke voted on patch set 63 CR +1 V +1 08/27/24, 03:01

  23. Nikita Hovratov voted on patch set 63 CR +1 V +1 08/27/24, 06:25

  24. Oli Bartsch voted on patch set 63 CR +2 V +2 08/27/24, 06:27

  25. Oli Bartsch merged patch set 64 08/27/24, 06:27