83747[FEATURE] Add record-based transformation Data Processor
Commit message
Review
-
Code review CR +2 Done
- Benni Mack +2
- Nikita Hovratov +1
- Andreas Kienast +1
-
Verified V +2 Done
- Benni Mack +2
- Nikita Hovratov +1
- Andreas Kienast +1
- core-ci +1 CI
Issues 1
Files 24
- typo3/sysext/core/Classes/Domain/Record/ComputedProperties.phpnew+55 −0
- typo3/sysext/core/Classes/Domain/Record/LanguageInfo.phpnew+47 −0
- typo3/sysext/core/Classes/Domain/Record/SystemProperties.phpnew+119 −0
- typo3/sysext/core/Classes/Domain/Record/VersionInfo.phpnew+55 −0
- typo3/sysext/core/Classes/Domain/RawRecord.phpnew+98 −0
- typo3/sysext/core/Classes/Domain/Record.php10new+132 −0
- typo3/sysext/core/Classes/Domain/RecordFactory.php26new+295 −0
- typo3/sysext/core/Classes/Domain/RecordInterface.php2new+46 −0
- typo3/sysext/core/Documentation/Changelog/13.2/Feature-103783-RecordTransformationDataProcessor.rst6new+169 −0
- typo3/sysext/core/Tests/Functional/Domain/RecordFactoryTest.phpnew+114 −0
- typo3/sysext/core/Tests/Functional/Fixtures/pages.csv+28 −27
- typo3/sysext/core/Tests/Unit/Domain/RecordFactoryTest.phpnew+387 −0
- typo3/sysext/frontend/Classes/DataProcessing/RecordTransformationProcessor.php4new+84 −0
- typo3/sysext/frontend/Configuration/Services.yaml+4 −0
- typo3/sysext/frontend/Tests/Functional/DataProcessing/Fixtures/PageLayouts/Default.tsconfignew+61 −0
- typo3/sysext/frontend/Tests/Functional/DataProcessing/Fixtures/PageLayouts/Home.tsconfignew+41 −0
- typo3/sysext/frontend/Tests/Functional/DataProcessing/Fixtures/PageLayouts/Productdetail.tsconfignew+61 −0
- typo3/sysext/frontend/Tests/Functional/DataProcessing/Fixtures/RecordTransform/Pages/Default.htmlnew+8 −0
- typo3/sysext/frontend/Tests/Functional/DataProcessing/Fixtures/RecordTransform/Pages/Home.htmlnew+8 −0
- typo3/sysext/frontend/Tests/Functional/DataProcessing/Fixtures/RecordTransform/Pages/Productdetail.htmlnew+8 −0
Show 4 more files
- typo3/sysext/frontend/Tests/Functional/DataProcessing/Fixtures/RecordTransform/Partials/SingleContent.htmlnew+6 −0
- typo3/sysext/frontend/Tests/Functional/DataProcessing/Fixtures/ContentScenario.yamlnew+70 −0
- typo3/sysext/frontend/Tests/Functional/DataProcessing/Fixtures/setup.typoscriptnew+9 −0
- typo3/sysext/frontend/Tests/Functional/DataProcessing/RecordTransformationProcessorTest.phpnew+92 −0
Review log 70105
Show 5180 earlier messages
-
Benni Mack uploaded patch set 1 04/11/24, 13:10
-
core-ci voted on patch set 1 V -1 04/11/24, 13:17 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55771
-
Benni Mack uploaded patch set 2
V -104/11/24, 14:09 -
core-ci voted on patch set 2 V -1 04/11/24, 14:16 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55775
-
Benni Mack uploaded patch set 3
V -104/11/24, 14:51 -
core-ci voted on patch set 3 V -1 04/11/24, 14:57 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55776
-
Benni Mack uploaded patch set 4
V -104/11/24, 15:03 -
core-ci voted on patch set 4 V -1 04/11/24, 15:09 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55777
-
Benni Mack uploaded patch set 5
V -104/11/24, 16:51 -
core-ci voted on patch set 5 V -1 04/11/24, 16:58 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55782
-
Nikita Hovratov rebased onto patch set 6
V -104/11/24, 20:34 -
core-ci voted on patch set 6 V -1 04/11/24, 20:43 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55790
-
Stefan Bürk uploaded patch set 7
V -104/12/24, 11:34 -
core-ci voted on patch set 7 V +1 04/12/24, 11:39 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55860
-
Stefan Bürk rebased onto patch set 8
V +104/26/24, 19:43 -
core-ci voted on patch set 8 V +1 04/26/24, 19:48 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56801
-
Nikita Hovratov uploaded patch set 9 04/26/24, 19:57
Copied Votes: * Verified+1 (copy condition: "**changekind:NO_CHANGE**") -
core-ci voted on patch set 9 V +1 04/26/24, 20:03 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56803
-
Benni Mack uploaded patch set 10
V +105/02/24, 20:07 -
core-ci voted on patch set 10 V +1 05/02/24, 20:12 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57114
-
Benni Mack uploaded patch set 11
V +105/02/24, 20:21 -
core-ci voted on patch set 11 V +1 05/02/24, 20:26 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57115
-
Oli Bartsch uploaded patch set 12
V +105/03/24, 11:19 -
core-ci voted on patch set 12 V -1 05/03/24, 11:24 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57171
-
Oli Bartsch uploaded patch set 13
V -105/03/24, 16:48 -
core-ci voted on patch set 13 V +1 05/03/24, 16:53 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57175
-
Benni Mack uploaded patch set 14
V +105/07/24, 20:23 -
core-ci voted on patch set 14 V +1 05/07/24, 20:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57362
-
Simon Schaufelberger commented on patch set 14 05/07/24, 23:07
Feature-103783-RecordTransformationDataProcessor.rst:20space missing -
Benni Mack commented on patch set 15
V +105/08/24, 06:24Published edit on patch set 14. -
Benni Mack commented on patch set 15 05/08/24, 06:24
Feature-103783-RecordTransformationDataProcessor.rst:20ReplyDone -
core-ci voted on patch set 15 V +1 05/08/24, 06:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57387
-
Simon Schaufelberger wrote 05/08/24, 07:50
Removed cc Simon Schaufelberger. -
Simon Praetorius commented on patch set 15 05/08/24, 09:38
Great effort to introduce value objects here. 😊RecordFactory.php:41Should this maybe throw an exception if the table does not exist in TCA?RecordFactory.php:59Aren't these two variables the same?RecordFactory.php:64I guess this covers type definitions that refer to fields that are not properly defined as columns in TCA? If so, maybe add a comment to explain this.RecordFactory.php:66I don't get the first part of this condition: Should other arbitrary fields be allowed that are not defined in TCA? Doesn't seem right to me.RecordFactory.php:117Is it intended for the timestamps to be UTC? If not: (new \DateTimeImmutable())->setTimestamp(...)RecordFactory.php:179Is the last fallback correct here? I think it should be $tcaForTable['types'][0].RecordFactory.php:188This looks confusing without a comment explaining it. Maybe also extract it to a helper method that can be used below as well?RecordTransformationProcessor.php:71I'm not sure if this is helpful. There might be cases where data isn't an array containing the raw record. -
Georg Ringer commented on patch set 15 05/08/24, 15:21
one small nitpicking: while the processor is called RecordTransformDataProcessor, the name is transform-records. singular vs plural and ordering. I would go with TransformRecordDataProcessor and transform-record -
Nikita Hovratov commented on patch set 15 05/08/24, 18:13
RecordFactory.php:62This should be $typeFieldName !== null => the value could also be "0" but with an existing type field. -
Nikita Hovratov commented on patch set 15 05/08/24, 18:30
RecordFactory.php:124isset($properties[$tcaConfig['ctrl']['descriptionColumn']) -
Nikita Hovratov commented on patch set 15 05/08/24, 18:50
RecordFactory.php:50I don't fully understand the reasoning behind this special treatment of tt_content. It only makes it harder to use in scenarios where you need the actual database table. E.g. if you want to render nested Content Elements you need the table name and the raw record: <f:cObject.render typoscriptObjectPath="{record.fullType}" table="{record.mainType}" data="{record.rawRecord}"/> Is there any plan to introduce the naming / fallback layer for "content" => "tt_content" in other places, too? -
Nikita Hovratov commented on patch set 15 05/08/24, 19:05
RecordFactory.php:105Import FQCNRecordInterface.php:46It's not possible right now to use this in Fluid Context, is it? Maybe we could add a getter for the array representation? Could be useful if ViewHelper expect an array (for type hinted methods).Feature-103783-RecordTransformationDataProcessor.rst:50I think we also need some examples on how to use this variable in Fluid. The problem is, the debug output is not helpful for integrators and they will fall on their nose trying to retrieve {record.properties.xyz} because it is displayed in the var dump. See example how I did this for Content Blocks here: https://docs.typo3.org/p/contentblocks/content-blocks/main/en-us/Templating/Index.html#accessing-variables -
Nikita Hovratov uploaded patch set 16
V +105/09/24, 14:01 -
Nikita Hovratov commented on patch set 16 05/09/24, 14:02
RecordFactory.php:62ReplyDoneRecordFactory.php:105ReplyDoneRecordFactory.php:124ReplyDone -
core-ci voted on patch set 16 V +1 05/09/24, 14:05 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57445
-
Nikita Hovratov uploaded patch set 17
V +105/09/24, 14:58 -
Nikita Hovratov commented on patch set 17 05/09/24, 14:59
RecordFactory.php:179ReplyDone -
core-ci voted on patch set 17 V +1 05/09/24, 15:05 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57446
-
Nikita Hovratov uploaded patch set 18
V +105/09/24, 15:31 -
Nikita Hovratov commented on patch set 17 05/09/24, 15:31
RecordFactory.php:41ReplyDoneRecordFactory.php:59ReplyDoneRecordFactory.php:64ReplyDoneRecordFactory.php:66ReplyDoneRecordFactory.php:188ReplyDone -
core-ci voted on patch set 18 V +1 05/09/24, 15:35 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57447
-
Nikita Hovratov uploaded patch set 19
V +105/09/24, 16:07 -
core-ci voted on patch set 19 V -1 05/09/24, 16:14 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57448
-
Nikita Hovratov uploaded patch set 20
V -105/09/24, 16:17 -
core-ci voted on patch set 20 V -1 05/09/24, 16:22 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57449
-
Nikita Hovratov uploaded patch set 21
V -105/09/24, 17:26 -
core-ci voted on patch set 21 V +1 05/09/24, 17:31 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57450
-
Nikita Hovratov commented on patch set 21 05/09/24, 17:34
Record.php:37There is no way to access special properties like `isDisabled` or `publishAt` in Fluid yet. Should we define getter for them, too?Record.php:119This is either the original uid, when in workspace context or the overlaid uid when on a translation. The naming is hard here, because it is more usage related (uid used to later resolve relations, I guess). Maybe we should split this in `getOriginalUid` and `getLocalizedUid` for usages in Fluid? -
Nikita Hovratov commented on patch set 21 05/09/24, 17:35
Reply
Yes, this should be streamlined. -
Nikita Hovratov commented on patch set 21 05/09/24, 17:35
Feature-103783-RecordTransformationDataProcessor.rst:50ReplyDone -
Nikita Hovratov commented on patch set 21 05/09/24, 17:45
Record.php:77One problem that might pop up is, that "type" is a really common name. In most cases "type" is also used as the TCA `type` field, so this would result in the same output. However, for any other usage this would make it impossible to retrieve the property. In Content Blocks I named this `typeName`. Do you think we could stick to this name? This would also ease the transition to Content Blocks later. -
Nikita Hovratov uploaded patch set 22
V +105/09/24, 18:04 -
core-ci voted on patch set 22 V +1 05/09/24, 18:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57451
-
Nikita Hovratov uploaded patch set 23
V +105/09/24, 18:29 -
core-ci voted on patch set 23 V +1 05/09/24, 18:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57452
-
Nikita Hovratov commented on patch set 23 05/09/24, 18:36
RecordTransformationProcessor.php:71ReplyYeah, this will cause a bug when there is a field called `data` in the record. I changed things up a bit, so that the input variable name is now `data`. This works for FLUIDTEMPLATE and DataBaseQueryProcessor without causing side-effects. @benni@typo3.org I'm not sure if we actually should handle the previous default case "If variableName left empty, use all properties as input". I have no knowledge of such a case. -
Nikita Hovratov uploaded patch set 24
V +105/09/24, 18:36 -
core-ci voted on patch set 24 V +1 05/09/24, 18:41 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57453
-
Nikita Hovratov uploaded patch set 25
V +105/10/24, 13:16 -
Nikita Hovratov commented on patch set 25 05/10/24, 13:18
Record.php:77ReplyDoneRecordFactory.php:50ReplyDone -
core-ci voted on patch set 25 V +1 05/10/24, 13:22 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57479
-
Nikita Hovratov uploaded patch set 26
V +105/10/24, 13:31 -
Nikita Hovratov commented on patch set 25 05/10/24, 13:33
Reply
I chose to change the identifier to record-transformation and keep the class name the same. This seems more in line with current naming without "verb-style" names. E.g. database-query vs. query-database. -
core-ci voted on patch set 26 V +1 05/10/24, 13:37 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57481
-
Nikita Hovratov uploaded patch set 27
V +105/10/24, 13:48 -
Nikita Hovratov commented on patch set 27 05/10/24, 13:49
RecordFactory.php:117ReplyDone -
core-ci voted on patch set 27 V +1 05/10/24, 13:58 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57484
-
Nikita Hovratov uploaded patch set 28
V +105/10/24, 15:58 -
Nikita Hovratov commented on patch set 28 05/10/24, 15:59
Record.php:37ReplyDone -
core-ci voted on patch set 28 V +1 05/10/24, 16:04 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57486
-
Nikita Hovratov uploaded patch set 29
V +105/10/24, 16:49 -
Nikita Hovratov commented on patch set 29 05/10/24, 16:50
Record.php:119ReplyDone -
core-ci voted on patch set 29 V -1 05/10/24, 16:54 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57487
-
Nikita Hovratov uploaded patch set 30
V -105/10/24, 17:10 -
core-ci voted on patch set 30 V +1 05/10/24, 17:15 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57488
-
Andreas Kienast commented on patch set 30 CR -1 05/13/24, 06:42
RecordFactory.php:192The load of `unset()` calls clutters the code very much. IMO the same could be achieved by `$properties = array_diff_key($properties, $systemProperties);`. Unfortunately, removing the `unset()` calls apparently doesn't let RecordFactory related tests fail. -
Andreas Kienast commented on patch set 30 05/13/24, 07:13
RecordFactory.php:230When creating a new record, `tstamp` matches `crdate`. Perhaps it makes sense to compare these two values and write `lastUpdatedAt` only if there is a difference.Feature-103783-RecordTransformationDataProcessor.rst:87Perhaps this is something `TYPO3\CMS\Extbase\Utility\DebuggerUtility` could deal with in the future(tm).RecordTransformationProcessor.php:56): array -
Benni Mack uploaded patch set 31
CR -1V +105/13/24, 19:50 -
Benni Mack uploaded patch set 32 05/13/24, 19:52
-
Benni Mack commented on patch set 32 05/13/24, 19:55
RecordFactory.php:192Replythat does not work unfortunately. The systemProperties also contains the "language" key (whereas the properties have l18n_parent or l10n_parent, depending on the configuration).RecordFactory.php:230Replycurrently this only reflects the current DB state, we could add a getter later-on IMHORecordInterface.php:46Replylet's handle this in the next patch of this lineFeature-103783-RecordTransformationDataProcessor.rst:87Replyyes. good follow up to spend some love to the DebuggerUtilityRecordTransformationProcessor.php:56ReplyDone -
core-ci voted on patch set 31 V +1 05/13/24, 19:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57643
-
core-ci voted on patch set 32 V +1 05/13/24, 19:57 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57644
-
Nikita Hovratov voted on patch set 32 CR +1 V +1 05/13/24, 20:30
-
Andreas Kienast commented on patch set 32 CR -1 05/14/24, 14:04
Record.php:66Should this be `($this->systemProperties?->toArray() ?? [])` since `$this->systemProperties` is nullable? -
Nikita Hovratov uploaded patch set 33
CR +1V +105/14/24, 14:09 -
Nikita Hovratov commented on patch set 33 05/14/24, 14:09
Record.php:66ReplyDone -
core-ci voted on patch set 33 V -1 05/14/24, 14:14 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57719
-
Andreas Kienast commented on patch set 33 CR -1 05/14/24, 14:16
Record.php:66ReplyYou missed the parentheses. This will now trigger an exception: `Unsupported operand types: array + null`. -
Andreas Kienast uploaded patch set 34 05/14/24, 14:17
Copied Votes: * Code-Review-1 (copy condition: "**changekind:NO_CHANGE** OR is:MIN") * Verified-1 (copy condition: "**changekind:NO_CHANGE**") -
Andreas Kienast uploaded patch set 35
CR -1V -105/14/24, 14:18 -
Andreas Kienast commented on patch set 35 05/14/24, 14:18
Record.php:66ReplyDone -
Nikita Hovratov uploaded patch set 36 05/14/24, 14:18
-
core-ci voted on patch set 35 V +1 05/14/24, 14:24 Copied votes on follow-up patch sets have been updated: * Verified+1 has been copied to patch set 36 (copy condition: "changekind:NO_CHANGE"). Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57721
-
core-ci voted on patch set 36 V +1 05/14/24, 14:25 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57722
-
Andreas Kienast voted on patch set 36 CR +1 V +1 05/14/24, 14:35
-
Nikita Hovratov voted on patch set 36 CR +1 V +1 05/14/24, 14:39
-
Benni Mack commented on patch set 36 CR +2 V +2 05/14/24, 14:40
Thanks! -
Benni Mack merged patch set 37 05/14/24, 14:40