90405[!!!][TASK] Use Record API in List Module
Commit message
Review
-
Code review CR +2 Done
- Benni Mack +2
- Oli Bartsch +1
- Stefan Bürk +1
-
Verified V +2 Done
- Benni Mack +2
- Oli Bartsch +1
- Stefan Bürk +1
- core-ci +1 CI
Issues 1
Files 10
- typo3/sysext/backend/Classes/Controller/RecordListController.php+4 −3
- typo3/sysext/backend/Classes/RecordList/DatabaseRecordList.php29+302 −224
- typo3/sysext/backend/Classes/RecordList/ElementBrowserRecordList.php+8 −12
- typo3/sysext/backend/Classes/Routing/PreviewUriBuilder.php+5 −1
- typo3/sysext/backend/Classes/Utility/BackendUtility.php8+29 −4
- typo3/sysext/backend/Classes/View/RecordSearchBoxComponent.php+2 −1
- typo3/sysext/backend/Tests/Functional/Utility/BackendUtilityTest.php+1 −1
- typo3/sysext/core/Classes/Authentication/BackendUserAuthentication.php+7 −2
- typo3/sysext/core/Documentation/Changelog/14.0/Breaking-107356-UseRecordAPIInListModule.rst4new+63 −0
- typo3/sysext/install/Configuration/ExtensionScanner/Php/MethodCallMatcher.php2+63 −0
Review log 3448
Show 1523 earlier messages
-
Benni Mack uploaded patch set 1 08/15/25, 09:10
-
core-ci voted on patch set 1 V -1 08/15/25, 09:17 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/83443
-
Benni Mack uploaded patch set 2
V -108/15/25, 12:26 -
core-ci voted on patch set 2 V -1 08/15/25, 12:40 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/83454
-
Benni Mack uploaded patch set 3
V -108/15/25, 12:49 -
core-ci voted on patch set 3 V +1 08/15/25, 12:57 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/83456
-
Benni Mack updated the commit message in patch set 4
V +108/17/25, 20:31 -
core-ci voted on patch set 4 V +1 08/17/25, 20:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/83538
-
Garvin Hicking commented on patch set 4 08/17/25, 21:07
-
Benni Mack uploaded patch set 5
V +108/29/25, 14:08 -
Benni Mack commented on patch set 5 08/29/25, 14:10
DatabaseRecordList.php:654Replyhuge paginated list = ? - if we have 200 records, we fetch 200 full DB rows now. do you think this is too much? Record API only works with full records, if we have serious concerns on this topic, we shouldn't switch to Record API. Please elaborate. -
core-ci voted on patch set 5 V -1 08/29/25, 14:21 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/83995
-
Garvin Hicking commented on patch set 5 08/29/25, 15:06
DatabaseRecordList.php:654ReplySo currently let's say I have a recordlist that paginages a table "tx_geo_shapes". The table has columns: uid, pid, title, geojson plus various text/mm-relations (ecological data, animal populations, audience, state, postcode ...). The table has 10.000 records of every park in germany (this is a real-life project example). The shapes are stored in the DB instead of JSON files because Geospation operations are performed, and a mapserver is powered by the same table. The largest park has 50MB of shapes, the average is 5MB. Total data is 50GB. Currently, a recordlist will only fetch uid, pid and title for the list. TCA does not specify the JSON data, it is populated from outside imports (type=passthrough). Fetching 100 records and paginate through it until item 10.000 to also order/ filter by selected columns with metadata, and also export by CSV. Now if I understood this patch (sorry if I'm mistaken) now not only 10.000 x 255 bytes or so would be fetched and stored in PHP variables and transferred across the network (if DB is not local!), but 100x5MB=500 MB. And that column would not even be evaluated. Since the recordlist here is in a sysfolder with all other MM tables. displaying the recorda there would have a considerable performance / resource impact, right? -
Benni Mack commented on patch set 5 08/29/25, 15:30
DatabaseRecordList.php:654ReplyOK. Thanks for the example. - MM relations are not evaluated directly (lazy loading) and only used when the column is shown. - If you show 100 parks at once and each row has 5MB in their geojson field, then yes, you will fetch 100*5MB through the network, even if the record is passthrough. So, once we work with larger records we run into this problem already. Extbase also does '*' and if you do a listing of 100 parks in the frontend, I assume you run into the same problem that you have a large database. Either we a) should stick with arrays here for list module b) come up with a "PartialRecord" concept that allows to only include selective fields c) we can keep the DB query as before and soften the concept of the existing Record API to also work with "partial" DB rows. -
Benni Mack uploaded patch set 6
V -108/29/25, 15:40 -
core-ci voted on patch set 6 V -1 08/29/25, 15:54 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/84000
-
Garvin Hicking commented on patch set 6 08/29/25, 16:22
DatabaseRecordList.php:654ReplyIn Extbase/Frontend, I can work around this (custom queries, custom repositories). But in the Backend I couldn't. Maybe we could add an event that allows to modify "*" then, but I find it hacky. I would rather evaluate all registered TCA columns that are not pass-through (or unlisted). Then it could be "record api complete", as the extra rows would not contain so large data sets? Maybe with an event still we could filter type=json out as well....? -
Benni Mack uploaded patch set 7
V -108/29/25, 16:42 -
Benni Mack uploaded patch set 8 08/29/25, 16:53
-
Benni Mack commented on patch set 8 08/29/25, 16:56
DatabaseRecordList.php:654ReplySee my code adaptions, works now. -
core-ci voted on patch set 7 V -1 08/29/25, 17:03 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/84002
-
core-ci voted on patch set 8 V -1 08/29/25, 17:09 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/84007
-
Garvin Hicking commented on patch set 8 08/29/25, 18:16
DatabaseRecordList.php:654ReplyThanks so much. Looking good, I will try to find time to test/review this properly (also post-merge, if you can get it in sooner, not blocking).Breaking-107356-UseRecordAPIInListModule.rst:21"expect" or "require" instead of "accept" maybe? Otherwise people may think it's "array|RecordInterface"?l (yes, the "instead" already expresses that, but I'd prefer to be explicit here)Breaking-107356-UseRecordAPIInListModule.rst:45Shall we use the "xclass" buzzword here for a better "search experience" of developers?
-
Benni Mack uploaded patch set 9
V -109/01/25, 13:37 -
core-ci voted on patch set 9 V +1 09/01/25, 13:44 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/84159
-
Benni Mack commented on patch set 9 09/01/25, 13:50
Breaking-107356-UseRecordAPIInListModule.rst:21ReplyDoneBreaking-107356-UseRecordAPIInListModule.rst:45ReplyDone -
Benni Mack updated the commit message in patch set 10
V +109/01/25, 14:53 -
core-ci voted on patch set 10 V +1 09/01/25, 15:00 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/84166
-
Georg Ringer commented on patch set 10 V -1 09/01/25, 15:13
getting error now: Table "fe_groups" has capability "InternalDescription" set but the record misses the corresponding field "description" -
Benni Mack uploaded patch set 11
V +109/01/25, 15:19 -
core-ci voted on patch set 11 V +1 09/01/25, 15:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/84172
-
Georg Ringer voted on patch set 11 CR +1 V +1 09/01/25, 15:38
-
Oli Bartsch commented on patch set 11 V -1 09/02/25, 08:57
MethodCallMatcher.php:6442Adding those methods here does not work as they are reported all the time, even if called with the correct arguments. -
Benni Mack uploaded patch set 12
CR +1V +109/02/25, 09:24 -
Benni Mack commented on patch set 12 09/02/25, 09:24
MethodCallMatcher.php:6442ReplyDone -
core-ci voted on patch set 12 V +1 09/02/25, 09:32 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/84225
-
Oli Bartsch uploaded patch set 13
V +109/02/25, 10:08 -
Oli Bartsch commented on patch set 13 09/02/25, 10:11
BackendUtility.php:2982Why not use capabilities here?BackendUtility.php:2989This field might be excluded, right?BackendUtility.php:2992SortBy is added below in L:3002BackendUtility.php:2995This field might be excluded, right? -
core-ci voted on patch set 13 V +1 09/02/25, 10:16 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/84230
-
Oli Bartsch commented on patch set 13 09/02/25, 10:39
DatabaseRecordList.php:598What about DiffSourceField?DatabaseRecordList.php:806Can we just use $record->getLanguageId() here?DatabaseRecordList.php:1154Since we change the method signature, should we not directly use `Record $record` instead of `array $row`?DatabaseRecordList.php:1267IMO a real benefit would be to get rid of getProcessedValueExtra() here since we have the resolved record value already. I assume we however still have some distinctions here?DatabaseRecordList.php:1630This could also be `$record->getLanguageId()`?DatabaseRecordList.php:2096`$record->getLanguageInfo()` should be used?DatabaseRecordList.php:2201I think we should go with `RecordInterface $record` here. There is only one call passing an array, which we should adjust accordingly, reducing those `getRawRecord()->toArray()` calls as much as possible.DatabaseRecordList.php:2307We should work with the $record directly in this method.DatabaseRecordList.php:3139$record->getLanguageInfo()DatabaseRecordList.php:3140$record->getLanguageId() -
Oli Bartsch commented on patch set 13 09/02/25, 11:05
DatabaseRecordList.php:598ReplyDoneDatabaseRecordList.php:806ReplyNo, as the interface does not cover this. But we can also not just call get() since this could trigger exceptions on missing fields.DatabaseRecordList.php:1580This needs to be RecordInterface -
Oli Bartsch uploaded patch set 14
V +109/02/25, 13:21 -
Oli Bartsch commented on patch set 14 09/02/25, 13:21
DatabaseRecordList.php:1154ReplyDoneDatabaseRecordList.php:1267ReplyNot to be done in this patch i guessDatabaseRecordList.php:1580ReplyDoneDatabaseRecordList.php:1630ReplyDoneDatabaseRecordList.php:2096ReplyDoneDatabaseRecordList.php:2201ReplyDoneDatabaseRecordList.php:2307ReplyDoneDatabaseRecordList.php:3139ReplyDoneDatabaseRecordList.php:3140ReplyDoneBackendUtility.php:2982ReplyDoneBackendUtility.php:2989ReplyDoneBackendUtility.php:2992ReplyDoneBackendUtility.php:2995ReplyDone -
core-ci voted on patch set 14 V +1 09/02/25, 13:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/84251
-
Oli Bartsch voted on patch set 14 CR +1 V +1 09/02/25, 13:29
-
Stefan Bürk voted on patch set 14 CR +1 V +1 09/02/25, 13:49
-
Benni Mack commented on patch set 14 CR +2 V +2 09/02/25, 13:50
Thanks for the work here. -
Benni Mack merged patch set 15 09/02/25, 13:50
DatabaseRecordList.php:654