Review main Merged

83713[FEATURE] Add record list download presets

Garvin Hicking CR +2 V +2 Expert +1418 −6 updated 06/13/24, 12:56

Commit message

The modal to download selected data from a record listing
(Web > List) is enhanced to display a list of configured
presets.

These presets can be defined via page TSconfig or user
TSconfig. The listed columns take personal "exclude
fields" into consideration.

A preset definition looks like:

```
mod.web_list.downloadPresets {
    pages {
        10 {
            label = Test-Preset-1
            columns = uid, title, crdate, slug
        }

        20 {
            label = LLL:EXT:myext/Resources/Private/Language/locallang.xlf:preset2.label
            columns = uid, title, slug
            identifier = some-identifier
        }
    }
}
```

Multiple presets for one table can exist, and as many
presets for as many tables as needed are possible.

Extensions can provide such preset definitions in their
`Configuration/page.tsconfig` file, or in Site Sets.

A new PSR-14 event `BeforeRecordDownloadPresetsAreDisplayedEvent`
allows to modify the list of presets as well.

A new PSR-14 event `BeforeRecordDownloadIsExecutedEvent` allows
to modify the actual download data, replacing the old
hook usage of $GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']
['TYPO3\CMS\Recordlist\RecordList\DatabaseRecordList']
['customizeCsvHeader'] and $GLOBALS['TYPO3_CONF_VARS']
['SC_OPTIONS']['TYPO3\CMS\Recordlist\RecordList\DatabaseRecordList']
['customizeCsvRow'].

These hooks have been deprecated and are scheduled for
removal with TYPO3 v14; the only public TYPO3
extension currently utilizing this hook is gridelements.

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Oli Bartsch +1
    • Garvin Hicking +1
  • Verified V +2 Done
    • Benni Mack +2
    • Oli Bartsch +1
    • Garvin Hicking +1
    • core-ci +1 CI

Issues 1

Files 17

Review log 5579

Show 3854 earlier messages
  1. Garvin Hicking uploaded patch set 1 04/09/24, 07:26

  2. core-ci voted on patch set 1 V +1 04/09/24, 07:31 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55590

  3. Jörg Bösche voted on patch set 1 CR +1 04/11/24, 13:34

  4. Chris Müller uploaded patch set 2 CR +1 V +1 04/13/24, 17:01

  5. core-ci voted on patch set 2 V +1 04/13/24, 17:06 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55916

  6. Guido Schmechel commented on patch set 2 04/14/24, 13:07

    DatabaseRecordList.php:474

    Does it make sense to outsource this to a separate function? Or is that too small-scale?

    RecordDownloadSettings.html:53

    I once tested this with a very long preset name. Presumably it makes sense to limit the number of characters in the preset using CSS? An Edge case, of course

    Feature-102337-PresetsForRecordListDownload.rst:64

    bodytext for pages? Maybe choose a different field name to avoid confusion?
  7. Garvin Hicking commented on patch set 2 04/14/24, 15:59

    (also thanks @chris for rst work!)

    DatabaseRecordList.php:474 Reply

    Good idea. Will refactor to something like "applyPresetToColumbo($columnsToRender)". Then the early returns make more sense in a dedicated method.

    RecordDownloadSettings.html:53 Reply

    I'm not to great with CSS but I'll see if an easy max-width on the select can work.

    Feature-102337-PresetsForRecordListDownload.rst:64 Reply

    Haha. A leftover from my early test cases where I checked if invalid columns work properly. Makes no sense here, good catch, thanks!
  8. Garvin Hicking uploaded patch set 3 V +1 04/15/24, 14:38

    Refactoring
  9. Garvin Hicking commented on patch set 3 04/15/24, 14:43

    DatabaseRecordList.php:474 Reply

    Done

    RecordDownloadSettings.html:53 Reply

    Looks as if just omitting "w-auto" here allows to render a large select properly...

    Feature-102337-PresetsForRecordListDownload.rst:64 Reply

    Done
  10. core-ci voted on patch set 3 V +1 04/15/24, 14:45 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55997

  11. Georg Ringer voted on patch set 3 CR +1 V +1 04/15/24, 17:30

  12. Benni Mack commented on patch set 3 04/15/24, 18:46

    looks really cool - and I have this use-case quite often. It would be also cool to have like a "processor" option (a registered service) where the data can be processed / filtered before being sent to the output.
  13. Garvin Hicking commented on patch set 3 04/15/24, 19:09

    Reply

    Thanks for the feedback! I have this ability to post-process response data on my todo-list already actually. But wanted it as a separate PR. Introducing a PSR-14 event there shouldn't be hard. If this patch gets through I can start work on it... - waiting for Oli to recover from surfing so I can ping him about it :-) Do you think a PSR-14 event for manipulating the Preset-list itself is worthwile, or is TSconfig sufficient? I'm unsure about unneccessary complexity for this feature.
  14. Benni Mack commented on patch set 3 04/15/24, 19:20

    Reply

    Yes, an Event might be even better
  15. Henrik Ziegenhain voted on patch set 3 CR +1 V +1 05/06/24, 08:38

  16. Benni Mack commented on patch set 4 CR +1 V +1 05/06/24, 14:49

    Published edit on patch set 3.
  17. core-ci voted on patch set 4 V +1 05/06/24, 14:58 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57293

  18. Garvin Hicking commented on patch set 4 05/06/24, 15:11

    Reply

    I found out there's a hook already for modifying CSV data (not the query/fields though IIRC) $GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['TYPO3\CMS\Recordlist\RecordList\DatabaseRecordList']['customizeCsvRow'] I could add a new PSR-14 event and we could add a deprecation to the hook for the next version. Also I would add a PSR-14 event to modify/add/remove presets through an event (instead of only TSConfig). Still working on it, albeit a bit short on time for the next 1-2 weeks.
  19. Garvin Hicking uploaded patch set 5 V +1 05/13/24, 12:40

    Add PSR 14 event for preset manipulation
  20. core-ci voted on patch set 5 V -1 05/13/24, 12:45 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57620

  21. Garvin Hicking uploaded patch set 6 V -1 05/13/24, 13:03

    Fix request parameter in test
  22. core-ci voted on patch set 6 V +1 05/13/24, 13:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57622

  23. Garvin Hicking uploaded patch set 7 V +1 05/13/24, 16:27

    Add another PSR14 event and deprecate old hooks
  24. core-ci voted on patch set 7 V +1 05/13/24, 16:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57634

  25. Garvin Hicking commented on patch set 7 05/13/24, 16:35

    Reply

    Thanks to being ill I could finally work on this :D Now implemented two PSR-14 events both for the data processing/download and the preset manipulation, with some tests. I've never replaced a hook with an PSR-14 event and hope this kind of deprecation (and non-removal) is ok for the v13 LTS release process.
  26. Benni Mack commented on patch set 7 06/03/24, 07:16

    In general, I would rename the Event in a way that reflects WHEN the event is fired "BeforeRecordListExportIsSent" instead of "Modify..." - Modify is what the Event Listener is doing, not the Event itself.

    RecordListDownloadController.php:89

    can be readonly

    ModifyRecordListDownloadDataEvent.php:43

    do you have an example where the request can be null?

    ModifyRecordListDownloadPresetEvent.php:37

    Can the request be null?
  27. Garvin Hicking uploaded patch set 8 V +1 06/03/24, 10:44

    apply code review suggestions
  28. Garvin Hicking commented on patch set 7 06/03/24, 10:46

    Reply

    Thanks for the idea. I just adhered to the other events that were in this directory (`ModifyRecordListHeaderColumns`, `ModifyRecordListRecordActions`, `ModifyRecordListTableActions`), but I agree that the Before/After terminology is better.

    RecordListDownloadController.php:89 Reply

    Done

    ModifyRecordListDownloadDataEvent.php:43 Reply

    You're right, this shouldn't be nullable.

    ModifyRecordListDownloadPresetEvent.php:37 Reply

    same, shouldn't be nullable.
  29. core-ci voted on patch set 8 V -1 06/03/24, 10:53 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58617

  30. Garvin Hicking uploaded patch set 9 V -1 06/03/24, 11:49

    Fix phpdoc
  31. core-ci voted on patch set 9 V +1 06/03/24, 11:53 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58619

  32. Benni Mack voted on patch set 9 V +1 06/03/24, 21:25

  33. Georg Ringer voted on patch set 9 CR +1 V +1 06/06/24, 23:40

  34. Henrik Ziegenhain voted on patch set 9 CR +1 06/07/24, 18:21

  35. Benni Mack commented on patch set 9 CR -1 06/07/24, 19:50

    one few last remarks: mod.web_list.recordListDownloadPresets -> let's call it mod.web_list.downloadPresets and -> how about one Feature.RST for the both PSR-14 Events?
  36. Garvin Hicking commented on patch set 9 06/08/24, 01:32

    Reply

    I can rename this, yes - was too afraid it may sound too broad without a restriction to the recordList, but it does look nicer when shorter. ;) About the feature rsts though IMO its easier to read when one one event is addressed, so that examples are not easily confused and the focus is clearer, I'd prefer distinct files here. Would also make later doc linking be more specific...?
  37. Benni Mack commented on patch set 9 06/09/24, 15:20

    let's keep the two RSts then. You're the docs guy!

    Reply

    let's keep the two RSts then. You're the docs guy! please rename the option though
  38. Garvin Hicking uploaded patch set 10 CR +1 V +1 06/09/24, 19:06

    Rename recordListDownloadPresets to downloadPresets
  39. Garvin Hicking commented on patch set 10 06/09/24, 19:07

    Reply

    Done
  40. core-ci voted on patch set 10 V -1 06/09/24, 19:13 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58958

  41. core-ci voted on patch set 10 V +1 06/09/24, 19:34 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58958

  42. Benni Mack voted on patch set 10 CR +1 V +1 06/09/24, 19:51

  43. Oli Bartsch commented on patch set 10 CR -1 06/10/24, 09:27

    Reviewing
  44. Garvin Hicking updated the commit message in patch set 11 CR +1 V +1 06/10/24, 12:02

    fix typo in commit msg
  45. core-ci voted on patch set 11 V +1 06/10/24, 12:08 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59003

  46. Oli Bartsch uploaded patch set 12 V +1 06/10/24, 14:40

  47. core-ci voted on patch set 12 V +1 06/10/24, 14:45 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59018

  48. Oli Bartsch uploaded patch set 13 V +1 06/10/24, 14:46

  49. core-ci voted on patch set 13 V +1 06/10/24, 14:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59019

  50. Oli Bartsch uploaded patch set 14 V +1 06/10/24, 15:42

  51. Garvin Hicking uploaded patch set 15 06/10/24, 15:51

    Use value object for preset transfer
  52. core-ci voted on patch set 15 V -1 06/10/24, 16:02 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59045

  53. core-ci voted on patch set 14 V +1 06/10/24, 16:02 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59042

  54. Garvin Hicking uploaded patch set 16 V -1 06/10/24, 16:04

    CGL
  1. core-ci voted on patch set 16 V -1 06/10/24, 16:12 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59048

  2. Oli Bartsch commented on patch set 16 CR -1 06/10/24, 16:17

    RecordListDownloadController.php:177

    Those variable assignments could be skipped.

    RecordListDownloadController.php:198

    Lets do the transformation in the event in the setPresets method and call this method in the constructor => So we ensure on creating the event, the data is correct and also when listeners update the data is correct. This should be covered with a test.

    RecordListDownloadController.php:200

    Should we include the $id as well?

    DatabaseRecordList.php:493

    Transformation to RecordListDownloadPreset is missing here

    DatabaseRecordList.php:497

    So label is used as an identifier here. I would like to use a separat logic for this. Especially because I would like to have the label being able to use locallang.

    DatabaseRecordList.php:507

    At this point it has still not been validated whether the string is a valid comma-separated list

    RecordListDownloadPreset.php:22

    Let's call this just DownloadPreset

    RecordListDownloadPreset.php:24

    The properties should not be optional (having a default value)

    RecordListDownloadPreset.php:26

    This should be an array

    RecordListDownloadPreset.php:48

    Having no label should trigger an error as we currently need it for display and also for evaluation in the getColumnsToRender() method.

    RecordListDownloadPreset.php:82

    Let's use instanceof.

    RecordDownloadSettings.html:53

    This should be translateable

    RecordDownloadSettings.html:57

    Please remove this as we should not force devs to add presets for every table. Also editors (which see this message) are usually not able to handle this and might not understand the message at all => just leading to confusion.
  3. Garvin Hicking commented on patch set 16 06/10/24, 16:38

    Thank you for the very detailed feedback, Oliver. I'll address this, but it'll take some time. For some of them I gave some feedback, everything else I completely agree on.

    RecordListDownloadController.php:200 Reply

    I was thinking this would be included in `request`, but you're right, for convenience this would be good to have.

    RecordListDownloadPreset.php:26 Reply

    The later processing of selected columns is all based on strings... so that would IMO warrant a pre-patch but also possibly break some things. Or would it be okay for you to make the getter return the imploded array then? Also it would require first splitting up the whole typoscript notation into an array too when fromArray() is called...?

    RecordListDownloadPreset.php:48 Reply

    The "label" here is only "label of the preset". The column labels are assigned at another place later on.

    RecordListDownloadPreset.php:82 Reply

    I didn't find places in the TYPO3 core making use of "instanceof" (outside Tests), it's all "is_a" everywhere, still change this?

    RecordDownloadSettings.html:57 Reply

    I see your point, and I thought a bit about this too. You found the easter egg. ;-) What I often find hard is to make integrators aware of the possibility they have here. Without any kind of indicator, nobody will probably really learn to utilize that (let's face it, many integrators don't read changelogs...). Could we find some middle ground for this? A small icon/tooltip? An empty dropdown saing "no presets defined" or something like that? Anything that would hint people at "oh, here be dragons^W presets".
  4. Oli Bartsch commented on patch set 16 06/10/24, 17:23

    RecordListDownloadPreset.php:26 Reply

    Could you show where we currently handle the columns as string? AFAICS do all usages in RecordListDownloadController and DownloadRecordList use the `columnsToRender` as array of column names.

    RecordListDownloadPreset.php:43

    This should only take one "preset" and return itself. Creating multiple DownloadPreset instances should be done by the calling code by iterating over such array an call formArray() for each preset separately. Thats's how a factory method should work.

    RecordListDownloadPreset.php:48 Reply

    The label is currently compared in the `getColumnsToRender()` method to get the currently selected preset: `if (($presetData['label'] ?? '') === $selectedPreset) {`. Additionally we need some kind of identifier to allow overwriting presets. Relying on the numeric array index is not reliable.

    RecordListDownloadPreset.php:75

    This should be done in the BeforeRecordDownloadPresetsAreDisplayedEvent. This should just be the value object.

    RecordListDownloadPreset.php:82 Reply

    Actually I find both "variants". I would go with instanceof but would not force this.

    RecordDownloadSettings.html:57 Reply

    As discussed in slack, please just remove this, integrators should read changelogs :)
  5. Garvin Hicking commented on patch set 16 06/10/24, 17:40

    RecordListDownloadPreset.php:26 Reply

    You are right, sorry; I misremembered this and confused it with some other area I was working on. So yes, let's go with consistent columns as array.

    RecordListDownloadPreset.php:82 Reply

    All good then, if you found those I didn't look well enough. I find instanceof better readable but thought I would break convention. Let's go for instanceof.

    RecordDownloadSettings.html:57 Reply

    Just to state that here too, yes, I agree on the point of consistency here, it can be removed (may need acceptance test adaptation, unsure)
  6. Garvin Hicking uploaded patch set 17 CR -1 V -1 06/11/24, 06:16

    Rename RecordListDownloadPreset to DownloadPreset
  7. core-ci voted on patch set 17 V -1 06/11/24, 06:20 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59109

  8. Garvin Hicking uploaded patch set 18 V -1 06/11/24, 08:08

    Code review
  9. Garvin Hicking commented on patch set 16 06/11/24, 08:09

    RecordListDownloadController.php:177 Reply

    Done

    RecordListDownloadController.php:198 Reply

    Tried to address this, can you check if that's suitable like that. Test needs to be added.

    RecordListDownloadController.php:200 Reply

    Added now.

    DatabaseRecordList.php:493 Reply

    Done

    DatabaseRecordList.php:497 Reply

    Note done yet, could you maybe address this? Maybe we could use the typoscript array notations (10, 20, 30...) to provide the identifier? Or a new subkey "identifier" next to "label" and "columns" would be needed, but then together with the 10/20/30... that would feel kind of bloaty in Typoscript definition?

    DatabaseRecordList.php:507 Reply

    Hope this is addressed now with the constructor set+fromArray logic?

    RecordListDownloadPreset.php:22 Reply

    Done

    RecordListDownloadPreset.php:24 Reply

    Done

    RecordListDownloadPreset.php:26 Reply

    Done

    RecordListDownloadPreset.php:43 Reply

    Needs addressing, maybe you can help me here?

    RecordListDownloadPreset.php:48 Reply

    (see note above about identifier/label, marking this as resolved for better overview)

    RecordListDownloadPreset.php:82 Reply

    Done

    RecordDownloadSettings.html:53 Reply

    Still needs addressing; maybe sufficient to use <f:translate> here and let it fallback to a string?!

    RecordDownloadSettings.html:57 Reply

    Done
  10. Garvin Hicking updated the commit message in patch set 19 06/11/24, 08:10

    commit message keeps popping up
  11. core-ci voted on patch set 18 V -1 06/11/24, 08:15 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59127

  12. core-ci voted on patch set 19 V -1 06/11/24, 08:17 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59128

  13. Garvin Hicking uploaded patch set 20 V -1 06/11/24, 08:52

    Use named arrays
  14. core-ci voted on patch set 20 V +1 06/11/24, 08:58 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59138

  15. Oli Bartsch uploaded patch set 21 V +1 06/12/24, 13:42

  16. Oli Bartsch uploaded patch set 22 06/12/24, 13:48

  17. core-ci voted on patch set 21 V -1 06/12/24, 13:52 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59332

  18. core-ci voted on patch set 22 V -1 06/12/24, 13:55 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59333

  19. Oli Bartsch uploaded patch set 23 V -1 06/12/24, 13:56

  20. core-ci voted on patch set 23 V +1 06/12/24, 14:01 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59334

  21. Oli Bartsch commented on patch set 23 CR +1 V +1 06/12/24, 14:12

    Reply

    Done

    RecordListDownloadController.php:198 Reply

    Done

    DatabaseRecordList.php:497 Reply

    Done

    RecordListDownloadPreset.php:43 Reply

    Done

    RecordListDownloadPreset.php:75 Reply

    Done

    RecordDownloadSettings.html:53 Reply

    Done
  22. Garvin Hicking commented on patch set 23 06/12/24, 14:41

    A huge thank you to you, Oliver. Now I grasp what you hinted at, and it is of course much cleaner. Tested the current version and works well for me.
  23. Garvin Hicking voted on patch set 23 CR +1 V +1 06/12/24, 14:53

  24. Benni Mack voted on patch set 23 CR +2 V +2 06/13/24, 12:56

  25. Benni Mack merged patch set 24 06/13/24, 12:56