Review main Merged

83749[FEATURE] Introduce Extbase file upload and deletion handling

Torben Hansen CR +2 V +2 Nightmare +3973 −7 updated 09/13/24, 15:12

Commit message

This change introduces file upload handling in Extbase, aiming
to simplify and standardize the process of managing file uploads
and deletions in TYPO3 extensions with simple attributes/annotations.

Recognizing the complexity of file upload scenarios, this feature
is designed to cover direct domain model properties only. It
provides support for handling file uploads and deletions for
properties of types `FileReference`, and
`ObjectStorage<FileReference>` in non-nested domain models.

The new API uses extbase validators to validate file size,
allowed MIME types and image dimensions, ensuring the integrity
and security of uploaded files. The feature also supports
custom validators, allowing developers to extend validation logic
according to specific needs.

Review

  • Code review CR +2 Done
    • Garvin Hicking +2
    • Oliver Hader +2
    • Nikita Hovratov +1
  • Verified V +2 Done
    • Garvin Hicking +2
    • Oliver Hader +1
    • core-ci +1 CI

Issues 2

Files 33

Show 13 more files

Review log 147231

Show 127206 earlier messages
  1. Torben Hansen uploaded patch set 1 04/11/24, 16:44

  2. Torben Hansen marked it work in progress 04/11/24, 16:45

  3. core-ci voted on patch set 1 V -1 04/11/24, 16:53 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55781

  4. Torben Hansen uploaded patch set 2 V -1 04/11/24, 17:08

  5. core-ci voted on patch set 2 V +1 04/11/24, 17:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55783

  6. Torben Hansen updated the commit message in patch set 3 V +1 04/12/24, 04:44

  7. core-ci voted on patch set 3 V +1 04/12/24, 04:49 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55820

  8. Garvin Hicking uploaded patch set 4 V +1 04/12/24, 07:08

    Spelling
  9. core-ci voted on patch set 4 V +1 04/12/24, 07:14 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55834

  10. Garvin Hicking commented on patch set 4 04/12/24, 07:33

    Having this as a feature would be a HUGE help to developers and integrators. I would love having this in v13 and the Rst entry reads great. I can't comment much on the code-side of things sadly at this point. How about maybe allowing to use the "common-(image|media|file)-types" shortcut (from the TCA type=file config.allowed) in the Validators? Not sure if that should be integrated into the MimeValidator or be it's own FilenameValidator.

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:182

    I think "suffix" is the more commonly known term for this, maybe change this in the code and also here then?

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:225

    Couldn't we use a "resetValidator()" method for cases like this? Then you could optionally (depending on other arguments and action handling) for example remove .php upload restrictions if a specific frontend user is logged in and checked? Then again, one could probably use their own validator which does these checks, and not use any of the applied validators. This alternative could be mentioned in the scope of later documentation, not necessarily this changelog (but it would help for the docs, if that is mentioned to be fleshed out).
  11. Garvin Hicking commented on patch set 4 V +1 04/12/24, 08:10

    Made the initial test with Torbens example extension and it works great in all cases provided. Seeing how easy this is to implement in an extbase extension is just awesome.
  12. Torben Hansen uploaded patch set 5 V +1 04/14/24, 12:27

  13. Torben Hansen commented on patch set 4 04/14/24, 12:27

    This change is ready for review.

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:182 Reply

    Done
  14. Torben Hansen marked it ready for review 04/14/24, 12:27

  15. Torben Hansen marked it work in progress 04/14/24, 12:27

  16. core-ci voted on patch set 5 V +1 04/14/24, 12:34 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55949

  17. Garvin Hicking voted on patch set 5 V +1 04/14/24, 15:34

  18. Mathias Brodala commented on patch set 5 04/15/24, 05:41

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:67

    > ... new ... Or > ... newly introduced ... Otherwise it sounds like the attribute always was there and the reader was just not smart enough to notice it. BTW: it must be "attribute" everywhere, not "annotation".
  19. Garvin Hicking uploaded patch set 6 V +1 04/15/24, 14:38

    Refactoring
  20. core-ci voted on patch set 6 V +1 04/15/24, 14:43 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/55996

  21. Torben Hansen uploaded patch set 7 V +1 04/15/24, 17:14

  22. core-ci voted on patch set 7 V +1 04/15/24, 17:20 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56003

  23. Torben Hansen uploaded patch set 8 V +1 04/20/24, 07:39

  24. Torben Hansen commented on patch set 7 04/20/24, 07:39

    This change is ready for review.

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:67 Reply

    Done

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:225 Reply

    The .php file upload restriction must IMO always be in affect, since PHP file upload will result in possible remote code execution. But I added a `resetValidators` method now, which resets all validators and adds the `DenyPhpUploadValidator` as the only one to the array of validators.
  25. Torben Hansen marked it ready for review 04/20/24, 07:39

  26. core-ci voted on patch set 8 V +1 04/20/24, 07:43 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56382

  27. Torben Hansen uploaded patch set 9 V +1 04/20/24, 12:04

  28. Torben Hansen marked it work in progress 04/20/24, 12:05

  29. core-ci voted on patch set 9 V +1 04/20/24, 12:10 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56383

  30. Torben Hansen uploaded patch set 10 V +1 04/20/24, 13:26

  31. core-ci voted on patch set 10 V +1 04/20/24, 13:31 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56385

  32. Torben Hansen rebased onto patch set 11 V +1 04/22/24, 12:01

  33. core-ci voted on patch set 11 V +1 04/22/24, 12:08 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56488

  34. Torben Hansen updated the commit message in patch set 12 V +1 04/22/24, 12:24

  35. core-ci voted on patch set 12 V +1 04/22/24, 12:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56489

  36. Torben Hansen uploaded patch set 13 V +1 04/27/24, 11:23

  37. Torben Hansen rebased onto patch set 14 04/27/24, 11:27

  38. core-ci voted on patch set 13 V +1 04/27/24, 11:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56846

  39. core-ci voted on patch set 14 V +1 04/27/24, 11:35 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/56848

  40. Torben Hansen uploaded patch set 15 V +1 05/02/24, 17:43

  41. core-ci voted on patch set 15 V +1 05/02/24, 17:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57107

  42. Torben Hansen uploaded patch set 16 V +1 05/04/24, 10:58

  43. core-ci voted on patch set 16 V +1 05/04/24, 11:03 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57205

  44. Torben Hansen uploaded patch set 17 V +1 05/04/24, 12:42

  45. core-ci voted on patch set 17 V +1 05/04/24, 12:47 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57206

  46. Torben Hansen uploaded patch set 18 V +1 05/05/24, 05:54

  47. core-ci voted on patch set 18 V -1 05/05/24, 05:59 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57231

  48. Torben Hansen uploaded patch set 19 V -1 05/05/24, 08:50

  49. core-ci voted on patch set 19 V -1 05/05/24, 08:55 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57233

  50. Torben Hansen uploaded patch set 20 V -1 05/05/24, 09:16

  51. core-ci voted on patch set 20 V +1 05/05/24, 09:21 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57234

  52. Torben Hansen rebased onto patch set 21 V +1 05/07/24, 13:33

  53. core-ci voted on patch set 21 V +1 05/07/24, 13:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57345

  54. Torben Hansen uploaded patch set 22 V +1 05/09/24, 10:04

  55. core-ci voted on patch set 22 V +1 05/09/24, 10:10 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57443

  56. Torben Hansen uploaded patch set 23 V +1 05/11/24, 04:25

  57. core-ci voted on patch set 23 V +1 05/11/24, 04:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57510

  58. Torben Hansen uploaded patch set 24 V +1 05/11/24, 04:34

  59. core-ci voted on patch set 24 V +1 05/11/24, 04:39 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57512

  60. Torben Hansen uploaded patch set 25 V +1 05/14/24, 12:36

  61. Torben Hansen uploaded patch set 26 05/14/24, 12:38

  62. Torben Hansen uploaded patch set 27 05/14/24, 12:41

  63. core-ci voted on patch set 25 V +1 05/14/24, 12:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57708

  64. core-ci voted on patch set 26 V +1 05/14/24, 12:43 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57709

  65. core-ci voted on patch set 27 V +1 05/14/24, 12:46 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57710

  66. Torben Hansen uploaded patch set 28 V +1 05/15/24, 06:45

  67. core-ci voted on patch set 28 V +1 05/15/24, 07:39 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57762

  68. Stephan Großberndt commented on patch set 29 V +1 05/16/24, 19:55

    Published edit on patch set 28.
  69. core-ci voted on patch set 29 V +1 05/16/24, 20:00 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57833

  70. Stephan Großberndt commented on patch set 29 05/16/24, 20:12

    Thank you for the awesome addition making the life of Extbase developers so much easier.

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:182 Reply

    Changing postfix to suffix is marked as done but has not been changed. I also think suffix would be a better choice. It seems to me you do not use suffix as this might cause the impression that the file suffix, e.g. "txt" in "my_uploaded_file.txt" will get that random addition resulting in "my_uploaded_file.txt-p9dsa09ads80tgbn". Maybe adding an example here would be more helpful to make it clear "my_uploaded_file.txt" will become "my_uploaded_file-p9dsa09ads80tgbn.txt" instead? https://twominenglish.com/postfix-vs-suffix/
  71. Torben Hansen uploaded patch set 30 V +1 05/17/24, 07:43

  72. Torben Hansen commented on patch set 30 05/17/24, 07:47

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:182 Reply

    Good spot. I was pretty sure I changed this, but obviously not. Now the patch is updated to use the term `suffix` instead of `postfix`.
  73. core-ci voted on patch set 30 V +1 05/17/24, 07:48 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57854

  74. Torben Hansen uploaded patch set 31 V +1 05/17/24, 11:33

  75. core-ci voted on patch set 31 V +1 05/17/24, 11:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57863

  76. Torben Hansen uploaded patch set 32 V +1 05/18/24, 10:31

  77. core-ci voted on patch set 32 V +1 05/18/24, 10:36 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57889

  78. Torben Hansen uploaded patch set 33 V +1 05/21/24, 10:51

  79. core-ci voted on patch set 33 V +1 05/21/24, 10:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/57992

  80. Torben Hansen uploaded patch set 34 V +1 05/21/24, 14:28

  81. core-ci voted on patch set 34 V +1 05/21/24, 14:36 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58006

  82. Torben Hansen uploaded patch set 35 V +1 05/21/24, 17:14

  83. core-ci voted on patch set 35 V +1 05/21/24, 17:21 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58007

  84. Torben Hansen uploaded patch set 36 V +1 05/24/24, 11:29

  85. core-ci voted on patch set 36 V +1 05/24/24, 11:36 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58151

  86. Torben Hansen updated the commit message in patch set 37 V +1 05/25/24, 07:04

  87. core-ci voted on patch set 37 V +1 05/25/24, 07:08 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58192

  88. Torben Hansen uploaded patch set 38 V +1 05/25/24, 09:13

  89. core-ci voted on patch set 38 V +1 05/25/24, 09:18 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58198

  90. Mogens Fiebrandt commented on patch set 38 05/25/24, 14:13

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:230

    @derhansen@gmail.com Imo it would be better to make the DuplicationBehavior::RENAME as default. Assume implementing an Upload for application documents. The REPLACE could lead to strange results, when multiple users upload documents with same name.
  91. Mogens Fiebrandt commented on patch set 38 05/25/24, 14:47

    FileUpload.php:28

    I would love to have some kind of constants for default configuration of the annotation. So it would be possible to call the annotation like: #[FileUpload(FileUpload::IMAGE, [ 'uploadFolder' => '1:/user_upload/my_upload_folder/' ])]
  92. Mogens Fiebrandt commented on patch set 38 05/25/24, 22:09

    locallang.xlf:85

    There is a typo here. It must say: The file must not be larger than %s in size.
  93. Mogens Fiebrandt commented on patch set 38 05/25/24, 22:11

    locallang.xlf:85 Reply

    Or the “not” has to be deleted.
  94. Mogens Fiebrandt commented on patch set 38 05/26/24, 11:31

    ImageDimensionsValidator.php:59

    I was thinking of an PSR-14 event here to have the option to apply some crooping or resizing actions on the image. Not sure if this is the right place to do so. In fact the imageinfo would need to be fetched again afterwards. But if doing the manipulation on the image before fetching the imageinfo, you can't be sure that it's really an imagefile. What do you think about this?
  95. Mogens Fiebrandt commented on patch set 38 05/26/24, 11:35

    ImageDimensionsValidator.php:59 Reply

    This could also affect on the filesize and maybe cause problems or lead to strange behavior in the filesizevalidator.
  96. Mogens Fiebrandt commented on patch set 38 05/26/24, 11:40

    FileSizeValidator.php:54

    I was thinking of an PSR-14 event here to have the possibility to shrink the image-file-size before validating it. The fileinfo would need to be fetched again after doing so.
  97. Torben Hansen uploaded patch set 39 V +1 05/26/24, 12:08

  98. Torben Hansen commented on patch set 38 05/26/24, 12:08

    This change is ready for review.

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:230 Reply

    Agree, done.

    FileUpload.php:28 Reply

    Although this may make it easier to setup, it does not read clear, what configuration applies (e.g. file size, image size, mime types). So I would not add an additional attribute/annotation for this scenario.

    FileSizeValidator.php:54 Reply

    The same as for the `ImageDimensionsValidator`. A. validator must always only validate the object/value and not modify it.

    ImageDimensionsValidator.php:59 Reply

    A validator should only validate an object and not modify it in any way. If it is required to modify the file after upload , we have the PSR-14 event `TYPO3\CMS\Core\Resource\Event\AfterFileAddedEvent`.

    locallang.xlf:85 Reply

    Done
  99. Torben Hansen marked it ready for review 05/26/24, 12:08

  100. core-ci voted on patch set 39 V +1 05/26/24, 12:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58276

  101. Torben Hansen marked it work in progress 05/27/24, 08:34

  102. Torben Hansen uploaded patch set 40 V +1 06/08/24, 10:37

  103. core-ci voted on patch set 40 V +1 06/08/24, 10:41 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/58912

  104. Oliver Hader commented on patch set 40 06/10/24, 11:32

    MimeTypeValidator.php:52

    This seems to just check the mime-type, independent of the file extension (see `\TYPO3\CMS\Form\Mvc\Validation\MimeTypeValidator` for comparison) It has to be avoided, that e.g. `malicious.exe` with a (crafted) mime-type file-header `image/gif` is accepted. I'm not sure, whether it should be separate `FileExtensionValidator` and `FileExtensionMimeTypeValidator` (combining file-extension and mime-type). Additionally that might be an option in any of the the existing validators, e.g. `validateFileExtension` (default to `true`)
  105. Mathias Brodala commented on patch set 40 06/10/24, 12:06

    MimeTypeValidator.php:52 Reply

    But the $mimeType here is detected from the temporarily stored uploaded file, not from any HTTP header. Thus the filename doesn't matter. With your example if you upload a valid image file as "malicious.exe", then that'd be fine and only raise a few eyebrows. If you instead upload a Windows ELF file named "image.gif" then it'd be rejected.
  106. Torben Hansen uploaded patch set 41 V +1 06/15/24, 17:06

  107. Torben Hansen commented on patch set 40 06/15/24, 17:09

    This change is ready for review.

    MimeTypeValidator.php:52 Reply

    I added the additional check to the validator. So now, the given file extension is validated against a list of allowed file extensions for the detected mime type. The `MimeTypeValidator` has the validator option `ignoreFileExtensionCheck`, which detaults to `false`. If `true`, the file extension check is ignored.
  108. Torben Hansen marked it ready for review 06/15/24, 17:09

  109. Torben Hansen uploaded patch set 42 06/15/24, 17:10

  110. core-ci voted on patch set 41 V +1 06/15/24, 17:11 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59647

  111. core-ci voted on patch set 42 V +1 06/15/24, 17:14 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59648

  112. Torben Hansen uploaded patch set 43 V +1 06/22/24, 05:23

  113. core-ci voted on patch set 43 V -1 06/22/24, 05:28 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60061

  114. Torben Hansen uploaded patch set 44 V -1 06/22/24, 08:37

  115. core-ci voted on patch set 44 V +1 06/22/24, 08:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60068

  116. Torben Hansen uploaded patch set 45 V +1 06/22/24, 10:34

  117. core-ci voted on patch set 45 V -1 06/22/24, 10:41 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60070

  118. Torben Hansen uploaded patch set 46 V -1 06/22/24, 11:12

  119. core-ci voted on patch set 46 V +1 06/22/24, 11:19 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60073

  120. Torben Hansen uploaded patch set 47 V +1 06/24/24, 06:55

  121. core-ci voted on patch set 47 V +1 06/24/24, 07:00 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60158

  122. Torben Hansen uploaded patch set 48 V +1 06/29/24, 11:47

  123. core-ci voted on patch set 48 V +1 06/29/24, 11:52 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60628

  124. Torben Hansen rebased onto patch set 49 V +1 07/09/24, 13:48

  125. Torben Hansen uploaded patch set 50 07/09/24, 13:49

  126. core-ci voted on patch set 49 V +1 07/09/24, 13:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61240

  127. core-ci voted on patch set 50 V +1 07/09/24, 13:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61241

  128. Torben Hansen uploaded patch set 51 V +1 07/13/24, 10:41

  129. core-ci voted on patch set 51 V +1 07/13/24, 10:49 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61539

  130. Oliver Klee wrote 07/13/24, 11:12

    Removed cc Oliver Klee.
  131. Torben Hansen uploaded patch set 52 V +1 07/16/24, 13:24

  132. core-ci voted on patch set 52 V -1 07/16/24, 13:36 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61636

  133. Torben Hansen uploaded patch set 53 V -1 07/16/24, 16:36

  134. core-ci voted on patch set 53 V +1 07/16/24, 16:44 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61647

  135. Torben Hansen uploaded patch set 54 V +1 07/19/24, 12:13

  136. core-ci voted on patch set 54 V +1 07/19/24, 12:21 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61819

  137. Torben Hansen uploaded patch set 55 V +1 07/19/24, 13:32

  138. Torben Hansen commented on patch set 55 07/19/24, 13:36

    Reply

    The MimeType validator now also ensures, that the file extension matches. However, from the current point of view, I prefer to let extension developers manually configure allowed mimetypes for each individual property and not use the global defaults for common file types.
  139. core-ci voted on patch set 55 V +1 07/19/24, 13:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61829

  140. Torben Hansen uploaded patch set 56 V +1 07/19/24, 13:39

  141. core-ci voted on patch set 56 V +1 07/19/24, 13:44 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61830

  142. Torben Hansen uploaded patch set 57 V +1 07/22/24, 06:10

  143. core-ci voted on patch set 57 V +1 07/22/24, 06:17 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61920

  144. Torben Hansen uploaded patch set 58 V +1 07/22/24, 12:53

  145. core-ci voted on patch set 58 V +1 07/22/24, 12:58 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61931

  146. Torben Hansen uploaded patch set 59 V +1 07/24/24, 16:35

  147. core-ci voted on patch set 59 V +1 07/24/24, 16:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62117

  148. Georg Ringer voted on patch set 59 CR +1 V +1 07/26/24, 06:17

  149. Christian Kuhn commented on patch set 59 CR -1 07/26/24, 20:17

    looking forward to a chat on this.

    FileUploadConfiguration.php:48

    i'm not a fan of these DO / Service class mixups. could we separate state from service funcionality, please?

    Backend.php:562

    this change should probably be extracted as pre-patch and explained, what do you think?
  150. Torben Hansen commented on patch set 59 07/27/24, 10:11

    Reply

    I'm at DevDays next week and looking forward for discussions and work on this topic 😊
  151. Torben Hansen uploaded patch set 60 CR +1 V +1 08/02/24, 13:05

  152. Torben Hansen marked it work in progress 08/02/24, 13:06

  153. core-ci voted on patch set 60 V +1 08/02/24, 13:10 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62730

  154. Torben Hansen uploaded patch set 61 V +1 08/04/24, 06:36

  155. core-ci voted on patch set 61 V +1 08/04/24, 06:41 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62819

  156. Torben Hansen uploaded patch set 62 V +1 08/04/24, 07:58

  157. core-ci voted on patch set 62 V +1 08/04/24, 08:03 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62822

  158. Torben Hansen uploaded patch set 63 V +1 08/06/24, 09:30

  159. core-ci voted on patch set 63 V -1 08/06/24, 09:39 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62888

  160. Torben Hansen uploaded patch set 64 V -1 08/06/24, 11:57

  161. Torben Hansen commented on patch set 64 08/06/24, 11:58

    FileUploadConfiguration.php:48 Reply

    Removed the validator initialization from the constructor.

    Backend.php:562 Reply

  162. core-ci voted on patch set 64 V +1 08/06/24, 12:03 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62916

  163. Torben Hansen uploaded patch set 65 V +1 08/06/24, 17:19

  164. core-ci voted on patch set 65 V -1 08/06/24, 17:27 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/62962

  165. Torben Hansen uploaded patch set 66 V -1 08/21/24, 18:04

  166. core-ci voted on patch set 66 V -1 08/21/24, 18:12 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/63929

  167. Torben Hansen uploaded patch set 67 V -1 08/23/24, 11:22

  168. core-ci voted on patch set 67 V -1 08/23/24, 11:30 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64063

  169. Torben Hansen uploaded patch set 68 V -1 08/23/24, 12:07

  170. core-ci voted on patch set 68 V -1 08/23/24, 12:14 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64069

  171. Torben Hansen uploaded patch set 69 V -1 08/23/24, 12:20

  172. core-ci voted on patch set 69 V -1 08/23/24, 12:30 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64071

  173. Torben Hansen uploaded patch set 70 V -1 08/23/24, 12:36

  174. core-ci voted on patch set 70 V +1 08/23/24, 12:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64077

  175. Torben Hansen uploaded patch set 71 V +1 08/23/24, 12:59

  176. core-ci voted on patch set 71 V +1 08/23/24, 13:07 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64078

  177. Torben Hansen uploaded patch set 72 V +1 08/25/24, 12:21

  178. core-ci voted on patch set 72 V +1 08/25/24, 12:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64152

  179. Torben Hansen marked it ready for review 08/25/24, 12:34

  180. Mathias Brodala commented on patch set 72 08/26/24, 07:19

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:63

    IMO it would be nice if we could pass in arbitrary Extbase validation attributes/validators instead here. See e.g. Symfony's MapUploadedFile attribute: https://symfony.com/doc/current/controller.html#controller_map-uploaded-file
  181. Torben Hansen commented on patch set 72 08/26/24, 07:36

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:63 Reply

    Would be nice, but this does not cover min/max values for the uploaded file (which IMO is a must have). So lets keep it as it is (simply and only for the descibed use cases). If users need additional validations, they can add it as described in the docs.
  182. Mathias Brodala commented on patch set 72 08/26/24, 08:01

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:63 Reply

    What do you mean exactly? We have "required" which could be covered by the "NotEmpty" validator. If people try to upload multiple files for a single-file relation, one would expect only the 1st file to be mapped. Maybe check how Symfony handles this case.
  183. Torben Hansen commented on patch set 72 08/26/24, 08:16

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:63 Reply

    But for minFiles and maxFiles there is nothing we can do with the extbase validators, since here calculation is required (e.g. consideration of new file uploads and existing file relations minus file deletions). Also the FileNameValidator is always automatically added in this patch and this is a must-have in terms of security, which users may forget if this must be done manally. Do not get me wrong: I think you idea is nice and this could possibly be archieved somehow and would maybe also be better/more flexible, but the idea comes IMO way too late 3 weeks before the feature freeze, because it requires a bigger conceptial change.
  184. Mathias Brodala commented on patch set 72 08/26/24, 08:22

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:63 Reply

    Well, you marked this ready for review yesterday so you should expect more reviews and discussions incoming from now on. IMO this should not be rushed just to be ready before freeze. But in the end my vote doesn't count much as usual, so feel free to ignore it.
  185. Torben Hansen uploaded patch set 73 V +1 08/29/24, 16:08

  186. core-ci voted on patch set 73 V +1 08/29/24, 16:24 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64485

  187. Garvin Hicking uploaded patch set 74 V +1 08/30/24, 17:21

    Spelling and wording changes
  188. Garvin Hicking commented on patch set 74 V +1 08/30/24, 17:24

    I spent a few hours on this and reviewed most of it; couldn't finish reviewing tests. A few questions/ideas, a few spelling fixes, but I still feel this is quite a solid approach to making extbase file uploads WAY MORE accessible for developers (and integrators). I think it follows the existing 'extbase magic rules' and would like to stick my head out for this one. I'll continue review in the next week. I hope we could also still iron out possible scenarios and enhancements in the post-FF stabilization phase.

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:314

    We should list the arguments to that event and give an example use of what the event could be used for (see other comment about naming)

    ModifyUploadedFileTargetFilenameEvent.php:25

    How about calling this "BeforeUploadedFilenameIsPersisted" or something like that? Most core events seem to align with a Before/After naming scheme. This would make it maybe more clear that the event can be used to adjust a target filename.

    Argument.php:346

    should this be an "elseif" for clarity?

    Argument.php:348

    Do we need another instanceOf check at this place, too? To ensure that the iterated uploadedFiles indeed holds acceptable instances?

    Argument.php:352

    $exception isn't used, maybe drop the local variable here?

    FileHandlingServiceConfiguration.php:41

    Should we name this "fileUploadDeletionConfigurations" maybe, to make that connection more clear, that it is related to uploaded files? If I read the code correctly it should only allow to delete files handed over by the f:form.uploadDeleteCheckbox viewhelper. (If I read that right, security wise the signed file reference UID doesn't allow manipulation to delete any random reference UID)

    FileUploadConfiguration.php:30

    Just to be sure, this one and FileHandlingServiceConfiguration is meant as non-internal API?

    FileUploadConfiguration.php:190

    Maybe use `GeneralUtility::makeInstance(StorageRepository::class)->findByCombinedIdentifier($combinedIdentifier);` here and actually ensure the storage identifier is both matching the syntax AND exists?

    FileHandlingService.php:311

    Also add a safety check on "instanceof FileReference"?

    FileHandlingService.php:339

    Also add a safety check on "instanceof FileReference"?

    FileHandlingService.php:377

    I might have missed a spot where the clientFilename is sanitized? UploadedFile::getClientFilename() forwards sanitizing to the consumer of the method so there should be some place that maybe replaces invalid characters or path references like "../../" out of it?

    FileHandlingService.php:389

    Use named arguments here? I know many events don't use it, but for new ones we probably should follow "named arguments usage rules"

    FileHandlingService.php:395

    I feel a bit reluctant for this casual auto-creation. Do you know if the core/extbase has other scenarios of this? As an integrator I wouldn't feel too bad receiving an exception here and having to setup the storages properly... (This would also go well with the idea to check the combined identifier storage validity at the earlier place I referred to)

    UploadDeleteCheckboxViewHelper.php:43

    Should we use some kind of "_delete" identifier here to not accidentally clash with people using "delete" as some other kind of object variable?
  189. core-ci voted on patch set 74 V +1 08/30/24, 17:36 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64543

  190. Garvin Hicking commented on patch set 75 V +1 09/04/24, 12:26

    Published edit on patch set 74.
  191. core-ci voted on patch set 75 V +1 09/04/24, 12:31 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/64753

  192. Garvin Hicking commented on patch set 75 V +1 09/04/24, 12:38

    Had a talk with Torben about this (thanks!!), he'll address some (minor) changes and resolve/explain some of my comments. We're working hard make this votable before feature-freeze.
  193. Sebastian Fischer voted on patch set 75 V +1 09/05/24, 17:59

  194. Oliver Hader commented on patch set 75 09/06/24, 08:40

    Argument.php:44

    Please add type hints to `UploadedFileInterface` - e.g. `array<string, UploadedFileInterface|list<UploadedFileInterface>>`

    Argument.php:338

    `@return list<UploadedFileInterface>`

    FileUploadConfiguration.php:30 Reply

    I think, this class also might be ``` @internal Only to be used within Extbase, not part of TYPO3 Core API. ```

    FileUploadObjectStorageUploadedFile.php:28

    I guess `validation` needs to be arbitrary (support any properties, since there might be custom validator classes)? I case that's not required, it might be better to separate the attributes, avoid the nested config-array and probably get better auto-completion, e.g. ``` #[FileUpload(addRandomSuffix: true, duplicationBehavior: DuplicationBehavior::CANCEL)] #[FileUploadValidation(required: true, allowedMimeTypes: ['image/png'])] ```
  195. Oliver Hader commented on patch set 75 09/06/24, 09:00

    UploadDeleteCheckboxViewHelper.php:47

    I think, this constant should be in the `FileHandlingService` - thus, the view-helper uses a global name - not the service relying on a view-helper constant.

    UploadDeleteCheckboxViewHelper.php:96

    (side-note for me) I need to debug and test these hashes in the service class
  196. Torben Hansen uploaded patch set 76 V +1 09/07/24, 16:10

  197. Torben Hansen commented on patch set 76 09/07/24, 16:12

    Feature-103511-IntroduceExtbaseFileUploadHandling.rst:314 Reply

    I restructured this section, so the new event has a dedicated chapter, where it is described, what the event can be used to and where all evert arguments/methods are mentioned.

    ModifyUploadedFileTargetFilenameEvent.php:25 Reply

    I think, that the current name fits better. Additionally, we often use the `Modify*Event` notation in core, so I would like to keep the current event name as it is.

    Argument.php:44 Reply

    Done

    Argument.php:338 Reply

    Done

    Argument.php:346 Reply

    Done

    Argument.php:348 Reply

    The value in `$this-uploadedFiles` is taken from `\TYPO3\CMS\Core\HTTP\ServerRequest::getUploadedFiles`, which always contains an array of `UploadedFileInterface`. We therefore need no additional check at this point.

    Argument.php:352 Reply

    Done

    FileHandlingServiceConfiguration.php:41 Reply

    As disussed, I have now renamed the property and class.

    FileUploadConfiguration.php:30 Reply

    Done

    FileHandlingService.php:311 Reply

    Done

    FileHandlingService.php:339 Reply

    Done

    FileHandlingService.php:377 Reply

    Files are persisted using FAL logic (see `ResourceStorage::addUploadedFile()`), where the given filename is cleaned from not allowed chars (see `DriverInterface::sanitizeFileName()`.

    FileHandlingService.php:389 Reply

    I could not find a single PSR-14 event core usage which use named arguments, but since the docs suggest this (https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/CodingGuidelines/CodingBestPractices/NamedArguments.html#cgl-named-arguments-pcpp-value-objects), I changed it here.

    FileHandlingService.php:395 Reply

    I basically just copied this with some minor adjustments from ext:form, so yes, we have other scenarios of this. What do you think of, if we manke this configurable? I can add a new property to `FileUploadConfiguration` names `$createUploadFolderIfNotExist = true` and provide getter/setter to allow extension authors to modify this setting. If the property is false, we throw an exception, if not, we create the folder (if possible)

    FileUploadObjectStorageUploadedFile.php:28 Reply

    The validation configuration must be part of the `FileUpload` attribute/annotation, since we also add the `FileNameValidator` (which prevents PHP file upload) automatically when the attribute/annotation is used. This is IMO the most easy solution for developers and the file upload validation part will not be forgotten by mistake. Adding custom validators via the attribute/annotation is not supported in the current version (may change in later TYPO3 versions). If custom validators are required, they must ba added manually by code as shown in the RST file.

    UploadDeleteCheckboxViewHelper.php:43 Reply

    Renamed it to `@delete` and additionally replaced the public constant in the class with a fixed string.

    UploadDeleteCheckboxViewHelper.php:47 Reply

    Agree, done
  198. core-ci voted on patch set 76 V -1 09/07/24, 16:17 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65020

  199. Torben Hansen uploaded patch set 77 V -1 09/07/24, 16:23

  200. Torben Hansen uploaded patch set 78 09/07/24, 16:31

  201. core-ci voted on patch set 77 V -1 09/07/24, 16:32 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65021

  202. core-ci voted on patch set 78 V +1 09/07/24, 16:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65022

  203. Garvin Hicking commented on patch set 78 09/07/24, 18:44

    FileHandlingService.php:395 Reply

    That sounds like an interesring middle-ground. I only thought about my own implementations here, not the case where 3rd party extensions would like to store data without explicit configuration/setup. I'm unsure though what the default here would be, and if just everybody would never turn it of. Curious for other people's thoughts.
  204. Torben Hansen uploaded patch set 79 V +1 09/08/24, 17:47

  205. Torben Hansen commented on patch set 79 09/08/24, 17:48

    FileHandlingService.php:395 Reply

    Implemented the new property in `FileUploadConfiguration` which defaults to `true`.
  206. core-ci voted on patch set 79 V +1 09/08/24, 17:52 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65044

  1. Garvin Hicking uploaded patch set 80 V +1 09/09/24, 07:55

    set createUploadFolderIfNotExist and some doc changes
  2. core-ci voted on patch set 80 V +1 09/09/24, 08:00 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65061

  3. Torben Hansen uploaded patch set 81 V +1 09/09/24, 09:57

  4. core-ci voted on patch set 81 V -1 09/09/24, 10:07 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65076

  5. core-ci voted on patch set 81 V +1 09/09/24, 10:26 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65076

  6. Garvin Hicking commented on patch set 81 CR +1 V +1 09/10/24, 18:36

    I support this patch and believe it is an important step forward to make file upload in recent TYPO3 versions easier for developers. The outdated "de facto" state of using helhum/upload-example with TypeConverters (thanks so much for this) for this was always a bit cumbersome to implement for functionality one would expect to be contained within a framework like Extbase, and various implementations of this floating around made it even harder. In my opinion, with this patch, a solid groundwork is laid that should allow us to improve this in the future. The attribute/annotation allows for forward compatibility by adding more options like a `cacheOnValidationFail` key, and additional `customValidators` list of Validators FQCNs to be applied, and we can also influence default settings for that. These two issues were considered beyond scope for the initial implementation, but will allow isolated steps for iterative improvements. The patch as it is now should be non-breaking for any custom implementation of TypeConverters or manual UploadedFile handling, as it is an opt-in attribution of properties. I also firmly believe it is in sync with how Extbase consumers would expect file uploads to work in first place.
  7. Simon Schaufelberger commented on patch set 81 09/11/24, 11:38

    ClassSchema.php:237

    micro-optimization: can this be optimized like so as well? `if ($fileUploadAnnotations !== []) {`
  8. Torben Hansen commented on patch set 81 09/11/24, 13:55

    ClassSchema.php:237 Reply

    Yes, this can be changed. But I prefer to change this after this patch has been merged, because we also have the same scenario for `$validateAnnotations` in this class. If this patch gets merged, I will push a follow-up patch to replace the `count()` comparisons.
  9. Nikita Hovratov voted on patch set 81 CR +1 09/11/24, 20:42

  10. Oliver Hader commented on patch set 81 09/13/24, 09:38

    FileUploadObjectStorageUploadedFile.php:28 Reply

    Fine for me. It basically was just meant to be an idea, not a requirement 😊

    UploadDeleteCheckboxViewHelper.php:96 Reply

    Done
  11. Oliver Hader voted on patch set 81 CR +1 09/13/24, 09:51

  12. Torben Hansen uploaded patch set 82 CR +1 V +1 09/13/24, 10:39

  13. Garvin Hicking voted on patch set 82 CR +1 V +1 09/13/24, 10:51

  14. core-ci voted on patch set 82 V +1 09/13/24, 10:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65661

  15. Oliver Hader voted on patch set 82 CR +1 09/13/24, 11:15

  16. Oliver Hader commented on patch set 82 09/13/24, 13:19

    FileHandlingServiceConfiguration.php:218

    file-u-handling...
  17. Oliver Hader commented on patch set 83 CR +1 V +1 09/13/24, 13:20

    Published edit on patch set 82.
  18. Oliver Hader commented on patch set 83 09/13/24, 13:20

    FileHandlingServiceConfiguration.php:218 Reply

    Done
  19. Oliver Hader commented on patch set 83 CR +1 V +1 09/13/24, 13:21

    Impressive 😊
  20. core-ci voted on patch set 83 V +1 09/13/24, 13:27 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/65680

  21. Nikita Hovratov voted on patch set 83 CR +1 09/13/24, 13:29

  22. Garvin Hicking voted on patch set 83 CR +1 V +1 09/13/24, 14:06

  23. Oliver Hader voted on patch set 83 CR +2 09/13/24, 15:10

  24. Garvin Hicking voted on patch set 83 CR +2 V +2 09/13/24, 15:11

  25. Garvin Hicking merged patch set 84 09/13/24, 15:12