Review main Merged

90219[TASK] Enhance Extbase file upload consistency checks

Oliver Hader CR +2 V +2 Hard +271 −48 updated 08/28/25, 19:03

Commit message

The file upload handling has a `MimeTypeValidator` which internally
also performs a file extension to mime-type consistency check.
However, in case the `mimeType` validator is not explicitly set,
the consistency check won't be executed either.

This change introduces the `FileExtensionMimeTypeConsistencyValidator`
which performs a generic file extension to mime-type consistency check
(in contrast, `MimeTypeValidator` would compare the assumed mime-types
to the configured allowed mime-types additionally).

In addition the `FileExtensionValidator` is added, allowing to check
for specific file extensions. This is considered a missing feature.

Review

  • Code review CR +2 Done
    • Oliver Hader +2
    • Garvin Hicking +1
    • Torben Hansen +1
  • Verified V +2 Done
    • Oliver Hader +2
    • Garvin Hicking +1
    • Torben Hansen +1
    • core-ci +1 CI

Issues 2

Files 10

Review log 2330

Show 35 earlier messages
  1. Oliver Hader uploaded patch set 1 08/01/25, 11:28

  2. core-ci voted on patch set 1 V -1 08/01/25, 11:35 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82629

  3. Oliver Hader uploaded patch set 2 V -1 08/01/25, 11:43

  4. core-ci voted on patch set 2 V +1 08/01/25, 11:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82631

  5. Oliver Hader uploaded patch set 3 V +1 08/01/25, 12:09

  1. core-ci voted on patch set 3 V +1 08/01/25, 12:18 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82633

  2. Oliver Hader commented on patch set 3 08/01/25, 12:57

    @todo Adjust RST documentation
  3. Garvin Hicking commented on patch set 3 08/01/25, 14:01

    FileExtensionValidator.php:36

    Should we have the sum of imagefile+miscfile+docfile as allowed file extensions for convenience? Or have an option like "useSystemConfigueationAllowedFileExtensions" which can be set that populates it from there? I think it would be nice to have a shortcut here for developers to say "upload whatever is allowed in backend filelist uploads".
  4. Oliver Hader commented on patch set 3 08/01/25, 19:13

    FileExtensionValidator.php:36 Reply

    I did not see a strong demand for that. But I'm fine with adding that as option, e.g. `useStorageDefaults`.
  5. Oliver Hader uploaded patch set 4 V +1 08/01/25, 19:41

  6. Oliver Hader commented on patch set 4 08/01/25, 19:43

    This change is ready for review.

    Reply

    I'm not sure how/whether documentation should be changed to make it clear that the feature set was extended for a particular TYPO3 v13.4.x version. Docs experts, please go ahead and adjust documentation if necessary.

    FileExtensionValidator.php:36 Reply

    Done
  7. core-ci voted on patch set 4 V +1 08/01/25, 19:48 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82646

  8. Garvin Hicking commented on patch set 4 08/01/25, 19:51

    Reply

    I will take care of this. We will add a "versionchanged" notice in the docs, and the new option be highlighted+described. I'm leaving this unresolved to remind me of it so I can prepare a PR (and set it to draft to be reviewed).

    FileExtensionValidator.php:36 Reply

    Thanks! I'm working on a test matrix to review this thoroughly.
  9. Garvin Hicking commented on patch set 5 V +1 08/01/25, 20:54

    Published edit on patch set 4.
  10. core-ci voted on patch set 5 V +1 08/01/25, 21:01 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82647

  11. Garvin Hicking commented on patch set 5 V -1 08/01/25, 21:37

    I've created a test matrix (sorry...): https://docs.google.com/spreadsheets/d/1fp75LapxayDmvXf9mV7CtORQYgNkyZaqSxrExdt68Pw/edit?usp=sharing and a test extension: https://github.com/garvinhicking/extbase_upload/tree/v14-mime-matrix I have tested this also with patch https://review.typo3.org/c/Packages/TYPO3.CMS/+/90152 applied. Much of this works, however there are cases we still need to address before I test the matrix with the other three feature flag variations: #1 - "allowedFileExtensions" ============================ I had to make the "allowedFileExtensions" argument not be required, otherwise validation would always fail; I've adjusted the patch already #2 - Allowing "exe" =================== The attributes where I tried to allow "exe" upload files did not work, for example: ``` Files with FileExtensionValidator (defaults, e.g. "zip" plus "exe") #[FileUpload([ 'validation' => [ 'fileExtension' => [ 'allowedFileExtensions' => ['exe'], 'useStorageDefaults' => true ], ], 'uploadFolder' => '1:/user_upload/', ])] ``` this case yields a validator error: ``` 1754045716: The resolved media type "application/x-dosexec" is not allowed for file extension "exe". ``` I'm unsure if this is a bug in our DefaultConfiguration MIME mapping? However I would expect "exe" files to be allowed using the configuration above. #3 - Uploading "php" files now possible ======================================= With the patch(es) applied, this configuration: ``` Any file #[FileUpload([ 'validation' => [ 'maxFiles' => 1, ], 'uploadFolder' => '1:/user_upload/', ])] ``` now allows to upload .php files which was not possible before, because the defaule FileNameValidator was applied. I am unsure why this no longer works, I couldn't check this yet. Given a configuration as above, I would expect one of these outcomes: Version A: Uploading in Extbase would work as if I had set: ---------- ``` 'validation' => [ 'fileExtension' => ['useStorageDefaults' => true], ], ``` This would IMO be least breaking for expectations of developers, and has the highest security if `enforceAllowedFileExtensions` is set. It would prevent ".php" files and allow all other file types of media/misc/textfile_ext. Version B: Uploading ANY file would be allowed --------- Since there is no restriction made in the attribute configuration, all "non flagged files" should be allowed (no matter if they are in media/misc/textfile_ext). BUT: ".php" still should always be prevented as before. Version C: Breaking Change --------- We could flag a configuration as above as too dangerous, and return a validation failure in v13, and throw an exception in v14. This would obviously be a breaking change, and we would then require that people either setup 'mimeType' or the new 'fileExtension' validator. I would personally prefer Version A, would be fine with Version B. I see Version C as too impactful for v13.
  12. Garvin Hicking commented on patch set 5 08/02/25, 12:39

    Reply

    Hm, something was off with my test of variant #3 - when I wanted to xdebug it today I noticed that PHP files are in fact properly denied. I don't know what I tested there yesterday. I will setup a proper playwright test scenario for this. Sorry for that extra noise. I will also check why the 'exe' upload fails specifically, probably some IANA mime type mismatch.
  13. Garvin Hicking uploaded patch set 6 V +1 08/02/25, 18:52

  14. core-ci voted on patch set 6 V +1 08/02/25, 19:03 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82671

  15. Garvin Hicking commented on patch set 6 08/03/25, 11:32

    Reply

    I am transforming my tests to actual functional tests bere: https://review.typo3.org/c/Packages/TYPO3.CMS/+/90226 (wip)
  16. Garvin Hicking commented on patch set 6 08/03/25, 16:33

    FileExtensionMimeTypeConsistencyValidator.php:64

    This does not cover the MimeTypeGuesser that evaluates possibly needed remapping of $GLOBALS['TYPO3_CONF_VARS']['SYS']['FileInfo']['mimeTypeCompatibility']. Also, since PHP versions differently evaluate .exe mime types we sadly definitely need a compatibility layer here. I'll prepare a side-patch for it.
  17. Garvin Hicking commented on patch set 6 V +1 08/03/25, 19:25

    Reply

    Done now: https://review.typo3.org/c/Packages/TYPO3.CMS/+/90226 That patch creates tests and fixes two bugs I've faced with this patch here. Generally it also proves that this task here is IMO the way-to-go (with the additional fixes applied). It makes these assumptions: * The MIME-Type integrity validator is not allowed to be disabled in extbase. It is independent from features.security.system.enforceFileExtensionMimeTypeConsistency, and we need this safety net. * The FileExtensionValidator is optional, and adds much needed convenience. The downside is, if it is not utilized by extbase plugin developers, the default FileUpload attribute minimum configuration will allow to upload files like exe and html and everything else that is not rejected by the FileNameValidator (php/perl etc. files, denyPattern). This validator is also independent from features.security.system.enforceAllowedFileExtensions. IMO both assumptions are sane. Please check out my test and we could discuss how to disect it into a proper patch chain (proposals in my patch).
  18. Garvin Hicking uploaded patch set 7 V +1 08/18/25, 08:52

  19. core-ci voted on patch set 7 V +1 08/18/25, 09:01 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/83559

  20. Garvin Hicking commented on patch set 7 CR +1 V +1 08/18/25, 09:09

    Reply

    I've now integrated the changes of that patch, so that we can decided how/if we want the additional tests to be implemented as a follow-up. For me, this patch plus https://review.typo3.org/c/Packages/TYPO3.CMS/+/89987 and https://review.typo3.org/c/Packages/TYPO3.CMS/+/90152/2 would improve the situation a lot, and the tests I've made seem to indicate this works. ;)
  21. Torben Hansen voted on patch set 7 CR +1 V +1 08/23/25, 11:24

  22. Garvin Hicking commented on patch set 7 08/23/25, 11:43

    FileExtensionMimeTypeConsistencyValidator.php:64 Reply

    Done
  23. Oliver Hader commented on patch set 7 08/28/25, 19:02

    Reply

    Thx! 😊
  24. Oliver Hader voted on patch set 7 CR +2 V +2 08/28/25, 19:02

  25. Oliver Hader merged patch set 8 08/28/25, 19:03