Review main Merged

64450[TASK] Integrate server response security checks

Oliver Hader CR +2 V +2 Hard +564 −1 updated 05/12/20, 09:15

Commit message

In order to evaluate potential server misconfigurations and to reduce
the potential of security implications in general, a new HTTP response
check is integrated to "Environment Status" and the "Security" section
in the reports module.

It is evaluated whether non-standard file extensions lead to unexpected
handling on the server-side, such as `test.php.wrong` being evaluated
as PHP or `test.html.wrong` being served with `text/html` content type.

Review

  • Code review CR +2 Done
    • Oliver Hader +2
  • Verified V +2 Done
    • Oliver Hader +2
    • Benjamin Franzke +1
    • TYPO3com +1 CI

Issues 1

Files 5

Review log 2333

Show 58 earlier messages
  1. Oliver Hader uploaded patch set 1 05/10/20, 19:03

  2. TYPO3com voted on patch set 1 V -1 05/10/20, 19:14 Completed build in 10m 57s on Sun, 10 May, 07:14 PM Test Summary: 73696 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3293

  3. Oliver Hader uploaded patch set 2 05/10/20, 19:23

  4. Georg Ringer commented on patch set 2 CR -1 05/10/20, 19:25

    can we have a feature + rst for that?

    FileDeclaration.php:1

    can we have a declare strict?
  5. TYPO3com voted on patch set 2 V -1 05/10/20, 19:34 Completed build in 11m 11s on Sun, 10 May, 07:34 PM Test Summary: 6 of 74180 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3294

  6. Oliver Hader uploaded patch set 3 05/10/20, 19:46

  7. Oliver Hader commented on patch set 3 05/10/20, 19:48

    Reply Unresolved

    Sure, if YOU create one... ;-)

    FileDeclaration.php:1 Reply Unresolved

    yes, done
  8. TYPO3com voted on patch set 3 V +1 05/10/20, 19:58 Completed build in 11m 15s on Sun, 10 May, 07:58 PM Test Summary: 73696 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3295

  1. Helmut Hummel commented on patch set 3 05/10/20, 20:20

    I like the idea. One little change request and some general thoughts

    ServerResponseCheck.php:70

    typo3temp is meant to be a private directory, not accessible by the webserver. please use typo3temp/assets instead, which is the only directory in typo3temp meant to be publicly accessible

    ServerResponseCheck.php:157

    minor remark here: especially in very restricted hosting environments (but also some others, like basic auth protected sites, load balanced setups,...), webservers might not not be able to connect to the host name within the HTTP headers. This means all these checks will fail. not sure if it makes sense to make some client options configurable, or whether it might be better to do such checks in JS. But better have these checks in for now and think about improvements for these later
  2. Oliver Hader uploaded patch set 4 05/10/20, 20:51

  3. Oliver Hader commented on patch set 4 05/10/20, 20:57

    ServerResponseCheck.php:70 Reply Unresolved

    Good point, I've adjusted it to `typo3temp/assets/...`

    ServerResponseCheck.php:157 Reply Unresolved

    Also a good point which I'd like to skip for the time being - if the request fails, it's shown as a warning in the report. I also was thinking about some (additional) JavaScript approach, but was more thinking about web servers behind load-balancers or proxied servers and requests would have to take a service layer roundtrip... For sure there's more to come, e.g. testing proper `Fetch-Sec-*` headers or similar, ensuring they are not stripped by some proxy. The would just work with JavaScript requests... → later™
  4. Oliver Hader uploaded patch set 5 05/10/20, 21:04

  5. TYPO3com voted on patch set 4 V +1 05/10/20, 21:07 Completed build in 16m 02s on Sun, 10 May, 09:07 PM Test Summary: 73696 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3297

  6. TYPO3com voted on patch set 5 V +1 05/10/20, 21:21 Completed build in 17m 04s on Sun, 10 May, 09:21 PM Test Summary: 73696 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3298

  7. Benni Mack commented on patch set 5 05/11/20, 06:24

    I guess the two new files are also meant to be @internal (not part of TYPO3's core API)? I did this with almost everything in EXT:install
  8. Oliver Hader uploaded patch set 6 05/11/20, 22:04

  9. TYPO3com voted on patch set 6 V +1 05/11/20, 22:15 Completed build in 11m 38s on Mon, 11 May, 10:15 PM Test Summary: 73705 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3319

  10. Josef Glatz commented on patch set 6 CR -1 05/12/20, 03:07

    Feature-91354-IntegrateServerResponseSecurityChecks.rst:27

    There is no explicit security guidelines documentation for 10.4 or higher. according to https://docs.typo3.org/m/typo3/guide-security/master/en-us/ the security guideline was merged into TYPO3 Explained docs.
  11. Josef Glatz commented on patch set 6 CR 0 05/12/20, 03:09

    Feature-91354-IntegrateServerResponseSecurityChecks.rst:27 Reply

    Ah, I‘ve overseen something
  12. Josef Glatz commented on patch set 6 05/12/20, 03:10

    Feature-91354-IntegrateServerResponseSecurityChecks.rst:27 Reply

    Done
  13. Markus Klein commented on patch set 6 CR -1 05/12/20, 05:50

    ServerResponseCheck.php:137

    that's probably wrong. should be withExpectedContentType
  14. Oliver Hader commented on patch set 6 05/12/20, 06:54

    ServerResponseCheck.php:137 Reply Unresolved

    Yepp
  15. Oliver Hader uploaded patch set 7 05/12/20, 07:25

  16. TYPO3com voted on patch set 7 V +1 05/12/20, 07:53 Completed build in 23m 32s on Tue, 12 May, 07:53 AM Test Summary: 73705 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3326

  17. Benjamin Franzke commented on patch set 7 V -1 05/12/20, 08:01

    Nested requests to TYPO3_REQUEST_HOST will also cause deadlocks with the PHP development server (e.g. `php -S localhost:8000 -t public`). The PHP development server runs single threaded and can not handle parallel requests. Therefore, when the *current* request starts one additional request to itself, it will deadlock. With this commit it is currently impossible to open the reports module using the development server, the request never completes. Now, why is that a problem? The TYPO3 base distribution doesn't advise to use the PHP development server, but it presents it as an option: https://github.com/TYPO3/TYPO3.CMS.BaseDistribution#quickstart We should skip the checks if PHP_SAPI === 'cli-server'
  18. Oliver Hader commented on patch set 7 05/12/20, 08:10

    > Patch Set 7: Verified-1 > > Nested requests to TYPO3_REQUEST_HOST will also cause deadlocks with the PHP development server (e.g. `php -S localhost:8000 -t public`). > > The PHP development server runs single threaded and can not handle parallel requests. Therefore, when the *current* request starts one additional request to itself, it will deadlock. > > With this commit it is currently impossible to open the reports module using the development server, the request never completes. > > Now, why is that a problem? > The TYPO3 base distribution doesn't advise to use the PHP development server, but it presents it as an option: > https://github.com/TYPO3/TYPO3.CMS.BaseDistribution#quickstart > > We should skip the checks if PHP_SAPI === 'cli-server' ?!? Sounds like a totally different thing? Since this is probably the last 9.5 regular release, I'd really like to see that integrated...
  19. Oliver Hader uploaded patch set 8 05/12/20, 08:29

  20. Benjamin Franzke voted on patch set 8 V +1 05/12/20, 08:35

  21. TYPO3com voted on patch set 8 V -1 05/12/20, 09:04 Completed build in 31m 05s on Tue, 12 May, 09:04 AM Test Summary: 4 of 73871 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3327

  22. TYPO3com commented on patch set 8 05/12/20, 09:11 Completed build in 1h 09m 51s on Tue, 12 May, 09:11 AM Test Summary: 1 of 74189 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3327

  23. TYPO3com voted on patch set 8 V +1 05/12/20, 09:15 Completed build in 1h 51m 59s on Tue, 12 May, 09:15 AM Test Summary: 73705 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3327

  24. Oliver Hader voted on patch set 8 V +2 CR +2 05/12/20, 09:15

  25. Oliver Hader merged patch set 9 05/12/20, 09:15