Review main Merged

77131[BUGFIX] Catch any transfer exception in HTTP host header check

Oliver Hader CR +2 V +2 Easy +12 −2 updated 12/15/22, 09:57

Commit message

ServerResponseCheck triggers a HTTP host header check which is
expected to fail. The more generic TransferException is used to
catch any other failed request, not only those with 4xx or 5xx
HTTP status codes. Besides that, TLS certificates shall not be
verified, and HTTP location redirects not be followed.

Review

  • Code review CR +2 Done
    • Oliver Hader +2
    • Benjamin Franzke +1
    • André Buchmann +1
  • Verified V +2 Done
    • Oliver Hader +2
    • Benjamin Franzke +1
    • André Buchmann +1
    • core-ci +1 CI

Issues 1

Files 1

Review log 2228

Show 23 earlier messages
  1. Oliver Hader uploaded patch set 1 12/14/22, 13:31

  2. Oliver Hader commented on patch set 2 12/14/22, 13:34

    Published edit on patch set 1.
  3. core-ci voted on patch set 1 V -1 12/14/22, 13:39 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35101

  1. core-ci voted on patch set 2 V +1 12/14/22, 13:43 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35102

  2. Andreas Kienast voted on patch set 2 CR +1 12/14/22, 13:43

  3. André Buchmann voted on patch set 2 V +1 CR +1 12/14/22, 13:56

  4. Benjamin Franzke commented on patch set 2 CR -1 12/14/22, 14:01

    The issue states: > Could not resolve host: 1eae60a836.random.example.org Hm, not sure, but that error renders the test useless, since no request happened, so we do not know whether the server would have behaved correctly. My question is: Why did curl actually try to read out to 1eae60a836.random.example.org, it's only a header and the URL should have been something different. Any hints how to reproduce that?
  5. André Buchmann commented on patch set 2 12/14/22, 14:07

    Reply

    From Slack André Buchmann 5 minutes ago Can't reproduce it in my local ddev instance. Is this caused due to different curl versions? Server with the error: curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.53.1 zlib/1.2.7 libidn/1.28 libssh2/1.8.0 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz unix-sockets ddev: curl 7.74.0 (aarch64-unknown-linux-gnu) libcurl/7.74.0 OpenSSL/1.1.1n zlib/1.2.11 brotli/1.0.9 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh2/1.9.0 nghttp2/1.43.0 librtmp/2.3 Release-Date: 2020-12-09 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp Features: alt-svc AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets Oliver Hader 3 minutes ago Maybe that non existing host does resolve within DDEV… that might be related to CURL, but also related to (Docker) networking in general
  6. Oliver Hader commented on patch set 2 12/14/22, 14:31

    Reply

    Are there any specific settings in the `HTTP` section of `LocalConfiguration.php` (https://github.com/TYPO3/typo3/blob/10.4/typo3/sysext/core/Configuration/DefaultConfiguration.php#L1410-L1426)?
  7. André Buchmann commented on patch set 2 12/14/22, 14:50

    Reply

    [HTTP][cert] empty [HTTP][connect_timeout] = 10 [HTTP][proxy] empty [HTTP][ssl_key] empty [HTTP][timeout] 0 [HTTP][verify] = 1 [HTTP][version] = 1.1
  8. Benjamin Franzke commented on patch set 2 12/14/22, 15:31

    Reply

    Ok, so reading the slack messages I now see that this may also be caused due to being called from a different context, as you are using https://github.com/georgringer/t3monitoring_client (in combination with https://github.com/georgringer/t3monitoring ) …which executes the status reports in frontend context. Didn't verify it yet, just saying before we merge this too early.
  9. Oliver Hader uploaded patch set 3 CR +1 V +1 12/14/22, 17:42

  10. core-ci voted on patch set 3 V +1 12/14/22, 17:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35112

  11. Oliver Hader uploaded patch set 4 V +1 12/14/22, 18:22

  12. core-ci voted on patch set 4 V +1 12/14/22, 18:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35118

  13. Oliver Hader uploaded patch set 5 V +1 12/14/22, 20:58

  14. core-ci voted on patch set 5 V +1 12/14/22, 21:06 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35121

  15. Oliver Hader uploaded patch set 6 V +1 12/14/22, 21:39

  16. Oliver Hader commented on patch set 6 12/14/22, 21:45

    Reply

    The part that is using proper backend URIs (`/typo3/`) is handled in → https://review.typo3.org/c/Packages/TYPO3.CMS/+/77159
    Finally, please vote/test/review again. Thx a bunch!
  17. core-ci voted on patch set 6 V +1 12/14/22, 21:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35128

  18. Benjamin Franzke voted on patch set 6 CR +1 12/15/22, 06:40

  19. Benjamin Franzke voted on patch set 6 V +1 12/15/22, 07:03

  20. André Buchmann commented on patch set 6 CR -1 12/15/22, 08:24

    ServerResponseCheck.php:238

    This comment states, that the response was invalid at this point. But below there are only error messages for three conditions. Shouldn't we add a general fallback message here? Apart from that PS6 is working.
  21. Oliver Hader commented on patch set 6 12/15/22, 09:05

    ServerResponseCheck.php:238 Reply

    In case the server is configured correctly, there would be an exception earlier (caught at `TransferException`), which is the expected behavior. Anything else is an indicator, that the server processed an invalid request - and to make sure, it is really caused by the injected host header ('...example.org'), there are those checks. I think it's okay like this. What should a general fallback message contain from your point of view?
  22. André Buchmann commented on patch set 6 12/15/22, 09:23

    ServerResponseCheck.php:238 Reply

    Maybe like that? if ($serverServerName === $randomHost) { $messageParts[] = sprintf('SERVER_NAME contained unexpected "%s"', $randomHost); } if ($messageParts === []) { $messageParts[] = sprintf('Server processed the HTTP request with invalid host headers "%s"', $randomHost); } $messageQueue->addMessage( new FlashMessage( $this->wrapList($messageParts, (string)$url, self::WRAP_FLAT), 'Unexpected server response', ContextualFeedbackSeverity::ERROR ) ); Are there maybe valid cases when we reach the section behind the catch clause? Then the comment would be missleading.
  23. André Buchmann commented on patch set 6 V +1 CR +1 12/15/22, 09:33

    ServerResponseCheck.php:238 Reply

    Okay, let's leave it as is for now
  24. Oliver Hader voted on patch set 6 V +2 CR +2 12/15/22, 09:57

  25. Oliver Hader merged patch set 7 12/15/22, 09:57