Review main Work in progress

88148[BUGFIX] Resolve site for paths with invalid UTF-8

Georg Ringer CR +1 V +1 Normal +91 −1 updated 11/10/26, 09:56

Commit message

A request path containing percent-encoded bytes that are not valid
UTF-8, such as "%F6" (ISO-8859-1 "ö"), did not resolve any site, even
if host and base matched. The site routes are compiled with the "utf8"
option, and the Symfony URL matcher reports no match for a path that
is not valid UTF-8. The request ended up on the generic "No site
configuration found" page instead of the error handling of the site.

Such a path is a syntactically valid URI (RFC 3986), only the page
cannot be found. The invalid bytes of the path are now replaced
before matching, so site and language are resolved by host and base,
and the site renders its configured 404 page. The bytes are replaced
instead of passed on, so the page slug lookup never queries the
database with invalid UTF-8.

Test this change

Run a command in your clone of the TYPO3 Core. Then try what the change does and vote on Verified in Gerrit. How to review a patch

Check out patch set 5
git fetch https://review.typo3.org/Packages/TYPO3.CMS refs/changes/48/88148/5 && git checkout FETCH_HEAD
Cherry-pick onto main
git fetch https://review.typo3.org/Packages/TYPO3.CMS refs/changes/48/88148/5 && git cherry-pick FETCH_HEAD

Review

  • Code review CR +1 Needs a vote
    • Anton Danilov +1
  • Verified V +1 Needs a vote
    • Anton Danilov +1
    • core-ci +1 CI

Issues 2

Files 3

Review log 1116

  1. Georg Ringer uploaded patch set 1 02/10/25, 19:53

  2. Georg Ringer commented on patch set 1 02/10/25, 19:53

    This change is ready for review.
    is this the correct way to go? thanks for your eyes
  3. core-ci voted on patch set 1 V -1 02/10/25, 20:03 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/73929

  4. Oliver Hader uploaded patch set 2 V -1 02/11/25, 01:22

  5. Oliver Hader uploaded patch set 3 02/11/25, 01:23

  6. core-ci voted on patch set 2 V -1 02/11/25, 01:25 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/73951

  7. core-ci voted on patch set 3 V +1 02/11/25, 01:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/73952

  8. Oliver Hader commented on patch set 3 02/11/25, 01:31

    Alternative: Deny the request earlier, since the URI is malformed (encoded as ISO-8859-1 instead of UTF-8 for the introduced test case with `%E8`)
  9. Rémy DANIEL commented on patch set 3 12/15/25, 09:08

    Reply

    This is a good idea. What would be the error page then? Still a 404?
  10. Anton Danilov commented on patch set 3 10/20/26, 10:22

    Reply

    I'd like to help move this WIP forward: #106146 is still present on current `main`, and #111145 is the newer duplicate. On the "deny the request earlier" alternative, I looked at what the standards say and think it is the weaker option here. - `%E8` is a syntactically valid URI. RFC 3986 §2.1 defines percent-encoding as a data octet, and §2 does not mandate any particular character encoding; only its interpretation as UTF-8 fails https://www.rfc-editor.org/rfc/rfc3986#section-2.1 https://www.rfc-editor.org/rfc/rfc3986#section-2 - RFC 9110 §2.4 allows a recipient to attempt recovery of a usable protocol element from an invalid construct; answering 400 is a policy choice, not a conformance requirement https://www.rfc-editor.org/rfc/rfc9110.html#section-2.4 - These requests do reach servers in practice: WHATWG URL keeps percent-encoded bytes in the path undecoded, and §1.3 treats a failed UTF-8 decode as inadvisable rather than invalid. Where a consumer does decode with replacement, invalid sequences become U+FFFD rather than an error https://url.spec.whatwg.org/#percent-encoded-bytes https://encoding.spec.whatwg.org/#utf-8-decoder - 400 is for a perceived client error, with malformed syntax, invalid framing or deceptive routing as examples (RFC 9110 §15.5.1). Here host and base are valid and the site is identifiable; only the page path fails - that is 404 (§15.5.5), and the site owns its 404 handling. So, I'd propose to normalise the path, resolve site/language from host + base, and let `PageRouter` render the site's configured 404. If that's the right direction, I can prepare a rebased patch set.
  11. Benni Mack uploaded patch set 4 V +1 11/09/26, 20:37

  12. Benni Mack commented on patch set 4 11/09/26, 20:38

    Reply

    Anton, can you check the updated approach?
  13. Benni Mack rebased onto patch set 5 11/09/26, 20:38

  14. core-ci voted on patch set 5 V +1 11/09/26, 20:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/113782

  15. core-ci voted on patch set 4 V +1 11/09/26, 21:05 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/113781

  16. Anton Danilov commented on patch set 5 CR +1 V +1 11/10/26, 09:56

    Reply

    Thanks for moving this forward, @benni@typo3.org! This approach works for me. One thing, not blocking: the unit test asserts the site and language but never `getTail()`, and the tail is what `PageRouter` resolves the page from. Asserting the expected tail as well ("grüße/\u{FFFD}" for the last case) would cover the replacement itself, not only that a site was found.