Review main Abandoned

72441[BUGFIX] Ensure URIs are parsed using plain POSIX/C locale

Oliver Hader CR 0 V 0 Easy +22 −1 updated 12/01/21, 22:45

Commit message

Using parse_url on unicode characters and not using a plain POSIX/C
locale might lead to negative side-effects, destroying the unicode
sequence of those special characters.

Review

  • Code review CR 0
  • Verified V 0

Issues 1

Files 2

Review log 912

  1. Oliver Hader uploaded patch set 1 12/01/21, 14:21

  2. core-ci voted on patch set 1 V -1 12/01/21, 14:24 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/20326

  3. Oliver Hader uploaded patch set 2 12/01/21, 14:37

  4. core-ci voted on patch set 2 V -1 12/01/21, 14:51 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/20327

  5. Oliver Hader uploaded patch set 3 12/01/21, 17:59

  6. Christian Kuhn commented on patch set 3 12/01/21, 18:12

    HttpUtility.php:107 Unresolved

    we have a series of issues with locales, though: first, not all systems have it at all. for instance it's missing in musl - alpine based php builds. worse, the api is a desaster and you never know if a setlocale() call worked. we have a history of issues with this when calling setlocale() in tests on various systems. and last, setlocale() is no thread-safe and changes global php state. i came to the conclusion that we need to get rid of it entirely. unsure on side effects in case we hack it in here.

    HttpUtility.php:110 Unresolved

    may fail. return of setlocale(.., '0') is not reliable.
  7. core-ci voted on patch set 3 V +1 12/01/21, 18:18 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/20350

  8. Christian Kuhn commented on patch set 3 CR -1 12/01/21, 18:42

    I'll actually need to vote this down: We're having trouble with the non-mb-safe php parse_url() function and the patch tries to work around that using non-thread-safe setlocale() which has an api that we never know it a call actually worked. I'd suggest to go away from parse_url() instead - this ancient php function does not look as if it will be fixed anywhere soon, neither does set/getlocale(). I think we should look at php ecosystem to see if there is a reliable, multibyte and thread safe, well tested, good codebase and high usage user-land implementation of parse_url() and use that instead. If not, we should write our own and hammer it with any absurd url we can possibly think of in testing.
  9. Oliver Hader commented on patch set 3 12/01/21, 20:27

    Yes, it is working around a drawback with locale settings... This change fixes a real bug, it is encapsulated in a static function and well contained. For stable branches I don't see the need to introduce a new external library, accept their new dependencies and modify our codebase.
  10. Christian Kuhn commented on patch set 3 12/01/21, 20:37

    Reply

    I'm not convinced this does not introduce new hidden side bugs due to the problematic introduction of set/getlocale(). With the inclusion in Uri.php this code will be executed for each call, much more often than any other setlocale/getlocale() call we had before that are more hidden elsewhere. This significantly increases chances for side effects which will be extremely hard to track down if a hosting hits problems for whatever reason. Why not have an own parser for this issue to *really* fix this language inability instead of hacking around a problematic language function by introducing a second even more problematic language function?
  11. Oliver Hader commented on patch set 3 12/01/21, 22:20

    Reply Unresolved

    I totally understand what you were writing, however I have a different point of view and don't look at it as such a **BIG PROBLEM**... * in TYPO3 main branch, there are 43 occurrences of `setlocale()` already * we have features like $GLOBALS['TYPO3_CONF_VARS']['SYS']['systemLocale'] which exists for a long time already * e.g. PathUtility & CommandUtility are using a very similar approach compared to my change-set * thus, if we really have a problem with setlocale(), then we have that already - even without my change... The "thread-safe" topic is a valid concern, however it requires that locales are initialized correctly (again and again). Exactly this is currently done already in the SiteResolver middleware during frontend rendering. Introducing a new library basically means * we accept and handle new dependencies to other projects (I wrote that above already) - but it would be okay, since we have to deal with and maintain symfony, doctrine and co already * we have to replace a lot of parse_url() code, and basically would consider that for TYPO3 v12 only * it still has the chance of introducing new security vulnerabilities (just briefly google for libraries that tried to implement their own interpretation of URI/URL RFCs and failed - e.g. Google Chrome vuln was a prominent one with "GURL")
  12. Oliver Hader abandoned the change 12/01/21, 22:45

    I don't want to enforce this change... let's drop it...