Review main Merged

53208[BUGFIX] Use mb_* methods directly instead of CharsetConverter

Benni Mack CR +2 V +2 Normal +83 −49 updated 06/20/17, 16:28

Commit message

There are several places where mb_* methods are still not used, although
they are even better to use instead of the $csConv->parseCharset() method
for instance, as it has more aliases defined already.

Same goes for "entities_to_utf8" which is a PHP user-space implementation
of html_entity_decode() dated back to Kaspers' change in 2003.

Using native PHP methods should also increase performance marginally.

Review

  • Code review CR +2
    • Benni Mack +2
    • Markus Klein +1
    • Wouter Wolters +1
    • Anja Leichsenring +1
  • Verified V +2
    • Benni Mack +2
    • Markus Klein +1
    • Anja Leichsenring +1
    • TYPO3com +1 CI

Issues 1

Files 10

Review log 2228

Show 23 earlier messages
  1. Benni Mack uploaded patch set 1 06/14/17, 12:14

  2. TYPO3com voted on patch set 1 V +1 06/14/17, 12:23 Completed build in 09m 43s on Wed, 14 Jun, 02:23 PM Test Summary: 56499 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-12569

  3. Benni Mack updated the commit message in patch set 2 06/14/17, 12:34

  1. TYPO3com voted on patch set 2 V +1 06/14/17, 12:44 Completed build in 09m 44s on Wed, 14 Jun, 02:44 PM Test Summary: 56499 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-12570

  2. Benni Mack updated the commit message in patch set 3 06/14/17, 15:53

  3. TYPO3com voted on patch set 3 V +1 06/14/17, 16:03 Completed build in 10m 18s on Wed, 14 Jun, 06:03 PM Test Summary: 56514 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-12578

  4. Georg Ringer voted on patch set 3 CR +1 V +1 06/14/17, 16:25

  5. Markus Klein commented on patch set 3 CR -1 06/14/17, 20:53

    SuggestWizardDefaultReceiver.php:177

    this uses the internal encoding, which could be different from utf8. we should specify this explicitly to avoid side effects like we had with http_build_query()
  6. Markus Klein commented on patch set 3 V -1 06/14/17, 21:03

    PageRenderer.php:2090

    do we know a reason why this protection was installed here?

    PageRenderer.php:2108

    using this instead of ->conv() is only correct if $toCharset is utf8, I opt for hardcoding it therefore.

    ContentObjectRenderer.php:2295

    is it safe to kick the synonym-functionality of parse_charset. the mb_* do not do this.

    TypoScriptFrontendController.php:4187

    IMO this is not safe. this is only true if metaCharset is indeed utf8
  7. Benni Mack commented on patch set 3 06/15/17, 04:47

    SuggestWizardDefaultReceiver.php:177 Reply

    yes, you're right!

    PageRenderer.php:2108 Reply

    ack!

    ContentObjectRenderer.php:2295 Reply

    yes. that's what I meant in the first place. mb_* do have aliases for encodings, which you can use automatically. They do not only have the same but even more than we added originally: You can get all aliases for "ascii" here which you can use transparently as well: http://php.net/manual/en/function.mb-encoding-aliases.php

    TypoScriptFrontendController.php:4187 Reply

    Yes, true, it's not 100% safe, I will remove it from this change. However, this is indeed an interesting piece of code, as the current code never uses any mb_* code but just uses "kasper-unit-tested" home-made code, which is a bad idea instead of relying to proper and wide-spread mb_* functions. but will do that separately.
  8. Benni Mack uploaded patch set 4 06/15/17, 04:56

  9. TYPO3com voted on patch set 4 V -1 06/15/17, 05:06 Completed build in 10m 34s on Thu, 15 Jun, 07:06 AM Test Summary: 36 of 56517 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-12590

  10. Markus Klein commented on patch set 3 06/15/17, 08:45

    ContentObjectRenderer.php:2295 Reply

    The docs isn't very clear about the handling of aliases and which are known.
  11. Benni Mack commented on patch set 3 06/15/17, 14:17

    ContentObjectRenderer.php:2295 Reply

    I know, that's why I tested it :)
  12. TYPO3com commented on patch set 4 06/15/17, 14:24 Completed build in 9h 38m 24s on Thu, 15 Jun, 04:24 PM Test Summary: 36 of 56517 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-12590

  13. Benni Mack uploaded patch set 5 06/15/17, 16:49

  14. TYPO3com voted on patch set 5 V +1 06/15/17, 17:00 Completed build in 10m 18s on Thu, 15 Jun, 07:00 PM Test Summary: 56514 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-12601

  15. Markus Klein commented on patch set 5 06/16/17, 08:53

    PageRenderer.php:2090

    still not sure, why we can drop that?
  16. Benni Mack commented on patch set 5 06/16/17, 09:04

    PageRenderer.php:2090 Reply

    you cannot switch the backend to sth else than UTF-8 (default since 4.5, hard-coded since 6.0 in about 500 places) - and charset is always UTF-8. Yes, there might be an extension that completely cracks up the backend with custom entrypoints (non-UTF8, and then a custom parameter for the charset other than UTF-8) but that is just plain "not according to the standards". Like somebody who still wants to run a latin1 DB - it might be possible, but we don't care about this use-case anymore.
  17. Markus Klein commented on patch set 5 CR +1 V +1 06/16/17, 09:58

    PageRenderer.php:2090 Reply

    Ah, I didn't think about the getCharSet in detail. clear then.
  18. Wouter Wolters voted on patch set 5 CR +1 06/16/17, 10:06

  19. Anja Leichsenring voted on patch set 5 CR +1 V +1 06/16/17, 10:17

  20. Benni Mack voted on patch set 5 CR +2 V +2 06/16/17, 10:19

  21. Gerrit merged patch set 5 06/16/17, 10:19

  22. Andreas Kienast commented on patch set 6 06/17/17, 11:53

    GeneralUtility.php:264

    post -1: $len is undefined.
  23. Markus Klein commented on patch set 6 06/17/17, 19:35

  24. Markus Klein commented on patch set 6 06/20/17, 16:17

    TypoScriptFrontendController.php:4175

    regression here: we lack a strtolower() here now, which was in the old parse_charset
  25. Stephan Großberndt commented on patch set 6 06/20/17, 16:28