Review main Merged Breaking

87193[!!!][TASK] Simplify CharsetConverter

Christian Kuhn CR +2 V +2 Impossible +118 −64903 updated 11/25/24, 15:43

Commit message

This change touches a very old codebase: The CharsetConverter
(which was called differently back then) exists since Kaspers
initial commit in essentially the same form as it is now, with
little maintenance, refactorings and improvements over the years.

The class contains various "from charset x to y" and character
substitution helpers. This includes solutions for some nowadays
rather obscure character sets like the EUC family [1], which
were prevelant in asia before utf-8 became a thing. With the
vast majority of web sites being bound to utf-8 nowadays, there
are only few use cases left that justify a general framework
solution. A seldom use case are importers and exporters
that need to create something that (still) can't talk utf-8. Such
cases are seldom, and custom extensions that deal with such things
can and should establish own solutions to deal with conversions,
maybe based on 3rd party libraries that are more transparent and
better tested than the codebase delivered with CharsetConverter.

The goal of this patch is to reduce the class code to make it
more slim, more transparent, more easy to grasp, and to get rid
of things we don't need anymore.

The patch itself is the result of a straight removal chain:
* conv() is unused in core and removed
* specCharsToASCII() is always called with 'utf-8' as first
  argument since #105686, the lower level method
  utf8_char_mapping() can be used instead
* the above renders euc_char_mapping(), sb_char_mapping(),
  specCharsToASCII(), utf8_decode() and utf8_encode() unused,
  which can be removed
* the above renders protected initCharset() and initToASCII()
  unused, which can be removed together with all files within
  EXT:core/Resources/Private/Charsets/csconvtbl/

[1] https://en.wikipedia.org/wiki/Extended_Unix_Code

Review

  • Code review CR +2 Done
    • Christian Kuhn +2
    • Andreas Nedbal +1
    • Stefan Bürk +1
  • Verified V +2 Done
    • Christian Kuhn +2
    • Andreas Nedbal +1
    • Stefan Bürk +1
    • core-ci +1 CI

Issues 2

Files 43

Show 23 more files

Review log 1219

  1. Christian Kuhn uploaded patch set 1 11/22/24, 15:06

  2. core-ci voted on patch set 1 V +1 11/22/24, 15:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70047

  3. Christian Kuhn uploaded patch set 2 V +1 11/22/24, 17:32

  4. core-ci voted on patch set 2 V +1 11/22/24, 17:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70059

  5. Christian Kuhn uploaded patch set 3 V +1 11/22/24, 17:52

  6. core-ci voted on patch set 3 V +1 11/22/24, 17:58 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70062

  7. Christian Kuhn uploaded patch set 4 V +1 11/22/24, 17:58

  8. core-ci voted on patch set 4 V +1 11/22/24, 18:05 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70063

  9. Christian Kuhn rebased onto patch set 5 V +1 11/25/24, 11:28

  10. core-ci voted on patch set 5 V +1 11/25/24, 11:34 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70129

  11. Christian Kuhn uploaded patch set 6 V +1 11/25/24, 13:33

  12. Christian Kuhn updated the commit message in patch set 7 11/25/24, 13:36

  13. core-ci voted on patch set 6 V +1 11/25/24, 13:41 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70138

  14. core-ci voted on patch set 7 V +1 11/25/24, 13:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70139

  15. Andreas Nedbal voted on patch set 7 CR +1 V +1 11/25/24, 13:54

  16. Stefan Bürk voted on patch set 7 CR +1 11/25/24, 14:02

  17. Stefan Bürk voted on patch set 7 V +1 11/25/24, 14:14

  18. Christian Kuhn voted on patch set 7 CR +2 V +2 11/25/24, 15:43

  19. Christian Kuhn merged patch set 8 11/25/24, 15:43