Review main Merged

87308[TASK] Offload charset table generation

Andreas Kienast CR +2 V +2 Nightmare +2908 −392 updated 02/06/25, 12:35

Commit message

CharsetConverter previously took care of generating a charset
translation table based on definition files. This approach had
a few flaws:

* The generated cache file always based on static, immutable data
* The cache file was written into `var/charsets/`, with the effect
  that nothing prunes that file once the incoming data changes

For these reasons, the generation of said charset translation tables
is offloaded into an external script meant to be executed once the
incoming data changes.

This approch is similar to the country iso database: TYPO3 core
ships compiled data created and maintained by a build script.

Review

  • Code review CR +2 Done
    • Andreas Kienast +2
    • Stefan Bürk +1
    • Christian Kuhn +1
    • Oliver Hader +1
  • Verified V +2 Done
    • Andreas Kienast +2
    • Stefan Bürk +1
    • Christian Kuhn +1
    • core-ci +1 CI

Issues 2

Files 17

Review log 2332

Show 57 earlier messages
  1. Andreas Kienast uploaded patch set 1 12/02/24, 16:54

  2. Andreas Kienast uploaded patch set 2 12/02/24, 16:54

  3. Andreas Kienast commented on patch set 2 12/02/24, 16:59

    @lolli@schwarzbu.ch since you recently worked on the CharsetConverter, can I please have your opinion on this?
  4. core-ci voted on patch set 1 V -1 12/02/24, 17:04 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70473

  5. core-ci voted on patch set 2 V -1 12/02/24, 17:06 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70474

  6. Christian Kuhn commented on patch set 2 12/02/24, 17:31

    yep. i had a similar thing in mind 😊 i would suggest to create an (internal) DO class, similar to what we do with countries. this would be auto-loadable and we don't need resources stuff and manual require.
  7. Andreas Kienast uploaded patch set 3 V -1 12/03/24, 07:28

  1. core-ci voted on patch set 3 V -1 12/03/24, 07:35 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70503

  2. Andreas Kienast uploaded patch set 4 V -1 02/04/25, 14:04

  3. core-ci voted on patch set 4 V -1 02/04/25, 14:19 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/73497

  4. Andreas Kienast uploaded patch set 5 V -1 02/04/25, 14:34

  5. Andreas Kienast uploaded patch set 6 02/04/25, 14:34

  6. core-ci voted on patch set 6 V -1 02/04/25, 14:46 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/73503

  7. Andreas Kienast uploaded patch set 7 V -1 02/04/25, 14:57

  8. Andreas Kienast updated the commit message in patch set 8 02/04/25, 14:57

  9. core-ci voted on patch set 8 V -1 02/04/25, 15:10 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/73508

  10. Andreas Kienast uploaded patch set 9 V -1 02/04/25, 15:40

  11. core-ci voted on patch set 9 V +1 02/04/25, 15:46 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/73511

  12. Oliver Hader commented on patch set 9 02/06/25, 07:54

    I guess the GifBuilder changes are not related to CharsetConverter? Are they?
  13. Christian Kuhn commented on patch set 9 02/06/25, 08:39

    i really like this. one thing: we could/should have a CI run in 'integration' that calls the script to verify nothing changes. this prevents that someone fiddles in the data class manually.

    CharsetProvider.php:26

    do we really need the alias? this is mainly an internal service used once/very seldomly?

    CharsetProvider.php:27

    not needed in this case since the class has no further dependencies anyways.

    CharsetProvider.php:28

    final readonly would be great in this case.

    CharsetProvider.php:35

    lets make this a constant.
  14. Andreas Kienast commented on patch set 9 02/06/25, 08:44

    Reply

    Ack.

    Reply

    Yesn't. The changes in the GifBuilderTest became necessary by introducing DI to CharsetConverter.

    CharsetProvider.php:26 Reply

    Honestly, I bluntly copied this from the CountryProvider class.

    CharsetProvider.php:27 Reply

    Acknowledged

    CharsetProvider.php:28 Reply

    Acknowledged

    CharsetProvider.php:35 Reply

    Acknowledged
  15. Andreas Kienast uploaded patch set 10 V +1 02/06/25, 08:53

  16. Christian Kuhn commented on patch set 10 02/06/25, 08:57

    CharsetProvider.php:26 Reply

    Done
  17. Christian Kuhn commented on patch set 10 CR -1 02/06/25, 08:57

    on it.
  18. core-ci voted on patch set 10 V +1 02/06/25, 08:59 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/73635

  19. Christian Kuhn uploaded patch set 11 CR -1 V +1 02/06/25, 09:42

  20. core-ci voted on patch set 11 V +1 02/06/25, 09:50 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/73643

  21. Christian Kuhn commented on patch set 11 CR +1 V +1 02/06/25, 09:52

    played around with this and tuned details.
  22. Oliver Hader voted on patch set 11 CR +1 02/06/25, 10:03

  23. Stefan Bürk voted on patch set 11 CR +1 V +1 02/06/25, 11:46

  24. Andreas Kienast voted on patch set 11 CR +2 V +2 02/06/25, 12:35

  25. Andreas Kienast merged patch set 12 02/06/25, 12:35