Review main Merged

88272[TASK] Modernize JavaScript and CSS buildchain

Benjamin Franzke CR +2 V +2 Impossible +8131 −20614 updated 03/26/25, 07:37

Commit message

Use ECMAScript 2023 as target for TypeScript, rollup and esbuild,
which is understood by all supported browsers. Plus: We already
use ES2023 features anyway, so it's fine to let tools optimize for
this target.

Replace clean-css with cssnano
------------------------------

Replace clean-css with cssnano, which is a native postcss plugin
that can operate on the postcss CSS AST directly, without the need
for an own CSS parser implementation.
This means that this plugin is less likely to miss support for new
CSS functionality like container queries (which clean-css – in the
version pulled in by postcss-clean – does not yet support).

Replace html-minifier with htmlnano
-----------------------------------

Replace the ancient library minify-html-literals with litnano (htmlnano
for lit template literals), as minify-html-literals pulls in insecure
dependencies and does not allow for asynchronous css processing
(needed for cssnano usage).

Replace terser with esbuild
---------------------------

Esbuild can be used to bundle and terse dependencies in one step
and is much faster than terser. It can also replace our existing
UMD wrappers and therefore is now used as third-party bundler
for all the "simple" cases (excluding ckeditor and lit).

Also our own TypeScript buildchain now uses esbuild minification
as final minification sink.

Allow generation of (temporary) JavaScript and CSS sourcemaps
-------------------------------------------------------------

The TypeScript, SASS and thirdparty package buildchain is adapted to
support temporary generation of sourcemaps, which may become helpful for
debugging and development, now that more variables are being mangled due
to the usage of esbuild minification.

A sourcemap can be built via:

  npm run build -- --sourcemaps

  npm run build-js -- --sourcemaps
  npm run build-css -- --sourcemaps

  # Or only build sourcemaps for changed files
  npm run build -- watch --sourcemaps

Please note that the sourcemap are inlined into the generated files
and needs to be removed by rebuiling without `--sourcemaps` enabled
in order to be mergable. (CI will verify)

Asynchronous execution task
---------------------------

The TypeScript post-processing-copy task and the minification task
(previously terser, now esbuild) are merged into a new combined
asynchronous grunt task, allowing to minify lit html
literals first and apply JavaScript minification afterwards.
(Previously terser was executed first and minify-html-literals
afterwards, which worked because terser did not mangle `html`
literals without `module` mode being enabled, now that we use
esbuild, top level variable names will be mangled into shorter
variants)

The combined – and most importantly asynchronous – grunt task allows
to perform the switch to cssnano instead of clean-css,
which also requires asynchronous execution.

Miscellaneous build optimizations
---------------------------------

 * Fully parallelize CPU-bound tasks, `npm run build` is now 4x faster
   (parallelization improved `npm run build` runtime from ~40sec to
   ~18sec and the switch to esbuild caused another 9sec decrease,
   so we're at ~9sec build time)
 * Remove concurrent wrappers for io-bound copy operations.
   Thread creation creates unneeded overhead, resulting in 200-400ms
   slowdowns instead of performance gains.
 * Drop tsconfig.json processing, instead adapt the paths of dashed
   extensions to match the bare JavaScript module prefix
 * Avoid intermediate files where possible (simplifies sourcemap
   generation and avoids dozen "changed" tracked files while rebuilding)
 * Move Build/util to Build/lib and use ES6 modules (can be loaded via
   require in nodejs 22 LTS)
 * Avoid fake thirdparty module paths (mark.js, chart.js, flatpickr)
   where possible and provide custom TypeScript mappings where missing
 * Merge sass processing into the postcss grunt task to ease sourcemap
   generation and to reduce configuration boilerplate

Commands executed:
  npm remove \
    es-module-lexer \
    terser \
    grunt-npmcopy \
    grunt-sass \
    grunt-terser \
    @rollup/plugin-terser \
    rollup-plugin-postcss \
    postcss-clean \
    minify-html-literals
  npm install --save-dev \
    esbuild@^0.25 \
    rollup@^4.37 \
    rollup-plugin-esbuild@^6.2 \
    @rollup/plugin-commonjs@^28 \
    @rollup/plugin-node-resolve@^16 \
    postcss@^8.5 \
    cssnano@^7 \
    litnano@^1 \
    autoprefixer@^10.4.21 \
    @csstools/postcss-sass@^5.1
  # Update to depend on cssnano v7 instead of v6
  npm install --save-dev @ckeditor/ckeditor5-dev-utils@^46
  git mv Build/Sources/TypeScript/indexed_search \
    Build/Sources/TypeScript/indexed-search
  git mv Build/Sources/TypeScript/rte_ckeditor \
    Build/Sources/TypeScript/rte-ckeditor
  git mv Build/Sources/TypeScript/sys_note \
    Build/Sources/TypeScript/sys-note
  grep -rl clean-css Build/Sources | xargs sed -i '/clean-css/d'
  npm audit fix
  npm run build

Review

  • Code review CR +2 Done
    • Benjamin Franzke +2
    • Georg Ringer +1
    • Andreas Kienast +1
  • Verified V +2 Done
    • Benjamin Franzke +2
    • Georg Ringer +1
    • Andreas Kienast +1
    • core-ci +1 CI

Issues 2

Files 530

Show 510 more files

Review log 3759

Show 1934 earlier messages
  1. Benjamin Franzke uploaded patch set 1 02/19/25, 08:01

  2. core-ci voted on patch set 1 V +1 02/19/25, 08:07 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/74365

  3. Benjamin Franzke uploaded patch set 2 V +1 03/08/25, 06:23

  4. core-ci voted on patch set 2 V +1 03/08/25, 06:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75344

  5. Benjamin Franzke uploaded patch set 3 V +1 03/08/25, 07:01

  6. core-ci voted on patch set 3 V -1 03/08/25, 07:09 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75346

  7. Benjamin Franzke uploaded patch set 4 V -1 03/08/25, 17:09

  8. core-ci voted on patch set 4 V +1 03/08/25, 17:16 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75353

  9. Benjamin Franzke uploaded patch set 5 V +1 03/11/25, 08:12

  10. core-ci voted on patch set 5 V -1 03/11/25, 08:21 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75456

  11. Benjamin Franzke uploaded patch set 6 V -1 03/11/25, 08:29

  12. core-ci voted on patch set 6 V +1 03/11/25, 08:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75463

  13. Benjamin Franzke uploaded patch set 7 V +1 03/11/25, 10:53

  14. core-ci voted on patch set 7 V +1 03/11/25, 10:59 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75473

  15. Benni Mack updated the commit message in patch set 8 V +1 03/11/25, 13:05

  16. core-ci voted on patch set 8 V +1 03/11/25, 13:12 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75478

  17. Benjamin Franzke marked it work in progress 03/11/25, 14:05

  18. Benjamin Franzke uploaded patch set 9 V +1 03/22/25, 16:03

  19. core-ci voted on patch set 9 V +1 03/22/25, 16:11 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76032

  20. Benjamin Franzke uploaded patch set 10 V +1 03/22/25, 20:22

  21. core-ci voted on patch set 10 V +1 03/22/25, 20:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76043

  22. Benjamin Franzke uploaded patch set 11 V +1 03/22/25, 20:32

  23. core-ci voted on patch set 11 V +1 03/22/25, 20:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76044

  24. Simon Schaufelberger commented on patch set 11 03/23/25, 16:42

    TypeScript
  25. Benjamin Franzke marked it work in progress 03/23/25, 17:01

  26. Benjamin Franzke uploaded patch set 12 V +1 03/24/25, 07:44

  27. Benjamin Franzke commented on patch set 12 03/24/25, 07:44

    Reply

    Done
  28. core-ci voted on patch set 12 V -1 03/24/25, 07:52 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76127

  29. Benjamin Franzke uploaded patch set 13 V -1 03/24/25, 08:01

  30. core-ci voted on patch set 13 V +1 03/24/25, 08:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76128

  31. Benjamin Franzke uploaded patch set 14 V +1 03/24/25, 10:24

  32. core-ci voted on patch set 14 V +1 03/24/25, 10:32 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76135

  33. Benni Mack updated the commit message in patch set 15 V +1 03/24/25, 11:26

  34. core-ci voted on patch set 15 V +1 03/24/25, 11:34 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76143

  1. Andreas Kienast commented on patch set 15 V -1 03/24/25, 15:34

    Gruntfile.js:239

    Why do we need this at all? This leads to errors on my macOS machine: ``` Running "exec:rollup" (exec) task >> /bin/sh: /Users/akienast/Library/Application: No such file or directory >> Exited with code: 127. >> Error executing child process: Error: Process exited with code 127. ```
  2. Andreas Kienast commented on patch set 15 03/24/25, 15:58

    message-interface.js:13

    Can we somehow avoid these empty files at all? `import type ...` or alike?

    drag-uploader.js:14

    This does not look minified.
  3. Benjamin Franzke commented on patch set 15 03/24/25, 16:55

    Gruntfile.js:239 Reply

    ah, good to know. I wanted to assure that rollup is executed with the exact node version that is being used to execute grunt. but not needed for nvm or runTests users, so I'll revert that.

    message-interface.js:13 Reply

    yea, we could analyze the AST and see that the file is entirely empty and maybe we can skip emitting the file. Will have a look.

    drag-uploader.js:14 Reply

    It is, esbuild decided it's better to print one *real* newline (1char) instead of a psuedo newline (`\n`) which is two chars. Important: it's not a tagged lit template literal, which is minified by `litnano`, but a regular template literal postprocessed by innerHTML 😱
  4. Benjamin Franzke uploaded patch set 16 V +1 03/24/25, 17:00

  5. Benjamin Franzke commented on patch set 16 03/24/25, 17:00

    message-interface.js:13 Reply

    Done
  6. core-ci voted on patch set 16 V +1 03/24/25, 17:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76155

  7. Benjamin Franzke uploaded patch set 17 V +1 03/25/25, 04:21

  8. core-ci voted on patch set 17 V +1 03/25/25, 04:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76201

  9. Benjamin Franzke uploaded patch set 18 V +1 03/25/25, 04:44

  10. core-ci voted on patch set 18 V +1 03/25/25, 04:52 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76207

  11. Benjamin Franzke uploaded patch set 19 V +1 03/25/25, 05:35

  12. core-ci voted on patch set 19 V +1 03/25/25, 05:45 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76210

  13. Andreas Kienast voted on patch set 19 CR +1 V +1 03/25/25, 08:15

  14. Benjamin Franzke uploaded patch set 20 CR +1 V +1 03/25/25, 16:07

    Rebase
  15. core-ci voted on patch set 20 V +1 03/25/25, 16:16 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76289

  16. Benjamin Franzke uploaded patch set 21 V +1 03/25/25, 16:16

  17. Benjamin Franzke commented on patch set 21 03/25/25, 16:16

    eslint.config.mjs:35

    Removed "sourceType", we want "module", but module is default, see: https://eslint.org/docs/latest/use/configure/language-options
  18. core-ci voted on patch set 21 V +1 03/25/25, 16:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76294

  19. Andreas Kienast voted on patch set 21 CR +1 V +1 03/25/25, 19:56

  20. Benjamin Franzke uploaded patch set 22 CR +1 V +1 03/26/25, 05:25

    Rebase
  21. core-ci voted on patch set 22 V +1 03/26/25, 05:34 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76337

  22. Andreas Kienast voted on patch set 22 CR +1 V +1 03/26/25, 06:53

  23. Georg Ringer voted on patch set 22 CR +1 V +1 03/26/25, 06:59

  24. Benjamin Franzke voted on patch set 22 CR +2 V +2 03/26/25, 07:28

  25. Benjamin Franzke merged patch set 23 03/26/25, 07:37