Review main Merged

70244[TASK] Run Grunt scripts in parallel

Andreas Kienast CR +2 V +2 Normal +108 −36 updated 08/31/21, 11:06

Commit message

This patch groups Grunt tasks that can run simultaneously to improve the
build performance. To achieve this, the package `grunt-concurrent` is
used.

On a 2018 Dell XPS 15 with Intel i7-9750H this saves roughly 12 seconds.
To check the difference, the command `time grunt build` was executed
twice:

Sequential build:

  48,09s user 10,06s system 154% cpu 37,565 total
  50,60s user 11,89s system 154% cpu 40,539 total

Parallel build:

  77,91s user 5,96s system 261% cpu 28,311 total
  67,43s user 5,05s system 262% cpu 27,619 total

Due to issues with `grunt-contrib-imagemin`, the subscript `imagemin` is
excluded from the `minify_assets` group for now.

Review

  • Code review CR +2 Done
    • Tymoteusz Motylewski +2
    • Oli Bartsch +1
  • Verified V +2 Done
    • Tymoteusz Motylewski +2
    • Oli Bartsch +1
    • Jonas Eberle +1
    • core-ci +1 CI

Issues 1

Files 3

Review log 1721

  1. Andreas Kienast uploaded patch set 1 08/06/21, 09:00

  2. core-ci voted on patch set 1 V -1 08/06/21, 09:13 Core CI is not happy: https://gitlab.typo3.org/typo3/CI/cms/-/pipelines/14189

  3. Andreas Kienast uploaded patch set 2 08/06/21, 09:19

  4. core-ci voted on patch set 2 V -1 08/06/21, 09:34 Core CI is not happy: https://gitlab.typo3.org/typo3/CI/cms/-/pipelines/14191

  5. Mathias Brodala commented on patch set 2 08/06/21, 09:59

    Maybe I'm missing something but isn't this slower (user+system) than before? Also why are there 2 build lines here instead of 1? My understanding: https://stackoverflow.com/a/556411/6812729
  6. Andreas Kienast commented on patch set 2 08/06/21, 10:23

    Reply

    First of all: I've triggered two builds before and after that change to get different build times. The builds are faster, please see the "total" time at the end of each output in the commit message. The patch currently saves roughly 10 seconds per build. The execution time for "user" is higher because more CPU cores are used, the execution time is cumulated across all cores.
  7. Mathias Brodala commented on patch set 2 08/06/21, 10:26

    Reply Unresolved

    Understood, seems like I still don't understand "time" fully. ;-)
  8. Andreas Kienast commented on patch set 2 08/27/21, 14:30

    I figured out why the build fails. Since TYPO3 v10 the imagemin step fails with > Warning: Error: spawn /builds/typo3/CI/cms/Build/node_modules/optipng-bin/vendor/optipng ENOENT in file ../typo3/sysext/core/Resources/Public/Icons/Flags/AD.png Use --force to continue. but that didn't let the whole CI fail. Since these steps are run in parallel now, imagemin let's the whole group fail, leaving other files untouched.
  9. Andreas Kienast uploaded patch set 3 08/27/21, 14:46

  10. core-ci voted on patch set 3 V +1 08/27/21, 15:07 Core CI is happy: https://gitlab.typo3.org/typo3/CI/cms/-/pipelines/15732

  11. Tymoteusz Motylewski commented on patch set 3 08/27/21, 18:37

    parallel is longer than seq?
  12. Andreas Kienast commented on patch set 3 08/27/21, 18:46

    Reply

    No, please read the previous comments in this PR. The "user" time is the CPU time. As more CPUs are involved in parallel builds, it's the logical consequence the user time is higher. The important metric here is "total", which is lower than in sequential builds.
  13. Andreas Kienast uploaded patch set 4 08/30/21, 20:37

  14. core-ci voted on patch set 4 V +1 08/30/21, 20:56 Core CI is happy: https://gitlab.typo3.org/typo3/CI/cms/-/pipelines/15897

  15. Jonas Eberle voted on patch set 4 V +1 08/31/21, 06:14

  16. Tymoteusz Motylewski voted on patch set 4 V +1 CR +1 08/31/21, 10:46

  17. Tymoteusz Motylewski commented on patch set 4 08/31/21, 10:47

    tested it with: - no patch cd Build yarn install grunt build git status -> no changes in files cherry pick change yarn install grunt build git status -> no changes in files
  18. Oli Bartsch commented on patch set 4 V +1 CR +1 08/31/21, 10:54

    Before: 98.48s After: 80.50s Thanks a lot :)
  19. Tymoteusz Motylewski voted on patch set 4 V +2 CR +2 08/31/21, 11:04

  20. Tymoteusz Motylewski commented on patch set 4 08/31/21, 11:05

    Reply

    Ack
  21. Tymoteusz Motylewski merged patch set 5 08/31/21, 11:06