Review main Merged

64481[TASK] Convert ckeditor4 sources from CRLF to LF in grunt npmcopy

Benjamin Franzke CR +2 V +2 Easy +18 −0 updated 05/22/20, 18:16

Commit message

Git converts CRLF to LF when plaintext files are staged. The existing
copies of the rte_ckeditor Contrib/* sources have therefore already been
converted to LF by git [1]. Initially these files had been copied as CRLF
from the ckeditor4 sources in node_modules by grunt npmcopy.

Now, when `yarn build` is executed, the copy operation is performed again,
which means the files are reverted back to CRLF. Git therefore needs to
perform the CRLF to LF conversion again. (Which itself needs to be
triggered by the developer by staging the changed files)

We do now mimic git`s autocrlf behaviour and replace CRLF by LF in the
files copied from ckeditor Contrib/* folders to prevent the files
from clobbering the `git status` or `git diff` output.

By passing `encoding: null` to the grunt.file.copy options we ensure
that binary files will be copied as is.

Also configure *.svg files to be checked out as LF on all platforms
(namely windows) like we do for other plaintext files as well.
This ensures svg files do not show up (in windows) as changed because
their original from node_modules was stored as LF.
*.patch is added as patching jquery on windows would fail otherwise.

[1] https://git-scm.com/docs/gitattributes#_end_of_line_conversion

Review

  • Code review CR +2 Done
    • Susi Moog +2
    • Andreas Kienast +1
  • Verified V +2 Done
    • Susi Moog +2
    • Andreas Kienast +1
    • TYPO3com +1 CI

Issues 1

Files 2

Review log 1827

Show 12 earlier messages
  1. Benjamin Franzke uploaded patch set 1 05/12/20, 22:32

  2. TYPO3com voted on patch set 1 V -1 05/12/20, 22:45 Completed build in 12m 55s on Tue, 12 May, 10:45 PM Test Summary: 4 of 74193 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3357

  1. TYPO3com voted on patch set 1 V +1 05/12/20, 22:57 Completed build in 11m 46s on Tue, 12 May, 10:57 PM Test Summary: 73708 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3358

  2. Josef Glatz updated the commit message in patch set 2 05/13/20, 04:40

  3. TYPO3com voted on patch set 2 V +1 05/13/20, 04:52 Completed build in 12m 09s on Wed, 13 May, 04:52 AM Test Summary: 73708 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3359

  4. Josef Glatz updated the commit message in patch set 3 05/13/20, 04:53

  5. Josef Glatz commented on patch set 3 V -1 CR -1 05/13/20, 05:54

    Tested successfully on - macOS Mojave Can't test it successfully with - Windows 10 Pro in Git Bash where I have installed nvm, yarn, node12 via "choco" Beside that I'm pretty sure that the backport works straight forward
  6. Josef Glatz commented on patch set 3 05/13/20, 06:19

    to be more clearly for my failing win 10 tests: * the patch could not be applied when running `yarn install` file * other files (e.g. svg files) are converted from LF to CRLF (typo3/sysext/rte_ckeditor/Resources/Public/JavaScript/Contrib/plugins/copyformatting/cursors/cursor-disabled.svg)
  7. Benjamin Franzke uploaded patch set 4 05/13/20, 06:34

  8. TYPO3com voted on patch set 4 V +1 05/13/20, 06:53 Completed build in 17m 55s on Wed, 13 May, 06:53 AM Test Summary: 73708 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3362

  9. Susi Moog commented on patch set 4 V -1 05/14/20, 04:46

    WIN 10, Powershell - after yarn build I still get 91 modified files that disappear when staging them.

    .gitattributes:32

    we should probably add .patch, too - currently on powershell Win10 yarn install fails if I don't manually change the jquery patch file to LF.
  10. Benjamin Franzke uploaded patch set 5 1 comment 05/14/20, 05:54

  11. TYPO3com voted on patch set 5 V -1 05/14/20, 06:07 Completed build in 13m 18s on Thu, 14 May, 06:07 AM Test Summary: 1 of 73870 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3371

  12. Josef Glatz commented on patch set 5 CR -1 05/14/20, 08:52

    With patchset 5 and the following tasks in the Windows GIT Bash (right after cherry-picking the patchset) * rm -rf Build/patches/jquery+3.4.1.patch * git checkout -- Build/patches/jquery+3.4.1.patch the command "yarn install && yarn build" command delivers me following changed files: > .../backend/Resources/Public/Fonts/FontAwesome/fontawesome-webfont.svg | 0 .../Public/JavaScript/Contrib/plugins/codesnippet/lib/highlight/LICENSE | 0 .../JavaScript/Contrib/plugins/copyformatting/cursors/cursor-disabled.svg | 0 .../Public/JavaScript/Contrib/plugins/copyformatting/cursors/cursor.svg | 0 .../Resources/Public/JavaScript/Contrib/plugins/emoji/assets/iconsall.svg | 0 typo3/sysext/t3editor/Resources/Public/JavaScript/Contrib/cm/AUTHORS | 0 typo3/sysext/t3editor/Resources/Public/JavaScript/Contrib/cm/LICENSE | 0 .../t3editor/Resources/Public/JavaScript/Contrib/cm/bin/source-highlight | 0 8 files changed, 0 insertions(+), 0 deletions(-) It looks like we are on a good way ¯\_(ツ)_/¯ right now! But I actually have no idea how we should deal with the leftover ones. - Should we leave them? - Should we add a note in the contribution documentation? - Should we add them also to .gitattributes (even if it feels bad to not having only generic rules insed the file)?
  13. Josef Glatz commented on patch set 5 05/14/20, 10:13

    > Patch Set 5: Code-Review-1 > > With patchset 5 and the following tasks in the Windows GIT Bash (right after cherry-picking the patchset) > > * rm -rf Build/patches/jquery+3.4.1.patch > * git checkout -- Build/patches/jquery+3.4.1.patch > > the command "yarn install && yarn build" command delivers me following changed files: > > > .../backend/Resources/Public/Fonts/FontAwesome/fontawesome-webfont.svg | 0 > .../Public/JavaScript/Contrib/plugins/codesnippet/lib/highlight/LICENSE | 0 > .../JavaScript/Contrib/plugins/copyformatting/cursors/cursor-disabled.svg | 0 > .../Public/JavaScript/Contrib/plugins/copyformatting/cursors/cursor.svg | 0 > .../Resources/Public/JavaScript/Contrib/plugins/emoji/assets/iconsall.svg | 0 > typo3/sysext/t3editor/Resources/Public/JavaScript/Contrib/cm/AUTHORS | 0 > typo3/sysext/t3editor/Resources/Public/JavaScript/Contrib/cm/LICENSE | 0 > .../t3editor/Resources/Public/JavaScript/Contrib/cm/bin/source-highlight | 0 > 8 files changed, 0 insertions(+), 0 deletions(-) > > > It looks like we are on a good way ¯\_(ツ)_/¯ right now! > > But I actually have no idea how we should deal with the leftover ones. > - Should we leave them? > - Should we add a note in the contribution documentation? > - Should we add them also to .gitattributes (even if it feels bad to not having only generic rules insed the file)? # Update on my previous comment (regarding Windows): If I run "git add -A" once after the procedure described in my last comment, than the mentioned changed files are no longer shown. Running `yarn && yarn build` again showing a clean working tree. With this experience I definitely would suggest to add a note in the contribution guide as there might be unforeseen problems in the future in this area.
  14. Benjamin Franzke commented on patch set 5 05/19/20, 06:16

    You need to re-checkout the SVG files on windows for GIT to respect the LF .gitattributes setting. That applies to you/us because we already checked out these files with the old gitattributes file. That means with these set of commands: rm -rf typo3/sysext/rte_ckeditor/ typo3/sysext/t3editor/ typo3/sysext/backend/ git checkout -- typo3 cd Build && yarn build git status You should currently be left with the following unchanged files on windows: $ git status HEAD detached at FETCH_HEAD Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) modified: ../typo3/sysext/rte_ckeditor/Resources/Public/JavaScript/Contrib/plugins/codesnippet/lib/highlight/LICENSE modified: ../typo3/sysext/t3editor/Resources/Public/JavaScript/Contrib/cm/AUTHORS modified: ../typo3/sysext/t3editor/Resources/Public/JavaScript/Contrib/cm/LICENSE modified: ../typo3/sysext/t3editor/Resources/Public/JavaScript/Contrib/cm/bin/source-highlight Will push a patchset to force LF for these files via gitattributes.

    .gitattributes:32 Reply Unresolved

    Now, that's a good catch, yes. Andi said it worked for him, but the jquery patch failed for me and Josef in powershell as well. But I do now understand why: Andi initially added that file and therefore had it stored in LF. While "we" checked it out as CRLF…
  15. Benjamin Franzke uploaded patch set 6 05/19/20, 06:20

  16. Benjamin Franzke commented on patch set 6 05/19/20, 06:22

    Please test this patch on windows by using a fresh checkout of TYPO3 sources (for the .gitattributes to change effect) E.g. in Git-Bash on Windows: rm -rf Build/ typo3/sysext/rte_ckeditor/ typo3/sysext/t3editor/ typo3/sysext/backend/ git checkout -- typo3/ Build/ cd Build && yarn build
  17. Andreas Kienast voted on patch set 6 V +1 CR +1 05/19/20, 06:53

  18. TYPO3com voted on patch set 6 V -1 05/19/20, 06:56 Completed build in 36m 03s on Tue, 19 May, 06:56 AM Test Summary: 5 of 73960 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3440

  19. TYPO3com commented on patch set 6 05/19/20, 07:23 Completed build in 13m 39s on Tue, 19 May, 07:23 AM Test Summary: 71752 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3441

  20. TYPO3com voted on patch set 6 V +1 05/19/20, 07:23 Completed build in 1h 39m 04s on Tue, 19 May, 07:23 AM Test Summary: 73799 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3440

  21. TYPO3com commented on patch set 6 05/19/20, 08:46 Completed build in 1h 50m 38s on Tue, 19 May, 08:46 AM Test Summary: 73799 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3441

  22. Susi Moog voted on patch set 6 V +2 CR +2 05/22/20, 14:35

  23. Susi Moog commented on patch set 6 05/22/20, 14:35

    Tested with Powershell on Windows
  24. Susi Moog merged patch set 7 05/22/20, 14:35

  25. Josef Glatz commented on patch set 7 05/22/20, 18:16

    Hey folks! Thank you windows girls and boys!