Review main Merged

65187[BUGFIX] Process cropped images only once

Helmut Hummel CR +2 V +2 Normal +79 −86 updated 12/21/20, 16:08

Commit message

To avoid loss of quality and spawning unnecessary imagemagick
processes, cropping and scaling of images is now done
with a single imagemagick process.

By doing so, the code for SVG processing is streamlined.
SVG processing code can further be improved later,
by putting it into a dedicated file processing task processor.

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Markus Klein +1
  • Verified V +2 Done
    • Benni Mack +2
    • Markus Klein +1
    • TYPO3com +1 CI

Issues 1

Files 2

Review log 4962

Show 2537 earlier messages
  1. Helmut Hummel uploaded patch set 1 08/05/20, 11:24

  2. TYPO3com voted on patch set 1 V -1 08/05/20, 11:24 Completed build in 24s on Wed, 5 Aug, 11:24 AM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4420

  3. Helmut Hummel uploaded patch set 2 08/05/20, 11:26

  4. TYPO3com voted on patch set 2 V -1 08/05/20, 11:37 Completed build in 11m 06s on Wed, 5 Aug, 11:37 AM Test Summary: 72957 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4421

  5. Helmut Hummel uploaded patch set 3 08/05/20, 11:58

  6. TYPO3com voted on patch set 3 V +1 08/05/20, 12:09 Completed build in 11m 32s on Wed, 5 Aug, 12:09 PM Test Summary: 72957 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4422

  7. Helmut Hummel uploaded patch set 4 08/05/20, 15:03

  8. TYPO3com voted on patch set 4 V +1 08/05/20, 15:15 Completed build in 11m 31s on Wed, 5 Aug, 03:14 PM Test Summary: 72957 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4425

  9. Andreas Kienast voted on patch set 4 CR +1 08/07/20, 09:44

  10. Oliver Klee commented on patch set 4 08/07/20, 10:13

    Unresolved

    Should this be "spawning" instead of "spanning"?
  11. Oliver Klee commented on patch set 4 08/07/20, 10:14

    GraphicalFunctions.php:2133

    As this is a boolean condition, would false as a fall-back value make more sense?
  12. Andreas Kienast rebased onto patch set 5 08/07/20, 12:05

  13. Andreas Kienast voted on patch set 5 CR +1 08/07/20, 12:06

  14. Andreas Kienast commented on patch set 5 08/07/20, 12:09

    GraphicalFunctions.php:2133 Reply Unresolved

    No, because $options['crop'] is an object. If we'd change this, it should be `($options['crop'] ?? null) instanceof Area`.
  15. Andreas Kienast updated the commit message in patch set 6 08/07/20, 12:10

  16. TYPO3com voted on patch set 5 V -1 08/07/20, 12:22 Completed build in 15m 57s on Fri, 7 Aug, 12:22 PM Test Summary: 9 of 73467 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4472

  17. TYPO3com voted on patch set 6 V +1 08/07/20, 12:29 Completed build in 13m 49s on Fri, 7 Aug, 12:29 PM Test Summary: 72983 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4475

  18. Andreas Kienast uploaded patch set 7 08/07/20, 12:36

  19. Oliver Klee wrote 08/07/20, 12:40

    Removed reviewer Oliver Klee.
  20. TYPO3com voted on patch set 7 V +1 08/07/20, 12:56 Completed build in 19m 58s on Fri, 7 Aug, 12:56 PM Test Summary: 72983 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4479

  21. Helmut Hummel wrote 08/11/20, 14:14

    Topic set to image-processing
  22. Tim Schreiner commented on patch set 7 V +1 CR +1 08/12/20, 19:02

    Tested with jpg and png formats with different cropping configurations
  23. Jörg Bösche voted on patch set 7 CR +1 08/21/20, 07:20

  24. Helmut Hummel uploaded patch set 8 08/22/20, 13:47

  25. TYPO3com voted on patch set 8 V -1 08/22/20, 13:58 Completed build in 10m 43s on Sat, 22 Aug, 01:58 PM Test Summary: 73021 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4853

  26. Helmut Hummel uploaded patch set 9 08/22/20, 14:04

  27. Christian Kuhn voted on patch set 9 V +1 CR +1 08/22/20, 14:13

  28. TYPO3com voted on patch set 9 V -1 08/22/20, 14:19 Completed build in 13m 59s on Sat, 22 Aug, 02:18 PM Test Summary: 60353 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4854

  29. TYPO3com commented on patch set 9 08/23/20, 15:31 Completed build in 11m 07s on Sun, 23 Aug, 03:31 PM Test Summary: 72958 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4873

  30. TYPO3com voted on patch set 9 V +1 08/24/20, 07:34 Completed build in 16h 25m 02s on Mon, 24 Aug, 07:34 AM Test Summary: 73074 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4873

  31. Oliver Hader commented on patch set 9 08/24/20, 08:01

    GraphicalFunctions.php:2137

    Previous code had `!` addition to ignore original aspect ratio (https://imagemagick.org/script/command-line-processing.php#geometry)

    GraphicalFunctions.php:2146

    Can colorspace be omitted then? (probably checking that again)

    LocalCropScaleMaskHelper.php:201

    Does it mean `$configuraton['crop`]` is instanceof `Area` or is an array support as well? (probably add a comment here)

    LocalCropScaleMaskHelper.php:204

    This might be moved to a new Area::fromString function to have the factory-related part in one place (`fromString` can support both JSON-encoded and the TypoScript setting)
  32. Helmut Hummel commented on patch set 9 09/02/20, 06:59

    GraphicalFunctions.php:2137 Reply

    true. the geometry instruction had a `!` however before and now the geometry instruction is using the original width and height as input, so the result would be exactly the same. Should I add the ! anyway?

    GraphicalFunctions.php:2146 Reply

    in case to 'additionalParams' is not set as processing instruction the colorspace option is properly applied, previously it was applied twice. but yes, if 'additionalParams' is set, now the colorspace is omitted. we may want to change that

    LocalCropScaleMaskHelper.php:201 Reply

    with or without this change, an array as crop instruction currently isn't supported

    LocalCropScaleMaskHelper.php:204 Reply

    yes, this should be cleaned up at some point. this might end up in a processing configuration normalisation step, so we can drop it from here. but I'd like to pospone this decision for a alter change
  33. Helmut Hummel uploaded patch set 10 09/07/20, 09:47

  34. Helmut Hummel commented on patch set 10 09/07/20, 09:47

    GraphicalFunctions.php:2146 Reply

    I now enforced the colorspace (and quality) to always be set when required for the file type. I's say than 'additionalParams' should be trated as "additional" and not like before as "alternative" params.

    LocalCropScaleMaskHelper.php:201 Reply

    added a comment

    LocalCropScaleMaskHelper.php:204 Reply

    At this point in code, it should only be allowed to have one format. conversion should be done on TypoScript parsing. But as mentioned before, this is another cleanup to be done
  35. Helmut Hummel uploaded patch set 11 09/07/20, 09:56

  36. Helmut Hummel commented on patch set 11 09/07/20, 09:57

    GraphicalFunctions.php:2137 Reply

    added the ! nontheless
  37. TYPO3com voted on patch set 11 V +1 09/12/20, 13:24 Completed build in 13m 24s on Sat, 12 Sep, 01:24 PM Test Summary: 73207 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-5558

  1. Georg Ringer voted on patch set 11 V +1 CR +1 09/28/20, 12:16

  2. Markus Klein commented on patch set 11 CR -1 09/28/20, 12:50

    GraphicalFunctions.php:2147

    Has this been dropped on purpose?
  3. Helmut Hummel commented on patch set 11 09/28/20, 15:14

    GraphicalFunctions.php:2147 Reply

    Yes. See https://review.typo3.org/c/Packages/TYPO3.CMS/+/65187/9/typo3/sysext/core/Classes/Imaging/GraphicalFunctions.php#b2146 for a history of this change. tl;dr: colorspace is still applied, as it is part of default $params that are initialized above
  4. Markus Klein commented on patch set 11 CR +1 09/28/20, 21:23

    GraphicalFunctions.php:2147 Reply

    Ack
  5. Markus Klein commented on patch set 11 CR -1 10/06/20, 09:02

    If I remove all processed images and debug the FE output, I see this command being executed -auto-orient -geometry 640x640! -crop 366x201+141+365 +repage -colorspace sRGB -quality 85 -sharpen 1x2 -auto-orient -geometry 366x201! -colorspace sRGB -quality 85 -sharpen 1x2 IMO this is a bit too much
  6. Markus Klein commented on patch set 11 V -1 10/06/20, 09:07

    GraphicalFunctions.php:2145

    $this->scalecmd and $params is applied twice. Here and in line 2133
  7. Helmut Hummel commented on patch set 11 10/06/20, 17:07

    > Patch Set 11: Code-Review-1 > > If I remove all processed images and debug the FE output, I see this command being executed > > -auto-orient -geometry 640x640! -crop 366x201+141+365 +repage -colorspace sRGB -quality 85 -sharpen 1x2 -auto-orient -geometry 366x201! -colorspace sRGB -quality 85 -sharpen 1x2 > > IMO this is a bit too much Why do you think this is too much? Please elaborate
  8. Georg Ringer commented on patch set 11 10/06/20, 17:08

    > > -auto-orient -geometry 640x640! -crop 366x201+141+365 +repage -colorspace sRGB -quality 85 -sharpen 1x2 -auto-orient -geometry 366x201! -colorspace sRGB -quality 85 -sharpen 1x2 > > > > IMO this is a bit too much > > Why do you think this is too much? Please elaborate I guess because "-quality 85 -sharpen 1x2 " is used twice
  9. Helmut Hummel commented on patch set 11 10/06/20, 17:10

    GraphicalFunctions.php:2145 Reply Unresolved

    That is actually EXACTLY how it was before, except that only one IM/GM command is used instead of 2 Besides that, the scale command above uses the original dimensions, while this one uses the scaled dimensions
  10. Helmut Hummel commented on patch set 11 10/06/20, 17:22

    > I guess because "-quality 85 -sharpen 1x2 " is used twice OK, thanks. quality is fine to be used multiple times, I'm not sure about sharpen though. Will check.
  11. Helmut Hummel uploaded patch set 12 10/06/20, 17:43

  12. Helmut Hummel commented on patch set 11 10/06/20, 17:44

    > OK, thanks. quality is fine to be used multiple times, I'm not sure about sharpen though. Will check. Did not check, but fixed it to only be applied once
  13. TYPO3com voted on patch set 12 V +1 10/06/20, 17:57 Completed build in 14m 20s on Tue, 6 Oct, 05:57 PM Test Summary: 73785 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-6296

  14. Markus Klein commented on patch set 12 10/06/20, 22:28

    GraphicalFunctions.php:2176

    I now get this command if an image in a content element with a custom crop area: -auto-orient -geometry 640x640! -crop 366x201+141+365 +repage -colorspace sRGB -quality 85 -auto-orient -geometry 366x201! -colorspace sRGB -quality 85 -sharpen 1x2 If I understood this correctly, the operations are done left to right, so after cropping to 366x201 and a quality reduction to 85% we do another resize (aka geometry) to the same size and another quality reduction to 85%. I guess the problem described in the ticket (double quality loss) is still present, right? IMO the command should be: -auto-orient -geometry 640x640! -crop 366x201+141+365 +repage -colorspace sRGB -quality 85 -sharpen 1x2
  15. Helmut Hummel commented on patch set 12 10/07/20, 17:23

    GraphicalFunctions.php:2176 Reply

    > Patch Set 12: > > I guess the problem described in the ticket (double quality loss) is still present, right? How about testing this out? I did. My findings were that the quality setting isn't applied for internal processing within one IM/GM command. Do you have different findings? > -auto-orient -geometry 640x640! -crop 366x201+141+365 +repage -colorspace sRGB -quality 85 -auto-orient -geometry 366x201! -colorspace sRGB -quality 85 -sharpen 1x2 Oh, I've overlooked, that in this case the second resize can be omitted in this case. It should however still not affect the final output
  16. Markus Klein commented on patch set 12 10/07/20, 20:12

    GraphicalFunctions.php:2176 Reply

    > How about testing this out? I did. My findings were that the quality setting isn't applied for internal processing within one IM/GM command. Do you have different findings? Not yet, but I can double check of course. > It should however still not affect the final output Correct, the output still looks valid. But my intention is more that the whole process might speed up if we don't do superfluous operations on an image.
  17. Markus Klein commented on patch set 12 10/08/20, 13:55

    GraphicalFunctions.php:2176 Reply

    Okay, I double checked now with docs: The operations in the given order do not cause problems, but are mostly superfluous. It would suffice to have: -auto-orient -geometry 640x640! -crop 366x201+141+365 +repage -colorspace sRGB -quality 85 -sharpen 1x2
  18. Markus Klein commented on patch set 12 V +1 CR +1 10/08/20, 13:59

    I would say we fix the superfluous operations in a separate patch. Lets get that in.
  19. Benni Mack commented on patch set 12 11/03/20, 13:53

    GraphicalFunctions.php:2176 Reply

    How to continue here?
  20. Markus Klein commented on patch set 12 11/03/20, 13:58

    GraphicalFunctions.php:2176 Reply Unresolved

    Should be fixed with a different patch, but not critical at all
  21. Benni Mack voted on patch set 12 V +2 CR +2 11/17/20, 13:58

  22. Benni Mack merged patch set 13 11/17/20, 13:58

  23. Markus Klein commented on patch set 13 12/16/20, 19:00

    LocalCropScaleMaskHelper.php:209

    seems like we have a regression here. I need: `new Area($cropData->default->cropArea->x, $cropData->default->cropArea->y, $cropData->default->cropArea->width, $cropData->default->cropArea->height);` here, otherwise error. Since we moved this code down from above, I wonder if this might have been a problem before?
  24. Markus Klein commented on patch set 13 12/16/20, 20:18

    LocalCropScaleMaskHelper.php:209 Reply Unresolved

    Okay, not really a regression directly, but still formerly working code (also inside the news extension) is broken now. Might still be an edge-case. https://github.com/georgringer/news/pull/1409
  25. Benni Mack wrote 12/21/20, 16:08

    Created a revert of this change as If3f38cfeeb860e1a13648d239f05d2754f2f9102