Review main Merged

65279[TASK] Improve and fix ThumbnailController

Helmut Hummel CR +2 V +2 Easy +29 −21 updated 09/02/20, 06:36

Commit message

The ThumbnailController outputs raw data of image files
as a response to a file process request.
This is a massive and unnecessary performance hit, when using
a remote storage, as the files need to get downloaded from
the remote to be then delivered by TYPO3.

This is now changed to simply redirect to the file URL.

Also the mime-type check for images is replaced with
an isImage() call to the processed file, in order
to also properly deliver thumbnails of PDF files.

Also clarify in code, that ProcessedFile::CONTEXT_IMAGEPREVIEW
isn't capable of handling any other processing instruction
than "width" and "height".

Also fix delivering an icon as thumbnail,
when the processed file isn't an image.

Last but not least, fix the 404 response by using
a HtmlResponse instead of a plain Response object,
because the plain Response object needs a resource
and cannot deal with plain strings.

Review

  • Code review CR +2 Done
    • Helmut Hummel +2
    • Markus Klein +1
    • Jörg Bösche +1
    • Benni Mack +1
  • Verified V +2 Done
    • Helmut Hummel +2
    • Markus Klein +1
    • Jörg Bösche +1
    • Benni Mack +1
    • TYPO3com +1 CI

Issues 1

Files 6

Review log 3543

Show 1318 earlier messages
  1. Helmut Hummel uploaded patch set 1 08/11/20, 14:31

  2. Helmut Hummel wrote 08/11/20, 14:31

    Topic set to image-processing
  3. TYPO3com voted on patch set 1 V +1 08/11/20, 14:42 Completed build in 11m 39s on Tue, 11 Aug, 02:42 PM Test Summary: 72989 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4576

  4. Tim Schreiner commented on patch set 1 V +1 08/11/20, 16:55

    We might should write an info or even warning to the log if unsupported configurations are used in preview context (maxWidth, minWidth, maxHeight, minHeight) to give a developer a better hint when they are used in the ThumbnailViewHelper.
  5. Tim Schreiner commented on patch set 1 CR +1 08/11/20, 17:06

    > Patch Set 1: Verified+1 > > We might should write an info or even warning to the log if unsupported configurations are used in preview context (maxWidth, minWidth, maxHeight, minHeight) to give a developer a better hint when they are used in the ThumbnailViewHelper. I just thought about that we plan to deprecate the preview context in the near future and use CropScale context instead. From this point of view, logging unsupported options are not really necessary.
  6. Tymoteusz Motylewski commented on patch set 1 08/11/20, 21:00

    ThumbnailController

    LocalPreviewHelper.php:53

    Will this force regeneration of the all existing thimbnails (as config would be different)?
  7. Helmut Hummel commented on patch set 1 08/12/20, 09:42

    LocalPreviewHelper.php:53 Reply

    It will force regeneration of all thumbnail, that provide other configuration than width and height, yes. As far as I have seen, most thumbnails provide only these properties, but haven't investigated in detail. I also think the impact isn't that big, as not many thumbnails are genrated in a single request and currently also most of them are generated using the thumb script and thus are generated asynchronously already
  8. Helmut Hummel updated the commit message in patch set 2 08/12/20, 09:46

  9. Helmut Hummel commented on patch set 2 08/12/20, 09:47

    Reply

    Done
  10. Helmut Hummel commented on patch set 2 08/12/20, 09:53

    LocalPreviewHelper.php:53 Reply

    Additionally this change will (hopefully finally) consolidate thumbnail generation as currently the exact same thumbnail is created multiple times, because of differing configuration, which has no effect at all. So in the best case nothing is regenrated, but a thumb with only width and height as config was already generated and will be reused.
  11. TYPO3com voted on patch set 2 V +1 08/12/20, 09:57 Completed build in 10m 42s on Wed, 12 Aug, 09:57 AM Test Summary: 72986 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4592

  12. Helmut Hummel uploaded patch set 3 08/12/20, 10:20

  13. Helmut Hummel commented on patch set 3 08/12/20, 10:21

    LocalPreviewHelper.php:53 Reply

    checked again, found that the filter expression was wrong, fixed it and evaluated, that almost all thumbs need to be regenerated when applying this, because unfortunately the ThumbnailController always adds 'crop' => null to the configuration
  14. TYPO3com voted on patch set 3 V -1 08/12/20, 10:31 Completed build in 11m 19s on Wed, 12 Aug, 10:31 AM Test Summary: 72658 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4593

  15. Helmut Hummel commented on patch set 3 08/12/20, 10:45

    LocalPreviewHelper.php:53 Reply Unresolved

    For my use case of 21 images, and a certain test scenario to show thumbs of them in different contexts, I found that with this change 42 processed thumb files are generated, before it were 66. So basically 24 (so over a third) were generated in excess. 21 of them only had width and height in the configuration (file picker). This also means that a system that has tumbs generated needs before this change, after this change there will be 21 new records in the processed_file table, but since correct thumbs were available (from file picker), no thumb needs to be re-generated at all. My conclusion is, that this change will not cause trouble after an update
  16. Helmut Hummel commented on patch set 3 08/12/20, 10:56

    On my dev system, showing a folder with 21 images with thumbnails from a local storage is roughly 25% faster than before this change. for remote storages the performance improvement will be even more significantly faster
  17. TYPO3com voted on patch set 3 V +1 08/12/20, 11:00 Completed build in 11m 00s on Wed, 12 Aug, 11:00 AM Test Summary: 72986 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4594

  18. TYPO3com commented on patch set 3 08/12/20, 11:04 Completed build in 13m 50s on Wed, 12 Aug, 11:03 AM Test Summary: 72986 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4595

  1. Benjamin Franzke commented on patch set 3 CR -1 08/12/20, 11:26

    ThumbnailController.php:115

    No configurable permanent (301) redirects please – we'll never be able to change the destination again, so possible bugs can not be fixed. If we (really) wanted a redirect here, that should be a 302 instead, but: We actually have support for streaming files as a PSR-7 response (also available in v9.5), added in https://review.typo3.org/c/Packages/TYPO3.CMS/+/55585/ So in theory (not tested with process files yes), we should rather do something like the following here: return $processedImage->getStorage()->streamFile($processedImage);
  2. Markus Klein commented on patch set 3 08/12/20, 19:35

    ThumbnailController.php:115 Reply

    I suppose the sent headers will be different if PHP responds with the image directly in comparison to the webserver delivering the image.
  3. Helmut Hummel commented on patch set 3 08/13/20, 14:03

    ThumbnailController.php:115 Reply

    @Benjamin: how would a streamFile implementation of a Amazon S3 driver look like? Or Dropbox driver? Might be possible to do, I still don't see point in doing so, when a simple redirect solves what we need. Maybe you can elaborate on the benefits of streaming? Regarding 301 vs. 302: fine with me to use a 302. I'd be better than before and this controller (and the concept) will be removed in 11 anyway.
  4. Helmut Hummel commented on patch set 3 08/13/20, 15:52

    ThumbnailController.php:115 Reply

    On the other hand, the URL that issues the permanent redirect, always contains a token, which is unique for every login session. so a simple re-login would void the cache anyway.
  5. Helmut Hummel uploaded patch set 4 08/13/20, 15:55

  6. TYPO3com voted on patch set 4 V -1 08/13/20, 15:56 Completed build in 25s on Thu, 13 Aug, 03:56 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4646

  7. Helmut Hummel commented on patch set 4 08/13/20, 15:56

    ThumbnailController.php:115 Reply

    changed to 302 redirect now
  8. Helmut Hummel uploaded patch set 5 08/13/20, 15:58

  9. Helmut Hummel commented on patch set 5 08/13/20, 15:58

    ThumbnailController.php:115 Reply

    Done
  10. TYPO3com voted on patch set 5 V +1 08/13/20, 16:10 Completed build in 11m 32s on Thu, 13 Aug, 04:10 PM Test Summary: 72986 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4647

  11. Markus Klein updated the commit message in patch set 6 08/13/20, 18:09

  12. Markus Klein commented on patch set 6 08/13/20, 18:13

    ThumbnailController.php:114

    why do we need the true param here?

    LocalPreviewHelper.php:55

    lets add typehints here
  13. TYPO3com voted on patch set 6 V +1 08/13/20, 18:20 Completed build in 10m 53s on Thu, 13 Aug, 06:20 PM Test Summary: 72986 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4648

  14. Helmut Hummel commented on patch set 6 08/14/20, 08:30

    ThumbnailController.php:114 Reply

    because we are in /typo3/ and the public url of files is (for whatever reason) relative, which would lead to paths typo3/fileadmin/...

    LocalPreviewHelper.php:55 Reply

    we can't. value might be an object or an array or a string or an integer. array key should be string, but I'm not sure whether I'd let the code fatal before doing a deprecation process
  15. Markus Klein commented on patch set 6 08/14/20, 08:43

    ThumbnailController.php:114 Reply

    Ah thanks. I overlooked the behaviour of localtionHeaderUrl in this regard.

    LocalPreviewHelper.php:55 Reply

    huh? $configuration[xxx] can be an object? I missed that, totally. All clear then.
  16. Markus Klein voted on patch set 6 CR +1 08/14/20, 08:43

  17. Helmut Hummel commented on patch set 6 08/14/20, 09:29

    LocalPreviewHelper.php:55 Reply

    yeah, $configuration['crop'] currently is an Area object when a crop is set
  18. Markus Klein voted on patch set 6 V +1 08/18/20, 06:09

  19. Jörg Bösche voted on patch set 6 CR +1 08/27/20, 12:31

  20. Benni Mack voted on patch set 6 V +1 CR +1 09/01/20, 06:49

  21. Benni Mack commented on patch set 6 09/01/20, 06:50

    ThumbnailController.php:112

    FYI: It might be that "isImage()" is not available in 9.5. We need to recheck this on the v9 backport.
  22. Jörg Bösche voted on patch set 6 V +1 09/01/20, 08:05

  23. Helmut Hummel voted on patch set 6 V +2 CR +2 09/02/20, 06:33

  24. Helmut Hummel commented on patch set 6 09/02/20, 06:36

    ThumbnailController.php:112 Reply Unresolved

    yeah, I know, thanks. Will consider this during the backport.
  25. Helmut Hummel merged patch set 7 09/02/20, 06:36