Review main Merged

53834[BUGFIX] Catch Exception while extracting metadata

Ricky Mathew CR +2 V +2 Easy +10 −3 updated 04/04/18, 08:57

Commit message

Catch InsufficientFileAccessPermissionsException while extracting
metadata. Reason for this exception could be that the file extension is
not allowed by the ['BE']['fileDenyPattern'] setting.

This patch makes sure that the indexer doesn't break and is able to
extract the other files from storage.

Review

  • Code review CR +2
    • Georg Ringer +2
    • Alexander Opitz +1
    • Wouter Wolters +1
    • Ricky Mathew +1
  • Verified V +2
    • Georg Ringer +2
    • Ricky Mathew +1
    • TYPO3com +1 CI

Issues 1

Files 1

Review log 2933

Show 78 earlier messages
  1. Ricky Mathew uploaded patch set 1 08/30/17, 09:11

  2. TYPO3com voted on patch set 1 V -1 08/30/17, 09:26 Completed build in 15m 04s on Wed, 30 Aug, 11:26 AM Test Summary: 1 of 58448 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-14099

  3. Frans Saris commented on patch set 1 08/30/17, 10:57

    The try catch didn't work?
  4. Ricky Mathew commented on patch set 1 08/31/17, 05:36

    > The try catch didn't work? Try catch isn't working.. it tends to fail
  5. Frans Saris commented on patch set 1 08/31/17, 05:57

    It fails because of another exception?
  6. Ricky Mathew commented on patch set 1 08/31/17, 06:13

    > It fails because of another exception? Yes
  7. Frans Saris commented on patch set 1 08/31/17, 10:52

    Can you post the stacktrace here? I'm more in favor of catching the exception then handling the config check here as there could be more reasons why a file couldn't be processed. With catching the correct exceptions we fix the problem more generic
  8. Ricky Mathew commented on patch set 1 09/07/17, 07:07

    > Can you post the stacktrace here? I'm more in favor of catching the > exception then handling the config check here as there could be > more reasons why a file couldn't be processed. > > With catching the correct exceptions we fix the problem more > generic Here is the debug backtrace #0 /typo3/sysext/core/Classes/Resource/ResourceStorage.php(1621): TYPO3\CMS\Core\Resource\ResourceStorage->assureFileReadPermission(Object(TYPO3\CMS\Core\Resource\File)) #1 /typo3/sysext/core/Classes/Resource/File.php(137): TYPO3\CMS\Core\Resource\ResourceStorage->getFileContents(Object(TYPO3\CMS\Core\Resource\File)) #3 /typo3/sysext/core/Classes/Resource/Index/Indexer.php(111): TYPO3\CMS\Core\Resource\Index\Indexer->extractMetaData(Object(TYPO3\CMS\Core\Resource\File)) #4 /typo3/sysext/scheduler/Classes/Task/FileStorageExtractionTask.php(49): TYPO3\CMS\Core\Resource\Index\Indexer->runMetaDataExtraction(30) #5 /typo3/sysext/scheduler/Classes/Scheduler.php(181): TYPO3\CMS\Scheduler\Task\FileStorageExtractionTask->execute() #6 /typo3/sysext/scheduler/Classes/Controller/SchedulerModuleController.php(867): TYPO3\CMS\Scheduler\Scheduler->executeTask(Object(TYPO3\CMS\Scheduler\Task\FileStorageExtractionTask)) #7 /typo3/sysext/scheduler/Classes/Controller/SchedulerModuleController.php(243): TYPO3\CMS\Scheduler\Controller\SchedulerModuleController->executeTasks() #8 /typo3/sysext/scheduler/Classes/Controller/SchedulerModuleController.php(158): TYPO3\CMS\Scheduler\Controller\SchedulerModuleController->getModuleContent() #9 /typo3/sysext/scheduler/Classes/Controller/SchedulerModuleController.php(314): TYPO3\CMS\Scheduler\Controller\SchedulerModuleController->main() The function assureFileReadPermission() of TYPO3\CMS\Core\Resource\ResourceStorage catch and throws the exception.But i think this function is not intended for scheduler alone and that's why it's throwing the exception.SO for scheduler we don't need to throw exception on file indexing , we just need to skip indexing the file.So it is better to solve the problem from upper layer and not to go deep.
  1. Frans Saris uploaded patch set 2 11/14/17, 17:57

  2. Frans Saris commented on patch set 2 11/14/17, 17:58

    Ricky could you give my patch a try?
  3. TYPO3com voted on patch set 2 V +1 11/14/17, 18:08 Completed build in 10m 39s on Tue, 14 Nov, 07:08 PM Test Summary: 56768 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-16347

  4. Ricky Mathew commented on patch set 2 V -1 11/15/17, 09:43

    > Ricky could you give my patch a try? I just tried the patch.But it's not working at all and it fails.Because the function $this->extractMetaData() in try block goes through a series of functions and finally reaches the 'assureFileReadPermission()' function in TYPO3\CMS\Core\Resource\ResourceStorage where it directly throws the exception and thus stops the execution of the scheduler.
  5. Frans Saris commented on patch set 2 11/15/17, 11:52

    The try catch catches only InsufficientFileAccessPermissionsException exceptions. So the only reason that you still would get an exception if it's a different exception. Can you provide a full stack trace. I'm really interested in what exception you run into
  6. Ricky Mathew commented on patch set 2 11/15/17, 12:23

    It's 'IllegalFileExtensionException' that i ran into. So a tweak in code like the following worked for me in 'catch' section catch (InsufficientFileAccessPermissionsException | IllegalFileExtensionException $e) { // We skip files that are not accessible }
  7. Frans Saris uploaded patch set 3 11/19/17, 09:47

  8. Frans Saris commented on patch set 2 11/19/17, 09:48

    tnx for tracking this down. As we still need to be compatible with PHP 7 we need 2 separate catch lines. Could you test this patch?
  9. TYPO3com voted on patch set 3 V +1 11/19/17, 09:58 Completed build in 11m 06s on Sun, 19 Nov, 10:58 AM Test Summary: 56802 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-16472

  10. Michael Oehlhof commented on patch set 3 11/19/17, 12:00

    Indexer.php:116

    What about logging files that are not accessible?
  11. Frans Saris commented on patch set 3 11/19/17, 12:15

    IMO not needed. Logs will fast fill with these messages and you are not able to resolve the error if you have/need such a file in your file storage
  12. Ricky Mathew commented on patch set 3 CR +1 V +1 11/20/17, 05:01

    > tnx for tracking this down. As we still need to be compatible with > PHP 7 we need 2 separate catch lines. > > Could you test this patch? I just checked and working fine for me.
  13. Alexander Opitz commented on patch set 3 CR +1 11/20/17, 08:21

    Indexer.php:120

    Only by the way: Shouldn't this comment be moved in the else part?
  14. Henning Liebe commented on patch set 4 11/20/17, 09:47

    Published edit on patch set 3.
  15. TYPO3com voted on patch set 4 V +1 11/20/17, 09:58 Completed build in 11m 26s on Mon, 20 Nov, 10:58 AM Test Summary: 56802 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-16487

  16. Ricky Mathew commented on patch set 4 CR +1 V +1 11/20/17, 09:59

    > Patch Set 4: Published edit on patch set 3.
  17. Alexander Opitz voted on patch set 4 CR +1 11/20/17, 13:05

  18. Wouter Wolters voted on patch set 4 CR +1 11/20/17, 14:00

  19. Georg Ringer voted on patch set 4 CR +2 V +2 11/20/17, 19:29

  20. Gerrit merged patch set 4 11/20/17, 19:30

  21. Georg Ringer commented on patch set 5 11/20/17, 19:30

    Cherry Picked This patchset was cherry picked to branch TYPO3_8-7 as commit 16b26860dbaeb203162b077bcdaf2a60bcb04c67
  22. Stephan Großberndt commented on patch set 5 12/20/17, 16:44

    Backport to 7.6 is missing
  23. Stephan Großberndt commented on patch set 5 02/19/18, 20:52

    ping
  24. Stephan Großberndt commented on patch set 5 04/03/18, 15:51

    ping
  25. Alexander Opitz commented on patch set 5 04/04/18, 08:57