Review main Merged

19684[BUGFIX] Files with unclean path indexed multiple times

Steffen Ritter CR +2 V +2 Hard +213 −39 updated 07/06/13, 22:44

Commit message

When adding a file or requesting a file by an identifier
cleanup any . and .. in the path before handing off
to the driver so files are not indexed multiple times.

Review

  • Code review CR +2
    • Wouter Wolters +2
  • Verified V +2
    • Wouter Wolters +2

Files 4

Review log 54

Show 29 earlier messages
  1. Stefan Neufeind uploaded patch set 2 05/27/13, 17:19

  2. Stefan Neufeind commented on patch set 2 05/27/13, 17:19

    Files with unclean identifier also lead to files being indexed multiple times. So this patch-set aims for generic "cleanup" for both cases, emulating what realpath() does (but without checking if a file exists). @Steffen: Please see if you're fine with this patch. Otherwise please apologize and let's talk :-)
  3. Stefan Neufeind commented on patch set 2 05/27/13, 17:19

    for testing the case with an unclean identifier (wherever they might come from) with a file under fileadmin/ try: page.10 = IMAGE page.10.file = 1:./abc/../something.jpg
  4. Stefan Neufeind uploaded patch set 3 05/27/13, 17:19

  5. Stefan Neufeind uploaded patch set 4 05/27/13, 17:19

  6. Stefan Neufeind commented on patch set 4 05/27/13, 17:19

    (1 inline comment)
  7. Dmitry Dulepov commented on patch set 5 05/27/13, 17:19

    While don't you use realpath()?
  8. Stefan Neufeind commented on patch set 5 05/27/13, 17:19

    It also checks for the existence of a file. We just want to clean up (relative) paths here. That's why it "almost" works like realpath in this regard.
  9. Stefan Neufeind uploaded patch set 5 05/27/13, 17:19

  10. Stefan Neufeind commented on patch set 5 05/27/13, 17:19

    (1 inline comment) CGL: added two more spaces :-)
  11. Alexander Opitz commented on patch set 1 05/27/13, 17:19

    I would prefer that you didn't submit this

    ResourceStorage.php:30

    You know ... you should change all occurences of TYPO3\CMS\Core\Utility\GeneralUtility ;-)
  12. Alexander Opitz commented on patch set 5 05/27/13, 17:19

    (1 inline comment)
  13. Stefan Neufeind uploaded patch set 7 05/27/13, 17:19

  14. Stefan Neufeind commented on patch set 7 05/27/13, 17:19

    Added testcases After working on the tests: Could it be that we (no longer?) need the separate cleanDirectoryNameAndFile() to clean a dirname with a file? Or which cases exist where that is still needed?
  15. Stefan Neufeind commented on patch set 7 05/27/13, 17:19

    Somebody please have a look. Imho we're quite close to a good solution with this? :-)
  16. Alexander Opitz uploaded patch set 6 05/27/13, 17:19

  17. Alexander Opitz commented on patch set 6 05/27/13, 17:19

    Verified; Looks good to me, but someone else must approve
  18. Steffen Ritter commented on patch set 2 05/27/13, 17:19

    I would prefer that you didn't submit this if we introduce a new function, please put it as static function to PathUtility so we do not need to instanciate an class.
  19. Philipp Gampe commented on patch set 1 05/27/13, 17:19

    Looks good to me, but someone else must approve
  20. Philipp Gampe commented on patch set 7 05/27/13, 17:19

    (4 inline comments) I do not think we are fully there yet, but looks good already
  21. Peter Niederlag commented on patch set 8 2 comments 05/27/13, 17:19

    I would prefer that you didn't submit this tests would like some polish
  22. Philipp Gampe commented on patch set 8 1 comment 05/27/13, 17:19

    Verified; Looks good to me, but someone else must approve Some more unit tests (also includes existing tests again). This patchset also takes care to preserve the slash at the beginning. The needs some testing with windows paths.
  23. Philipp Gampe commented on patch set 8 2 comments 05/27/13, 17:19

    Fails
  24. Philipp Gampe uploaded patch set 8 05/27/13, 17:19

  25. Philipp Gampe uploaded patch set 9 05/27/13, 17:19

  26. Philipp Gampe commented on patch set 9 05/27/13, 17:19

    fixed the comments
  27. Philipp Gampe commented on patch set 9 05/27/13, 17:19

    Looks good to me, but someone else must approve I would still like to see a test on windows or with windows paths
  28. Anja Leichsenring commented on patch set 1 05/27/13, 17:19

    (1 inline comment)

    ResourceStorage.php:30 Reply

    ... except for the ones in doc comments. IDEs do not handle this well.
  29. Mr. Jenkins commented on patch set 1 05/27/13, 17:19

  1. Mr. Jenkins commented on patch set 2 05/27/13, 17:19

  2. Mr. Jenkins commented on patch set 3 05/27/13, 17:19

  3. Mr. Jenkins commented on patch set 4 05/27/13, 17:19

  4. Mr. Jenkins commented on patch set 5 05/27/13, 17:19

  5. Mr. Jenkins commented on patch set 6 05/27/13, 17:19

  6. Mr. Jenkins commented on patch set 7 05/27/13, 17:19

  7. Mr. Jenkins commented on patch set 8 05/27/13, 17:19

  8. Mr. Jenkins commented on patch set 9 05/27/13, 17:19

  9. Wouter Wolters commented on patch set 1 05/27/13, 17:19

    Verified; Looks good to me, but someone else must approve
  10. Steffen Ritter commented on patch set 9 V -1 CR -1 07/04/13, 17:48

    The path-cleanup is just wrong :)
  11. Steffen Ritter uploaded patch set 10 07/04/13, 18:22

  12. Mr. Jenkins commented on patch set 10 07/04/13, 18:23

  13. Jigal van Hemert commented on patch set 10 V +1 CR +1 07/06/13, 21:07

    Tested on Windows. Unit test for path test are green.
  14. Stefan Neufeind commented on patch set 10 CR +1 07/06/13, 21:14

    BasicFileUtility.php:449

    for backport to 6.0 leave out this deprecated-annotation

    BasicFileUtility.php:449

    for backport to 6.0 leave out this deprecated-annotation
  15. Alexander Opitz commented on patch set 10 CR +1 07/06/13, 21:18

    ResourceStorage.php:413

    GeneralUtility added in the use, so replace it in the complete file.
  16. Wouter Wolters commented on patch set 10 07/06/13, 21:27

    ResourceStorage.php:154

    $basicFileUtility does not belong there.

    PathUtility.php:197

    remove empty line
  17. Wouter Wolters uploaded patch set 11 07/06/13, 21:30

  18. Mr. Jenkins commented on patch set 11 07/06/13, 21:31

  19. Wouter Wolters commented on patch set 11 V +2 CR +2 07/06/13, 21:32

    No functional changes, only a minor CGL cleanup which I discussed with Steffen Ritter.
  20. Wouter Wolters merged patch set 12 07/06/13, 21:32

  21. Wouter Wolters commented on patch set 12 07/06/13, 21:32

    Cherry Picked This patchset was cherry picked to change: I324c4637621165a56470fb95864eeff1254f62dc
  22. Andreas Wolf commented on patch set 12 07/06/13, 21:38

    I think this change should not have been merged because all drivers that use non-hierarchical identifiers might be more broken now. I did not carefully review the changes, but tampering with the identifier on a storage-level seems to be error-prone. If at all, this should be done on the driver level. There we clearly need such a feature for e.g. the local storage (and WebDAV, Amazon S3...)
  23. Steffen Ritter commented on patch set 12 07/06/13, 21:40

    non hierarchichal storages don't use pathes - and the problem is not in driver-level; driver already handles that correctly - aka it does not care! Indexing/Storage cannot handle that! Therefore we definetely need that at that point...
  24. Wouter Wolters commented on patch set 12 07/06/13, 22:43

    Reverted This patchset was reverted in change: I7135b9478e01a5f95ad20df3a6b24bb5d9715ec8
  25. Wouter Wolters commented on patch set 12 07/06/13, 22:44

    Reverted because it broke several tests: There were 2 errors: 1) TYPO3\CMS\Core\Tests\Unit\Resource\ResourceStorageTest::addFileCallsDriverWithCorrectArguments InvalidArgumentException: File "vfs:/base-51d8985331d94/mount-51d8985331ce2/file.ext" does not exist. /var/www/websites/master/htdocs/typo3/sysext/core/Classes/Resource/ResourceStorage.php:722 /var/www/websites/master/htdocs/typo3/sysext/core/Tests/Unit/Resource/ResourceStorageTest.php:249 /var/www/websites/master/htdocs/typo3conf/ext/phpunit/Classes/TestRunner/ManualCliTestRunner.php:45 /var/www/websites/master/htdocs/typo3conf/ext/phpunit/Classes/TestRunner/ManualCliTestRunner.php:51 /var/www/websites/master/htdocs/typo3/cli_dispatch.phpsh:65 2) TYPO3\CMS\Core\Tests\Unit\Resource\ResourceStorageTest::addFileChangesFilenameIfFileExists InvalidArgumentException: File "vfs:/base-51d898533c069/mount-51d898533bfba/file.ext" does not exist. /var/www/websites/master/htdocs/typo3/sysext/core/Classes/Resource/ResourceStorage.php:722 /var/www/websites/master/htdocs/typo3/sysext/core/Tests/Unit/Resource/ResourceStorageTest.php:271 /var/www/websites/master/htdocs/typo3conf/ext/phpunit/Classes/TestRunner/ManualCliTestRunner.php:45 /var/www/websites/master/htdocs/typo3conf/ext/phpunit/Classes/TestRunner/ManualCliTestRunner.php:51 /var/www/websites/master/htdocs/typo3/cli_dispatch.phpsh:65 FAILURES!