Review main Merged

47169[TASK] Use PathUtility wrappers for basename/dirname

Jigal van Hemert CR +2 V +2 Normal +141 −88 updated 05/12/18, 18:52

Commit message

Basename and dirname are locale aware and can damage path and file names
if they are used with UTF-8 file/path names while a UTF-8 locale is not
loaded.
The PathUtility wrappers solve this.

Review

  • Code review CR +2
    • Susi Moog +2
    • Stefan Neufeind +1
    • Mathias Schreiber +1
  • Verified V +2
    • Susi Moog +2
    • Stefan Neufeind +1
    • Mathias Schreiber +1
    • TYPO3com +1 CI

Issues 1

Files 37

Show 17 more files

Review log 5167

Show 3242 earlier messages
  1. Jigal van Hemert uploaded patch set 1 03/10/16, 09:37

  2. Daniel Goerz voted on patch set 1 CR +1 03/10/16, 16:10

  3. Frank Nägler voted on patch set 1 CR +1 03/11/16, 10:01

  4. Frank Nägler commented on patch set 1 V -1 03/11/16, 11:59

    The usage in SystemEnvironmentBuilder is a problem, because it looks like $GLOBALS['TYPO3_CONF_VARS'] is not set at this point. Undefined index: TYPO3_CONF_VARS in /var/www/t3/Web/TYPO3.CMS/typo3/sysext/core/Classes/Utility/PathUtility.php on line 188
  5. Nicole Hummel commented on patch set 1 CR -1 03/11/16, 14:51

    This is a simple search and replace. I would suggest to rethink were utf-8 files are really should work (e.g. in the Cache-Backends I wouldn't support it!)
  6. Jigal van Hemert commented on patch set 1 03/11/16, 17:48

    > This is a simple search and replace. I would suggest to rethink > were utf-8 files are really should work (e.g. in the Cache-Backends > I wouldn't support it!) Search and replace... hmm... I thought I was alone at home with nobody watching while I prepared the patch. We can't be sure about the location of the installation. As UTF-8 file systems are quite common there can very well be segments which contain for example a client/company name. On non-Western installations it can be quite common; consider one of our Korean friends who puts the code in C:\Users\준영\Documents\Projects\TYPO3\ IMO we simply cannot assume where UTF-8 file/path names can be used and where not.
  7. Nicole Hummel commented on patch set 1 03/11/16, 18:14

    Just see the two commented places

    FileBackend.php:138

    no path allowed anyways. no need to test utf-8 ensured

    SimpleFileBackend.php:225

    no path allowed anyways. no need to test utf-8 ensured
  8. Jigal van Hemert commented on patch set 1 03/11/16, 20:02

    FileBackend.php:138 Reply

    It already goes wrong with an identifier starting with a diacritic: basename('üexample') -> 'example' if you don't have a UTF-8 locale loaded.

    SimpleFileBackend.php:225 Reply

    See previous comment
  9. Markus Klein commented on patch set 1 03/14/16, 21:47

    If the sysenvbuilder already fails, does it help to make the remainder of the core locale aware?
  10. Jigal van Hemert uploaded patch set 2 03/25/16, 23:00

  11. Jigal van Hemert commented on patch set 2 03/26/16, 10:11

    A bit of explanation: we use dirname() and basename() all over the place, but these two functions can't handle UTF-8 character unless a UTF-8 locale is loaded. Many file systems support UTF-8 characters in directory and file names. If an installation is located in something like /var/www/äbersold/html/ (customer name in path) things go horribly wrong. There is already an issue to automatically set the system locale; this is problematic, so I'll try to modify that in an Install Tool option that tries to detect a locale and suggest one to the user. This is just a side note to emphasize that we simply must support UTF-8 file systems more.
  12. Mathias Schreiber commented on patch set 2 CR -1 V -1 03/31/16, 16:58

    can't reproduce :)
  13. Susi Moog commented on patch set 2 01/22/17, 14:20

    What's the status of this issue? Is it still needed and is someone working on it?
  14. Jigal van Hemert uploaded patch set 3 07/21/17, 13:49

  15. TYPO3com voted on patch set 3 V -1 07/21/17, 13:49 Completed build in 15s on Fri, 21 Jul, 03:49 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-13376

  16. Jigal van Hemert commented on patch set 3 CR -2 07/21/17, 15:04

    Just rebased and tracked code that was moved/deleted. Needs further checking and fixing on rebase actions.
  17. Jigal van Hemert uploaded patch set 4 07/21/17, 21:38

  18. TYPO3com voted on patch set 4 V -1 07/21/17, 21:38 Completed build in 15s on Fri, 21 Jul, 11:38 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-13379

  19. Jörg Bösche uploaded patch set 5 07/31/17, 09:15

  20. TYPO3com voted on patch set 5 V +1 07/31/17, 09:26 Completed build in 11m 04s on Mon, 31 Jul, 11:26 AM Test Summary: 56928 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-13502

  21. Jigal van Hemert commented on patch set 5 CR +1 08/01/17, 20:53

    Thanks for the last changes; couldn't get the cgl fix script working on windows.
  22. Susi Moog commented on patch set 5 08/07/17, 07:17

    GeneralUtility.php:2757

    would it make sense to add the regex to a central utility function? this is the second usage and it's not clear what the regex does from just viewing.
  23. Jigal van Hemert uploaded patch set 6 08/08/17, 17:50

  24. Jigal van Hemert commented on patch set 5 08/08/17, 17:54

    GeneralUtility.php:2757 Reply

    Done
  25. TYPO3com voted on patch set 6 V +1 08/08/17, 18:01 Completed build in 11m 11s on Tue, 8 Aug, 08:01 PM Test Summary: 57426 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-13595

  26. Susi Moog uploaded patch set 7 11/28/17, 15:22

  27. TYPO3com voted on patch set 7 V -1 11/28/17, 15:22 Completed build in 20s on Tue, 28 Nov, 04:22 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-16831

  28. Susi Moog uploaded patch set 8 11/28/17, 16:20

  29. TYPO3com voted on patch set 8 V +1 11/28/17, 16:33 Completed build in 12m 58s on Tue, 28 Nov, 05:33 PM Test Summary: 56870 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-16838

  30. Stefan Neufeind uploaded patch set 9 12/09/17, 05:47

  31. Stefan Neufeind commented on patch set 9 12/09/17, 05:52

    @Jigal: Could you have a look whether in the following files additional calls might need to be updated as well? for dirname: typo3/sysext/lowlevel/Classes/Command/RteImagesCommand.php typo3/sysext/core/Classes/TypoScript/Parser/TypoScriptParser.php for basename: typo3/sysext/install/Classes/Updates/BackendLayoutIconUpdateWizard.php typo3/sysext/lowlevel/Classes/Command/RteImagesCommand.php typo3/sysext/core/Classes/Localization/Parser/AbstractXmlParser.php typo3/sysext/core/Classes/TypoScript/Parser/TypoScriptParser.php
  32. TYPO3com voted on patch set 9 V +1 12/09/17, 05:57 Completed build in 09m 41s on Sat, 9 Dec, 06:57 AM Test Summary: 57053 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-17449

  33. Jigal van Hemert uploaded patch set 10 12/11/17, 19:12

  34. TYPO3com voted on patch set 10 V -1 12/11/17, 19:12 Completed build in 16s on Mon, 11 Dec, 08:12 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-17592

  35. Oliver Klee commented on patch set 10 12/11/17, 19:25

    Also 8.7?
  36. Jigal van Hemert commented on patch set 10 12/11/17, 19:26

    The failing tests are also present in current master. I've found a few dirname() calls in composer InstallerScripts classes, but I have no idea what we can use here; can we call the PathUtility with the functions that use regexps?
  37. Mathias Schreiber updated the commit message in patch set 11 02/18/18, 14:23

  38. Mathias Schreiber commented on patch set 11 02/18/18, 14:23

    micro change to the commit message to retrigger bamboo build
  39. TYPO3com voted on patch set 11 V -1 02/18/18, 14:23 Completed build in 18s on Sun, 18 Feb, 03:23 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-19834

  40. Andreas Kienast uploaded patch set 12 02/18/18, 14:35

  41. TYPO3com voted on patch set 12 V +1 02/18/18, 14:44 Completed build in 09m 01s on Sun, 18 Feb, 03:44 PM Test Summary: 30902 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-19835

  42. Tobi Kretschmann voted on patch set 12 CR +1 02/19/18, 09:12

  1. Jan Stockfisch voted on patch set 12 CR +1 02/19/18, 10:02

  2. Stefan Neufeind voted on patch set 12 CR +1 02/19/18, 10:54

  3. Anja Leichsenring uploaded patch set 13 04/14/18, 15:44

  4. TYPO3com voted on patch set 13 V -1 04/14/18, 15:44 Completed build in 18s on Sat, 14 Apr, 05:44 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-22331

  5. Anja Leichsenring uploaded patch set 14 04/14/18, 15:45

  6. Anja Leichsenring voted on patch set 13 V +1 CR +1 04/14/18, 15:48

  7. Anja Leichsenring voted on patch set 14 V +1 CR +1 04/14/18, 15:48

  8. TYPO3com voted on patch set 14 V +1 04/14/18, 15:55 Completed build in 09m 47s on Sat, 14 Apr, 05:55 PM Test Summary: 30948 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-22332

  9. Stefan Neufeind commented on patch set 14 V +1 CR +1 04/15/18, 10:16

    Not easy to verify each and every case. But basically works fine for me and looks okay by review.
  10. Frank Nägler commented on patch set 14 05/09/18, 11:42

    @Jigal: This patch needs a rebase, can you take care? After rebase we should get this patch asap in.
  11. Jigal van Hemert uploaded patch set 15 05/10/18, 19:09

  12. TYPO3com voted on patch set 15 V -1 05/10/18, 19:09 Completed build in 23s on Thu, 10 May, 09:09 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-23290

  13. Jigal van Hemert uploaded patch set 16 05/10/18, 19:29

  14. TYPO3com voted on patch set 16 V -1 05/10/18, 19:38 Completed build in 08m 59s on Thu, 10 May, 09:38 PM Test Summary: 1 of 31251 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-23292

  15. TYPO3com voted on patch set 16 V +1 05/11/18, 10:13 Completed build in 14h 52m 59s on Fri, 11 May, 12:13 PM Test Summary: 31003 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-23292

  16. Susi Moog voted on patch set 16 V +1 CR +1 05/11/18, 13:13

  17. Susi Moog commented on patch set 16 CR -1 05/11/18, 13:21

    on it

    ErrorHandler.php:119

    is this a rebase error?
  18. Susi Moog uploaded patch set 17 05/11/18, 13:23

  19. Mathias Schreiber voted on patch set 17 V +1 CR +1 05/11/18, 13:25

  20. TYPO3com voted on patch set 17 V +1 05/11/18, 13:31 Completed build in 07m 42s on Fri, 11 May, 03:31 PM Test Summary: 31003 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-23323

  21. Susi Moog voted on patch set 17 V +2 CR +2 05/11/18, 13:47

  22. Susi Moog merged patch set 18 05/11/18, 13:48

  23. Stefan Neufeind voted on patch set 18 V +1 CR +1 05/11/18, 13:55

  24. Christian Kuhn commented on patch set 18 05/11/18, 21:03

    PathUtility.php:277

    these two methods would have been great opportunities for unit tests ...
  25. Stefan Neufeind commented on patch set 18 05/12/18, 18:52

    Please see https://review.typo3.org/56944 for unittests (and a fix for the basename-fetching).