Review main Abandoned

40871[TASK] Added a local cache for the results of retrieveFileAndFoldersInPath

Michael Oehlhof CR 0 V 0 Easy +15 −2 updated 10/19/17, 21:56

Commit message

It prevents calling retrieveFileAndFoldersInPath twice with the same parameters
when calling get[Files|Folders]InFolder directly after count[Files|Folders]InFolder.

Review

  • Code review CR 0
  • Verified V 0

Issues 1

Files 1

Review log 35

Show 10 earlier messages
  1. Michael Oehlhof uploaded patch set 1 07/02/15, 19:59

  2. Andreas Kienast commented on patch set 1 CR -1 07/03/15, 05:13

    I guess the variable `$folderNameFilterCallbacks` must be taken into account, too, as this array may change the list.
  3. Andreas Kienast uploaded patch set 2 07/03/15, 05:18

  4. Alexander Opitz commented on patch set 2 CR -1 07/03/15, 06:49

    LocalDriver.php:498

    md5(json_encode(func_get_args())) has the same result in getFoldersInFolder and in getFilesInFolder for same parameters. But both have a different meaning and content. Also Both function will get duplicated code. IMHO it would be better if we add a getDirectoryItemListCached function.
  5. Andreas Kienast commented on patch set 2 CR -1 07/03/15, 07:09

    You're right, I completely overlooked that.
  6. Michael Oehlhof uploaded patch set 3 07/03/15, 18:49

  7. Alexander Opitz voted on patch set 3 V +1 CR +1 07/06/15, 08:42

  8. Andreas Kienast voted on patch set 3 CR +1 07/06/15, 13:19

  9. Markus Klein commented on patch set 3 CR -1 07/06/15, 18:56

    When do you invalidate the cache?
  10. Michael Oehlhof commented on patch set 3 07/07/15, 21:03

    I think we do not need invalidation, because the cache exists only during the runtime of the script. Every request starts with an empty cache.
  1. Markus Klein commented on patch set 3 07/07/15, 21:23

    Sure, but can you make sure that the files do not change during one request?
  2. Alexander Opitz commented on patch set 3 V 0 CR 0 07/08/15, 08:33

    Hmmm right, reread directory after adding a file is an issue with this solution.
  3. Stefan Froemken commented on patch set 3 07/14/15, 11:11

    So what should we implement? A new MySQL-Cache? How should it work? Should we flush cache with each request to work like current memory solution? Should we create a cache to keep the information as long as possible in cache? Be careful. Last solution needs a fine tuned caching logic.
  4. Alexander Opitz commented on patch set 3 07/15/15, 10:21

    Maybe come back to the issue and search the solution there? How about not calling count and get*InFolder?
  5. Stefan Froemken commented on patch set 3 07/15/15, 18:58

    "back to the issue" haha. I don't find any "count" in this patch, so I don't get your idea. I have similar problems with current sfdropbox from github, so I'm very interessted in your solution.
  6. Alexander Opitz commented on patch set 3 07/15/15, 21:24

    Cause the patch tries to cache the calls which are behind the count and get*InFolder functions. If you read the issue for this patch then you maybe get a hint.
  7. Michael Oehlhof uploaded patch set 4 07/19/15, 17:03

  8. Alexander Opitz commented on patch set 4 CR -1 07/28/15, 08:21

    Again, I don't think this is the solution. Subprocesses which change the files aren't respected.
  9. Markus Klein commented on patch set 4 07/28/15, 10:15

    @opitz: what do you mean with subprocesses? what did we overlook?
  10. Alexander Opitz commented on patch set 4 07/28/15, 12:12

    Now, discussed this with Markus. We should add a new API fucntion like "clearDriverCache()" which should reset such caches on the specific driver.
  11. Benni Mack wrote 08/12/15, 20:21

    Topic set to FAL
  12. Frank Nägler commented on patch set 4 CR -1 10/05/15, 07:41

    hmm, can you please provide some data about the performance improvement? is it really faster? and how much times faster? We have so much caches and I don't think we need it in all cases. Please create some measuring or a test script to measure the performance with different PHP versions.
  13. Oliver Hader uploaded patch set 5 10/09/15, 00:09

  14. Alexander Opitz commented on patch set 5 10/09/15, 10:20

    @Frank It's only a runtime cache ... and it will be around 50% faster.
  15. Markus Klein commented on patch set 5 10/09/15, 10:33

    I still consider this a valuable addition. The problem though is that we suggest to extend the driver interface with a clearcache() functionality (much like the file system on OS level, which has a stat_cache) and this breaking change can't go into CMS 7 anymore, which is a pity. I'm sorry I didn't pay more attention to this patch, but I was way to busy with other stuff.
  16. Michael Oehlhof uploaded patch set 6 11/01/15, 12:54

  17. Michael Oehlhof uploaded patch set 7 11/01/15, 12:57

  18. Michael Oehlhof commented on patch set 7 11/01/15, 13:01

    Make the patch more simple to only handle the use case of the forge item. It prevents executing of the sortDirectoryEntries function when the get*InFolder function follows directly after the count*InFolder function with the same parameters.
  19. Markus Klein commented on patch set 7 11/01/15, 20:00

    I fully understand the idea, but I'm still worried about retrieveFileAndFoldersInPath() also being called in moveFolderWithinStorage and renameFolder
  20. Benni Mack commented on patch set 7 11/29/15, 21:39

    master-only, as I consider this one non-urgent / non-critical for 6.2
  21. Frank Nägler commented on patch set 7 12/14/15, 13:15

    LocalDriver.php:548

    maybe I don't get it now, but why reset the key here? Also imho the cache should be implemented in this way: if (!empty($this->localCache[$cacheKey])) { $this->localCache[$cacheKey] }

    LocalDriver.php:586

    should be: $this->localCache[$cacheKey] = $this->sortDirectoryEntries($directoryEntries, $sort, $sortRev);

    LocalDriver.php:587

    return $this->localCache[$cacheKey];
  22. Stefan Neufeind commented on patch set 7 11/15/16, 15:07

    Benni decided for master-only
  23. Markus Sommer commented on patch set 7 12/03/16, 15:37

    Ping. :) is this patch relevant for 8 Master?
  24. Markus Klein commented on patch set 7 10/19/17, 21:56

    LocalDriver.php:546

    Finding a correct cache key is actually really hard here. Because not only the paramters matter, but the cache item must also be evicted if the underlying resource changes - aka the file system. This is impossible here, or at least rather insane to do efficiently.
  25. Markus Klein abandoned the change 10/19/17, 21:56

    Due to the major conceptional issues we face here and the lack of activity over 2 years, I hereby abandon this change request.