Review main Abandoned

79234[TASK] Avoid expensive preg_match calls in addHash method

Alexander Schnitzler CR 0 V 0 Normal +151 −75 updated 07/06/23, 09:43

Commit message

VariableProcessor::addHash() is called to convert URI path
variables that don't comply with certain rules to a hash,
which does.

First of all, variables must not exceed a certain length
of 32, second of all, variables must only contain word
characters aka "\w" or "[A-Za-z0-9_]".

So whenever a variable is too long or contains invalid
characters, it is converted to an md5 hash which complies
with both rules.

The check for length is not a problem but the check for
allowed characters via preg_match is. It's expensive and
given that addHash() is called no matter if a hash already
exists or not, preg_match is possibly peformed many hundred
times on the same value.

The fewer routes and route aspects defined, the smaller
the impact. But beyond a certain threshold, this is very
noticable.

To avoid this issue, all values are hashed and an early
return will avoid recalculating existing hashes again
and again.

Review

  • Code review CR 0
  • Verified V 0

Issues 1

Files 2

Review log 1014

  1. Alexander Schnitzler uploaded patch set 1 06/08/23, 11:21

  2. core-ci voted on patch set 1 V -1 06/08/23, 11:33 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/42297

  3. Alexander Schnitzler uploaded patch set 2 V -1 06/08/23, 19:30

  4. core-ci voted on patch set 2 V -1 06/08/23, 19:41 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/42322

  5. Oliver Hader commented on patch set 2 06/27/23, 12:19

    VariableProcessor.php:41

    It means, that any value that is lower than 31 characters (that's a `symfony/routing` limit) which at the same time does not contain any non-word chars are used as-is (allowed are `a-zA-Z0-9_`, disallowed is anything else, like `-` or `@`). The current change would hash all parameter names.
  6. Oliver Hader uploaded patch set 3 V -1 06/27/23, 12:56

  7. Oliver Hader commented on patch set 3 06/27/23, 12:59

    With hashing all parameters, processing time might be faster - however, memory consumption would increase. Anyway, I've adjusted the patch - test should be green again.

    VariableProcessor.php:41 Reply

    ...
  8. core-ci voted on patch set 3 V -1 06/27/23, 13:06 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/43030

  9. Oliver Hader uploaded patch set 4 V -1 06/27/23, 13:27

  10. Oliver Hader commented on patch set 4 06/27/23, 13:29

    Reply Unresolved

    Adjusted the unit-tests as well - basically nobody would understand that anymore, debugging in `VariableProcessor` becomes more challenging with this patch. Alternative: Just use a check whether a hash has already created in `addHash()`.
  11. core-ci voted on patch set 4 V -1 06/27/23, 13:41 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/43031

  12. Oliver Hader commented on patch set 4 06/27/23, 14:48

    The title of the forge issue was outlining an alternative approach. I've created an alternative patch that uses a shared runtime cache - this way, the necessity to hash and the hash-sum of a value are only processed once for each value. → https://review.typo3.org/c/Packages/TYPO3.CMS/+/79522
  13. Alexander Schnitzler commented on patch set 4 07/06/23, 09:43

    Reply

    Thank you very much! I will benchmark my project without and with your patch next week.
  14. Alexander Schnitzler abandoned the change 07/06/23, 09:43

    Not needed any more.