72048[BUGFIX] Fix too long regular expression error in ContentObjectRenderer
Commit message
Review
-
Code review CR +2 Done
- Christian Kuhn +2
- Nikita Hovratov +1
-
Verified V +2 Done
- Christian Kuhn +2
- Nikita Hovratov +1
- core-ci +1 CI
Issues 1
Files 2
Review log 3341
Show 1216 earlier messages
-
Anja Leichsenring uploaded patch set 1 11/04/21, 11:24
-
core-ci voted on patch set 1 V -1 11/04/21, 11:46 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18867
-
Stefan Bürk uploaded patch set 2 11/04/21, 12:53
-
Stefan Bürk commented on patch set 2 CR -1 11/04/21, 12:55
One first solution - but not sure, if it is really good to do it that way to prevent the compilation error - or if there are untested/unregonized sideeffects. Maybe someone can take this as a startpoint - can look into it further after work. -
Stefan Bürk wrote 11/04/21, 12:56
Removed Code-Review-1 by Stefan Bürk <stefan@buerk.tech> -
Stefan Bürk voted on patch set 2 CR -1 11/04/21, 12:56
-
core-ci voted on patch set 2 V +1 11/04/21, 13:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18878
-
Simon Gilli uploaded patch set 3 11/04/21, 15:00
-
Anja Leichsenring commented on patch set 3 11/04/21, 15:07
ContentObjectRenderer.php:3046why 1000? The value has been randomly chosen by me to be 'above breaking point'. The highest value no error is raised is 962.ContentObjectRenderer.php:3050while you are on it: queal should be equal ;) -
Stefan Bürk commented on patch set 3 11/04/21, 15:12
ContentObjectRenderer.php:3046ReplyIn the issue was something with little over 1000. As it was a first try in ~15mins in the launch break. So not take this as the real number. Just saving a first try. Simon has some more insights, and talking with him - he has a play. But one point would be to determine on which value this must be done etc.ContentObjectRenderer.php:3050ReplyAck. Sure ;) -
core-ci voted on patch set 3 V -1 11/04/21, 15:20 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18896
-
Simon Gilli commented on patch set 3 CR -1 11/04/21, 15:21
Stefan and I are on it. So fare I have fixed the tests to show the issue with the current solution. We will hopefully come up with the right solution this evening... The rest is discussed at Slack for the time beeing. And another side note, the best solution here would probably be using a DOM parser and not abuse regex for it. But that's another story so for the moment it's fine to just fix the current code. -
Jonas Eberle commented on patch set 3 11/05/21, 12:44
Reply
Maybe \TYPO3\CMS\Core\Html\HtmlParser can help here to simplify the logic in that function? -
Stefan Bürk uploaded patch set 4 11/05/21, 15:40
-
core-ci voted on patch set 4 V +1 11/05/21, 15:58 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18960
-
Stefan Bürk uploaded patch set 5 11/05/21, 17:59
-
Stefan Bürk commented on patch set 5 11/05/21, 18:02
Added fix on the right place now - took little longer, as it was a little tricky.ContentObjectRenderer.php:3046ReplyDoneContentObjectRenderer.php:3050ReplyDone -
Stefan Bürk wrote 11/05/21, 18:04
Removed reviewer Stefan Bürk. -
core-ci voted on patch set 5 V +1 11/05/21, 18:20 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18966
-
Simon Gilli voted on patch set 5 V +1 CR +1 11/06/21, 10:04
-
Christian Kuhn updated the commit message in patch set 6 11/09/21, 17:18
-
Christian Kuhn voted on patch set 6 V +1 CR +1 11/09/21, 17:18
-
core-ci voted on patch set 6 V +1 11/09/21, 17:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/19078
-
Nikita Hovratov commented on patch set 6 11/10/21, 00:59
ContentObjectRenderer.php:3072This is the culprit btw (the pipe). This causes some sort of backtracking states (not an regex expert, just read some stackoverflow: https://stackoverflow.com/questions/33976303/php-why-am-i-being-warned-that-my-regular-expression-is-too-large/33977966#33977966). I guess this causes us to reach this 64k limit fairly easy. Probably also related to this 962 value. Best solution would be to get rid of it somehow. Have played around a bit, but it's not so easy to satisfy all test cases with a new implementation. -
Nikita Hovratov commented on patch set 6 11/10/21, 03:18
ContentObjectRenderer.php:3071Should we use mb_substr here? -
Stefan Bürk commented on patch set 6 11/10/21, 07:27
ContentObjectRenderer.php:3071ReplySure, thanks for pointing out. Will change that.ContentObjectRenderer.php:3072ReplyPlayed arround with regex and other options too for some hours. The first group is there to find the htmlentitie styles and count them as 1 instead of the full length, which compiles in a broader way. And in combanitation with the "u" modifier flag for unicode this get's even worser. This whole method should get a refactor as already mentioned, as doing these regexp things on html is always nasty. I do not see to get a regexp here which do which has do be done and don't reach the limit. Normal way would be to splitt each group part out and do it on the own, which would make it even worser. I would say, that the current solution with splitting the to chunks is a solid working solution for now (as bugfix) - and gives us time to implement a working refactor/reimplemantion. -
Stefan Bürk uploaded patch set 7 11/10/21, 07:43
-
Stefan Bürk commented on patch set 7 11/10/21, 07:46
ContentObjectRenderer.php:3071ReplyDone -
core-ci voted on patch set 7 V +1 11/10/21, 07:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/19102
-
Nikita Hovratov commented on patch set 7 11/10/21, 09:21
ContentObjectRenderer.php:3072ReplyYeah, this is what I understood too. This can't be fixed with another regex I think. The chunk solution is a valid bugfix, thanks for that! I maybe have a clean(er) solution, at least all test cases are green now. The only regex is now to find html entities. Not sure about unicode though. Neverheless, let's get this bugfix in first as it's also for v10. And then we can see, if this can be improved on master. -
Stefan Bürk commented on patch set 7 11/10/21, 10:09
ContentObjectRenderer.php:3072ReplyI think having that chunk solution as working (first) would be a good thing, as you said. Anyway I'm interesseted in your solution .. may you share your "regexp" here as comment - or maybe ping me on slack ? -
Nikita Hovratov voted on patch set 7 V +1 CR +1 11/10/21, 10:12
-
Stefan Bürk commented on patch set 7 11/10/21, 11:06
ContentObjectRenderer.php:3072Reply^ done. We take this as safe solution. -
Stefan Bürk commented on patch set 7 11/10/21, 11:07
ContentObjectRenderer.php:3072Replydone -
Sybille Peters commented on patch set 7 11/10/21, 12:20
Just a minor comment: It sounds like we are fixing preg_match here ;-) Maybe "Fix error using preg_match" or something. -
Stefan Bürk updated the commit message in patch set 8 11/10/21, 12:24
-
Stefan Bürk commented on patch set 8 11/10/21, 12:25
Reply
Done -
core-ci voted on patch set 8 V +1 11/10/21, 12:36 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/19123
-
Nikita Hovratov voted on patch set 8 V +1 CR +1 11/10/21, 13:36
-
Christian Kuhn voted on patch set 8 V +2 CR +2 11/10/21, 18:57
-
Christian Kuhn merged patch set 9 11/10/21, 18:57