78787[TASK] Concurrently fetch URLs in HashProxy
Commit message
Review
-
Code review CR +2 Done
- Andreas Kienast +2
- Georg Ringer +1
-
Verified V +2 Done
- Andreas Kienast +2
- Georg Ringer +1
- core-ci +1 CI
Issues 1
Files 9
- typo3/sysext/backend/Classes/Middleware/ContentSecurityPolicyHeaders.php+3 −1
- typo3/sysext/backend/Configuration/Services.yaml+4 −0
- typo3/sysext/core/Classes/Security/ContentSecurityPolicy/HashProxy.php6+60 −17
- typo3/sysext/core/Classes/Security/ContentSecurityPolicy/HashValue.php+3 −1
- typo3/sysext/core/Classes/Security/ContentSecurityPolicy/ModelService.php+9 −1
- typo3/sysext/core/Classes/Security/ContentSecurityPolicy/Policy.php+4 −2
- typo3/sysext/core/Classes/Security/ContentSecurityPolicy/SourceValueInterface.php+3 −1
- typo3/sysext/frontend/Classes/Middleware/ContentSecurityPolicyHeaders.php+3 −1
- typo3/sysext/frontend/Configuration/Services.yaml+4 −0
Review log 3147
Show 1522 earlier messages
-
Christian Eßl uploaded patch set 1 04/21/23, 14:58
-
Christian Eßl uploaded patch set 2 04/21/23, 15:09
-
core-ci voted on patch set 1 V -1 04/21/23, 15:16 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41044
-
Christian Eßl rebased onto patch set 3 04/21/23, 15:18
-
Christian Eßl uploaded patch set 4 04/21/23, 15:20
-
core-ci voted on patch set 2 V -1 04/21/23, 15:26 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41045
-
core-ci voted on patch set 3 V -1 04/21/23, 15:26 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41046
-
Oliver Klee commented on patch set 4 04/21/23, 15:48
-
core-ci voted on patch set 4 V -1 04/21/23, 15:51 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41048
-
Christian Eßl uploaded patch set 5
V -104/21/23, 16:06 -
Christian Eßl uploaded patch set 6 04/21/23, 16:07
-
Christian Eßl commented on patch set 6 04/21/23, 16:08
HashProxy.php:178ReplyDone -
Christian Eßl uploaded patch set 7 04/21/23, 16:14
-
core-ci voted on patch set 6 V +1 04/21/23, 16:25 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41054
-
Oliver Klee commented on patch set 7 04/21/23, 16:29
Do you have any numbers on how much this improves things? -
core-ci voted on patch set 7 V +1 04/21/23, 16:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41056
-
Christian Eßl commented on patch set 7 04/21/23, 16:55
Reply
At the moment it is just theoretical, as the resource contents of files etc. afaik aren't compiled yet in the Content-Security-Policy feature. But when HashProxies are fully used in the future, it would depend on the number of files in the frontend/backend, that use such a hash source. -
Oliver Hader commented on patch set 7 04/21/23, 20:36
HashProxy.php:179What do you think about using `GuzzleHttp\Pool` instead to limit the max. number of concurrent request (to e.g. 10 or 20)? -
Oliver Hader commented on patch set 7 04/21/23, 20:57
I don't know, how intense this will be used in TYPO3 projects... but I think introducing a local cache for "hash per URI" would make sense to avoid those recurring URI fetches. -
Oliver Hader uploaded patch set 8
V +104/21/23, 21:00 -
core-ci voted on patch set 8 V +1 04/21/23, 21:08 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41065
-
Oliver Hader commented on patch set 8 04/21/23, 21:12
How to test: see https://forge.typo3.org/issues/100708#note-9 Testes with `hey`: * before: Average: 0.9083 secs, Requests/sec: 1.9180 * after: Average: 0.7375 secs, Requests/sec: 2.2789
-
Oliver Klee wrote 04/21/23, 21:54
Removed cc Oliver Klee. -
Oliver Hader commented on patch set 8 04/21/23, 22:33
I've been working on a cache (much better results): `Average: 0.0365 secs, Requests/sec: 54.8154` -
Oliver Hader uploaded patch set 9
V +104/21/23, 22:34 -
Oliver Hader uploaded patch set 10 04/21/23, 22:35
-
Oliver Hader commented on patch set 10 V +1 CR +1 04/21/23, 22:37
Reply
DoneHashProxy.php:179Replyskipped -
core-ci voted on patch set 10 V -1 04/21/23, 22:55 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41068
-
core-ci voted on patch set 9 V -1 04/21/23, 22:55 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41067
-
core-ci commented on patch set 10 04/21/23, 23:03 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41068
-
Oliver Hader uploaded patch set 11
CR +1V +104/21/23, 23:07 -
core-ci voted on patch set 11 V -1 04/22/23, 00:15 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41071
-
Oliver Hader rebased onto patch set 12
V -104/22/23, 05:50 -
core-ci voted on patch set 12 V -1 04/22/23, 06:06 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41078
-
Oliver Hader uploaded patch set 13
V -104/22/23, 06:27 -
core-ci voted on patch set 13 V +1 04/22/23, 06:35 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41080
-
Oliver Hader rebased onto patch set 14
V +104/22/23, 08:14 -
Oliver Hader commented on patch set 15 04/22/23, 08:20
Published edit on patch set 14. -
core-ci voted on patch set 14 V +1 04/22/23, 08:22 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41082
-
core-ci voted on patch set 15 V +1 04/22/23, 08:27 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41083
-
Oliver Hader uploaded patch set 16
V +104/24/23, 07:24 -
Georg Ringer voted on patch set 16 V +1 CR +1 04/24/23, 07:34
-
core-ci voted on patch set 16 V +1 04/24/23, 08:02 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/41121
-
Andreas Kienast commented on patch set 16 04/24/23, 08:05
HashProxy.php:184If we'd filter out null values here, we could drop the whole null handling in the loop and in the `compileHashValues()` method (line 177). -
Oliver Hader commented on patch set 16 04/24/23, 08:09
HashProxy.php:184ReplyIf we'd do this, invalid/non-existing URLs would be fetched again and again with each request. This way `null` (not found) is written to the cache. -
Andreas Kienast voted on patch set 16 V +2 CR +2 04/24/23, 08:22
-
Andreas Kienast merged patch set 17 04/24/23, 08:22
HashProxy.php:178