Review main Merged

66241[BUGFIX] Make sure external link checks time out

Sybille Peters CR +2 V +2 Normal +172 −0 updated 11/20/20, 15:57

Commit message

The HTTP requests for checking external links did not have a
timeout. The timeout from Global Configuration was used
which is set to 0 (not timeout) by default.

This could cause the link checking to run
indefinitely and scheduler tasks to never terminate.

Usually, there is a connect timeout, a read timeout
and a total timeout. We set the total timeout and make
this configurable via page TSconfig.

If this is not set via TSconfig, it will default to the
value set in $GLOBALS['TYPO3_CONF_VARS']['HTTP']['timeout'].

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Christian Kuhn +1
  • Verified V +2 Done
    • Benni Mack +2
    • Sybille Peters +1
    • Christian Kuhn +1
    • TYPO3com +1 CI

Issues 1

Files 5

Review log 2229

Show 34 earlier messages
  1. Sybille Peters uploaded patch set 1 10/21/20, 17:32

  2. TYPO3com voted on patch set 1 V +1 10/21/20, 17:50 Completed build in 15m 11s on Wed, 21 Oct, 05:50 PM Test Summary: 73488 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-6757

  3. Chris Müller voted on patch set 1 V +1 CR +1 10/22/20, 17:32

  4. Oli Bartsch commented on patch set 1 V -1 10/22/20, 20:25

    Introducing new configuration should get at least an important rst (especially when backporting to older branches).
  1. Sybille Peters commented on patch set 1 10/23/20, 08:12

    > Patch Set 1: Verified-1 > > Introducing new configuration should get at least an important rst (especially when backporting to older branches). I can do that. Even with "Important" - I am also concerned this patch will not get merged at all. Incidentally, I believe there used to be a configuration for a timeout (not in linkvalidator, but in the core), that was before Guzzle see documentation: > You are advised to use cURL to check for links. Linkvalidator uses the HTTP request library shipped with TYPO3. Please have a look in the Install Tool at the section “All Configuration” which includes an HTTP section at the end. > There you may define a default timeout and you may change from using sockets to using the cURL library. https://docs.typo3.org/c/typo3/cms-linkvalidator/10.4/en-us/Administration/Index.html I will handle documentation updates and fixes separately.
  2. Sybille Peters commented on patch set 1 10/23/20, 08:25

    It might also be a possibility to add the timeout with default 10s and backport this and add the configuration option only in master. I think having a default timeout of 10s is a good solution - it is long enough so it should not cause problems, but it will solve the very nasty problem of (possibly) indefinitely running scheduler tasks. I would even be up for making 60s the default before master and making it configurable and changing default to 10 in master. What do you think, Oliver and Chris?
  3. Sybille Peters commented on patch set 1 10/23/20, 08:55

    Ok, such a core option actually exists: The entire $GLOBALS['TYPO3_CONF_VARS']['HTTP'] array is used in the core GuzzleClientFactory. Setting $GLOBALS['TYPO3_CONF_VARS']['HTTP']['timeout'] should have the same effect as what I did. I should have checked this before. I will check to make sure this is actually used. If that is the case, I must rethink my patch. It might still be a good idea to override a timeout of 0 (no timeout) in global configuration via linkvalidator configuration. But the timeout in global configuration should be considered by default. It might also be enough to point this out in the documentation and change the - now outdated - https://docs.typo3.org/c/typo3/cms-linkvalidator/10.4/en-us/Administration/Index.html
  4. Sybille Peters uploaded patch set 2 10/23/20, 10:43

  5. Sybille Peters uploaded patch set 3 10/23/20, 10:48

  6. Sybille Peters uploaded patch set 4 10/23/20, 10:53

  7. Sybille Peters updated the commit message in patch set 5 10/23/20, 10:55

  8. TYPO3com voted on patch set 2 V -1 10/23/20, 10:56 Completed build in 12m 39s on Fri, 23 Oct, 10:56 AM Test Summary: 1 of 73949 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-6831

  9. TYPO3com voted on patch set 3 V +1 10/23/20, 11:01 Completed build in 13m 01s on Fri, 23 Oct, 11:01 AM Test Summary: 73464 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-6832

  10. TYPO3com voted on patch set 4 V +1 10/23/20, 11:08 Completed build in 14m 51s on Fri, 23 Oct, 11:08 AM Test Summary: 73462 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-6833

  11. TYPO3com voted on patch set 5 V +1 10/23/20, 11:12 Completed build in 15m 46s on Fri, 23 Oct, 11:12 AM Test Summary: 73464 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-6834

  12. Chris Müller voted on patch set 5 V +1 CR +1 10/23/20, 13:00

  13. Sybille Peters wrote 10/23/20, 17:16

    Topic set to Linkvalidator
  14. Ingo Fabbri voted on patch set 5 CR +1 10/24/20, 12:26

  15. Anja Leichsenring commented on patch set 5 V -1 11/20/20, 14:17

    at least a (int) cast is missing ... we are on it.
  16. Sybille Peters commented on patch set 5 11/20/20, 14:20

    ah, thanks for catching that.
  17. Christian Kuhn uploaded patch set 6 11/20/20, 14:44

  18. Christian Kuhn commented on patch set 6 11/20/20, 14:45

    removed 9.5 from release list (imho not a critical bug) added 'int' property type hint - MUST be removed for v10 backport.

    ExternalLinktype.php:128 Unresolved

    int cast?!
  19. Christian Kuhn uploaded patch set 7 11/20/20, 14:50

  20. TYPO3com voted on patch set 6 V +1 11/20/20, 14:58 Completed build in 13m 23s on Fri, 20 Nov, 02:58 PM Test Summary: 32391 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-7470

  21. TYPO3com voted on patch set 7 V +1 11/20/20, 15:02 Completed build in 10m 48s on Fri, 20 Nov, 03:02 PM Test Summary: 32391 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-7471

  22. Sybille Peters commented on patch set 7 V +1 11/20/20, 15:02

    Just did a quick test - looks good.
  23. Christian Kuhn voted on patch set 7 V +1 CR +1 11/20/20, 15:16

  24. Benni Mack voted on patch set 7 V +2 CR +2 11/20/20, 15:57

  25. Benni Mack merged patch set 8 11/20/20, 15:57