Review main Merged

67241[TASK] Refactor low level console commands to avoid full boot

Benjamin Franzke CR +2 V +2 Hard +298 −165 updated 02/19/21, 21:27

Commit message

This will allow for low level commands like cache:warmup (or flush)
to be implemented.
Commands that're registered in (internal!) service providers are now
defined to be lowlevel. That allows to change behaviour for internal
commands only, without being breaking.

Low level commands…
 * will only be defined by TYPO3 core, not by third party extensions
   or libraries (despite by using internal interfaces)
 * will not use any $GLOBALS in constructor
 * will only inject dependencies defined by service providers
 * must not require a database connection during construction
 * will only be defined in internal service providers
 * may use the internal BootService to bootstrap to a certain
   boot level

Regular commands…
 * are allowed to access TYPO3_CONF_VARS in constructor
 * are allowed to inject any service via constructor
 * are allowed to access TCA in constructor
 * may perform database calls in constructors (although this is not
   recommended)
 * may fail during construction, e.g. because of a stale DI container
   (an upcoming low level cache:flush/warmup command will be
   provided for such scenarios)

The command list `bin/typo3 list`…
 * will show all low level commands
 * will degrade to low level only commands if the DI
   container is not loadable

The upgrade command therefore now runs fully uncached and does not
require a full boot upfront.

The symfony command `help` is treated as non low level command,
as command constructors need to be executed in order for arguments
to be configured and available for help output. Therefore a full
boot is required. This causes the symfony DI container,
ext_localconf and ext_tables to be loaded.
That means: An invalid container cache will cause help to fail,
`list` will degrade do lowlevel commands, but an
(upcoming) low level console command cache:flush/warmup will
still be invokable and is intended for scenarios where 3rd party
code is changed and therefore DI/localconf/TCA cache became stale.

`typo3/cms-cli`, which provides vendor/bin/typo3 in composer mode,
is adapted in: https://github.com/TYPO3/cms-cli/pull/5

Commands used for updating typo3/cms-cli:

  composer require typo3/cms-cli:^3.0
  composer require typo3/cms-cli:^3.0 \
    --no-update --working-dir=typo3/sysext/core

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Helmut Hummel +1
  • Verified V +2 Done
    • Benni Mack +2
    • Helmut Hummel +1
    • TYPO3com +1 CI
    • core-ci +1 CI

Issues 2

Files 13

Review log 2754

Show 1229 earlier messages
  1. Benjamin Franzke uploaded patch set 1 12/22/20, 13:11

  2. TYPO3com voted on patch set 1 V -1 12/22/20, 13:20 Completed build in 08m 48s on Tue, 22 Dec, 01:20 PM Test Summary: 32540 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-8432

  3. Benjamin Franzke uploaded patch set 2 12/22/20, 13:39

  4. TYPO3com voted on patch set 2 V -1 12/22/20, 13:48 Completed build in 09m 10s on Tue, 22 Dec, 01:48 PM Test Summary: 1 of 32781 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-8433

  5. TYPO3com commented on patch set 2 12/22/20, 14:09 Completed build in 09m 42s on Tue, 22 Dec, 02:09 PM Test Summary: 1 of 32781 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-8434

  6. TYPO3com voted on patch set 2 V +1 12/22/20, 19:42 Completed build in 08m 48s on Tue, 22 Dec, 07:42 PM Test Summary: 32540 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-8442

  7. Benjamin Franzke uploaded patch set 3 12/28/20, 15:14

  8. TYPO3com voted on patch set 3 V +1 12/28/20, 15:25 Completed build in 09m 56s on Mon, 28 Dec, 03:25 PM Test Summary: 32540 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-8472

  9. Benjamin Franzke uploaded patch set 4 12/28/20, 21:07

  10. Benjamin Franzke uploaded patch set 5 12/28/20, 21:10

  11. TYPO3com voted on patch set 4 V +1 12/28/20, 21:17 Completed build in 09m 48s on Mon, 28 Dec, 09:17 PM Test Summary: 32540 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-8481

  12. TYPO3com voted on patch set 5 V +1 12/28/20, 21:20 Completed build in 09m 56s on Mon, 28 Dec, 09:20 PM Test Summary: 32540 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-8482

  13. Benjamin Franzke uploaded patch set 6 01/26/21, 20:06

  14. TYPO3com voted on patch set 6 V +1 01/26/21, 20:17 Completed build in 10m 53s on Tue, 26 Jan, 08:17 PM Test Summary: 32601 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9074

  15. Benjamin Franzke uploaded patch set 7 02/04/21, 14:50

  16. TYPO3com voted on patch set 7 V -1 02/04/21, 14:50 Completed build in 18s on Thu, 4 Feb, 02:50 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9263

  17. Benjamin Franzke uploaded patch set 8 02/04/21, 16:31

  18. TYPO3com voted on patch set 8 V -1 02/04/21, 16:31 Completed build in 20s on Thu, 4 Feb, 04:31 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9265

  19. core-ci voted on patch set 8 V +1 02/04/21, 16:43 Core CI is happy: https://core.b13.com/typo3-core/typo3.cms/-/pipelines/380

  20. Benjamin Franzke uploaded patch set 9 02/05/21, 00:15

  21. TYPO3com voted on patch set 9 V -1 02/05/21, 00:15 Completed build in 19s on Fri, 5 Feb, 12:15 AM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9270

  22. Benjamin Franzke uploaded patch set 10 02/05/21, 00:16

  23. TYPO3com voted on patch set 10 V -1 02/05/21, 00:17 Completed build in 22s on Fri, 5 Feb, 12:17 AM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9271

  24. core-ci voted on patch set 9 V +1 02/05/21, 00:26 Core CI is happy: https://core.b13.com/typo3-core/typo3.cms/-/pipelines/387

  25. core-ci voted on patch set 10 V +1 02/05/21, 00:32 Core CI is happy: https://core.b13.com/typo3-core/typo3.cms/-/pipelines/388

  26. Benjamin Franzke uploaded patch set 11 02/05/21, 12:32

  27. TYPO3com voted on patch set 11 V -1 02/05/21, 12:33 Completed build in 01m 07s on Fri, 5 Feb, 12:33 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9293

  28. core-ci voted on patch set 11 V +1 02/05/21, 12:42 Core CI is happy: https://core.b13.com/typo3-core/typo3.cms/-/pipelines/410

  29. Helmut Hummel commented on patch set 11 02/09/21, 09:52

    CommandApplication.php:119

    This check will always fail when abbreviated command names are used. e.g. "bin/typo3 du" will result in a full boot for the dumpautoload command The only idea to solve that is to let the command registry know whether the command wnats full boot or not and then boot right before running a command (with implementing an console Application class)

    CommandApplication.php:130

    this isn't and never will be true for TYPO3. I think we can skip that check
  1. Benjamin Franzke uploaded patch set 12 02/09/21, 15:34

  2. Benjamin Franzke commented on patch set 12 02/09/21, 15:34

    CommandApplication.php:119 Reply

    Fixed by booting the DI container if an exact low-level command match cannot be found. If `Application::find()` then returns a low level command, ext_localconf is not loaded. That means `bin/typo3 dumpautoload` is "safer" than `bin/typo3 du`, because latter may fail if the container is broken. But I think that is fine, as people (at least I do), will use full command names in scripts/deployments/etc…

    CommandApplication.php:130 Reply

    Ack
  3. TYPO3com voted on patch set 12 V +1 02/09/21, 15:42 Completed build in 07m 57s on Tue, 9 Feb, 03:42 PM Test Summary: 32597 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9402

  4. core-ci voted on patch set 12 V +1 02/09/21, 15:45 Core CI is happy: https://core.b13.com/typo3-core/typo3.cms/-/pipelines/539

  5. Benjamin Franzke uploaded patch set 13 02/10/21, 05:26

  6. TYPO3com voted on patch set 13 V +1 02/10/21, 05:35 Completed build in 09m 27s on Wed, 10 Feb, 05:35 AM Test Summary: 32597 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9426

  7. core-ci voted on patch set 13 V +1 02/10/21, 05:38 Core CI is happy: https://core.b13.com/typo3-core/typo3.cms/-/pipelines/566

  8. Benjamin Franzke uploaded patch set 14 02/12/21, 20:47

    Resolve Merge Conflict
  9. TYPO3com voted on patch set 14 V +1 02/12/21, 20:56 Completed build in 09m 08s on Fri, 12 Feb, 08:56 PM Test Summary: 32611 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9512

  10. core-ci voted on patch set 14 V +1 02/12/21, 21:00 Core CI is happy: https://core.b13.com/typo3-core/typo3.cms/-/pipelines/698

  11. Benni Mack commented on patch set 14 02/16/21, 07:35

    BootService.php:37

    you can use PHP 7.4 types already here.
  12. Benjamin Franzke uploaded patch set 15 02/16/21, 08:11

  13. Benjamin Franzke commented on patch set 15 02/16/21, 08:11

    BootService.php:37 Reply

    Ack (background: As this class is moved from EXT:install to EXT:core I kept it as is – but you're right, it can be adapted).
  14. TYPO3com voted on patch set 15 V +1 02/16/21, 08:18 Completed build in 06m 51s on Tue, 16 Feb, 08:18 AM Test Summary: 32621 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9561

  15. core-ci voted on patch set 15 V +1 02/16/21, 08:24 Core CI is happy: https://core.b13.com/typo3-core/typo3.cms/-/pipelines/762

  16. Benni Mack voted on patch set 15 V +1 CR +1 02/17/21, 21:12

  17. Helmut Hummel commented on patch set 15 02/19/21, 15:04

    Looks great already. Have some minor remarks, though
    This isn't true any more, right?

    CommandApplication.php:88

    This variable isn't used, is it?

    UpgradeWizardRunCommand.php:81

    Make this part of the BootService API (basically include this call into initializeBackendAuthentication)? Or do we have a use case to only call one but not the other?

    UpgradeWizardRunCommand.php:82

    This is part of the BootService API. Maybe use it here?

    LateBootService.php:28

    Unused?
  18. Benjamin Franzke uploaded patch set 16 02/19/21, 20:16

  19. Benjamin Franzke commented on patch set 16 02/19/21, 20:16

    Reply

    Oh, yes, true. Rephrased.

    CommandApplication.php:88 Reply

    right, leftover from a previous patchset. Removed.

    UpgradeWizardRunCommand.php:81 Reply

    I thought about adding this to the BootService, but wasn't sure how, because currently this is also called in CommandApplication, and ideally I'd like to remove it there (for lowlevel commands). I furthermore don't know why this is added at all here. I'd like this as task for another patch/cleanup. So far the history, I then forgot that I added initializeBackendAuthentication() to BootService, but didn't actually use it anywhere. The problem was that the BootService is not just though for CLI but also for install tool, so I don't want to hardcode CommandLineUserAuthentication there. Would you be ok when I leave this as-is and rather remove unintended initializeBackendAuthentication from BootService? I'd like to sort the backendUser handling out later, and I'd be fine if someone added backendUser/Auth Handling to the BootService.

    UpgradeWizardRunCommand.php:82 Reply

    Removed BootService->initializeBackendAuthentication, per comment for line 81.

    LateBootService.php:28 Reply

    Ack, removed.
  20. TYPO3com voted on patch set 16 V +1 02/19/21, 20:24 Completed build in 07m 56s on Fri, 19 Feb, 08:24 PM Test Summary: 32684 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-9641

  21. core-ci voted on patch set 16 V +1 02/19/21, 20:27 Core CI is happy: https://core.b13.com/typo3-core/typo3.cms/-/pipelines/861

  22. Helmut Hummel commented on patch set 16 02/19/21, 20:35

    UpgradeWizardRunCommand.php:81 Reply

    Ack
  23. Helmut Hummel voted on patch set 16 V +1 CR +1 02/19/21, 20:50

  24. Benni Mack voted on patch set 16 V +2 CR +2 02/19/21, 21:27

  25. Benni Mack merged patch set 17 02/19/21, 21:27