Review main Merged

77232[FEATURE] Adopt symfony messenger as message bus and queue

Susi Moog CR +2 V +2 Expert +1303 −6 updated 01/23/23, 17:40

Commit message

See the Feature.rst file for details.

The changes in workspaces are meant as a usage example.

The goal is to provide a basic implementation as a first step.
Further features (for example extending the capabilities of the
consume command or providing additional transportFactories or
multi-bus support) can be done in additional patches
(or even – in case of transports – in additional packages).

The default transport is synchronous to avoid forcing users to
configure the long running message consumer worker.

Command executed:
 composer req symfony/doctrine-messenger:^6.2 symfony/messenger:^6.2
 composer req symfony/doctrine-messenger:^6.2 symfony/messenger:^6.2 \
   -d typo3/sysext/core --no-update

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Benjamin Franzke +1
    • Daniel Goerz +1
    • Frank Nägler +1
    • Susi Moog +1
  • Verified V +2 Done
    • Benni Mack +2
    • Benjamin Franzke +1
    • core-ci +1 CI

Issues 1

Files 20

Review log 4564

Show 2539 earlier messages
  1. Susi Moog uploaded patch set 1 12/28/22, 08:37

  2. core-ci voted on patch set 1 V -1 12/28/22, 08:49 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35372

  3. Susi Moog uploaded patch set 2 V -1 12/28/22, 12:03

  4. core-ci voted on patch set 2 V -1 12/28/22, 12:14 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35375

  5. Susi Moog uploaded patch set 3 V -1 12/29/22, 08:34

  6. core-ci voted on patch set 3 V +1 12/29/22, 08:46 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35387

  7. Susi Moog uploaded patch set 4 V +1 12/29/22, 08:52

  8. core-ci voted on patch set 4 V +1 12/29/22, 09:00 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35390

  9. Susi Moog uploaded patch set 5 V +1 01/06/23, 07:55

  10. core-ci voted on patch set 5 V +1 01/06/23, 08:02 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/35519

  11. Susi Moog uploaded patch set 6 V +1 01/18/23, 15:50

  12. core-ci voted on patch set 6 V -1 01/18/23, 15:58 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36115

  13. Susi Moog uploaded patch set 7 V -1 01/18/23, 16:21

  14. Susi Moog marked it ready for review 01/18/23, 16:21

  15. core-ci voted on patch set 7 V +1 01/18/23, 16:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36123

  16. Susi Moog commented on patch set 8 V +1 01/18/23, 16:54

    Published edit on patch set 7.
  17. core-ci voted on patch set 8 V +1 01/18/23, 17:02 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36129

  18. Benni Mack commented on patch set 8 01/18/23, 19:44

    DataHandlerHook.php:186

    looks ugly
  19. Georg Ringer commented on patch set 8 01/18/23, 20:23

    ConsumeMessagesCommand.php:2

    declare strict

    MessageHandlerPass.php:39

    could be proctected readonly?

    MessageHandlerPass.php:45

    no need for doc comment

    Feature-97700-AdoptSymfonyMessengerAsAMessageBusAndQueue.rst:14

  20. Frank Nägler uploaded patch set 9 V +1 01/18/23, 20:34

  21. Oliver Hader wrote 01/18/23, 20:43

    Unset private
  22. core-ci voted on patch set 9 V -1 01/18/23, 20:47 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36147

  23. Frank Nägler rebased onto patch set 10 V -1 01/19/23, 08:50

  24. core-ci voted on patch set 10 V -1 01/19/23, 09:01 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36170

  25. Anja Leichsenring uploaded patch set 11 V -1 01/19/23, 09:07

  26. Anja Leichsenring uploaded patch set 12 01/19/23, 09:14

  27. core-ci voted on patch set 11 V +1 01/19/23, 09:14 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36173

  28. Anja Leichsenring commented on patch set 12 01/19/23, 09:14

    I added the declare(strict_types=1); to all classes, leaving the other comments to you, Susi

    ConsumeMessagesCommand.php:2 Reply

    Done

    MessageHandlerPass.php:45 Reply

    Done
  29. core-ci voted on patch set 12 V +1 01/19/23, 09:22 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36174

  30. Frank Nägler voted on patch set 12 V +1 01/19/23, 11:40

  31. Frank Nägler uploaded patch set 13 V +1 01/19/23, 12:42

  32. Frank Nägler uploaded patch set 14 01/19/23, 12:43

  33. Frank Nägler rebased onto patch set 15 01/19/23, 12:56

  34. Georg Ringer commented on patch set 16 01/19/23, 13:05

    Published edit on patch set 15.
  35. core-ci voted on patch set 15 V -1 01/19/23, 13:18 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36237

  36. core-ci voted on patch set 16 V +1 01/19/23, 13:34 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36241

  37. Frank Nägler voted on patch set 16 V +1 CR +1 01/19/23, 15:42

  38. Benni Mack commented on patch set 16 01/19/23, 16:11

    MessageHandlerPass.php:39 Reply

    you mean private readonly?

    DataHandlerHook.php:186 Reply

    Done
  39. Stefan Bürk commented on patch set 16 01/19/23, 23:48

    MessageHandlerPass.php:39 Reply

    Doesn't make much difference in a final class. But would also vote for `private readonly` here. Beside this, can't wie DI the DpendencyOrderingService here too and having it as `private readonly` constructor parameter promotion ?

    BusFactory.php:24

    Should we not make this factory `final` ?

    DoctrineTransportFactory.php:25

    Same, shouldn't we make this factory `final` ?

    HandlersLocatorFactory.php:24

    Shouldn' we make this factory `final` ?

    HandlersLocatorFactory.php:27

    Would add a empty line between proprty and constructor.

    DataHandlerHook.php:57

    missing empty line ... maybe we should move the constructor below the properties ?

    StageChangeMessage.php:28

    We should use private readonly here and add corresponding getters. We are doing this with event classes also, as it makes easier for deprecation at later point. and eventually renamings with keeping old getters with deprecation triggers etc ...
  1. Benni Mack uploaded patch set 17 CR +1 V +1 01/20/23, 16:57

  2. Benni Mack commented on patch set 17 01/20/23, 16:59

    MessageHandlerPass.php:39 Reply

    I chose not to use constructor property promotion here to be consistent with the other property for the time being

    BusFactory.php:24 Reply

    Done

    DoctrineTransportFactory.php:25 Reply

    Done

    HandlersLocatorFactory.php:24 Reply

    Done

    HandlersLocatorFactory.php:27 Reply

    Done

    Feature-97700-AdoptSymfonyMessengerAsAMessageBusAndQueue.rst:14 Reply

    Done

    DataHandlerHook.php:57 Reply

    Done

    StageChangeMessage.php:28 Reply

    Done
  3. Benni Mack uploaded patch set 18 01/20/23, 16:59

  4. core-ci voted on patch set 17 V +1 01/20/23, 17:05 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36375

  5. core-ci voted on patch set 18 V +1 01/20/23, 17:10 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36376

  6. Chris Müller uploaded patch set 19 V +1 01/20/23, 17:21

  7. core-ci voted on patch set 19 V +1 01/20/23, 17:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36377

  8. Frank Nägler commented on patch set 19 V +1 CR +1 01/20/23, 18:02

    still works 😎
  9. Stefan Bürk commented on patch set 20 CR +1 V +1 01/20/23, 21:33

    Published edit on patch set 19.
  10. Stefan Bürk voted on patch set 20 CR +1 01/20/23, 21:36

  11. core-ci voted on patch set 20 V +1 01/20/23, 21:41 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36391

  12. Benjamin Franzke commented on patch set 20 CR -1 01/22/23, 12:48

    on it

    ConsumeMessagesCommand.php:139

    Let's avoid symfonyisms.. (I know this is copied from there, still I'd rather avoid that in our codebase)

    BusFactory.php:47

    MessageBus allows for a IteratorAggregate to be passed (instead of an instantiated middleware array). We should do that in order to lazily instantiate middeware classes only if used.

    TransportLocator.php:72

    Hm, thinking about it we should actually create a (symfony) ServiceLocator for the services tagged as message transport, instead of relying on the name, otherwise we have redundancy in configuration mechanisms (the consumer-command currently uses the tags, while this locator uses a service-name-convention). I'll change that.

    Services.yaml:473

    This should be identifier: 'default'

    StageChangeMessage.php:28 Reply

    I have to say: I strongly disagree. We currently have a mixture of both ways of doing it in core and I don't see any problems with deprecating public readonly properties. They simply will be magic getter/setter then… I mean, the property deprecation is actually the easy part, as you won't be able to deprecate constructor arguments without relying named argument calls. (After all: The constructor is also API in this case).
  13. Benjamin Franzke commented on patch set 20 01/22/23, 13:37

    Services.yaml:477

    We need to hook up this transport to the TYPO3 schema manager, in order to be hooked up in the TYPO3 installtool. Otherwise a removal of the `messenger_messages` table (that is autogenerated by symfony messenger) will be generated: ALTER TABLE `messenger_messages` RENAME TO `zzz_deleted_messenger_messages`
  14. Benjamin Franzke commented on patch set 20 01/22/23, 13:50

    Feature-97700-AdoptSymfonyMessengerAsAMessageBusAndQueue.rst:123

    Hm, i don't get this. Why `default`? `default` should be the SyncTransport, so the argument should be `doctrine`, no? So basically: vendor/bin/typo3 messenger:consume doctrine
  15. Benjamin Franzke commented on patch set 20 01/22/23, 16:09

    Feature-97700-AdoptSymfonyMessengerAsAMessageBusAndQueue.rst:127

    Nice suggestion, yes. We should actually make sure our command is properly usable with systemd by using an exit code that signals that we *want* to be restarted and that we did not fail and were not stopped because the service was stopped (as with `systemd stop typo3-message-consumer`). That'd be something like the following in typo3-message-consumer.service: Restart=on-failure # Restart on exit code 133 (which is returned by the app when receiving SIGUP) RestartForceExitStatus=133 # ..but do not interpret exit code 111 as an error (as is a restart request be the app) SuccessExitStatus=133 From what I saw so far, this is currently not supported by the workers – meeh, but there were similar feature requests for symfony: https://github.com/symfony/symfony/issues/42029 Will look into it, how we could provide it in our command.
  16. Benjamin Franzke uploaded patch set 21 CR +1 V +1 01/22/23, 17:34

  17. Benjamin Franzke commented on patch set 21 01/22/23, 17:34

    ConsumeMessagesCommand.php:139 Reply

    Done

    BusFactory.php:47 Reply

    Done

    TransportLocator.php:72 Reply

    Done

    Services.yaml:473 Reply

    Done

    Services.yaml:477 Reply

    I've disabled autocreation by doctrine now and rather imported the schema into ext_tables.sql and used a custom table name (sys_messenger_messages) which fit's better into our tablename scheme. If someone wants to improve this, by connecting our schema migrator with the doctrine schema provided by symfony/messenger, but for now this should be good.

    Feature-97700-AdoptSymfonyMessengerAsAMessageBusAndQueue.rst:123 Reply

    Changed this now. Please give me a HEADS up in case I misunterstood something in here.

    Feature-97700-AdoptSymfonyMessengerAsAMessageBusAndQueue.rst:127 Reply

    Done

    StageChangeMessage.php:28 Reply

    I've now changed this back to Susi's original code. It really made sense. StageChangeMessage is actually an implementation detail (marked it as internal now), so we can really be nice to ourselves and avoid such ugly getters.
  18. Benjamin Franzke voted on patch set 21 V +1 CR +1 01/22/23, 17:35

  19. core-ci voted on patch set 21 V +1 01/22/23, 17:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/36428

  20. Susi Moog commented on patch set 21 CR +1 01/23/23, 07:21

    Thanks @Benjamin Franzke for the improvements!
  21. Frank Nägler voted on patch set 21 CR +1 01/23/23, 07:34

  22. Daniel Goerz voted on patch set 21 CR +1 01/23/23, 08:32

  23. Benni Mack voted on patch set 21 V +2 CR +2 01/23/23, 11:18

  24. Benni Mack merged patch set 22 01/23/23, 11:18

  25. Helmut Hummel commented on patch set 22 01/23/23, 17:40

    Thanks everybody for your work on this! I think this is an awesome feature and will make a lot of cool things possible!