Merge conflict Gerrit cannot merge this change. It needs a rebase onto main.
94709[BUGFIX] Fix sending spooled emails using HttpTransport
Commit message
Test this change
Run a command in your clone of the TYPO3 Core. Then try what the change does and vote on Verified in Gerrit. How to review a patch
- Check out patch set 12
-
git fetch https://review.typo3.org/Packages/TYPO3.CMS refs/changes/09/94709/12 && git checkout FETCH_HEAD - Cherry-pick onto main
-
git fetch https://review.typo3.org/Packages/TYPO3.CMS refs/changes/09/94709/12 && git cherry-pick FETCH_HEAD
Review
-
Code review CR 0 Needs a vote
-
Verified V 0 Needs a vote
- core-ci +1 CI
Issues 2
Files 6
Review log 3345
Show 1620 earlier messages
-
Stephan Großberndt uploaded patch set 1 07/08/26, 13:47
-
core-ci voted on patch set 1 V +1 07/08/26, 13:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/104519
-
Simon Schaufelberger commented on patch set 1 07/08/26, 14:04
would it be possible to cover this with a test? -
Stephan Großberndt commented on patch set 1 07/08/26, 14:15
Reply
Yes, probably by adding one in TYPO3\CMS\Core\Tests\Unit\Mail\TransportFactoryTest. Could you help me on how to properly test this? The problem is that it only occurs with a `AbstractApiTransport`, which is not simply usable -
Markus Klein voted on patch set 1 CR +1 07/08/26, 14:15
-
Stephan Großberndt commented on patch set 1 07/08/26, 14:25
Reply
It would be necessary to set up a fake transport to make sure a http transport via AbstractApiTransport is used, send a mail to the spool and trigger the sending from the spool. -
Stephan Großberndt commented on patch set 1 07/08/26, 14:44
I verified that emails can be sent - using spool=file and `symfony/microsoft-graph-mailer` (did not work before) - using spool=memory and `symfony/microsoft-graph-mailer` (did not work before) - using spool=file and smtp (still works) - using spool=memory and smtp (still works) -
Stephan Großberndt uploaded patch set 2
CR +1V +107/20/26, 08:29 -
core-ci voted on patch set 2 V -1 07/20/26, 08:39 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105016
-
Stephan Großberndt uploaded patch set 3
V -107/20/26, 08:41 -
core-ci voted on patch set 3 V +1 07/20/26, 08:50 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105017
-
Simon Schaufelberger commented on patch set 3 CR +1 V +1 07/20/26, 09:43
Pair programming with Stephan was fun! -
Stefan Bürk commented on patch set 3 CR -1 07/20/26, 09:57
Reviewing (and commit message needs polishing, line length ;) ) -
Stephan Großberndt updated the commit message in patch set 4
CR +1V +107/21/26, 15:26 -
Stephan Großberndt commented on patch set 4 07/21/26, 15:26
Reply
Updated commit message :-) -
core-ci voted on patch set 4 V +1 07/21/26, 15:35 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105110
-
Stephan Großberndt commented on patch set 4 07/21/26, 15:51
Reply
Thanks again for your help and time, Simon! Fun for me as well! -
Stefan Bürk uploaded patch set 5
V +107/21/26, 15:51 -
Stefan Bürk updated the commit message in patch set 6 07/21/26, 15:58
-
core-ci voted on patch set 5 V +1 07/21/26, 15:59 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105111
-
Stephan Großberndt updated the commit message in patch set 7 07/21/26, 16:01
-
Stefan Bürk commented on patch set 7 07/21/26, 16:04
FileSpool.php:172`SentMessage` adds a `Message ID` header with a generated message id for the `raw` message based on the original message if not already set as header (cloned from the original message). That would mean that it's possible discarded and the information lost. I'm uncertain if we should add a similar check here ... I'm not deep enough in that area / api part of symfony. See `SentMessage::__construct()`: ``` public function __construct( RawMessage $message, private Envelope $envelope, ) { $message->ensureValidity(); $this->original = $message; if ($message instanceof Message) { $message = clone $message; $headers = $message->getHeaders(); if (!$headers->has('Message-ID')) { $headers->addIdHeader('Message-ID', $message->generateMessageId()); } $this->messageId = $headers->get('Message-ID')->getId(); $this->raw = new RawMessage($message->toIterable()); } else { $this->raw = $message; } } ``` -
core-ci voted on patch set 6 V +1 07/21/26, 16:07 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105112
-
core-ci voted on patch set 7 V +1 07/21/26, 16:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105113
-
Markus Klein commented on patch set 7 07/21/26, 19:24
I'm not sure, if this actually a bug in Symfony. PHPstorm already complains within \Symfony\Component\Mailer\Transport\AbstractApiTransport that the types do not match. -
Stephan Großberndt commented on patch set 7 07/23/26, 08:42
Reply
Indeed it is an additional bug in symfony, which should not hinder the merge of this change. I'm unsure on how to report it as it won't be possible to trigger it within symfony mailer itself standalone.FileSpool.php:172ReplyI don't understand where you want to add what kind of check. Before the change the FileSpool used the RawMessage, which misses information as it is constructed as a new RawMessage instead of FluidEmail, Email or Message. AbstractTransport::send() creates a SentMessage: On SentMessage::__construct() the message original message is saved as $this->original and a newly constructed RawMessage based on the original message is created in $this->raw. AbstractTransport::doSend() is called, which for the spooled transports does not actually send the message but serializes it and saves it in the file system. On FileSpool::flushQueue() the message is unserialized and actually sent. We do want to use the original message in this $transport->send() call of the filespool so the real transport, which is used here, receives the original message. -
Markus Klein commented on patch set 7 07/24/26, 15:05
FileSpool.php:172ReplyWell, there is even more detail to this. Lets consider a FluidEmail $m being sent via SMTP without a spooler. * Our Mailer class will pass $m to the SmtpTansport::send method. * which will run [AbstractTransport::send](https://github.com/symfony/symfony/blob/8.2/src/Symfony/Component/Mailer/Transport/AbstractTransport.php#L61) * which will create a SentMessage($m,…) $sm * and run [SmtpTransport::doSend](https://github.com/symfony/symfony/blob/8.2/src/Symfony/Component/Mailer/Transport/Smtp/SmtpTransport.php#L196) * [There](https://github.com/symfony/symfony/blob/8.2/src/Symfony/Component/Mailer/Transport/Smtp/SmtpTransport.php#L215) it will use `$message->toIterable()` `toIterable` actually uses `raw` [here](https://github.com/symfony/symfony/blob/8.2/src/Symfony/Component/Mailer/SentMessage.php#L101) and not the `original`. The resulting SentMessage $sm will hold the message-id used to send the message. With the spooler plugged into the middle, things actually get tricky now. The spooler again walks the way of using the actual transport. The transport shall receiver the same call as it would be, when using it without a spooler. This would be the `originalMessage`. BUT, after sending the message a new SentMessage is returned (which is discarded of course), which MAY have a different message-id than the one received from the $sm by the calling code. This is probably the reason, we currently pass the `raw` message to the real transport, as it guarantees a stable message-id. The AbstractHttpTransport/AbstractApiTransport are completely different in this regard. They use the [`originalMessage`](https://github.com/symfony/symfony/blob/8.2/src/Symfony/Component/Mailer/Transport/AbstractApiTransport.php#L32) during sending. So IMO the AbstractApiTransport in Symfony is broken here. It never ensures a message-id is generated and it has broken data types with the `MessageConverter::toEmail($message->getOriginalMessage())` call. -
Markus Klein commented on patch set 7 07/24/26, 15:24
Unresolved
-
Markus Klein commented on patch set 7 07/24/26, 15:24
-
Stephan Großberndt uploaded patch set 8
V +107/30/26, 10:29 -
Stephan Großberndt updated the commit message in patch set 9 07/30/26, 10:33
-
core-ci voted on patch set 8 V +1 07/30/26, 10:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105679
-
Stephan Großberndt commented on patch set 9 07/30/26, 10:39
Reply
DoneRebased due to merge conflict. symfony/mailer has been updated in the meantime, which brings an update that replaces the PHP type error with an explicit exception in case a RawMessage is passed in doSendHttp(): https://github.com/symfony/mailer/commit/2a380924b56e034076c4a658d40e140ba150e44c#diff-7e921fed88ab5a0298bc8b2c29475e7a7acdccfbcfa6c25a9ff724ec7136e7e8 How to proceed with the message-id problem? -
core-ci voted on patch set 9 V +1 07/30/26, 10:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/105680
-
Stephan Großberndt commented on patch set 9 09/03/26, 12:49
FileSpool.php:172ReplyThe symfony ticket has been closed by Nicolas Grekas, who left a comment: https://github.com/symfony/symfony/issues/64964#issuecomment-5467567471 @markus.klein@typo3.org @stefan@buerk.tech -
Stephan Großberndt uploaded patch set 10
V +110/02/26, 07:33 -
core-ci voted on patch set 10 V +1 10/02/26, 07:44 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/109884
-
Garvin Hicking updated the commit message in patch set 11
V +110/02/26, 07:47 -
Garvin Hicking commented on patch set 11 10/02/26, 07:48
(I've scoped this to v14+ because I think for 13.4 the a-bit-edge-caseyness might not classify as "high importance bug fixes". We'll need to close down on 13.4 backports at this time now) -
core-ci voted on patch set 11 V +1 10/02/26, 07:59 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/109888
-
Markus Klein commented on patch set 11 10/02/26, 11:13
Unresolved
I still have that one on my todo to revisit. I'll also comment on the upstream issue again, because the info is nice there, but actually needs to be documented like that. -
Stephan Großberndt rebased onto patch set 12
V +110/07/26, 07:32 -
core-ci voted on patch set 12 V +1 10/07/26, 09:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/110292
-
Markus Klein commented on patch set 12 10/09/26, 11:45
FileSpool.php:172Reply UnresolvedFollowing the comment that Nicolas left, I suggest we start adding a message-id on the Email in the filespool here.