92002[FEATURE] Implement system resource publishing
Commit message
Review
-
Code review CR +2 Done
- Helmut Hummel +2
- Jochen Roth +1
- Garvin Hicking +1
-
Verified V +2 Done
- Helmut Hummel +2
- Jochen Roth +1
- Garvin Hicking +1
- core-ci +1 CI
Issues 1
Files 30
- typo3/sysext/core/Classes/Command/Output/MessageRenderer.php4new+52 −0
- typo3/sysext/core/Classes/Command/AssetPublishCommand.php4new+94 −0
- typo3/sysext/core/Classes/Composer/CommandExecutionFailedException.php4new+30 −0
- typo3/sysext/core/Classes/Composer/ConsoleCommand.php2new+102 −0
- typo3/sysext/core/Classes/Composer/InstallerScripts.php+11 −1
- typo3/sysext/core/Classes/Composer/PackageArtifactBuilder.php+1 −136
- typo3/sysext/core/Classes/DependencyInjection/AssetFileSystemPublisherPass.phpnew+71 −0
- typo3/sysext/core/Classes/Messaging/FlashMessageQueue.php+4 −2
- typo3/sysext/core/Classes/Package/Exception/PackageAssetsPublishingFailedException.php+0 −1
- typo3/sysext/core/Classes/Package/Initialization/PublishAssetsOnPackageInitialization.php2new+38 −0
- typo3/sysext/core/Classes/Package/Resource/ResourceCollection.php+1 −1
- typo3/sysext/core/Classes/Package/Resource/ResourceCollectionInterface.php2+5 −0
- typo3/sysext/core/Classes/Resource/ResourceFactory.php+12 −1
- typo3/sysext/core/Classes/SystemResource/Publishing/FileSystem/FileSystemPublisherInterface.php2new+38 −0
- typo3/sysext/core/Classes/SystemResource/Publishing/FileSystem/JunctionPublisher.php3new+80 −0
- typo3/sysext/core/Classes/SystemResource/Publishing/FileSystem/MirrorPublisher.phpnew+74 −0
- typo3/sysext/core/Classes/SystemResource/Publishing/FileSystem/PublishingConfiguration.php4new+60 −0
- typo3/sysext/core/Classes/SystemResource/Publishing/FileSystem/SymlinkPublisher.phpnew+69 −0
- typo3/sysext/core/Classes/SystemResource/Publishing/DefaultSystemResourcePublisher.php2+87 −4
- typo3/sysext/core/Classes/SystemResource/Publishing/DefaultSystemResourceUriGenerator.php+2 −1
Show 10 more files
- typo3/sysext/core/Classes/SystemResource/Publishing/SystemResourcePublisherInterface.php+2 −1
- typo3/sysext/core/Classes/Utility/File/FileSystem.php10new+209 −0
- typo3/sysext/core/Classes/ServiceProvider.php+22 −1
- typo3/sysext/core/Configuration/DefaultConfiguration.php+3 −0
- typo3/sysext/core/Configuration/DefaultConfigurationDescription.yaml2+11 −0
- typo3/sysext/core/Configuration/Services.php+3 −0
- typo3/sysext/core/Documentation/Changelog/14.2/Feature-109163-Implement-public-system-resources-publishing.rst20new+144 −0
- typo3/sysext/core/Tests/Functional/SystemResource/Publishing/DefaultResourcePublisherTest.php+54 −0
- typo3/sysext/install/Classes/Middleware/AssetPublishing.phpnew+46 −0
- typo3/sysext/install/Classes/ServiceProvider.php+11 −0
Review log 5474
Show 3349 earlier messages
-
Helmut Hummel uploaded patch set 1 12/04/25, 22:23
-
Helmut Hummel marked it work in progress 12/04/25, 22:26
-
core-ci voted on patch set 1 V -1 12/04/25, 22:38 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/91616
-
Helmut Hummel uploaded patch set 2
V -112/09/25, 11:09 -
core-ci voted on patch set 2 V -1 12/09/25, 11:17 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/91878
-
Helmut Hummel uploaded patch set 3
V -112/09/25, 12:27 -
core-ci voted on patch set 3 V +1 12/09/25, 12:35 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/91884
-
Helmut Hummel uploaded patch set 4
V +103/03/26, 18:45 -
core-ci voted on patch set 4 V +1 03/03/26, 18:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97212
-
Helmut Hummel updated the commit message in patch set 5
V +103/03/26, 18:58 -
Helmut Hummel marked it ready for review 03/03/26, 18:58
-
core-ci voted on patch set 5 V +1 03/03/26, 19:07 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97213
-
Garvin Hicking commented on patch set 5 03/03/26, 19:47
Great this looks interesting! Are you by chance already working on a ReST? That would allow me to better test this patch (if needed/wanted) and it would save me some time for the broader pictureFileSystem.php:76Return annotation could be dropped, superfluousFileSystem.php:98Can be dropped tooFileSystem.php:149Fan be dropped tooFileSystem.php:195How do you stand in terms of "no @throws@ annotations"? 😇 -
Helmut Hummel uploaded patch set 6
V +103/05/26, 10:17 -
Helmut Hummel commented on patch set 5 03/05/26, 10:18
Reply
on itFileSystem.php:76ReplyDoneFileSystem.php:98ReplyDoneFileSystem.php:149ReplyDoneFileSystem.php:195ReplyI like them, if they are used consistently -
core-ci voted on patch set 6 V -1 03/05/26, 10:54 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97402
-
Helmut Hummel uploaded patch set 7
V -103/05/26, 12:37 -
Helmut Hummel uploaded patch set 8 03/05/26, 12:38
-
Helmut Hummel commented on patch set 7 03/05/26, 12:39
Reply
started with the docs, not finished yet, but contains the most important info I guess. will work on that later -
core-ci voted on patch set 7 V -1 03/05/26, 12:47 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97405
-
core-ci voted on patch set 8 V -1 03/05/26, 12:50 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97406
-
Simon Schaufelberger commented on patch set 8 03/05/26, 20:41
AssetPublishCommand.php:31Add `AsCommand`, see LockBackendCommand as an example.AssetPublishCommand.php:62I would use `...` instead of the short versionCommandExecutionFailedException.php:30remove empty lineConsoleCommand.php:51Shall we use `CommandUtility::exec` here?PublishAssetsOnPackageInitialization.php:33usually EventListeners have an identifierResourceCollectionInterface.php:30add new line afterFileSystemPublisherInterface.php:29add empty line -
Helmut Hummel commented on patch set 8 03/06/26, 11:52
AssetPublishCommand.php:31Replythis does not work for lowlevel commandsAssetPublishCommand.php:62ReplyI wouldn't because its the correct ellipsisConsoleCommand.php:51Replyno, because this is executed in Composer Build time, thus we must use the API that is available there -
Helmut Hummel commented on patch set 8 03/06/26, 11:55
PublishAssetsOnPackageInitialization.php:33Replynone of the already merged listeners for PackageInitializationEvent have oone -
Helmut Hummel uploaded patch set 9
V -103/06/26, 12:23 -
Helmut Hummel commented on patch set 8 03/06/26, 12:23
CommandExecutionFailedException.php:30ReplyDoneResourceCollectionInterface.php:30ReplyDoneFileSystemPublisherInterface.php:29ReplyDone -
Helmut Hummel uploaded patch set 10 03/06/26, 12:39
-
Simon Schaufelberger voted on patch set 10 CR +1 03/06/26, 12:55
-
core-ci voted on patch set 10 V -1 03/06/26, 13:22 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97499
-
core-ci voted on patch set 9 V -1 03/06/26, 13:32 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97494
-
core-ci voted on patch set 9 V +1 03/06/26, 17:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97494
-
core-ci voted on patch set 10 V +1 03/06/26, 18:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97499
-
Helmut Hummel uploaded patch set 11
CR +1V +103/09/26, 09:45 -
Helmut Hummel commented on patch set 10 03/09/26, 09:45
Reply
Done -
Helmut Hummel updated the commit message in patch set 12 03/09/26, 09:45
-
core-ci voted on patch set 12 V +1 03/09/26, 09:57 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97592
-
core-ci voted on patch set 11 V +1 03/09/26, 10:24 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97591
-
Helmut Hummel uploaded patch set 13
V +103/09/26, 10:46 -
core-ci voted on patch set 13 V -1 03/09/26, 10:56 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97599
-
Helmut Hummel uploaded patch set 14
V -103/09/26, 12:30 -
core-ci voted on patch set 14 V +1 03/09/26, 12:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97604
-
Garvin Hicking commented on patch set 14 03/09/26, 12:39
Feature-109163-Implement-public-system-resources-publishing.rst:51(Note to myself for review: what happens with files that existed in previous runs and which were removed, do these files remain? Or are they cleared on every invocation? Does a copy always copy file anew, or only if the file differs [network/bandwidth])Feature-109163-Implement-public-system-resources-publishing.rst:66Could you add a small example on where this "register as alia" would be done alongside a small publisher example? -
Helmut Hummel commented on patch set 14 03/09/26, 13:59
Feature-109163-Implement-public-system-resources-publishing.rst:51ReplyEverything is copied completely. If files in the source dir are deleted, they remain in the target dir. It is only made sure that the target contains most recent files. There is still no purge in any way, not even on extension removal -
Garvin Hicking commented on patch set 14 03/09/26, 14:16
Feature-109163-Implement-public-system-resources-publishing.rst:51ReplyI think, then we need to mention this in the changelog specifically. I think it could lead to confuusion, if people want to "depublish" contents. With the symlink strategy that would happen to never "leak" old files. Because this is then actually not a "mirror" type, but rather a "one-way-copy". Mirroring IMO implies deleting files no longer present on the left-side. Something like: ``` .. warning: Please bear in mind that the 'mirror' mode will not remove any files that were previously published. There is no specific "unpublish" action. Files that were once published, but no longer should be, need to be manually removed from the publish location on the target server(s)/builds. This is especially important for any TYPO3 installations where composer operations are performed "live on premise" instead being part of packaging/deployment. ``` -
Simon Schaufelberger wrote 03/09/26, 14:19
Removed reviewer Simon Schaufelberger. -
Georg Ringer commented on patch set 14 03/10/26, 04:10
Feature-109163-Implement-public-system-resources-publishing.rst:51Replywhat about handing options to the publishingtype configuration - just like used from cacheconfigurations? then someone could set option delete=true of mirroring, see https://symfony.com/doc/current/components/filesystem.html#mirror -
Jochen Roth voted on patch set 14 CR +1 V +1 03/10/26, 07:00
-
Helmut Hummel commented on patch set 14 03/10/26, 09:57
Feature-109163-Implement-public-system-resources-publishing.rst:66ReplyI would like to leave this for further additions in this area. All interfaces are still marked internal for a reason. Once they are settled, I'm happy to add more examples how to do so. -
Helmut Hummel commented on patch set 14 03/10/26, 10:00
Feature-109163-Implement-public-system-resources-publishing.rst:51Reply> if people want to "depublish" contents I don't really share this concern. In classic mode, there is no file system publishing at all. in Composer mode, new code is published by a new deployment, which then wipes out old _assets folder anyway. what case do you have in mind, where someone in Composer mode updates code, but leaves the _assets folder there?
-
Helmut Hummel commented on patch set 14 03/10/26, 10:01
Feature-109163-Implement-public-system-resources-publishing.rst:51Reply> what about handing options to the publishingtype configuration I would not expose the option. But we could set these options, if it makes everybody feel more comfortable -
Helmut Hummel uploaded patch set 15
CR +1V +103/10/26, 11:05 -
Helmut Hummel commented on patch set 15 03/10/26, 11:06
Feature-109163-Implement-public-system-resources-publishing.rst:51Replyfixed by removing stale files (sf filesystem mirror option) -
Helmut Hummel commented on patch set 15 03/10/26, 11:06
Feature-109163-Implement-public-system-resources-publishing.rst:66Replyresolved for now -
core-ci voted on patch set 15 V +1 03/10/26, 11:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97680
-
Garvin Hicking commented on patch set 15 03/10/26, 11:43
Feature-109163-Implement-public-system-resources-publishing.rst:51ReplyThanks. My case was a "composer is used on production systems via shell" case. So no deployment. This is actually done in many cases if you check the support channels, so I'm afraid we have to deal with it. Haven't checked yet but we should havw a sentence in the "mirror" description that says stale files are removed to point it out.Feature-109163-Implement-public-system-resources-publishing.rst:66ReplyI think then it does not make sense yet to state that third parties can make use of this. Either we document it or we say "it's internal. Look but don't touch". Thats fair,but we should not give the impression it can really be used. -
Garvin Hicking commented on patch set 15 03/10/26, 14:12
DefaultConfigurationDescription.yaml:202```suggestion mirror: 'Copy all files and folders to the public directory (deletes stale public files)' ``` -
Helmut Hummel commented on patch set 15 03/11/26, 07:40
Feature-109163-Implement-public-system-resources-publishing.rst:66ReplyIt for sure can be used. It's only hard(er) for me to come up with a simple example. -
Helmut Hummel uploaded patch set 16
V +103/11/26, 08:04 -
core-ci voted on patch set 16 V +1 03/11/26, 08:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97739
-
Helmut Hummel commented on patch set 15 03/11/26, 08:13
DefaultConfigurationDescription.yaml:202ReplyDoneFeature-109163-Implement-public-system-resources-publishing.rst:66Replyfinally had an idea for an example -
Jochen Roth commented on patch set 16 CR +1 V +1 03/11/26, 08:33
Just a side note: When i switch from "link" to "mirror" or vice versa, it fails. > Could not publish public resources for extension "container_example" by using the "symlink" strategy Despite that it is working and i guess this is not a real-world problem. -
Garvin Hicking commented on patch set 16 V +1 03/11/26, 20:57
I tested this also in my "monorepo as composer mode" setup, as well as a classic mode install and both setups seemed to work flawlessly. Also tried your docs example for custom publishing which also works. That example was VERY helpful to me.MessageRenderer.php:25@internal?JunctionPublisher.php:25Is this official API? Or better internal? Same for the other publishers.FileSystem.php:28Is this public API offered by TYPO3 Core?Feature-109163-Implement-public-system-resources-publishing.rst:72```suggestion :caption: EXT:my_extension/Classes/Service/ExampleResourcePublisher.php (Simple example how to directly generate URIs for a CDN) ```Feature-109163-Implement-public-system-resources-publishing.rst:78```suggestion namespace MyVendor\MyExtension\Service; ```Feature-109163-Implement-public-system-resources-publishing.rst:103```suggestion // There could be some additional logic here to publish files to a CDN ``` -
Helmut Hummel commented on patch set 16 03/12/26, 08:15
MessageRenderer.php:25ReplyI would say this is totally fine to be used in third party extensions. The API should be fine like that and the output can be iterated on, so that third party usages benefit from it.JunctionPublisher.php:25ReplyEverything is prepared that this could be public. but let's keep it internal for now. will adapt -
Helmut Hummel uploaded patch set 17
CR +1V +103/12/26, 08:18 -
Helmut Hummel commented on patch set 16 03/12/26, 08:22
JunctionPublisher.php:25ReplyDoneFileSystem.php:28Replygood point. I'll make this internal for now. It is basically only helper for the filesystem publishers. it could evolve to sth more stable in the future though (hence the generic namespace)Feature-109163-Implement-public-system-resources-publishing.rst:72ReplyDoneFeature-109163-Implement-public-system-resources-publishing.rst:78ReplyDoneFeature-109163-Implement-public-system-resources-publishing.rst:103ReplyDone -
Helmut Hummel uploaded patch set 18 03/12/26, 08:22
-
core-ci voted on patch set 17 V +1 03/12/26, 08:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97798
-
core-ci voted on patch set 18 V +1 03/12/26, 08:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97799
-
Jochen Roth voted on patch set 18 CR +1 V +1 03/12/26, 08:45
-
Garvin Hicking commented on patch set 18 CR +1 V +1 03/12/26, 10:34
👍 -
Helmut Hummel voted on patch set 18 CR +2 V +2 03/12/26, 11:01
-
Helmut Hummel merged patch set 19 03/12/26, 11:01
-
Andreas Kienast commented on patch set 18 03/12/26, 11:03
MessageRenderer.php:25Is this class meant to be extended by 3rd party? Otherwise I'd suggest to make it final and readonly.CommandExecutionFailedException.php:24`$errorOutput` is apparently not used. Was it forgotten somehow?DefaultSystemResourcePublisher.php:103Do we need some special handling in case no available publisher is able to publish something?PublishingConfiguration.php:31I couldn't find a validation whether only `auto`, `link` or `mirror` are passed. Do I miss something?PublishingConfiguration.php:53This method seems to be unused. -
Helmut Hummel commented on patch set 19 03/12/26, 11:55
MessageRenderer.php:25ReplyAcknowledgedCommandExecutionFailedException.php:24ReplyAcknowledgedDefaultSystemResourcePublisher.php:103Replyyeah, good point. once this becomes public API, we might need to check for that. since it is internal for now and the case can not happen for shipped publishers when config is correct, I think it is fine.PublishingConfiguration.php:31Replyother values could be evaluated by custom publishers (see below)PublishingConfiguration.php:53Replyit is not. it is prepared for being used in custom publishers, but this is out of scope for now, as API is still marked internal. could be used for experiments though