Review main Merged

92002[FEATURE] Implement system resource publishing

Helmut Hummel CR +2 V +2 Expert +1336 −149 updated 03/12/26, 11:55

Commit message

Implement the publishing part for the new system resources API.

With this in place, publishing has become more flexible and
it opens opportunities for further features.

A cli command `asset:publish` is added,
that uses the API and replaces the previously
hard wired publishing code that is run
during Composer build time.

For backwards compatibility, the command is also executed
during Composer build time, by the newly added
ConsoleCommand installer script.

Additionally, extension assets are published
during extension activation
(extension:setup command or in extension manager).

TYPO3 currently only ships file system based publishing.
This means that extension resources reside in a directory
outside of document root and are meant to be public,
will be copied to the public directory by filesystem operations.

Third party code however, can implement publishing to a CDN
(and generate links accordingly), which is the reason
publishing assets is also invoked in TYPO3 classic mode,
but does nothing for file system publishing, since all extension
assets still reside inside the document root in that mode.

Review

  • Code review CR +2 Done
    • Helmut Hummel +2
    • Jochen Roth +1
    • Garvin Hicking +1
  • Verified V +2 Done
    • Helmut Hummel +2
    • Jochen Roth +1
    • Garvin Hicking +1
    • core-ci +1 CI

Issues 1

Files 30

Show 10 more files

Review log 5474

Show 3349 earlier messages
  1. Helmut Hummel uploaded patch set 1 12/04/25, 22:23

  2. Helmut Hummel marked it work in progress 12/04/25, 22:26

  3. core-ci voted on patch set 1 V -1 12/04/25, 22:38 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/91616

  4. Helmut Hummel uploaded patch set 2 V -1 12/09/25, 11:09

  5. core-ci voted on patch set 2 V -1 12/09/25, 11:17 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/91878

  6. Helmut Hummel uploaded patch set 3 V -1 12/09/25, 12:27

  7. core-ci voted on patch set 3 V +1 12/09/25, 12:35 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/91884

  8. Helmut Hummel uploaded patch set 4 V +1 03/03/26, 18:45

  9. core-ci voted on patch set 4 V +1 03/03/26, 18:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97212

  10. Helmut Hummel updated the commit message in patch set 5 V +1 03/03/26, 18:58

  11. Helmut Hummel marked it ready for review 03/03/26, 18:58

  12. core-ci voted on patch set 5 V +1 03/03/26, 19:07 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97213

  13. Garvin Hicking commented on patch set 5 03/03/26, 19:47

    Great this looks interesting! Are you by chance already working on a ReST? That would allow me to better test this patch (if needed/wanted) and it would save me some time for the broader picture

    FileSystem.php:76

    Return annotation could be dropped, superfluous

    FileSystem.php:98

    Can be dropped too

    FileSystem.php:149

    Fan be dropped too

    FileSystem.php:195

    How do you stand in terms of "no @throws@ annotations"? 😇
  14. Helmut Hummel uploaded patch set 6 V +1 03/05/26, 10:17

  15. Helmut Hummel commented on patch set 5 03/05/26, 10:18

    Reply

    on it

    FileSystem.php:76 Reply

    Done

    FileSystem.php:98 Reply

    Done

    FileSystem.php:149 Reply

    Done

    FileSystem.php:195 Reply

    I like them, if they are used consistently
  16. core-ci voted on patch set 6 V -1 03/05/26, 10:54 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97402

  17. Helmut Hummel uploaded patch set 7 V -1 03/05/26, 12:37

  18. Helmut Hummel uploaded patch set 8 03/05/26, 12:38

  19. Helmut Hummel commented on patch set 7 03/05/26, 12:39

    Reply

    started with the docs, not finished yet, but contains the most important info I guess. will work on that later
  20. core-ci voted on patch set 7 V -1 03/05/26, 12:47 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97405

  21. core-ci voted on patch set 8 V -1 03/05/26, 12:50 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97406

  22. Simon Schaufelberger commented on patch set 8 03/05/26, 20:41

    AssetPublishCommand.php:31

    Add `AsCommand`, see LockBackendCommand as an example.

    AssetPublishCommand.php:62

    I would use `...` instead of the short version

    CommandExecutionFailedException.php:30

    remove empty line

    ConsoleCommand.php:51

    Shall we use `CommandUtility::exec` here?

    PublishAssetsOnPackageInitialization.php:33

    usually EventListeners have an identifier

    ResourceCollectionInterface.php:30

    add new line after

    FileSystemPublisherInterface.php:29

    add empty line
  23. Helmut Hummel commented on patch set 8 03/06/26, 11:52

    AssetPublishCommand.php:31 Reply

    this does not work for lowlevel commands

    AssetPublishCommand.php:62 Reply

    I wouldn't because its the correct ellipsis

    ConsoleCommand.php:51 Reply

    no, because this is executed in Composer Build time, thus we must use the API that is available there
  24. Helmut Hummel commented on patch set 8 03/06/26, 11:55

    PublishAssetsOnPackageInitialization.php:33 Reply

    none of the already merged listeners for PackageInitializationEvent have oone
  25. Helmut Hummel uploaded patch set 9 V -1 03/06/26, 12:23

  26. Helmut Hummel commented on patch set 8 03/06/26, 12:23

    CommandExecutionFailedException.php:30 Reply

    Done

    ResourceCollectionInterface.php:30 Reply

    Done

    FileSystemPublisherInterface.php:29 Reply

    Done
  27. Helmut Hummel uploaded patch set 10 03/06/26, 12:39

  28. Simon Schaufelberger voted on patch set 10 CR +1 03/06/26, 12:55

  29. core-ci voted on patch set 10 V -1 03/06/26, 13:22 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97499

  30. core-ci voted on patch set 9 V -1 03/06/26, 13:32 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97494

  31. core-ci voted on patch set 9 V +1 03/06/26, 17:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97494

  32. core-ci voted on patch set 10 V +1 03/06/26, 18:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97499

  33. Helmut Hummel uploaded patch set 11 CR +1 V +1 03/09/26, 09:45

  34. Helmut Hummel commented on patch set 10 03/09/26, 09:45

    Reply

    Done
  35. Helmut Hummel updated the commit message in patch set 12 03/09/26, 09:45

  36. core-ci voted on patch set 12 V +1 03/09/26, 09:57 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97592

  37. core-ci voted on patch set 11 V +1 03/09/26, 10:24 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97591

  38. Helmut Hummel uploaded patch set 13 V +1 03/09/26, 10:46

  39. core-ci voted on patch set 13 V -1 03/09/26, 10:56 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97599

  40. Helmut Hummel uploaded patch set 14 V -1 03/09/26, 12:30

  41. core-ci voted on patch set 14 V +1 03/09/26, 12:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97604

  42. Garvin Hicking commented on patch set 14 03/09/26, 12:39

    Feature-109163-Implement-public-system-resources-publishing.rst:51

    (Note to myself for review: what happens with files that existed in previous runs and which were removed, do these files remain? Or are they cleared on every invocation? Does a copy always copy file anew, or only if the file differs [network/bandwidth])

    Feature-109163-Implement-public-system-resources-publishing.rst:66

    Could you add a small example on where this "register as alia" would be done alongside a small publisher example?
  43. Helmut Hummel commented on patch set 14 03/09/26, 13:59

    Feature-109163-Implement-public-system-resources-publishing.rst:51 Reply

    Everything is copied completely. If files in the source dir are deleted, they remain in the target dir. It is only made sure that the target contains most recent files. There is still no purge in any way, not even on extension removal
  44. Garvin Hicking commented on patch set 14 03/09/26, 14:16

    Feature-109163-Implement-public-system-resources-publishing.rst:51 Reply

    I think, then we need to mention this in the changelog specifically. I think it could lead to confuusion, if people want to "depublish" contents. With the symlink strategy that would happen to never "leak" old files. Because this is then actually not a "mirror" type, but rather a "one-way-copy". Mirroring IMO implies deleting files no longer present on the left-side. Something like: ``` .. warning: Please bear in mind that the 'mirror' mode will not remove any files that were previously published. There is no specific "unpublish" action. Files that were once published, but no longer should be, need to be manually removed from the publish location on the target server(s)/builds. This is especially important for any TYPO3 installations where composer operations are performed "live on premise" instead being part of packaging/deployment. ```
  45. Simon Schaufelberger wrote 03/09/26, 14:19

    Removed reviewer Simon Schaufelberger.
  46. Georg Ringer commented on patch set 14 03/10/26, 04:10

    Feature-109163-Implement-public-system-resources-publishing.rst:51 Reply

    what about handing options to the publishingtype configuration - just like used from cacheconfigurations? then someone could set option delete=true of mirroring, see https://symfony.com/doc/current/components/filesystem.html#mirror
  47. Jochen Roth voted on patch set 14 CR +1 V +1 03/10/26, 07:00

  48. Helmut Hummel commented on patch set 14 03/10/26, 09:57

    Feature-109163-Implement-public-system-resources-publishing.rst:66 Reply

    I would like to leave this for further additions in this area. All interfaces are still marked internal for a reason. Once they are settled, I'm happy to add more examples how to do so.
  49. Helmut Hummel commented on patch set 14 03/10/26, 10:00

    Feature-109163-Implement-public-system-resources-publishing.rst:51 Reply

    > if people want to "depublish" contents I don't really share this concern. In classic mode, there is no file system publishing at all. in Composer mode, new code is published by a new deployment, which then wipes out old _assets folder anyway. what case do you have in mind, where someone in Composer mode updates code, but leaves the _assets folder there?
  1. Helmut Hummel commented on patch set 14 03/10/26, 10:01

    Feature-109163-Implement-public-system-resources-publishing.rst:51 Reply

    > what about handing options to the publishingtype configuration I would not expose the option. But we could set these options, if it makes everybody feel more comfortable
  2. Helmut Hummel uploaded patch set 15 CR +1 V +1 03/10/26, 11:05

  3. Helmut Hummel commented on patch set 15 03/10/26, 11:06

    Feature-109163-Implement-public-system-resources-publishing.rst:51 Reply

    fixed by removing stale files (sf filesystem mirror option)
  4. Helmut Hummel commented on patch set 15 03/10/26, 11:06

    Feature-109163-Implement-public-system-resources-publishing.rst:66 Reply

    resolved for now
  5. core-ci voted on patch set 15 V +1 03/10/26, 11:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97680

  6. Garvin Hicking commented on patch set 15 03/10/26, 11:43

    Feature-109163-Implement-public-system-resources-publishing.rst:51 Reply

    Thanks. My case was a "composer is used on production systems via shell" case. So no deployment. This is actually done in many cases if you check the support channels, so I'm afraid we have to deal with it. Haven't checked yet but we should havw a sentence in the "mirror" description that says stale files are removed to point it out.

    Feature-109163-Implement-public-system-resources-publishing.rst:66 Reply

    I think then it does not make sense yet to state that third parties can make use of this. Either we document it or we say "it's internal. Look but don't touch". Thats fair,but we should not give the impression it can really be used.
  7. Garvin Hicking commented on patch set 15 03/10/26, 14:12

    DefaultConfigurationDescription.yaml:202

    ```suggestion mirror: 'Copy all files and folders to the public directory (deletes stale public files)' ```
  8. Helmut Hummel commented on patch set 15 03/11/26, 07:40

    Feature-109163-Implement-public-system-resources-publishing.rst:66 Reply

    It for sure can be used. It's only hard(er) for me to come up with a simple example.
  9. Helmut Hummel uploaded patch set 16 V +1 03/11/26, 08:04

  10. core-ci voted on patch set 16 V +1 03/11/26, 08:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97739

  11. Helmut Hummel commented on patch set 15 03/11/26, 08:13

    DefaultConfigurationDescription.yaml:202 Reply

    Done

    Feature-109163-Implement-public-system-resources-publishing.rst:66 Reply

    finally had an idea for an example
  12. Jochen Roth commented on patch set 16 CR +1 V +1 03/11/26, 08:33

    Just a side note: When i switch from "link" to "mirror" or vice versa, it fails. > Could not publish public resources for extension "container_example" by using the "symlink" strategy Despite that it is working and i guess this is not a real-world problem.
  13. Garvin Hicking commented on patch set 16 V +1 03/11/26, 20:57

    I tested this also in my "monorepo as composer mode" setup, as well as a classic mode install and both setups seemed to work flawlessly. Also tried your docs example for custom publishing which also works. That example was VERY helpful to me.

    MessageRenderer.php:25

    @internal?

    JunctionPublisher.php:25

    Is this official API? Or better internal? Same for the other publishers.

    FileSystem.php:28

    Is this public API offered by TYPO3 Core?

    Feature-109163-Implement-public-system-resources-publishing.rst:72

    ```suggestion :caption: EXT:my_extension/Classes/Service/ExampleResourcePublisher.php (Simple example how to directly generate URIs for a CDN) ```

    Feature-109163-Implement-public-system-resources-publishing.rst:78

    ```suggestion namespace MyVendor\MyExtension\Service; ```

    Feature-109163-Implement-public-system-resources-publishing.rst:103

    ```suggestion // There could be some additional logic here to publish files to a CDN ```
  14. Helmut Hummel commented on patch set 16 03/12/26, 08:15

    MessageRenderer.php:25 Reply

    I would say this is totally fine to be used in third party extensions. The API should be fine like that and the output can be iterated on, so that third party usages benefit from it.

    JunctionPublisher.php:25 Reply

    Everything is prepared that this could be public. but let's keep it internal for now. will adapt
  15. Helmut Hummel uploaded patch set 17 CR +1 V +1 03/12/26, 08:18

  16. Helmut Hummel commented on patch set 16 03/12/26, 08:22

    JunctionPublisher.php:25 Reply

    Done

    FileSystem.php:28 Reply

    good point. I'll make this internal for now. It is basically only helper for the filesystem publishers. it could evolve to sth more stable in the future though (hence the generic namespace)

    Feature-109163-Implement-public-system-resources-publishing.rst:72 Reply

    Done

    Feature-109163-Implement-public-system-resources-publishing.rst:78 Reply

    Done

    Feature-109163-Implement-public-system-resources-publishing.rst:103 Reply

    Done
  17. Helmut Hummel uploaded patch set 18 03/12/26, 08:22

  18. core-ci voted on patch set 17 V +1 03/12/26, 08:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97798

  19. core-ci voted on patch set 18 V +1 03/12/26, 08:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/97799

  20. Jochen Roth voted on patch set 18 CR +1 V +1 03/12/26, 08:45

  21. Garvin Hicking commented on patch set 18 CR +1 V +1 03/12/26, 10:34

    👍
  22. Helmut Hummel voted on patch set 18 CR +2 V +2 03/12/26, 11:01

  23. Helmut Hummel merged patch set 19 03/12/26, 11:01

  24. Andreas Kienast commented on patch set 18 03/12/26, 11:03

    MessageRenderer.php:25

    Is this class meant to be extended by 3rd party? Otherwise I'd suggest to make it final and readonly.

    CommandExecutionFailedException.php:24

    `$errorOutput` is apparently not used. Was it forgotten somehow?

    DefaultSystemResourcePublisher.php:103

    Do we need some special handling in case no available publisher is able to publish something?

    PublishingConfiguration.php:31

    I couldn't find a validation whether only `auto`, `link` or `mirror` are passed. Do I miss something?

    PublishingConfiguration.php:53

    This method seems to be unused.
  25. Helmut Hummel commented on patch set 19 03/12/26, 11:55

    MessageRenderer.php:25 Reply

    Acknowledged

    CommandExecutionFailedException.php:24 Reply

    Acknowledged

    DefaultSystemResourcePublisher.php:103 Reply

    yeah, good point. once this becomes public API, we might need to check for that. since it is internal for now and the case can not happen for shipped publishers when config is correct, I think it is fine.

    PublishingConfiguration.php:31 Reply

    other values could be evaluated by custom publishers (see below)

    PublishingConfiguration.php:53 Reply

    it is not. it is prepared for being used in custom publishers, but this is out of scope for now, as API is still marked internal. could be used for experiments though