73196[WIP][FEATURE] Add configuration object system
Commit message
Review
-
Code review CR 0
-
Verified V 0
Issues 1
Files 42
- Build/phpstan/phpstan.neon3+9 −0
- typo3/sysext/core/Classes/Config/FormDef/CheckboxFormField.phpnew+21 −0
- typo3/sysext/core/Classes/Config/FormDef/FieldTypeIncompatible.phpnew+24 −0
- typo3/sysext/core/Classes/Config/FormDef/FormDefinition.phpnew+20 −0
- typo3/sysext/core/Classes/Config/FormDef/FormField.phpnew+104 −0
- typo3/sysext/core/Classes/Config/FormDef/FormFieldType.phpnew+18 −0
- typo3/sysext/core/Classes/Config/FormDef/GroupFormField.phpnew+22 −0
- typo3/sysext/core/Classes/Config/FormDef/NumberFormField.phpnew+37 −0
- typo3/sysext/core/Classes/Config/FormDef/PasswordFormField.phpnew+12 −0
- typo3/sysext/core/Classes/Config/FormDef/SelectFormField.phpnew+23 −0
- typo3/sysext/core/Classes/Config/FormDef/TextAreaFormField.phpnew+25 −0
- typo3/sysext/core/Classes/Config/FormDef/TextFormField.phpnew+50 −0
- typo3/sysext/core/Classes/Config/Config.phpnew+48 −0
- typo3/sysext/core/Classes/Config/ConfigFactory.php3new+137 −0
- typo3/sysext/core/Classes/Config/ConstructorHydratable.phpnew+23 −0
- typo3/sysext/core/Classes/Config/GraphicsConfig.php1new+52 −0
- typo3/sysext/core/Classes/Config/Http.php2new+77 −0
- typo3/sysext/core/Classes/Config/Hydratable.phpnew+26 −0
- typo3/sysext/core/Classes/Config/MailConfig.php4new+192 −0
- typo3/sysext/core/Classes/Config/MailConfigDsn.php1new+24 −0
Show 22 more files
- typo3/sysext/core/Classes/Config/MailConfigMbox.php1new+23 −0
- typo3/sysext/core/Classes/Config/MailConfigSendmail.php1new+24 −0
- typo3/sysext/core/Classes/Config/MailConfigSmtp.php1new+76 −0
- typo3/sysext/core/Classes/Config/MailFormat.phpnew+15 −0
- typo3/sysext/core/Classes/DependencyInjection/ConfigDefinitionPass.php4new+82 −0
- typo3/sysext/core/Classes/Mail/Mailer.php+12 −66
- typo3/sysext/core/Classes/Mail/TransportFactory.php+124 −141
- typo3/sysext/core/Classes/ServiceProvider.php4+19 −0
- typo3/sysext/core/Configuration/Services.php2+1 −0
- typo3/sysext/core/Configuration/Services.yaml4+27 −0
- typo3/sysext/core/Tests/Functional/Mail/MailerTest.php2new+122 −0
- typo3/sysext/core/Tests/Unit/Config/ConfigFactoryTest.php2new+283 −0
- typo3/sysext/core/Tests/Unit/Mail/MailConfigSmtpTest.php1new+29 −0
- typo3/sysext/core/Tests/Unit/Mail/MailConfigTest.phpnew+90 −0
- typo3/sysext/core/Tests/Unit/Mail/MailerTest.php+24 −120
- typo3/sysext/core/Tests/Unit/Mail/TransportFactoryTest.php+62 −154
- typo3/sysext/core/composer.json+4 −0
- typo3/sysext/filelist/Classes/Controller/FileListController.php3+4 −2
- typo3/sysext/install/composer.json2+1 −0
- typo3/sysext/linkvalidator/Tests/Unit/Linktype/ExternalLinktypeTest.php+1 −0
- composer.json+4 −0
- composer.lock2+396 −1
Review log 4265
Show 2640 earlier messages
-
crell uploaded patch set 1 01/28/22, 22:22
-
core-ci voted on patch set 1 V -1 01/28/22, 22:24 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22641
-
Stefan Bürk commented on patch set 1 01/28/22, 23:40
some early/first notes btw. add the pre-commit hook which checky cgl locally - at least run now the fix locally to fix the cgl issues ;)again: main, not master.GfxConfig.php:22main is min PHP8.1 - so todo can directly solved ;) - use "constructor property promotion" to eliminate the properties and make them readonly directlyHttp.php:25What are these attributes (for) ? Is this where you need one of your libs/packages you have been building ? If not used yet, should be avoided - otherwise include your package and demonstrate it ... -
Stefan Bürk updated the commit message in patch set 2 01/29/22, 00:15
-
core-ci voted on patch set 2 V -1 01/29/22, 00:18 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22646
-
crell marked it work in progress 01/29/22, 17:42
-
crell commented on patch set 2 01/29/22, 17:44
This change is ready for review.Reply
Forgot to mark WIP on this. I thought it would go to the previous issue. Please ignore for now, it's just a rebase of the old patch so I can start working on the Serde-based version. -
crell uploaded patch set 3 02/01/22, 00:14
-
core-ci voted on patch set 3 V -1 02/01/22, 00:16 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22842
-
crell updated the commit message in patch set 4 02/01/22, 00:16
-
core-ci voted on patch set 4 V -1 02/01/22, 00:18 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22843
-
crell commented on patch set 4 02/01/22, 19:41
Reply
AckGfxConfig.php:22ReplyDoneHttp.php:25ReplyDone -
Oliver Klee commented on patch set 4 02/01/22, 20:16
In general, please let's note have a period at the end of commit message subject lines: https://chiamakaikeanyi.dev/how-to-write-good-git-commit-messages/ -
crell uploaded patch set 5 02/01/22, 20:21
-
core-ci voted on patch set 5 V -1 02/01/22, 20:23 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22893
-
Stefan Bürk commented on patch set 5 02/01/22, 20:30
* remove: "Build/phpunit/.phpunit.result.cache" which MUST not be in the repo, also this SHOULD not created anymore, so delete it !!! (and remove it from the patch). * rebase your patch on current main solving merge conclict: see https://docs.typo3.org/m/typo3/guide-contributionworkflow/main/en-us/Setup/Git/Index.html#set-autosetuprebase and https://docs.typo3.org/m/typo3/guide-contributionworkflow/main/en-us/HandlingAPatch/Rebase.html#method-2-git-pull-rebase * Either add "[POC]" to commit topic line and/or switch to WIP mode - or remove "Proof of concept from commit message" * again, use either cgl script manually or in combination as pre-commit hook and fix cgl before pushing - would help you to not have failing pipelines. At least not for the code file parts. See: https://docs.typo3.org/m/typo3/guide-contributionworkflow/main/en-us/Setup/Git/Index.html#pre-commit-hook > phpunit result cache has been disabled a longer time ago, and thus removed from .gitignore. Cleanup your coredev installation It's confusing is this patch is ready for real review or if it still is in work. -
Stefan Bürk commented on patch set 5 02/01/22, 20:31
Reply
"Resolving" directly was not intended. So reopen it. -
Oliver Klee wrote 02/01/22, 20:57
Removed cc Oliver Klee. -
crell marked it work in progress 02/01/22, 23:46
-
crell commented on patch set 5 02/01/22, 23:48
This change is ready for review.Reply
I keep setting this to WIP. I don't know why it's getting overridden. There's nothing to see here yet. Please ignore for now. And I have no idea why the result cache ended up in the patch... Shouldn't that be in gitignore? -
Stefan Bürk commented on patch set 5 02/02/22, 02:33
Reply
That file should not be there at all ! phpunit configuration for core is setup to not creating/using a result file at all. So it should not be written pysically. Pls. delete that file AND remove it from the patch. After removing from patch ensure that file is absolutly killed in your instance/filesystem. Do you run phpunit with a custom configuration and not the core one / Build/Scripts/runTests.sh ? -
crell uploaded patch set 6 02/02/22, 20:41
-
core-ci voted on patch set 6 V -1 02/02/22, 20:43 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22924
-
crell uploaded patch set 7 02/03/22, 18:02
-
core-ci voted on patch set 7 V -1 02/03/22, 18:05 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22966
-
crell uploaded patch set 8 02/03/22, 22:23
-
core-ci voted on patch set 8 V -1 02/03/22, 22:35 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22980
-
crell uploaded patch set 9 02/03/22, 23:38
-
core-ci voted on patch set 9 V -1 02/03/22, 23:40 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22984
-
crell uploaded patch set 10 02/08/22, 21:08
-
core-ci voted on patch set 10 V -1 02/08/22, 21:10 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23240
-
crell uploaded patch set 11 02/08/22, 21:11
-
core-ci voted on patch set 11 V -1 02/08/22, 21:19 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23242
-
crell uploaded patch set 12 02/09/22, 16:03
-
core-ci voted on patch set 12 V -1 02/09/22, 16:18 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23291
-
crell uploaded patch set 13 02/09/22, 20:46
-
core-ci voted on patch set 13 V -1 02/09/22, 20:59 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23304
-
crell uploaded patch set 14 02/14/22, 15:16
-
core-ci voted on patch set 14 V -1 02/14/22, 15:29 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23525
-
crell uploaded patch set 15 02/14/22, 16:56
-
core-ci voted on patch set 15 V -1 02/14/22, 17:09 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23535
-
Benni Mack commented on patch set 15 02/14/22, 17:20
Just added a few comments while reading through the code first time. One thing I do not understand yet: I guess the config objects are built during runtime (= when needed), because we don't know if they will change through ext_localconf.php etc. which makes sense. However, if we actually use the objects a lot (imagine "FE" is used in some objects A LOT, e.g. creating such an object for each link that is generated in PageLinkBuilder for example, when you search for TYPO3_CONF_VARS[FE]) then we somehow should re-use the objects that are built (but only after the boot-state = after all extensions have loaded ext_localconf.php) in the container. right?GfxConfig.php:29how would one deprecate properties like this in the future? I think getters do their job in this regard to easily move forward when removing / deprecating properties. One example would be $processorPathLzw which does not serve any "real" use except for working with TIFF files.ConfigDefinitionPass.php:59looks like a lot of magic here :) pipe()? Unsure if we should introduce fp to TYPO3 code as a "side-effect" with this change.ServiceProvider.php:102are we going to have "serde" wired for everybody available in TYPO3? If so, I think we should go with a stable version / API here. We do use packages like "phpstan" which did not have a major version yet, but these are just dev dependencies unlike serde.FileListController.php:32I'd recommend if we introduce object-based configuration reading, we should name it in a more speaking fashion than the cryptic naming. So I suggest using "\TYPO3\CMS\Core\Configuration\GraphicsConfiguration" -
crell uploaded patch set 16 02/14/22, 17:34
-
Benjamin Franzke commented on patch set 16 02/14/22, 17:48
phpstan.neon:39This is for sure a opinionated "error" that phpstan repots here, still I agree to the phpstan authors that this is code smell and I'm against introducing readonly properties that aren't initialized in the constructor. This isn't properly statically analyzable, therefore it's actually better we avoid it.ConfigDefinitionPass.php:54All this compiler pass does is plain definitions of config classes. This could basically just be the following in Services.yaml: TYPO3\CMS\Core\Config\GfxConfig: factory: ['@TYPO3\\CMS\\Core\\Config\\ConfigFactory', 'getConfigClass'] arguments: ['GFX', 'TYPO3\CMS\Core\Config\ConfigFactory'] Doesn't seem reasonable for me to introduce two new composer dependencies and a compiler pass just to avoid that tiny bit of (actually very readable and introspectable) boilerplate in Services.yamlMemorySpool.php:71no, use makeInstance, it will always use getContainer (which is internal and shouldn't be used here).ServiceProvider.php:112Such Utility function services no purpose in this kind of a service provider. The `core` ServiceProvider is a service provider for the core extension, not a general utility-trash-bin.Services.php:11Please follow the existing coding style. use-statements are avoided if relative class-references are possible in this file.Services.yaml:357redundant with definition in core/Classes/ServiceProvider.phpServices.yaml:369All the "class: …" definitions added here are redundant and therefore superfluous.composer.json:22dependency unused in EXT:install, therefore superfluous in this composer.json -
core-ci voted on patch set 16 V -1 02/14/22, 17:48 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23541
-
crell commented on patch set 16 02/14/22, 17:52
Reply
The config objects are exposed as services, so the container will cache them like any other object already. The first time it's requested the factory will build it and return it, and any subsequent requests will just pull it out of the container's existing cache.GfxConfig.php:29Reply UnresolvedI haven't fully modeled Gfx yet; this is just a dumb-straight-conversion at the moment. In general, though, there's a couple of ways to migrate properties. 1. Serde supports aliases, so you can have a property foo that will pull from the "foo" key in the incoming data (array in this case), and if not found will pull from "bar". 2. The constructor can set $this->bar = $this->foo to create an alias, if we rename bar to foo at some point. Both can be readonly, but only one promoted (the new one, presumably). 3. If #2 is too clumsy or clunky in a specific case, we can also lazily instantiate a readonly property like so: https://peakd.com/hive-168588/@crell/php-tricks-lazy-public-readonly-properties I think that should be enough options to avoid otherwise-redundant getter methods in most cases.ConfigDefinitionPass.php:59Reply UnresolvedIt's utilities in the Crell/fp library that Serde already uses, so it's pulled in either way. I've used it a few times in this code because it's about 1/4 the size of trying to do map, filter, and index-by the slow way, and honestly it's also easier to read once you get used to it. We could do it the foreach()-way, but at least for the moment I went with the way that seemed more ergonomic and compact.ServiceProvider.php:102Reply UnresolvedI don't have a 1.0 of Serde yet, mainly because I'm not 100% certain that some additions won't require a small API change, and in the process of writing these patches I'm spotting an edge case or two in some of the dependencies. (The 0.3.1 release was to account for a bug fix in Crell/fp that this patch found.). It should be fairly stable though, certainly the main front-end usage. (The only thing I can think of to maybe change there would be optional serialization groups, which would only be an optional extra argument.) I can call it 1.0 at whatever point we are comfortable calling it that, though.FileListController.php:32ReplyFine by me. I will rename it to GraphicsConfig in the next patch. I have been using Config as a suffix for now since "Configuration" is already used. I don't have particularly strong feelings about that, though, so I'm OK with most consensus options. (A good IDE will make renaming it to whatever we go with trivial.) -
crell commented on patch set 16 02/14/22, 18:11
phpstan.neon:39ReplyForbidding RO properties not assigned in the constructor would forbid a large number of very valid, and for serialization necessary actions. Specifically, exposing reflection to the attributes simply cannot be done in the constructor. The alternative is to not use readonly properties at all, which would mandate redundant getter methods for every single property, something PHP 8.1 is specifically designed to make unnecessary. I don't see any value in SA dogmatism here if it means making every config class 5x larger. Not every technical code smell is actually smelly. I firmly believe phpstan is incorrect on this one, and the ergonomic cost of following that overly-strict opinion is far too high. It cuts off half the use cases for readonly properties.ConfigDefinitionPass.php:54Reply UnresolvedIf it were 3-4 services I'd probably agree. However, the intent is that it would be a much larger number, and would grow as we split up the deep TYPO3_CONF_VARS arrays and extensions are added and include their own config. So we'd be looking at dozens, in practice. For that, having the classes auto-detect and register automatically is a much better experience for extension developers. It doesn't look like it now, because getClassList() is hard coded. The intent is to make that dynamic by scanning the file system, but I'm not sure how to do that yet. It also makes it a bit easier to change the registration mechanism, which right now is still in development. (I have to add the YAML file lookup yet.)MemorySpool.php:71Reply UnresolvedI ran into problems with that before, which is why I changed it. I can't seem to replicate it now, so I'll try switching it back and see if the bot takes issue with it.ServiceProvider.php:112Reply UnresolvedThis is a little vestigial at the moment. The HTTP config is used during the installer, so it does need to be in ServiceProvider. Likely others are too. I had the Http config object available here previously, but then realized that right now Http has a property that is expected to be callables, which... I don't know what to do with at the moment, so I backed that out. I just didn't back out these pieces. However, this code or something akin to it *will* be necessary for config objects to be available in the installer.Services.php:11ReplyAckServices.yaml:357Reply UnresolvedPer other comment, which one should I remove for now?Services.yaml:369Reply UnresolvedHow do you indicate to the container to load that class, then? It's not in an auto-wired directory, and having the class-name-as-service without any keys under it is invalid YAML syntax, no?composer.json:22Reply UnresolvedAs above, technically true right now but won't be eventually. (Also, is the guideline for what to depend on where documented somewhere? I've mostly just seen "run these lines" recommendations, which include install.) I could back it out for now, but it will come back sooner or later. -
crell uploaded patch set 17 02/15/22, 16:48
-
core-ci voted on patch set 17 V +1 02/15/22, 16:59 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23642
-
crell uploaded patch set 18 02/16/22, 00:03
-
core-ci voted on patch set 18 V +1 02/16/22, 00:11 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23697
-
crell uploaded patch set 19 02/16/22, 22:50
-
core-ci voted on patch set 19 V -1 02/16/22, 23:01 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23763
-
crell commented on patch set 19 02/16/22, 23:03
I think this is now in a finally reviewable state. :-) Some notes: * As noted in some of the previous comments, I included some changes to ServiceProvider that will be necessary to expose config objects to the installer. I added them as part of the HTTP config object, but had to back that out when I realized it had union types, which I don't know how to deal with yet. It's not used currently, but will be eventually. * I am not using the core PHP cache at the moment, as it's currently configured BY the config system, creating a circular dependency. There may be a way to break that, and I'm fine with switching if we can, but this works for the moment. It's not necessarily final. * There's a few @todos in the code for places that are still known rough or are pending some other work first. * I originally tried to use a type map to fold all of the transport-specific objects into a single property with a common interface. I forget why exactly I ran into a wall there but it didn't work properly, so I backed that out. There are other things we can do to fiddle with property names to make them more readable quite easily, thanks to Serde, but that's something we can bikeshed later. * The location/namespace/name of various classes is probably wrong. "Configuration" is already taken. I'm flexible on what pattern we want to go with here. * Still pending is figuring out how to detect config classes. I do not believe it's a good DX for extension developers to manually register them in the container themselves. It's much better to detect them by location or attribute, but I'm not sure of the best way to do that at the moment. Suggestions welcome. * There is no UI or write-back support yet, as expected.FileListController.php:32ReplyDone -
crell uploaded patch set 20 02/16/22, 23:09
-
core-ci voted on patch set 20 V -1 02/16/22, 23:18 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23764
-
crell uploaded patch set 21 02/16/22, 23:46
-
core-ci voted on patch set 21 V +1 02/16/22, 23:54 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23767
-
Benjamin Kott commented on patch set 21 02/17/22, 10:19
Questions: 1. How can I dump the default config for reference? 2. How can I dump the current config? 3. Why is the config within for example mail.yaml not prefixed? General: 1. All directly used packages must be defined as root requirement 2. ... developingphpstan.neon:43UnresolvedI would prefer having each and every occurrence explicitly ignored if nessesary. Even it´s an php stan error, phpstan will then also complain if the error does not occur as soon as the problem is resolved.composer.lock:122Unresolveddireclty used dependencies are not declared in root composer.jsoncomposer.lock:186Unresolveddireclty used dependencies are not declared in root composer.jsonConfigFactory.php:53Unresolvedneeds to have a follow up/ticket afterwards so it does not get lost.ConfigFactory.php:64Unresolvedunresolved todo, iirc - you cannot this needs to be protected by default through the packaged htaccess, IIS config and adjusted documentation for nginx usage. NOTE: this is only the case for "classic" non composer installations, for composer installations the var/cache directory is outside the root.ConfigFactory.php:81Unresolvedhow can the cache be cleared? i guess we are not using the caching framework because it will at some point also handle the config for the caching framework?GraphicsConfig.php:23Unresolvedwhy do we not implement an interface that forces the hydratable?MailConfig.php:51Unresolvedhow is validated that the array keys are integers to be sortable? does that mean we have to reset all values if we only want to extend it? so how does our config look like? a) mail: layoutRootPaths: 30: EXT:superduper/Resources/Private/Layouts/ b) mail: layoutRootPaths: 0: EXT:core/Resources/Private/Layouts/ 10: EXT:backend/Resources/Private/Layouts/ 20: EXT:superduper/Resources/Private/Layouts/MailConfig.php:58Unresolvedthis is ultra hard to read because of missing empty lines after each argumentMailConfig.php:76Unresolvedcan we export the comment block? otherwise the documentation is full of html markup only makes it hard to readMailConfig.php:200Unresolvedthe config object should not contain utility validation methods. the config object should not be able to be constructed with invalid settings also consider `validateEmail` or `isValidEmail` for future implementationsMailConfigDsn.php:7UnresolvedMailDsnConfig?MailConfigMbox.php:10UnresolvedMailMboxConfig?MailConfigSendmail.php:10UnresolvedMailSendmailConfig?MailConfigSmtp.php:11UnresolvedMailSmtpConfig?MailerTest.php:82Unresolvedcode is commented out, can be removedMailerTest.php:94Unresolvedcode is commented out, can be removedConfigFactoryTest.php:47Unresolvedupdate serde please and replace, as benni mentioned if serde is not stable i would not consider it usable for now.ConfigFactoryTest.php:168Unresolvedi am not a fan of that yaml config definitions are not prefixed and the filename seems to be used as context identifierMailConfigSmtpTest.php:10UnresolvedMailSmtpConfigTest? -
crell uploaded patch set 22 02/18/22, 00:52
-
core-ci voted on patch set 22 V -1 02/18/22, 01:01 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23846
-
crell uploaded patch set 23 02/19/22, 00:24
-
core-ci voted on patch set 23 V -1 02/19/22, 00:37 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23893
-
crell abandoned the change 08/01/22, 18:05