64074[BUGFIX] Correctly override storagePid
Commit message
Review
-
Code review CR 0
-
Verified V 0
Issues 1
Files 4
- typo3/sysext/extbase/Classes/Configuration/AbstractConfigurationManager.php+1 −1
- typo3/sysext/extbase/Classes/Configuration/FrontendConfigurationManager.php16+30 −4
- typo3/sysext/extbase/Tests/Unit/Configuration/AbstractConfigurationManagerTest.php2+372 −0
- typo3/sysext/extbase/Tests/Unit/Configuration/FrontendConfigurationManagerTest.php+19 −1
Review log 4855
Show 2430 earlier messages
-
Christian Futterlieb uploaded patch set 1 04/04/20, 14:43
-
TYPO3com voted on patch set 1 V +1 04/04/20, 14:55 Completed build in 11m 38s on Sat, 4 Apr, 02:55 PM Test Summary: 73294 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2169
-
Alexander Schnitzler wrote 04/04/20, 15:15
Topic set to extbase -
Alexander Schnitzler commented on patch set 1 CR -1 04/04/20, 15:28
I think a unit test is not enough here. I need to see that this has an effect later on when the stdWrap is actually applied. And we should probably move this change into the AbstractConfigurationManager to have also the backend benefit from this change. -
Christian Futterlieb commented on patch set 1 04/04/20, 15:35
> I think a unit test is not enough here. I need to see that this has an effect later on when the stdWrap is actually applied. Ok, I'll add another test for AbstractConfigurationManager::getConfiguration() > And we should probably move this change into the AbstractConfigurationManager to have also the backend benefit from this change. As far as I can see, this does not apply to backend at all. It is just about the overriding, when $contentObject->data['pages'] is set. Which only happens in frontend. Or am I missing something? -
Alexander Schnitzler commented on patch set 1 04/04/20, 15:36
> Patch Set 1: > > > I think a unit test is not enough here. I need to see that this has an effect later on when the stdWrap is actually applied. > > Ok, I'll add another test for AbstractConfigurationManager::getConfiguration() > > > And we should probably move this change into the AbstractConfigurationManager to have also the backend benefit from this change. > > As far as I can see, this does not apply to backend at all. It is just about the overriding, when $contentObject->data['pages'] is set. Which only happens in frontend. Or am I missing something? Yes, you are right. I am also digging into the code right now. -
Alexander Schnitzler commented on patch set 1 04/04/20, 16:21
So, I built a test setup but I fail to reproduce the issue. If I understand typoscript right, the ifEmpty uses the assigned value (2 in your test) whenever the persistence.storagePid is empty. I wonder when this is the case as the AbstractConfigurationManager falls back to 1 in this case with getDefaultBackendStoragePid(). I also don't understand the description on forge very well. Can you please explain in more detail? -
Christian Futterlieb commented on patch set 1 04/04/20, 17:39
> So, I built a test setup but I fail to reproduce the issue. If I understand typoscript right, the ifEmpty uses the assigned value (2 in your test) whenever the persistence.storagePid is empty. I wonder when this is the case as the AbstractConfigurationManager falls back to 1 in this case with getDefaultBackendStoragePid(). First: getDefaultBackendStoragePid() would return AbstractConfigurationManager::DEFAULT_BACKEND_STORAGE_PID which is zero. Second: it is not about the 'ifEmpty' construct, but the fact, that the 'startingpoint' (tt_content.pages) is ignored in any scenario, where storagePid is configured with stdWrap properties. Example TypoScript: plugin.tx_myext.persistence { storagePid.append.cObject = TEXT storagePid.append.cObject.value = ,999 } Combined with tt_content.pages = 111 this SHOULD lead to '111,999' as storagePid. But it would be ',999'. > I also don't understand the description on forge very well. Can you please explain in more detail? The problem lies in two places: 1. ArrayUtility::mergeRecursiveWithOverrule() would not overwrite a value in $array1, when it is an array and the same offset in $array2 is not. Which is the case in this very situation. I don't know if this is a problem on it's own, but I could imagine, that changing this might break one thing or another.. 2. When storagePid is an array, it will be changed to plain TypoScript by TypoScriptService::convertPlainArrayToTypoScriptArray() (in AbstractConfigurationManager::getConfiguration()). In this method, the value of storagePid gets reconstituted from $storagePid['_typoScriptNodeValue']. Therefor, the storagePid value must be assigned as '_typoScriptNodeValue' if the storagePid configuration is an array. I hope, I explained it better now. -
Alexander Schnitzler commented on patch set 1 04/05/20, 08:08
Ok, I understand it better now but I still see some issues with this patch. The change in the FrontendConfigurationManager is only executed if tt_content.pages is set (a non empty string). This means, the ifEmpty is only applied later on if tt_content.pages is set which makes the ifEmpty useless. What you are trying to fix with this patch is two different things: 1) Evaluating persistence.storagePid of it's an array (like when it's a stdWrap) 2) Applying ifEmpty via stdWrap if tt_content.pages is not set. With this patch, 1) is solved but 2) isn't. -
Alexander Schnitzler uploaded patch set 2 04/05/20, 08:23
-
Alexander Schnitzler commented on patch set 2 04/05/20, 08:25
FrontendConfigurationManager.php:134What do you think about this? If storagePid is an array, it's node value should be set to an empty string imo. This makes stdWrap possible. And if stdWrap is not used, the node get a proper value. -
TYPO3com voted on patch set 2 V +1 04/05/20, 08:35 Completed build in 11m 47s on Sun, 5 Apr, 08:35 AM Test Summary: 73384 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2188
-
Christian Futterlieb commented on patch set 2 04/05/20, 10:33
> Ok, I understand it better now but I still see some issues with this patch. The change in the FrontendConfigurationManager is only executed if tt_content.pages is set (a non empty string). Correct. > This means, the ifEmpty is only applied later on if tt_content.pages is set which makes the ifEmpty useless. That's where you got it wrong. As I wrote before, it is not about the ifEmpty construction (see also the ts example in my last comment). The point is, that one would want the value of tt_content.pages be fed into ContentObjectRenderer::stdWrap($content, $conf) as the first argument. I'll walk you through the problem. 1. If we take this TypoScript: plugin.tx_myext.persistence { storagePid = 1 } 2. and tt_content.pages is empty 3. then we'd get the result of: ContentObjectRenderer::stdWrap('1', []); as $configuration['persistence']['storagePid'] from AbstractConfigurationManager::getConfiguration(). Which is to be expected so far. -- Changing the situation to: 1. TypoScript: plugin.tx_myext.persistence { storagePid = 1 } 2. tt_content.pages = 3,4 3. Resulting stdWrap: ContentObjectRenderer::stdWrap('3,4', []); Still ok, right? Right. -- Another change: 1. TypoScript: plugin.tx_myext.persistence { storagePid.append = TEXT storagePid.append.value = ,5 } 2. tt_content.pages = 3,4 3. Resulting stdWrap: v--- This is the problem. Here should be the value of ContentObjectRenderer::stdWrap('', [ tt_content.pages. But it is the value of 'append' => 'TEXT' plugin.tx_myext.persistence.storagePid (which is 'append.' => [ empty in this example) 'value' => ',5' ] ]); -
Christian Futterlieb commented on patch set 2 CR -1 04/05/20, 10:39
FrontendConfigurationManager.php:134ReplyThis is (excuse me) a terrible idea. With this, you'd remove any value, that was set via TypoScript in plugin.tx_myext.persistence.storagePid before. And if $frameworkConfiguration['persistence']['storagePid']['_typoScriptNodeValue'] was empty (or not set) before, there would be no point in setting it again to nothing. -
Alexander Schnitzler commented on patch set 2 04/05/20, 16:23
FrontendConfigurationManager.php:134Reply Unresolved> This is (excuse me) a terrible idea. With this, you'd remove any value, that was set via TypoScript in plugin.tx_myext.persistence.storagePid before Looking at my additional tests I tend to disagree with you here. > And if $frameworkConfiguration['persistence']['storagePid']['_typoScriptNodeValue'] was empty (or not set) before, there would be no point in setting it again to nothing. If $frameworkConfiguration['persistence']['storagePid'] is an array, we assume the user is using a stdWrap, right? In that case, We need a structure like this to be put into the stdWrap method: $frameworkConfiguration['persistence']['storagePid'] = ''; $frameworkConfiguration['persistence']['storagePid.'] = ['ifEmpty' => '2']; If we don't set $frameworkConfiguration['persistence']['storagePid']['_typoScriptNodeValue'] to an empty string, the following is missing after the conversion of the typoscript syntax: $frameworkConfiguration['persistence']['storagePid'] = ''; And without that key set, the call of stdWrap fails. Please see my tests, disable this additional block of code and then explain how the right solution looks like. With just your additional block of code, the stdWrap is not executed. -
Christian Futterlieb commented on patch set 2 04/06/20, 11:22
Just a short heads-up: as I'm digging deeper into the topic, it occurs to me, that the problem is broader than I thought. The storagePid overriding is not just done once, but (potentially) thrice in: 1. overrideStoragePidIfStartingPointIsSet() 2. overrideConfigurationFromPlugin() 3. overrideConfigurationFromFlexForm() I'm currently working on a solution, that fixes this. -
Christian Futterlieb uploaded patch set 3 04/06/20, 12:37
-
TYPO3com voted on patch set 3 V -1 04/06/20, 12:37 Completed build in 24s on Mon, 6 Apr, 12:37 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2229
-
Christian Futterlieb commented on patch set 3 04/06/20, 12:50
> > This is (excuse me) a terrible idea. With this, you'd remove any value, that was set via TypoScript in plugin.tx_myext.persistence.storagePid before > Looking at my additional tests I tend to disagree with you here. I just moved the test to AbstractConfigurationManager, when stdWrap is called (using $conf['storagePid'] ?? '' as first argument). This will solve the problem in every case without (potentially) overriding anything. Thanks Alexander for your unit tests. I changed them to be fed from a DataProvider and added some additional use cases. I could not get the tests to work with flexform config. I added them as comment anyway, if you are willing to make this work..? Sorry, I'm not very familiar with testing in TYPO3 :( -
Alexander Schnitzler commented on patch set 3 04/06/20, 13:00
> Patch Set 3: > > > > This is (excuse me) a terrible idea. With this, you'd remove any value, that was set via TypoScript in plugin.tx_myext.persistence.storagePid before > > > Looking at my additional tests I tend to disagree with you here. > > I just moved the test to AbstractConfigurationManager, when stdWrap is called (using $conf['storagePid'] ?? '' as first argument). This will solve the problem in every case without (potentially) overriding anything. > > Thanks Alexander for your unit tests. I changed them to be fed from a DataProvider and added some additional use cases. I could not get the tests to work with flexform config. I added them as comment anyway, if you are willing to make this work..? Sorry, I'm not very familiar with testing in TYPO3 :( I will have a look. Will you fix the cgl stuff and push a new patch set? -
Alexander Schnitzler uploaded patch set 4 04/06/20, 14:06
-
Christian Futterlieb commented on patch set 4 04/06/20, 14:33
> Will you fix the cgl stuff and push a new patch set? Sorry, just got back. Thx for fixing yourself anyway! -
TYPO3com voted on patch set 4 V +1 04/06/20, 14:59 Completed build in 40m 38s on Mon, 6 Apr, 02:59 PM Test Summary: 73426 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2232
-
Stefan Froemken uploaded patch set 5 04/06/20, 17:34
-
TYPO3com voted on patch set 5 V -1 04/06/20, 17:46 Completed build in 12m 05s on Mon, 6 Apr, 05:46 PM Test Summary: 73426 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2239
-
Oliver Klee commented on patch set 5 04/06/20, 17:50
Ie. -> i.e.Make the other array do what exactly?FrontendConfigurationManager.php:288The comma needs to go away.FrontendConfigurationManager.php:294Please let's have a void return type declaration here. If I see this correctly, $frameworkConfiguration is not used in this method. Is this a bug? Or can we remove the argument?AbstractConfigurationManagerTest.php:388Please let's have a type annotation for $expectedResult. -
Stefan Froemken uploaded patch set 6 04/06/20, 18:08
-
Stefan Froemken commented on patch set 6 04/06/20, 18:09
Reply
DoneReply
DoneFrontendConfigurationManager.php:288ReplyDoneFrontendConfigurationManager.php:294ReplyDoneFrontendConfigurationManager.php:305UnresolvedSorry, I don't want to see that value here in Code. That's why I parse the newConfiguration value throw convertTypoScriptArrayToPlainArray, too. The only problem is: If storagePid in newConfiguration does not have any sub-array "storagePid", the "_typoScriptNodeValue" key will not be created. That's why I added an invalidKey to newConfiguration. Now we have the same structure of arrays on both sides.AbstractConfigurationManagerTest.php:388ReplyDone -
TYPO3com voted on patch set 6 V +1 04/06/20, 18:20 Completed build in 12m 33s on Mon, 6 Apr, 06:20 PM Test Summary: 73426 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2240
-
Christian Futterlieb commented on patch set 6 CR -1 04/06/20, 18:22
FrontendConfigurationManager.php:306Hey Stefan I don't get why you removed the parameter $frameworkConfiguration? It is necessary to adjust both configurations: if either of those has an array in persistence.storagePid, the other one must have it too, that was the whole point.
-
Oliver Klee commented on patch set 6 04/06/20, 18:38
FrontendConfigurationManager.php:306ReplyThat might have been due to my comment that the parameter was unused: https://review.typo3.org/c/Packages/TYPO3.CMS/+/64074/5/typo3/sysext/extbase/Classes/Configuration/FrontendConfigurationManager.php#294 -
Christian Futterlieb commented on patch set 6 04/06/20, 18:40
FrontendConfigurationManager.php:306ReplyI meant the decision of Stefan before to not expand $frameworkConfiguration -
Christian Futterlieb commented on patch set 6 04/06/20, 18:45
FrontendConfigurationManager.php:306ReplyAnd: I see that there's a good chance, that we'll end up with an array in $newConfiguration['persistence']['storagePid'], which cannot be the right way. This value then gets passed as first argument into ContentObjectRenderer::stdWrap(), which is expected to be string. Just to clarify things: I get, that you don't want '_typoScriptNodeValue' to appear in the code. And that we seem to have to 'swallow' a superfluous conversion therefor. -
Christian Futterlieb commented on patch set 6 04/06/20, 19:30
Hey, please checkout this idea: https://gist.github.com/christianfutterlieb/766bdba35ac78f43f8c389031265144d I guess this would solve the problem with the _typoScriptNodeValue and not leave out the expansion of $frameworkConfiguration, what do you think? -
Stefan Froemken commented on patch set 6 04/06/20, 20:27
FrontendConfigurationManager.php:306ReplyAs FrameworkConfiguration is already a result of typoScriptConverter, so there is no need for further investigations on this side. The problem is the newConfiguration. Don't forget that the next called method is the mergeRescursiveWithOverrule. After that you have a perfect array and all tests are green. If you think that something went wrong please create a new UnitTests to get things clearer. -
Christian Futterlieb uploaded patch set 7 04/07/20, 00:59
-
TYPO3com voted on patch set 7 V -1 04/07/20, 01:16 Completed build in 16m 40s on Tue, 7 Apr, 01:16 AM Test Summary: 6 of 73913 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2245
-
Christian Futterlieb commented on patch set 7 04/07/20, 01:16
FrontendConfigurationManager.php:306ReplyI added the new test to prove my point: there we end up with an array in persistence.storagePid AFTER forming it back to a typoscript array, leading to an erroneous array-to-string conversion in ContentObjectRenderer::stdWrap(). I think it is key to make the persistence.storagePid in both ($frameworkConfiguration and $newConfiguration) array, if the other already is. -
Alexander Schnitzler commented on patch set 7 04/07/20, 07:53
FrontendConfigurationManager.php:306ReplyHe guys, I feel this discussion is currently not leading anywhere. Let's come back to a discussion that is not about code but about what we want to achieve and then come back to its implementation. I have a general question regarding this method. What is its specific purpose? From just reading the method name I don't know what it should do. Please describe the idea you have with this method. What is the problem this method has a solution for? -
Christian Futterlieb commented on patch set 7 04/07/20, 09:00
> He guys, I feel this discussion is currently not leading anywhere. Let's come back to a discussion that is not about code but about what we want to achieve and then come back to its implementation. I'd like to fix the problem initially described in the forge issue. While working on the patch I realized, that it is more generally about the combination of a) merging the different configurations and b) running a sub-section of the result through TypoScriptService::convertTypoScriptArrayToPlainArray() and stdWrap() My intended solution is the adjustment of the affected parts of the configuration arrays (if needed) right before merging them together. As described in https://gist.github.com/christianfutterlieb/766bdba35ac78f43f8c389031265144dFrontendConfigurationManager.php:306Reply UnresolvedOk, I wrote my arguments in the main comments section. > I have a general question regarding this method. What is its specific purpose? From just reading the method name I don't know what it should do. Please describe the idea you have with this method. What is the problem this method has a solution for? It should adjust the value of persistence.storagePid in $frameworkConfiguration and $newConfiguration to avoid problems afterwards with ArrayUtility::mergeRecursiveWithOverrule(). Therefor the name would be better 'adjustStoragePidStructureBeforeMerge()' -
Alexander Schnitzler commented on patch set 7 04/07/20, 09:04
> Patch Set 7: > > (1 comment) > > > He guys, I feel this discussion is currently not leading anywhere. Let's come back to a discussion that is not about code but about what we want to achieve and then come back to its implementation. > > I'd like to fix the problem initially described in the forge issue. While working on the patch I realized, that it is more generally about the combination of > a) merging the different configurations and > b) running a sub-section of the result through TypoScriptService::convertTypoScriptArrayToPlainArray() and stdWrap() > > My intended solution is the adjustment of the affected parts of the configuration arrays (if needed) right before merging them together. As described in https://gist.github.com/christianfutterlieb/766bdba35ac78f43f8c389031265144d What I don't understand is why you want to fiddle around with the TS-Configuration before it gets merged aka overridden with more specific typoscript from the extension-ts, plugin-ts or flexform-ts. The main idea is to first fetch all the data and then apply the stdWrap, right? So why should we adjust the incoming TS before we have the final TS available? -
Christian Futterlieb commented on patch set 7 04/07/20, 09:44
We must adjust them because ArrayUtility::mergeRecursiveWithOverrule() would lose some options when they are not the same structure Example situation 1: mergeRecursiveWithOverrule( [ 'persistence' => [ 'storagePid' => [ '_typoScriptNodeValue' => '111', 'someStdWrapProp' => '' ] ] ], [ 'persistence' => [ 'storagePid' => '222' ] ] ); This will completely ignore the second storagePid (222). Example situation 2: mergeRecursiveWithOverrule( [ 'persistence' => [ 'storagePid' => '111' ] ], [ 'persistence' => [ 'storagePid' => [ 'wrap' => '|,222' ] ] ] ); This will completely ignore the first storagePid (111), leading not to '111,222' as expected, but ',222'. Note: if persistence.storagePid would always contain ints or comma separated int-lists, there wouldn't be a problem. That's what all existing unit tests expect. -
Alexander Schnitzler commented on patch set 7 04/07/20, 17:11
> Patch Set 7: > > We must adjust them because ArrayUtility::mergeRecursiveWithOverrule() would lose some options when they are not the same structure > > Example situation 1: > > mergeRecursiveWithOverrule( > [ > 'persistence' => [ > 'storagePid' => [ > '_typoScriptNodeValue' => '111', > 'someStdWrapProp' => '' > ] > ] > ], > [ > 'persistence' => [ > 'storagePid' => '222' > ] > ] > ); > > This will completely ignore the second storagePid (222). > > Example situation 2: > > mergeRecursiveWithOverrule( > [ > 'persistence' => [ > 'storagePid' => '111' > ] > ], > [ > 'persistence' => [ > 'storagePid' => [ > 'wrap' => '|,222' > ] > ] > ] > ); > > This will completely ignore the first storagePid (111), leading not to '111,222' as expected, but ',222'. > > Note: if persistence.storagePid would always contain ints or comma separated int-lists, there wouldn't be a problem. That's what all existing unit tests expect. OK, this made things much clearer. You want to check for an array structure in all available configurations. I will take your path set and test with the data you provided via data providers. -
Christian Futterlieb commented on patch set 7 04/07/20, 18:09
> OK, this made things much clearer. You want to check for an array structure in all available configurations. Exactly. > I will take your path set and test with the data you provided via data providers. Cool, thx. Just to be sure: hope you mean the code here: https://gist.github.com/christianfutterlieb/766bdba35ac78f43f8c389031265144d =) -
Benni Mack commented on patch set 7 11/04/21, 11:30
Hey Christian, care to update this patch with the comments from Alex? -
Benni Mack updated the commit message in patch set 8 11/18/21, 10:44
-
Christian Futterlieb commented on patch set 8 11/18/21, 20:05
Reply
Sorry my late answer. Yes, I'll do it next week -
Benni Mack wrote 11/28/21, 22:06
Removed cc Benni Mack. -
Benni Mack wrote 11/29/21, 10:21
Change destination moved from master to main -
Benni Mack updated the commit message in patch set 9 11/29/21, 14:09
-
Oliver Klee wrote 11/29/21, 14:19
Removed cc Oliver Klee. -
Alexander Schnitzler wrote 04/01/23, 17:17
Removed reviewer Alexander Schnitzler. -
Benni Mack updated the commit message in patch set 10 05/03/23, 09:41
-
Benni Mack commented on patch set 10 07/04/23, 07:39
anyone up for rebasing this one? Otherwise I will abandon this change, as we don't have an update in the past 1.5 years -
Benni Mack abandoned the change 02/05/24, 07:19
abandoning this change