Review main Merged

63152[BUGFIX] Reintroduce Extbase referring argument handling

Oliver Hader CR +2 V +2 Hard +563 −3 updated 04/17/20, 13:48

Commit message

Change https://review.typo3.org/c/Packages/TYPO3.CMS/+/61223 (v10 only)
modified Extbase referring argument handling.

This change reintroduces `$internalArguments['__referrer']['arguments']`
to be used when redirecting back to original request action in case some
validation errors occurred - otherwise those arguments are lost.

This basically reverts commit 19b5ee0f3706bf996a4287e6e89449dd71c4bef3
(adjusted for code clean up and refactorings that happened in between)

Review

  • Code review CR +2 Done
    • Alexander Schnitzler +2
    • Torben Hansen +1
  • Verified V +2 Done
    • Alexander Schnitzler +2
    • Torben Hansen +1
    • TYPO3com +1 CI

Issues 1

Files 10

Review log 3548

Show 1523 earlier messages
  1. Oliver Hader uploaded patch set 1 02/02/20, 22:00

  2. TYPO3com voted on patch set 1 V +1 02/02/20, 22:14 Completed build in 14m 04s on Sun, 2 Feb, 10:14 PM Test Summary: 71247 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-40495

  3. Alexander Schnitzler wrote 02/28/20, 16:00

    Topic set to extbase
  4. Alexander Schnitzler commented on patch set 1 04/06/20, 09:18

    Hi Olly, what is the status here? Is this still WIP or ready to be reviewed?
  5. Oliver Hader uploaded patch set 2 04/08/20, 13:12

  6. Oliver Hader updated the commit message in patch set 3 04/08/20, 13:13

  7. Oliver Hader commented on patch set 3 04/08/20, 13:13

    This change is ready for review.
  8. Oliver Hader uploaded patch set 4 04/08/20, 13:14

  9. TYPO3com voted on patch set 2 V -1 04/08/20, 13:21 Completed build in 27s on Wed, 8 Apr, 01:21 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2298

  10. TYPO3com voted on patch set 3 V -1 04/08/20, 13:21 Completed build in 38s on Wed, 8 Apr, 01:21 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2299

  11. TYPO3com voted on patch set 4 V -1 04/08/20, 13:32 Completed build in 11m 46s on Wed, 8 Apr, 01:32 PM Test Summary: 62896 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2300

  12. Oliver Hader uploaded patch set 5 04/08/20, 14:04

  13. TYPO3com voted on patch set 5 V -1 04/08/20, 14:51 Completed build in 39m 05s on Wed, 8 Apr, 02:51 PM Test Summary: 73375 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2304

  14. Oliver Hader uploaded patch set 6 04/08/20, 14:56

  15. Oliver Hader uploaded patch set 7 04/08/20, 15:09

  16. TYPO3com voted on patch set 6 V -1 04/08/20, 15:43 Completed build in 43m 14s on Wed, 8 Apr, 03:43 PM Test Summary: 73375 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2307

  17. TYPO3com voted on patch set 7 V +1 04/08/20, 15:49 Completed build in 24m 01s on Wed, 8 Apr, 03:49 PM Test Summary: 73375 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2308

  18. Oliver Hader commented on patch set 7 04/09/20, 08:08

    Serializer.php:99 Unresolved

    Not sure yet, about this strategy... since "insecure deserialization" is still possible with other class dependencies...
  19. Oliver Hader updated the commit message in patch set 8 04/09/20, 13:22

  20. TYPO3com voted on patch set 8 V +1 04/09/20, 13:51 Completed build in 29m 22s on Thu, 9 Apr, 01:51 PM Test Summary: 73381 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2348

  21. Alexander Schnitzler commented on patch set 8 04/12/20, 10:49

    Hi folks, I had a look at this patch and since we are about to introduce a Serializer now, I thought about the benefits of using the native (PHP) serialization ob objects here and if it's not possible to have a completely different strategy here. Might be a real stupid idea but as I was working a lot with symfony/serializer these days, I wondered if we can't normalize each object that is passed on into an array and then serialize as json/xml/youNameIt. That way, when passing data to an action, it is supposed to be an array of property values which can either be set directly (public properties/often used in dto's) or via setters. That way, we don't need to take care of what types of objects are allowed to be deserialized. We simply don't allow it. I know, this is a totally different idea but I really wonder if we need native (de)serialization at this point.
  22. Oliver Hader commented on patch set 8 04/12/20, 11:18

    > Patch Set 8: > > Hi folks, > I had a look at this patch and since we are about to introduce a Serializer now, I thought about the benefits of using the native (PHP) serialization ob objects here and if it's not possible to have a completely different strategy here. Please don't focus too much on the name `Serializer` here, it's just a class name and not to be mixed with other concepts you referred to. I'd be fine calling it `SerializationService` or `SomeOtherThingy`... > Might be a real stupid idea but as I was working a lot with symfony/serializer these days, I wondered if we can't normalize each object that is passed on into an array and then serialize as json/xml/youNameIt. All serialization approaches have a problem with security - since they are going to assign (magically) properties and behavior to objects at some point. The vulnerability then stems from the fact that this data is user submitted... In general, using `symfony/serializer` is a different discussion for solving a different problem & use-case. Besides that, models needs to define what can be serialized and how it's supposed to be used. > That way, when passing data to an action, it is supposed to be an array of property values which can either be set directly (public properties/often used in dto's) or via setters. That way, we don't need to take care of what types of objects are allowed to be deserialized. We simply don't allow it. That's the regression that shall be solved with this patch - especially for nested objects. But again: Just using scalar types does not protect against security vulnerabilities since the model and their consumers decide how properties are used. > I know, this is a totally different idea but I really wonder if we need native (de)serialization at this point. Currently we do - previous request arguments are stored (serialized) in a HTML form and handed over to the next request. A more secure approach would avoid round-trip information via HTTP. The functional test cases demonstrate the problem, feel free to play around with them and get your own impression.
  23. Torben Hansen updated the commit message in patch set 9 04/13/20, 12:26

  1. TYPO3com voted on patch set 9 V +1 04/13/20, 12:42 Completed build in 15m 15s on Mon, 13 Apr, 12:42 PM Test Summary: 73421 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2442

  2. Alexander Schnitzler commented on patch set 9 V +1 CR -1 04/13/20, 13:30

    Serializer.php:57 Unresolved

    This is pretty much the only thing I'd change for now. The exception message could be more declarative. Something like: The request contained potentially insecure serialized objects (classes with either __destruct or __wakeup methods) which is not allowed. Since we are introducing this new security features as a breaking change, we should also add a breaking rst to make sure this change is seen by our users, especially those that update their extensions after the merge of this patch and those that didn't realise the former change already.
  3. Torben Hansen voted on patch set 9 V +1 04/13/20, 17:58

  4. Alexander Schnitzler uploaded patch set 10 04/15/20, 14:41

  5. TYPO3com voted on patch set 10 V -1 04/15/20, 14:41 Completed build in 23s on Wed, 15 Apr, 02:41 PM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2587

  6. Alexander Schnitzler uploaded patch set 11 04/15/20, 14:47

  7. Alexander Schnitzler commented on patch set 11 04/15/20, 14:49

    GenericViewResolver.php:18 Unresolved

    Needs to be reverted -> separate patch

    ViewResolverInterface.php:5 Unresolved

    Needs to be reverted -> separate patch
  8. TYPO3com voted on patch set 11 V +1 04/15/20, 14:59 Completed build in 11m 56s on Wed, 15 Apr, 02:59 PM Test Summary: 73535 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2588

  9. Oliver Hader commented on patch set 11 CR -1 04/15/20, 15:28

    on it...
  10. Francois Suter commented on patch set 11 04/15/20, 17:23

    It works for me. However the patch does not apply cleanly and touches on some classes which don't exist anymore: "typo3/sysext/extbase/Classes/Mvc/View/GenericViewResolver.php" and "typo3/sysext/extbase/Classes/Mvc/View/ViewResolverInterface.php".
  11. Alexander Schnitzler commented on patch set 11 04/15/20, 17:32

    > Patch Set 11: > > It works for me. However the patch does not apply cleanly and touches on some classes which don't exist anymore: "typo3/sysext/extbase/Classes/Mvc/View/GenericViewResolver.php" and "typo3/sysext/extbase/Classes/Mvc/View/ViewResolverInterface.php". If the patch does bot apply properly your base revision is outdated. The classes exist in master but the changes are superfluous nonetheless.
  12. Francois Suter commented on patch set 11 04/15/20, 17:39

    > Patch Set 11: > > > Patch Set 11: > > > > It works for me. However the patch does not apply cleanly and touches on some classes which don't exist anymore: "typo3/sysext/extbase/Classes/Mvc/View/GenericViewResolver.php" and "typo3/sysext/extbase/Classes/Mvc/View/ViewResolverInterface.php". > > If the patch does bot apply properly your base revision is outdated. The classes exist in master but the changes are superfluous nonetheless. Ah I see. I'm not indeed on latest revision.
  13. Oliver Hader commented on patch set 11 04/15/20, 17:44

    I had a call with Helmut this afternoon (thanks to him for the suggestions). In general I'd just focus on bringing back the previous behavior (basically reverting the regression) without adding more checks on magic methods and the link. Basically checking for __destruct, __wakeup was just a mitigation... however there's much more and it's basically never really secure. Thus, having the HMAC signature on those arguments will then be the "only" protection - which is fine, since it requires the `encryptionKey` - my initial plan to harden that area even further failed. Sorry for the hassle my changes in Dec 2019 have caused...
  14. Alexander Schnitzler commented on patch set 11 04/15/20, 18:29

    > Patch Set 11: > > I had a call with Helmut this afternoon (thanks to him for the suggestions). > In general I'd just focus on bringing back the previous behavior (basically reverting the regression) without adding more checks on magic methods and the link. > > Basically checking for __destruct, __wakeup was just a mitigation... however there's much more and it's basically never really secure. Thus, having the HMAC signature on those arguments will then be the "only" protection - which is fine, since it requires the `encryptionKey` - my initial plan to harden that area even further failed. > > Sorry for the hassle my changes in Dec 2019 have caused... Ok, so should we then use my patch I abandoned these days? https://review.typo3.org/c/Packages/TYPO3.CMS/+/62866
  15. Oliver Hader commented on patch set 11 04/16/20, 10:01

    > Patch Set 11: > > > Patch Set 11: > > > > I had a call with Helmut this afternoon (thanks to him for the suggestions). > > In general I'd just focus on bringing back the previous behavior (basically reverting the regression) without adding more checks on magic methods and the link. > > > > Basically checking for __destruct, __wakeup was just a mitigation... however there's much more and it's basically never really secure. Thus, having the HMAC signature on those arguments will then be the "only" protection - which is fine, since it requires the `encryptionKey` - my initial plan to harden that area even further failed. > > > > Sorry for the hassle my changes in Dec 2019 have caused... > > Ok, so should we then use my patch I abandoned these days? > https://review.typo3.org/c/Packages/TYPO3.CMS/+/62866 No, because that on still uses `json_encode` and does not solve the regression... :(
  16. Alexander Schnitzler commented on patch set 11 04/16/20, 10:14

    > Patch Set 11: > > > Patch Set 11: > > > > > Patch Set 11: > > > > > > I had a call with Helmut this afternoon (thanks to him for the suggestions). > > > In general I'd just focus on bringing back the previous behavior (basically reverting the regression) without adding more checks on magic methods and the link. > > > > > > Basically checking for __destruct, __wakeup was just a mitigation... however there's much more and it's basically never really secure. Thus, having the HMAC signature on those arguments will then be the "only" protection - which is fine, since it requires the `encryptionKey` - my initial plan to harden that area even further failed. > > > > > > Sorry for the hassle my changes in Dec 2019 have caused... > > > > Ok, so should we then use my patch I abandoned these days? > > https://review.typo3.org/c/Packages/TYPO3.CMS/+/62866 > > No, because that on still uses `json_encode` and does not solve the regression... :( I am unable to follow any more. Will you create a patch that reverts your initial changes?
  17. Oliver Hader commented on patch set 11 04/16/20, 10:28

    > Patch Set 11: > > > Patch Set 11: > > > > > Patch Set 11: > > > > > > > Patch Set 11: > > > > > > > > I had a call with Helmut this afternoon (thanks to him for the suggestions). > > > > In general I'd just focus on bringing back the previous behavior (basically reverting the regression) without adding more checks on magic methods and the link. > > > > > > > > Basically checking for __destruct, __wakeup was just a mitigation... however there's much more and it's basically never really secure. Thus, having the HMAC signature on those arguments will then be the "only" protection - which is fine, since it requires the `encryptionKey` - my initial plan to harden that area even further failed. > > > > > > > > Sorry for the hassle my changes in Dec 2019 have caused... > > > > > > Ok, so should we then use my patch I abandoned these days? > > > https://review.typo3.org/c/Packages/TYPO3.CMS/+/62866 > > > > No, because that on still uses `json_encode` and does not solve the regression... :( > > I am unable to follow any more. Will you create a patch that reverts your initial changes? Yes, unfortunately it's a mess... a simple revert is not possible and it does not help that code formatting changes and blindly moving around code was done. With having more tests in place I tried to document and have the behavior reproducible in code... seems that this as well was too complicated... Sure, I'm going to create patches to get back to a previous and working state having the possibility to make use of instances in Extbase arguments - similar as it has been in TYPO3 v9.
  18. Oliver Hader uploaded patch set 12 04/16/20, 11:06

  19. TYPO3com voted on patch set 12 V -1 04/16/20, 11:06 Completed build in 25s on Thu, 16 Apr, 11:06 AM Test Summary: No tests found Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2646

  20. Oliver Hader uploaded patch set 13 04/16/20, 11:16

  21. TYPO3com voted on patch set 13 V +1 04/16/20, 11:27 Completed build in 11m 16s on Thu, 16 Apr, 11:27 AM Test Summary: 73567 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-2647

  22. Alexander Schnitzler voted on patch set 13 V +1 CR +1 04/16/20, 11:52

  23. Torben Hansen voted on patch set 13 V +1 CR +1 04/16/20, 12:39

  24. Alexander Schnitzler voted on patch set 13 V +2 CR +2 04/17/20, 13:48

  25. Alexander Schnitzler merged patch set 14 04/17/20, 13:48