Review main Merged

75417[FEATURE] Let class schema detect multiple property types

Alexander Schnitzler CR +2 V +2 Expert +1121 −78 updated 07/26/23, 18:07

Commit message

Extbase creates class schema objects for classes to gather
information for further analysis. One part of that information
is property types. In recent TYPO3 and PHP versions, extbase did
expect a property to have zero or one type.

Due to changes to PHP (Union Types/Intersection Types), extbase's
internal behaviour needs to change and detect all declared types.

This is especially useful in models whose properties can now be
declared as

public Entity|LazyLoadingProxy|null $reference = null;

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Oliver Klee +1
    • Markus Klein +1
    • Stefan Froemken +1
  • Verified V +2 Done
    • Benni Mack +2
    • Oliver Klee +1
    • Markus Klein +1
    • Stefan Froemken +1
    • core-ci +1 CI

Issues 1

Files 21

Show 1 more file

Review log 7091

Show 5066 earlier messages
  1. Alexander Schnitzler uploaded patch set 1 08/11/22, 12:01

  2. Alexander Schnitzler wrote 08/11/22, 12:19

    Topic set to extbase
  3. core-ci voted on patch set 1 V -1 08/11/22, 13:02 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/30038

  4. Alexander Schnitzler uploaded patch set 2 08/11/22, 13:22

  5. core-ci voted on patch set 2 V -1 08/11/22, 13:32 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/30045

  6. Alexander Schnitzler uploaded patch set 3 08/11/22, 13:45

  7. core-ci voted on patch set 3 V -1 08/11/22, 13:53 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/30047

  8. Benni Mack commented on patch set 3 08/11/22, 15:11

    The Extbase master is back again. I love it!
  9. Alexander Schnitzler uploaded patch set 4 08/11/22, 16:16

  10. core-ci voted on patch set 4 V +1 08/11/22, 16:24 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/30050

  11. Alexander Schnitzler marked it work in progress 08/11/22, 16:56

  12. Alexander Schnitzler uploaded patch set 5 08/12/22, 09:45

  13. core-ci voted on patch set 5 V -1 08/12/22, 09:54 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/30070

  14. Alexander Schnitzler uploaded patch set 6 08/12/22, 09:55

  15. core-ci voted on patch set 6 V +1 08/12/22, 10:03 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/30071

  16. Alexander Schnitzler uploaded patch set 7 08/12/22, 10:21

  17. core-ci voted on patch set 7 V +1 08/12/22, 10:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/30075

  18. Alexander Schnitzler marked it ready for review 08/12/22, 10:31

  19. Oliver Klee commented on patch set 7 08/12/22, 10:52

    Awesome! I've added some remarks.

    DataMapFactory.php:167

    We're accessing $propertyTypes[0] multiple times here. Maybe we can reduce this using a well-named local variable.

    DataMapper.php:605

    Please let's avoid assignments in conditions as much as possible, and move the assignment to a separate line (which also would allow us to get rid of the hidden Yoda condition).

    DataMapper.php:770

    To make the code more readable, I propose moving $primaryType->getCollectionValueTypes()[0] to a local variable with a null fallback.

    ObjectConverter.php:132

    Is this TODO still valid? If so, should it get resolved in this change, or should we have a ticket instead so it doesn't get lost?

    PersistentObjectConverter.php:140

    Ditto about the assignment in a condition and the hidden Yoda condition.

    PersistentObjectConverter.php:146

    Ditto about moving $primaryType->getCollectionValueTypes()[0] to a local variable.

    Property.php:91

    Ditto about assignment and Yoda.

    ValidatorResolver.php:117

    Are these TODOs still valid?

    DateTimeImmutableExample.php:28

    Can we move all this cleanup to a separate pre-patch?

    PropertyWithTypeDeclarationsTest.php:56

    Ditto about data providers.

    PropertyWithoutTypeDeclarationsTest.php:37

    Can me make this more specific with assertSame([], …)?

    PropertyWithoutTypeDeclarationsTest.php:74

    The structure and similarity of these tests really beg for being converted to data provides.
  20. Alexander Schnitzler commented on patch set 7 08/12/22, 10:57

    ValidatorResolver.php:117 Reply

    Yes, the Resolver still needs to be reworked.

    DateTimeImmutableExample.php:28 Reply

    This is kind of relevant for the test suite. We could of course only convert the union types here (ObjectStorages and LazyLoadingProxy) but is it really worth the hassle? I am dealing with very limited time. If that change is important, would you mind taking over those changes?
  21. Oliver Klee commented on patch set 7 08/12/22, 11:13

    DateTimeImmutableExample.php:28 Reply

    Yes, I can do that. Is there an easy way for me to know the cleanup from the non-cleanup?
  22. Alexander Schnitzler commented on patch set 7 08/12/22, 11:16

    DateTimeImmutableExample.php:28 Reply

    Well, we could consider those changes a cleanup where we switch from phpdoc to property type declaration for single types. As this patch introduces the detection of union types, those are the changes that are 100% relevant for the test suite.
  23. Oliver Klee commented on patch set 7 08/12/22, 11:17

    DateTimeImmutableExample.php:28 Reply

    Ack
  24. Oliver Klee commented on patch set 7 08/12/22, 14:52

    DateTimeImmutableExample.php:28 Reply

  25. Oliver Klee commented on patch set 7 08/12/22, 15:47

    Post.php:87

    I thinks ObjectStorage|LazyObjectStorage (and ObjectStorage<…>|LazyObjectStorage<…>) is redundant as LazyObjectStorage is a subclass of ObjectStorage. (Ditto for the other places where we have a union type of both classes.)
  26. Alexander Schnitzler commented on patch set 7 08/12/22, 16:39

    Post.php:87 Reply

    I tend to disagree because we usually instantiate those properties with an ObjectStorage in the constructor and only hold a LazyObjectStorage when instantiated by the DataMapper via createEmptyObject(). Just using the one or the other type declaration produces issues in the opposite case.
  27. Alexander Schnitzler commented on patch set 7 08/13/22, 07:43

    Post.php:87 Reply

    Has been resolved via direct messages.
  28. Alexander Schnitzler uploaded patch set 8 08/13/22, 08:40

  29. Alexander Schnitzler commented on patch set 8 08/13/22, 08:40

    ObjectConverter.php:132 Reply

    It's a valid todo but has a different impact. If we no longer primarily look at the setters for type determination, we introduce a breaking change, hence this is something for another patch.
  30. core-ci voted on patch set 8 V +1 08/13/22, 08:47 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/30136

  31. Oliver Klee commented on patch set 8 08/13/22, 08:52

    DataMapFactory.php:167 Reply

    Done

    DataMapper.php:605 Reply

    Done

    DataMapper.php:770 Reply

    Done

    ObjectConverter.php:132 Reply

    Ack

    PersistentObjectConverter.php:140 Reply

    Done

    PersistentObjectConverter.php:146 Reply

    Done

    Property.php:91 Reply

    This is still open.

    ValidatorResolver.php:117 Reply

    Ack

    PropertyWithoutTypeDeclarationsTest.php:37 Reply

    Done
  32. Alexander Schnitzler rebased onto patch set 9 09/20/22, 10:59

  33. Alexander Schnitzler uploaded patch set 10 09/20/22, 11:07

  34. core-ci voted on patch set 9 V +1 09/20/22, 11:08 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/31314

  35. Alexander Schnitzler commented on patch set 10 09/20/22, 11:10

    Property.php:91 Reply

    Done

    PropertyWithTypeDeclarationsTest.php:56 Reply

    Hi Oli, please tell me if you have time to rework this according to your taste. I don't have time for this right now myself. If you feel like we can leave the tests as is, please mark these remaining threads as resolved. Thanks in advance.
  36. core-ci voted on patch set 10 V +1 09/20/22, 11:16 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/31315

  37. Oliver Klee commented on patch set 10 09/20/22, 11:22

    PropertyWithTypeDeclarationsTest.php:56 Reply

    Yes, I will do it in a second.
  38. Oliver Klee uploaded patch set 11 09/20/22, 11:27

  39. Oliver Klee commented on patch set 10 09/20/22, 11:29

    PropertyWithTypeDeclarationsTest.php:56 Reply

    Ah, sorry, wrong comment. I currently do not have the time to rework the tests accordingly, but have pushed fixes to some more Yodas.

    PropertyWithoutTypeDeclarationsTest.php:37 Reply

    I've fixed some more Yoda instances and repushed.

    PropertyWithoutTypeDeclarationsTest.php:74 Reply

    Not going to do right now.
  40. core-ci voted on patch set 11 V -1 09/20/22, 11:40 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/31316

  41. Oliver Klee uploaded patch set 12 09/20/22, 11:53

  42. core-ci voted on patch set 12 V +1 09/20/22, 12:02 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/31318

  43. Christian Kuhn commented on patch set 12 09/21/22, 14:12

    we should be able to resolve this with the patch, which should fix another phpstan hit. https://github.com/TYPO3/typo3/blob/main/typo3/sysext/belog/Classes/Domain/Model/LogEntry.php#L128
  44. Markus Klein commented on patch set 12 10/12/22, 15:24

    DataMapFactory.php:167

    do we need to document this somewhere? This is somehow crucial to have the actual type as the first one in the set
  45. Oliver Klee commented on patch set 12 02/18/23, 14:48

    DataMapFactory.php:167 Reply

    I'd prefer to change this to make the order irrelevant. This would amount the number of surprising behavior in the code.
  46. Alexander Schnitzler commented on patch set 12 02/19/23, 08:28

    DataMapFactory.php:167 Reply

    I'd prefer this too but what would the alternative look like? What we do here, is to gather the type and possible element type aka TValue if type indicates we deal with a collection. How would we make this happen without a big refactoring in this and other classes? And, what are the cases where this occurs? Do people annotate: `@var ObjectStorage<TValue>|Entity`? Yes, people can do this but it's unsupported, why would they? And should the order in this case actually matter? What I have in mind right now is the question, if we actually need to support union types except nullable (if that is a type...)? Extbase has now logic that makes union types a desired feature, hence I'd simply not support it and maybe throw an exception in places where just one type is expected but a union type is used. wdyt?
  47. Oliver Klee commented on patch set 12 02/19/23, 11:05

    DataMapFactory.php:167 Reply

    The common use cases I have for union types is ObjectStorage<Thing>|LazyLoadingProxy and Thing|LazyLoadingProxy|null. I currently cannot think of any other meaningful types, but heavily suspect relevant usescases will come up. So maybe it suffices to consider null and LazyLoadingProxy as non-primary types. What do you think?
  48. Markus Klein commented on patch set 12 02/24/23, 14:24

    DataMapFactory.php:167 Reply

    I'd only add LazyObjectStorage as non-primary
  49. Alexander Schnitzler commented on patch set 12 03/04/23, 08:57

    DataMapFactory.php:167 Reply

    Alrighty. > The common use cases I have for union types is ObjectStorage<Thing>|LazyLoadingProxy and Thing|LazyLoadingProxy|null. I currently cannot think of any other meaningful types Ok, we need to distinguish the cases where users make use of union types to be able to use type declarations for lazy loaded property values and the cases where a value can actually be of n types but not mixed. The latter is something Christian mentioned a while back. https://github.com/TYPO3/typo3/blob/main/typo3/sysext/belog/Classes/Domain/Model/LogEntry.php#L128 This patch is not primarily about those cases because in order to support something like string|int, we need more logic. If the framework can choose between int and string, what does it do with a numeric value, stored in a varchar? Do we actually want this depth of logic? I don't want to go down that road, hence the idea of primary types. This saves us a lot of logic for desired type detection and ordering and what not and does not really give much of a benefit. In case of `LogEntry::$newId`, we do store non-numeric string values in the db but the property is initialized with `int (0)`. I'd say this is wrong but whatever. If we assume this is right and we have those real world examples, all I want to achieve with this patch, is that the author can use type declarations without having extbase choke over it and not mapping the database value onto the property. All this can be achieved with a primary type, extbase looks at. Now, let's talk about > `ObjectStorage<Thing>|LazyObjectStorage` `LazyObjectStorage` is not an issue. We talked about it a while back. As `LazyObjectStorage` is a child of `ObjectStorage`, we don't have a need for union types. All is working fine already. > `Thing|LazyLoadingProxy|null` is a different issue as `LazyLoadingProxy` is a different class that Thing. We don't need to consider `null` here btw. `null` is not a type, it's the absence of type and works for as long as php has objects. If extbase encounters a `LazyLoadingProxy`, I'd suggest not detecting it as a primary type at all, no matter where it is positioned. `LazyLoadingProxy|Thing` should result in primary type `Thing`. --- Conclusion: Please let's focus on the day to day use cases that are currently broken and not think about possible use cases of the future that require a ton more logic anyway. I suggest writing a Feature.rst, explaining the concept of primary types and their impact on extbase model properties. Is this a path we can take? I'd really like to get this in soon and not have it end on my pile of unfinished patches.
  50. Oliver Klee commented on patch set 12 03/04/23, 09:33

    DataMapFactory.php:167 Reply

    Yes, sounds like a good plan. Particularly, this would solve the last pain point for me. > If extbase encounters a `LazyLoadingProxy`, I'd suggest not detecting it as a primary type at all, no matter where it is positioned. `LazyLoadingProxy`|`Thing` should result in primary type `Thing`. Thanks, Alexander!
  51. Alexander Schnitzler uploaded patch set 13 V +1 03/04/23, 09:45

  52. Alexander Schnitzler commented on patch set 13 03/04/23, 09:46

    Patchset 13 is just a rebase.
  53. core-ci voted on patch set 13 V +1 03/04/23, 09:52 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/38426

  54. Alexander Schnitzler uploaded patch set 14 V +1 03/04/23, 11:30

  55. Alexander Schnitzler commented on patch set 14 03/04/23, 11:32

    DataMapFactory.php:167 Reply

    Done
  56. core-ci voted on patch set 14 V +1 03/04/23, 11:37 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/38434

  57. Simon Schaufelberger voted on patch set 14 CR +1 03/04/23, 13:34

  58. Markus Klein commented on patch set 14 03/05/23, 22:35

    DataMapper.php:345

    why do we need this explicit check against true?

    ObjectConverter.php:122

    @todo

    NoPropertyTypesException.php:20

    maybe better a LogicException?
  59. Alexander Schnitzler commented on patch set 14 03/06/23, 08:13

    DataMapper.php:345 Reply

    I don't know any more. Will revert it.

    ObjectConverter.php:122 Reply

    Ack

    NoPropertyTypesException.php:20 Reply

    Good catch. I love when people understand the difference between \Runtime- and \LogicException. That's rare.
  60. Alexander Schnitzler uploaded patch set 15 CR +1 V +1 03/06/23, 08:16

  61. Alexander Schnitzler commented on patch set 15 03/06/23, 08:17

    DataMapper.php:345 Reply

    Done

    ObjectConverter.php:122 Reply

    Done

    NoPropertyTypesException.php:20 Reply

    Done
  62. Markus Klein commented on patch set 15 03/06/23, 08:34

    Thanks for the changes. Will do my next review round soon.
  63. core-ci voted on patch set 15 V -1 03/06/23, 08:45 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/38514

  64. Alexander Schnitzler rebased onto patch set 16 V -1 03/06/23, 08:50

  65. core-ci voted on patch set 16 V +1 03/06/23, 09:04 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/38521

  66. Stefan Froemken commented on patch set 16 V +1 CR +1 03/06/23, 18:07

    Tested with my v12 example extension. After adding Lazy to my property I can also add both, the model and LazyLoadingProxy, to my Getter to get it working again.

    ObjectConverter.php:122

    Confirmed...costs me hours to find that out
  1. Alexander Schnitzler rebased onto patch set 17 CR +1 V +1 03/07/23, 19:49

  2. core-ci voted on patch set 17 V +1 03/07/23, 19:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/38624

  3. Stefan Froemken voted on patch set 17 V +1 CR +1 03/07/23, 19:58

  4. Alexander Schnitzler uploaded patch set 18 CR +1 V +1 03/08/23, 08:48

  5. core-ci voted on patch set 18 V +1 03/08/23, 08:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/38645

  6. Oliver Klee voted on patch set 18 V +1 03/08/23, 15:21

  7. Oliver Klee commented on patch set 18 03/08/23, 15:36

    Looks very good to me! I've added some minor comments on improving (human-)readability and on clamping down on the types.

    DataMapFactory.php:161

    `$propertyTypes` contains only a subset of the types. So I suggest to change the name to reflect this: `$nonProxyPropertyTypes`

    DataMapper.php:218

    Ditto.

    DataMapper.php:651

    For the benefit of the human reader, I'd prefer this instead: ``` if (!$primaryType instanceof …) { ```

    DataMapper.php:656

    Ditto: if (!is_string($className)) {

    DataMapper.php:703

    Ditto.

    DataMapper.php:812

    Ditto.

    PersistentObjectConverter.php:133

    Ditto about instanceof.

    NoPropertyTypesException.php:20

    Should this class be `@internal`?

    Property.php:38

    We now can drop the PHPDoc type annotation.

    Property.php:75

    The `@var` needs to be in backticks in order to not confuse static type checkers.

    Property.php:81

    Probably all these types can be `non-empty-string|null`, I suppose?

    Property.php:88

    The `@var` needs to be in backticks in order to not confuse static type checkers.

    Property.php:92

    `list<Type>`?

    Property.php:108

    `list<Type>`

    ValidatorResolver.php:115

    instanceof
  8. Oliver Klee uploaded patch set 19 V +1 03/09/23, 17:04

  9. Oliver Klee commented on patch set 18 V +1 03/09/23, 17:05

    DataMapFactory.php:161 Reply

    Done

    DataMapper.php:218 Reply

    Done

    DataMapper.php:651 Reply

    Done

    DataMapper.php:656 Reply

    Done

    DataMapper.php:703 Reply

    Done

    DataMapper.php:812 Reply

    Done

    PersistentObjectConverter.php:133 Reply

    Done

    NoPropertyTypesException.php:20 Reply

    Done

    Property.php:38 Reply

    Done

    Property.php:75 Reply

    Done

    Property.php:81 Reply

    Done

    Property.php:88 Reply

    Done

    Property.php:92 Reply

    Done

    Property.php:108 Reply

    Done

    ValidatorResolver.php:115 Reply

    Done
  10. Oliver Klee voted on patch set 19 V +1 CR +1 03/09/23, 17:05

  11. core-ci voted on patch set 19 V +1 03/09/23, 17:12 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/38709

  12. Markus Klein commented on patch set 19 03/10/23, 08:05

    Feature-98132-LetClassSchemaDetectMultiplePropertyTypes.rst:16

    Extbase. It is usually written capitalized. To be changed in the whole rst

    Feature-98132-LetClassSchemaDetectMultiplePropertyTypes.rst:18

    ..<space><space>code-block

    Feature-98132-LetClassSchemaDetectMultiplePropertyTypes.rst:39

    unions

    Feature-98132-LetClassSchemaDetectMultiplePropertyTypes.rst:63

    and has always been working.

    Feature-98132-LetClassSchemaDetectMultiplePropertyTypes.rst:65

    code-block missing

    DataMapFactory.php:173

    We have this exact check now in 2 or 3 places. Shall we somehow make this a method "somewhere"?

    DataMapper.php:659

    Let's add also a word about how to resolve the situation. "... class name. Check the type declaration of the property to use a valid class name"

    DataMapper.php:725

    this has been here long enough. lets remove the todo and add a comment what this check is supposed to cover. It checks whether the class name contains `_` or `\`. I think we can drop the `_` check and I'm unsure about the `\` check. Is the class name always fully qualified?

    ObjectConverter.php:122

    I suggest adding a forge ticket for this right away and reference it here

    Property.php:53

    This is not needed anymore, since the nullable property is part of a type

    ValidatorResolver.php:117

    todo can be removed
  13. Alexander Schnitzler uploaded patch set 20 CR +1 V +1 03/10/23, 08:40

  14. Alexander Schnitzler commented on patch set 20 03/10/23, 08:43

    Feature-98132-LetClassSchemaDetectMultiplePropertyTypes.rst:16 Reply

    Done

    Feature-98132-LetClassSchemaDetectMultiplePropertyTypes.rst:18 Reply

    Done

    Feature-98132-LetClassSchemaDetectMultiplePropertyTypes.rst:39 Reply

    Done

    Feature-98132-LetClassSchemaDetectMultiplePropertyTypes.rst:63 Reply

    Done

    Feature-98132-LetClassSchemaDetectMultiplePropertyTypes.rst:65 Reply

    Done

    DataMapper.php:725 Reply

    Let's tackle this in another patch please. I think the solution is to check `$primaryType->getClassName() !== null` here but I don't want to risk anything and do it independently, covered with tests.

    ObjectConverter.php:122 Reply

    Done

    Property.php:53 Reply

    Done

    ValidatorResolver.php:117 Reply

    Let's tackle this in another patch, I have something on my stash already.
  15. core-ci voted on patch set 20 V +1 03/10/23, 08:47 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/38758

  16. Alexander Schnitzler uploaded patch set 21 V +1 03/10/23, 09:34

  17. Alexander Schnitzler commented on patch set 21 03/10/23, 09:34

    DataMapFactory.php:173 Reply

    I agree and would also postpone it and refactor the api a little later. I'd really like to get this in and have it polished later when I have more time.

    DataMapper.php:659 Reply

    Done
  18. core-ci voted on patch set 21 V +1 03/10/23, 10:01 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/38759

  19. Oliver Klee voted on patch set 21 V +1 CR +1 03/10/23, 10:18

  20. Stefan Froemken voted on patch set 21 V +1 CR +1 03/10/23, 10:49

  21. Markus Klein commented on patch set 21 V +1 CR +1 03/10/23, 12:25

    DataMapFactory.php:173 Reply

    Ack

    DataMapper.php:725 Reply

    Ack

    ValidatorResolver.php:117 Reply

    Ack
  22. Benni Mack voted on patch set 21 V +2 CR +2 03/10/23, 12:27

  23. Benni Mack merged patch set 22 03/10/23, 12:28

  24. Christian Kuhn commented on patch set 22 06/15/23, 22:56

    Property.php:77

    this one is still used.

    Property.php:137

    this one is still used.
  25. Alexander Schnitzler commented on patch set 22 07/26/23, 18:07

    Property.php:77 Reply

    Done

    Property.php:137 Reply

    Done