Review main Abandoned Breaking

73989[!!!][TASK] Add native typing for more Utility classes

crell CR 0 V 0 Hard +230 −397 updated 04/13/22, 16:48

Commit message

Adds native types for CsvUtility.
Adds native types for DebugUtility.
Adds native types for DiffUtility.
Adds native types for ExtensionManagementUtility.
Add native types for HttpUtility.
Add native types for MailUtility.

DiffUtility had some interesting interactions around nulls, as some
parameters were docblock-typed for string but occaionally called
with null, specifically from debug().  I fixed that by normalizing
to a string inside debug().

I also improved the documentation for renderDump(), which had
silently nullable booleans without explaining why, but changing it
to non- nullable broke. I added to the documentation to better
explain why.

The one change to BackendUtility is because getProcessedValue()'s
result gets passed directly to DiffUtility::makeDiffDisplay()
by ImportExport. However, it's of undefined type while
DiffDisplay() wants a string only, so it was somehow passing
an integer through. For now, since BackendUtility is in weak
type mode just specifying a return type seems to fix the issue.
That will have to get cleaned up further at some point later.

Review

  • Code review CR 0
  • Verified V 0

Issues 3

Files 11

Review log 2935

Show 610 earlier messages
  1. crell uploaded patch set 1 03/17/22, 20:46

  2. core-ci voted on patch set 1 V +1 03/17/22, 20:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/24973

  3. Oli Bartsch commented on patch set 1 V -1 03/17/22, 21:39

    Adding type hints parameters of API methods is a breaking change which has to be documented as such.
  4. crell uploaded patch set 2 04/08/22, 19:15

  5. crell updated the commit message in patch set 3 04/08/22, 19:17

  6. core-ci voted on patch set 3 V -1 04/08/22, 19:28 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/25857

  7. core-ci voted on patch set 2 V -1 04/08/22, 19:28 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/25856

  8. crell uploaded patch set 4 04/08/22, 21:00

  9. core-ci voted on patch set 4 V +1 04/08/22, 21:08 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/25864

  10. Oli Bartsch commented on patch set 4 04/08/22, 22:41

    I would prefer to stick to one utility per patch as you did with the other classes. Furthermore, do the currently linked issue not match the current patch set. Only "CsvUtility" and "DebugUtility" are linked.
  1. Oli Bartsch commented on patch set 4 04/08/22, 22:42

    Reply

    unresolved
  2. crell commented on patch set 4 04/08/22, 22:55

    Reply

    There are no forge issues for the others at the moment. I hadn't pre-made that far previously. The main reason I switched to a combined patch is expediency. Most of these are fairly small and trivial to review. It's just one or two that have non-trivial changes to review. I initially wanted to just do them in parallel, but because they all would touch the same part of the same changelog file that would result in even more work rerolling all of them after each commit. This seemed like the least-clumsy option.
  3. Anja Leichsenring voted on patch set 4 V +1 CR +1 04/09/22, 09:39

  4. Stefan Bürk commented on patch set 4 CR -1 04/09/22, 09:53

    on it
  5. Oliver Klee commented on patch set 4 04/09/22, 10:12

    BackendUtility.php:1508

    If $value is mixed, the return value of the method also need to be mixed (or we need to do some casting to ensure the string type).

    ExtensionManagementUtility.php:84

    Please let's separate the assignment and reading/returning the value. (Or more general, please let's not have side effects.)

    ExtensionManagementUtilityTest.php:1515

    Faulty -> Empty
  6. Stefan Bürk commented on patch set 4 04/09/22, 10:43

    -1 stays, because of 'BackendUtility::getProcessedValue(...)' return type. First, it's not ensured that return value is a string or null, thus eventually breaking / erroring. You added it as a "simple" solution to avoid other pipeline failures, or better to say hiding an issue away. E.g. when calling for 'DiffUtility::makeDiffDisplay()' which has core usages combined with 'BackendUtility::getProcessedValue()'. This hides issues away to slip strict types in ignoring issues. 'BackendUtility::getProcessedValue()' has to be properly overworked and ensured first instead of simply adding a strict return type.
    maybe change this more to a point-list and differenciate where it has been fully done along with declare strict. E.g. BackendUtility was "only" a return type hint, but not fully typed. Would give a better overview.
    seems in regaredy of core tests, but introduces other issues which are hidden "currently". It seems to be a solution to hide the test errors, but's not a fully/final or proper solution. Adding return type is okay, but problems should not be shifted.

    BackendUtility.php:1505 Unresolved

    Not sure about the consequences hard typing the return type now anyway - even if it now matches the docblock. It is not ensured that the return value is string or null. Needs more work than simply adding a strict return type here.

    BackendUtility.php:1508 Unresolved

    Need check or at lease type cast here. $value may be something else, as it is not ensured that it really is a string - and thus fataling because of return type missmatch. It may transform "uid" to string instead of "leaving" it as is - beside the types in docblock.

    ExtensionManagementUtility.php:53

    empty line can be removed

    ExtensionManagementUtility.php:55 Unresolved

    empty line can be removed

    ExtensionManagementUtility.php:84 Reply

    Oliver, can you explain which side effects you mean ? Cannot see here any side effects her. And it's a valid replacement, granted that the syntax is new and not broadly known yet.

    MailUtility.php:57

    added return type '?string' is not guaranteed. needs proper handling/check

    MailUtility.php:78

    adding type check here may prepare strict typing "GeneralUtility::validEmail()" .. as string is not ensured yet.

    MailUtility.php:100

    adding type check here may prepare strict typing "GeneralUtility::validEmail()" .. as string is not ensured yet.

    GlobalDebugFunctions.php:19

    Missed adding type 'mixed' here ?
  7. Stefan Bürk commented on patch set 4 04/09/22, 10:45

    Reply

    Would also prefer to stick to one utility per patch also.
  8. Oliver Klee commented on patch set 4 04/09/22, 11:11

    Reply Unresolved

    Me too.

    ExtensionManagementUtility.php:84 Reply

    I'll do my best. :-) I'd like to separate setting static::$cacheManager from returning the cache manager into two instructions. (In this case, setting it is a side effect of returning it. Maybe side effect is not the best term for this.)
  9. Stefan Bürk commented on patch set 4 04/09/22, 11:18

    ExtensionManagementUtility.php:84 Reply

    Thanks for the explanation. Just to clarify .. it's not about the '??=', it's because it is directly used in the return statement ? Personally I would take this change, love this short syntax possibilties. (Wanted this kind of simply solutions for this kind of tasks since ages). So we are talking more about kind of taste or opionated view of code readability ? Cannot get any "technical" reson which would argument against it.
  10. Markus Klein commented on patch set 4 04/09/22, 11:43

    ExtensionManagementUtility.php:84 Reply

    I also like this new syntax. It's simple enough to read and reminds me of the typical `while($row = $res->fetchAssoc())`
  11. Oliver Klee commented on patch set 4 04/09/22, 11:49

    ExtensionManagementUtility.php:84 Reply

    > Thanks for the explanation. Just to clarify .. it's not about the '??=', it's because it is directly used in the return statement ? Yes, exactly. > So we are talking more about kind of taste or opionated view of code readability ? I would not call it a matter of taste for me (in the meaning of "taste varies, and it is not really important which one we choose"). It's certainly an opinionated view on code readability - my opinion is that code should communicate its intent as clearly as possibly in order to reduce the probability of people misreading what happens there (or of unintentionally breaking the code when they modify it). So it's both my opinion as a person who has read a lot about making code more readable and maintainable as well as my professional advice as a software engineer. (I read the principle in one of those books that code should either modify data or read it, but not both at the same time.)
  12. Oliver Klee commented on patch set 4 04/09/22, 11:53

    ExtensionManagementUtility.php:84 Reply

    This would be my suggestion in order to communicate the intent more clearly: static::$cacheManager ??= GeneralUtility::makeInstance(CacheManager::class); return static::$cacheManager;
  13. Stefan Bürk commented on patch set 4 04/11/22, 14:21

    MailUtility.php:78 Reply

    1. $GLOBALS['TYPO3_CONF_VARS']['MAIL']['defaultMailFromAddress']; may be unset/empty/not-a-string 2. GeneralUtility::validEmail($address) - docblock "string", not strict typed yet. So adding a type check for L78 before calling validEmail(..) in L79 will be a preparation to make "validEmail(..)" strict typed (type for arguments).
  14. crell commented on patch set 4 04/11/22, 15:48

    Reply Unresolved

    The change to BackendUtility is just a side effect of the others, much like the other patches in this series have had. It's not checking off any todo boxes.

    Reply Unresolved

    I agree it's suboptimal, but in all honesty I don't understand what all is going on inside getProcessedValue(). The root issue is that getProcessedValue() currently returns mixed (or possibly int|string, not sure), but makeDiffDisplay() expects a string, so those two functions *should not be compatible* in the first place. That they have been so far is only a convenient side effect of PHP's implicit type juggling and the data that happens to be requested. Another approach for the moment we could try is explicitly casting the return of getProcessedValue() to a string in various places when before passing to makeDiffDisplay(). I went with the added type to avoid having to track all of those down at the moment, but I can see how many of those there are and if that would be easier.

    BackendUtility.php:1508 Reply

    This file is still in weak type mode, so PHP will implicitly cast an int, float, or bool to a string. That's not ideal, but is effectively the same as sticking a (string) cast on every return statement. (It's different if the value is an array, but that seems to not come up.)

    ExtensionManagementUtility.php:53 Reply Unresolved

    Properties should have a blank line between them, no? That's what I've seen elsewhere.

    ExtensionManagementUtility.php:84 Reply

    I've been using this construct since ??= was introduced in 7.4. I've never had an issue understanding its communicated intent. If anything I find it more readable and self-evident than the traditional version, as it has fewer extra bits floating around I have to decipher. The only reason I've had to split it into two lines is if I want to stick a debugging line or breakpoint between them, which in this case would almost always make more sense inside whatever is called on the right side of the ??=. I'd prefer to leave this as is in the single-line version.

    MailUtility.php:57 Reply

    Done

    MailUtility.php:78 Reply

    Done

    MailUtility.php:100 Reply

    Done

    GlobalDebugFunctions.php:19 Reply

    Done

    ExtensionManagementUtilityTest.php:1515 Reply

    Done
  15. crell uploaded patch set 5 04/11/22, 15:48

  16. Oliver Klee voted on patch set 5 CR +1 04/11/22, 15:52

  17. core-ci voted on patch set 5 V -1 04/11/22, 16:02 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/25929

  18. core-ci voted on patch set 5 V +1 04/11/22, 17:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/25929

  19. Stefan Bürk commented on patch set 5 CR -1 04/11/22, 17:51

    BackendUtility.php:1508 Reply Unresolved

    If you add a return type, it must be ensured that the method returns only the allowed return type. Avoiding the work and argumenting with "php take care because of casting it in weak mode" is not an argument. Than do not add the return type. Sure, doctype is not correct. But we should do the full work, not only some surface cosmetics to suppress errors on the other hand. We should avoid this bad habit.
  20. Oliver Klee voted on patch set 5 CR -1 04/11/22, 20:14

  21. Oliver Klee commented on patch set 5 04/11/22, 20:17

    ExtensionManagementUtility.php:84 Reply Unresolved

    Please let's have this in two statements. It's not about whether you or I have no difficulties reading it, but how to do our best that as many readers as possible see as easily as possible that there are two distinct responsibilities: a) setting the cached value if there is none, and b) returning the cached value.
  22. Oliver Klee commented on patch set 5 04/11/22, 20:20

    And please let's split this up into one changeset per class. This helps getting everything in quicker. (The review time is (as a rule of thumb) exponential to the number of changed lines as the changed lines need to be re-reviewed again and again.) And I'm a lot more motivated to review more smaller changesets than fewer bigger changesets. Larry, would you be willing to split this up so we can get your changes in?
  23. crell commented on patch set 5 04/12/22, 14:21

    Reply

    It will be a pain at this point, but I can try. :-/ The reason I started combining was because the individual ones were still taking several days each, and because of the inevitable conflict in the changelog cannot be easily parallelized. (Committing one would cause all the others to need to be rerolled, which would clear any approvals they had.) We need some more efficient way to do this. If not batching them like this, then what's the alternative?
  24. Oliver Klee commented on patch set 5 04/12/22, 14:31

    Reply

    As far as conflicts in the changelog are concerned, you can at least parallelize 2 or 3 changesets and then add the corresponding new lines in the beginning, middle and end of the existing block. (Usually, smaller changesets tend to get merged quicker than bigger ones.)
  25. crell abandoned the change 04/13/22, 16:48

    Switching back to individual patches for each class.