Review main Merged

74634[BUGFIX] Fix type issues in Request

crell CR +2 V +2 Normal +51 −198 updated 07/07/22, 20:33

Commit message

This patch adds property and return types to the core Request class.
Parameter types cannot be added to methods from RequestInterface
as it has no typed version yet.

Doing so allows for the removal of a lot of now-unnecessary tests
as well as PHPstan baseline issues.

This patch also notes, but does not correct, the incorrect
implementation of getUri(), which this implementation permits
to return null even though that is a PSR-7 violation. Fixing
that has several knock-on effects better addressed in
future patches.

Review

  • Code review CR +2 Done
    • Stefan Bürk +2
    • Daniel Haupt +1
    • Simon Schaufelberger +1
  • Verified V +2 Done
    • Stefan Bürk +2
    • Daniel Haupt +1
    • Simon Schaufelberger +1
    • core-ci +1 CI

Issues 1

Files 6

Review log 4454

Show 2329 earlier messages
  1. crell uploaded patch set 1 05/12/22, 19:40

  2. core-ci voted on patch set 1 V +1 05/12/22, 19:47 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/27147

  3. Oliver Klee commented on patch set 1 CR -1 05/12/22, 20:07

    This looks generally very good. It still needs to be marked as breaking, and it needs a RST explaining the breaking changes.

    Request.php:233

    These two lines can be deleted now. (Ditto for the other type annotations that are noew redundant.)
  4. crell uploaded patch set 2 05/12/22, 20:29

  5. crell commented on patch set 2 05/12/22, 20:33

    Request.php:85

    $method has to still allow null for now. I tried changing it and it caused over 1000 unit test fails in entirely unrelated tests, which suggests an explicit null is being passed somewhere in the test framework. That will need to be identified and fixed first before the ? can be removed.

    Request.php:233 Reply

    Probably, though I deliberately didn't touch any of the docblocks on interface-specified methods as they are (I think) directly copied from the spec. Given their origin I preferred to not mess with them at all. Not a hill I'm going to even be injured on, though, if the consensus is to the contrary.
  6. crell commented on patch set 2 05/12/22, 20:33

    Reply

    The class is marked @internal, hence no breaking RST, no?
  7. core-ci voted on patch set 2 V -1 05/12/22, 20:38 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/27151

  8. crell rebased onto patch set 3 05/12/22, 20:39

  9. core-ci voted on patch set 3 V +1 05/12/22, 20:48 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/27152

  10. Oli Bartsch commented on patch set 3 05/12/22, 21:21

    Would it be possible to fix this completely so it's correct?

    Request.php:85 Reply

    Shouldn't we do this first and clean up this patch afterwards?
  11. crell commented on patch set 3 05/12/22, 21:45

    Reply

    The more-correct fix would be to require a URI in the constructor, which effectively punts the question to the caller. However, my IDE timed out tracking down all the constructor calls that would impact. Additionally, Uri currently allows a '/' URI anyway, rightly or wrongly. I discussed this issue with some other folks from PHP-FIG, and we concluded that this fix is sufficient for PSR compatibility, especially as a more aggressive fix becomes several orders of magnitude more aggressive. So, no, I don't think it's possible to fix completely at this time, but this does get us up to spec-compliance so it's sufficient.

    Request.php:85 Reply

    Both should be done; I don't think the order greatly matters, especially as it's an @internal class so we don't need to worry too much about API breakage. Coming back here later to remove the null once the test framework is fixed (assuming I am correct about that) is a trivial task we can do any time.
  12. Oli Bartsch commented on patch set 3 05/13/22, 07:46

    Request.php:85 Reply

    I’m not quite sure if this shouldn‘t be considered as breaking. While the class is marked as @internal, the methods are intended to be used in extension code, e.g. in PSR-15 middlewares or basically in any place, providing the request object (ServerRequest extends this class).
  13. Stefan Bürk commented on patch set 3 05/13/22, 10:04

    Request.php:85 Reply

    @Larry Had a quick look. Cannot say it for all cases, but for the first onces it is not testing-framework related, but related to test setup in unit tests: * (new ServerRequest()) instead of e.G. (new ServerRequest('/', 'GET')) * prophesized ServerRequests, thus missing constructor data: $this->prophesize(ServerRequest::class)->reveal() * ... and I would bet that all these things boils down to this in one or other way. Not checked functionals/acceptance tests, that was only for mentioned unit tests. There is a extending chain: Request (CORE) -> ServerRequest (CORE) -> InternalRequest (TF) TF internal request are used for functional tests / frontend requests. This is not marked as @internal - and should beused by tf users / extension tests. Recently we changed the way for subrequest execution (along with extending InternRequest from ServerRequest), and having a getUri() called there. InternalRequest has a constructor override, ensuring these things - so I think functional tests using InternalRequests should not be affected. Otherwise there is no magically or hard using null for method/uri. ``` /** * @param string|null $uri URI for the request, if any. */ public function __construct($uri = null) { if ($uri === null) { $uri = 'http://localhost/';; } $body = new Stream('php://temp', 'rw'); parent::__construct($uri, 'GET', $body); } ``` But if I'm remembering it right, there are functional tests creating ServerRequests and putting it directly as dependencie injections / instances in classes/methods without going through the frontend stack. They may be affected. Request/ServerRequest are marked as @internal, but as Oliver already mentioned, they are provided in several places. Extension developers may create custom Requests/ServerRequests to do additional internal calls with changed or faked request versions (we have such use cases), and thus may hav not provided these things in the constructor, but afterwards with the corresponding withXY() methods. So I think, we should handle Request/ServerRequest as "public" api, despite marked as @internal. So this should be handled at least as breaking - if not going the v12 deprecated / v13 breaking route. The inteface only states that it must be ensured that the defined methods in the interface matches / returns the correct things. Not that the constructor must ensure this. Don't get me wrong - have this insured with native types and in the constructor in the end should be goal - we just need to figure out the way to it, in which steps - for core and even more important for extension developers and agencies. Thus providing a sane way to this goal. We see it at least with our unit tests, which will be a hard piece of work to refactor them and correct the setup. Many people adopted how core tests things for their extension testings. Additionally, the getUri() method may returned null. We have a custom crawler implementation, which using a empty initalized request object in a event, so project specific extensions can listen to events and fill the request object. As it was "possible" whe checked for "getUri() === null" to have default handling codes outside. Checking for "(string)$request->getUri() === '/'" which have been choosen as default is not the same. So, this may have more impact than maybe seen in the first place. Granted, we used the "missing" native type here, even if the interface do not allow returning null for getUri(). So following should be done/discussed: * prepare / cleanup / adjust unit test setup in pre-patch(es) * check functional / acceptance tests * decide breaking or depreacation way And we should eventually leave than the baseline entry until it can be fully solved.
  14. Oliver Klee commented on patch set 3 05/13/22, 10:12

    Request.php:233 Reply

    The current approach indeed is to remove the type annotations when we add the corresponding native type declaration (unless the type annotation provides additional information that should be kept, e.g., a helpful description).
  15. crell uploaded patch set 4 05/13/22, 14:47

  16. core-ci voted on patch set 4 V +1 05/13/22, 14:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/27177

  17. crell commented on patch set 4 05/13/22, 16:13

    Request.php:233 Reply

    Done
  18. crell commented on patch set 4 05/13/22, 17:02

    Request.php:85 Reply

    The root issue for most of the test fails relating to null $method was indeed ServerRequest. It was passing null in many cases, which caused an exception/error, but some test somewhere did that after setting up GU, which left those set, and thus polluted every test that came after it. Resolving that made tracking down the remaining null cases trivial. (They were all in tests.)

    Request.php:277

    Splitting off from Stefan's comments in another thread, I'm unclear what to suggest here. The current return-null behavior is wrong. If other code is relying on that null as a sentinel value, then the sentinel is going to have to change on them at some point. That means for the transition code has to check for null OR whatever the "empty URI" object is we settle on. There's no way around that. Of course, '/' is a legal path, so we would need more than that to indicate a null object. That suggests to me we may need to do something like defining an EmptyUrl implements UrlInterface object, which we spawn and return if the internal value is null. Then callers can check "if $url == null || $url instanceof EmptyUrl` to determine if the value is empty, which would then work in both v11 and v12. I think that's the cleanest forward path for this issue I can see at the moment. Any objections?
  19. crell uploaded patch set 5 05/13/22, 17:02

  20. core-ci voted on patch set 5 V +1 05/13/22, 17:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/27183

  21. Oliver Klee wrote 05/18/22, 19:42

    Removed reviewer Oliver Klee.
  22. Stefan Bürk commented on patch set 5 05/18/22, 22:12

    Some summerizes (see my thoughts/opinion/suggestions at the end). \TYPO3\CMS\Core\Http\Request ---------------------------- Implements `\Psr\Http\Message\RequestInterface` and inherit other interface and implementations through extending `Message` and so on, which do not matter here for now. This class is mainly used to create request in a object oriented way, and used along with client libraries like Guzzle to make requests based on set information (properties). This class is marked as internal, and thus should not be used directly. Core provides a public api wrapper around this `Request` calls and to make requests, which is basicly documented in the TYPO3 Documentation: https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Http/Index.html Working horse for this is `\TYPO3\CMS\Core\Http\RequestFactory`, which provides two factory methods: * `createRequest(string $method, $uri): RequestInterface` * `request(string $uri, string $method = 'GET', array $options = []): ResponseInterface` `createRequest` do not have a stricted type for $uri, docblock however only says 'UriInterface|string $uri'. However, there are quite some (unit) tests, creating a `Request` instance as test subject, and setting all properies later on before testing things. Additionally this class acts as parent for `ServerRequest`, using the the implemented stuff from Request(). \TYPO3\CMS\Core\Http\ServerRequest ---------------------------------- Extends `\TYPO3\CMS\Core\Http\Request` and additionally implements the `\Psr\Http\Message\ServerRequestInterface` methods. This class is used to hold the request information of the incoming web request, and not to make requests. To make nonsense of this circumstance, the TYPO3 core testing-framework uses this class to "simulate" web requests, which are now directly inserted into the stack (through `InternalRequest` as further extending class of this one with some test related additions). In a real webrequest there is always a host, port and path given, from where the ServerRequest is populated (along with other data). Thus we can assume that having URI and METHOD is okay for this case. However, tests (unit/functionaltests) may have build incomplete test ServerRequests to inject it into parts of the core directly. Executing frontend requests uses InternalRequest(), which already normalizes the method and the uri object if not given. Steps/Opinion/Suggestion ------------------------ Thinking about all these things, I would say we need to do following: # pre-patch 1 adjust all test occurances which creates Request() without or with incorrect parameters ( avoid null for $uri, $method etc) # pre-patch 2 adjust `\TYPO3\CMS\Core\Http\RequestFactory->createRequest()` to not use/pass through uri as null and add proper native type. Additionally not passing null for body. This is a `[!!!][TASK]` breaking pre-patch. Not sure if we should donate this patch a dedicated breaking ReST file with proper code example to avoid null for $uri (which can be done in v11 as compatible prepration already, so would be fine). Second possiblity would be: make the uri null check here to add the uri default with path '/' and not providing empty/nullable body. + if uri is provied as null trigger a proper deprecation log entry and add a proper deprecation rest file (not breaking) with code examples. And make the change in v13. # pre-patch 3 adjust all functional tests creating ServerRequests as test subjects without providing $uri as string|uri and proper method as string (instead of null) -> simple TASK # finish this one change signature of Request and ServerRequest constructors not allowing null for $uri and $method (as you wanted to change for Request already), along with removal of the null handle code and checks. As these classes are marked as internal, and technical breaking. Still flipping between handling this change as breaking (with dedicated breaking ReST) file to notice developers so they can adapt the mandantory changes in there test suites BEFORE upgrading the core (which is possible) - or at least with a proper important rest file (maybe should be decided by @benni then). # get in touch with PSR and hopefully get nearly a pure PHP8.1 fully typed/stricted type interface version of the PSR interfaces => if done, raise to this version and Adjust all types/return types for Request/ServerRequest (which may be considered as breaking then) This avoids that we need to add kind of weired '/' uri default and having a proper path to cover all things. However, would feel better to have a confirmation or denial of this by a more experianced core developer / maintainer .. at best by @benni.

    Request.php:93

    Is this not redudant with the `instanceof` check above, and thus superflous here ?

    Request.php:277 Reply

    Moved most of it to a "general" comment instead of a specific line comment. In the end we should avoid "/" and even more intrudocung some kind of "EmptyUri" object thing. I have rethought deeply about all, so see generic comment.
  23. Stefan Bürk commented on patch set 5 05/19/22, 11:02

    Request.php:277 Reply

    Done
  24. Benni Mack commented on patch set 5 05/20/22, 12:22

    Reply

    Thanks for this good summary. * Pre-Patch 1 is good * Pre-Patch 2 is somehow ... I don't know. My current gut feeling is this: If we change this and add native types, basically we're getting more "narrow" with a breaking change. The purpose of the RequestFactory was to implement PSR-17. However, if we change this to a custom type like "string|UriInterface" but PHP-FIG then says "actually, we're going to use "string|null|UriInterface" then we'd be breaking AGAIN. This just seems too much work. Not passing "null" for body, should be done, nevertheless. In general, we should follow PSRs in their strictness (or looseness) and wait for that instead of rushing.
  25. crell commented on patch set 5 05/20/22, 18:10

    Reply

    Well, the main criteria here where we're not following the spec is getUri(), which is already documented as returning UriInterface only and forever, amen. A future 2.x that adds return types wouldn't change that, it would just enforce it better. Also, I talked to MWOP and he's very very hesitant about the 2.x/3.x versions of PSR-7 and is very very scared of breaking things for people or making a lot more work for implementation maintainers. We shouldn't expect releases of those before late summer, I imagine, at the earliest. (And he may decide to not do so at all.) So we shouldn't wait for that, but still need to follow the spec by eliminating that null. RequestFactory::createRequest() already disallows a null $uri, it's just not typed as such and cannot be until there is a PSR-17 update. We can add an explicit check for null but it doesn't change the API. (Anyone passing null is already wrong and it's their bug.) That means adding such a check is NOT an API break. Null for the body is the least problematic bit here. The main question, though: If we change all calls to Request() to pas a real URL... what should that real URL be? If we're going to demand a real URL in the constructor (which it outside the scope of the PSR), we should either have a reasonable default OR a documented "if you're not sure what to do, do this" answer. I'm also leaning toward backing out the UriInterface bugfix part and getting the rest of this patch committed first, as the rest has no API or breakage implications (I think). Would folks be OK with that?

    Request.php:93 Reply

    Mm, yes, yes it is.
  26. Benni Mack commented on patch set 5 06/13/22, 12:00

    Reply

    Fine by me
  27. crell uploaded patch set 6 06/15/22, 16:21

  28. crell commented on patch set 6 06/15/22, 16:22

    Reply

    Ack
    Backed out the URI fixes for another time.

    Request.php:93 Reply

    Done
  29. core-ci voted on patch set 6 V +1 06/15/22, 16:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/28100

  1. Oliver Klee commented on patch set 6 06/15/22, 16:30

    Request.php:74

    Possibly a typo? or -> of
  2. crell commented on patch set 6 06/15/22, 18:23

    Request.php:74 Reply

    Missing f in "for".
  3. crell uploaded patch set 7 06/15/22, 18:23

  4. core-ci voted on patch set 7 V +1 06/15/22, 18:32 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/28101

  5. Oliver Klee commented on patch set 7 06/15/22, 20:18

    Looks generally good. I‘ve commented on two smaller things.

    Request.php:50

    @var array<int, non-empty-string>

    Request.php:162

    Can we string-type $header here?
  6. crell commented on patch set 7 06/15/22, 20:22

    Request.php:50 Reply

    I think string[] is more accurate here, as the key is meaningless. Really this could just be a const or something, but I didn't want to mess with that.

    Request.php:162 Reply

    Unfortunately no, as that conflicts with RequestInterface, which does not yet have any primitive types. That means we can add return types safely, but not param types.
  7. crell uploaded patch set 8 06/15/22, 20:23

  8. core-ci voted on patch set 8 V +1 06/15/22, 20:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/28107

  9. Oliver Klee commented on patch set 9 06/15/22, 21:12

    Published edit on patch set 8.
  10. Oliver Klee voted on patch set 9 CR +1 06/15/22, 21:12

  11. core-ci voted on patch set 9 V +1 06/15/22, 21:19 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/28108

  12. crell voted on patch set 9 CR +1 06/21/22, 20:06

  13. Daniel Haupt commented on patch set 9 CR +1 06/21/22, 23:00

    Is there a specific strategy for verifying here? I am not sure what to check manually.
    nit: "we" could be removed here, I guess
  14. crell commented on patch set 9 06/22/22, 17:31

    Reply

    There should be no observable behavioral changes. So if nothing seems different to you or errors when it didn't used to, it's working correctly.

    Reply

    Ugh, yeah, that's just me typoing, or leftovers from an earlier version of the sentence or something. Does that need a new patch uploaded or can committers fix that so people don't need to vote again? :-)
  15. Daniel Haupt updated the commit message in patch set 10 06/22/22, 18:38

  16. Oliver Klee voted on patch set 10 CR +1 06/22/22, 18:39

  17. Daniel Haupt commented on patch set 10 V +1 CR +1 06/22/22, 18:40

    Reply

    I was not sure, so I just tried it. Unfortunately, it resets the votes. IMO, those small typos could be ignored, next time. Sorry for adding this extra noise.
  18. core-ci voted on patch set 10 V +1 06/22/22, 18:46 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/28339

  19. Oliver Klee wrote 06/24/22, 16:55

    Removed reviewer Oliver Klee with the following votes: * Code-Review+1 by Oliver Klee
  20. Simon Schaufelberger commented on patch set 10 07/04/22, 10:36

    Request.php:164

    why no type for header?

    Request.php:233

    can mixed be more specific? a preg_match only works on strings, or?

    Request.php:266

    why no string for the method here?
  21. Simon Schaufelberger commented on patch set 10 07/04/22, 15:26

    Request.php:164 Reply

    Not possible because of Interface

    Request.php:266 Reply

    Not possible because of Interface
  22. Simon Schaufelberger commented on patch set 10 07/04/22, 15:30

    Request.php:233 Reply

    not possible because of Interface.
  23. Simon Schaufelberger voted on patch set 10 V +1 CR +1 07/04/22, 15:32

  24. Stefan Bürk voted on patch set 10 V +2 CR +2 07/07/22, 20:33

  25. Stefan Bürk merged patch set 11 07/07/22, 20:33