74634[BUGFIX] Fix type issues in Request
Commit message
Review
-
Code review CR +2 Done
- Stefan Bürk +2
- Daniel Haupt +1
- Simon Schaufelberger +1
-
Verified V +2 Done
- Stefan Bürk +2
- Daniel Haupt +1
- Simon Schaufelberger +1
- core-ci +1 CI
Issues 1
Files 6
- Build/phpstan/phpstan-baseline.neon+0 −20
- typo3/sysext/core/Classes/Http/Request.php28+46 −58
- typo3/sysext/core/Classes/Http/ServerRequest.php+1 −1
- typo3/sysext/core/Tests/Unit/Http/RequestFactoryTest.php+0 −27
- typo3/sysext/core/Tests/Unit/Http/RequestTest.php+4 −65
- typo3/sysext/core/Tests/Unit/Http/ServerRequestFactoryTest.php+0 −27
Review log 4454
Show 2329 earlier messages
-
crell uploaded patch set 1 05/12/22, 19:40
-
core-ci voted on patch set 1 V +1 05/12/22, 19:47 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/27147
-
Oliver Klee commented on patch set 1 CR -1 05/12/22, 20:07
This looks generally very good. It still needs to be marked as breaking, and it needs a RST explaining the breaking changes.Request.php:233These two lines can be deleted now. (Ditto for the other type annotations that are noew redundant.) -
crell uploaded patch set 2 05/12/22, 20:29
-
crell commented on patch set 2 05/12/22, 20:33
Request.php:85$method has to still allow null for now. I tried changing it and it caused over 1000 unit test fails in entirely unrelated tests, which suggests an explicit null is being passed somewhere in the test framework. That will need to be identified and fixed first before the ? can be removed.Request.php:233ReplyProbably, though I deliberately didn't touch any of the docblocks on interface-specified methods as they are (I think) directly copied from the spec. Given their origin I preferred to not mess with them at all. Not a hill I'm going to even be injured on, though, if the consensus is to the contrary. -
crell commented on patch set 2 05/12/22, 20:33
Reply
The class is marked @internal, hence no breaking RST, no? -
core-ci voted on patch set 2 V -1 05/12/22, 20:38 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/27151
-
crell rebased onto patch set 3 05/12/22, 20:39
-
core-ci voted on patch set 3 V +1 05/12/22, 20:48 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/27152
-
Oli Bartsch commented on patch set 3 05/12/22, 21:21
Would it be possible to fix this completely so it's correct?Request.php:85ReplyShouldn't we do this first and clean up this patch afterwards? -
crell commented on patch set 3 05/12/22, 21:45
Reply
The more-correct fix would be to require a URI in the constructor, which effectively punts the question to the caller. However, my IDE timed out tracking down all the constructor calls that would impact. Additionally, Uri currently allows a '/' URI anyway, rightly or wrongly. I discussed this issue with some other folks from PHP-FIG, and we concluded that this fix is sufficient for PSR compatibility, especially as a more aggressive fix becomes several orders of magnitude more aggressive. So, no, I don't think it's possible to fix completely at this time, but this does get us up to spec-compliance so it's sufficient.Request.php:85ReplyBoth should be done; I don't think the order greatly matters, especially as it's an @internal class so we don't need to worry too much about API breakage. Coming back here later to remove the null once the test framework is fixed (assuming I am correct about that) is a trivial task we can do any time. -
Oli Bartsch commented on patch set 3 05/13/22, 07:46
Request.php:85ReplyI’m not quite sure if this shouldn‘t be considered as breaking. While the class is marked as @internal, the methods are intended to be used in extension code, e.g. in PSR-15 middlewares or basically in any place, providing the request object (ServerRequest extends this class). -
Stefan Bürk commented on patch set 3 05/13/22, 10:04
Request.php:85Reply@Larry Had a quick look. Cannot say it for all cases, but for the first onces it is not testing-framework related, but related to test setup in unit tests: * (new ServerRequest()) instead of e.G. (new ServerRequest('/', 'GET')) * prophesized ServerRequests, thus missing constructor data: $this->prophesize(ServerRequest::class)->reveal() * ... and I would bet that all these things boils down to this in one or other way. Not checked functionals/acceptance tests, that was only for mentioned unit tests. There is a extending chain: Request (CORE) -> ServerRequest (CORE) -> InternalRequest (TF) TF internal request are used for functional tests / frontend requests. This is not marked as @internal - and should beused by tf users / extension tests. Recently we changed the way for subrequest execution (along with extending InternRequest from ServerRequest), and having a getUri() called there. InternalRequest has a constructor override, ensuring these things - so I think functional tests using InternalRequests should not be affected. Otherwise there is no magically or hard using null for method/uri. ``` /** * @param string|null $uri URI for the request, if any. */ public function __construct($uri = null) { if ($uri === null) { $uri = 'http://localhost/';; } $body = new Stream('php://temp', 'rw'); parent::__construct($uri, 'GET', $body); } ``` But if I'm remembering it right, there are functional tests creating ServerRequests and putting it directly as dependencie injections / instances in classes/methods without going through the frontend stack. They may be affected. Request/ServerRequest are marked as @internal, but as Oliver already mentioned, they are provided in several places. Extension developers may create custom Requests/ServerRequests to do additional internal calls with changed or faked request versions (we have such use cases), and thus may hav not provided these things in the constructor, but afterwards with the corresponding withXY() methods. So I think, we should handle Request/ServerRequest as "public" api, despite marked as @internal. So this should be handled at least as breaking - if not going the v12 deprecated / v13 breaking route. The inteface only states that it must be ensured that the defined methods in the interface matches / returns the correct things. Not that the constructor must ensure this. Don't get me wrong - have this insured with native types and in the constructor in the end should be goal - we just need to figure out the way to it, in which steps - for core and even more important for extension developers and agencies. Thus providing a sane way to this goal. We see it at least with our unit tests, which will be a hard piece of work to refactor them and correct the setup. Many people adopted how core tests things for their extension testings. Additionally, the getUri() method may returned null. We have a custom crawler implementation, which using a empty initalized request object in a event, so project specific extensions can listen to events and fill the request object. As it was "possible" whe checked for "getUri() === null" to have default handling codes outside. Checking for "(string)$request->getUri() === '/'" which have been choosen as default is not the same. So, this may have more impact than maybe seen in the first place. Granted, we used the "missing" native type here, even if the interface do not allow returning null for getUri(). So following should be done/discussed: * prepare / cleanup / adjust unit test setup in pre-patch(es) * check functional / acceptance tests * decide breaking or depreacation way And we should eventually leave than the baseline entry until it can be fully solved. -
Oliver Klee commented on patch set 3 05/13/22, 10:12
Request.php:233ReplyThe current approach indeed is to remove the type annotations when we add the corresponding native type declaration (unless the type annotation provides additional information that should be kept, e.g., a helpful description). -
crell uploaded patch set 4 05/13/22, 14:47
-
core-ci voted on patch set 4 V +1 05/13/22, 14:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/27177
-
crell commented on patch set 4 05/13/22, 16:13
Request.php:233ReplyDone -
crell commented on patch set 4 05/13/22, 17:02
Request.php:85ReplyThe root issue for most of the test fails relating to null $method was indeed ServerRequest. It was passing null in many cases, which caused an exception/error, but some test somewhere did that after setting up GU, which left those set, and thus polluted every test that came after it. Resolving that made tracking down the remaining null cases trivial. (They were all in tests.)Request.php:277Splitting off from Stefan's comments in another thread, I'm unclear what to suggest here. The current return-null behavior is wrong. If other code is relying on that null as a sentinel value, then the sentinel is going to have to change on them at some point. That means for the transition code has to check for null OR whatever the "empty URI" object is we settle on. There's no way around that. Of course, '/' is a legal path, so we would need more than that to indicate a null object. That suggests to me we may need to do something like defining an EmptyUrl implements UrlInterface object, which we spawn and return if the internal value is null. Then callers can check "if $url == null || $url instanceof EmptyUrl` to determine if the value is empty, which would then work in both v11 and v12. I think that's the cleanest forward path for this issue I can see at the moment. Any objections? -
crell uploaded patch set 5 05/13/22, 17:02
-
core-ci voted on patch set 5 V +1 05/13/22, 17:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/27183
-
Oliver Klee wrote 05/18/22, 19:42
Removed reviewer Oliver Klee. -
Stefan Bürk commented on patch set 5 05/18/22, 22:12
Some summerizes (see my thoughts/opinion/suggestions at the end). \TYPO3\CMS\Core\Http\Request ---------------------------- Implements `\Psr\Http\Message\RequestInterface` and inherit other interface and implementations through extending `Message` and so on, which do not matter here for now. This class is mainly used to create request in a object oriented way, and used along with client libraries like Guzzle to make requests based on set information (properties). This class is marked as internal, and thus should not be used directly. Core provides a public api wrapper around this `Request` calls and to make requests, which is basicly documented in the TYPO3 Documentation: https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Http/Index.html Working horse for this is `\TYPO3\CMS\Core\Http\RequestFactory`, which provides two factory methods: * `createRequest(string $method, $uri): RequestInterface` * `request(string $uri, string $method = 'GET', array $options = []): ResponseInterface` `createRequest` do not have a stricted type for $uri, docblock however only says 'UriInterface|string $uri'. However, there are quite some (unit) tests, creating a `Request` instance as test subject, and setting all properies later on before testing things. Additionally this class acts as parent for `ServerRequest`, using the the implemented stuff from Request(). \TYPO3\CMS\Core\Http\ServerRequest ---------------------------------- Extends `\TYPO3\CMS\Core\Http\Request` and additionally implements the `\Psr\Http\Message\ServerRequestInterface` methods. This class is used to hold the request information of the incoming web request, and not to make requests. To make nonsense of this circumstance, the TYPO3 core testing-framework uses this class to "simulate" web requests, which are now directly inserted into the stack (through `InternalRequest` as further extending class of this one with some test related additions). In a real webrequest there is always a host, port and path given, from where the ServerRequest is populated (along with other data). Thus we can assume that having URI and METHOD is okay for this case. However, tests (unit/functionaltests) may have build incomplete test ServerRequests to inject it into parts of the core directly. Executing frontend requests uses InternalRequest(), which already normalizes the method and the uri object if not given. Steps/Opinion/Suggestion ------------------------ Thinking about all these things, I would say we need to do following: # pre-patch 1 adjust all test occurances which creates Request() without or with incorrect parameters ( avoid null for $uri, $method etc) # pre-patch 2 adjust `\TYPO3\CMS\Core\Http\RequestFactory->createRequest()` to not use/pass through uri as null and add proper native type. Additionally not passing null for body. This is a `[!!!][TASK]` breaking pre-patch. Not sure if we should donate this patch a dedicated breaking ReST file with proper code example to avoid null for $uri (which can be done in v11 as compatible prepration already, so would be fine). Second possiblity would be: make the uri null check here to add the uri default with path '/' and not providing empty/nullable body. + if uri is provied as null trigger a proper deprecation log entry and add a proper deprecation rest file (not breaking) with code examples. And make the change in v13. # pre-patch 3 adjust all functional tests creating ServerRequests as test subjects without providing $uri as string|uri and proper method as string (instead of null) -> simple TASK # finish this one change signature of Request and ServerRequest constructors not allowing null for $uri and $method (as you wanted to change for Request already), along with removal of the null handle code and checks. As these classes are marked as internal, and technical breaking. Still flipping between handling this change as breaking (with dedicated breaking ReST) file to notice developers so they can adapt the mandantory changes in there test suites BEFORE upgrading the core (which is possible) - or at least with a proper important rest file (maybe should be decided by @benni then). # get in touch with PSR and hopefully get nearly a pure PHP8.1 fully typed/stricted type interface version of the PSR interfaces => if done, raise to this version and Adjust all types/return types for Request/ServerRequest (which may be considered as breaking then) This avoids that we need to add kind of weired '/' uri default and having a proper path to cover all things. However, would feel better to have a confirmation or denial of this by a more experianced core developer / maintainer .. at best by @benni.Request.php:93Is this not redudant with the `instanceof` check above, and thus superflous here ?Request.php:277ReplyMoved most of it to a "general" comment instead of a specific line comment. In the end we should avoid "/" and even more intrudocung some kind of "EmptyUri" object thing. I have rethought deeply about all, so see generic comment. -
Stefan Bürk commented on patch set 5 05/19/22, 11:02
Request.php:277ReplyDone -
Benni Mack commented on patch set 5 05/20/22, 12:22
Reply
Thanks for this good summary. * Pre-Patch 1 is good * Pre-Patch 2 is somehow ... I don't know. My current gut feeling is this: If we change this and add native types, basically we're getting more "narrow" with a breaking change. The purpose of the RequestFactory was to implement PSR-17. However, if we change this to a custom type like "string|UriInterface" but PHP-FIG then says "actually, we're going to use "string|null|UriInterface" then we'd be breaking AGAIN. This just seems too much work. Not passing "null" for body, should be done, nevertheless. In general, we should follow PSRs in their strictness (or looseness) and wait for that instead of rushing. -
crell commented on patch set 5 05/20/22, 18:10
Reply
Well, the main criteria here where we're not following the spec is getUri(), which is already documented as returning UriInterface only and forever, amen. A future 2.x that adds return types wouldn't change that, it would just enforce it better. Also, I talked to MWOP and he's very very hesitant about the 2.x/3.x versions of PSR-7 and is very very scared of breaking things for people or making a lot more work for implementation maintainers. We shouldn't expect releases of those before late summer, I imagine, at the earliest. (And he may decide to not do so at all.) So we shouldn't wait for that, but still need to follow the spec by eliminating that null. RequestFactory::createRequest() already disallows a null $uri, it's just not typed as such and cannot be until there is a PSR-17 update. We can add an explicit check for null but it doesn't change the API. (Anyone passing null is already wrong and it's their bug.) That means adding such a check is NOT an API break. Null for the body is the least problematic bit here. The main question, though: If we change all calls to Request() to pas a real URL... what should that real URL be? If we're going to demand a real URL in the constructor (which it outside the scope of the PSR), we should either have a reasonable default OR a documented "if you're not sure what to do, do this" answer. I'm also leaning toward backing out the UriInterface bugfix part and getting the rest of this patch committed first, as the rest has no API or breakage implications (I think). Would folks be OK with that?Request.php:93ReplyMm, yes, yes it is. -
Benni Mack commented on patch set 5 06/13/22, 12:00
Reply
Fine by me -
crell uploaded patch set 6 06/15/22, 16:21
-
crell commented on patch set 6 06/15/22, 16:22
Reply
AckBacked out the URI fixes for another time.Request.php:93ReplyDone -
core-ci voted on patch set 6 V +1 06/15/22, 16:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/28100
-
Oliver Klee commented on patch set 6 06/15/22, 16:30
Request.php:74Possibly a typo? or -> of -
crell commented on patch set 6 06/15/22, 18:23
Request.php:74ReplyMissing f in "for". -
crell uploaded patch set 7 06/15/22, 18:23
-
core-ci voted on patch set 7 V +1 06/15/22, 18:32 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/28101
-
Oliver Klee commented on patch set 7 06/15/22, 20:18
Looks generally good. I‘ve commented on two smaller things.Request.php:50@var array<int, non-empty-string>Request.php:162Can we string-type $header here? -
crell commented on patch set 7 06/15/22, 20:22
Request.php:50ReplyI think string[] is more accurate here, as the key is meaningless. Really this could just be a const or something, but I didn't want to mess with that.Request.php:162ReplyUnfortunately no, as that conflicts with RequestInterface, which does not yet have any primitive types. That means we can add return types safely, but not param types. -
crell uploaded patch set 8 06/15/22, 20:23
-
core-ci voted on patch set 8 V +1 06/15/22, 20:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/28107
-
Oliver Klee commented on patch set 9 06/15/22, 21:12
Published edit on patch set 8. -
Oliver Klee voted on patch set 9 CR +1 06/15/22, 21:12
-
core-ci voted on patch set 9 V +1 06/15/22, 21:19 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/28108
-
crell voted on patch set 9 CR +1 06/21/22, 20:06
-
Daniel Haupt commented on patch set 9 CR +1 06/21/22, 23:00
Is there a specific strategy for verifying here? I am not sure what to check manually.nit: "we" could be removed here, I guess -
crell commented on patch set 9 06/22/22, 17:31
Reply
There should be no observable behavioral changes. So if nothing seems different to you or errors when it didn't used to, it's working correctly.Reply
Ugh, yeah, that's just me typoing, or leftovers from an earlier version of the sentence or something. Does that need a new patch uploaded or can committers fix that so people don't need to vote again? :-) -
Daniel Haupt updated the commit message in patch set 10 06/22/22, 18:38
-
Oliver Klee voted on patch set 10 CR +1 06/22/22, 18:39
-
Daniel Haupt commented on patch set 10 V +1 CR +1 06/22/22, 18:40
Reply
I was not sure, so I just tried it. Unfortunately, it resets the votes. IMO, those small typos could be ignored, next time. Sorry for adding this extra noise. -
core-ci voted on patch set 10 V +1 06/22/22, 18:46 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/28339
-
Oliver Klee wrote 06/24/22, 16:55
Removed reviewer Oliver Klee with the following votes: * Code-Review+1 by Oliver Klee -
Simon Schaufelberger commented on patch set 10 07/04/22, 10:36
Request.php:164why no type for header?Request.php:233can mixed be more specific? a preg_match only works on strings, or?Request.php:266why no string for the method here? -
Simon Schaufelberger commented on patch set 10 07/04/22, 15:26
Request.php:164ReplyNot possible because of InterfaceRequest.php:266ReplyNot possible because of Interface -
Simon Schaufelberger commented on patch set 10 07/04/22, 15:30
Request.php:233Replynot possible because of Interface. -
Simon Schaufelberger voted on patch set 10 V +1 CR +1 07/04/22, 15:32
-
Stefan Bürk voted on patch set 10 V +2 CR +2 07/07/22, 20:33
-
Stefan Bürk merged patch set 11 07/07/22, 20:33