72817[BUGFIX] Respect original limit in QueryResultPaginator
Commit message
Review
-
Code review CR 0
-
Verified V 0
Issues 1
Files 2
Review log 2429
Show 34 earlier messages
-
Daniel Haupt uploaded patch set 1 12/30/21, 15:37
-
core-ci voted on patch set 1 V +1 12/30/21, 15:52 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/21523
-
Stefan Bürk commented on patch set 1 01/03/22, 11:56
Thanks for the patch Daniel. I would love to see some tests here to cover this case (before/after) and prevents breaking in the future (regression). May I ask you to add some ? -
Daniel Haupt commented on patch set 1 01/03/22, 13:45
Reply
Hi Stefan, thanks for your feedback. Unfortunately, I am not experienced with writing tests, yet. Which kind of tests do you mean? I could try it!
-
Stefan Bürk commented on patch set 1 01/04/22, 20:34
Reply
Functional tests, as this should test real database behaviour. Not sure if there are some tests/examples regardion extbase pagination or something like that, so sry no short hint on that. Maybe you can try to find something, even if it would not fully work at the end. Otherwise I could have a look the next days. -
Daniel Haupt marked it work in progress 01/24/22, 13:27
-
Daniel Haupt rebased onto patch set 2 01/24/22, 13:27
-
Daniel Haupt uploaded patch set 3 01/24/22, 13:35
-
Daniel Haupt marked it ready for review 01/24/22, 13:36
-
Daniel Haupt commented on patch set 3 01/24/22, 13:39
Reply
Hi Stefan, it took me a little longer, but I have now added functional tests for the first time. Looking forward to your feedback for my tests. -
core-ci voted on patch set 2 V +1 01/24/22, 13:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22461
-
Stefan Bürk commented on patch set 3 01/24/22, 13:41
Reply
Thanks for adding the test. Will have a look later. Did not found the time to look for adding tests yet, so thanks anyway that you tried and stepped in. -
core-ci voted on patch set 3 V -1 01/24/22, 14:03 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22462
-
Daniel Haupt marked it work in progress 01/24/22, 14:05
-
Daniel Haupt uploaded patch set 4 01/25/22, 11:43
-
core-ci voted on patch set 4 V +1 01/25/22, 11:54 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22512
-
Daniel Haupt marked it ready for review 01/25/22, 12:06
-
Stefan Bürk commented on patch set 4 CR -1 01/26/22, 09:44
Thanks for the patch, adding the tests (and learn how to do it). My bad that I did not looked deeper into it before requesting for the tests - anyway they made some things clearer. However I have to put -1 on it (should be -2 from my point of view, but do not have the right for it), that's not to offend you, but from my point of view we do not have a bug here which needs to be fixed. See short or detailed answer below. Anyway this patch was helpfull to regonize thinks here, so it was not a waste, even if it would not be used as-is. Short answer: There is no bug to fix, as the Pagination (Paginator) works how it should work, and your solution would break the design principles of Pagination/Paginator as it shares the responsibility who are in charge for the pagination calculation between the pagination and the underlying query. But you showed that we may have missing a usefull property, which may be added as a 'FEATURE' (main only) to the pagination (Paginator) to hard control the 'total item limit'. Detailed answer: Design principle for Pagination/Paginator in conjunction with a database query is, that that class controls the "offset/limit" of the underlying query to do the needed tasks and calculation (total item count etc). DBMS do not have that pagination AND a third total result limit statement, and "paging" is done with offset/limit shift. Only using limit is not really a "total item" filter, as it should be phrased as "give me xx items from offset 0 (page1)". Your "calculation" generally is more or less correct, but it must not rely in information from the query, but the pagination class needs an optional configuration option 'totalResultItemLimit' and use this to ensure the cut on that, even if the "last page" may exceed that limit when "page * itemsPerPage" > "totalResultItemLimit" and enough records are in the database. And that "option" is missing. Pagination/Paginator MUST ignore offset/limit of the provided "base" query. If you do the "offset/limit" handling before, you are doing the "paging" on yourself, so the Pagination class is not suitable. If you want to controll that, you are free to implement your "own specialiced" pagination class in your project, but the core one should stay to general principles. And that's not only a Typo3 thinking, that's how Pagination/Paginator classes are implemented in all major Frameworks when used with DatabaseQueries. (ZendFramework/Laminas, Laravell, Symfony, you-name-it) - and is a well etablished principle. Doing the contraverse is not good. So doing this (invalid) mix as bugfix gets a decline from my side, but will lookup for confirmation/discussion/opionion from more core/extbase experienced people like lolli, Benni or Alexander Schnitzler for example - maybe my -1 get overruled. However I would suggest following from my side: * transform this into an (dedicated) feature adding the missing optional 'totalResultItemLimit' option and close/abandon this "bugfix" * use that in the calculation to use correct offset/(sub)limit values for retrieving the correct items for the wanted page -
Georg Ringer commented on patch set 4 01/26/22, 10:55
Reply
the main point is not if this should be added to this pagination or as a new pagination but IMO TYPO3 must ship a pagination which is suitable by many devs. However having multiple paginations like the new numberedpagination and a new which supports offset/limit and those can't be combined is again not useful -
Benni Mack commented on patch set 4 04/14/23, 10:12
any update here? As v10.4 will soon be out of our support, we should adapt or abandon this change. -
Daniel Haupt commented on patch set 4 04/22/23, 11:47
Reply
In general, I think the paginator should take the original limit and/or offset into account when applying the pagination. This is at least what I would expect as a developer. Especially since we paginate over a `QueryResult` and not directly a `Query`. In the current implementation, the paginator could modify the original expected result set. It basically acts as a `QueryPaginator` built from `QueryResult` instance, removing the previously set limit and offset. We are not paginating the result represented by the original `QueryResult` instance. -
Benni Mack updated the commit message in patch set 5
CR -1V +105/03/23, 09:46 -
Benni Mack rebased onto patch set 6 07/08/23, 12:03
-
core-ci voted on patch set 6 V +1 07/08/23, 12:12 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/43943
-
Stefan Bürk commented on patch set 6 07/11/23, 09:07
Reply
In my eyes this is still wrong. The Paginator is the responsible coordinator for the offset and limit. Not the underlying query. Even other frameworks does this for a paginator based on query. And this is what I would expect as developer. I would get crazy if the paginator would silenty and magically use the query limits which may be set by others. The paginator should and must overrule this, how it is done currently. If beside a normal itemsPerPage + maxPages setting is not enough and a mx items limits needs to be etablished this should be a option on the paginator, because he is responsible for it. Not the query beneath. I my eyes this approach is wrong. We should add a option flag for the "maxItemsLimit" and use this for calculation triage. -
Stefan Bürk commented on patch set 6 08/04/23, 09:34
Reply Unresolved
Similar change: https://review.typo3.org/c/Packages/TYPO3.CMS/+/80326 -
Stefan Bürk voted on patch set 6 CR -1 08/04/23, 10:12
-
Benni Mack updated the commit message in patch set 7
CR -1V +110/18/24, 14:10 -
Benni Mack abandoned the change 12/17/24, 20:29
I will abandon this change due to lack of feedback for 1+ year.