Review main Merged

64567[BUGFIX] Reintroduce tt_content sorting to both page modules

Jo Hasenau CR +2 V +2 Easy +13 −0 updated 08/25/20, 16:49

Commit message

With #90235 db based sorting in the main query that fetches
the content elements to render has been accidentially removed.
In many cases, this did not have any impact: The sorting field
is part of a db index that is used for the query and some
dbms like maria/mysql then tend to return rows determined by
that index. But it can go wrong, too.
The fix is to add proper ordering again of course.

Review

  • Code review CR +2 Done
    • Anja Leichsenring +2
    • Denis Mir +1
    • Daniel Siepmann +1
    • Christian Kuhn +1
    • Jürgen Venne +1
    • Richard Haeser +1
  • Verified V +2 Done
    • Anja Leichsenring +2
    • Denis Mir +1
    • Christian Kuhn +1
    • Jürgen Venne +1
    • TYPO3com +1 CI

Issues 2

Files 2

Review log 4049

Show 1924 earlier messages
  1. Jo Hasenau uploaded patch set 1 05/23/20, 17:02

  2. TYPO3com voted on patch set 1 V +1 05/23/20, 17:13 Completed build in 11m 38s on Sat, 23 May, 05:13 PM Test Summary: 73804 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3500

  3. Josef Glatz commented on patch set 1 V -1 05/25/20, 07:38

    Still only master. There was no branching since the release of 10.4y
  4. Jo Hasenau updated the commit message in patch set 2 05/25/20, 10:43

  5. Jo Hasenau commented on patch set 2 05/25/20, 10:44

    Done :-)

    Reply

    Done
  6. TYPO3com voted on patch set 2 V -1 05/25/20, 10:59 Completed build in 15m 10s on Mon, 25 May, 10:59 AM Test Summary: 2 of 74289 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3511

  7. TYPO3com voted on patch set 2 V +1 05/25/20, 11:14 Completed build in 46m 14s on Mon, 25 May, 11:14 AM Test Summary: 73805 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-3511

  8. Benni Mack updated the commit message in patch set 3 05/25/20, 18:54

  9. Jürgen Venne commented on patch set 3 V +1 CR +1 06/03/20, 12:08

    Works as expected also in Fluid Page module
  10. Oli Bartsch commented on patch set 3 V +1 CR -1 06/04/20, 14:49

    PageLayoutView.php:1920 Unresolved

    I think we should avoid this flag because of some reasons: 1. This method is only called once, without the last parameter, so it defaults to "true" automatically. 2. We should unify this with the new fluid page module where we also have no such flag. 3. As phpmd states "A boolean flag argument is a reliable indicator for a violation of the Single Responsibility Principle (SRP)."

    PageLayoutView.php:1930 Unresolved

    I know, this is just the code from before. But let's take the chance to refactor it a bit. The variable `$orderBy` is declared and then reused in the foreach. This is really illegible. The `$orderBys` declaration could be skipped because of its single usage in the foreach. So a suggestion (also applies to the fluid based page module) would be: $sortBy = (string)($GLOBALS['TCA'][$table]['ctrl']['sortby'] ?: $GLOBALS['TCA'][$table]['ctrl']['default_sortby']); foreach (QueryHelper::parseOrderBy($sortBy) as $orderBy) { $queryBuilder->addOrderBy($orderBy[0], $orderBy[1]); }
  11. Oli Bartsch uploaded patch set 4 08/01/20, 09:30

  12. Oli Bartsch voted on patch set 4 V +1 CR +1 08/01/20, 09:42

  13. TYPO3com voted on patch set 4 V +1 08/01/20, 09:42 Completed build in 12m 17s on Sat, 1 Aug, 09:42 AM Test Summary: 73002 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4345

  14. Jo Hasenau voted on patch set 4 V +1 CR +1 08/01/20, 12:32

  15. Christian Knauf commented on patch set 4 V +1 08/03/20, 22:01

    I imported the patch into an TYPO3 10.4.6 and the sorting works again
  16. Andreas Kienast voted on patch set 4 CR +1 08/05/20, 13:25

  17. Jo Hasenau commented on patch set 5 08/05/20, 15:09

    Published edit on patch set 4.
  18. TYPO3com voted on patch set 5 V +1 08/05/20, 15:20 Completed build in 11m 25s on Wed, 5 Aug, 03:20 PM Test Summary: 72957 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4426

  19. Benni Mack commented on patch set 5 08/05/20, 15:26

    Any chance to add some functional tests to it?
  20. Jürgen Venne voted on patch set 5 V +1 CR +1 08/05/20, 16:39

  21. Jo Hasenau commented on patch set 5 08/05/20, 20:17

    > Patch Set 5: > > Any chance to add some functional tests to it? If there is someone, who is able to do functional tests, feel free. But since this has been broken without requesting tests before, we should not make those tests mandatory for the fix now.
  22. Oliver Klee commented on patch set 5 08/07/20, 11:34

    > But since this has been broken without requesting tests before, we should not make those tests mandatory for the fix now. "Not having tests allowed us to break this without noticing. So we shouldn't have tests that would keep us from breaking this again."
  23. Jo Hasenau commented on patch set 5 08/07/20, 12:01

    > "Not having tests allowed us to break this without noticing. So we shouldn't have tests that would keep us from breaking this again." The point is not, that we shouldn't have tests at all, but that we should not force people, who already did the fix, into writing tests on top of that, although this was obviously not necessary while doing the breaking change. That's why I wrote "if there is someone, who is able to do functional test, feel free."
  24. Oliver Klee commented on patch set 5 CR -1 08/07/20, 12:48

    > The point is not, that we shouldn't have tests at all, but that we should not force people, who already did the fix, into writing tests on top of that I was thinking more along the lines of "not merging this until we have covered this with functional tests (regressions tests)", i.e., I was focusing on the patch, not on people. I don't know here the "forcing people to do something" sentiment comes from. I understood that you'd like to have this merged even without tests. I disagree. And if I have misunderstood you and you didn't want this to be merged even without tests, then everything is fine. :-)
  1. Jo Hasenau commented on patch set 5 08/07/20, 13:14

    > I was thinking more along the lines of "not merging this until we have covered this with functional tests (regressions tests)", i.e., I was focusing on the patch, not on people. I don't know here the "forcing people to do something" sentiment comes from. > > I understood that you'd like to have this merged even without tests. I disagree. And if I have misunderstood you and you didn't want this to be merged even without tests, then everything is fine. :-) So basically you confirmed, what I wrote before. By giving a -1 based on "code review" just because things are missing you consider to be mandatory, while others do not, you are enforcing someone else to write those tests, since IIRC you refuse to write them too. So the code will stay broken and people won't be able to sort records in the page module unless someone else steps in to fulfill your personal requirements. If you ever asked yourself, why people might step back from providing their spare time for core development - that's one of the reasons why.
  2. Oliver Klee commented on patch set 5 08/07/20, 13:29

    > So the code will stay broken and people won't be able to sort records in the page module unless someone else steps in to fulfill your personal requirements. > > If you ever asked yourself, why people might step back from providing their spare time for core development - that's one of the reasons why. This is not "my personal requirement" - this is my contribution as a professional software engineer with a focus on high-quality, maintainable software, judging and advising by my best professional knowledge. And don't you put the blame on people not contributing on my shoulders. That's out of proportion, and that's no way to discuss. I don't personally blame you for regressions either.
  3. Oliver Klee commented on patch set 5 08/07/20, 14:19

    For the record: I strongly advise to only merge this together with regressions tests. If a core team member thinks that merging this change without regression tests is acceptable concerning the risk of regressions, please feel free to override my -1.
  4. Oliver Klee commented on patch set 5 08/07/20, 14:56

    Also, my understanding is that this is a regressions that happened as we did not have sufficient tests for this part of the code. If I'm mistaken in this, please let me know.
  5. Oliver Klee wrote 08/07/20, 16:40

    Removed reviewer Oliver Klee with the following votes: * Code-Review-1 by Oliver Klee <typo3-coding@oliverklee.de>
  6. Andre Bardoun commented on patch set 5 V +1 CR +1 08/10/20, 09:26

    The function is essential for the editors. In my opinion, we can optimized that later. Please merge.
  7. Daniel Hoffmann commented on patch set 5 V +1 CR +1 08/24/20, 09:51

    Works and is essential for editors, please merge fast.
  8. Christian Kuhn commented on patch set 5 CR -1 08/25/20, 09:54

    working on it ...
  9. Christian Kuhn uploaded patch set 6 08/25/20, 10:12

  10. Simon Gilli updated the commit message in patch set 7 08/25/20, 10:15

  11. Christian Kuhn commented on patch set 6 V +1 CR +1 08/25/20, 10:20

    Let's get this done. Regarding tests: The changed functionality changes the view only, not a db state. Thus, a functional test wouldn't help here much. The only way to test this is with an acceptance test: Create a page, go to page module, add some content elements, drag&drop them around, verify order. I'd estimate half a day of work or so for this scenario. Nobody seems to be able to spend this for the patch currently. However, the issue is quite annoying and it is in one of our main interfaces that editors use all the time. We thus need to get this fix done. Additionally, the added order-by clauses are at an obvious place in the code now - i don't think anyone would accidentally remove this again. So yes, an acceptance test for this would be great, but we should leave it out if nobody can do that currently, to get the fix in. +1 on reading an testing.
  12. Christian Kuhn updated the commit message in patch set 8 08/25/20, 10:22

  13. Christian Kuhn voted on patch set 8 V +1 CR +1 08/25/20, 10:22

  14. TYPO3com voted on patch set 6 V -1 08/25/20, 10:37 Completed build in 25m 27s on Tue, 25 Aug, 10:37 AM Test Summary: 2 of 72856 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4939

  15. TYPO3com voted on patch set 8 V -1 08/25/20, 10:52 Completed build in 25m 23s on Tue, 25 Aug, 10:52 AM Test Summary: 72422 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4942

  16. TYPO3com voted on patch set 7 V -1 08/25/20, 10:53 Completed build in 33m 56s on Tue, 25 Aug, 10:53 AM Test Summary: 72109 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4941

  17. TYPO3com voted on patch set 8 V +1 08/25/20, 11:00 Completed build in 59m 11s on Tue, 25 Aug, 11:00 AM Test Summary: 73072 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-4942

  18. Daniel Siepmann commented on patch set 8 08/25/20, 11:22

    PageLayoutView.php:1927

    PHP should trigger a notice if the given table does not have `sortby` or `default_sortby`. I guess the only valid table here is `tt_content` anyway, which has it set by default?
  19. Christian Kuhn commented on patch set 8 08/25/20, 11:27

    PageLayoutView.php:1927 Reply Unresolved

    thought about that, too ... imho fine for now since $table will always be tt_content anyway. having $table flexible here is probably just a left over from before this class has been extracted from AbstractDatabaseRecordList in v9 (which has been used for list, filelist and page module before). imho good to keep like that for the time being, a refactoring could be possible later - maybe similar to what daniel windloff is currently doing for the list module.
  20. Daniel Siepmann voted on patch set 8 CR +1 08/25/20, 11:31

  21. Jürgen Venne voted on patch set 8 V +1 CR +1 08/25/20, 12:05

  22. Richard Haeser voted on patch set 8 CR +1 08/25/20, 12:16

  23. Denis Mir commented on patch set 8 V +1 CR +1 08/25/20, 14:01

    Verified the patch on systems that had the error.
  24. Anja Leichsenring voted on patch set 8 V +2 CR +2 08/25/20, 16:49

  25. Anja Leichsenring merged patch set 9 08/25/20, 16:49