Review main Merged

76443[BUGFIX] Speed up FormEngine with hundreds of categories

Benni Mack CR +2 V +2 Hard +116 −194 updated 11/08/22, 10:40

Commit message

FormEngine's TreeRendering for categories utilizes
the TreeDataProvider logic, which is handed over
a list of available items loaded by FormEngine's
AbstractItemProvider. These records have already
been validated and a full record was already loaded
by FormEngine at this point.

The TreeDataProvider construct then fetches ALL records
again, as it only works with the "IDs", but not with
already loaded full rows. This makes the construct
expensive. It needs a bigger overhaul.

To mitigate the performance impact of bigger category
trees, the patch applies a hack to suppress loading
single category rows multiple times by handing them
over from FormEngine to the TreeDataProvider construct.

As this change is also backported to 11.5, the
optimizations are only done for type=category to
minimize impact for other Tree usages.

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Oli Bartsch +1
    • Christian Kuhn +1
    • Susi Moog +1
  • Verified V +2 Done
    • Benni Mack +2
    • Oli Bartsch +1
    • Christian Kuhn +1
    • Susi Moog +1
    • core-ci +1 CI

Issues 5

Files 6

Review log 1724

  1. Benni Mack uploaded patch set 1 11/05/22, 14:19

  2. core-ci voted on patch set 1 V -1 11/05/22, 14:30 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/33145

  3. Benni Mack updated the commit message in patch set 2 11/05/22, 14:36

  4. core-ci voted on patch set 2 V -1 11/05/22, 14:46 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/33146

  5. Benni Mack uploaded patch set 3 11/05/22, 21:08

  6. core-ci voted on patch set 3 V -1 11/05/22, 21:22 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/33153

  7. Benni Mack uploaded patch set 4 11/06/22, 13:25

  8. core-ci voted on patch set 4 V -1 11/06/22, 13:43 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/33167

  9. Benni Mack uploaded patch set 5 11/06/22, 21:38

  10. core-ci voted on patch set 5 V -1 11/06/22, 21:52 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/33172

  11. Christian Kuhn commented on patch set 5 CR -1 11/07/22, 19:15

    working on it, will push.
  12. Christian Kuhn uploaded patch set 6 11/08/22, 09:34

  13. Christian Kuhn commented on patch set 6 11/08/22, 09:34

    AbstractItemProvider.php:544

    let's make this argument a bool to not move sql parts around.
  14. core-ci voted on patch set 6 V -1 11/08/22, 09:46 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/33282

  15. Christian Kuhn uploaded patch set 7 11/08/22, 09:49

  16. core-ci voted on patch set 7 V +1 11/08/22, 09:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/33284

  17. Christian Kuhn commented on patch set 7 11/08/22, 09:58

    this is of course a hack and we know it. it's fine from my side, to enable a v11 backport for this. mid-term we should however see if we could get rid of the entire TreeDataProvider stuff in v12, though.
  18. Christian Kuhn voted on patch set 7 V +1 CR +1 11/08/22, 10:00

  19. Oli Bartsch commented on patch set 7 CR -1 11/08/22, 10:03

    Reviewing ..
  20. Susi Moog voted on patch set 7 CR +1 11/08/22, 10:22

  21. Oli Bartsch voted on patch set 7 V +1 CR +1 11/08/22, 10:34

  22. Susi Moog voted on patch set 7 V +1 11/08/22, 10:35

  23. Benni Mack voted on patch set 7 V +2 CR +2 11/08/22, 10:40

  24. Benni Mack merged patch set 8 11/08/22, 10:40