81166[BUGFIX] Fix performance issue when resolving Extbase relations in workspaces
Commit message
Review
-
Code review CR +2 Done
- Benni Mack +2
- Stefan Bürk +1
-
Verified V +2 Done
- Benni Mack +2
- Stefan Bürk +1
- core-ci +1 CI
Issues 6
- Bug #99244DataMapper uses a lot of memory when retrieving MM-relations in workspaces
- Bug #88021Preview of a record related with FAL does not work when file is changed in a workspace
- Bug #82750Workspace preview doesn't preview MM relations properly
- Bug #82086Frontend Preview Mode And Missing Inline Elements in Workspace
- Bug #76993Getting a single extbase query result in a workspace for a relation that has been deleted and re-added returns no relation
- Task #57169Invalid relations of MM records in workspaces
Files 1
Review log 3547
Show 1622 earlier messages
-
Rémy DANIEL uploaded patch set 1 09/21/23, 20:49
-
core-ci voted on patch set 1 V -1 09/21/23, 20:59 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/47038
-
Rémy DANIEL uploaded patch set 2
V -109/21/23, 21:19 -
core-ci voted on patch set 2 V +1 09/21/23, 21:26 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/47039
-
Rémy DANIEL rebased onto patch set 3
V +106/17/24, 14:08 -
core-ci voted on patch set 3 V +1 06/17/24, 14:16 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59734
-
Friedemann Altrock commented on patch set 3 CR +1 V +1 08/30/24, 14:30
Just ran into this problem with hundreds of MM objects being hydrated and PHP running out of memory. Can confirm that the patch fixes it. -
Benni Mack updated the commit message in patch set 4
CR +1V +110/18/24, 13:47 -
Rémy DANIEL commented on patch set 4 10/18/24, 14:46
Hi Benni Why not backport this patch also to 12.4, where the issue still exists? Our 11.5 ELTS instances would also love to have it. -
Friedemann Altrock voted on patch set 4 CR +1 V +1 11/10/24, 13:26
-
Benni Mack commented on patch set 4 CR -1 11/15/24, 10:28
I'm fine adding 12.4 againDataMapper.php:547needs a if ($columMap->getParentKeyFieldName() !== null) wrapper -
Rémy DANIEL uploaded patch set 5
CR +1V +111/15/24, 15:12 -
Rémy DANIEL commented on patch set 5 11/15/24, 15:13
DataMapper.php:547ReplyDone -
core-ci voted on patch set 5 V +1 11/15/24, 15:19 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/69488
-
Benni Mack commented on patch set 5 11/15/24, 16:25
DataMapper.php:529this looks like it is moved out of resolveRelationValuesOfField() into this calling method. Is there any benefit on why to do this? -
Rémy DANIEL commented on patch set 5 11/15/24, 17:12
DataMapper.php:529ReplyYes. $parentId is now needed in 2 places: inside resolveRelationValuesOfField where it was calculated before, and in the new sql constraint. In order not to introduce duplicated code, I extracted $parentId calculation outside of resolveRelationValuesOfField. -
Benni Mack uploaded patch set 6
V +111/15/24, 22:19 -
Benni Mack commented on patch set 6 CR +1 11/15/24, 22:21
I hope my changes are OK, I find this change now much more readable. However, your remarks in this patch were really helpful! Can you also verify if this change is OK like it is, Rémy! -
Benni Mack commented on patch set 6 11/15/24, 22:21
DataMapper.php:529ReplyThanks for the feedback! -
core-ci voted on patch set 6 V -1 11/15/24, 22:30 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/69506
-
Benni Mack commented on patch set 7
CR +1V -111/15/24, 23:04Published edit on patch set 6. -
core-ci voted on patch set 7 V -1 11/15/24, 23:13 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/69510
-
Rémy DANIEL commented on patch set 7 11/18/24, 08:41
Reply
Thanks, your change is much more readable. -
Benni Mack rebased onto patch set 8
V -111/22/24, 16:31 -
core-ci voted on patch set 8 V -1 11/22/24, 16:41 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70058
-
Benni Mack rebased onto patch set 9
V -112/11/24, 16:33 -
core-ci voted on patch set 9 V -1 12/11/24, 16:48 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70955
-
Benni Mack uploaded patch set 10
V -112/11/24, 17:01 -
Benni Mack voted on patch set 10 CR +1 V +1 12/11/24, 17:01
-
core-ci voted on patch set 10 V +1 12/11/24, 17:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70960
-
Benni Mack rebased onto patch set 11
CR +1V +112/11/24, 18:41 -
core-ci voted on patch set 11 V +1 12/11/24, 18:49 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70975
-
Benni Mack voted on patch set 11 CR +1 V +1 12/13/24, 14:10
-
Rémy DANIEL voted on patch set 11 CR +1 V +1 12/23/24, 09:35
-
Stefan Bürk voted on patch set 11 CR -1 12/23/24, 10:49
-
Stefan Bürk commented on patch set 11 12/23/24, 11:10
DataMapper.php:542We are talking about a join here and this statement only adds an equal check against a value for one value: ``` parentKeyFieldName = $parentId ``` which is not guaranteed to be unique within an join sql statement - and even more open for conflicts with multiple joins. I'd say that we need to use the tablename + fieldname combination to make this concreate and deterministic from a sql point of view. ``` $query->equals( $columnMap->getParentKeyTableName() . '.' . $columnMap->getParentKeyFieldName(), $parentId ) ``` to have the sql as ``` parentTableName.parentFieldname = $parentId ``` Not that deep into the code here, but the table may be aliased and not sure if that is reflected in the ColumnMap table. That should also be investigated and the alias mapping used instead of the real tablename. This is about extbase - I would feel more confident having some kind of functional test for this, but not that deep into extbase to have it directly in mind to provide it directly. This solution seems to be incomplete for me or did I missed something ? -
Rémy DANIEL commented on patch set 11 12/23/24, 11:45
DataMapper.php:542ReplyHi Stefan Here is an example of the generated sql query: ``` SELECT `sys_category_record_mm`.*, `sys_category`.* FROM `sys_category_record_mm` LEFT JOIN `sys_category` `sys_category` ON ((`sys_category_record_mm`.`uid_local` = `sys_category`.`uid`) AND (((`sys_category`.`deleted` = 0) AND (`sys_category`.`hidden` = 0) AND (`sys_category`.`starttime` <= 1734954060) AND (((`sys_category`.`endtime` = 0) OR (`sys_category`.`endtime` > 1734954060)))))) WHERE ((((((`sys_category_record_mm`.`uid_local` IN (?)) AND (`sys_category_record_mm`.`uid_foreign` = ?))) AND (`sys_category_record_mm`.`fieldname` = ?))) AND (`sys_category_record_mm`.`tablenames` = ?) AND (`sys_category`.`deleted` = 0 AND ((`sys_category`.`t3ver_wsid` = 0) OR (`sys_category`.`t3ver_wsid` = 1)))) ORDER BY `sys_category_record_mm`.`sorting_foreign` ASC ``` You can see the table in the LEFT JOIN is aliased and the alias is used in combination with fieldnames in the ON clause. -
Stefan Bürk commented on patch set 11 12/30/24, 10:25
DataMapper.php:542ReplyThanks for the answer. That was not clear fro. the clasa/method desceiption. Going to have a look directly after the year switch and try to comeing up with some tests for this. -
Garvin Hicking commented on patch set 11 12/30/24, 12:08
Can you add a "Related: ..." to the issue of that referenced commit a1f22a427f3bc49ab3fc3424859a7e2d7beaa1c9 ? -
Stefan Bürk updated the commit message in patch set 12
CR +1V +101/02/25, 14:11 -
Stefan Bürk commented on patch set 12 CR +1 V +1 01/02/25, 14:15
Reply
DoneDataMapper.php:542ReplyNot easily doable - needs some pre-work/structure. Added to todo list to add these kind of tests for extbase repository/ORM handling in general. -
core-ci voted on patch set 12 V +1 01/02/25, 14:17 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/71829
-
Stefan Bürk updated the commit message in patch set 13
CR +1V +101/02/25, 15:01 -
Stefan Bürk voted on patch set 13 CR +1 V +1 01/02/25, 15:01
-
core-ci voted on patch set 13 V +1 01/02/25, 15:06 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/71832
-
Benni Mack voted on patch set 13 CR +2 V +2 01/02/25, 20:34
-
Benni Mack merged patch set 14 01/02/25, 20:43