Review main Merged

71374[BUGFIX] Enable Export/Import of page translations

Nikita Hovratov CR +2 V +2 Expert +1196 −3 updated 11/21/23, 15:19

Commit message

The impexp system extension is now able to export translated pages. This
essential functionality has not been implemented up until now.

Also, specific language variants can be exported, that will only
contain records matching the requested language. This "hidden
feature" can be accessed by right-clicking on a page's icon in the
translation overview and selecting "export". This feature was
available before, but the export never included actual translated
records before.

In addition, the relation field "l10n_source" is taken care of while
exporting/importing. Normally this would happen automatically, if there
was a refindex entry for that. However, no references are saved for
the tables "pages" and "tt_content". This seems to be another bug and
should be fixed in another patch. After this, some code can be cleaned
up.

Functional tests have been added to verify the exported data.

Review

  • Code review CR +2 Done
    • Christian Kuhn +2
    • Garvin Hicking +1
    • Stefan Bürk +1
  • Verified V +2 Done
    • Christian Kuhn +2
    • Garvin Hicking +1
    • Stefan Bürk +1
    • core-ci +1 CI

Issues 2

Files 8

Review log 4465

Show 2440 earlier messages
  1. Nikita Hovratov uploaded patch set 1 09/29/21, 17:56

  2. Nikita Hovratov uploaded patch set 2 09/29/21, 18:09

  3. core-ci voted on patch set 1 V -1 09/29/21, 18:11 Core CI is not happy: https://gitlab.typo3.org/typo3/CI/cms/-/pipelines/17318

  4. core-ci voted on patch set 2 V -1 09/29/21, 18:26 Core CI is not happy: https://gitlab.typo3.org/typo3/CI/cms/-/pipelines/17319

  5. Nikita Hovratov uploaded patch set 3 09/30/21, 06:56

  6. core-ci voted on patch set 3 V +1 09/30/21, 07:12 Core CI is happy: https://gitlab.typo3.org/typo3/CI/cms/-/pipelines/17326

  7. Nikita Hovratov marked it work in progress 10/07/21, 11:31

  8. Benni Mack wrote 11/29/21, 10:12

    Change destination moved from master to main
  9. Benni Mack updated the commit message in patch set 4 11/29/21, 13:37

  10. Nikita Hovratov abandoned the change 04/04/22, 18:06

    no time for that
  11. Nikita Hovratov restored the change 10/23/23, 06:29

  12. Nikita Hovratov rebased onto patch set 5 10/23/23, 06:30

  13. Nikita Hovratov commented on patch set 5 10/23/23, 06:30

    This change is ready for review.
  14. Garvin Hicking commented on patch set 5 10/23/23, 06:32

    will work on this
  15. core-ci voted on patch set 5 V -1 10/23/23, 06:42 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48064

  16. Garvin Hicking uploaded patch set 6 V -1 10/31/23, 11:12

  17. Garvin Hicking uploaded patch set 7 10/31/23, 11:18

  18. core-ci voted on patch set 6 V -1 10/31/23, 11:29 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48394

  19. core-ci voted on patch set 7 V -1 10/31/23, 11:32 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48396

  20. Garvin Hicking uploaded patch set 8 V -1 10/31/23, 11:49

  21. core-ci voted on patch set 8 V -1 10/31/23, 11:58 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48400

  22. Garvin Hicking uploaded patch set 9 V -1 10/31/23, 12:42

  23. Garvin Hicking commented on patch set 9 10/31/23, 12:44

    Stardate 20231031.1342, Garvin's supplemental. Rebased on top of current main (might make backports tougher due to recent changes; note QueryBuilder not using PARAM_STR_ARRAY). Also addressed that translated pages were not considered "inside page tree", which broke functional tests. So I moved them from "outside page tree" to their "inside" section. Basic export and import at this point seems to work nicely. Will test some more edge cases.
  24. core-ci voted on patch set 9 V -1 10/31/23, 12:53 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48401

  25. Garvin Hicking uploaded patch set 10 V -1 10/31/23, 12:55

  26. core-ci voted on patch set 10 V -1 10/31/23, 13:05 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48403

  27. Garvin Hicking uploaded patch set 11 V -1 10/31/23, 13:46

    Fix Tests by adjusting pagetree again
  28. core-ci voted on patch set 11 V -1 10/31/23, 13:53 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48406

  29. Garvin Hicking uploaded patch set 12 V -1 10/31/23, 13:57

    Move comments to iteration and remove redundant code
  30. core-ci voted on patch set 12 V +1 10/31/23, 14:05 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48407

  31. Garvin Hicking uploaded patch set 13 V +1 11/03/23, 12:55

    Support exporting a single language
  32. core-ci voted on patch set 13 V +1 11/03/23, 13:02 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48547

  33. Garvin Hicking uploaded patch set 14 V +1 11/03/23, 14:47

    Add very basic fixture and testing stub
  34. core-ci voted on patch set 14 V +1 11/03/23, 14:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48551

  35. Garvin Hicking uploaded patch set 15 V +1 11/10/23, 14:48

    Add proper tests and fixtures
  36. core-ci voted on patch set 15 V -1 11/10/23, 14:57 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48791

  37. Garvin Hicking uploaded patch set 16 V -1 11/10/23, 16:08

    Make Tests deterministic by using sorting table field
  38. core-ci voted on patch set 16 V -1 11/10/23, 16:15 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48794

  39. Garvin Hicking uploaded patch set 17 V -1 11/10/23, 16:18

    Fix CGL
  40. core-ci voted on patch set 17 V +1 11/10/23, 16:26 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/48795

  1. Garvin Hicking commented on patch set 17 V +1 CR +1 11/10/23, 16:37

    Open for review! And since I'm not listed as an author I can technically vote for it 😇
  2. Simon Schaufelberger commented on patch set 17 11/16/23, 00:42

    Export.php:312

    Would it make sense to put this in some kind of repository class?

    MultilingualPagesAndTtContentTest.php:26

    indentation doesn't look right here

    MultilingualPagesAndTtContentTest.php:30

    indentation doesn't look right here

    MultilingualPagesAndTtContentTest.php:33

    different code style: ``` protected array $recordTypesIncludeFields = [ 'pages' => [ 'title', 'deleted', 'doktype', 'hidden', 'perms_everybody', 'l10n_parent', 'l10n_source', 'sys_language_uid', 'sorting', ], 'tt_content' => [ 'CType', 'header', 'deleted', 'l18n_parent', // note l18n vs. l10n 'l10n_source', 'sys_language_uid', 'sorting', ], ]; ```
  3. Garvin Hicking uploaded patch set 18 CR +1 V +1 11/16/23, 08:54

    Rebased and adjusted indentation
  4. Garvin Hicking commented on patch set 18 11/16/23, 08:56

    Thanks Simon for getting into this. I fixed the indentation. I copied the Test from `typo3-core/typo3/sysext/impexp/Tests/Functional/Export/PagesAndTtContentTest.php` and that had the same "wrong" indentation. Not sure I should fix that file too, because it changes the scope of this patch. But now at least this test has proper indentation.

    Export.php:312 Reply

    (This part comes from Nikita) - I think due to the usage of 'yield' it might make it troublesome within a repository? Also I think the ImpExp-module needs some more major refactoring, so doing it at this point would change the scope of this bugfix (and also maybe make backporting to older versions harder). Purely architecturally, I fully agree with you of course!

    MultilingualPagesAndTtContentTest.php:26 Reply

    Done

    MultilingualPagesAndTtContentTest.php:30 Reply

    Done

    MultilingualPagesAndTtContentTest.php:33 Reply

    Done
  5. core-ci voted on patch set 18 V +1 11/16/23, 09:01 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49006

  6. Simon Schaufelberger commented on patch set 18 11/16/23, 11:38

    Export.php:312 Reply

    Ah yes, because the function is called as an iterator, it needs to be refactored in order to be used as a proper method within a class. Still, DB stuff should be kept in one place...
  7. Simon Schaufelberger voted on patch set 18 CR +1 11/16/23, 11:38

  8. Garvin Hicking voted on patch set 18 V +1 CR +1 11/16/23, 13:41

  9. Garvin Hicking uploaded patch set 19 CR +1 V +1 11/21/23, 10:16

    Made helper method protected changed scope to 12 and mainn
  10. core-ci voted on patch set 19 V +1 11/21/23, 10:26 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49209

  11. Christian Kuhn commented on patch set 18 11/21/23, 11:14

    Hey. I'd opt to remove v11 from backport list: This patch is a bit too intrusive, a bit too risky and the issue is not critical enough. If anyone *really* needs this in v11, it should backport the patch into the affected instance itself.
  12. Oliver Hader commented on patch set 19 11/21/23, 11:48

    Export.php:270

    Wouldn't `$this->dat['header']['pagetree'][(int)$pageTranslation['uid']]['uid'] = ...` enough here → thus, omitting the if-check

    Export.php:319

    Other parts of the export functionality seem to apply a restriction for the live workspace - which seems to be missing here

    Export.php:505

    `$restrictToLanguageIds !== []`? (`count` is not relevant here)

    Import.php:1055

    File and `l10n_source` - I did not check it, but it seems to be strange...
  13. Garvin Hicking uploaded patch set 20 V +1 11/21/23, 12:58

    Refactor to use ctrl TCA enable workspace restriction
  14. core-ci voted on patch set 20 V -1 11/21/23, 13:04 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49229

  15. Garvin Hicking uploaded patch set 21 V -1 11/21/23, 13:29

    CGL
  16. core-ci voted on patch set 21 V +1 11/21/23, 13:34 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49230

  17. Christian Kuhn commented on patch set 21 11/21/23, 13:55

    Export.php:270 Reply

    Done

    Export.php:319 Reply

    Done

    Export.php:505 Reply

    Done

    Import.php:1055 Reply

    Done
  18. Christian Kuhn uploaded patch set 22 V +1 11/21/23, 14:27

  19. core-ci voted on patch set 22 V +1 11/21/23, 14:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/49236

  20. Christian Kuhn voted on patch set 22 V +1 CR +1 11/21/23, 14:37

  21. Garvin Hicking commented on patch set 22 V +1 CR +1 11/21/23, 14:38

    Export.php:270 Reply

    Thanks, yes!

    Export.php:319 Reply

    Ack

    Export.php:505 Reply

    Ack

    Import.php:1055 Reply

    Probably it applies more to type=db rather than file. This is part of Nikitas original implementation, but I believe due to the fielname fixation to "l10n_source" it should help. Refactored to lookup the TCA ctrl here to be more sure.
  22. Stefan Bürk voted on patch set 22 V +1 CR +1 11/21/23, 14:45

  23. Simon Schaufelberger wrote 11/21/23, 15:03

    Removed reviewer Simon Schaufelberger.
  24. Christian Kuhn voted on patch set 22 V +2 CR +2 11/21/23, 15:19

  25. Christian Kuhn merged patch set 23 11/21/23, 15:19