Issue Bug Under Review Quiet

#105578Lost MM relations in workspaces

Ruben Zielke 11/12/24, 09:14 updated 10/03/26, 05:42

Description

Description

During the publishing process of related records the relations get lost.

Relations between two or more entities (e.g. products and references) are lost as soon as the products and the related references have an editing status in the editorial (non-live) workspace and are subsequently published.

See my TCA attached.

How to reproduce?

Initial situation

Products tx_solutions_domain_model_product
*References* tx_solutions_domain_model_reference
*MM Table* tx_solutions_domain_model_product_reference_mm

Products and references are linked via an MM table.
All data is in the live workspace

Products tx_solutions_domain_model_product
_.uid _.t3ver_oid _.t3ver_wsid _.t3ver_state
1 0 0 0
References tx_solutions_domain_model_reference.
_.uid _.t3ver_oid _.t3ver_wsid _.t3ver_state
1 0 0 0
2 0 0 0
Relation tx_solutions_domain_model_product_reference_mm
_.uid_local (product uid) _.uid_foreign (reference uid)
1 1
1 2

Editing reference

A reference (uid 1) is edited in the Editorial Workspace.
A copy with a new uid 3 of the data record is automatically created.
tx_solutions_domain_model_reference.t3ver_oid = 1
tx_solutions_domain_model_reference.t3ver_wsid = 1
tx_solutions_domain_model_reference.t3ver_state = 1

Products tx_solutions_domain_model_product
_.uid _.t3ver_oid _.t3ver_wsid _.t3ver_state
1 0 0 0
References tx_solutions_domain_model_reference.
_.uid _.t3ver_oid _.t3ver_wsid _.t3ver_state
1 0 0 0
2 0 0 0
3 1 1 1
Link tx_solutions_domain_model_product_reference_mm
_.uid_local (product uid) _.uid_foreign (reference uid)
1 1
1 2

Editing product

A product (uid 1) is edited in the Editorial Workspace.
A copy with a new uid 2 of the data record is automatically created.
tx_solutions_domain_model_product.t3ver_oid = 1
tx_solutions_domain_model_product.t3ver_wsid = 1
tx_solutions_domain_model_product.t3ver_state = 1

Products tx_solutions_domain_model_product
_.uid _.t3ver_oid _.t3ver_wsid _.t3ver_state
1 0 0 0
2 1 1 1
References tx_solutions_domain_model_reference.
_.uid _.t3ver_oid _.t3ver_wsid _.t3ver_state
1 0 0 0
2 0 0 0
3 1 1 1
Relation tx_solutions_domain_model_product_reference_mm
_.uid_local (product uid) _.uid_foreign (reference uid)
1 1
1 2
2 3

Publication of the product

The changes to the product are published.
The changes to the data record are transferred from ‘uid 2’ to ‘uid 1’.
In the relation table, the additional entry (uid_local 5) is removed and the original entry uid_local 1 is updated with the uid 4 of the edited reference.
However, the reference uid 3 is not available in the live workspace.

Products tx_solutions_domain_model_product
_.uid _.t3ver_oid _.t3ver_wsid _.t3ver_state
1 0 0 0
2 1 1 1
References tx_solutions_domain_model_reference.
_.uid _.t3ver_oid _.t3ver_wsid _.t3ver_state
1 0 0 0
2 0 0 0
3 1 1 1
Relation tx_solutions_domain_model_product_reference_mm
_.uid_local (product uid) _.uid_foreign (reference uid)
1 3
1 2
1 3

Publication of the reference

The changes to the reference are published.
The changes to the data record are transferred from uid 3 to uid 1.
There is no change in the relation table.
However, the reference uid 3 is not available in the live workspace.

Products tx_solutions_domain_model_product
_.uid _.t3ver_oid _.t3ver_wsid _.t3ver_state
1 0 0 0
References tx_solutions_domain_model_reference.
_.uid _.t3ver_oid _.t3ver_wsid _.t3ver_state
1 0 0 0
2 0 0 0
3 1 1 1
Relation tx_solutions_domain_model_product_reference_mm
_.uid_local (product uid) _.uid_foreign (reference uid)
1 3
1 2

I would have expected the entry in the relation table to be updated when publishing the reference.

BTW: If a related record is edited in a non-live workspace, the stored relation is no more visible in the record view (backend). An editor has no information if the relation is really lost or not shown because of the workspace overlay.
May be it is an separate issue.

Please let me know if you need any further information.

Reviews 4

Files 3

History 1213

  1. Christian Kuhn commented <span data-processor="relativetime" data-value="2024-11-27T08:16:21+00:00" > 11/27/24, 08:16 </span>

    Quick feedback, I had a ~2h look at this.

    I was able to replay it in a similar but not identical way on a main/v14. Changing the TCA slightly influences details, one area is select-foreign_table vs. group-allowed and then having fieldname/tablename in MM.

    Category relations – which are based on similar TCA do not show the behavior and don’t loose “existing” connections when the foreign side of a relation is workspace-changed after local side has been workspace-changed. I did not understand why, yet.

    I suppose this is a mixture of funny TCA plus a core issue, and we need to sort out details. And we need more functional test preparation: DataScenarios/CategoryManyToMany is currently very similar to DataScenarios/ManyToMany but misses the workspace scenarios. Those should be added, and DataScenarious/ManyToMany should then be changed to no longer use sys_category, but some “own” MM relation by an extension. This would add the test base for a non-sys_category bed we can play with, and to pin point differences to sys_category. I’d say these preparation steps are ~3 days for a dev who know whats going on.

  2. Francois Suter commented <span data-processor="relativetime" data-value="2025-09-24T10:03:18+00:00" > 09/24/25, 10:03 </span>

    I have observed the same issue. If I relate my experience with that of Ruben, the bug already pops up with the following scenario:
    • modify the reference in a workspace
    • publish the changes to the reference
      The result is that the product loses the relation to the reference.

    However I have tested this is a project with many such relations and it does not happen in 100% of the cases, although every case seems similar. I’m quite puzzled for now. This is a pretty hairy bug, but critical for one of my clients.

  3. Francois Suter commented and changed attached relation_test_1.0.0_202509260849.zip <span data-processor="relativetime" data-value="2025-09-26T06:56:22+00:00" > 09/26/25, 06:56 </span>

    I have created a simple test extension (attached here) with which I can reproduce the problem (at least the variant of the problem that affect me) consistently across TYPO3 12, 13 and 14/main. It is about having a MM table managing several relations and thus using fieldname/tablenames (like sys_category_record_mm).

    In the test extension, I have people and offices. There’s a relation field between people and offices, where several offices can be selected (I have created other relations, but between people and people, and offices and offices, but relations with self behave yet differently and I have left that aside for now). I create some people and offices records and assign people to various offices. Then I switch to a workspace and modify an office. In the MM table an extra record is created, for managing the relation between the office workspace copy and the employee. This entry has an empty “fieldname” field (“sorting_foreign” is also set to 0 instead of 1).

    When the change to the office is published, the new relation in the MM table replaces the old one, and the “fieldname” and “sorting_foreign” information is lost, thus – in effect – making the relation disappear.

    I hope this can help test and solve this issue.

  4. Francois Suter commented <span data-processor="relativetime" data-value="2025-09-29T14:45:35+00:00" > 09/29/25, 14:45 </span>

    I continue digging into this. In particular I compare behavior between my custom tables and sys_category. No result yet, but the observation that sys_category_record_mm is impacted by this bug in relation with the sorting field: the workspace copy uses the “sorting” field instead of the “sorting_foreign” field. This error is preserved upon publication, since it is the new relation record which is remapped with the live id.

  5. Francois Suter commented <span data-processor="relativetime" data-value="2025-10-01T13:06:06+00:00" > 10/01/25, 13:06 </span>

    Here are my latest updates, in case they can be useful in solving this bug:

    First of all, I had misunderstood how to set up the “MM_oppositeUsage” property correctly, mostly because I had looked at the code for sys_category, not realising that some magic happened in \TYPO3\CMS\Core\Preparations\TcaPreparation::configureCategoryRelations() to dynamically populate “MM_oppositeUsage”.

    With that corrected I had more consistent results. Most of the time the “fieldname” column was filled, but it failed when a table had more than one field related to the same foreign table. This is actually documented in \TYPO3\CMS\Core\Database\RelationHandler::completeOppositeUsageValues()

    The other error – which always happens – is that the sorting is stored in the “sorting” field instead of the “sorting_foreign” field. This is understandable since we seem to be acting from the foreign side, but is actually a wrong guess.

    I have managed to work around this issue by using a CmdMap hook to do the following:

    - gather all relation definitions (from the TCA) for the table where a new version has been created
    - gather all relations in MM tables to the original record
    - delete all relations in MM tables to the new version
    - recreate all relations based on the existing ones and changing the “uid_local” value

    This preserves both the “fieldname” and “sorting_foreign” information. When the change is published, no information is lost.

    I think the solution would be to copy the existing relations and updates them with the uid of the new version. However this is quite a major change from the actual way of doing things, especially since the RelationHandler is used in both workspace and LIVE scenarios. Maybe some parts of the RelationHandler could be reserved to LIVE operations and add events that could be used by the workspaces extension.

  6. Mathias Bolt Lesniak commented <span data-processor="relativetime" data-value="2025-10-01T16:30:02+00:00" > 10/01/25, 16:30 </span>

    Thanks for the updates, @francois. I enjoy reading them. It looks like you’re closing in on a solution. Good work!

  7. Francois Suter commented <span data-processor="relativetime" data-value="2025-12-26T17:01:53+00:00" > 12/26/25, 17:01 </span>

    I’m getting back to this topic, because the various workaround we use in our projects are not sufficient. From many tests I have made these last few days, what it seems to boil down to, is that the TYPO3 Core mistakenly creates relations to workspace copies in the database. These relations cannot be understood by the FormEngine and thus get lost if any further work on the related records is done. If no more work is done and the changes are published, it is okay, because the Core rewrites the relations with the live uids.

    Unless there are cases which I don’t know about, it just seems plain wrong – and useless – to create relations to workspace copies. Relations should always be registered using the live uid (which is also okay for records that are new in a workspace, since they keep their uid upon publication).

    I haven’t yet found the place where this happens and – of course – it remains to be seen what kind of side effects such a fix could have. A colleague of mine will try patching the Core. If all else fails we might fall back to using a hook again, checking the relations for the records that were just touched upon and rewrite their relations in the database.

  8. Georg Ringer commented <span data-processor="relativetime" data-value="2026-10-03T05:42:09+00:00" > 10/03/26, 05:42 </span>

    Thanks a lot for all the analysis and the test extension, Francois! I had a closer look and could reproduce the core bug on main. However, the test extension also contains configuration issues which cause part of the observed behaviour, so let me separate both.

    Configuration issues in the test extension

    The relation between offices and people is stored from the office side (tx_relationtest_domain_model_office.people, local side) and read from the people side (tx_relationtest_domain_model_people.offices with MM_opposite_field).

    • 'MM_oppositeUsage' => [] in office.people is empty. Without it, TYPO3 cannot know which field on the opposite side a relation belongs to, so relations written from the office side get an empty fieldname. The people side reads with MM_match_fields fieldname = offices and therefore does not see these relations. As you found out already, it has to be configured, e.g.:
    'MM_oppositeUsage' => [
        'tx_relationtest_domain_model_people' => ['offices'],
    ],
    • office.subsidiaries and office.people use the same MM table tx_relationtest_office_mm, both from the local side (uid_local is the office). office.people has no MM_match_fields and allowed = '*', so it reads all rows of the office, including the subsidiaries relations (tablenames = tx_relationtest_domain_model_office). Use a separate MM table for the subsidiaries, or restrict allowed to tx_relationtest_domain_model_people.

    The core bug

    With a correct configuration, relations are still lost as soon as both sides of a relation are modified in a workspace and only one of them is published. It can be reproduced with the core test extension test_mm:

    1. Live: tx_test_mm:28 is related to tt_content:297
    2. In a workspace, modify tt_content:297 (version 299), then modify tx_test_mm:28 (version 32). The workspace relation is stored as 32 -> 299.
    3. Publish tx_test_mm:28 only: the live relation 28 -> 297 is deleted, and the workspace relation becomes 28 -> 299, which points to the unpublished version of the content element.
    4. Discard the content element: the relation is gone for good.

    Publishing the content element first shows the same problem the other way round: live temporarily points to the workspace version 32 of tx_test_mm:28.

    The cause is in DataHandler::versionPublishManyToManyRelations(). When publishing a record, all live MM rows of the record are deleted and all MM rows of its workspace version are moved to the live uid, without checking whether the record on the opposite side is still an unpublished version. This is also why Ruben’s reference loses the relation.

    A patch with functional tests covering these scenarios is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/96342: on publish, live gets a relation to the live record of such a version, while the workspace keeps the relation to the version until it is published or discarded.

    Two further findings that will be handled separately:

    • When a version is created, the copied MM rows lose sorting_foreign (the sorting is written to sorting instead), as you already noticed.
    • Publishing a record that is new in the workspace does not touch MM rows at all, so the same situation may happen there.