Review main Merged

90401[FEATURE] Automatic history tracking for Extbase entities

Benni Mack CR +2 V +2 Hard +678 −5 updated 03/23/26, 18:32

Commit message

This change introduces automatic history tracking for all
Extbase domain entities by leveraging PSR-14
persistence events and TYPO3's `sys_history` table.

This feature is gated behind a feature toggle that needs
specific opt-in, due to possible data privacy / GDPR
and data-flooding implications.

The `ExtbaseHistoryTracker` event listener automatically captures:

- Entity creation (`EntityAddedToPersistenceEvent`)
- Entity updates (`EntityUpdatedInPersistenceEvent`)
- Entity deletion (`EntityRemovedFromPersistenceEvent`)

All operations include proper user context (frontend, backend, or
anonymous) and store complete entity data snapshots in `sys_history`,
making them visible in the backend's record history functionality.

This can be toggled per-table level, so that it can
also easily be configured in CLI context. TCA Schema API is
extended with this capability.

Review

  • Code review CR +2 Done
    • Georg Ringer +2
    • Sebastian Michaelsen +1
    • Garvin Hicking +1
    • Torben Hansen +1
  • Verified V +2 Done
    • Georg Ringer +2
    • Garvin Hicking +1
    • core-ci +1 CI

Issues 1

Files 20

Review log 5771

Show 3546 earlier messages
  1. Benni Mack uploaded patch set 1 08/14/25, 15:28

  2. Benni Mack uploaded patch set 2 08/14/25, 15:29

  3. Oli Bartsch commented on patch set 2 08/14/25, 15:31

    ExtbaseHistoryTrackerTest.php:36

    A Mock a day ;) Could we use functionals here?
  4. core-ci voted on patch set 1 V -1 08/14/25, 15:44 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/83401

  5. core-ci voted on patch set 2 V -1 08/14/25, 15:45 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/83402

  6. Stefan Froemken commented on patch set 2 08/14/25, 19:31

    ExtbaseHistoryTracker.php:113

    Oh...Our beloved ConfigurationManager. Do we need an extbase request check here? Maybe persistsAll will be called via Middleware and not via Plugin context.
  7. Garvin Hicking commented on patch set 2 09/06/25, 11:04

    Just dropping in to say I love this and want this in my extbase-life :-)
  8. Georg Ringer commented on patch set 2 09/06/25, 14:27

    I am not sure if it maybe better to disable it by tablename? because maybe I get some "non-important" tables and very important ones which should be really logged? additionally it is very hard to get the plugin name?
  9. Benni Mack commented on patch set 2 09/12/25, 21:26

    Reply

    hmm, I like the idea to have this on a per-table basis, but at the same time, we would loose information about this when restoring the information. The main goal to have this feature, is that we have a complete audit trail "This change was made on DD.MM.YYYY" etc. because this might be really helpful for compliance reasons
  10. Georg Ringer commented on patch set 2 09/13/25, 08:29

    Reply

    i love that feature as well and also the reason ^^, just not a per plugin configuration
  11. Garvin Hicking commented on patch set 2 09/13/25, 09:08

    Reply

    Isn't this even more a "per-extension" basis right now? So affects all tables of a single extension (not plugin ctype - then it would be "tx_new_list" or so, right?) A per-plugin basis would allow having low-level plugins/controllers that could disable the functionality based on specific actions of that instance. Per table would allow to exclude certain tables only (like if an extbase extension has its own tracking/logging table). The more I think about it, per-table sounds helpful to me, too... defaulting to "on" is good and hopefully no one needs to list all tables to be exlcuded, that would get cumbersome. Then we might need a "disableAll" per-extension like the current approach alongside - complicating it for the core code though.
  12. Daniel Corn commented on patch set 2 09/16/25, 07:10

    ExtbaseHistoryTracker.php:46

    Should `EntityAddedToPersistenceEvent` be used? I tried to use the code inside of an Extbase extension in TYPO3 **v13**. Updates to the (parent-)model were tracked as "add" alongside the "modify" history entry.
  13. Daniel Corn commented on patch set 2 09/16/25, 07:11

    ExtbaseHistoryTracker.php:46 Reply

    Sorry if trying this in TYPO3 **v13** is nonsense.
  14. Benni Mack commented on patch set 2 10/16/25, 19:32

    ExtbaseHistoryTracker.php:46 Reply

    Daniel, this is fine. Feel free to adapt this change and improve it (if you test it in v13, that's also a good thing)
  15. Garvin Hicking commented on patch set 2 10/17/25, 06:32

    Just to note this here: Georg suggested to add a PSR-14 event for fine-grained control over which table(s) in which cases (CLI, Frontend, Backend) should be tracked. And I agree this would be helpful. Then a "per-table" control could be done with the event, and we could leave the general integrator-friendly configuration on plugin-basis stay. wdyt?
  16. Oliver Klee updated the commit message in patch set 3 V -1 01/22/26, 08:42

  17. core-ci voted on patch set 3 V -1 01/22/26, 08:54 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/94020

  18. Garvin Hicking commented on patch set 3 01/22/26, 10:12

    I'll try to work on this soon(ish).

    ExtbaseHistoryTracker.php:113 Reply

    I'd say the try/catch and default return should take care of it.

    ExtbaseHistoryTrackerTest.php:36 Reply

    Will be reworked, fails anyways. Yes.
  19. Stefan Bürk commented on patch set 3 01/23/26, 00:33

    Reply

    Unpopular question: > Why not having this as TCA->ctrl->extbaseHistory true/false > with "default" as "true" to help people having the audit enabled, > forcing developer to explicitly disable that (or an integrator in > the sitepackage as override) ? TypoScript is FE context, which again bounds the next "config" to that request and hurts people or not having the option properly loaded in BE context or CLI or all the places people adding Extbase ORM. I would also abstain from having it as (PageTsConfig) or similar. Avoid all that loading related to a "page" and not having a "default" in some way.
  20. Garvin Hicking commented on patch set 3 01/23/26, 09:10

    Reply

    Sounds good to me actually. Much easier to access, and would even prevent us needing to create an event, if I understand correctly. I have this on my radar and would love to talk it over with Georg and you before feature freeze, and wrap this up.
  21. Josef Glatz commented on patch set 3 01/23/26, 19:11

    rst path needs to be adopted to /14.2/
  22. Garvin Hicking uploaded patch set 4 V -1 03/20/26, 15:07

    TCA Schema FTW
  23. Garvin Hicking commented on patch set 4 03/20/26, 15:09

    Reply

    I've implemented this as a TCA Schema Capability now. "works on my machine".

    Reply

    Should be better now with TCA-level configuration. The events themselves can be adjusted by custom listeners if needed.

    Reply

    Done
    I could need help on wrapping up the functional tests, anyone up for it?

    ExtbaseHistoryTracker.php:46 Reply

    Yes, this event worked better for me, it's now utilized in the updated version. Thanks!

    ExtbaseHistoryTrackerTest.php:36 Reply

    We should now be able to use the functional tests for this. I've added `todo` markers.
  24. core-ci voted on patch set 4 V -1 03/20/26, 15:18 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98306

  25. Garvin Hicking commented on patch set 5 V -1 03/20/26, 16:36

    Published edit on patch set 4.
  26. core-ci voted on patch set 5 V -1 03/20/26, 16:49 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98310

  27. Garvin Hicking commented on patch set 6 V -1 03/20/26, 17:06

    Published edit on patch set 5.
  28. core-ci voted on patch set 6 V -1 03/20/26, 17:16 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98317

  29. Sebastian Michaelsen commented on patch set 6 CR -1 03/20/26, 21:02

    ExtbaseHistoryTracker.php:106

    $object->_getCleanProperties() does still contain the old data at this point. We should use that to get the history right.
  30. Garvin Hicking commented on patch set 6 03/20/26, 21:43

    ExtbaseHistoryTracker.php:106 Reply

    That sounds excellent, I'm not deep enough into the states here. I'll check it out next week, won't have time this weekend. Thanks so much for pointing at this, I would love to have this better diffable.
  31. Garvin Hicking uploaded patch set 7 CR -1 V -1 03/21/26, 14:36

  32. Garvin Hicking marked it work in progress 03/21/26, 14:45

  33. Garvin Hicking commented on patch set 7 03/21/26, 14:46

    ExtbaseHistoryTracker.php:106 Reply

    Seems to work well. Will need to work on tests.
  34. core-ci voted on patch set 7 V -1 03/21/26, 14:50 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98350

  35. Sebastian Michaelsen uploaded patch set 8 V -1 03/21/26, 19:44

  36. core-ci voted on patch set 8 V -1 03/21/26, 19:54 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98358

  37. Sebastian Michaelsen uploaded patch set 9 V -1 03/21/26, 20:09

  38. core-ci voted on patch set 9 V +1 03/21/26, 20:18 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98359

  39. Sebastian Michaelsen uploaded patch set 10 V +1 03/22/26, 07:55

  40. Sebastian Michaelsen commented on patch set 10 03/22/26, 07:59

    ExtbaseHistoryTracker.php:106 Reply

    Done
  41. Sebastian Michaelsen commented on patch set 10 03/22/26, 08:02

    ExtbaseHistoryTrackerTest.php:36 Reply

    Done
  42. Garvin Hicking marked it ready for review 03/22/26, 08:04

  43. core-ci voted on patch set 10 V -1 03/22/26, 08:08 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98377

  44. Garvin Hicking commented on patch set 10 CR +1 V +1 03/22/26, 08:24

    Thanks Sebastian! Failing tests are unrelated flakey e2e. I'll spend the rest of the day kicking them, I guess.
  45. core-ci commented on patch set 10 03/22/26, 08:30 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98377

  46. Torben Hansen commented on patch set 10 03/22/26, 08:42

    I tested the feature and I think it is a great improvement 👍 I have however concerns to enable it by default. 1. Possible Database Denial of Service If an attacker finds a frontend plugin that persists data (e.g., a registration form or a comment section built with Extbase), they can script thousands of requests resulting in sys_history table being flooded with data. Especially for extbase entities with many properties and relations, history_data field can contain several KB of data. This may lead to Disk Space Exhaustion if persistence operations are not limited. 2. Privacy and GDPR Compliance Issues If an Extbase model handles personal user data, this data is now duplicated into sys_history. If a user requests their data to be deleted, a developer must now also take care of removing data in sys_history (either manually or by using a cleanup scheduler task) 3. Log pollution In the TYPO3 Backend, the record history / undo module is primary designed to help editors to revert changes made by editors. If an extbase entitiy is updated multiple times a day in frontend context, it gets harder for editors in backend context to identify and revert changes made by editors. 4. Possible logging of sensitive information Depending on the type of application created with extbase, it might now be possible, that sensitive information is logged to sys_history table. We should somehow (maybe in docs) make deveopers aware of, that the feature might lead to logging of sensitive data. Additionally I think it might be good to provide an example on how to prevent logging of sensitive data. I therefore think, this features should be opt in, so developers/site administrators must enable it manually for each table.
  1. core-ci voted on patch set 10 V +1 03/22/26, 11:04 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98377

  2. Garvin Hicking commented on patch set 10 03/22/26, 11:05

    Reply

    1) We have that ratelimiting now ;) I think the DOS issue would be the same, the concept of flooding could be applied also without history tracking. 2+4) That's a valid concern. We could add thst to the changelog/docs? 3) But you normally filter data on the recordlist module? Having said that: I would be fine to make it opt in. But then we need another global default switch for it. Possibly DefaultConfiguration with a feature flag. What do you think? We could leave that off by default for now and collect feedback. Thanks for the helpful feedback!
  3. Torben Hansen commented on patch set 10 03/22/26, 11:26

    Reply

    1) Missing rate limit is often claimed by pentesters and every applications (mass) persisting data should have it. What I meant here is, that if we by default have the feature on, it would be more easy for attackers to flood a TYPO3 core table than it currently is. 2+4) Changelog is good. 3) AFAIK the record history has no filtering options. With opt in I mean, that the default value for `enableHistoryTracking` should be `false`. Extension authors can enable it in TCA and site owners can override it via TCA overrides if needed.
  4. Garvin Hicking commented on patch set 10 03/22/26, 11:50

    Reply

    1) was more of a wink. I do understand the concern. I think we should not have it default to off. We should default to "null" then, and a feature flag can then be evaluated in the listener (not the TCA Schemacapability API to prevent a feature flag dependency I guess). I would say the instance must easily decide if history tracking is enables. Not authors of ext:news or ext:powermail or so. So the feature flag should IMO override the TCA setting then, or an opt-in makes less sense, because I would have to take over controls of extensions regularly. Managing the TCA setting for many tables makes it quite ovearhead-y. Thoughts?
  5. Torben Hansen commented on patch set 10 03/22/26, 12:01

    Reply

    I think, if the feature is not enabled by default and can be enabled globally via feature flag, this is ok. Extension authors/site owners should still be able to manually control the setting per table (e.g. for tables, where history tracking is not required).
  6. Garvin Hicking commented on patch set 10 CR 0 V 0 03/22/26, 12:09

    Reply

    Yes. I would turn the feature flag off by default due to your feedback. So: - ff off (default) TCA undefined (default): off - ff off, TCA "true" => on - ff off, TCA "false@ => on - ff on, TCA undefined: on - ff on, TCA "true" => on - ff on, TCA "false" => off ?
  7. Garvin Hicking commented on patch set 10 03/22/26, 12:10

    Reply

    Cannot edit made a boo: ff off (default) TCA undefined (default): off ff off, TCA "true" => on ff off, TCA "false" => off ff on, TCA undefined: on ff on, TCA "true" => on ff on, TCA "false" => off
  8. Torben Hansen commented on patch set 10 03/22/26, 12:56

    Reply

    I suggest to change this: ff off, TCA "true" => off
  9. Garvin Hicking commented on patch set 10 03/22/26, 13:21

    Reply

    Ack. I'll ping @benni@typo3.org for feedback before getting on this. I might get feedback that I'm overcomplicating things so maybe we'll defer to just "TCA=false" by default. ;)
  10. Garvin Hicking uploaded patch set 11 V +1 03/23/26, 07:37

    feature flag all the things
  11. Garvin Hicking commented on patch set 11 CR +1 V +1 03/23/26, 07:41

    Reply

    @derhansen@gmail.com Please check out the recent patch set which contains the Feature Flag gating (as authoritative "total opt out") and changes to the ReST to mention security, privacy and flooding concerns. Hope that's fine!
  12. Sebastian Michaelsen commented on patch set 11 CR +1 03/23/26, 07:47

    I think I would have gone for just one CSV file indicating "no history written" and reused it in all cases where no history should be written. But it's certainly not wrong to go create distinct CSV files for each test case.
  13. core-ci voted on patch set 11 V +1 03/23/26, 07:49 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98423

  14. Josef Glatz commented on patch set 11 03/23/26, 07:54

    Feature-107289-AutomaticHistoryTrackingForExtbaseEntities.rst:89

    I think GDPR is totally fine, since all of our docs are english? Should we put DE here?
  15. Garvin Hicking commented on patch set 11 03/23/26, 08:25

    Reply

    I thought about this too, but then wanted to ensure no cross-bleed between tests could occur (in the future).

    Feature-107289-AutomaticHistoryTrackingForExtbaseEntities.rst:89 Reply

    I think we have a large DE-based audience and I would like to use this trigger keyword here.
  16. Torben Hansen commented on patch set 11 03/23/26, 13:58

    Feature-107289-AutomaticHistoryTrackingForExtbaseEntities.rst:97

    The impact now only applies, when the feature flag is enabled. I suggest to change the section, that it now is possible to track changes of extbase entities when the feature flag is enabled.
  17. Torben Hansen commented on patch set 11 03/23/26, 14:02

    ExtbaseHistoryTracker.php:91

    Lets move this at the very first beginning of the function. No need to resolte dataMap or get schema, when the feature is disabled.
  18. Garvin Hicking uploaded patch set 12 CR +1 V +1 03/23/26, 14:45

  19. Garvin Hicking commented on patch set 12 CR +1 V +1 03/23/26, 14:46

    Reply

    Done

    Feature-107289-AutomaticHistoryTrackingForExtbaseEntities.rst:97 Reply

    Done

    ExtbaseHistoryTracker.php:91 Reply

    Done
  20. core-ci voted on patch set 12 V +1 03/23/26, 14:54 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/98492

  21. Sebastian Michaelsen voted on patch set 12 CR +1 03/23/26, 17:46

  22. Torben Hansen voted on patch set 12 CR +1 03/23/26, 18:15

  23. Georg Ringer voted on patch set 12 CR +1 V +1 03/23/26, 18:32

  24. Georg Ringer voted on patch set 12 CR +2 V +2 03/23/26, 18:32

  25. Georg Ringer merged patch set 13 03/23/26, 18:32