Review main Merged

88675[TASK] Migrate scheduler database to new storage format

Benni Mack CR +2 V +2 Hard +683 −161 updated 04/10/25, 10:44

Commit message

TYPO3's scheduler system extension was included in TYPO3 4.3,
based on the "gabriel" extension. The concepts back then
were amazing but have been around now for more than 15 years
in core.

A few flaws are the way the Tasks are registered, and
how the AbstractTask object is convoluted.

One major flaw is that the actual Task object is stored
in a serialized way completely.

This led to very very many workarounds in the past years
when Task classes were renamed, Type-safety added etc.

This has now been addressed by splitting up the actual
classnames or tasktypes (for CLI commands) into a new field
"tasktype" and the options / arguments / settings into a
new JSON-serialized "parameters" field, as well as
"execution_details" into a separate JSON field. These
fields are added to the existing `tx_scheduler_task`
database table.

An upgrade wizard handles the migration from
"serialized_task_object" to "tasktype" et al.

As a further change we could also thin out the Task object,
and make the Task object less convoluted with
regular object work.

This paves the way to
- introduce TCA for tx_scheduler_task (import/export)
- hopefully make the UI editable via FormEngine at some point
- use DI in any task object in the future
- Change the ugly additionalFieldProvider + task registration logic

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Garvin Hicking +1
    • Christian Kuhn +1
  • Verified V +2 Done
    • Benni Mack +2
    • Garvin Hicking +1
    • Christian Kuhn +1
    • core-ci +1 CI

Issues 1

Files 33

Show 13 more files

Review log 4768

Show 2843 earlier messages
  1. Benni Mack uploaded patch set 1 03/21/25, 17:45

  2. core-ci voted on patch set 1 V -1 03/21/25, 17:57 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75965

  3. Benni Mack uploaded patch set 2 V -1 03/21/25, 18:13

  4. Benni Mack uploaded patch set 3 03/21/25, 18:19

  5. core-ci voted on patch set 2 V -1 03/21/25, 18:21 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75966

  6. core-ci voted on patch set 3 V -1 03/21/25, 18:28 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75967

  7. Benni Mack uploaded patch set 4 V -1 03/21/25, 19:14

  8. core-ci voted on patch set 4 V +1 03/21/25, 19:23 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75972

  9. Benni Mack marked it work in progress 03/22/25, 12:35

  10. Benni Mack uploaded patch set 5 V +1 03/31/25, 15:59

  11. Benni Mack uploaded patch set 6 03/31/25, 16:01

  12. core-ci voted on patch set 5 V -1 03/31/25, 16:08 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76726

  13. core-ci voted on patch set 6 V -1 03/31/25, 16:10 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76727

  14. Benni Mack uploaded patch set 7 V -1 04/02/25, 06:20

  15. core-ci voted on patch set 7 V -1 04/02/25, 06:27 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76840

  16. Benni Mack uploaded patch set 8 V -1 04/03/25, 06:50

  17. core-ci voted on patch set 8 V -1 04/03/25, 07:00 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76924

  18. Benni Mack uploaded patch set 9 V -1 04/03/25, 10:25

  19. core-ci voted on patch set 9 V -1 04/03/25, 10:33 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76938

  20. Benni Mack uploaded patch set 10 V -1 04/03/25, 10:38

  21. core-ci voted on patch set 10 V +1 04/03/25, 10:44 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76942

  22. Christian Kuhn commented on patch set 10 04/08/25, 16:09

    SchedulerDatabaseStorageMigration.php:32

    we should move this wizard to ext:scheduler

    TaskSerializer.php:88

    do we really need the b/w layer in v14? couldn't we just say that running the upgrade wizard is mandatory in v14? we should expect people run the upgrade wizards when doing major upgrades. the task hydration is imho not API, so we should be able to do this in one go?
  23. Benni Mack uploaded patch set 11 V +1 04/08/25, 20:03

  24. core-ci voted on patch set 11 V -1 04/08/25, 20:11 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77238

  25. Benni Mack uploaded patch set 12 V -1 04/09/25, 13:57

  26. Benni Mack marked it ready for review 04/09/25, 13:57

  27. core-ci voted on patch set 12 V -1 04/09/25, 14:03 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77285

  28. Benni Mack uploaded patch set 13 V -1 04/09/25, 14:03

  29. Benni Mack commented on patch set 13 04/09/25, 14:04

    SchedulerDatabaseStorageMigration.php:32 Reply

    Done

    TaskSerializer.php:88 Reply

    Done
  30. core-ci voted on patch set 13 V +1 04/09/25, 14:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77286

  31. Christian Kuhn voted on patch set 13 CR +1 V +1 04/09/25, 17:19

  32. Stefan Bürk voted on patch set 13 CR +1 V +1 04/09/25, 17:59

  33. Garvin Hicking commented on patch set 13 04/09/25, 18:00

    Execution.php:299

    ```suggestion // The type cast is necessary as long as the migration (DB migration) exists, ```

    Execution.php:303

    Use a "todo" marker here?

    SchedulerDatabaseStorageMigration.php:46

    ```suggestion return 'Each Scheduler Task was previously stored in a serialized object format in the database. This update wizard migrates records of this type to a JSON-formatted storage in the database.'; ```
  34. Benni Mack commented on patch set 13 CR +2 V +2 04/09/25, 18:00

    Thank you, that's an internal milestone!
  35. Benni Mack voted on patch set 13 V 0 CR -1 04/09/25, 18:01

  36. Benni Mack uploaded patch set 14 CR +1 V +1 04/09/25, 18:03

  37. Stefan Bürk voted on patch set 14 CR +1 V +1 04/09/25, 18:07

  38. Christian Kuhn voted on patch set 14 CR +1 V +1 04/09/25, 18:07

  39. core-ci voted on patch set 14 V +1 04/09/25, 18:10 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77306

  40. Garvin Hicking commented on patch set 14 CR -1 04/09/25, 18:12

    reviewing

    Execution.php:299 Reply

    Done

    Execution.php:303 Reply

    Done

    SchedulerDatabaseStorageMigration.php:46 Reply

    Done
  41. Garvin Hicking uploaded patch set 15 CR +1 V +1 04/09/25, 19:39

  42. core-ci voted on patch set 15 V +1 04/09/25, 19:48 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77316

  43. Benni Mack commented on patch set 15 CR -1 04/09/25, 19:49

    needs more work
  1. Benni Mack marked it work in progress 04/09/25, 19:49

  2. Garvin Hicking commented on patch set 15 CR -1 04/09/25, 19:50

    (Still on it) Missing a few non-EXT:scheduler tasks: typo3/sysext/extensionmanager/Classes/Task/UpdateExtensionListTask.php typo3/sysext/linkvalidator/Classes/Task/ValidatorTask.php typo3/sysext/recycler/Classes/Task/CleanerTask.php typo3/sysext/reports/Classes/Task/SystemStatusUpdateTask.php For these the wizard upgrade doesn't populate 'parameters' properly and loses their state/content.
  3. Benni Mack uploaded patch set 16 CR -1 V +1 04/09/25, 21:05

  4. Benni Mack marked it ready for review 04/09/25, 21:05

  5. Benni Mack commented on patch set 16 04/09/25, 21:09

    Reply

    they do not need to be upgraded. The issue as you mentioned is solved now
  6. core-ci voted on patch set 16 V +1 04/09/25, 21:10 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77322

  7. Benni Mack rebased onto patch set 17 V +1 04/09/25, 22:43

  8. core-ci voted on patch set 17 V +1 04/09/25, 22:50 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77367

  9. Garvin Hicking uploaded patch set 18 V +1 04/10/25, 07:14

  10. Garvin Hicking commented on patch set 18 CR +1 V +1 04/10/25, 07:16

    Reply

    I'm thinking it would be good to keep a few non-upgraded ones around for some time to catch issues better.
    Fixed one last detail that came up with testing (dynamic property assignment due to a serialized configuration of an old task). I really enjoy this change, thank you for pushing through.
  11. Garvin Hicking commented on patch set 18 04/10/25, 07:16

    Reply

    Done
  12. core-ci voted on patch set 18 V -1 04/10/25, 07:23 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77390

  13. core-ci commented on patch set 18 04/10/25, 08:36 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77390

  14. Garvin Hicking commented on patch set 19 CR +1 V +1 04/10/25, 08:39

    Published edit on patch set 18.
  15. core-ci voted on patch set 19 V +1 04/10/25, 08:57 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77412

  16. Garvin Hicking voted on patch set 19 CR +1 V +1 04/10/25, 09:11

  17. Stefan Bürk voted on patch set 19 CR +1 V +1 04/10/25, 09:24

  18. Stefan Bürk commented on patch set 19 CR 0 V 0 04/10/25, 09:28

    AbstractTask.php:477

    ```suggestion if (property_exists($this, $key)) { ``` I'd say that this should be a proper property check .. isset() would return false even if the property exists but is set to "null" (nullable property)
  19. Stefan Bürk commented on patch set 20 CR +1 V +1 04/10/25, 09:36

    Published edit on patch set 19.
  20. core-ci voted on patch set 20 V +1 04/10/25, 09:43 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77419

  21. Stefan Bürk commented on patch set 20 04/10/25, 09:43

    AbstractTask.php:477 Reply

    My bad, was not on the latest changeset.
  22. Garvin Hicking voted on patch set 20 CR +1 V +1 04/10/25, 09:52

  23. Christian Kuhn voted on patch set 20 CR +1 V +1 04/10/25, 10:39

  24. Benni Mack commented on patch set 20 CR +2 V +2 04/10/25, 10:44

    Thank you, everybody
  25. Benni Mack merged patch set 21 04/10/25, 10:44