Review main Merged

73130[BUGFIX] Stabilize unserialization of scheduler tasks

Nikita Hovratov CR +2 V +2 Hard +399 −67 updated 02/05/22, 10:10

Commit message

The unserialization of scheduler tasks was very
error-prone, due to a lack of exception handling.

To stabilize and unify the unserialization task, a
dedicated method is added, which handles all kinds
of errors, that might happen. It will return a valid
Task or throw an exception with added information
about the failures.

Review

  • Code review CR +2 Done
    • Christian Kuhn +2
  • Verified V +2 Done
    • Christian Kuhn +2
    • core-ci +1 CI

Issues 2

Files 11

Review log 3240

Show 1115 earlier messages
  1. Nikita Hovratov uploaded patch set 1 01/23/22, 00:04

  2. core-ci voted on patch set 1 V -1 01/23/22, 00:20 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22422

  3. Georg Ringer commented on patch set 1 01/23/22, 09:39

    should we also add some more info for the user which covers what to do: delete it and readd it?
  4. Christian Kuhn commented on patch set 1 CR -1 01/23/22, 09:53

    SchedulerModuleController.php:688

    please don't introduce a third notification thing here: scheduler has flash messages already and BU->writelog at some places. in this case it's enough to simply show the according task in the 'missing classes' section. that section shows a header text that the class is broken already.

    TaskList.html:408

    skip all that and use the 'missing classes' display. maybe slightly adapt the wording of that section.
  5. Christian Kuhn commented on patch set 1 01/23/22, 11:12

    Ok, here is the thing: The fact that tasks are serialized to DB in the first place is a central misconception in the scheduler. This leads to all sorts of problems - for instance when tasks need injects, you have to take care of __wakeUp(). The fact that 'additionalFields' are properties of tasks is even more awful: It leads to the situation that 'extisting' tasks fail upon unserialization when adding / removing additional fields. We will eventually have v12 changes taking care of a broader ext:scheduler re-design that will fix these things as well. However, for now, I'd say we should do the following: Extract the unserialize call of 'serialized_task_object' string to a helper method in scheduler class. This one should perform the unserialization (including an @unserialize() call, btw), and then check the instance. It should catch possible exceptions and return ?AbstractTask. Consuming code should check for null. The list display should check null and move the task to "missingClasses" (which could be renamed a bit), rephrasing the message to some more general message like "This scheduled task is broken for some reason, please delete and re-create it". Note it's also possible to "edit" a broken task: Add a task, edit it, set a shortcut to that edit view, then destroy the task (for instance by removing a char from the serialized string in DB), then use the shortcut to edit that one again -> boom. So, let's have one method that checks this stuff and adapt existing unserialize calls of that string to use that method. Note there is a second serialized string in db ('serialized_executions'), which is awful as well, but more robust - the latter one does not need to be touched, I'd say.
  6. Nikita Hovratov uploaded patch set 2 01/23/22, 14:00

  7. core-ci voted on patch set 2 V -1 01/23/22, 14:16 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22432

  8. Nikita Hovratov uploaded patch set 3 01/23/22, 14:24

  9. core-ci voted on patch set 3 V +1 01/23/22, 14:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22434

  10. Nikita Hovratov uploaded patch set 4 01/23/22, 17:00

  11. Nikita Hovratov commented on patch set 4 01/23/22, 17:10

    SchedulerModuleController.php:688 Reply

    Done

    TaskList.html:408 Reply

    Done
  12. core-ci voted on patch set 4 V +1 01/23/22, 17:11 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22436

  13. Oliver Hader commented on patch set 4 01/24/22, 11:45

    In the long run PHP `unserialize` should be dropped here... What about moving the new functionality to a new `SchedulerUnserializer` to have things encapsulated. Besides that, I feel we should extend the test cases a bit to check the good and bad cases (spoiler: there are currently no tests for that particular scope)...

    Scheduler.php:539

    `[^"]+` - to avoid the greedy operator and keep boundaries... I know, the code was like for a long time...
  14. Oliver Hader marked it work in progress 01/24/22, 14:03

  15. Oliver Hader uploaded patch set 5 01/24/22, 14:20

  1. Oliver Hader commented on patch set 5 01/24/22, 14:22

    Extracted TaskSerializer - which allows to have dedicated tests → I'll be working on them now
  2. core-ci voted on patch set 5 V +1 01/24/22, 14:37 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22465

  3. Nikita Hovratov commented on patch set 5 01/24/22, 15:20

    Reply

    Thanks for picking this up, Oliver!
  4. Oliver Hader uploaded patch set 6 01/24/22, 15:56

  5. Oliver Hader commented on patch set 6 V +1 CR +1 01/24/22, 15:59

    This change is ready for review.
    I made the TaskSerialization aspects a little bit more explicit, separated between resolving class-name of existing task and extracting from serialized data. Besides that, test cases have been added.
  6. Oliver Hader commented on patch set 6 01/24/22, 16:03

    I fiddled around with serializations in my local database, missing classes are really a blocker. Fine for current main and 11.5 branch from my side. As Christian pointed out already, `unserialize` on whole class instances is a misconception in general and target to be changed in TYPO3 v12 - but that's a different story.
  7. Oliver Hader commented on patch set 6 01/24/22, 16:04

    Reply

    *missing → invalid classes like `AbstractWriter` which cannot be deserialized
  8. Oliver Hader commented on patch set 6 01/24/22, 16:06

    Scheduler.php:539 Reply

    Done
  9. core-ci voted on patch set 6 V +1 01/24/22, 16:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22469

  10. Benjamin Franzke commented on patch set 6 01/24/22, 16:50

    The patch isn't easy to cherry-pick into 11.5, there are many conflicts because of https://review.typo3.org/c/Packages/TYPO3.CMS/+/73051 Does anyone want to take the work to backport this, or should we skip 11.5?
  11. Christian Kuhn commented on patch set 6 CR -1 01/24/22, 16:58

    i'll review this and will take care of the backport.

    Reply

    i think we should basically 'hack it somehow' in v11 due to my massive v12 refactoring of that class in v12. georg has a patch version that could be targeted for v11.
  12. Nikita Hovratov commented on patch set 6 01/24/22, 19:07

    SchedulerModuleController.php:693

    $taskClass === null and !class_exists($taskClass) is unreachable. We are ensured that $taskObject is an existing class, which extends AbstractTask at this point. Scheduler->isValidTaskObject scans additionally whether $task->getExecution() is not an PHP incomplete class. So we should remove everything, besides line 697 here. We could also of course add class_exists to the catch part, to overrule the 'Class is not an AbstractTask' message, to give the user a better error message.
  13. Oliver Hader uploaded patch set 7 01/25/22, 07:55

  14. Oliver Hader commented on patch set 6 V +1 CR +1 01/25/22, 07:55

    SchedulerModuleController.php:693 Reply

    Good catch, I missed the `$task->getExecution()` part in `isValidTaskObject($task)`
  15. Oliver Hader voted on patch set 7 V +1 CR +1 01/25/22, 07:56

  16. core-ci voted on patch set 7 V +1 01/25/22, 08:08 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22501

  17. Nikita Hovratov commented on patch set 7 V +1 01/25/22, 08:26

    Leaving Code-Review +1 out for @lolli to look at.
  18. Christian Kuhn commented on patch set 7 V +1 CR +1 01/25/22, 09:59

    this reads clean and well now! unsure about backport, though ... could become a bit tricky in the controller.
  19. Nikita Hovratov voted on patch set 7 CR +1 01/25/22, 13:11

  20. Christian Kuhn uploaded patch set 8 02/05/22, 09:43

  21. core-ci voted on patch set 8 V +1 02/05/22, 09:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/23050

  22. Christian Kuhn commented on patch set 8 V +2 CR +2 02/05/22, 10:03

    I will merge this now since i "only" rebased and resoloved some merge conflicts. For v11, I'll go with a more straight approach as georg suggested with https://review.typo3.org/c/Packages/TYPO3.CMS/+/65995
  23. Christian Kuhn merged patch set 9 02/05/22, 10:04

  24. Christian Kuhn commented on patch set 9 02/05/22, 10:09

  25. Christian Kuhn commented on patch set 9 02/05/22, 10:10