Review main Merged

84902[TASK] Use ISO8601 dates in FormEngine datetime handling

Andreas Kienast CR +2 V +2 Hard +200 −320 updated 07/15/24, 12:41

Commit message

Historically, FormEngine UI mangles dates from one format to another and
feeds the result back to the date picker, crossing fingers nothing goes
wrong. This bites hard when it comes to non-standard date formats,
basically everything but `d-m-Y`.

To lay the groundwork on tackling the date format issue, FormEngine now
always works with ISO8601 dates internally, allowing to get rid of some
helper methods.

Review

  • Code review CR +2 Done
    • Andreas Kienast +2
    • Benjamin Franzke +1
    • Georg Ringer +1
    • Benni Mack +1
  • Verified V +2 Done
    • Andreas Kienast +2
    • Benjamin Franzke +1
    • Georg Ringer +1
    • Benni Mack +1
    • core-ci +1 CI

Issues 1

Files 18

Review log 5886

Show 3761 earlier messages
  1. Andreas Kienast uploaded patch set 1 06/25/24, 13:13

  2. core-ci voted on patch set 1 V -1 06/25/24, 13:20 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60297

  3. Andreas Kienast uploaded patch set 2 V -1 06/25/24, 13:46

  4. core-ci voted on patch set 2 V -1 06/25/24, 13:56 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60305

  5. Andreas Kienast uploaded patch set 3 V -1 07/02/24, 13:35

  6. core-ci voted on patch set 3 V -1 07/02/24, 13:41 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60842

  7. Andreas Kienast uploaded patch set 4 V -1 07/02/24, 14:28

  8. core-ci voted on patch set 4 V -1 07/02/24, 14:35 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60846

  9. Andreas Kienast uploaded patch set 5 V -1 07/02/24, 14:59

  10. core-ci voted on patch set 5 V -1 07/02/24, 15:04 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60849

  11. Andreas Kienast uploaded patch set 6 V -1 07/03/24, 12:23

  12. core-ci voted on patch set 6 V -1 07/03/24, 12:29 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60907

  13. Andreas Kienast uploaded patch set 7 V -1 07/03/24, 15:27

  14. core-ci voted on patch set 7 V -1 07/03/24, 15:36 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60912

  15. Andreas Kienast uploaded patch set 8 V -1 07/04/24, 15:25

  16. core-ci voted on patch set 8 V -1 07/04/24, 15:31 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60976

  17. Andreas Kienast uploaded patch set 9 V -1 07/04/24, 15:45

  18. core-ci voted on patch set 9 V -1 07/04/24, 15:51 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60978

  19. Andreas Kienast voted on patch set 9 CR -1 07/04/24, 16:18

  20. Andreas Kienast uploaded patch set 10 CR -1 V -1 07/04/24, 17:51

  21. Garvin Hicking commented on patch set 10 07/06/24, 08:34

    Just a short note/question, because I believe you addressed this. In https://forge.typo3.org/issues/91508 it came up again that due to our initialization of the flatpickr with "dd-MM-yyyy" no dates with 3 digits can be handled (due to "yyyy" requiring four). Using the ISO date here, and trying to make sense of this complex code should fix the issue too, right? (Thanks for your efforts in this matter, much appreciated and valued!)
  22. Andreas Kienast commented on patch set 10 07/08/24, 06:41

    Reply

    Honestly, I have no idea whether this patch fixes the issue as well.
  23. Garvin Hicking commented on patch set 10 07/08/24, 07:26

    Reply

    I just checked; it properly now uses "0999" and is able to pick dates then. Yay, you! :)
  24. Andreas Kienast uploaded patch set 11 07/08/24, 09:13

  25. core-ci voted on patch set 11 V -1 07/08/24, 09:20 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61170

  26. Andreas Kienast uploaded patch set 12 V -1 07/08/24, 12:41

  27. Andreas Kienast commented on patch set 12 V -1 07/08/24, 12:43

    Scheduler task creation is broken now (╯°□°)╯︵ ┻━┻
  28. core-ci voted on patch set 12 V -1 07/08/24, 12:47 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61172

  29. Andreas Kienast uploaded patch set 13 V -1 07/08/24, 13:42

  30. core-ci voted on patch set 13 V -1 07/08/24, 13:50 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61175

  31. Andreas Kienast uploaded patch set 14 V -1 07/08/24, 14:56

  32. core-ci voted on patch set 14 V -1 07/08/24, 15:04 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61177

  33. Andreas Kienast uploaded patch set 15 07/08/24, 15:25

    Copied Votes: * Verified-1 (copy condition: "**changekind:NO_CHANGE**")
  34. core-ci voted on patch set 15 V -1 07/08/24, 15:30 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61180

  35. Andreas Kienast uploaded patch set 16 V -1 07/09/24, 11:42

  36. core-ci voted on patch set 16 V +1 07/09/24, 11:49 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61233

  37. Andreas Kienast uploaded patch set 17 V +1 07/09/24, 13:33

  38. Benjamin Franzke commented on patch set 16 07/09/24, 13:38

    date-time-picker.ts:141

    Hm, true for DateTimeInterface::ATOM (and the broken DateTimeInterface::ISO8601 constant) do not support milliseconds, but PHP DateTime can handle milliseconds: ``` const ISO8601U = "Y-m-d\TH:i:s.uO"; var_dump(DateTime::createFromFormat(ISO8601U, '2024-07-07T15:30:00.123Z')); object(DateTime)#1 (3) { ["date"]=> string(26) "2024-07-07 15:30:00.123000" ["timezone_type"]=> int(2) ["timezone"]=> string(1) "Z" } ``` Also see https://www.php.net/manual/en/datetime.format.php > **u** Microseconds. Note that date() will always generate 000000 since it takes an int parameter, whereas DateTime::format() does support microseconds if DateTime was created with microseconds. That means this comment is not true. If there's another argument to not use milliseconds, we'd need to name that (like complication with existing stuff that generated/sends ISO8601 without milliseconds?) or because of fears with "the DataHandler"?... IMHO we should try to make it right and migrate all cases to generate ISO8601 with milliseconds and desired by ECMAScript: https://tc39.es/ecma262/multipage/numbers-and-dates.html#sec-date-time-string-format > ECMAScript defines a string interchange format for date-times based upon a simplification of the ISO 8601 calendar date extended format. The format is as follows: **YYYY-MM-DDTHH:mm:ss.sssZ**

    DatetimeElement.php:161

    I'm not sure this is still "ok", DatabaseRowDateTimeFields will create ISO8601, yes, but timezone-dependant ones. Shouldn't we normalized them to be Z-based here? Otherwise we'll have `00:00Z` for integer based fields and stuff like `00:00+01:00` for native datetime fields.
  39. core-ci voted on patch set 17 V +1 07/09/24, 13:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61238

  40. Andreas Kienast uploaded patch set 18 V +1 07/09/24, 14:30

  41. Andreas Kienast commented on patch set 18 CR -2 07/09/24, 14:35

  42. core-ci voted on patch set 18 V +1 07/09/24, 14:36 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61244

  43. Andreas Kienast uploaded patch set 19 V +1 07/09/24, 19:48

    Copied Votes: * Code-Review-2 (copy condition: "changekind:NO_CHANGE OR **is:MIN**")
  44. Andreas Kienast voted on patch set 19 CR 0 07/09/24, 19:49

  45. core-ci voted on patch set 19 V +1 07/09/24, 19:53 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61255

  46. Andreas Kienast commented on patch set 19 07/09/24, 20:36

    DatetimeElement.php:161 Reply

    In general I think it'd make sense to have a unified format. However, I'd like to keep that out of this patch if that is fine for you.
  47. Andreas Kienast uploaded patch set 20 V +1 07/09/24, 20:37

  48. core-ci voted on patch set 20 V +1 07/09/24, 20:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61258

  49. Andreas Kienast uploaded patch set 21 V +1 07/10/24, 07:22

  50. core-ci voted on patch set 21 V +1 07/10/24, 07:27 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61288

  51. Andreas Kienast commented on patch set 21 V -1 07/10/24, 07:41

    belog filtering is broken
  52. Andreas Kienast uploaded patch set 22 V +1 07/10/24, 08:00

  53. core-ci voted on patch set 22 V +1 07/10/24, 08:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61292

  54. Andreas Kienast uploaded patch set 23 V +1 07/10/24, 10:36

  55. core-ci voted on patch set 23 V +1 07/10/24, 10:41 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61306

  56. Andreas Kienast uploaded patch set 24 V +1 07/10/24, 10:44

  57. core-ci voted on patch set 24 V +1 07/10/24, 10:52 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61308

  58. Benjamin Franzke uploaded patch set 25 V +1 07/11/24, 10:07

  59. Benjamin Franzke commented on patch set 25 07/11/24, 10:07

    date-time-picker.ts:141 Reply

    Done

    DatetimeElement.php:161 Reply

    Acknowledged
  60. core-ci voted on patch set 25 V +1 07/11/24, 10:15 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61370

  61. Benjamin Franzke commented on patch set 25 07/11/24, 10:28

    date-time-picker.ts:221

    Which bug? I can not find a bug without using `showMonths = 12`
  1. Andreas Kienast commented on patch set 25 07/11/24, 10:33

    date-time-picker.ts:221 Reply

    Testable with `inputdatetime_5` (also discovered by AC tests): * Focus field * Flush content * Enter an arbitrary time, e.g. "13:30" * Press TAB key * JS error: `TypeError: Cannot set properties of undefined (setting 'tabIndex')` flatpickr somehow thinks it's a great idea to update a month selector even if there is no month selector at all.
  2. Benjamin Franzke uploaded patch set 26 V +1 07/11/24, 13:31

  3. Benjamin Franzke commented on patch set 26 07/11/24, 13:31

    date-time-picker.ts:221 Reply

    Ack, added a comment, and changed to directly configure the relevant option `monthSelectorType`, as mentioned on https://flatpickr.js.org/options/ > If showMonths has a value greater than 1, the month is always shown as static.

    ElementsBasicInputDateCest.php:126

    We now got milliseconds here, and I guess that's because Formengine now uses luxon toIso() (which will generate milliseconds) as intermediate format. For `date*` fields this intermediate format is formatted somehwere™️ before being added to the hidden field, while `time*` fields are not post-processes. Otherwise we have cases with and without milliseconds send to the server (which currently works, but I think we should try to streamline that). Fixed by suppressing milliseconds in the time/timesec processor.
  4. core-ci voted on patch set 26 V +1 07/11/24, 13:36 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61391

  5. Simon Schaufelberger commented on patch set 26 07/12/24, 00:03

    Do you want to rebase?
  6. Andreas Kienast uploaded patch set 27 V +1 07/12/24, 06:31

  7. Andreas Kienast commented on patch set 27 07/12/24, 06:31

    Reply

    Done
  8. core-ci voted on patch set 27 V +1 07/12/24, 06:37 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61451

  9. Benjamin Franzke commented on patch set 27 07/12/24, 07:08

    date-time-picker.ts:139

    I kinda "dislike" that this is a very implicit hack that crops-away the (admittedly intermediate) locale-offset. I understand that this is desired (as we use js-date localtime as container for UTC, as flatpickr only supporse localtime Date, not UTC 🤦). I know it looks "worse", but I'm thinking about using an explicit approach like: ``` const dt = DateTime.fromJSDate(date); const iso = dt .setZone('utc') .plus(dt.offset * 60 * 1000) .toISO({ suppressMilliseconds: true }); ``` WDYT @a.fernandez@scripting-base.de ?
  10. Benjamin Franzke uploaded patch set 28 V +1 07/12/24, 08:22

  11. core-ci voted on patch set 28 V +1 07/12/24, 08:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61464

  12. Benjamin Franzke commented on patch set 28 CR +1 V +1 07/12/24, 12:25

    date-time-picker.ts:139 Reply

    Done
  13. Christian Kuhn commented on patch set 28 07/12/24, 13:35

    sure about v12? isn't this a bit too risky for v12?
  14. Andreas Kienast commented on patch set 28 07/12/24, 13:41

    Reply

    We need this as a preparation patch to fix broken FormEngine behavior if `SYS/ddmmyy` is configured to be e.g. `m/d/Y`.
  15. Benjamin Franzke commented on patch set 28 07/12/24, 14:17

    Reply

    I was very sceptical as well, but went through this patch a dozen times in order to ensure that this is good and does not introduce breaking changes. I'm confident that this can be merged.
  16. Georg Ringer voted on patch set 28 CR +1 V +1 07/12/24, 17:18

  17. Georg Ringer commented on patch set 28 07/12/24, 17:19

    is there a potential need for an important rst because extension authors might need to adopt code, e.g. when using own backend modules?
  18. Benjamin Franzke uploaded patch set 29 CR +1 V +1 07/12/24, 19:06

  19. core-ci voted on patch set 29 V +1 07/12/24, 19:15 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61506

  20. Benjamin Franzke commented on patch set 29 07/12/24, 19:15

  21. Benjamin Franzke commented on patch set 29 CR +1 V +1 07/15/24, 06:17

    Reply

  22. Benni Mack commented on patch set 29 CR +1 V +1 07/15/24, 12:02

    Thanks for taking care of that one!
  23. Georg Ringer voted on patch set 29 CR +1 V +1 07/15/24, 12:19

  24. Andreas Kienast voted on patch set 29 CR +2 V +2 07/15/24, 12:41

  25. Andreas Kienast merged patch set 30 07/15/24, 12:41