Review main Ready for review

96745[BUGFIX] DataHandler: Streamline `null` handling

Daniel Windloff CR 0 V +1 Nightmare +2836 −50 updated 11/20/26, 11:29

Commit message

DataHandler allows all values to be set to `null` even
if the TCA option is not allowed for this field type.
This `null`-Handling has been streamlined:

The following TCA field types allow `null` without
further options:
- passthrough
- user

For the following TCA field types the
option `nullable` is allowed:
- input
- color
- country
- email
- text
- number
- password
- link
- datetime (depending on the format)

The following TCA field types are not nullable:
- uuid
- imageManipulation
- check
- radio
- slug
- language
- folder
- flex
- file
- inline
- category
- select
- group
- json

The field type text now respect `nullable`.
The field type country now saves null or
an empty string depending on `nullable`.

A TCA migration will automatically remove and
log any usage of nullable for field types
without the option

Test this change

Run a command in your clone of the TYPO3 Core. Then try what the change does and vote on Verified in Gerrit. How to review a patch

Check out patch set 11
git fetch https://review.typo3.org/Packages/TYPO3.CMS refs/changes/45/96745/11 && git checkout FETCH_HEAD
Cherry-pick onto main
git fetch https://review.typo3.org/Packages/TYPO3.CMS refs/changes/45/96745/11 && git cherry-pick FETCH_HEAD

Review

  • Code review CR 0 Needs a vote
  • Verified V +1 Needs a vote
    • Oliver Klee +1
    • core-ci +1 CI

Merge conflicts 32

Issues 1

Files 12

Review log 2839

Show 914 earlier messages
  1. Daniel Windloff uploaded patch set 1 10/16/26, 05:45

  2. core-ci voted on patch set 1 V -1 10/16/26, 05:58 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/111312

  3. Daniel Windloff uploaded patch set 2 V -1 10/16/26, 06:37

  4. core-ci voted on patch set 2 V -1 10/16/26, 06:46 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/111315

  5. Benni Mack wrote 10/16/26, 06:47

    Hashtag added: DataHandler
  6. Benni Mack wrote 10/16/26, 06:47

    Topic set to DataHandler
  7. Daniel Windloff uploaded patch set 3 V -1 10/17/26, 05:16

  8. Daniel Windloff uploaded patch set 4 10/17/26, 05:29

  9. core-ci voted on patch set 3 V +1 10/17/26, 05:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/111424

  10. core-ci voted on patch set 4 V +1 10/17/26, 05:39 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/111425

  11. Oliver Klee voted on patch set 4 V +1 10/17/26, 17:07

  12. Alexander Schnitzler uploaded patch set 5 V +1 10/18/26, 14:49

  13. Alexander Schnitzler commented on patch set 5 V -1 10/18/26, 15:01

    I would prefer to have more tests here covering all the different changes. The image manipulation regression slipped right through, there is possibly more broken without test coverage.

    DataHandler.php:1369

    The match result is cast with `(array)`, so this yields `[0 => '...']` and the value is dropped. Should be `['value' => (string)$value]`. See `imageManipulationValueIsStored()` in PS5 please.
  14. core-ci voted on patch set 5 V -1 10/18/26, 15:04 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/111535

  1. Daniel Windloff commented on patch set 5 CR -1 V -1 10/19/26, 06:06

    I will take care of it this evening.
  2. Daniel Windloff uploaded patch set 6 CR -1 V -1 10/19/26, 20:01

  3. core-ci voted on patch set 6 V -1 10/19/26, 20:16 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/111709

  4. Daniel Windloff commented on patch set 6 10/20/26, 04:57

    Reply

    Done

    Reply

    Yes, that’s correct. Added some more testcases. There are additional issues regarding minimum date limits when using `datetime` with `NOT NULL` across different database engines as the default empty value can't be saved everywhere. I hadn’t anticipated that; I actually thought this was a pre-patch intended to help address known issues in other `checkValueFor*` methods.

    DataHandler.php:1369 Reply

    Done
  5. Alexander Schnitzler commented on patch set 6 CR -1 10/20/26, 07:48

    Thanks for the extensive test coverage — I verified that `SetValuesToNullTest` fails in 19 cases against the old DataHandler, so it really pins the change. A few points: 1. CI is red because of the data set `type "datetime_notnull_date" without nullable true`. It expects `'0000-00-00'` to be stored, but MariaDB (strict mode) and Postgres reject that value in a `NOT NULL` date column, so the insert fails there and the record is never created. Only SQLite accepts it. The datetime code path isn't changed by this patch, and the handling of empty dates is being worked on separately (#109572, https://review.typo3.org/c/Packages/TYPO3.CMS/+/95162). Please remove this data set from `setValuesToNullProvider()`. While you're at it: the fixture default `'default' => '1970-00-00'` of `datetime_notnull_date` is not a valid date, I'd drop it too. 2. In `RecordFieldTransformerTest::handlesNullFieldValueForCountry()` the patch deletes the assertions after the first `transformField()` call. Part of them checks the non-nullable field `typo3tests_contentelementb_country`, which this patch doesn't change at all, so that coverage is lost for no reason. Please keep them and only change the expectation for the nullable field: `typo3tests_contentelementb_country_nullable` now resolves to `''` instead of `null`, so `self::assertNull(...)` becomes `self::assertSame('', ...)` for both the `transformField()` result and `$resolvedRecord->get(...)`. 3. In `setValuesProvider()`, the data set `case "imageManipulation_null"` uses `'field' => 'imageManipulation_notnull'`, which is the same field as the data set before it. So the field `imageManipulation_null` is never tested with a crop value. Please change it to `'field' => 'imageManipulation_null'`. 4. `checkValueForFlex()` now returns `['value' => '']` instead of `[]` when `$field === ''` (that's the case of a flex field nested inside a FlexForm). The caller in the flex traversal only overwrites the stored value if `isset($res['value'])`. Before, `[]` left the submitted value untouched; now it is replaced by `''`. Is that change intended? If yes, please explain it in the commit message and add a test. If not, please go back to `return [];` (all tests stay green with it). 5. The RST describes the new rules, but not what changes for existing installations. Two things should be mentioned there: - `type=text` without `nullable` used to store `NULL` when `null` was passed, now it stores `''`. That's why the impexp fixtures and `DefaultValuesTest` had to be adapted, and it can affect extensions too. - `'nullable' => true` on a type that doesn't support it (e.g. `select`, `group`, `check`, `country`) used to make DataHandler store `NULL`, now it is ignored. Integrators can remove the option; for a select field, an item with `'value' => null` is the supported way to get `NULL`. Nits: - The commit message says `isNullable()` "now always returns `null`", but it returns `false`. - In the RST, `:php-short:` is meant for PHP class names. For `null`, `passthrough` and `'nullable' => true` please use double backticks instead. - RST wording: "allow to persist" → "allow persisting", "does not persist" → "do not persist".
  6. Daniel Windloff commented on patch set 6 10/23/26, 07:24

    Reply

    Thank you for the detailed review. I will provide a new patch set soon.
  7. Daniel Windloff uploaded patch set 7 CR -1 V -1 10/24/26, 03:56

  8. core-ci voted on patch set 7 V +1 10/24/26, 04:04 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/112379

  9. Daniel Windloff uploaded patch set 8 V +1 10/24/26, 04:31

  10. Daniel Windloff commented on patch set 8 10/24/26, 04:34

    Reply Unresolved

    1. Yes, format date without nullable defaults to values that are out of range. I had to set a default date there; otherwise, the record cannot be saved. This could be solved in another bug fix. 2. If the test for null in the country should stay, nullable must be allowed. Therefore, I adjusted the handling so that nullable is allowed. As the validation of the type country seems to fail, I could not add tests for the case where invalid countries are selected and the fallback is null or an empty string, depending on the nullable config. 3. Done 4. Done, reset to former behaviour. 5. I changed the description. value => null for select was/is broken before and wasn’t supported, as FormEngine / DataHandler hasn’t implemented a way to differentiate between empty string and null.
  11. core-ci voted on patch set 8 V +1 10/24/26, 04:39 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/112385

  12. Daniel Windloff commented on patch set 8 CR -1 10/24/26, 07:24

    uuid is wrong
  13. Daniel Windloff marked it work in progress 10/24/26, 11:40

  14. Daniel Windloff commented on patch set 8 10/24/26, 11:41

    This change is ready for review.
    It is better to have other bugfixes first.
  15. Daniel Windloff marked it work in progress 10/24/26, 11:42

  16. Daniel Windloff uploaded patch set 9 CR -1 V +1 11/07/26, 07:43

  17. core-ci voted on patch set 9 V +1 11/07/26, 07:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/113561

  18. Daniel Windloff uploaded patch set 10 V +1 11/07/26, 08:57

  19. core-ci voted on patch set 10 V +1 11/07/26, 09:06 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/113562

  20. Daniel Windloff commented on patch set 10 11/07/26, 09:10

    This change is ready for review.
    - Rebased after country and uuid changes are in - Changed behavior of "country" - Added TCA migration to carify what has been changed
  21. Daniel Windloff commented on patch set 10 11/07/26, 09:11

    Reply

    Done

    Reply

    Done

    Reply

    Done
  22. Oliver Klee voted on patch set 10 V +1 11/07/26, 13:18

  23. Daniel Windloff uploaded patch set 11 V +1 11/20/26, 11:08

  24. core-ci voted on patch set 11 V +1 11/20/26, 11:14 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/115070

  25. Oliver Klee voted on patch set 11 V +1 11/20/26, 11:29