Review main Merged

88596[TASK] Upgrade to `doctrine/dbal:~4.3.3`

Stefan Bürk CR +2 V +2 Expert +289 −804 updated 10/06/25, 12:54

Commit message

This change upgrades doctrine/dbal to use `4.3.3` as
the lowest version along with required adjustments to the
extended Doctrine DBAL classes to adapt to ongoing upstream
changes:

MySQL and MariaDB
-----------------

The merged pull-request [1] changes column definition
parsing for MySQL and MariaDB platforms, to which our cloned
`MySQLSchemaManager::getPortableTableColumnDefinition`
is mostly incompatible.

For example, it has been adjusted to keep doc-block based
Doctrine type-configuration working, and ensures `TEXT`
field default values.

Copied code is updated to the current upstream state,
including minor new code cloning.

PostgreSQL
----------

The merged pull-request [2] simplified the schema column
introspection for the PostgresSQL platform. The change
requires the TYPO3 layer of the `PostgreSQLSchemaManager`
to be adopted accordingly.

The underlying code is aligned to the 4.3.x version,
with TYPO3 related adjustments still in place.

SQLite
------

The merged pull-request [3] does not require a change
to the TYPO3 layer, so that only a comment is placed
to align with the other `SchemaManager` changes.

Schema comparison
-----------------

Doctrine added `@final` annotations to two classes as
soft-flag for an upcoming breaking change in Doctrine DBAL 5
and therefore we add these errors to our PHPStan baseline for
now, as the extended classes are still needed and will require
a larger refactoring of our database analyzer stack.

CustomDoctrineTypesColumnDefinitionTrait
----------------------------------------

The purpose of this trait was to handle `enum` and `set`
types during database comparison actions, including
proper sanitization of column default values and reading the
values for these types, to mitigate comparison issues.

Only MySQL and MariaDB support these types. They we initially
implemented directly in TYPO3 and have been – in case of the
`EnumType` – moved to Doctrine in the meantime.

The trait checked for the `dbType` to decide if action needs to be taken
or not, making the trait useless in the following extended SchemaManager:

* PostgreSQLSchemaManager
* SQLiteSchemaManager

The original intention to use the trait in all extended
SchemaManager instances was to have custom handling in
one place and decide on the platform or types.

Meanwhile, the `enum` default value and values parsing
has been integrated in Doctrine DBAL directly and is
no longer needed for this. Having one method cloned
and adjusted to restore custom comment-based type
declaration is not worth a trait. Therefore the trait is now
removed and `set` type handling is integrated directly into the cloned
method in `MySQLSchemaManager::parentGetPortableTableColumnDefinition`.

Trait usage is removed from all schema managers.

SchemaManager unit tests
------------------------

Extended schema manager tests have been implemented as unit
tests using mocked method results, which are technically
wrong. PostgreSQL and SQLite do not support `ENUM` and
`SET` types and the test setup and expectation were wrong.
Corresponding tests along with unit test fixture classes
are now removed.

`MySQLSchemaManagerTest` generally supports `ENUM` and `SET`,
but covering this with a unit test along with
fixture schema manager and mocked column parsing is not
guaranteed to cover future upstream by Doctrine anyway. The tested
values reading from database structure have been integrated into
the `SchemaManagerTest` functional test, testing the creation
for `ENUM` and `SET` fields.

All schema manager unit tests along with fixture extensions are
removed due to being obsolete or replaced with functional tests.

See:

* `SchemaManagerTest::enumTypeFieldCanBeCreated()`
* `SchemaManagerTest::setTypeFieldCanBeCreated()`

Closing words
-------------

Maintaining code adoptions like these is similar to
TYPO3 extensions relying on XCLASSing. The TYPO3 Core
Team strives to align with the Doctrine Team to find
patterns that allow for better integration in frameworks
like TYPO3. This is not a possible short-term scenario
though.

Custom `ENUM` and `SET` type value parsing is adjusted
to respect parsing of a changed database structure due
to the Doctrine DBAL changes. Unit tests are adopted
to provide changes in column data for the expected
structure.

`doctrine/dbal 4.3.0` [1] also contains an important,
performance related change [5] and also the `4.3.1`
bugfix release, which is taken as minimum rule. Using
the `~4.3.3` version constraints limits Doctrine DBAL
to `>=4.3.3 <4.4` following a decision to narrow
Doctrine to patch level upgrades, mitigating possible
breaking updates when newer minor versions of
`doctrine/dbal` doctrine are released, which are possibly
imcompatible with TYPO3 specific platform adaptions and
overrides.

Used command(s):

```
BIN_COMPOSER="$( which composer )" \
&& ${BIN_COMPOSER} require \
  -d typo3/sysext/core --no-update --no-install \
  'doctrine/dbal':'~4.3.3' \
&& ${BIN_COMPOSER} require \
  -d typo3/sysext/install --no-update --no-install \
  'doctrine/dbal':'~4.3.3' \
&& ${BIN_COMPOSER} require \
  -d typo3/sysext/redirects --no-update --no-install \
  'doctrine/dbal':'~4.3.3' \
&& ${BIN_COMPOSER} require -w \
  'doctrine/dbal':'~4.3.3' \
&& Build/Scripts/runTests.sh -s phpstanGenerateBaseline
```

[1] https://github.com/doctrine/dbal/pull/6932
[2] https://github.com/doctrine/dbal/pull/6933
[3] https://github.com/doctrine/dbal/pull/6944
[4] https://github.com/doctrine/dbal/releases/tag/4.3.0
[5] https://github.com/doctrine/dbal/pull/6926
[6] https://github.com/doctrine/dbal/releases/tag/4.3.1
[7] https://github.com/doctrine/dbal/releases/tag/4.3.2
[8] https://github.com/doctrine/dbal/releases/tag/4.3.3

Review

  • Code review CR +2 Done
    • Stefan Bürk +2
    • Benjamin Franzke +1
    • Christian Kuhn +1
  • Verified V +2 Done
    • Stefan Bürk +2
    • Benjamin Franzke +1
    • Oliver Klee +1
    • Christian Kuhn +1
    • core-ci +1 CI

Issues 1

Files 18

Review log 4173

Show 2448 earlier messages
  1. Stefan Bürk uploaded patch set 1 03/15/25, 14:08

  2. core-ci voted on patch set 1 V -1 03/15/25, 14:17 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/75641

  3. Stefan Bürk marked it work in progress 03/15/25, 17:57

  4. Stefan Bürk uploaded patch set 2 V -1 03/30/25, 14:16

  5. core-ci voted on patch set 2 V -1 03/30/25, 14:25 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76669

  6. Stefan Bürk uploaded patch set 3 V -1 03/30/25, 15:41

  7. core-ci voted on patch set 3 V -1 03/30/25, 15:48 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76670

  8. Stefan Bürk uploaded patch set 4 V -1 03/30/25, 16:15

  9. core-ci voted on patch set 4 V -1 03/30/25, 16:21 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76671

  10. Stefan Bürk uploaded patch set 5 V -1 03/30/25, 16:31

  11. core-ci voted on patch set 5 V -1 03/30/25, 16:37 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76672

  12. Stefan Bürk uploaded patch set 6 V -1 03/30/25, 16:40

  13. core-ci voted on patch set 6 V +1 03/30/25, 16:46 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76673

  14. core-ci voted on patch set 6 V -1 03/30/25, 17:03 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76674

  15. Stefan Bürk uploaded patch set 7 V -1 06/04/25, 14:26

  16. core-ci voted on patch set 7 V -1 06/04/25, 14:37 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79868

  17. Stefan Bürk uploaded patch set 8 V -1 06/04/25, 17:21

  18. core-ci voted on patch set 8 V -1 06/04/25, 17:27 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79884

  19. Stefan Bürk uploaded patch set 9 V -1 06/04/25, 21:25

  20. core-ci voted on patch set 9 V -1 06/04/25, 21:32 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79891

  21. Stefan Bürk uploaded patch set 10 V -1 06/04/25, 21:36

  22. core-ci voted on patch set 10 V -1 06/04/25, 21:45 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79892

  23. Stefan Bürk rebased onto patch set 11 V -1 06/05/25, 14:34

  24. core-ci voted on patch set 11 V -1 06/05/25, 14:44 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79934

  25. Stefan Bürk rebased onto patch set 12 V -1 06/05/25, 15:07

  26. core-ci voted on patch set 12 V -1 06/05/25, 15:18 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79937

  27. Stefan Bürk uploaded patch set 13 V -1 06/05/25, 15:30

  28. core-ci voted on patch set 13 V -1 06/05/25, 15:41 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79940

  29. Stefan Bürk rebased onto patch set 14 V -1 06/05/25, 16:49

  30. core-ci voted on patch set 14 V +1 06/05/25, 16:58 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79944

  31. core-ci commented on patch set 14 06/05/25, 17:32 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79946

  32. Stefan Bürk uploaded patch set 15 06/05/25, 17:58

    Copied Votes: * Verified+1 (copy condition: "**changekind:NO_CHANGE**")
  33. core-ci voted on patch set 15 V +1 06/05/25, 18:07 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79948

  34. Stefan Bürk uploaded patch set 16 V +1 06/05/25, 19:40

  35. core-ci voted on patch set 16 V +1 06/05/25, 19:49 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/79951

  36. Stefan Bürk uploaded patch set 17 V +1 06/10/25, 04:36

  37. core-ci voted on patch set 17 V +1 06/10/25, 04:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/80095

  38. Stefan Bürk uploaded patch set 18 V +1 08/01/25, 09:42

  39. core-ci voted on patch set 18 V +1 08/01/25, 09:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82622

  40. core-ci commented on patch set 18 08/01/25, 10:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82624

  41. Stefan Bürk updated the commit message in patch set 19 V +1 08/01/25, 10:22

  42. Stefan Bürk marked it ready for review 08/01/25, 10:22

  43. core-ci voted on patch set 19 V +1 08/01/25, 10:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82626

  44. Benni Mack updated the commit message in patch set 20 V +1 08/01/25, 10:51

  45. core-ci voted on patch set 20 V +1 08/01/25, 10:58 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82627

  46. Oliver Klee updated the commit message in patch set 21 V +1 08/01/25, 11:01

  47. core-ci voted on patch set 21 V +1 08/01/25, 11:08 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/82628

  48. Benjamin Franzke commented on patch set 21 08/01/25, 12:27

    CustomDoctrineTypesColumnDefinitionTrait.php:87

    The method `processCustomDoctrineTypesColumnDefinition` is used for all platforms (mariadb, postgres and sqlite), but `column_type` refers to the mariadb specific `COLUMN_TYPE` extension [1], right? How is it ensured that this code works for postgres and sqlite? I ask because I'm dump and do not understand this. I see that test cases are adapted with the expection that postgres and sqlite deliver this value, but I don't see where the platforms, doctrine or our codebase would set this value. Maybe you can shed some light on that? [1] https://mariadb.com/docs/server/reference/system-tables/information-schema/information-schema-tables/information-schema-columns-table#:~:text=%2C%20otherwise%20NULL.-,COLUMN_TYPE,-Column%20definition%2C%20a
  1. Stefan Bürk marked it work in progress 08/01/25, 13:55

  2. Stefan Bürk uploaded patch set 22 V +1 09/25/25, 11:00

  3. core-ci voted on patch set 22 V -1 09/25/25, 11:07 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/85502

  4. Stefan Bürk uploaded patch set 23 V -1 09/25/25, 11:21

  5. core-ci voted on patch set 23 V +1 09/25/25, 11:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/85504

  6. core-ci voted on patch set 23 V -1 09/25/25, 11:59 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/85505

  7. core-ci voted on patch set 23 V +1 09/25/25, 12:24 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/85505

  8. Stefan Bürk commented on patch set 23 09/25/25, 13:25

    CustomDoctrineTypesColumnDefinitionTrait.php:87 Reply

    Sorry for the late answer. > I ask because I'm dump and do not understand this You are not dump and even the experienced and best people may need someone to shed some light onto the way to avoid getting lost from time to time, so I will try that now and answer your question while also explaining the changes made during investigating the change again based on your more than valid question and remark! Originally, doctrine/dbal used a normalized look onto things and splitted them up into vendor specific things, as you already mention `COLUMN_TYPE` for MySQL/MariaDB and you are right, that for "PostgreSQL/SQLite" that should be something else. The trait had a conditional check for the dbType (enum|set) in https://review.typo3.org/c/Packages/TYPO3.CMS/+/88596/21/typo3/sysext/core/Classes/Database/Schema/SchemaManager/CustomDoctrineTypesColumnDefinitionTrait.php#57 and as only MySQL/MariaDB supported that type (from the TYPO3 supported databases) the code matched MariaDB/MySQL and was safe. I added the trait back then to all SchemaManagers in case there would be other generic/dedicated custom changes required, not making the "condition" clear enough. Instead of "creating" that now dedicated for each of them, and due to the fact that doctrine/dbal already does a lot of that already directly (which we have cloned) I decided to remove the trait now completly, removing the custom/fallback to doctrine original (cloned) from the extended SchemaManager. The required adoptions for the TYPO3 custom `SET` type handling has been added to `MySQLSchemaManager` (documented with code comments). With that I finally dropped the unit tests, which were nasty, did the wrong thing for PostgresSQL/SQLite not supporting it anyway hidden by mocked method returns not reflecting the reality. For MySQL/MariaDB I added the "value" parsing verification to the functional `SchemaManagerTests` literally testing the same. I also added addtional stuff testing the "support" of the custom types, which revealed that the TYPO3 SetType is not only registered but also mapped to all connections even for connection platforms not supporting `SET`. That should be streamlined to make that also correct and I added comment in the connection pool now. Regarding "custom" doctrine type handling there are also request pending to make that kind of configurable/extendable, so I postponed that "streamlining" as part of looking into that "feature" and documented in the test for the SET time the "incorrect" expected platforms. Hope that is okay for everyone.
  9. Stefan Bürk uploaded patch set 24 V +1 09/25/25, 13:28

  10. core-ci voted on patch set 24 V +1 09/25/25, 13:35 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/85521

  11. Stefan Bürk marked it ready for review 09/26/25, 14:55

  12. Oliver Klee voted on patch set 24 V +1 09/26/25, 19:02

  13. Simon Schaufelberger uploaded patch set 25 V +1 10/01/25, 19:37

  14. core-ci voted on patch set 25 V +1 10/01/25, 19:44 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/85835

  15. Oliver Klee voted on patch set 25 V +1 10/01/25, 20:02

  16. Stefan Bürk uploaded patch set 26 V +1 10/02/25, 21:52

  17. Stefan Bürk commented on patch set 26 10/02/25, 21:52

    CustomDoctrineTypesColumnDefinitionTrait.php:87 Reply

    Done
  18. Oliver Klee voted on patch set 26 V +1 10/02/25, 21:58

  19. core-ci voted on patch set 26 V +1 10/02/25, 21:59 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/85945

  20. Benjamin Franzke uploaded patch set 27 V +1 10/06/25, 08:54

  21. core-ci voted on patch set 27 V +1 10/06/25, 09:03 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/86064

  22. Oliver Klee voted on patch set 27 V +1 10/06/25, 09:22

  23. Christian Kuhn voted on patch set 27 CR +1 V +1 10/06/25, 11:47

  24. Stefan Bürk voted on patch set 27 CR +2 V +2 10/06/25, 12:02

  25. Stefan Bürk merged patch set 28 10/06/25, 12:54