Review main Merged

86436[BUGFIX] Ensure correct type resolving in `AbstractSchemaManager`

Stefan Bürk CR +2 V +2 Hard +818 −21 updated 10/11/24, 15:18

Commit message

Doctrine DBAL 4.x removed the feature to save specific type
information as part of the column comment and for reading
the database schema with `AbstractSchemaManager` hierarchy
classes.

That also includes the option to enable and disable this
feature along with setting in type classes which needs to
write the custom information in case used database do not
support it natively - or for implemented custom type.

Doctrine postulate that it is the responsibility of the
application or framework to keep track of a precise and
cross database schema on own behalf, where the provided
schema information retrieved with the Doctrine DBAL
`AbstractSchemaManager` implementation are types which
would produce the same database structure for that vendor.

The possible TYPO3 counterpart for expected field types is
`$GLOBALS['TCA']` (TcaSchemaFactory), but does not really
ensure to hold all information, because TYPO3 allows extension
authors to define tables not managed by TYPO3 (TCA) with
`ext_tables.sql` files.

With #99062 the introduction of the native JSON database field
support it has been postulated that simply providing an array
for `Connection` methods like `insert` or `update` and using
`DataHandler` for TCA-type JSON handles this for all databases,
even if not nativly supported. That broke in TYPO3 v13 due to
the breaking change of Doctrine.

This mitigate this issue, this change reintroduces column
comment based type handling by ...

* introducing `GetColumnDeclarationSQLCommentTypeAwareTrait`
  used within extended `AbstractPlatform` classes to ensure
  adding column comment type hints for types on platforms
  needing it.
* introducing `ColumnTypeCommentMethodsTrait` used within
  extended `AbstractSchemaManager` classes to ensure that
  the correct type is set when reading database structure
  information.
* modifying extended `AbstractSchemaManager` classes to use
  the trait methods in suitable places, partly needing to
  clone code from Doctrine DBAL classes.
* removeing `ConnectionMigrator` SQLite specific override
  code switching to `TextType` for unsupported `JsonType`
  and resolving now the comment and todo with this change.

[1] https://github.com/doctrine/dbal/blob/54be50e2f2ea4cdbbdee35b9d0cf08cd9cb9bcb8/UPGRADE.md#bc-break-deployed-database-schema-no-longer-contains-the-information-about-abstract-data-types
[2] https://github.com/doctrine/dbal/blob/54be50e2f2ea4cdbbdee35b9d0cf08cd9cb9bcb8/UPGRADE.md#bc-break-removed-platform-commented-type-api

Review

  • Code review CR +2 Done
    • Stefan Bürk +2
    • Benjamin Franzke +1
    • Christian Kuhn +1
  • Verified V +2 Done
    • Stefan Bürk +2
    • Benjamin Franzke +1
    • Christian Kuhn +1
    • core-ci +1 CI

Issues 4

Files 19

Review log 2637

Show 812 earlier messages
  1. Stefan Bürk uploaded patch set 1 10/05/24, 21:55

  2. core-ci voted on patch set 1 V -1 10/05/24, 22:12 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67164

  3. Stefan Bürk updated the commit message in patch set 2 V -1 10/05/24, 22:27

  4. core-ci voted on patch set 2 V -1 10/05/24, 22:44 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67167

  5. Stefan Bürk uploaded patch set 3 V -1 10/06/24, 12:23

  6. Stefan Bürk updated the commit message in patch set 4 10/06/24, 12:33

  7. core-ci voted on patch set 3 V +1 10/06/24, 12:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67181

  8. core-ci voted on patch set 4 V +1 10/06/24, 12:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67182

  9. Benjamin Franzke commented on patch set 4 10/07/24, 13:09

    BUGFIX
    Seems to resolve https://forge.typo3.org/issues/104781 as well(?)
  10. Benjamin Franzke commented on patch set 4 10/07/24, 13:21

  11. Stefan Bürk updated the commit message in patch set 5 V +1 10/07/24, 13:42

  12. Stefan Bürk commented on patch set 5 10/07/24, 13:42

    Reply

    Done

    Reply

    Sorry, no. This change does not solve any of the two mentioned issues. That need other work.
  1. core-ci voted on patch set 5 V +1 10/07/24, 13:59 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67240

  2. Stefan Bürk commented on patch set 5 10/10/24, 17:35

    Reply

    Done
  3. Stefan Bürk rebased onto patch set 6 V +1 10/10/24, 17:35

  4. core-ci voted on patch set 6 V +1 10/10/24, 17:43 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67525

  5. Stefan Bürk rebased onto patch set 7 V +1 10/11/24, 09:08

  6. Christian Kuhn commented on patch set 7 10/11/24, 09:20

    these two links are 404
  7. core-ci voted on patch set 7 V +1 10/11/24, 09:26 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67576

  8. Stefan Bürk uploaded patch set 8 V +1 10/11/24, 09:29

  9. Stefan Bürk commented on patch set 8 10/11/24, 09:30

    Reply

    Changed.
  10. core-ci voted on patch set 8 V -1 10/11/24, 09:39 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67579

  11. core-ci voted on patch set 8 V +1 10/11/24, 09:50 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67579

  12. Benjamin Franzke voted on patch set 8 V +1 10/11/24, 10:11

  13. Benjamin Franzke commented on patch set 8 10/11/24, 10:12

    GetColumnDeclarationSQLCommentTypeAwareTrait.php:25

    I guess this code has been extracted from doctrine v2? Can we link to the source, wherever we've taken code from?
  14. Andreas Kienast commented on patch set 8 10/11/24, 10:52

    ConnectionMigrator.php:299

    What issue is solved by moving the condition into the loop? This looks like a no-op for sqlite/postgres.
  15. Christian Kuhn voted on patch set 8 CR +1 V +1 10/11/24, 11:06

  16. Stefan Bürk commented on patch set 8 10/11/24, 11:35

    GetColumnDeclarationSQLCommentTypeAwareTrait.php:25 Reply

    Good point, will update. I have to use commit hash gitbub urls for that, because doctrine deletes branches all along. Origin was dbal v3.

    ConnectionMigrator.php:299 Reply

    Good point. That change is a left over. In the first tries to solve this issue, I hooked here in - and needed it for all platforms, and just doing the maria/mysql stuff - thus moved the condition in to avoid at least here two foreach loops on these two platforms. Will revert that, not needed now.
  17. Stefan Bürk uploaded patch set 9 CR +1 V +1 10/11/24, 12:30

  18. Stefan Bürk uploaded patch set 10 10/11/24, 12:31

  19. Stefan Bürk commented on patch set 10 10/11/24, 12:32

    GetColumnDeclarationSQLCommentTypeAwareTrait.php:25 Reply

    Adde links to sources and some words on several places. Hope that I catched everything and that this is what you meant.

    ConnectionMigrator.php:299 Reply

    Done
  20. core-ci voted on patch set 9 V +1 10/11/24, 12:37 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67595

  21. core-ci voted on patch set 10 V +1 10/11/24, 12:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/67596

  22. Benjamin Franzke commented on patch set 10 CR +1 V +1 10/11/24, 13:48

    GetColumnDeclarationSQLCommentTypeAwareTrait.php:25 Reply

    yup, thanks a lot!
  23. Christian Kuhn voted on patch set 10 CR +1 V +1 10/11/24, 15:08

  24. Stefan Bürk voted on patch set 10 CR +2 V +2 10/11/24, 15:18

  25. Stefan Bürk merged patch set 11 10/11/24, 15:18