85103[WIP][TASK] Introduce linter for SQL
Commit message
Review
-
Code review CR 0
-
Verified V 0
Files 25
- Build/Scripts/runTests.sh+7 −0
- Build/sqlfluff/typo3.cfgnew+93 −0
- typo3/sysext/core/Classes/Database/Schema/SqlReader.php+3 −0
- typo3/sysext/core/Resources/Private/Sql/Cache/Backend/Typo3DatabaseBackendCache.sql+4 −4
- typo3/sysext/core/Resources/Private/Sql/Cache/Backend/Typo3DatabaseBackendTags.sql+5 −5
- typo3/sysext/core/Tests/Unit/Database/Schema/SqlReaderTest.php+24 −0
- typo3/sysext/core/ext_tables.sql+52 −52
- typo3/sysext/dashboard/ext_tables.sql+5 −5
- typo3/sysext/extensionmanager/ext_tables.sql+21 −21
- typo3/sysext/felogin/ext_tables.sql+2 −2
- typo3/sysext/filemetadata/ext_tables.sql+6 −5
- typo3/sysext/frontend/ext_tables.sql+8 −8
- typo3/sysext/impexp/ext_tables.sql+13 −13
- typo3/sysext/indexed_search/ext_tables.sql+81 −81
- typo3/sysext/linkvalidator/ext_tables.sql+2 −2
- typo3/sysext/reactions/ext_tables.sql+1 −1
- typo3/sysext/redirects/ext_tables.sql+1 −1
- typo3/sysext/scheduler/ext_tables.sql+2 −2
- typo3/sysext/seo/ext_tables.sql+1 −1
- typo3/sysext/styleguide/ext_tables.sql+68 −68
Review log 89
-
Ayke Halder uploaded patch set 1 07/04/24, 10:05
-
core-ci voted on patch set 1 V +1 07/04/24, 10:14 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/60958
-
Christian Kuhn commented on patch set 1 CR -1 07/04/24, 10:20
hey. thanks for your efforts. i think there is little point in doing this: ext_tables.sql is a custom dialect, handled by our internal lexer, and it is not even bound to a specific dbms. quoting for instance happens on a different layer. these files are not supposed to be executed onto some dmbs directly. -
Ayke Halder commented on patch set 1 07/04/24, 10:28
Reply
This still enhances the code by checking some code-style rules, that in the end have an impact on the DB. E.g. rules like "do not use upper-case characters for column identifiers". (The quoting is not for quoting, but to mark identifiers breaking rules.) -
Christian Kuhn commented on patch set 1 07/04/24, 12:13
Reply
no. and we will also most likely not change such "offending" field names in the foreseeable future since that would create heavily breaking changes without serious benefit. there is no point in marking them as 'breaking rules', and we are NOT breaking rules since all DBMS support this, and ext_tables.sql is a custom dialect. what we will do: we will postulate clearly that only [A-Z,a-z,_] (need to re-check) are allowed for field and table names, since that will allow us to use latin1 for fields that contain these names, but that's a different todo for a different time when the parser is able to handle this. -
Ayke Halder commented on patch set 1 07/04/24, 12:56
Reply
For sure these already "offending" field names can not be changed because of the serious breaking changes this would introduce. Most of the field and table names only use underscore and lower-case `[_a-z]` right now, why should we allow/introduce upper-case `[A-Z]` in the future? This is more about freezing the status quo for already introduced field names. For field names newly introduced in the future we might set strict rules. ---- About the 'custom SQL dialect': As of now this is really close to mysql/mariadb SQL dialect - actually I'd say we are just using a subset of mysql/mariadb SQL dialect. (On postgres comparison: a special point where we are not compatible with postgres is comments starting with a hash `#` - does not matter as TYPO3s `SqlReader` is removing comments and is preparing the code db-engine-wise anyway.) ---- Just to have an example: We are using code-style rules for PHP so that the look of the code does not vary: ```php // as-is: if ($isDevelopment) { $foo = 'bar'; } // also working: if ( $isDevelopment ) { $foo = 'bar'; } if($isDevelopment){ $foo='bar'; } if ($isDevelopment) { $foo = 'bar'; } if ($isDevelopment) $foo = 'bar'; ``` Why should we not introduce code-style rules for TYPO3s 'custom SQL dialect'? As we are using a subset of mysql/mariadb SQL dialect, it looks do-able to introduce code-style rules here. ```sql /* E.g. how TYPO3s 'custom SQL dialect' should look (TBD) */ CREATE TABLE be_dashboards ( cruser_id int(11) unsigned DEFAULT 0 NOT NULL ); /* How it should not look */ CREATE TABLE be_dashboards ( cruser_id INT( 11 ) UNSIGNED DEFAULT 0 NOT NULL ) ; create table be_dashboards ( cruser_id int(11) unsigned default 0 not null ); Create Table be_dashboards ( cruser_id Int(11) Unsigned Default 0 Not NULL ); ``` -
Christian Kuhn commented on patch set 1 07/04/24, 13:56
Reply
Ayke. Please stop this. We will not add a linter for these files for the mentioned reasons, or pollute the files with some "-- noqa: RF04" magic strings, a linter wants to have now for a format that we do not bow to. We'll also not create infrastructure for this, or run it in CI. ext_tables.sql are became significantly smaller, even in core extensions, and most ext devs will not bother much with them in the future anymore. There is no point in adding something from a third party that checks something that does not match. Please abandon. -
Ayke Halder abandoned the change 07/04/24, 15:55
-
Ayke Halder commented on patch set 1 07/04/24, 15:55
Reply
Acknowledged