Issue Bug New Orphaned

#106585database:updateschema breaks on Data Type 'VECTOR'

Nikolai Townsend 04/17/25, 14:26 updated 05/23/25, 21:21

Description

When running vendor/bin/typo3 database:updateschema on CLI or “Analyze Database Structure” in the Install Tool, they throw an error.

This happens when you got a ext_tables.sql in your project that adds a field with the new Data Type ‘VECTOR’.

myext/ext_tables.sql

CREATE TABLE tx_myext_domain_model_chunk
(
    pid       tinyint(3) unsigned DEFAULT '0' NOT NULL,
    title     VARCHAR(255) NOT NULL DEFAULT '',
    url       VARCHAR(255) NOT NULL DEFAULT '',
    content   LONGTEXT     NOT NULL DEFAULT '',
    embedding VECTOR(1024) NOT NULL,
    VECTOR    INDEX (embedding)
);

This new data type was introduced on in MariaDB 11.7.2: https://mariadb.com/kb/en/vector-overview/

This data type is used in AI for search of text by their coordinates in an multi dimensional space.

Database analysis failed
[SQL Error] line 0, col 240: Error: Expected BIT, TINYINT, SMALLINT, MEDIUMINT, INT, INTEGER, BIGINT, REAL, DOUBLE, FLOAT, DECIMAL, NUMERIC, DATE, TIME, TIMESTAMP, DATETIME, YEAR, CHAR, VARCHAR, BINARY, VARBINARY, TINYBLOB, BLOB, MEDIUMBLOB, LONGBLOB, TINYTEXT, TEXT, MEDIUMTEXT, LONGTEXT, ENUM, SET, or JSON, got 'VECTOR' in statement: CREATE TABLE tx_nitosite_domain_model_chunk ( pid tinyint(3) unsigned DEFAULT '0' NOT NULL, title VARCHAR(255) NOT NULL DEFAULT '', url VARCHAR(255) NOT NULL DEFAULT '', content LONGTEXT NOT NULL DEFAULT '', embedding VECTOR(1024) NOT NULL, VECTOR INDEX (embedding) );

When I add this field manually to the DB, TYPO3 is working fine, I can even add vectors to this table with doctrine.

When I add it manually and remove it from ext_tables.sql it still crashes, just with stack trace:

(1/1) Doctrine\DBAL\Exception\InvalidArgumentException
Unknown database type "vector" requested, TYPO3\CMS\Core\Database\Platform\MariaDB1010Platform may not support it.

in /var/www/html/vendor/doctrine/dbal/src/Platforms/AbstractPlatform.php line 354

        $dbType = strtolower($dbType);

        if (! isset($this->doctrineTypeMapping[$dbType])) {
            throw new InvalidArgumentException(sprintf(
                'Unknown database type "%s" requested, %s may not support it.',
                $dbType,
                static::class,
            ));
at Doctrine\DBAL\Platforms\AbstractPlatform->getDoctrineTypeMapping('vector')
in /var/www/html/vendor/typo3/cms-core/Classes/Database/Schema/SchemaManager/ColumnTypeCommentMethodsTrait.php line 57
     */
    private function determineColumnType(string $dbType, array &$tableColumn): string
    {
        $platform = ($this instanceof AbstractPlatform) ? $this : $this->platform;
        $type = $dbType !== '' ? $platform->getDoctrineTypeMapping($dbType) : '';
        if (isset($tableColumn['comment'])) {
            $type = $this->extractDoctrineTypeFromComment($tableColumn['comment'], $type);
            $tableColumn['comment'] = $this->removeDoctrineTypeFromComment($tableColumn['comment'], $type);
        }
at TYPO3\CMS\Core\Database\Schema\SchemaManager\MySQLSchemaManager->determineColumnType('vector', array('table_name' => 'tx_nitosite_domain_model_chunk', 'field' => 'embedding', 'type' => 'vector(1024)', 'null' => 'NO', 'key' => 'MUL', 'default' => null, 'extra' => '', 'comment' => '', 'characterset' => null, 'collation' => null, 'name' => ''))
in /var/www/html/vendor/typo3/cms-core/Classes/Database/Schema/SchemaManager/MySQLSchemaManager.php line 241
        // Following line differs from \Doctrine\DBAL\Schema\MySQLSchemaManager::_getPortableTableColumnDefinition,
        // taken from:
        // - https://github.com/doctrine/dbal/blob/61446f07fcb522414d6cfd8b1c3e5f9e18c579ba/src/Schema/MySQLSchemaManager.php#L186-L192
        // - https://github.com/doctrine/dbal/blob/61446f07fcb522414d6cfd8b1c3e5f9e18c579ba/src/Schema/PostgreSQLSchemaManager.php#L427-L429
        $type = $this->determineColumnType($dbType, $tableColumn);

        switch ($dbType) {
            case 'char':
            case 'binary':
at TYPO3\CMS\Core\Database\Schema\SchemaManager\MySQLSchemaManager->parentGetPortableTableColumnDefinition(array('table_name' => 'tx_nitosite_domain_model_chunk', 'field' => 'embedding', 'type' => 'vector(1024)', 'null' => 'NO', 'key' => 'MUL', 'default' => null, 'extra' => '', 'comment' => '', 'characterset' => null, 'collation' => null, 'name' => ''))
in /var/www/html/vendor/typo3/cms-core/Classes/Database/Schema/SchemaManager/MySQLSchemaManager.php line 108
        /** @var DoctrineMariaDBPlatform|DoctrineMySQLPlatform $platform */
        $platform = $this->platform;
        $tableColumn = $this->normalizeTableColumnData($tableColumn, $platform);
        return $this->processCustomDoctrineTypesColumnDefinition($tableColumn, $platform)
            ?? $this->parentGetPortableTableColumnDefinition($tableColumn);
    }

    /**
     * @param array<string, mixed> $tableColumn
at TYPO3\CMS\Core\Database\Schema\SchemaManager\MySQLSchemaManager->_getPortableTableColumnDefinition(array('TABLE_NAME' => 'tx_nitosite_domain_model_chunk', 'field' => 'embedding', 'type' => 'vector(1024)', 'null' => 'NO', 'key' => 'MUL', 'default' => null, 'EXTRA' => '', 'comment' => '', 'characterset' => null, 'collation' => null))
in /var/www/html/vendor/doctrine/dbal/src/Schema/AbstractSchemaManager.php line 670
    protected function _getPortableTableColumnList(string $table, string $database, array $tableColumns): array
    {
        $list = [];
        foreach ($tableColumns as $tableColumn) {
            $column = $this->_getPortableTableColumnDefinition($tableColumn);

            $name        = strtolower($column->getQuotedName($this->platform));
            $list[$name] = $column;
        }

The people from DDEV said, they are planing to add MariaDB 11.8 when the LTS is released.

Beside the DB compare, TYPO3 12 + 13 seem to work well with MariaDB 11.7.x.

Reviews 0

No change resolves or names this issue in its commit message, and no comment links one.

Fix this issue

Start the commit message of your patch with this. Add what the patch does, and the versions it goes to in "Releases:". How to create a patch

Commit message
[BUGFIX] database:updateschema breaks on Data Type 'VECTOR' Resolves: #106585 Releases: main