#106585database:updateschema breaks on Data Type 'VECTOR'
Description
When running vendor/bin/typo3 database:updateschema on CLI or “Analyze Database Structure” in the Install Tool, they throw an error.
This happens when you got a ext_tables.sql in your project that adds a field with the new Data Type ‘VECTOR’.
myext/ext_tables.sql
CREATE TABLE tx_myext_domain_model_chunk
(
pid tinyint(3) unsigned DEFAULT '0' NOT NULL,
title VARCHAR(255) NOT NULL DEFAULT '',
url VARCHAR(255) NOT NULL DEFAULT '',
content LONGTEXT NOT NULL DEFAULT '',
embedding VECTOR(1024) NOT NULL,
VECTOR INDEX (embedding)
);
This new data type was introduced on in MariaDB 11.7.2: https://mariadb.com/kb/en/vector-overview/
This data type is used in AI for search of text by their coordinates in an multi dimensional space.
Database analysis failed [SQL Error] line 0, col 240: Error: Expected BIT, TINYINT, SMALLINT, MEDIUMINT, INT, INTEGER, BIGINT, REAL, DOUBLE, FLOAT, DECIMAL, NUMERIC, DATE, TIME, TIMESTAMP, DATETIME, YEAR, CHAR, VARCHAR, BINARY, VARBINARY, TINYBLOB, BLOB, MEDIUMBLOB, LONGBLOB, TINYTEXT, TEXT, MEDIUMTEXT, LONGTEXT, ENUM, SET, or JSON, got 'VECTOR' in statement: CREATE TABLE tx_nitosite_domain_model_chunk ( pid tinyint(3) unsigned DEFAULT '0' NOT NULL, title VARCHAR(255) NOT NULL DEFAULT '', url VARCHAR(255) NOT NULL DEFAULT '', content LONGTEXT NOT NULL DEFAULT '', embedding VECTOR(1024) NOT NULL, VECTOR INDEX (embedding) );
When I add this field manually to the DB, TYPO3 is working fine, I can even add vectors to this table with doctrine.
When I add it manually and remove it from ext_tables.sql it still crashes, just with stack trace:
(1/1) Doctrine\DBAL\Exception\InvalidArgumentException
Unknown database type "vector" requested, TYPO3\CMS\Core\Database\Platform\MariaDB1010Platform may not support it.
in /var/www/html/vendor/doctrine/dbal/src/Platforms/AbstractPlatform.php line 354
$dbType = strtolower($dbType);
if (! isset($this->doctrineTypeMapping[$dbType])) {
throw new InvalidArgumentException(sprintf(
'Unknown database type "%s" requested, %s may not support it.',
$dbType,
static::class,
));
at Doctrine\DBAL\Platforms\AbstractPlatform->getDoctrineTypeMapping('vector')
in /var/www/html/vendor/typo3/cms-core/Classes/Database/Schema/SchemaManager/ColumnTypeCommentMethodsTrait.php line 57
*/
private function determineColumnType(string $dbType, array &$tableColumn): string
{
$platform = ($this instanceof AbstractPlatform) ? $this : $this->platform;
$type = $dbType !== '' ? $platform->getDoctrineTypeMapping($dbType) : '';
if (isset($tableColumn['comment'])) {
$type = $this->extractDoctrineTypeFromComment($tableColumn['comment'], $type);
$tableColumn['comment'] = $this->removeDoctrineTypeFromComment($tableColumn['comment'], $type);
}
at TYPO3\CMS\Core\Database\Schema\SchemaManager\MySQLSchemaManager->determineColumnType('vector', array('table_name' => 'tx_nitosite_domain_model_chunk', 'field' => 'embedding', 'type' => 'vector(1024)', 'null' => 'NO', 'key' => 'MUL', 'default' => null, 'extra' => '', 'comment' => '', 'characterset' => null, 'collation' => null, 'name' => ''))
in /var/www/html/vendor/typo3/cms-core/Classes/Database/Schema/SchemaManager/MySQLSchemaManager.php line 241
// Following line differs from \Doctrine\DBAL\Schema\MySQLSchemaManager::_getPortableTableColumnDefinition,
// taken from:
// - https://github.com/doctrine/dbal/blob/61446f07fcb522414d6cfd8b1c3e5f9e18c579ba/src/Schema/MySQLSchemaManager.php#L186-L192
// - https://github.com/doctrine/dbal/blob/61446f07fcb522414d6cfd8b1c3e5f9e18c579ba/src/Schema/PostgreSQLSchemaManager.php#L427-L429
$type = $this->determineColumnType($dbType, $tableColumn);
switch ($dbType) {
case 'char':
case 'binary':
at TYPO3\CMS\Core\Database\Schema\SchemaManager\MySQLSchemaManager->parentGetPortableTableColumnDefinition(array('table_name' => 'tx_nitosite_domain_model_chunk', 'field' => 'embedding', 'type' => 'vector(1024)', 'null' => 'NO', 'key' => 'MUL', 'default' => null, 'extra' => '', 'comment' => '', 'characterset' => null, 'collation' => null, 'name' => ''))
in /var/www/html/vendor/typo3/cms-core/Classes/Database/Schema/SchemaManager/MySQLSchemaManager.php line 108
/** @var DoctrineMariaDBPlatform|DoctrineMySQLPlatform $platform */
$platform = $this->platform;
$tableColumn = $this->normalizeTableColumnData($tableColumn, $platform);
return $this->processCustomDoctrineTypesColumnDefinition($tableColumn, $platform)
?? $this->parentGetPortableTableColumnDefinition($tableColumn);
}
/**
* @param array<string, mixed> $tableColumn
at TYPO3\CMS\Core\Database\Schema\SchemaManager\MySQLSchemaManager->_getPortableTableColumnDefinition(array('TABLE_NAME' => 'tx_nitosite_domain_model_chunk', 'field' => 'embedding', 'type' => 'vector(1024)', 'null' => 'NO', 'key' => 'MUL', 'default' => null, 'EXTRA' => '', 'comment' => '', 'characterset' => null, 'collation' => null))
in /var/www/html/vendor/doctrine/dbal/src/Schema/AbstractSchemaManager.php line 670
protected function _getPortableTableColumnList(string $table, string $database, array $tableColumns): array
{
$list = [];
foreach ($tableColumns as $tableColumn) {
$column = $this->_getPortableTableColumnDefinition($tableColumn);
$name = strtolower($column->getQuotedName($this->platform));
$list[$name] = $column;
}
The people from DDEV said, they are planing to add MariaDB 11.8 when the LTS is released.
Beside the DB compare, TYPO3 12 + 13 seem to work well with MariaDB 11.7.x.
Reviews 0
No change resolves or names this issue in its commit message, and no comment links one.
Fix this issue
Start the commit message of your patch with this. Add what the patch does, and the versions it goes to in "Releases:". How to create a patch
- Commit message
-
[BUGFIX] database:updateschema breaks on Data Type 'VECTOR' Resolves: #106585 Releases: main
Related issues 3
History 6
-
Stefan Bürk commented and changed assigned to Stefan Bürk <span data-processor="relativetime" data-value="2025-04-17T17:01:49+00:00" > 04/17/25, 17:01 </span>
-
Stefan Bürk commented <span data-processor="relativetime" data-value="2025-05-23T15:01:03+00:00" > 05/23/25, 15:01 </span>
It seams that doctrine itself does not have support for VECTOR yet.
https://github.com/doctrine/dbal/issues/6703
We won’t add anythin in that area as long as doctrine does not support it directly.
-
Nikolai Townsend commented <span data-processor="relativetime" data-value="2025-05-23T21:21:11+00:00" > 05/23/25, 21:21 </span>
Stefan Bürk wrote in #note-2:
> It seams that doctrine itself does not have support for VECTOR yet.
>
> https://github.com/doctrine/dbal/issues/6703
>
> We won’t add anythin in that area as long as doctrine does not support it directly.In a raw way it is actually possible:
$connection->executeStatement( 'INSERT INTO tx_myext_domain_model_chunk (pid, title, url, content, embedding) VALUES (:pid, :title, :url, :content, VEC_FromText(:embedding))', [ 'pid' => $pid, 'title' => $title, 'url' => $url, 'content' => $content, 'embedding' => $embedding, ] );There is no way to handle VEC_FromText, VEC_ToText, VEC_DISTANCE* in an object oriented way with methods etc.
As a workaround, to get the ability to do a db-compare back, I build me a command that:
- backup my tx_myext_domain_model_chunk
- delete tx_myext_domain_model_chunk
- run database:updateschema
- restore tx_myext_domain_model_chunk
Not the nicest solution, but it might do the job for a while.
-
Georg Ringer changed Related to #106719 <span data-processor="relativetime" data-value="2025-09-15T19:50:36+00:00" > 09/15/25, 19:50 </span>
-
Georg Ringer changed Related to #93637 <span data-processor="relativetime" data-value="2025-10-02T19:59:42+00:00" > 10/02/25, 19:59 </span>
-
Julian Mair changed Related to #110761 <span data-processor="relativetime" data-value="2026-09-22T09:45:34+00:00" > 09/22/26, 09:45 </span>
@nito Thanks for the report.
The first direct thing is, that TYPO3 has no official support for MariaDB 11.0 or newer
and our core tests are failing with these newer versions which has to be tackled first.
At least for 13.4 and main. We will not announce official support for v12 as long as we
cannot do that for 13.4 or main (v14).
The second thing here is, that
ext_tables.sqlprovides only a defined subset of MySQLDDL to allow TYPO3 to ensure compatibility across all supported database systems and
versions with the same behaviour.
I need to check the new field type and how a full compat can be archieved. In general,
that includes extending the TYPO3 parser for it and match it to a Doctrine DBAL Type,
which is a first line if Doctrine DBAL has full support for this new field accross all
Database Systems in the 4.2 (or upcoming 4.3.0) version.
TYPO3 v12 is in priority bugfix mode, so don’t assume that this will happen in TYPO3 v12,
even v13 is most likely not a candidate for it. When possible, it may be a option for v13.
But all that is not written in stone, but you should be aware of that.
This is not a bug, it’s a feature request as the required MariaDB version is not officially
supported nor did TYPO3 ever annonced to support all database field types in
ext_tables.sql.I keep it for now as “bug” report, but this may be changed when investigating this in more
detail.