Issue Bug Closed

#57103FAL - Migrate file metadata from sys_file to an external metadata table fails with big data set

Krystian Szymukowicz 03/20/14, 13:23 updated 10/23/17, 21:40

Description

I have like 20k sys_files records.
While doing migrate file metadata from sys_file to sys_file_metadata in Install Tool I ended in “Fail”.

It turns out that 20k files is too much to cope with.
Doing insertion by chunks was “Successful”.

- class FileTableSplittingUpdate
- funkction checkForUpdate

....

$filesToMigrateUids = array_unique($filesToMigrateUids);
$filesToMigrateUidsParts = array_chunk($filesToMigrateUids, 100);
foreach($filesToMigrateUidsParts as $filesToMigrateUidsPart) {
          $dataToMove = $GLOBALS['TYPO3_DB']->exec_SELECTgetRows(
            implode(',', $this->detectFieldsToMigrate()) . ', uid AS file',
            'sys_file',
            'uid IN (' . implode(',', $filesToMigrateUidsPart) . ')'
            );
          $resultObject = $GLOBALS['TYPO3_DB']->exec_INSERTmultipleRows($this->metaDataTable, array_keys(current($dataToMove)), $dataToMove);
      if($resultObject === FALSE) {
        break;
    }
}
return $resultObject !== FALSE;

Reviews 1