#94609Moving a file to a different storage will not invalidate or modify the processed entry
Description
Even after flushing the page cache, the processed file is still loaded from the old location (e.g. fileadmin/_processed_/7/4/csm_file_f2a9dd6eb0.png). Ideally, when moving a file to a different storage, the processed file entries in DB table sys_file_processedfile should be removed or updated and the processed files itself should be removed or moved.
Tested with 10.4.18.
Reviews 1
Related issues 1
History 56
-
Sybille Peters changed Related to #77988 <span data-processor="relativetime" data-value="2021-07-21T12:23:32+00:00" > 07/21/21, 12:23 </span>
-
Sybille Peters commented <span data-processor="relativetime" data-value="2021-10-20T19:53:26+00:00" > 10/20/21, 19:53 </span>
-
Stefan Terborg commented and changed TYPO3 Version
1011 PHP Version 7.4 <span data-processor="relativetime" data-value="2023-07-11T07:52:16+00:00" > 07/11/23, 07:52 </span>I have a news record with a pdf file from storage S1. When I move the file to storage S2 the frontend still shows the old path. The file download leads to a 404. In the news record the file is marked as missing.
This should be fixed as the file relation seems to be broken.
-
Sybille Peters commented <span data-processor="relativetime" data-value="2024-12-01T13:14:21+00:00" > 12/01/24, 13:14 </span>
@“Stefan Terborg” Is your example with the pdf file with a processed file? If not, could you open another issue for this if it still exists and set this issue as related? Did you flush the cache?
-
Thomas Oliver Moll commented and changed TYPO3 Version
1113 PHP Version7.48.4 <span data-processor="relativetime" data-value="2026-11-18T11:51:54+00:00" > 11/18/26, 11:51 </span>I can confirm this is still present in 13.4.40 (verified in the source).
ResourceStorage::moveFile()updatessys_file.storageandsys_file.identifierfor the original file but never touchessys_file_processedfile. There is no listener onAfterFileMovedEventthat cleans up or re-homes processed files (unlikeAfterFileDeletedEvent/AfterFileReplacedEvent, which are handled byFileDeletionAspect).In practice the inconsistency is invisible as long as the old storage still exists, because
ProcessedFile::reconstituteFromDatabaseRecord()re-resolves the storage from the DB row. However, once the old storage is removed fromsys_file_storage, every processed file record pointing to it becomes unresolvable:StorageRepository::findByUid()returnsnull, and the nextgetStorage()call throws a RuntimeException. In our case this affected 800+ records and broke both the backend file list (thumbnail view) and frontend image rendering.A possible fix would be a listener on
AfterFileMovedEventthat, when the storage UID has changed, deletes the processed file records (and optionally the physical files in the old_processed_folder). TYPO3 will regenerate them on next access, so no data is lost. This mirrors whatFileDeletionAspect::cleanupProcessedFiles()already does for delete/replace.As a defensive measure,
ProcessedFile::reconstituteFromDatabaseRecord()could also fall back to the original file’s storage when the stored storage UID no longer exists, instead of leaving$this->storageas null.For now we just fixed it by doing:
delete FROM `sys_file_processedfile` WHERE storage != (select storage FROM sys_file AS F WHERE F.uid = sys_file_processedfile.original);
I hope this does not break something
————————————
Appendix: proposed patch (AI-generated, untested)
The patch below was drafted with the help of an AI assistant and has not been tested by us. The bug itself was found and reproduced by real users in production; the patch is only meant as helpful extra information for whoever picks this up. We do not have the knowledge and capabilities to thoroughly test such a core change for production use, so we deliberately refrained from creating a pull request.
It consists of two parts:
- A new listener in FileDeletionAspect that removes the processed file records (and physical files) when a file is moved to a different storage. TYPO3 regenerates them on next access, so no data is lost. Note the comparison uses the persisted storage property, because moveFile() only updates the DB property – the in-memory storage object still points to the source storage.
- A defensive guard in ProcessedFile::reconstituteFromDatabaseRecord() so that a processed file whose storage no longer exists falls back to the original file’s storage instead of ending up with a null storage.
diff --git a/typo3/sysext/core/Classes/Resource/ProcessedFile.php b/typo3/sysext/core/Classes/Resource/ProcessedFile.php --- a/typo3/sysext/core/Classes/Resource/ProcessedFile.php +++ b/typo3/sysext/core/Classes/Resource/ProcessedFile.php @@ -133,7 +133,10 @@ class ProcessedFile extends AbstractFile $this->processingUrl = $databaseRow['processing_url'] ?? ''; if (!empty($databaseRow['storage']) && (int)$this->storage->getUid() !== (int)$databaseRow['storage']) { - $this->storage = GeneralUtility::makeInstance(StorageRepository::class)->findByUid($databaseRow['storage']); + $storage = GeneralUtility::makeInstance(StorageRepository::class)->findByUid($databaseRow['storage']); + if ($storage !== null) { + $this->storage = $storage; + } } } diff --git a/typo3/sysext/core/Classes/Resource/Processing/FileDeletionAspect.php b/typo3/sysext/core/Classes/Resource/Processing/FileDeletionAspect.php --- a/typo3/sysext/core/Classes/Resource/Processing/FileDeletionAspect.php +++ b/typo3/sysext/core/Classes/Resource/Processing/FileDeletionAspect.php @@ -21,6 +21,7 @@ use TYPO3\CMS\Core\Attribute\AsEventListener; use TYPO3\CMS\Core\Database\ConnectionPool; use TYPO3\CMS\Core\Resource\Event\AfterFileAddedEvent; use TYPO3\CMS\Core\Resource\Event\AfterFileDeletedEvent; +use TYPO3\CMS\Core\Resource\Event\AfterFileMovedEvent; use TYPO3\CMS\Core\Resource\Event\AfterFileReplacedEvent; use TYPO3\CMS\Core\Resource\File; use TYPO3\CMS\Core\Resource\FileInterface; @@ -52,6 +53,22 @@ final class FileDeletionAspect $this->cleanupProcessedFiles($event->getFile()); } + #[AsEventListener('delete-processed-files-after-move')] + public function cleanupProcessedFilesPostFileMove(AfterFileMovedEvent $event): void + { + $file = $event->getFile(); + if (!$file instanceof File) { + return; + } + // moveFile() only updates the persisted storage property, the in-memory + // storage object still points to the source storage, so compare the + // persisted UID against the original folder's storage + if ((int)$file->getProperty('storage') === (int)$event->getOriginalFolder()->getStorage()->getUid()) { + return; + } + $this->cleanupProcessedFiles($file); + } + #[AsEventListener('delete-processed-files-after-delete')] public function removeFromRepositoryAfterFileDeleted(AfterFileDeletedEvent $event): void { -
Gerrit Code Review pushed patch set 1 for main to change 97807 status
NewUnder Review <span data-processor="relativetime" data-value="2026-11-18T21:06:36+00:00" > 11/18/26, 21:06 </span>
Can still reproduce this in 11 master.
Even after moving the file to a diffent storage (e.g. f/ instead of fileadmin/) and flushing cache, the processed file is still loaded from the old location, e.g. fileadmin/_processed_/e/4/csm_file_17c9247468.png