Issue Bug Under Review

#94609Moving a file to a different storage will not invalidate or modify the processed entry

Sybille Peters 07/21/21, 12:23 updated 11/18/26, 21:06

Description

Even after flushing the page cache, the processed file is still loaded from the old location (e.g. fileadmin/_processed_/7/4/csm_file_f2a9dd6eb0.png). Ideally, when moving a file to a different storage, the processed file entries in DB table sys_file_processedfile should be removed or updated and the processed files itself should be removed or moved.

Tested with 10.4.18.

Reviews 1

History 56

  1. Sybille Peters commented <span data-processor="relativetime" data-value="2021-10-20T19:53:26+00:00" > 10/20/21, 19:53 </span>

    Can still reproduce this in 11 master.

    Even after moving the file to a diffent storage (e.g. f/ instead of fileadmin/) and flushing cache, the processed file is still loaded from the old location, e.g. fileadmin/_processed_/e/4/csm_file_17c9247468.png

  2. Stefan Terborg commented and changed TYPO3 Version 10 11 PHP Version 7.4 <span data-processor="relativetime" data-value="2023-07-11T07:52:16+00:00" > 07/11/23, 07:52 </span>

    I have a news record with a pdf file from storage S1. When I move the file to storage S2 the frontend still shows the old path. The file download leads to a 404. In the news record the file is marked as missing.

    This should be fixed as the file relation seems to be broken.

  3. Sybille Peters commented <span data-processor="relativetime" data-value="2024-12-01T13:14:21+00:00" > 12/01/24, 13:14 </span>

    @“Stefan Terborg” Is your example with the pdf file with a processed file? If not, could you open another issue for this if it still exists and set this issue as related? Did you flush the cache?

  4. Thomas Oliver Moll commented and changed TYPO3 Version 11 13 PHP Version 7.4 8.4 <span data-processor="relativetime" data-value="2026-11-18T11:51:54+00:00" > 11/18/26, 11:51 </span>

    I can confirm this is still present in 13.4.40 (verified in the source).

    ResourceStorage::moveFile() updates sys_file.storage and sys_file.identifier for the original file but never touches sys_file_processedfile . There is no listener on AfterFileMovedEvent that cleans up or re-homes processed files (unlike AfterFileDeletedEvent / AfterFileReplacedEvent , which are handled by FileDeletionAspect ).

    In practice the inconsistency is invisible as long as the old storage still exists, because ProcessedFile::reconstituteFromDatabaseRecord() re-resolves the storage from the DB row. However, once the old storage is removed from sys_file_storage , every processed file record pointing to it becomes unresolvable: StorageRepository::findByUid() returns null , and the next getStorage() call throws a RuntimeException. In our case this affected 800+ records and broke both the backend file list (thumbnail view) and frontend image rendering.

    A possible fix would be a listener on AfterFileMovedEvent that, when the storage UID has changed, deletes the processed file records (and optionally the physical files in the old _processed_ folder). TYPO3 will regenerate them on next access, so no data is lost. This mirrors what FileDeletionAspect::cleanupProcessedFiles() already does for delete/replace.

    As a defensive measure, ProcessedFile::reconstituteFromDatabaseRecord() could also fall back to the original file’s storage when the stored storage UID no longer exists, instead of leaving $this->storage as null.

    For now we just fixed it by doing:

    delete FROM `sys_file_processedfile` WHERE storage != (select storage FROM sys_file AS F WHERE F.uid = sys_file_processedfile.original);

    I hope this does not break something

    ————————————

    Appendix: proposed patch (AI-generated, untested)

    The patch below was drafted with the help of an AI assistant and has not been tested by us. The bug itself was found and reproduced by real users in production; the patch is only meant as helpful extra information for whoever picks this up. We do not have the knowledge and capabilities to thoroughly test such a core change for production use, so we deliberately refrained from creating a pull request.

    It consists of two parts:

    1. A new listener in FileDeletionAspect that removes the processed file records (and physical files) when a file is moved to a different storage. TYPO3 regenerates them on next access, so no data is lost. Note the comparison uses the persisted storage property, because moveFile() only updates the DB property – the in-memory storage object still points to the source storage.
    2. A defensive guard in ProcessedFile::reconstituteFromDatabaseRecord() so that a processed file whose storage no longer exists falls back to the original file’s storage instead of ending up with a null storage.
    diff --git a/typo3/sysext/core/Classes/Resource/ProcessedFile.php b/typo3/sysext/core/Classes/Resource/ProcessedFile.php
    --- a/typo3/sysext/core/Classes/Resource/ProcessedFile.php
    +++ b/typo3/sysext/core/Classes/Resource/ProcessedFile.php
    @@ -133,7 +133,10 @@ class ProcessedFile extends AbstractFile
             $this->processingUrl = $databaseRow['processing_url'] ?? '';
    
             if (!empty($databaseRow['storage']) && (int)$this->storage->getUid() !== (int)$databaseRow['storage']) {
    -            $this->storage = GeneralUtility::makeInstance(StorageRepository::class)->findByUid($databaseRow['storage']);
    +            $storage = GeneralUtility::makeInstance(StorageRepository::class)->findByUid($databaseRow['storage']);
    +            if ($storage !== null) {
    +                $this->storage = $storage;
    +            }
             }
         }
    
    diff --git a/typo3/sysext/core/Classes/Resource/Processing/FileDeletionAspect.php b/typo3/sysext/core/Classes/Resource/Processing/FileDeletionAspect.php
    --- a/typo3/sysext/core/Classes/Resource/Processing/FileDeletionAspect.php
    +++ b/typo3/sysext/core/Classes/Resource/Processing/FileDeletionAspect.php
    @@ -21,6 +21,7 @@ use TYPO3\CMS\Core\Attribute\AsEventListener;
     use TYPO3\CMS\Core\Database\ConnectionPool;
     use TYPO3\CMS\Core\Resource\Event\AfterFileAddedEvent;
     use TYPO3\CMS\Core\Resource\Event\AfterFileDeletedEvent;
    +use TYPO3\CMS\Core\Resource\Event\AfterFileMovedEvent;
     use TYPO3\CMS\Core\Resource\Event\AfterFileReplacedEvent;
     use TYPO3\CMS\Core\Resource\File;
     use TYPO3\CMS\Core\Resource\FileInterface;
    @@ -52,6 +53,22 @@ final class FileDeletionAspect
             $this->cleanupProcessedFiles($event->getFile());
         }
    
    +    #[AsEventListener('delete-processed-files-after-move')]
    +    public function cleanupProcessedFilesPostFileMove(AfterFileMovedEvent $event): void
    +    {
    +        $file = $event->getFile();
    +        if (!$file instanceof File) {
    +            return;
    +        }
    +        // moveFile() only updates the persisted storage property, the in-memory
    +        // storage object still points to the source storage, so compare the
    +        // persisted UID against the original folder's storage
    +        if ((int)$file->getProperty('storage') === (int)$event->getOriginalFolder()->getStorage()->getUid()) {
    +            return;
    +        }
    +        $this->cleanupProcessedFiles($file);
    +    }
    +
         #[AsEventListener('delete-processed-files-after-delete')]
         public function removeFromRepositoryAfterFileDeleted(AfterFileDeletedEvent $event): void
         {