Issue Bug Closed

#98037Problem with renaming folders

Anke Altintop 07/27/22, 08:21 updated 09/24/26, 10:01

Description

I have problems with FAL and renaming folders. Something goes wrong with changing the identifier in the sys_file entries, some have changed and some not. There are also duplicate entries. The identifier_hash and folder_hash are different for the duplcate entries.
Running FAL indexer scheduler task or updating the reference index doesn’t solve the problem (I guess because of the different hashes).

The System is TYPO3 10. I guess there might be a timeout, the folder structure is quite deep.

Reviews 0

No change resolves or names this issue in its commit message, and no comment links one.

History 8

  1. Łukasz Uznański commented <span data-processor="relativetime" data-value="2023-10-03T14:46:40+00:00" > 10/03/23, 14:46 </span>

    Can’t reproduce this issue on v13. I created very deep structure, but renaming is working without any problems. Maybe this related to your environment setup?

    Could you provide more details? How to reproduce this issue? Directory overview or environment setup?

  2. Anke Altintop commented <span data-processor="relativetime" data-value="2023-10-09T06:48:47+00:00" > 10/09/23, 06:48 </span>

    The directory structure is at least 7 levels deep.
    It is
    TYPO3 10.4.37
    PHP 7.3.33
    Unix
    MySQL 10.4.31-MariaDB

    It’s possible the server had a timeout, but isn’t it strange, that the identifier_hash and folder_hash are different for the duplcate entries?

  3. Marcin Sągol commented <span data-processor="relativetime" data-value="2023-10-30T08:34:27+00:00" > 10/30/23, 08:34 </span>

    @kiwi2101 did you check if those files with incorrect hashes in the DB exist on the server? At the moment, I see that if I remove a file manually and change the folder name where this file was stored, the file record in the DB stays untouched (no hashes recalculated, etc.). Can you verify if the problematic entries in the DB are not marked as missing=1?

    The whole process looks like this (TYPO3 13-dev):

    - You rename a folder in the TYPO3 panel.
    - TYPO3 fetches all files from the DB in the changed folder (ResourceStorage::renameFolder() => $fileObjects = $this->getAllFileObjectsInFolder($folderObject)).
    - TYPO3 changes the folder name on the server and returns an array with mapping (old identifier => new identifier for files, etc.).
    - TYPO3 updates data for each file in the DB based on the mapping configuration.

    This data update is done by executing separate DB queries for each file. If something goes wrong during this process, you can end up with database rows where not all identifiers were updated to the new one. Then, if you run the indexer task from the Scheduler, it will create new records in the DB for files in the renamed folder and its subfolders. At the same time, it will mark files with old identifiers as missing=1 (those that were not processed when something went wrong as mentioned before).

    Can you verify if that might be your case?

  4. Anke Altintop commented <span data-processor="relativetime" data-value="2023-10-30T08:53:07+00:00" > 10/30/23, 08:53 </span>

    At this point it is a bit difficult to recap everything because we cleaned up the system after we found the mistake. But I am pretty sure, that the files existed and the editor only renamed the folder on lets say first or second level. So somewhere on a deeper level of the folder structure the errors popped up.

    I will try to reproduce the behavior on a staging system as soon as possible.

  5. Anke Altintop commented <span data-processor="relativetime" data-value="2023-11-29T09:16:59+00:00" > 11/29/23, 09:16 </span>

    I can’t reproduce the behavior anymore. Maybe it is fixed anyway at v13.
    We can also close the ticket at this point. If it pops up again, I will create a new ticket with detailed information.

    Thanks for your tests so far.

  6. Riccardo De Contardi commented and changed status New Closed <span data-processor="relativetime" data-value="2024-03-25T09:38:16+00:00" > 03/25/24, 09:38 </span>

    @Anke Altintop Thank you for your feedback; I apologize for this late reply.

    Issue closed in agreement with the reporter.

    If you think that this is the wrong decision or experience the issue again on recent TYPO3 versions, please open a new issue with a reference to this one.

    Thank you

  7. Jonas Eberle commented <span data-processor="relativetime" data-value="2026-09-24T10:01:11+00:00" > 09/24/26, 10:01 </span>

    We have seen that in a v11 and unless there has been specific work done I think it can still happen by big folders/many files or a slow database or file system.

    Folder gets renamed, files get moved, but database records haven’t thus file links/references dangle. Then “new” files get discovered, leading to new sys_file entries that are basically duplicates.

    I was able to reproduce by reducing the PHP may_execution_time to some seconds.

    I think this is still a hazard for serious database inconsistency. All it needs is a (temporarily) slow server or many files.

    I have not checked in latest code if there has been work done and I currently cannot.

    In my opinion it should be tried to make renaming folders (aka file moving) safe: Either by doing very small chunks of work that leave a non-finished, but consistent state (reduce risk, but not a full remedy) or by setting the database into a state that signals that there is still work to do (and that it is currently inside a rename-operation) so that another job can pick it up and finish it.