88102[BUGFIX] Prevent accidental deletion in lowlevel
Commit message
Review
-
Code review CR 0
-
Verified V 0
Issues 1
Files 1
Review log 1112
-
Sybille Peters uploaded patch set 1 02/08/25, 13:12
-
core-ci voted on patch set 1 V +1 02/08/25, 13:19 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/73804
-
Stefan Bürk voted on patch set 1 CR -1 02/08/25, 21:40
-
Christian Kuhn commented on patch set 1 02/10/25, 12:11
well. the main issue with cleanup:orphanrecords and cleanup:missingrelations is that they are both insanely slow. this is roughly due to this strategy: get a page, do x queries per page for *all* existing tables if there are some records. compile all that into an humongous array to then work on it, easily leading to hundreds of megabytes memory. with my "not *that* huge" local instance, a run already takes 10 minutes, and if i unload ext:styleguide (which brings many tables), it is *still* 3 minutes. these commands are simply unusable for any not trivially small instance. i fixed a similar strategy with cleanup:flexforms in v13 and the result was stunning: from many minutes to a couple of seconds with nearly no memory. codesize shrinked to 1/3. as such, i wonder if it is really useful to hack on these commands and adding patches that add quite a bit of code. we are probably much better off refactoring the codebase to a sane solution first, to then see what still needs to be done. as such, i'm not a fan of the patch at the moment, and would rather try to pick it up after a general modernization of the two commands - i think it will be much more easy to solve it afterwards and probably won't add additional runtime overhead again. note these commands have *never* been looked at or touched in details since 3.8 or so. they are rotting around for a very long time and nobody ever cared to work on them. i wonder how many instances actively use them. what do you think? -
Sybille Peters commented on patch set 1 02/11/25, 07:27
Reply
You are right, no doubt (about the inefficiency, memory, etc.). My approach was: There is a really nasty bug. I mean this command deleting non-orphaned pages. And doing this fix took me less than 30 minutes. It may be better to completely overhaul cleanup, but who is willing to do it and when? My main goal is to fix this nasty bug ASAP (issue has been open 4 years). (we could reduce amount of changed code by reverting $io => $this->io, if this would help). So, is this overhaul happening any time soon? And wouldn't it be better to move "dbdoctor" to core and add it there? Or would it be an alternative to add a warning somewhere (the docs tell you to backup. but they don't tell you to lock the Backend before running the command). -
Sybille Peters commented on patch set 1 02/11/25, 09:11
Additional comment: if this has no chance of getting merged, let's abandon and not spend additional time on it. -
Christian Kuhn commented on patch set 1 02/11/25, 09:55
Reply
Let's keep it pending for a while - I'm kinda certain we'll refactor cleanup:orphanrecords quite soon, and depending on its implementation we can see wether this change still makes sense.Reply
Well, dbdoctor will not be merged to core as such - there are too many loose ends (i hope to get a v13 compat version ready, soon, though). But i looked at dbdoctor for cleanup:orphanrecords already: dbdoctor has two health checks that - combined - do what cleanup:orphanrecords does. It shouldn't be *that* hard to grab the dbdoctor code as basis and create a refactored cleanup:orphanrecords from it. It's probably a few hours to do, and I was close to hack on that already, but got lost in details: I for instance thought about dropping the DH call that does the deletion in favor of a direct hard delete, but was not 100% sure on it, yet. -
Sybille Peters commented on patch set 1 02/11/25, 17:54
Reply
ok -
Christian Kuhn commented on patch set 1 02/13/25, 23:14
Reply
hey. I quickly hacked up a PoC/WIP change: https://review.typo3.org/c/Packages/TYPO3.CMS/+/88213 This creates a list of valid pages without creating a full tree. One DB query per TCA table, minimal memory impact. Runtime before: >10 minutes, after: 0.3 seconds. Could you maybe play around with this a bit to see if it finds the same data set that the old implementation comes up with? -
Christian Kuhn commented on patch set 1 02/17/25, 19:04
OrphanRecordsCommand.php:166i'm skipping this in new implementation: these tables have no TCA. -
Christian Kuhn commented on patch set 1 02/17/25, 19:05
Reply
i'll abandon here now: i'm about to push a solution with https://review.typo3.org/c/Packages/TYPO3.CMS/+/88213.Reply
Done -
Christian Kuhn abandoned the change 02/17/25, 19:06