Review main Merged

84729[FEATURE] Add setting for backend lock file location, drop typo3conf

Garvin Hicking CR +2 V +2 Hard +225 −54 updated 08/07/24, 12:49

Commit message

As a follow-up to #104074, the directory "typo3conf" is no longer
required to store the `LOCK_BACKEND` file.

A new configuration option TYPO3_CONF_VARS[BE][lockBackendFile]
is added alongside a distinct `BackendLocker` service class to
indicate where the backend lock and unlock commands store
their lockfile. By default this uses `var/lock/` for composer mode,
and `config/` for legacy installations as a fallback for when
the setting is empty (default).

Review

  • Code review CR +2 Done
    • Garvin Hicking +2
    • Markus Klein +1
    • Helmut Hummel +1
  • Verified V +2 Done
    • Garvin Hicking +2
    • Markus Klein +1
    • Helmut Hummel +1
    • core-ci +1 CI

Issues 3

Files 10

Review log 4559

Show 2534 earlier messages
  1. Garvin Hicking uploaded patch set 1 06/17/24, 09:31

  2. core-ci voted on patch set 1 V -1 06/17/24, 09:36 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59717

  3. Mathias Brodala commented on patch set 1 06/17/24, 09:37

    WIP, I know, but still. ;-)

    Environment.php:290

    How about "getTransientStatePath" instead? At least the current usage is not related to "configuration" but "state" instead. Also I'd question if this is really transient. The backend can be locked for a long time and ENABLE_INSTALL_TOOL can be persistent if it contains the correct content. So maybe reduce this to getStatePath() and consider using a dedicated directory?
  4. Garvin Hicking uploaded patch set 2 V -1 06/17/24, 10:52

  5. Garvin Hicking commented on patch set 2 06/17/24, 10:56

    Environment.php:290 Reply

    Thank you. I will leave this open up for discussion, I'm unsure to introduce such a directory myself. I think you are right though, it would probably be put into the `config/` path then, that resembles the old `typo3conf` directory more?
  6. core-ci voted on patch set 2 V -1 06/17/24, 10:57 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59720

  7. Garvin Hicking uploaded patch set 3 V -1 06/17/24, 10:57

  8. Mathias Brodala commented on patch set 2 06/17/24, 10:58

    Important-104126-ReferenceIndexRebuildRequired.rst:14

    IMO: > Composer Mode Thus "a thing" (capitalized) and no quotes since it's not something alien by now.

    Important-104126-ReferenceIndexRebuildRequired.rst:23

    > Composer Mode

    Important-104126-ReferenceIndexRebuildRequired.rst:24

    There was nothing "unproper" before, so I'd drop this.

    Important-104126-ReferenceIndexRebuildRequired.rst:26

    Both paths should be written explicitly and clearly, e.g. using bullets.

    Important-104126-ReferenceIndexRebuildRequired.rst:30

    This actually suggests that there is a command to enable the install tool. ;-)
  9. Mathias Brodala commented on patch set 3 06/17/24, 10:59

    Environment.php:290 Reply

    Yup, that's not where these files belong. The "config" path may not even be writable after deployment.
  10. core-ci voted on patch set 3 V +1 06/17/24, 11:02 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59721

  11. Garvin Hicking uploaded patch set 4 V +1 06/17/24, 11:18

    Docs
  12. Garvin Hicking commented on patch set 4 06/17/24, 11:18

    Important-104126-ReferenceIndexRebuildRequired.rst:14 Reply

    Done

    Important-104126-ReferenceIndexRebuildRequired.rst:23 Reply

    Done

    Important-104126-ReferenceIndexRebuildRequired.rst:24 Reply

    But it was "typo3conf" previously, even in Composer Mode. I do see this as "unproper"? I'll resolve this, but "proper" directory storage for this is also left up for discussion, so needs adjustment anyways.

    Important-104126-ReferenceIndexRebuildRequired.rst:30 Reply

    Done
  13. core-ci voted on patch set 4 V +1 06/17/24, 11:24 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59723

  14. Garvin Hicking updated the commit message in patch set 5 V +1 06/17/24, 12:31

    restrict to main
  15. core-ci voted on patch set 5 V +1 06/17/24, 12:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59729

  16. Garvin Hicking uploaded patch set 6 V +1 06/17/24, 13:17

    rephrasing
  17. core-ci voted on patch set 6 V +1 06/17/24, 13:22 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59730

  18. Helmut Hummel commented on patch set 6 06/17/24, 14:00

    Environment.php:290 Reply

    What about var/lock ?
  19. Helmut Hummel commented on patch set 6 06/17/24, 14:00

    AjaxLoginController.php:117

    why has this path a trailing slash?
  20. Garvin Hicking uploaded patch set 7 V +1 06/18/24, 06:48

    Change to var lock
  21. Garvin Hicking commented on patch set 7 06/18/24, 06:49

    AjaxLoginController.php:117 Reply

    True, it's not in line with the other paths. Changed now to remove trailing slash from the getTransientPath/getLockPath return.

    Environment.php:290 Reply

    I think that's the best option, changed this path to use it.

    Important-104126-ReferenceIndexRebuildRequired.rst:24 Reply

    Done

    Important-104126-ReferenceIndexRebuildRequired.rst:26 Reply

    Done
  22. core-ci voted on patch set 7 V +1 06/18/24, 06:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59779

  23. Mathias Brodala commented on patch set 7 CR -1 06/18/24, 07:06

    LockedBackendGuard.php:43

    The new path should be mentioned first, the legacy path afterwards. However, "LOCK_BACKEND" by itself without path would be enough.

    LockedBackendGuard.php:75

    See above. Also "lock", not "transient" if kept.

    LockedBackendGuard.php:96

    See above; if kept, mention Composer first, then legacy. Also how about inserting the actual path (sans web path prefix) instead here? Then people will know right away which one to check. Also we should explicitly mention the CLI command instead of "CLI-scripts"

    Important-104126-DropTypo3confDirectoryFromSystemStatusAndBackendLocking.rst:27

    No mention of ENABLE_INSTALL_TOOL anymore? Also "... is now ... (unchanged)" reads odd. Please only state the new paths here or the old one in parentheses.

    Important-104126-DropTypo3confDirectoryFromSystemStatusAndBackendLocking.rst:33

    Edge case but it should be mentioned that upgrading from an instance with files in typo3conf will "lose" its state (backend unlocked, install tool disabled) since these files are ignored after the switch to the new release.

    EnableFileService.php:168

    The legacy one must stay "typo3conf/", or am I wrong?
  24. Garvin Hicking uploaded patch set 8 CR -1 V +1 06/18/24, 09:00

    Address code review
  25. Garvin Hicking commented on patch set 8 06/18/24, 09:01

    Thank you so much @mbrodala@pagemachine.de for the detailed comments and review, this means a lot to me. I've tried to address your feedback, with one remaining question. :-)

    LockedBackendGuard.php:43 Reply

    Shuffled them. I'd like to leave the full path here, then one can search for "typo3conf/LOCK_BACKEND". I've also addressed that the file may actually reside in three places (typo3conf, config, var/lock). (To make fulltext search work I decided to spell out all three places and not do something like `(typo3conf|config)/LOCK_BACKEND`.)

    LockedBackendGuard.php:75 Reply

    Done

    LockedBackendGuard.php:96 Reply

    Thank you, I missed this one. I believe the lookup for the actual path might prove a bit to finicky, I'd rather have this resolving beyond the scope of this PR.

    Important-104126-DropTypo3confDirectoryFromSystemStatusAndBackendLocking.rst:27 Reply

    Yes I removed this because no longer a central function is used to resolve the directory for ENABLE_INSTALL_TOOL and BACKEND_LOCK because they are now put into distinct directories. Nothing for ENABLE_INSTALL_TOOL changes, so I removed the mention here. Also rephrased the list so that I try to point out when it's typo3conf/ and when it's config/ instead (changing this would also be outside the scope of this PR)

    Important-104126-DropTypo3confDirectoryFromSystemStatusAndBackendLocking.rst:33 Reply

    That's a good idea. I also added a note about persisting LOCK_BACKEND for deployments.

    EnableFileService.php:168 Reply

    OK so this one needs a bit discussion, I hope you bear with me: typo3conf/ is used when project path = public path config/ is used in all else NON-COMPOSER installs The latter would IMO be the more preferrable to have. This method actually is only used to display a path, it's not used for the actual directory retrieval. So I wanted to enforce people "recognizing" the "new" config/ directory here, that's why I changed the display output. I could leave it at `typo3conf/` of course, but that might not apply to a larger user base anymore. I did not want to change this too much by for example evaluating the "real" path used with string replacement and introducing a chance of accidentally disclosing directory information. I could place "config/ or typo3conf/" as a string in here, though. Thoughts?
  26. core-ci voted on patch set 8 V +1 06/18/24, 09:07 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/59786

  27. Helmut Hummel commented on patch set 8 06/18/24, 15:04

    EnableFileService.php:168 Reply

    @gh@faktor-e.de Consider the following to be true in all non composer installs: project path = public path In non composer installs, you can only achieve something different, when you really know what you are doing and change env vars accordingly
  28. Garvin Hicking commented on patch set 8 06/18/24, 15:20

    EnableFileService.php:168 Reply

    @typo3@helhum.io OK, but since it's technically possible people have this in config/ or typo3conf/ it might then be best to describe all three paths? As for the output then, revealing "config/" as the target directory (for public=project) of LOCK_BACKEND would be better than "typo3conf/", do I understand you right? So for the casual user, they only see "config/" here, and can place the file for their legacy installs there. And the people who re-configured their setup in a way that typo3conf/ might be used, they'll know where to look, right?
  29. Benni Mack commented on patch set 8 CR -1 06/20/24, 10:41

    Sorry, but the Environment API was built in a way that you can override / initialize the paths. You now introduce two new "paths" without having the option to override parts. getLockPath() seems to be VERY specific IMHO, and shouldn't belong in a class which always exists from the first microsecond of every TYPO3 Request. Or maybe it should? Just putting this aside. getLockPath() actually reminds me about the actual "locking" in our Locking API. So I'm not a huge fan of the name per sé.
  30. Garvin Hicking commented on patch set 8 06/20/24, 10:57

    Reply

    I understand, and agree this is a very specific utility method. I wonder though were else we could implement it, because the paths need to be accessed in two different commands, plus controller plus middleware. Adding a new class for these two directories feels odd. Would you have a suggestion on where you would see this?
  31. Mathias Brodala commented on patch set 8 06/20/24, 11:03

    Reply

    Also I'd then question getLabelsPath() which also derives from the var-path without being configurable. Adding more accessors deriving from var-path seem valid then.
  32. Markus Klein commented on patch set 8 CR -1 07/16/24, 11:23

    Please do not use var/lock for anything else but files related to locking-mechanisms. IMO /var/transient is a better place
  33. Benni Mack uploaded patch set 9 CR -1 V +1 07/16/24, 18:32

  34. core-ci voted on patch set 9 V +1 07/16/24, 18:37 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61648

  1. Mathias Brodala commented on patch set 9 07/17/24, 13:24

    BackendLocker.php:52

    Since this is used at exactly one location, maybe rename to "getRedirectUri"? This gives this arbitrary string juggling a bit more semantic.
  2. Garvin Hicking uploaded patch set 10 V +1 07/17/24, 13:28

    Make lock location configurable
  3. core-ci voted on patch set 10 V +1 07/17/24, 13:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61688

  4. Garvin Hicking uploaded patch set 11 V +1 07/17/24, 13:35

    Rename variables and methods for clearer intent
  5. Garvin Hicking commented on patch set 11 07/17/24, 13:36

    Reply

    This has now been addressed by Bennis Patchset with a distinct BackendLocker service, which no longer needs to "interfer" with the Environment API.
    Thanks for the group effort here, much appreciated. Current iteration should hopefully address all needs and adjusted documentation for it.

    Reply

    We discussed this internally, there were several possibilities and none fit perfectly. So now this location is configurable for custom needs and depending on lock/filesystem setup, and only the default is set to var/lock and can be overriden.

    BackendLocker.php:52 Reply

    I agree and made a proposal in Patchset 11.

    EnableFileService.php:168 Reply

    I'm resolving this, summary: * The returned path here is only for display reasons * TYPO3v12+ location will be config/ and not typo3conf/
  6. core-ci voted on patch set 11 V +1 07/17/24, 13:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61689

  7. Garvin Hicking uploaded patch set 12 V +1 07/17/24, 14:00

    Tag as feature instead of task
  8. core-ci voted on patch set 12 V +1 07/17/24, 14:06 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61693

  9. Markus Klein commented on patch set 12 07/17/24, 16:24

    DefaultConfiguration.php:1312

    I still have to think about the naming. Maybe go with "backendlockFilePath" or "backendlockFileLocation"
  10. Garvin Hicking commented on patch set 12 07/17/24, 17:38

    DefaultConfiguration.php:1312 Reply

    I had like 5 iterations of this - I would like to keep the "lock" prefix here, because the other settings that are kind of related use it too. Generally I don't mind changing it, but hope for consent. "location" feels like you might be able to enter a URL here. i think we'll need the terms "lock", "backend" and "file" (because its not a path, but the full name) for it?
  11. Mathias Brodala commented on patch set 12 07/18/24, 10:25

    BackendLocker.php:39

    Sorry, noticed too late: this should also say $redirectUri and also all consumers of this method should use suitable variable names. Again, this makes it clearer what this content is meant for. (And yes, for type safety this should actually be UriInterface everywhere but that's likely too much for now.)
  12. Garvin Hicking uploaded patch set 13 V +1 07/18/24, 22:08

    Rename internal variable for clarity
  13. Garvin Hicking commented on patch set 13 07/18/24, 22:10

    BackendLocker.php:39 Reply

    Thanks for spotting this! I tried to unify that with a clear variable name $redirectUriFromLockFileContent now. I would like to keep it as a string for now, to keep the complexity of this patch at bay, if that's ok with you.
  14. core-ci voted on patch set 13 V +1 07/18/24, 22:13 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61765

  15. Markus Klein commented on patch set 13 07/19/24, 17:39

    DefaultConfiguration.php:1312 Reply

    Acknowledged
  16. Markus Klein commented on patch set 13 07/19/24, 17:51

    LockBackendCommand.php:46

    While we are at it: "redirect to the locking state" does not make any sense to me. I think we should rephrase this.

    DefaultConfigurationDescription.yaml:237

    "must be writable for the PHP user" this may be different in some cases

    DefaultConfigurationDescription.yaml:237

    Directory *must be* relative

    Feature-104126-AddConfigurationSettingToDefineBackendLockingFile.rst:30

    "or" is superfluous
  17. Garvin Hicking uploaded patch set 14 V +1 07/19/24, 19:35

    Apply code review suggestions
  18. Garvin Hicking commented on patch set 14 07/19/24, 19:37

    LockBackendCommand.php:46 Reply

    Good one, I didn't even read that yet. Tried to rephrase this, what do you think?

    DefaultConfigurationDescription.yaml:237 Reply

    Done

    DefaultConfigurationDescription.yaml:237 Reply

    Done

    Feature-104126-AddConfigurationSettingToDefineBackendLockingFile.rst:30 Reply

    Actually I wanted to express three possibilities: * Change your deployment script for the new location * Adjust the setting to point to "the old location" * Use the CLI I see that this was not phrased well and tried another stab, how about that?
  19. core-ci voted on patch set 14 V +1 07/19/24, 19:42 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/61837

  20. Markus Klein commented on patch set 14 CR +1 07/25/24, 16:08

    LockBackendCommand.php:46 Reply

    Done

    Feature-104126-AddConfigurationSettingToDefineBackendLockingFile.rst:30 Reply

    Done
  21. Garvin Hicking commented on patch set 14 07/31/24, 08:04

    Would anyone like to talk about this on T3DD24 to talk about if/how to push this forward?
  22. Helmut Hummel voted on patch set 14 CR +1 V +1 08/07/24, 10:30

  23. Markus Klein voted on patch set 14 V +1 08/07/24, 12:32

  24. Garvin Hicking voted on patch set 14 CR +2 V +2 08/07/24, 12:49

  25. Garvin Hicking merged patch set 15 08/07/24, 12:49