Review main Merged

94563[BUGFIX] Set short cache lifetime for robots.txt in root-htaccess

Stig Nørgaard Færch CR +2 V +2 Easy +12 −0 updated 07/09/26, 11:55

Commit message

The shipped root-htaccess template configures mod_expires with a
one-month default and then overrides the typically per-request
content types (text/html, application/json, application/xml, …)
back to "access plus 0 seconds". text/plain is not in that list,
so a dynamically generated robots.txt inherits ExpiresDefault and
is served with a one-month max-age.

For a control file that is meant to be short-lived this is the
wrong default. Without a shared cache it is mostly a hygiene
issue, but behind a CDN or reverse proxy a stale robots.txt can
be served for up to a month after the rules change.

Add a robots.txt-specific rule that unsets Expires and sends a
one-hour, revalidating Cache-Control header. Scoping it to the
file instead of the whole text/plain type leaves caching of
static .txt assets untouched.

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Georg Ringer +1
    • Stig Nørgaard Færch +1
    • Oliver Klee +1
    • Simon Schaufelberger +1
  • Verified V +2 Done
    • Benni Mack +2
    • Georg Ringer +1
    • Stig Nørgaard Færch +1
    • Simon Schaufelberger +1
    • core-ci +1 CI

Issues 1

Files 1

Review log 89

  1. Stig Nørgaard Færch uploaded patch set 1 06/24/26, 10:57

  2. core-ci voted on patch set 1 V +1 06/24/26, 11:05 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/103805

  3. Georg Ringer voted on patch set 1 CR +1 V +1 07/05/26, 18:41

  4. Simon Schaufelberger voted on patch set 1 CR +1 V +1 07/07/26, 00:29

  5. Oliver Klee voted on patch set 1 CR +1 07/07/26, 07:31

  6. Stig Nørgaard Færch voted on patch set 1 V +1 07/07/26, 07:36

  7. Stig Nørgaard Færch voted on patch set 1 CR +1 07/07/26, 07:36

  8. Benni Mack voted on patch set 1 CR +2 V +2 07/09/26, 11:55

  9. Benni Mack merged patch set 2 07/09/26, 11:55