Issue Task Closed

#110094Default .htaccess: dynamically-served text/plain (e.g. robots.txt) inherits ExpiresDefault "access plus 1 month"

Stig Nørgaard Færch 06/24/26, 10:22 updated 09/08/26, 10:03

Description

Question / observation

The shipped default .htaccess template configures mod_expires with a 1-month default and then overrides specific content types. Dynamically generated text/plain responses — robots.txt being the canonical example — are not in the override list, so they inherit ExpiresDefault "access plus 1 month".

I don’t have a hard in-house blocker; I’m raising this as a question because the behaviour surprised me and the omission looks like it might be unintentional.

File: typo3/sysext/install/Resources/Private/FolderStructureTemplateFiles/root-htaccess

<IfModule mod_expires.c>
    ExpiresActive On
    ExpiresDefault                 "access plus 1 month"
    ...
    ExpiresByType text/html        "access plus 0 seconds"   # dynamic HTML: not cached
    ExpiresByType application/json "access plus 0 seconds"   # dynamic JSON: not cached
    ExpiresByType application/xml  "access plus 0 seconds"
    ExpiresByType text/xml         "access plus 0 seconds"
    # (no ExpiresByType text/plain)  -> falls back to ExpiresDefault = 1 month
</IfModule>

Why it’s noticeable

The list already neutralises the cache for content types that are typically generated per-request (text/html, application/json, application/xml, …). text/plain is the one common dynamic type left out, and a dynamically served robots.txt is exactly that. Result on Apache + mod_expires:

Cache-Control: max-age=2592000
Expires: <now + 1 month>

For a control file that’s meant to be short-lived, a 1-month TTL is the wrong default. With no shared cache it’s mostly a hygiene issue (browsers honour it); but behind a CDN or reverse-proxy cache it becomes an active problem — a stale robots.txt can be served for up to a month after the rules change.

Steps to reproduce

  1. Vanilla TYPO3 on Apache with mod_expires enabled (default .htaccess).
  2. Serve a text/plain response — e.g. a robots.txt produced by a route or extension (anything that doesn’t hit a more specific ExpiresByType).
  3. curl -I https://<host>/robots.txt → max-age=2592000, Expires ~1 month.

The question

Is text/plain defaulting to 1 month intentional, or an oversight in an otherwise dynamic-aware list?

If it’s worth changing, a few options (no strong preference):

  • Add ExpiresByType text/plain "access plus 0 seconds" — symmetric with the other dynamic types. Trade-off: also affects static .txt assets, which some may want cached.
  • A robots.txt-specific rule instead, e.g. <Files "robots.txt"> … </Files>, leaving other .txt untouched.
  • Leave as-is and simply document the behaviour.

Currently I work around it per-vhost:

<IfModule mod_headers.c>
    <Files "robots.txt">
        Header unset Expires
        Header set Cache-Control "public, max-age=3600, must-revalidate"
    </Files>
</IfModule>

Reviews 1

History 23

  1. Stig Nørgaard Færch commented and changed status Under Review Resolved done 0 % 100 % <span data-processor="relativetime" data-value="2026-07-09T12:00:11+00:00" > 07/09/26, 12:00 </span>

    Applied in changeset commit:44734a1e72cbcdf64ed250f32bdeeb5f7da363f4.