#110808Extension Scanner: the mandatory restFiles reference makes rules impossible for undocumented API removals
Description
This is a request for a decision, not a patch proposal. I ran into a structural limit of the Extension Scanner while measuring, and I would like to know which direction the team considers acceptable before writing anything.
The constraint
A scanner rule must reference a changelog file, and this is enforced twice:
- Load time —
AbstractCoreMatcher::validateMatcherDefinitionKeys()throwsInvalidArgumentException1500496068, “Each configuration must have at least one referenced restFiles entry”, whenrestFilesis empty. - Display time —
UpgradeControllerresolves every entry withFinder->name($fileName)againstEXT:core/Documentation/Changelogand throwsRuntimeException1499803909 when the result is not exactly one file.
So a rule can only exist for an API change that has its own .rst file in the core. There is no placeholder, no external link, and no way to express “this was removed, and it is not documented anywhere”.
Why that is a practical limit, measured
The core’s own changelog metadata says that for v14.0, of 99 breaking changes 34 are FullyScanned, 5 PartiallyScanned and 58 NotScanned. This ticket is about a structural reason why that ratio is hard to improve.
I diffed the public API of typo3/cms-core v13.4.35 against v14.3.7 and generated candidate scanner rules from the diff: 636 rules, of which 590 are not covered by the shipped rule set (2424 rules in 13.4.35). Then I tried to give each one a restFiles reference by matching it against a corpus of TYPO3 changelog entries (3824 entries; 787 of them in the 13.0–14.3.x series).
| Result for the 590 rules | count | share |
| symbol matched verbatim in an entry | 30 | 5.1 % |
| only the class name matched | 321 | 54.4 % |
| nothing matched at all | 239 | 40.5 % |
| usable as a restFiles reference | 45 | 7.6 % |
The 321 class-only hits are mostly coincidence. In a sample of six, five did not mention the member anywhere in the matched document — the class simply appears in an unrelated entry. Entering those as restFiles would show users a wrong explanation, which is worse than none. Relaxing the rule to “class matched and the member name appears somewhere in the same document” raises the yield to 93, but also produces mismatches. The defensible statement is a range:
> 45 to 93 of 590 rules — 8 to 16 % — can be given a changelog reference mechanically.
For the 239 with no match I checked whether they might be documented in an older series (deprecated in v12, removed in v14). Across all 3824 entries exactly 4 more appear, and those are coincidental too.
> 235 of 590 rules — 39.8 % — describe API removals that appear in no changelog entry at all.
Limits of the measurement
- The corpus is from 16 July 2026, v14.3.7 is from 8 September 2026. Entries added after that are missing, so the range is a lower bound.
- The corpus was converted to Markdown, so the exact
.rstfile names are not available. The measurement answers “is there an entry”, not “which file”. - The assessment of the class-only hits is a sample: 6 of 321 were read.
- Only
typo3/cms-corewas diffed for the rule generation.
Three ways forward
1. Contribute the 45–90 rules that can be given a hand-checked reference. Honest, small, immediately doable — and it covers roughly a sixth of the gap.
2. Change what restFiles requires. For instance: allow a rule without a reference and let the UI say “no changelog entry”; or allow one shared umbrella entry for undocumented removals; or make the display-time lookup non-fatal. This is the decision I am asking about.
3. Write the ~235 missing changelog entries. Arguably the correct answer, but a different order of magnitude, and not something one person does on the side.
I have no preference to push. Option 3 may well be what the team wants, and then the useful contribution of this measurement is simply the number: 235.
A side observation
The display-time check throws RuntimeException 1499803909 when a referenced file is not found. A renamed or removed .rst therefore does not degrade to “link missing” — it makes the whole scan result view fail. That may deserve its own ticket regardless of what is decided here.
What I can contribute
The diff tooling and the mapping script exist and the measurement is reproducible. I am happy to hand over the data, to hand-check the 45–90 references, or to write the patch once there is a direction.
Reviews 0
No change resolves or names this issue in its commit message, and no comment links one.
Fix this issue
Start the commit message of your patch with this. Add what the patch does, and the versions it goes to in "Releases:". How to create a patch
- Commit message
-
[FEATURE] Extension Scanner: the mandatory restFiles reference makes rules impossible for undocumented API removals Resolves: #110808 Releases: main
Related issues 1
Files 7
History 6
-
Olivier Dobberkau commented and changed attached lab-api-diff.sh attached lab-rulegen.sh attached lab-restfiles-map.py <span data-processor="relativetime" data-value="2026-09-26T20:58:33+00:00" > 09/26/26, 20:58 </span>
-
Georg Ringer changed Related to #110859 <span data-processor="relativetime" data-value="2026-09-30T11:30:38+00:00" > 09/30/26, 11:30 </span>
-
Georg Ringer commented and changed priority
Should haveCould have <span data-processor="relativetime" data-value="2026-11-12T18:59:05+00:00" > 11/12/26, 18:59 </span>Thanks for the measurement and for laying out the options. I could not reproduce the headline number, so here is what I checked, with its limits.
Constraint confirmed
Both checks exist as described:
AbstractCoreMatcher::validateMatcherDefinitionKeys()(1500496068) and theFinderlookup inUpgradeController(1499803909). The side observation is valid as well: a referenced.rstthat is renamed or removed breaks the whole scan result view instead of degrading to a missing link.Spot check of the “undocumented” share
I took all public methods of
typo3/sysext/core/Classesthat exist in v13.4.0 but not in v14.3.7. Only files present in both versions are considered, so removed or moved classes are not part of this count. Each method was then searched by name in the changelog entries of 13.* and 14.*.Result Methods removed public methods 115 of those marked internal@ (method or class)59 named in a changelog entry 47 named in no changelog entry 9 So in this subset about 8 % of the removed public methods have no changelog entry, not about 40 %. Half of the removals were
internal@ and need no scanner rule in the first place. Spot-checked examples that do have an entry but no scanner rule yet:CharsetConverter->conv()(Breaking-105695) andPageRenderer->getCompressJavascript()(Breaking-108055). Both name the method verbatim, so arestFilesreference is possible for them.Limits of this check
- The name match is loose. A hit can be coincidental (e.g.
instance), so 47 is an upper bound and 9 a lower bound. - I did not check whether the 9 moved to a parent class or trait.
- Removed classes, signature changes and other packages are not covered. The difference to the 590 candidates might partly come from there.
Question
Could you tell whether the 235 “no changelog entry” candidates contain
internal@ symbols, and whether the class-only/member matching handles the:php:/:php-short:notation of the changelog entries? In the cases I looked at, the entries did name the member.I would hold off on option 3 (writing about 235 entries) until that number is confirmed. Independent of the outcome, making the display-time lookup non-fatal looks worthwhile on its own.
- The name match is loose. A hit can be coincidental (e.g.
-
Georg Ringer changed status
NewNeeds Feedback <span data-processor="relativetime" data-value="2026-11-12T18:59:39+00:00" > 11/12/26, 18:59 </span> -
Olivier Dobberkau commented <span data-processor="relativetime" data-value="2026-11-13T09:11:52+00:00" > 11/13/26, 09:11 </span>
Thanks Georg, your spot check was right, and checking your two questions against my scripts turned up three defects in my measurement. I withdraw the 235 / 40 % figure. Details below.
1. @internal
lab-api-diff.shonly filtersinternal@ on the class docblock (everything above theclasskeyword). Methods markedinternal@ inside a public class are not filtered. So yes, the 590 and the 235 containinternal@ symbols. Together with the missing inheritance and alias resolution, this is the main reason why the corrected run below finds 113 candidates for cms-core instead of 636.2.
:php:/:php-short:notationThe mapping only counts a member hit if
<FQCN>->memberor<FQCN>::memberappears verbatim. That fails in two common cases:- The Markdown conversion of my corpus turned
:php-short:`\TYPO3\CMS\Core\Charset\CharsetConverter->conv()`intoCharsetConverter->conv(). The FQCN is gone, so Breaking-105695 counts only as a “class” hit, although it names the method. - Many entries list bare method names under the class, e.g. Breaking-108055: “removed from the class
\TYPO3\CMS\Core\Page\PageRenderer:” followed bygetCompressJavascript()and eleven others. The script cannot match that pattern at all.
So the “class only” bucket contains real hits, and entries that use only the short class name land in “nothing matched”. Both effects push the “undocumented” number up.
3. Wrong baseline rule set
The “not covered by the shipped rule set” filter used the rules of v13.4.35 (2425
restFilesblocks), not those of v14.3.7 (2624). The v14 rules added with the removals were therefore counted as missing. One correction to your examples as well:CharsetConverter->convandPageRenderer->getCompressJavascriptboth already have aMethodCallMatcherrule in v14.3.7. They were added in the same commits as the removals (e.g. 0020f3463fa for the CharsetConverter).Corrected measurement for typo3/cms-core
I rebuilt the candidates from git with three fixes:
internal@ on class and method level, v14.3.7 as the rule baseline, and matching against the original.rstfiles. Method inheritance via parent classes, traits and interfaces is resolved, and the v14 class aliases are honoured (e.g.FlexFormService). Basis: public methods and classes oftypo3/sysext/core/Classes, v13.4.35 → v14.3.7. Candidates are removed classes, removed methods, dropped arguments and newly required arguments.Result count public, non-@internal@ candidates 113 already have a rule in v14.3.7 46 (of those, already in the v13.4.35 rule set) (20) no rule: constructors, almost all of DI services 25 no rule: other API 42 The 42 without a rule:
Changelog situation count entry names class and member (incl. bare method list below the class) 17 entry names the member through its interface (Breaking-33747, Breaking-107310) 4 no 13.x/14.x entry names the member 21 14 of the 21 are cache backend setters and getters (
RedisBackend->setHostname()and seven more,Typo3DatabaseBackend->getCacheTable()etc.,NullBackend->setCompression(),AbstractBackend->flushByTags()). They disappeared in the cache typing rework (Breaking-107315), but that entry does not list them. The remaining 7:CharsetConverter->UnumberToChar(),DataHandler->deleteEl(),DataHandler->updateDB(),LocalImageProcessor->processTaskWithLocalFile(), the new required argument ofJavaScriptRenderer->render(), and the exception classesUnknownCharsetExceptionandInvalidSinglePointerFieldException.So for cms-core the honest numbers are: about 21 undocumented removals, not 235, and at most 21 rule candidates with a usable changelog reference. Several will drop out on a hand check:
AbstractFile->copyTo(),moveTo()andrename()only moved down toFile, so a rule matching by method name would flag every validFile->copyTo()call.PageRepository->enableFields()andBackendUserAuthentication->returnWebmounts()already have rules under other keys (see #110807). All 46 existing rules were found by the corrected matching (sanity check), so the matching itself looks sound. Limits: methods and classes only (no constants or properties), cms-core only, and the name match is loose, so the 21 per bucket are approximate. The complete list per symbol is attached as CSV.What this means
- Option 3 (writing ~235 entries) is off the table. At most a handful of entries are missing; the cache setters might deserve a list in Breaking-107315.
- Option 1 becomes concrete: I can prepare rules for the 21 documented candidates, each reference hand-checked. That would be one patch with its own ticket.
- I have to take back the side observation as well, and we both agreed to it too quickly. The integrity check
runTests.sh -s checkExtensionScannerRst(Build/Scripts/extensionScannerRstFileReferences.php, part of the pre-merge pipeline) requires everyrestFilesname to exist exactly once, which is the same condition as the runtime check. The rule set cannot be extended by extensions, andDocumentation/Changelogis part of the dist package. Exception 1499803909 is therefore only reachable when a deployment strips the documentation or the core is patched. I see no need for a separate ticket.
Disclosure: I checked the scripts and drafted this reply with AI assistance (Claude).
- The Markdown conversion of my corpus turned
-
Olivier Dobberkau commented and changed attached cms-core-v13.4.35-v14.3.7-scanner-candidates.csv attached nameonly.py attached analyze.py attached extract-api.php <span data-processor="relativetime" data-value="2026-11-13T09:23:19+00:00" > 11/13/26, 09:23 </span>
Added artifacts.
How the measurement was produced
The three scripts are attached. Two caveats up front: their inline comments are in German, and they were written for one lab, not as a product — I am happy to translate or clean them up if anyone wants to build on them. The method matters more than the scripts, so here is what each one does.
1. The API diff —
lab-api-diff.shlab-api-diff.sh <from> <to> [package …|--all], for examplelab-api-diff.sh v13.4.35 v14.3.7 --allover the 34 system extensions.autoload.psr-4from eachcomposer.jsonto find the namespace and the source directory, instead of assumingClasses/. That is not pedantry:typo3fluid/fluidusessrc/, and the first version of the script skipped it silently.internal@ by looking at the docblock above the class declaration.Two honest limits. It is method-level: removed classes only show up through their methods, and constants, properties and interfaces are not measured at all — so 1287 is a floor, not a total. And the graph step is not essential to the method: the query is just “give me the exported methods with their signatures”, which any static analyser over
nikic/php-parsercan answer. I used an off-the-shelf indexer (codebase-memory-mcp) because it was at hand.2. Rule generation —
lab-rulegen.shlab-rulegen.sh <api-diff.json|--all>. It must run against an installation of the target version, because it resolves every symbol by reflection through the real autoloader — class aliases, traits and inherited methods cannot be resolved reliably from the outside. As attached it shells out viaddev exec php; that is the one line a reviewer would adapt.ClassNameMatcherMethodCallMatcher/MethodCallStaticMatcherMethodArgumentDroppedMatcherBoth halves of that filter are needed. A pure existence check classified two real breaks as false positives:
FlexFormServicesurvives through a class alias but went from three parameters to one.Calibration against the hand-written rules: of the 35 hand-written v14.0 method rules in
TYPO3\CMS\Core, the generator reproduced 34. The single outlier turned out to be a defect in the hand-written rule, not a gap in the generator — that is #110807. Measured precision over 157 symbols: 152 correct, 96.8 %.The generated file carries a header saying
restFilesstill has to be added before anything can go into the core. That is how I ran into the subject of this ticket.3. The changelog mapping —
lab-restfiles-map.pylab-restfiles-map.py --corpus <changelog-root> --rules <rules.json> [--out report.json] [--all-series]— the script behind the numbers in the description. The corpus is one directory per series (13.0, 13.4, 14.0 …);--rulesis a JSON list of{matcher, key}.Three hit levels per rule:
ClassNameMatcher, where the class is the symbol, usually a coincidence for everything elseBy default it searches only the series that can plausibly document the measured jump (v13 announces, v14 removes).
--all-seriesis the cross-check I ran over all 3824 entries: it added exactly 4 further hits, all coincidental, so the restriction costs nothing.What a reviewer would need to redo it
A checkout of both versions, one installation of the target version for the reflection step, and a changelog corpus. The corpus I used was a Markdown conversion, which is why the exact
.rstfile names are missing from my numbers — for realrestFilesentries one would run the same mapping againstEXT:core/Documentation/Changelogof a v14 core, where the file names are whatFinder->name()expects.