95325[BUGFIX] Ignore diacritics when filtering select fields
Commit message
Review
-
Code review CR +2 Done
- Georg Ringer +2
- Stefan Bürk +1
- Benjamin Kott +1
- Benni Mack +1
-
Verified V +2 Done
- Georg Ringer +2
- Stefan Bürk +1
- Benjamin Kott +1
- Benni Mack +1
- core-ci +1 CI
Issues 1
Files 10
- Build/Sources/TypeScript/backend/form-engine/element/extra/select-box-filter.ts+8 −5
- Build/Sources/TypeScript/backend/form-engine/element/select-tree.ts+16 −4
- Build/Sources/TypeScript/backend/tests/form-engine/element/extra/select-box-filter-test.tsnew+215 −0
- Build/Sources/TypeScript/backend/tests/form-engine/element/select-tree-test.ts+45 −0
- Build/Sources/TypeScript/core/tests/utility/text-normalizer-test.tsnew+138 −0
- Build/Sources/TypeScript/core/utility/text-normalizer.tsnew+98 −0
- typo3/sysext/backend/Resources/Public/JavaScript/form-engine/element/extra/select-box-filter.js+1 −1
- typo3/sysext/backend/Resources/Public/JavaScript/form-engine/element/select-tree.js+1 −1
- typo3/sysext/core/Documentation/Changelog/14.3.x/Important-83788-DiacriticInsensitiveSelectFilter.rstnew+70 −0
- typo3/sysext/core/Resources/Public/JavaScript/utility/text-normalizer.jsnew+13 −0
Review log 1620
-
Georg Ringer uploaded patch set 1 08/13/26, 19:17
-
Georg Ringer commented on patch set 1 08/13/26, 19:18
This change is ready for review. -
core-ci voted on patch set 1 V +1 08/13/26, 19:24 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/106675
-
Markus Klein updated the commit message in patch set 2
V +108/14/26, 22:23 -
core-ci voted on patch set 2 V +1 08/14/26, 22:31 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/106733
-
Stefan Bürk commented on patch set 2 CR -1 08/15/26, 18:07
Thanks for finally taking #83788 on, and thanks in particular for the unit test - testing both directions (creme -> Creme brulee and crem -> Cremeschnitte) is exactly the pair that proves needle and haystack are both folded. I also verified there is no match highlighting in this component, so the usual "NFD shifted my highlight offsets" trap does not apply here. Two things hold me back, though. 1. toLocaleLowerCase() - please make it toLowerCase(). Without a locale argument this uses the browser UI locale, which has nothing to do with the backend user language or the language of the labels. On a tr-TR browser, 'I' lowercases to the dotless variant, so an option labelled Import is no longer found by typing import, and Istanbul with the dotted capital I is no longer found by typing istanbul. The old new RegExp(..., 'i') did not have that problem: ECMA-262 Canonicalize for a non-u regex is explicitly locale-independent. So this is a regression, and it is invisible to CI because the test runner's Chrome is not Turkish. toLowerCase() is also what every other filter in Build/Sources/TypeScript/ uses - backend/settings/editor.ts:454, backend/utility/collapse-state-search.ts:90, install/module/upgrade/upgrade-docs.ts:228, form/backend/form-editor-tree.ts:223 - there is currently no other toLocaleLowerCase() in the tree. 2. [BUGFIX] plus Releases: main, 14.3. #83788 is tracked as a Feature, and it was deliberately re-scoped from bugfix to feature by Oliver Hader in the issue itself back in 2018 ("I've modified [...] the scope to be a 'feature' instead of a 'bug fix'"). 14.3 is 14.3.7-dev, i.e. the v14 LTS patch line - this would ship a changed backend search semantic in a patch release, with no changelog: an editor who types Muller today to isolate it from the umlaut spelling gets both rows after the update. And if we do call it a bugfix, then 13.4 (13.4.35-dev, still actively maintained, affected identically) is missing from Releases:. My preference is [FEATURE], Releases: main, plus a Feature-83788-*.rst that also documents the sharp-s / slashed-o / ae / l-stroke limitation you already describe so honestly in the commit message. Three smaller points: * \p{Mn} strips all nonspacing marks, not just diacritics. Thai and Devanagari vowel signs are stripped too, so for those scripts the filter stops narrowing anything (two visually distinct Thai syllables fold to the same base consonant). The explicit combining-diacritical-marks range U+0300-U+036F is the narrower idiom and still covers Latin, Greek, Cyrillic and all six Vietnamese marks. * availableOptions is already cached; please cache the normalized labels next to it instead of running three string passes per option per keystroke. A selectMultipleSideBySide over sys_category easily has a few thousand options. * Consistency with your 95326: that one moves the selectTree filter to includes() but without folding, so after both changes a selectTree and a selectMultipleSideBySide in the same form answer "creme" differently. Since there is no folding helper in the TS sources yet, this might be the moment to put one in @typo3/core/ and use it from both - or at least say in the commit message that the scope is deliberately this one filter. Two questions, no action needed: was Intl.Collator({sensitivity:'base'}) considered (I agree it is unusable, no substring search - worth a sentence for the next reader), and is the Hangul side effect intentional? NFD decomposes precomposed Hangul syllables to jamo, so a syllable now matches inside a different syllable sharing the same leading jamo. Test-wise, if you touch it again: please assert the documented non-goals (strasse must not find the sharp-s spelling, mueller must not find the umlaut spelling) and add one case for the .t3js-formengine-multiselect-filter-dropdown (change) path and for <optgroup> hiding. -
Benjamin Kott uploaded patch set 3
CR -1V +108/28/26, 10:42 -
core-ci voted on patch set 3 V +1 08/28/26, 10:49 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/107328
-
Benjamin Kott commented on patch set 3 08/28/26, 11:00
Reply
done -
Benjamin Kott voted on patch set 3 CR +1 V +1 08/28/26, 12:54
-
Benni Mack commented on patch set 3 CR -1 08/31/26, 09:25
Why do we have a RST for a bugfix? -
Benjamin Kott commented on patch set 3 08/31/26, 13:42
Reply
dont mind, we can drop it, but it is also changed / improved behavior -
Benni Mack commented on patch set 3 CR 0 08/31/26, 13:44
Reply
ideally every bugfix is a changed & improved behavior. Having a new utility (text-normalizer) would IMHO justify the RST though. -
Georg Ringer uploaded patch set 4
CR +1V +109/01/26, 08:22 -
core-ci voted on patch set 4 V +1 09/01/26, 08:30 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/107471
-
Benjamin Kott voted on patch set 4 CR +1 V +1 09/01/26, 16:18
-
Benni Mack voted on patch set 4 CR +1 V +1 09/01/26, 17:04
-
Georg Ringer voted on patch set 4 CR +2 V +2 09/01/26, 19:15
-
Stefan Bürk voted on patch set 4 CR +1 V +1 09/01/26, 19:16
-
Georg Ringer merged patch set 5 09/01/26, 19:29