Issue Bug Closed

#97949Linkvalidator does not handle german umlauts correctly

Kai Strecker 07/15/22, 13:25 updated 09/22/26, 18:21

Description

Steps to reproduce:
  1. Create a page of type “External link”.
  2. Set for example “https://bäder.friedrichshafen.de/wellenfreibad/” as url.

Result: The linkvalidator thinks the url is “https://b”, which leads to the error “Couldn’t resolve host.”.

This error also occurs, if the url is in a rte field.

Reviews 2

History 1017

  1. Georg Ringer commented and changed status New Needs Feedback <span data-processor="relativetime" data-value="2022-07-15T19:03:05+00:00" > 07/15/22, 19:03 </span>

    thanks for creating the issue. I checked this in latest 11 and 12-dev and can’t reproduce it. can you please retest on a clean installation again? that would be great!

  2. Oliver Bartsch commented and changed status Needs Feedback Accepted <span data-processor="relativetime" data-value="2022-07-15T20:32:32+00:00" > 07/15/22, 20:32 </span>

    I was able to reproduce this on current main.

  3. Stefan Bürk commented <span data-processor="relativetime" data-value="2022-07-31T01:14:04+00:00" > 07/31/22, 01:14 </span>

    Not exact the same, but kind of related:

    There is a patch pending (currently) which refactors the url softref parser.
    I added a test case for this issue outcommented to that patch, which shows
    that current and refactored behaviour do not handles urls with german umlauts
    properly.

    https://review.typo3.org/c/Packages/TYPO3.CMS/+/75200

    Also this report is not for the softref parser (or is this involved in this ?) but
    for a “external link” page type.

    I think, german umlaut domain support should be evaluated to all the link stuff in
    core.

  4. Albrecht Köhnlein commented <span data-processor="relativetime" data-value="2022-08-05T18:51:35+00:00" > 08/05/22, 18:51 </span>

    I can confirm the problem and will try to dig into it. #t3dd22

  5. Gerrit Code Review pushed patch set 1 for main to change 75377 status Accepted Under Review <span data-processor="relativetime" data-value="2022-08-05T20:17:47+00:00" > 08/05/22, 20:17 </span>

  6. Stefan Bürk commented <span data-processor="relativetime" data-value="2025-04-10T07:51:26+00:00" > 04/10/25, 07:51 </span>

    Abandond the pending patch, the technicall approach was not reasonable.
    Issue still stays valid and is not closed to allow someone come up with
    another (working) solution for the issue.

  7. Benni Mack commented and changed status Accepted Closed <span data-processor="relativetime" data-value="2026-09-22T18:21:03+00:00" > 09/22/26, 18:21 </span>

    I re-checked this on current main and I believe the underlying bug is already fixed separately.

    The original report was about UrlSoftReferenceParser (and the resulting external link check) truncating a URL at the first non-ASCII character, e.g. https://bäder.friedrichshafen.de/… became https://b. That was caused by the parser’s regex using ASCII a-z character classes for the domain part.

    This was fixed independently in UrlSoftReferenceParser by replacing the ASCII letter classes with the unicode \p{L} property with issue #105753 (also for 12+)

    I verified this against the exact example from this report (https://bäder.friedrichshafen.de/wellenfreibad/) and the domain is now matched in full instead of being cut off after “b”.

    Additionally:
    - ExternalLinktype::preprocessUrl() already converts the host to punycode via idn_to_ascii() before the actual HTTP request is made (in place since #55155), so the resolved host is correctly xn—bder-loa.friedrichshafen.de and the request no longer fails with “Couldn’t resolve host”.
    - The “External URL” page doktype no longer uses the old pages.url field at all. It now uses pages.link (type => link), which goes through TypolinkSoftReferenceParser + TypoLinkCodecService::decode() (CSV-based splitting, no restrictive regex), so that path was never affected by the original regex bug in the first place.

    Given this, I think #97949 can be closed as a duplicate/consequence of #105753 (and the pre-existing punycode handling from #55155).