#105753Linkvalidator does not parse domains with Umlauts correctly
Description
Steps to reproduce:
- Create “Link to External Url” page
- Insert Url with Umlaut (“https://gebäudehülle.swiss”)
- Run Linkvalidator Task for the created page
It will find the link “https://geb” and report it as broken.
The reason is the Regexp in UrlSoftReferenceParser does not match Umlauts.
Reviews 3
Files 1
History 813
-
Sybille Peters commented and changed attached linkvalidator_result.png status
NewAccepted <span data-processor="relativetime" data-value="2024-12-30T10:47:02+00:00" > 12/30/24, 10:47 </span> -
Sybille Peters commented <span data-processor="relativetime" data-value="2024-12-30T10:52:35+00:00" > 12/30/24, 10:52 </span>
The following works correctly in RTE:
select bodytext from tt_content where pid=913;<p>mit Umlauten <a href="https://xn--gebudehlle-s5a60a.swiss/">link (umlauts)</a></p> <p><a href="https://xn--gebudehlle-s5a60a.swiss/sdfsddfs">broken link (umlauts)</a></p> <p>Encoded: <a href="https://gebäudehülle.swiss">link (encoded)</a></p> <p><a href="https://gebäudehülle.swiss/sdfsddfs">broken link (encoded)</a></p> -
Sybille Peters commented <span data-processor="relativetime" data-value="2024-12-30T11:09:48+00:00" > 12/30/24, 11:09 </span>
For pages.url, if the already encoded value is used, it is ok:
Not ok, if this is used:
As a workaround you can use the encoded URL (the URL is already converted for me, when I copy it from the browser address bar, for example).
As mentioned above, the problem is in UrlSoftReferenceParser. Possible solutions:
- value is already converted when saved in pages.url (and other fields)
- linkvalidator does not use softref parser to “parse” fields where the content can be used as is (e.g. for pages.url which is of type “input” with “softref” = “url”)
- linkvalidator converts the URL before passing it to the sofref parser (not possible for fields where parsing should be performed FIRST)
- the softref parser UrlSoftReferenceParser converts the URL first. We already have code for this in ExternalLinktype::preprocessUrl :
protected function preprocessUrl(string $url): string { $url = html_entity_decode($url); $parts = parse_url($url); if ($parts['host'] ?? false) { try { $newDomain = (string)idn_to_ascii($parts['host']); if (strcmp($parts['host'], $newDomain) !== 0) { $parts['host'] = $newDomain; $url = HttpUtility::buildUrl($parts); } } catch (\Exception | \Throwable $e) { // ignore error and proceed with link checking } } return $url; } -
Sybille Peters commented <span data-processor="relativetime" data-value="2024-12-30T11:32:08+00:00" > 12/30/24, 11:32 </span>
Also, there is no problem with parsing sys_file_reference.link because it uses the TypolinkSoftReferenceParser.
I am wondering what is the point of having UrlSoftReferenceParser because TypolinkSoftReferenceParser can also parse urls.
Also, pages.url could use type=link and with allowedTypes=‘url’ (though that would not solve the problem with possibly other fields)
pages.url:
- type=input
- softref=urlsys_file_reference.link:
- type=link
- softref=typolink -
Gerrit Code Review pushed patch set 1 for main to change 87609 status
AcceptedUnder Review <span data-processor="relativetime" data-value="2025-01-01T09:57:38+00:00" > 01/01/25, 09:57 </span> -
Sybille Peters changed assigned to Sybille Peters <span data-processor="relativetime" data-value="2025-01-01T11:43:39+00:00" > 01/01/25, 11:43 </span>
-
Gerrit Code Review pushed patch set 2 for main to change 87609 <span data-processor="relativetime" data-value="2025-01-02T09:48:22+00:00" > 01/02/25, 09:48 </span>
-
Gerrit Code Review pushed patch set 3 for main to change 87609 <span data-processor="relativetime" data-value="2025-01-02T09:49:42+00:00" > 01/02/25, 09:49 </span>
-
Gerrit Code Review pushed patch set 1 for 13.4 to change 87618 <span data-processor="relativetime" data-value="2025-01-02T10:30:26+00:00" > 01/02/25, 10:30 </span>
-
Gerrit Code Review pushed patch set 1 for 12.4 to change 87619 <span data-processor="relativetime" data-value="2025-01-02T10:30:38+00:00" > 01/02/25, 10:30 </span>
-
Sybille Peters commented and changed status
Under ReviewResolved done0 %100 % <span data-processor="relativetime" data-value="2025-01-02T10:45:08+00:00" > 01/02/25, 10:45 </span>Applied in changeset commit:22265db3b933a7d2e40625a1f65e0eec73cbc2dd.
-
Benni Mack changed status
ResolvedClosed <span data-processor="relativetime" data-value="2025-12-02T14:31:40+00:00" > 12/02/25, 14:31 </span> -
Benni Mack changed Duplicated by #97949 <span data-processor="relativetime" data-value="2026-09-22T18:21:12+00:00" > 09/22/26, 18:21 </span>
I can reproduce this in v14 (main) and v12 (latest 12.4 branch using page type “link to external page” and the link in pages.url (as described).
I cannot reproduce it with content element CType=“textmedia” and links in tt_content.bodytext (RTE). There, the links are parsed correctly.