Review main Merged

72057[BUGFIX] Fix link parsing in LinkHandler

Sybille Peters CR +2 V +2 Hard +278 −4 updated 09/23/26, 12:30

Commit message

When link parsing is performed within Linkvalidator,
several SoftReferenceParsers are applied on the
existing content. More than one may match, e.g.
external URL and typolink.

This resulted in problems where 2 links were extracted
from the content even though only one existed. This
happened specifically, if the anchor text (link text)
contained an URL.

An earlier version of this patch tried to fix this by
replacing the field content with an empty string once
any SoftReferenceParser had matched something, via
SoftReferenceParserResult::createWithoutMatches(). Since
that method is static and ignores the parser result it is
called on, this always yielded an empty string, which in
turn made all other configured SoftReferenceParsers for
that field silently stop finding any further, unrelated
links (e.g. a distinct URL used elsewhere in the same
field), as already pointed out during review.

This patch instead tracks the actual link targets already
found for a given table/field/record combination across
all its configured SoftReferenceParsers and skips adding
a result for a link target that was already found by a
previous parser. This still keeps the original field
content intact for every parser, so unrelated links are
found as before, while a link that is matched more than
once (e.g. because its URL is also used as its own link
text) is only reported once.

Additionally, several tests are added for
LinkAnalyzer::analyzeRecord(), as
tests did not yet exist for this method.

Review

  • Code review CR +2 Done
    • Benni Mack +2
    • Wouter Wolters +1
    • Simon Schaufelberger +1
  • Verified V +2 Done
    • Benni Mack +2
    • Oliver Klee +1
    • Wouter Wolters +1
    • Simon Schaufelberger +1
    • core-ci +1 CI

Issues 1

Files 2

Review log 4871

Show 3146 earlier messages
  1. Sybille Peters uploaded patch set 1 11/05/21, 09:00

  2. Sybille Peters wrote 11/05/21, 09:00

    Topic set to Linkvalidator
  3. core-ci voted on patch set 1 V -1 11/05/21, 09:19 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18932

  4. Sybille Peters uploaded patch set 2 11/07/21, 13:09

  5. Sybille Peters commented on patch set 2 11/07/21, 13:11

    Without the fix in LinkAnalyzer, the tests labeled "regression" will fail. The other tests are there to ensure we don't break anything with this patch and in the future.
  6. core-ci voted on patch set 2 V -1 11/07/21, 13:12 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18990

  7. Sybille Peters updated the commit message in patch set 3 11/07/21, 13:16

  8. Sybille Peters uploaded patch set 4 11/07/21, 13:17

  9. Sybille Peters uploaded patch set 5 11/07/21, 13:19

  10. core-ci voted on patch set 3 V -1 11/07/21, 13:20 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18991

  11. Sybille Peters uploaded patch set 6 11/07/21, 13:20

  12. core-ci voted on patch set 4 V -1 11/07/21, 13:21 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18992

  13. Sybille Peters updated the commit message in patch set 7 11/07/21, 13:22

  14. core-ci voted on patch set 5 V -1 11/07/21, 13:23 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18994

  15. core-ci voted on patch set 6 V -1 11/07/21, 13:23 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18995

  16. core-ci voted on patch set 7 V -1 11/07/21, 13:25 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18996

  17. Sybille Peters uploaded patch set 8 11/07/21, 13:44

  18. core-ci voted on patch set 8 V +1 11/07/21, 14:01 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/18998

  19. Nikita Hovratov commented on patch set 8 11/07/21, 15:08

    LinkAnalyzer.php:339

    This will always return an empty string. And $parserResult->getContent() would return: <p><a class="external" href="{softref:3fb9fc1bac7b0720ce1101b7671daf66}">https://blah</a></p>; Which would still resolve in two matches. After thinking a bit about this, I'm not sure if this is an issue at all. These are two different results: One for the href content (typolink) and one for the tag content (url). If there are several same links inside of a text, they should all be found. Don't know if we should treat this specific (but common) case differently.
  20. Sybille Peters commented on patch set 8 11/07/21, 15:14

    LinkAnalyzer.php:339 Reply

    <a href="http://blah">http://blah</a>; was extracted as 2 links and 2 broken link records are created by linkvalidator. This is wrong. The (new) tests failed without my change in LinkAnalyzer. With my change, they are ok. But, as already mentionend it might be better so solve this in the parser - as the link text should probably not be parsed for links - I don't know - this is a matter of how you define it, but for linkvalidator it does not make sense.
  21. Nikita Hovratov commented on patch set 8 11/07/21, 15:29

    LinkAnalyzer.php:339 Reply

    The parser works correctly, I would argue. Everything not inside html attributes and is an URL will match. We should probably first define, which parser results should be displayed. I guess normal URLs (not typolink), should be displayed, too? Like it is already. Then only linktext should be ignored? If yes, this would be too complicated for the parser. Imagine something like this: <p>Lorem Ipsum<br>https://foo.bar.baz/abc/def/ghi/.</p>; For the parser there is no difference between a-tags and other tags. Regex for html is nasty and should be avoided anyway. I think your tests are a false-positive. Try to add more than one link.
  22. Sybille Peters commented on patch set 8 11/07/21, 16:09

    LinkAnalyzer.php:339 Reply

    I think, ideally, linkvalidator should have the same behaviour as when the page is rendered - if something is rendered as a link, it should be detected as link by linkvalidator (this is a bit simplified, but generally what we want). For tt_content.bodytext we have softref="typolink_tag,email[subst],url". I don't know if it is so great to extract "simple" URLs anyway. IMHO this is unprecise and the links should be "real" links in the rich text. But this goes too far - as I am unfamiliar with the URL handling in general - all I can say is there are these requirements: - linkvalidator should only detect links which are handled as links elsewhere, e.g. when content is rendered in FE. The rest is strictly speaking not a link (for linkvalidator) and should be ignored - links should only be detected once. My solution of removing the link that is already parsed and extracted - in analyzeRecord() - I found that a good way to avoid parsing twice. What we could also do is remove "url" from the softref list, but this might have more unintended side effects. I don't feel so comfortable tampering with that. In any case, we probably should add more tests - that could also be done in a separate patch. But there are tests with more than one link, see https://review.typo3.org/c/Packages/TYPO3.CMS/+/72057/8/typo3/sysext/linkvalidator/Tests/Unit/LinkAnalyzerTest.php#241 In any case, I would wait what is decided is the best way to go here.
  23. Stefan Bürk commented on patch set 8 11/07/21, 19:20

    LinkAnalyzer.php:339 Reply

    Nikita, I do not think that that is correct. In the past there was an option for auto-linkink uri styled text, meaning that if there were links in the text, but no surrounding a tag, it has rendered that as real a-link tags, with the "uri" as link lable. But only then. Not sure/checked, but I think this should be still possible, as it should be a function of parseFunc. And this is no other. If there is a surrounding a-tag, the text between the open and close tag should not reconizied as seperate url/uri to be processed, but instead handle it as "pure label text" and ignore it for passing. Otherwise you are right, uri styled parts should be regonized and checked/analyzed. But not if it is surrounded by the a-tag. Not looked into it if this is a thing which need to be splitet in Link-Analyser or in the parser, but generally it should be handled "correctly" - meaning ignored if surrounded by a-tag.
  24. Benni Mack commented on patch set 8 11/09/21, 07:18

    LinkAnalyzer.php:339 Reply

    The feature ya'll referring to is called "makeLinks" in ContentObjectRenderer.
  25. Benni Mack updated the commit message in patch set 9 11/18/21, 10:45

  26. Sybille Peters rebased onto patch set 10 11/18/21, 10:50

  27. core-ci voted on patch set 9 V +1 11/18/21, 11:32 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/19464

  28. core-ci voted on patch set 10 V +1 11/18/21, 12:20 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/19486

  29. Benni Mack wrote 11/29/21, 10:18

    Change destination moved from master to main
  30. Benni Mack updated the commit message in patch set 11 11/29/21, 14:07

  31. Sybille Peters commented on patch set 11 12/03/21, 10:01

    My proposal is to fix the problem in the core (with separate patch). Extracting an URI which is only in the link text (anchor text) is not correct IMHO. Please enter this in bodytext (as source): <a href="#">http://t3coredev/cde</a>;. After checking, a broken link will be displayed by linkvalidator. Makes no sense. That is problem by sofref parser, right? BUT, in the frontend we do not have this problem when rendering. Link is rendered correctly. Must be handled differently there. I also wrote elsewhere that I find extracting URLs directly from a rich text as links can be problematic (and is) in several cases. I would change this to only treat "real" links as links in bodytext. The patch here can still be useful as it provides test. Also, still think removing the already parsed links from text is useful. Putting this patch on hold for now (setting to WIP). If necessary to discuss maybe talk directly on Slack?
  32. Sybille Peters marked it work in progress 12/03/21, 10:01

  33. Sybille Peters rebased onto patch set 12 01/06/22, 16:40

  34. core-ci voted on patch set 12 V +1 01/06/22, 16:56 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/21721

  35. Sybille Peters rebased onto patch set 13 V +1 02/09/24, 09:22

  36. Sybille Peters commented on patch set 13 02/09/24, 09:23

    This change is ready for review.
  37. Sybille Peters voted on patch set 13 V +1 02/09/24, 09:25

  38. core-ci voted on patch set 13 V -1 02/09/24, 09:30 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/52943

  39. Sybille Peters uploaded patch set 14 V +1 02/09/24, 09:41

  40. core-ci voted on patch set 14 V -1 02/09/24, 09:46 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/52945

  41. Sybille Peters uploaded patch set 15 V -1 02/09/24, 10:16

  42. core-ci voted on patch set 15 V +1 02/09/24, 10:21 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/52946

  43. Oliver Hader updated the commit message in patch set 16 V +1 10/18/24, 14:07

  44. Oliver Hader rebased onto patch set 17 12/11/24, 14:35

  45. core-ci voted on patch set 17 V -1 12/11/24, 14:46 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/70949

  46. Benni Mack updated the commit message in patch set 18 V -1 03/28/25, 12:29

  1. Benni Mack rebased onto patch set 19 03/28/25, 12:29

  2. core-ci voted on patch set 18 V -1 03/28/25, 12:48 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76585

  3. core-ci voted on patch set 19 V -1 03/28/25, 12:48 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76586

  4. Stefan Bürk uploaded patch set 20 V -1 03/28/25, 16:16

  5. core-ci voted on patch set 20 V +1 03/28/25, 16:25 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/76604

  6. Stefan Bürk rebased onto patch set 21 V +1 04/07/25, 23:24

  7. core-ci voted on patch set 21 V +1 04/07/25, 23:29 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/77168

  8. Benni Mack wrote 04/21/26, 14:59

    Removed cc Benni Mack.
  9. Benni Mack updated the commit message in patch set 22 V +1 04/22/26, 06:51

  10. core-ci voted on patch set 22 V -1 04/22/26, 11:31 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/100817

  11. Benni Mack rebased onto patch set 23 V -1 09/11/26, 21:50

  12. core-ci voted on patch set 23 V +1 09/11/26, 21:59 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/108322

  13. Benni Mack uploaded patch set 24 V +1 09/22/26, 18:47

  14. Benni Mack wrote 09/22/26, 18:47

    Change message removed by: Benni Mack

    LinkAnalyzer.php Reply

    Addressed in patch set 24: this line no longer exists. Instead of nulling out the field content after the first match (which is what createWithoutMatches() effectively did here, since it's static and ignores $parserResult), analyzeRecord() now tracks the tokenValue of every link already found for the current table/field/record across all configured SoftReferenceParsers, and analyzeLinks()/analyzeTypoLinks() skip adding a result for a tokenValue that was already found by an earlier parser. Each parser still sees the full, original field content, so a distinct URL elsewhere in the same field is still found (see the new 'Deduplicate link with URL as anchor text but still find an unrelated plain URL' test case). Resolving this thread - please reopen if you still see a problem.
  15. Benni Mack voted on patch set 24 CR +1 V +1 09/22/26, 18:49

  16. Wouter Wolters commented on patch set 24 09/22/26, 18:53

    tests
  17. core-ci voted on patch set 24 V -1 09/22/26, 18:58 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/109007

  18. Benni Mack updated the commit message in patch set 25 CR +1 V +1 09/22/26, 19:07

  19. Benni Mack commented on patch set 25 CR +1 V +1 09/22/26, 19:07

    Reply

    Done
  20. Wouter Wolters voted on patch set 25 CR +1 V +1 09/22/26, 19:08

  21. core-ci voted on patch set 25 V +1 09/22/26, 19:21 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/109010

  22. Simon Schaufelberger voted on patch set 25 CR +1 V +1 09/22/26, 19:49

  23. Oliver Klee voted on patch set 25 V +1 09/23/26, 07:34

  24. Benni Mack voted on patch set 25 CR +2 V +2 09/23/26, 12:30

  25. Benni Mack merged patch set 26 09/23/26, 12:30