Review main Merged

80748[BUGFIX] Fix GeneralUtility::validEmail() with spaces in email

Thomas Hohn CR +2 V +2 Normal +67 −1 updated 12/07/23, 10:05

Commit message

`GeneralUtility::validEmail()` uses the package egulias/email-validator
for validating emails. It treats the email `test @testmail.com` as
valid, but triggers a warning that is not caught, since the syntax has
only been marked as deprecated by egulias/email-validator.

A check for warnings of the type CFWSNearAt will now be performed, which
results in the email `test @testmail.com` being marked as invalid.

Review

  • Code review CR +2 Done
    • Oli Bartsch +2
    • Garvin Hicking +1
    • André Buchmann +1
  • Verified V +2 Done
    • Oli Bartsch +2
    • Garvin Hicking +1
    • André Buchmann +1
    • core-ci +1 CI

Issues 1

Files 3

Review log 5469

Show 3544 earlier messages
  1. Thomas Hohn uploaded patch set 1 08/29/23, 10:49

  2. Oliver Klee commented on patch set 1 08/29/23, 10:54

    Please let's cover this with regression tests.
  3. core-ci voted on patch set 1 V +1 08/29/23, 10:55 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/45973

  4. Mathias Brodala commented on patch set 1 V -1 08/29/23, 10:56

    GeneralUtility.php:617

    This would deny valid email addresses like this: "yes spaces are fine"@example.org See https://en.wikipedia.org/wiki/Email_address#Local-part IMO this should be left to the more elaborate mail validators invoked below.
  5. Christian Kuhn commented on patch set 1 08/29/23, 11:00

    shouldn't this be handled in egulias/email-validator instead?
  6. Thomas Hohn commented on patch set 1 08/29/23, 11:04

    Reply

    Will have a look :)
  7. Thomas Hohn uploaded patch set 2 V +1 08/29/23, 13:27

  8. core-ci voted on patch set 2 V +1 08/29/23, 13:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/45983

  9. Stefan Bürk commented on patch set 2 08/29/23, 14:00

    GeneralUtility.php:617 Reply

    I would say we have issues at other places too (regexp grepping of email addresses etc). Albeit not looked up. But technically, right. Quotet it's allowed.
  10. Thomas Hohn commented on patch set 2 08/29/23, 14:11

    Reply

    I still believe that egulias/email-validator doesn't catches all cases

    GeneralUtility.php:617 Reply

    Please have a look at the new test cases - I believe that there are some obvious cases that are not caught by egulias/email-validator
  11. Thomas Hohn uploaded patch set 3 V +1 08/29/23, 14:19

  12. core-ci voted on patch set 3 V +1 08/29/23, 14:26 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/45985

  13. Thomas Hohn commented on patch set 3 08/29/23, 14:57

    GeneralUtility.php:617 Reply

    Done
  14. Thomas Hohn commented on patch set 3 08/29/23, 14:57

    Reply

    Done
  15. Christian Kuhn commented on patch set 3 08/29/23, 15:43

    Reply

    when egulias/email-validator does not catch all cases, it should probably be improved? has an issue been raised? any result?
  16. Thomas Hohn commented on patch set 3 08/29/23, 18:18

    Reply

    Hi! Did some more investigations on these 2 cases: test@ testmail.com => invalid test @testmail.com => valid, but contains a warning object(Egulias\EmailValidator\Warning\CFWSNearAt)#167407 (2) { ["message":protected]=> string(37) "Deprecated folding white space near @" ["rfcNumber":protected]=> int(0) } I can make an issue at egulias/email-validator and see if they wan't to fix it and otherwise we have to handle it on our side.
  17. Christian Kuhn commented on patch set 3 08/29/23, 22:59

    Reply

    so we could check for warnings and return false if any? and if "test @example.com" *is* invalid per rfc, then the library should actively go away from that warning and turn it into an error?!
  18. Thomas Hohn commented on patch set 3 08/30/23, 05:01

  19. Thomas Hohn uploaded patch set 4 V +1 08/30/23, 05:02

  20. core-ci voted on patch set 4 V -1 08/30/23, 05:13 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/46010

  21. Thomas Hohn uploaded patch set 5 V -1 08/30/23, 05:44

  22. core-ci voted on patch set 5 V +1 08/30/23, 05:59 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/46011

  23. Christian Kuhn commented on patch set 5 09/02/23, 06:17

    Reply

    https://github.com/egulias/EmailValidator/issues/374 It looks as if we should simply switch default configuration to use NoRFCWarnings validator instead. Let's try that and see if the new tests are fine with this when the change in GU is reverted?
  24. Thomas Hohn commented on patch set 5 09/02/23, 08:21

    Reply

    Switching to that will at least make these 2 tests fail: 1) TYPO3\CMS\Core\Tests\Unit\Utility\GeneralUtilityTest::validEmailReturnsTrueForValidMailAddress with data set "top level domain only" Failed asserting that false is true. /Volumes/dev/external-repos/t3coredev-13/typo3/sysext/core/Tests/Unit/Utility/GeneralUtilityTest.php:559 /Volumes/dev/external-repos/t3coredev-13/bin/phpunit:122 2) TYPO3\CMS\Core\Tests\Unit\Utility\GeneralUtilityTest::validEmailReturnsTrueForValidMailAddress with data set "quoted @ char" Failed asserting that false is true. /Volumes/dev/external-repos/t3coredev-13/typo3/sysext/core/Tests/Unit/Utility/GeneralUtilityTest.php:559 /Volumes/dev/external-repos/t3coredev-13/bin/phpunit:122 In addition if I run it on the added tests more failures arise. So I think we should keep the RFCValidation and add a custom validator that handles this specific use-case and then wait/hope that it is fixed in the RFCValidation later
  25. Thomas Hohn uploaded patch set 6 V +1 09/06/23, 10:44

  26. Thomas Hohn commented on patch set 6 09/06/23, 10:48

    Reply

    Instead of creating a custom Validator, which would be a breaking change since the validator is specified in the defaultConfiguration - a check for the specific warning, with a longer explanation has been added.
  27. core-ci voted on patch set 6 V +1 09/06/23, 10:52 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/46283

  28. Oliver Klee voted on patch set 6 V +1 09/06/23, 11:16

  29. Georg Ringer commented on patch set 6 09/07/23, 13:02

    I strongly suggest to add an important.rst as it changes the behaviour for current LTS versions!
    caught

    GeneralUtility.php:637

    if email is valid, check ... (add a missing ,)
  30. Georg Ringer voted on patch set 6 CR +1 09/07/23, 13:02

  31. André Buchmann commented on patch set 6 09/07/23, 16:03

    GeneralUtility.php:638

    it as an
  32. Thomas Hohn commented on patch set 6 09/10/23, 10:50

    Reply

    Ack

    GeneralUtility.php:638 Reply

    Ack
  33. Thomas Hohn uploaded patch set 7 CR +1 V +1 09/10/23, 10:58

  34. Thomas Hohn uploaded patch set 8 09/10/23, 11:04

  35. Thomas Hohn commented on patch set 7 09/10/23, 11:04

    GeneralUtility.php:637 Reply

    Done
  36. Oliver Klee voted on patch set 8 V +1 09/10/23, 11:05

  37. core-ci voted on patch set 7 V +1 09/10/23, 11:05 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/46439

  38. core-ci voted on patch set 8 V +1 09/10/23, 11:11 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/46440

  39. Thomas Hohn commented on patch set 9 V +1 09/11/23, 04:53

    Published edit on patch set 8.
  40. core-ci voted on patch set 9 V +1 09/11/23, 05:03 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/46450

  41. André Buchmann voted on patch set 9 V +1 CR +1 09/11/23, 15:40

  42. Simon Schaufelberger uploaded patch set 10 CR +1 V +1 09/22/23, 08:27

  43. Simon Schaufelberger uploaded patch set 11 09/22/23, 08:29

  44. Simon Schaufelberger uploaded patch set 12 09/22/23, 08:31

  1. core-ci voted on patch set 10 V +1 09/22/23, 08:37 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/47058

  2. core-ci voted on patch set 11 V -1 09/22/23, 08:40 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/47059

  3. core-ci voted on patch set 12 V +1 09/22/23, 08:41 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/47060

  4. Oliver Klee updated the commit message in patch set 13 V +1 09/22/23, 09:06

  5. Oliver Klee voted on patch set 13 V +1 09/22/23, 09:06

  6. core-ci voted on patch set 13 V +1 09/22/23, 09:20 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/47068

  7. André Buchmann voted on patch set 13 V +1 CR +1 09/22/23, 15:21

  8. Josef Glatz voted on patch set 13 CR +1 09/28/23, 04:42

  9. Oliver Klee voted on patch set 13 CR +1 10/11/23, 13:04

  10. Garvin Hicking commented on patch set 13 V +1 CR +1 10/11/23, 13:40

    I tried to offer some rephrasing and minor typo. (If in the future the library offers to remove the warning and use a "isInvalid" for mails like these instead, hopefully we remember this patch and remove the special treatment again :D ) - is there some kind of notice board/collection for "workarounds" in external dependencies that may be revised from time to time?)

    GeneralUtility.php:633

    Maybe either "time being" or easier make it shorter by "Currently, "?

    Important-101776-1694342579-ChangeInEmailValidation.rst:6

    Use of mid-sentence "Email" and "email" is not completely consistent here? Also maybe include that validation is specifically about "space(s) before @"? Suggestion: Email validation in GeneralUtility::validEmail() now rejects spaces before "@"

    Important-101776-1694342579-ChangeInEmailValidation.rst:22

    Rephrasing suggestion: This library treats an email adress like "email @example.com" with a space before the "@" character as valid, but issues a warning, which has previously not been caught by TYPO3. Warnings like these are defined as "deviations from the RFC that in a broader interpretation are accepted." In the context of TYPO3, such non-RFC mail adresses shall be rejected. Thus, this specific warning (:php:`CFWSNearAt`) will now be caught, and the warning is turned into an invalidation of the given mail address. This will have the effect, that if integrators previously accepted email addresses formatted like these, validation will now fail (as the RFC implies).
  11. Thomas Hohn commented on patch set 13 10/12/23, 04:07

    Reply

    Yes thats unfortunately the case that "someone" has to look it up or hope the maintainer remembers to notify me if he changes it :)
  12. Thomas Hohn uploaded patch set 14 CR +1 V +1 10/12/23, 04:33

  13. Thomas Hohn commented on patch set 14 10/12/23, 04:34

    GeneralUtility.php:633 Reply

    Done

    Important-101776-1694342579-ChangeInEmailValidation.rst:6 Reply

    Done

    Important-101776-1694342579-ChangeInEmailValidation.rst:22 Reply

    Done
  14. core-ci voted on patch set 14 V +1 10/12/23, 04:40 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/47699

  15. Garvin Hicking voted on patch set 14 V +1 CR +1 10/12/23, 05:23

  16. Oliver Klee voted on patch set 14 V +1 CR +1 10/12/23, 08:04

  17. Markus Klein voted on patch set 14 CR +1 12/05/23, 16:56

  18. Christian Kuhn commented on patch set 14 12/05/23, 21:47

    imho far too late at least for v11: not critical, not security relevant, imho too risky, especially with that important.rst. if anyone really needs this in v11, it should backport manually to affected instance.
  19. Thomas Hohn commented on patch set 14 12/06/23, 05:13

    Reply

    Hi @lolli@schwarzbu.ch - will remove 11 😊
  20. Thomas Hohn uploaded patch set 15 CR +1 V +1 12/06/23, 05:14

  21. core-ci voted on patch set 15 V +1 12/06/23, 05:20 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/50400

  22. Garvin Hicking voted on patch set 15 CR +1 V +1 12/06/23, 06:19

  23. André Buchmann voted on patch set 15 CR +1 V +1 12/06/23, 07:10

  24. Oli Bartsch voted on patch set 15 CR +2 V +2 12/07/23, 09:57

  25. Oli Bartsch merged patch set 16 12/07/23, 10:05