86840[TASK] Introduce test of error-handling for FormEngine's basic-inputs
Commit message
Review
-
Code review CR +2 Done
- Andreas Kienast +2
- Christian Kuhn +1
-
Verified V +2 Done
- Andreas Kienast +2
- Christian Kuhn +1
- core-ci +1 CI
Issues 1
Files 4
- typo3/sysext/core/Tests/Acceptance/Application/FormEngine/AbstractElementsBasicCest.php2+39 −0
- typo3/sysext/core/Tests/Acceptance/Application/FormEngine/ElementsBasicEmailCest.php13+45 −0
- typo3/sysext/core/Tests/Acceptance/Application/FormEngine/ElementsBasicInputSimpleCest.php+236 −0
- typo3/sysext/core/Tests/Acceptance/Application/FormEngine/ElementsBasicNumberCest.php+45 −0
Review log 2840
Show 915 earlier messages
-
Ayke Halder uploaded patch set 1 10/30/24, 21:59
-
core-ci voted on patch set 1 V -1 10/30/24, 22:07 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/68869
-
Ayke Halder uploaded patch set 2
V -110/30/24, 22:17 -
Ayke Halder uploaded patch set 3 10/30/24, 22:19
-
core-ci voted on patch set 2 V -1 10/30/24, 22:30 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/68870
-
core-ci voted on patch set 3 V -1 10/30/24, 22:31 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/68872
-
Ayke Halder uploaded patch set 4
V -110/30/24, 22:53 -
core-ci voted on patch set 4 V -1 10/30/24, 22:59 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/68874
-
Ayke Halder uploaded patch set 5
V -110/30/24, 23:01 -
core-ci voted on patch set 5 V +1 10/30/24, 23:08 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/68876
-
Andreas Kienast commented on patch set 5 CR -1 10/31/24, 09:58
-
Ayke Halder uploaded patch set 6
CR -1V +110/31/24, 17:15 -
Ayke Halder commented on patch set 6 10/31/24, 17:20
AbstractElementsBasicCest.php:58ReplyAcknowledged, the variables are renamed to better describe their function. -
core-ci voted on patch set 6 V +1 10/31/24, 17:21 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/68935
-
Andreas Kienast uploaded patch set 7
V +111/19/24, 10:27
-
Andreas Kienast voted on patch set 7 CR +1 V +1 11/19/24, 10:27
-
core-ci voted on patch set 7 V +1 11/19/24, 10:33 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/69683
-
Benjamin Franzke commented on patch set 7 CR -1 11/19/24, 14:36
ElementsBasicEmailCest.php:72Can we ideally merge this with `inputBehaviorVariants` in order to make clear what the actualy sequence of the test is.ElementsBasicEmailCest.php:73Optional: Also I feel like we actually want to test error-handling only, and additionally use `correct@example.com` to be able to cover this via `runInputFieldTest`. Maybe we should fork here and have a separate `testInputFieldSequence` (which only save's if actually requested)ElementsBasicEmailCest.php:75Alternative idea: `inputSequence` WDYT? -
Ayke Halder commented on patch set 7 11/19/24, 16:51
ElementsBasicEmailCest.php:72ReplyThere can not be an input-field with an open error at the end of the test-sequence - as this might result in follow-up tests not working as intended. How can we ensure that the last test-sequence of ~~inputBehaviorVariants~~ `'inputSequence'` always has `'expectError' => false` defined? Maybe implement something like the following code? ``` // In the end we try to save the tested record: so there is no error allowed for the last test in a test-sequence. $I->assertEquals( false, $inputSequence[array_key_last($inputSequence)]['expectError'], 'Last test in sequence is not allowed to expect an error: Implement a proper last test without an error.' ); ``` This one field-test defined outside of the test-sequence ensured that there is no error at the end of the test-sequence and the tested CE can be saved: * This ensures that the record has no 'hidden' problem anymore and really can be saved. * It also ensures that follow-up tests run in a clean environment.ElementsBasicEmailCest.php:73ReplyI'd like to really save any record after checking the error-handling sequence, to make sure that there are no 'hidden issues' left. One known 'hidden issue' is about all these browser-native input-field validations: Failing browser-native validation right now does not allow to save the record - even though there are absolutely no errors shown to the user.ElementsBasicEmailCest.php:75ReplyI'm fine with `inputSequence` too. (As long as we don't mix it up with `sequence test` or similar, as googling this turns up `sequential hypothesis testing` - which is not what our test implements here: https://en.wikipedia.org/wiki/Sequential_analysis ) -
Andreas Kienast commented on patch set 7 11/20/24, 08:18
ElementsBasicEmailCest.php:72Reply> as this might result in follow-up tests not working as intended I don't quite understand this. Each new test (case) re-opens the form and starts from the beginning. -
Andreas Kienast uploaded patch set 8
CR +1V +111/20/24, 09:45 -
core-ci voted on patch set 8 V -1 11/20/24, 09:51 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/69778
-
Andreas Kienast uploaded patch set 9
V -111/20/24, 10:00 -
core-ci voted on patch set 9 V +1 11/20/24, 10:07 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/69779
-
Ayke Halder commented on patch set 9 11/20/24, 13:39
ElementsBasicEmailCest.php:72ReplyYour refactoring with patchset 9 looks good! https://review.typo3.org/c/Packages/TYPO3.CMS/+/86840/7..9 I'd like to implement the possibility to have the last test of an input-sequence failing (`expectedError = true`). I'll change one of the test-sequences to have the last test with `expectedError = true` and see what happens.ElementsBasicEmailCest.php:75ReplyThis is changed to `testSequence` with patchset 9: https://review.typo3.org/c/Packages/TYPO3.CMS/+/86840/7..9 -
Ayke Halder uploaded patch set 10
V +111/21/24, 10:16 -
core-ci voted on patch set 10 V +1 11/21/24, 10:23 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/69911
-
Andreas Kienast commented on patch set 10 CR +1 V +1 11/21/24, 10:27
ElementsBasicEmailCest.php:72ReplyDoneElementsBasicEmailCest.php:73ReplyDone -
Ayke Halder commented on patch set 10 11/21/24, 10:31
I implemented an optional `comment` to better discern the separate test-sequences inside one data-provider. This implements full checks for all simple text field validations with `min`/`max`.ElementsBasicEmailCest.php:72ReplyI didn't find a proper example to naturally implement the case `expectedError = true` as a last test. Nevertheless I checked this case locally, and it is properly working.ElementsBasicEmailCest.php:73ReplyThe check for save-ability is separate issue. So this can be implemented by an separate patch. -
Christian Kuhn commented on patch set 10 CR +1 V +1 11/21/24, 10:42
let's go with it. as usual with ac tests, we'll see over time if they are stable enough already or if they need further finetuning. -
Andreas Kienast voted on patch set 10 CR +2 V +2 11/21/24, 10:42
-
Ayke Halder rebased onto patch set 11
CR +1V +111/21/24, 11:10 -
Andreas Kienast commented on patch set 11 CR -1 11/21/24, 11:14
Tests are now broken due to changes done in https://review.typo3.org/c/Packages/TYPO3.CMS/+/87136. -
Andreas Kienast uploaded patch set 12
CR -111/21/24, 11:23 -
core-ci voted on patch set 11 V -1 11/21/24, 11:23 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/69921
-
Andreas Kienast uploaded patch set 13 11/21/24, 11:23
-
core-ci voted on patch set 13 V +1 11/21/24, 11:38 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/69927
-
Christian Kuhn voted on patch set 13 CR +1 V +1 11/21/24, 11:43
-
Andreas Kienast voted on patch set 13 CR +2 V +2 11/21/24, 11:43
-
Andreas Kienast merged patch set 14 11/21/24, 11:43
AbstractElementsBasicCest.php:58