Review main Merged

73136[TASK] Add acceptance test reports to Gitlab

Jochen Roth CR +2 V +2 Normal +91 −16 updated 01/24/22, 17:42

Commit message

Gitlab is able to give a nice overview of tests
and how often a test failed recently which
makes it easier to investigate flaky/failed
tests. Also it takes away the pain of searching
for the actual failing test in a job because it only
displays information of the failed test
(no bootstrap log details and such)

Codeception AC test reports have been added to the
Gitlab GUI using the --xml flag to generate a Gitlab
readable report format and it has been added to the
jobs artifacts:reports:junit.

Review

  • Code review CR +2 Done
    • Christian Kuhn +2
    • Oli Bartsch +1
    • Stefan Bürk +1
  • Verified V +2 Done
    • Christian Kuhn +2
    • Oli Bartsch +1
    • Stefan Bürk +1
    • core-ci +1 CI

Issues 1

Files 5

Review log 1522

  1. Jochen Roth uploaded patch set 1 01/24/22, 09:59

  2. Jochen Roth updated the commit message in patch set 2 01/24/22, 10:05

  3. Jochen Roth uploaded patch set 3 01/24/22, 10:06

  4. core-ci voted on patch set 1 V -1 01/24/22, 10:07 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22447

  5. core-ci voted on patch set 3 V +1 01/24/22, 10:28 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22450

  6. Christian Kuhn commented on patch set 3 01/24/22, 10:34

    foo. i just dropped ac test reports.xml with https://review.typo3.org/c/Packages/TYPO3.CMS/+/73039 from the setup ^^
  7. Jochen Roth uploaded patch set 4 01/24/22, 10:38

  8. Jochen Roth uploaded patch set 5 01/24/22, 10:41

  9. core-ci voted on patch set 4 V +1 01/24/22, 10:51 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22451

  10. Jochen Roth uploaded patch set 6 01/24/22, 10:51

  11. core-ci voted on patch set 5 V +1 01/24/22, 11:00 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22452

  12. core-ci voted on patch set 6 V +1 01/24/22, 11:09 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22454

  13. Jochen Roth updated the commit message in patch set 7 01/24/22, 12:52

  14. core-ci voted on patch set 7 V +1 01/24/22, 13:04 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22459

  15. Christian Kuhn commented on patch set 7 V +1 CR +1 01/24/22, 13:43

    let us do this for the ac tests for now to see if it works well, and maybe later add unit/functionals if needed.
  16. Stefan Bürk commented on patch set 7 01/24/22, 13:48

    Reply

    Was my first tought too. Anyway it may be usefull, at least until AC's are kinda stabilised ?
    Jochen, thats the 4th tab "tests" in gitlab which now have the tests listed, right ? Maybe we should thing about reducing ci test names to not clutter the views - thought about that already, but not scope of this patch. Anyway startet manual nightly on it, maybe we get some more failures listed as demonstration: https://git.typo3.org/typo3/CI/cms/-/pipelines/22464
  17. Jochen Roth commented on patch set 7 01/24/22, 14:06

    Reply

    also useful to see which tests failed (how oftern they failed) in a job and see only their specific issue/output

    Reply

    > Jochen, thats the 4th tab "tests" in gitlab which now have the tests listed, right ? yes
  18. core-ci commented on patch set 7 01/24/22, 14:14 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/22464

  19. Oli Bartsch voted on patch set 7 V +1 CR +1 01/24/22, 14:38

  20. Stefan Bürk voted on patch set 7 V +1 CR +1 01/24/22, 17:32

  21. Christian Kuhn voted on patch set 7 V +2 CR +2 01/24/22, 17:42

  22. Christian Kuhn merged patch set 8 01/24/22, 17:42