Review main Abandoned

65577[WIP][DOCS] Change rendering of changelogs

Sybille Peters CR 0 V 0 Normal +106 −16 updated 03/31/21, 09:26

Commit message

This change makes it possible to render only the changelogs of a specific
TYPO3 version by supplying a jobfile.

The main reason for this change are performance reasons when rendering
the changelog: This change reduces the time it takes to render the
changelogs (by getting rid of duplicate rendering).

Also, an index is now available, which makes it possible to select
changelogs by category (e.g. TypoScript) when viewing them on
docs.typo3.org.

The jobfile.json uses exclude patterns to exclude some .rst files from
rendering.

This makes it possible to render only the changelogs of the current
version, thus separating the rendered changelogs per branch (like in
the rest of the documentation).

This change is non-invasive, meaning that if the jobfile.json is not
used, the rendering can be performed as before. Also all rst files are
kept, allowing extension scanner to work.
If the jobfile.json is used, only the master changelogs will be rendered
for master.

Impact:

* The jobfile must be supported by Bamboo
* some redirects should be created to redirect e.g.
  /cms-core/9.5/en-us/Changelog/8.7.x/Feature-something.html to
  /cms-core/8.7/en-us/Changelog/8.7.x/Feature-something.html
* On the docs server, the version selector must be used to
  view older changelogs
* An index is now available (as described above)

WIP:

* Discuss integration in Bamboo.
* Discuss usage of jobfile.json.

Review

  • Code review CR 0
  • Verified V 0

Issues 1

Files 7

Review log 1824

  1. Sybille Peters uploaded patch set 1 09/04/20, 10:47

  2. TYPO3com voted on patch set 1 V +1 09/04/20, 11:03 Completed build in 15m 56s on Fri, 4 Sep, 11:03 AM Test Summary: 73200 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-5229

  3. Daniel Siepmann commented on patch set 1 09/07/20, 06:24

    Important-92196-RenderOnlyChangelogsOfCurrentBranchAndAddIndex.rst:4

    I would not mark this change as important. As far as I know "Important" means people need to check this change when updating TYPO3.

    jobfile.json:1 Unresolved

    Right now I'm not aware Bamboo will select the jobfile.json. It is called with "-c jobfile /PROJECT/jobfile.json" and Changelog is not handled different then any other extension. We should discuss that with the maintainers before merging.
  4. Oliver Klee updated the commit message in patch set 2 09/07/20, 15:06

  5. TYPO3com voted on patch set 2 V +1 09/07/20, 15:18 Completed build in 11m 37s on Mon, 7 Sep, 03:18 PM Test Summary: 73227 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-5332

  6. Sybille Peters uploaded patch set 3 09/07/20, 16:11

  7. Sybille Peters commented on patch set 3 09/07/20, 16:11

    Important-92196-RenderOnlyChangelogsOfCurrentBranchAndAddIndex.rst:4 Reply

    Done
  8. TYPO3com voted on patch set 3 V +1 09/07/20, 16:33 Completed build in 13m 36s on Mon, 7 Sep, 04:33 PM Test Summary: 73227 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-5337

  9. Andreas Kienast commented on patch set 3 CR -1 09/07/20, 17:28

    Unresolved

    8.7 and 7.6 are ELTS and don't get rendered anymore
  10. Andreas Kienast updated the commit message in patch set 4 09/07/20, 17:29

  11. TYPO3com voted on patch set 4 V -1 09/07/20, 17:51 Completed build in 20m 34s on Mon, 7 Sep, 05:51 PM Test Summary: 1 of 73712 failed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-5341

  12. TYPO3com voted on patch set 4 V +1 09/07/20, 18:30 Completed build in 1h 20m 21s on Mon, 7 Sep, 06:30 PM Test Summary: 73227 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-5341

  13. Daniel Siepmann commented on patch set 4 09/08/20, 09:49

    Unresolved

    To me it looks like this is wrong right now. There is a single jobfile which excludes all files. What do I miss? I guess this is solved by backporting a different content of jobfile.json to each branch?

    Howto.rst:194

    1000:1000 is specific to some Unix distributions, and only if you are the first user. Mac OS is using other ids by default. Not sure if we should cover / explain that. One could also use some dynamic replacements, just like done with $(pwd).
  14. Sybille Peters commented on patch set 4 09/08/20, 14:03

    Howto.rst:194 Reply

    Do we want to decide first if this change generally makes sense and should be accepted? After that, it could get finetuned and improved. About rendering with userid etc.: I used what was in Documentation-GENERATED-temp/last-docker-run-command-GENERATED.sh.txt on my system. Of course, that would have to be changed or adapted. IMHO, we don't have to document there how to render the docs with jobfile, it is just that I had a hard time finding the information at all, so I wanted to include it somewhere. But, of course this could be documented elsewhere or if it already is made more easy to find and a link could be added. About missing jobfiles: In the first version, I had included a jobfile for each version e.g. master, 10.4, 9.5 etc. I then removed these and kept only the jobfile for master. The current jobfile excludes all except master. When backporting, this would have to get changed to use the respective version.
  15. Daniel Siepmann commented on patch set 4 CR +1 09/09/20, 06:03

    Howto.rst:194 Reply

    I'm fine with the change and it looks good and very valuable to me. It should solve all the issues without creating new ones. With your approach we still have all rst files for extension scanner, but only render the Changelog for a specific TYPO3 version. I've only one downside which is okay: You can't search via CTRL+F / CMD+F on the initial page for all changes, as they would be splited between versions. I like the idea to have a single jobfile.json which differs on all branches to match the actual version. We just should ensure that this is adjusted with each new LTS release (define the workflow somewhere). I've checked bamboo and the jobfile.json should be respected out of the box, as far as I can see. I've no access to the jobs, just the log. Maybe we should add the documentation about how to render to https://docs.typo3.org/m/typo3/docs-how-to-document/master/en-us/WritingDocsOfficial/LocalEditing.html ? That way it is at a single location and not part of each changelog version.
  16. Daniel Siepmann commented on patch set 4 CR 0 09/09/20, 06:09

    Howto.rst:194 Reply Unresolved

    Sorry, Bamboo expects the jobfile one level up, in the extension folder. Not sure whether that is in use already, or documented. But I would say we should change that.
  17. Daniel Siepmann uploaded patch set 5 09/09/20, 07:22

  18. TYPO3com voted on patch set 5 V +1 09/09/20, 07:38 Completed build in 15m 30s on Wed, 9 Sep, 07:38 AM Test Summary: 73173 passed Find logs and detail information at https://bamboo.typo3.com/browse/CORE-GTC-5383

  19. Martin Bless commented on patch set 5 CR -1 09/10/20, 09:58

    jobfile.json must not be in the hands of non-administrators as it is very powerful. Instead, I'll make it possible to configure the "excludes" settings in the Settings.cfg file.
  20. Martin Bless commented on patch set 5 09/10/20, 10:09

  21. Sybille Peters commented on patch set 5 03/30/21, 07:25

    I will abandon this now.
  22. Sybille Peters abandoned the change 03/30/21, 07:25

    Find other solutions for inital goal: make it possible to search / filter changelogs by version / category.
  23. Daniel Siepmann commented on patch set 5 03/31/21, 09:05

    Reply

    I guess recent releases of the container solved the performance issue. Hopefully the new upcoming search will help in finding things. But I would also welcome any proposal to improve the changelog, as long as it is maintainable.
  24. Sybille Peters commented on patch set 5 03/31/21, 09:26

    I raised issue in Slack channel and was referred to global search. So there is some overlap with search / filter but I think managing the changelog is another issue in itself. We already have great tools with viewing changelogs in BE and online and extension scanner etc. but from my practical experience I sometimes find the information overwhelming. However, I do not see a solution right now which would not be awkward and more of a workaround. Apart from that - some tasks are too big or too complicated and unless that is somehow turned into a project with people working on it, I think it is best to abandon it for now to avoid clutter in Forge and Gerrit.