[Documentation] software proposal for Armbian wiki

miked · June 28, 2016

mkdocs.yml

site_name: Armbian Documentations
theme: readthedocs 
markdown_extensions:
    - toc:
        permalink: True

This doesn't actually provide persistent links across edits, like we want.

As tkaiser has described, the links that are made (and shown with a link icon on the web page if permalink: true is there) are generated from the title text.

"Legacy or Vanilla?" will have a link of #legacy-or-vanilla. If the title text is changed to "Choosing a kernel" the new link becomes #choosing-a-kernel.

Either way the docs web pages work fine, but if someone posted a link to "http://docs.armbian.com/user-manual/#legacy-or-vanilla"in the forum, that link is broken after the section's title is changed.

... well, not terribly broken; it will link to the top of the page, instead of the correct section. Changing the page name will fully break the link.

Edit: To put this another way, a "permanent link" according to mkdocs is essentially a link to a particular string of text in a section title. It is permanent only as long as that text is permanent (which it isn't if we have ongoing documentation editing).

lanefu · June 29, 2016

So by giving up on the 'presentation engine' in our github wiki and simply using a home page that states 'Do just edit the pages accessible on the right, don't change page titles, don't change section names, only add stuff and edit structure very carefully' you might get the wiki approach to work IF we want to postprocess the stuff therein (no idea if WE want that still)

By adjusting structure levels inside the individual .md docs and minimizing their count the postprocessing efforts are MINIMAL. Just choose the right mkdocs theme, disable search therein and feed the output to htmldoc. And you get a full PDF manual with links, index and so on. And using a different theme you can use HTML output from mkdocs on armbian.com (given that nobody uses the github wiki like a wiki and starts there editing like crazy -- that's the whole point. As soon as people start to use the wiki in the usual wiki way all our persistent links get fucked up automatically)

But since only @lanefu looked that much into (and might be able to understand me) I finally stop here since the whole discussion gets boring.

I think i'm on the same page as @tkaiser. Let me see if I can break it down a bit.

If we simplify down to just single markdown files/pages in the github wiki with a proper naming convention. All of our desired outputs can be easily achieved with a few post processing scripts. Also this model makes it easier for others to contribute to docs.... all they have to do is make a page/file.

Here's my knowledge base script as example code to generate a markdown file. Won't take much to refactor... That code was designed to parse markdown files, and find the first header and insert that into the TOC. changing to just regex [ ] blocks out of the directory listing should be pretty easy. I don't have time to get to that tonight so if somebody else is interested be my guest.

I built a picture:

Igor · June 29, 2016

This doesn't actually provide persistent links across edits, like we want.

Consider the page as WIP. I didn't make any more progress and will update it with our findings when done.

I guess the location of the documentation will be o.k. "docs.armbian.com" ?

jeanrhum · June 30, 2016

To summarize the discussion because it was difficult for me to follow all the answers. The initial objective was to identify a software to manage and process the documentation:
- current input format is markdown
- 2 output formats are expected: html and pdf

Identified solutions:
- mkdocs seems interesting (simple and configurable) to generate static html pages
- pandoc can generate pdf with toc from many document formats
- github wiki is based on markdown and offers a simple way to generate html doc as we can see with current redirection.

Personnally, I used Sphinx for a previous project to generate from the same input files an html documentation and a pdf report (in fact generate latex and then pdf). The input format is restructured text, but markdown can be easily translated to that close language (e.g. with pandoc) more especially if there are already files for each section of the doc. I'm not sure this is the best solution for current case.

One major issue is to define how people can contribute while keeping documentation structure with valid permalink:

- public wiki solutions don't fit this requirement even if it is easy to rollback when problem occurs.

- pull requests on a git repository for the documentation can be chosen, it requires that admins of this repo validate and integrate contributions, but it may offer more guarantee about structure preservation.

The content of the documentation and its structure should also be addressed, but perhaps in another topic.

lanefu · June 30, 2016

One major issue is to define how people can contribute while keeping documentation structure with valid permalink:

- public wiki solutions don't fit this requirement even if it is easy to rollback when problem occurs.

- pull requests on a git repository for the documentation can be chosen, it requires that admins of this repo validate and integrate contributions, but it may offer more guarantee about structure preservation.

The content of the documentation and its structure should also be addressed, but perhaps in another topic.

IHMO The permalink is going to be a big ask... My assumption is a CMS, or something database backed is required for a permalink to work. The complexity probably doesn't merit the benefit. Things being renamed, retitled, moved implies someone is taking a very dedicated editor role. Most likely the level of that effort will be marginal. Critical items can have a redirect on a webserver. Just let the content flow in and keep the parent topics simple.

I liked @tkaiser 's suggestion to operate the docs wiki out of a separate repo that had more generous access.

Other than maybe identifying a few key parent categories for documentation, the content and structure should probably be focused on a separate discussion from the implementation/workflow discussion.

miked · June 30, 2016

IHMO The permalink is going to be a big ask... My assumption is a CMS, or something database backed is required for a permalink to work. The complexity probably doesn't merit the benefit. Things being renamed, retitled, moved implies someone is taking a very dedicated editor role. Most likely the level of that effort will be marginal. Critical items can have a redirect on a webserver. Just let the content flow in and keep the parent topics simple.

I liked @tkaiser 's suggestion to operate the docs wiki out of a separate repo that had more generous access.

Other than maybe identifying a few key parent categories for documentation, the content and structure should probably be focused on a separate discussion from the implementation/workflow discussion.

I continued my permanent link experiment, it is working locally. It's a simple bash script. I'm not 100% convinced it's a good idea. It is simply:

1. A human adds markers in the source document where you want to specify an anchor name, overriding what is automatically generated.

2. When html is built, script replaces the autogenerated anchors with the specified one, in mkdoc's output files (The anchor id, and links to it in the html files and search index).

The marker that goes in source can be something that will not be displayed even if not properly processed out, eg:

abuse a markdown reference: "[permalink]: my-anchor-name"

or fake html that browsers will ignore: "<permalink="my-anchor-name">

(the latter can be processed more cleanly).

It can be used sparsely in the source, only where desired (but then a site maintainer must add the tags when the scripts warn of potential broken links), or it can be used all over (but then either the doc writers must deal with them or a site maintainer).

Whether this is worth it is someone else's call, so i'll leave it until someone wants to move forward. Any decision on it should be easily changed later.

Tido · July 1, 2016

a comment of TK on another thread:
We simply (still) fail to point our users to the relevant portions of documentation.

Why is this necessary?
Doesn't a TOC and good structure do the job?
We are not RPi with 8years olds in the forum.

lanefu · July 1, 2016

Heads up: I'm planning on making a drastic commit to the Wiki later today. It will shuffle the naming of all the docs etc. If you all would rather I do a preview first on my own repo before moving forward let me know.

@Igor

lanefu · July 1, 2016

@all

I committed new structure to wiki. Sorry for any intermediate fallout. Going to keep pounding on this.. I'll add a doc on how to do documentation as well

https://github.com/igorpecovnik/lib/wiki

lanefu · July 1, 2016

Since I've started moving forward with implementation, See this thread for implementation status and discussion.

http://forum.armbian.com/index.php/topic/1540-400-implement-documentation-software-solution/

Kriston · July 3, 2016

Thanks, lanefu. I'll get up to speed on that solution. I'm more about writing than deciding on a tool, so I've elected to use whatever you folks have decided upon.

lanefu · July 3, 2016

Thanks, lanefu. I'll get up to speed on that solution. I'm more about writing than deciding on a tool, so I've elected to use whatever you folks have decided upon.

Glad to help. Connect with igor and get access to the new docs repo.. In the meantime you could clone and submit merge request

https://github.com/igorpecovnik/lib.docs

Kriston · July 3, 2016

Thanks, I'll forked a little while ago, so I'll re-base my fork and start submitting.

Thanks!!

Kriston

lanefu · July 3, 2016

great.. BTW there's a task topic already created for you. It'll inspire others if you can show status there

http://forum.armbian.com/index.php/topic/1532-392-documentation-rework/

miked · July 3, 2016

Looks like it's really coming along well!

Do you want UX feedback discussed here, or on https://github.com/igorpecovnik/lib.docs(are issues disabled for it???), or should we wait until it's ready for feedback?

For example... When I'm at docs Home, the User Guide sections are offscreen on my computer. Even if it is onscreen, it is below stuff like Task Tracking, Changelog etc. As a complete newb, if I go to the docs, I might skim the Home doc, maybe hit Next and stop reading by the FEX section at best. I might skim the sidebar topics... if I'm lucky I'll notice "Getting started" near the bottom. Otherwise I'll think that Home is all the getting started info there is and conclude that this all requires some expertise that I don't have.

I don't know if such an issue is already being addressed with a doc reorganization, or if it is now baked into the current design.

lanefu · July 4, 2016

Try to keep discussion on the forums. Task Tracking Process

Re: Content Order. yeah it needs to be optimized.. Right now the TOC is dynamically generated via a python script with no order other than grouping by topic.

Need to figure out if its worth adjusting the tool, or just maintain mkdocs.yml manually.

July 4, 2016

[splitting up documents, rearranging stuff for no other reason than a nice wiki presentation]

Yes, you are perfectly right about this. Nothing to add.

And here we go: https://github.com/igorpecovnik/lib.docs/tree/master/docs

15 documents instead of the few we had before (still: the less documents the better since this helps maintaining consistent structure!!!) and if this will be the basis for permalinks then good luck (Feel-boot instead of FEL/NFS boot, the most important part -- user documentation -- at the end and so on). The most recent entry in Release_Changelog.md is BS as well as the whole Hardware_Allwinner-H3.md (this is not related to hardware that much). It doesn't work splitting up a temporary FAQ that should be frozen with 5.15 release (no announcement?) for a new user base into static documents, this absolutely makes no sense at all but now leads to others wasting their time trying to maintain this mess.

BTW: When feeding current mkdocs output to htmldoc still everything breaks since header/TOC levels aren't consistent accross documents (that's why less documents are better than more)

Tido · July 4, 2016

And here we go: https://github.com/igorpecovnik/lib.docs/tree/master/docs

15 documents instead of the few we had before

I agree with you, that there are to many 'little' topics in the first order.

This was my suggestion - the order is not defined, just the Main-Topics (8)

Introduction, FAQ, Board specific, SoC specific, Basic, Advanced, Developer, Text for image.zip

If these are the basis for permalinks this would be good enough for me - so that the document can grow how the future wants it.

Why shall an FAQ be frozen, shouldn't it reflect the state of the latest version - to keep it easy ?

Igor · July 4, 2016

Yes, you are perfectly right about this. Nothing to add.

That is refereed that I'll be involved as less as possible ... and wait what will come out. My latest idea of structure is below and I don't insist with it. Have other things to cover up + leaving for summer vacations in few days.

drwxr-xr-x root/root         0 2016-06-28 21:10 ./docs/boards/
-rw-r--r-- root/root      2800 2016-06-28 16:40 ./docs/boards/Allwinner-hardware-conf.md
-rw-r--r-- root/root      4890 2016-06-28 16:40 ./docs/boards/H3.md
-rw-r--r-- root/root      2150 2016-06-28 21:48 ./docs/index.md
drwxr-xr-x root/root         0 2016-06-28 21:42 ./docs/images/
-rw-r--r-- root/root      3342 2016-06-27 14:42 ./docs/images/logo_middle.png
-rw-r--r-- root/root     18374 2016-06-28 21:14 ./docs/user-manual.md
-rw-r--r-- root/root     23199 2016-06-28 16:40 ./docs/Changelog.md
-rw-r--r-- root/root     17611 2016-06-28 21:19 ./docs/build-manual.md

Wiki. Docs are bad in any shape at this moment so having many versions of the same thing is pointless and bring only confusion. Current situation has at least good search, even the structure (and design) is not done properly yet.

lanefu · July 4, 2016

I have a few responses to some of the most recent concerns....

Too Many Documents--Creates TOC inconsistencies

To encourage contribution from as much of the community as possible, several smaller documents is more appealing. It strongly minimizes the risk of merge conflicts, allows for more content creation in parallel, and allows the contributor to stay focused on specific content.

TOC inconsistencies can easily be mitigated by provided a few empty markdown templates that provide scaffolding to keep the heading structure in place.

The key to getting contribution is to minimize the barrier to entry as much as possible. AKA: quick, simple, easy for the contributor

Here's the an example of an ideal workflow for adding documentation:

Contributor clones repo
Contributor copies ex: `templates/howto_template.md` to `docs/howto_new-topic.md`
Contributor creates content within the simple framework provided by template
Contributor commits and pushes to repo
post-commit job runs a more intelligent version of `tools/mkArmbianDocs.py` and generates new mkdocs.yml
post-commit builds new docs site, pushes, builds PDF (maybe--unless extra curation is desired)
Contributor is delighted that creating a single markdown file was all that was needed to impact the community
Contributor is inspired to create more content -- because it was soooooo easy.

Permalinks--No Hope....ever

On the short-term, permalinks will be unreliable.

In the long-term I would expect most content structure to normalize and permalink reliability will fall with in acceptable terms. Also--search will fill in the gaps.

Tido · July 4, 2016

Here's the an example of an ideal workflow for adding documentation:

Contributor clones repo

Contributor copies ex: `templates/howto_template.md` to `docs/howto_new-topic.md`

Contributor creates content within the simple framework provided by template

Contributor commits and pushes to repo

post-commit job runs a more intelligent version of `tools/mkArmbianDocs.py` and generates new mkdocs.yml

post-commit builds new docs site, pushes, builds PDF (maybe--unless extra curation is desired)

Contributor is delighted that creating a single markdown file was all that was needed to impact the community

Contributor is inspired to create more content -- because it was soooooo easy.

Will you add this to: Process_Contribute.md ?

lanefu · July 4, 2016

Will you add this to: Process_Contribute.md ?

Okay I did a quick add to `process_contribute.md`

I also pushed a simple documentation example template. I'm hoping that it'll be more compatible with PDF TOC stuff.. I'll try to test at some point.

miked · July 5, 2016

I feel that settling on filenames/URLs as quickly as possible should be a high priority. Are the current filenames settled? (including the "feel boot" typo?) If perfect permanent links are infeasible, breaking links and leaving temporary/WIP names for long periods should at least be avoided as much as we can. I don't know how much of a problem it could really be, but I imagine people googling a common question, and ending up at a permanent forum post somewhere that contains a broken link to the documentation.

I was curious about collapsible navigation links and came across this mkdocs theme example: http://developer.helpmonks.com/

Code and description can be found here: https://github.com/mkdocs/mkdocs/issues/588

This appears to be a WIP, has bugs and might be hacky, and is extra complication on top of readthedocs, so I don't recommend we use it, just an idea.

I downloaded the theme and tried it out on our docs. It doesn't display subsections in the navbar, which is a problem. It looks like their theme's toc.html removes it, and adds support for organizing .md files into subdirectories.

lanefu · July 5, 2016

@miked

It would be my preference that we leave the theme alone for now. We really just need something that works reliably from end to end. (Web, Mobile, PDF, Markdown Viewer) The current theme is sufficient.

Re: Filenames, parent categories..

Ready, Fire, Aim!

Nothing is locked in-now is the time to fix. Please follow the naming convention described here. Just to re-cap what I described yesterday, the end-game is for new content to be easily and rapidly deployed. Part of the reason for flat directory structure, and file-named based TOC is to let mkdocs and a few scripts do the work--often.

I do have to circle back and add some sorting preset functionality to the generator script. Feel free to participate on this thread.

Suggested Next Steps:

Identify Parent Topics and preferred order
Update / create documentation template(s) to assist in formatting (I made 1 example template. Adjust / replace as you see fit.)
Draft a documentation standard and post
Normalize formatting of exiting docs into new standard
Validate that new formatting standard renders cleanly in PDF format

Now that we're focusing on the actual documentation contents and formatting its probably time to move the conversation to the Documentation Task Topic.

lanefu · July 7, 2016

@miked @kriston @tido

I want to make sure you all connect with each other. You've all contacted me about documentation content, file structure etc. I want to make sure you all aren't repeating work. Please provide some status on what you're working on. sorry i meant to post this here

http://forum.armbian.com/index.php/topic/1540-400-implement-documentation-software-solution/

Edited July 7, 2016 by lanefu

Kriston · July 7, 2016

@miked @kriston @tido

I want to make sure you all connect with each other. You've all contacted me about documentation content, file structure etc. I want to make sure you all aren't repeating work. Please provide some status on what you're working on. sorry i meant to post this here

http://forum.armbian.com/index.php/topic/1540-400-implement-documentation-software-solution/

Forgive me if this isn't the place to reply, but I am concerning myself with body content and not tools or layout decisions. I am happy with whatever decision is made after the tools and layout are decided upon.

Kriston

miked · July 7, 2016

Please provide some status on what you're working on.

I'm not actively doing anything at the moment, to avoid stepping on toes.

I would like to QA/edit the organization and navigation of the docs (within the current tools/layout). I could probably do that now?

I might add to some of the content, for beginner readers, but only after others' rewrite tasks are done.

lanefu · July 7, 2016

I'm not actively doing anything at the moment, to avoid stepping on toes.

I would like to QA/edit the organization and navigation of the docs (within the current tools/layout). I could probably do that now?

I might add to some of the content, for beginner readers, but only after others' rewrite tasks are done.

yeah man get cracking.. nothing is holding you back! the files can be re-tooled now.. please see my post about templates.. i think thats a critical place to start.

Sign In

[Documentation] software proposal for Armbian wiki

Recommended Posts

miked

lanefu

Igor

jeanrhum

lanefu

miked

Tido

lanefu

lanefu

lanefu

Kriston

lanefu

Kriston

lanefu

miked

lanefu

Guest tkaiser

Tido

Igor

lanefu

Tido

lanefu

miked

lanefu

lanefu

Kriston

miked

lanefu

Forums

My Activity Streams

Download

Store

Important Information