Day 1 – Rendering down for Christmas

Last Christmas we got a new specification for RakuDoc, and this Christmas we have a generic rendering engine for RakuDoc V2. “So what?” some may say. “So WHAT !! ?? !!” shouts an inner demon of mine, having struggled for a year to get all the new stuff working, and wasting weeks on getting escaping some characters in HTML, while not double escaping them due to implicit recursion.

“Yeh, it’s boring,” says my inner geek, who sees friends’ eyes glaze when talking about my obsession. So why is RakuDoc such a big deal for me? And what can the new renderer do?

RakuDoc should replace MarkDown for complex websites

RakuDoc is a markup language that on one hand can be easily read in its source form, while at the same time has meta data that allows for a very rich and complex document. Again ‘so what?’, we have sophisticated editors like MS Word or LibreOffice’s Write, which can produce phenomenal output!

They are great for one-off documents, but what about complex documents such as descriptions of software, or more generally documentation of products or services? Suppose you then want to have these documents available in multiple formats such as Websites, or epub books? It can be done!

Consider, however, what nearly who has ever used a WYSIWYG editor, such as MS Word, encountered, namely a styling glitch, eg. the bolding can’t be turned off, or the cursor won’t go to the end of a table. It takes forever to get the text into the right format.

The problem is that the document has an underlying source, which cannot easily be examined, and if you do get, eg., an rtf of the source, a complex or heavily edited document will be swamped with redundant tags. These underlying formats are not intended to be human readable. RakuDoc is both human readable, while also having a simple way of extending the language through custom blocks.

The Christmas delivery

As to the new renderer. There is a generic render engine, which passes relevant source data to templates, collects document information for Table of Contents, and an index, and then outputs the final document in the chosen output format. The templates can be changed and extended, so each output format is defined via the templates.

The distribution RakuAST::RakuDoc::Render contains the four renderers which can be called using Raku (to Text, HTML-single page, HTML, and MarkDown). The documentation is at RakuAST renderer.

As can be seen from the title, this is one of the first modules to use RakuAST, which is the Raku v6.e milestone. The implementation difference between POD6 (or RakuDoc v1) and RakuDoc v2 needs an explanation.

The ‘old’ way of recovering the RakuDoc part of a Raku file is compile the file and recover the contents of the $=pod variable. The compilation stage was the same for a file containing executable code and for one just containing RakuDoc source code e.g., all the documantation files in the Raku/docs repository. In addition, the structures for the $=pod variable were very large. This has meant that when the documentation site is built, all the documents have to be compiled and cached, and then the $=pod is taken from the cache.

The ‘new’ way is to capture the AST of the program source, which is available before the program is run. Although similar in structure to a $=pod variable, the AST of a RakuDoc source is actually easier to handle. The consequence, however, is that the class describing the generic render engine has use experimental :rakuast, and that makes it difficult to work with fez and with zef. So the installation process is as follows:

1. clone the github repository into a directory (eg. rakuast-renderer)
2. from a terminal inside the directory, run zef install . -/precompile-install

The documentation gives a variety of ways to use the renderers, including raku --rakudoc=Generic docs/samples/101-basics.rakudoc > output.txt.

Better is use the distribution’s RenderDocs utility. For example, bin/RenderDocs --format=html README. Then in a browser open the file README.html in the CWD. This works because a file called README.rakudoc exists in the docs subdirectory.

Lots more options are available. Try bin/RenderDocs -h If you don’t like the way the HTML is formed or you want to use a different CSS framework instead of the Bulma framework, those can be changed as well. These are all described in the documentation.

Since this distribution is fresh for Christmas, there may be some flaws in the documentation, and the MarkDown renderer is in need of some TLC. Did I mention RakuDocs was customisable? So, there are a couple of custom blocks unfortunately only for HTML – not enough time for MarkDown, as follows   • Including maps into a document Using the Leaflet maps library

• Including graphs of all kinds Using the dot program

What was RakuDoc v2 difficult

There was actually a great deal of functionality in RakuDoc v1 that was never implemented. So all of these good things are now available:

• numbered lists, numbered headings, numbered definitions
• bulleted lists, with bullets that can be any Unicode glyph or icon
• Aliases, which can be declared in one place and then inserted multiple times in the text
• Semantic blocks, which can be declared in one place and hidden, but used later (suppose you want to maintain an AUTHORS list at the top of a file, but have it rendered at the bottom.
• Include a RakuDoc source from another file
• Systematic use of metadata, eg. a caption for a table to be included in the Contents
  • A memorable id for a heading, so that it can be linked to without the whole heading
• Configuration of blocks, all elements can be given metadata options that they will use by default. So the bullets on a list can be specified in the =item specification, or all =item blocks can be pre-configured to have some bullet type.
  • The configuration can be overridden with a new section, and when the section ends, the old configuration is restored.

An example of all the different things can be seen in a rendering compliance document See the HTML version of rakodociem-ipsum.

This is a toy that needs breaking!

Bear in mind that the whole distribution is newly complete. The documentation probably has some errors because the software changed while the documentation still has an older implementation idea. The Text and MarkDown renderers (more precisely their templates) need some Tender Loving Care. There are probably better ways to do certain things in MarkDown.

But this is a toy that needs breaking because to break it, you have to use it and test it. The more use and tests, the better.