Documentation guidelines

A brief set-of-rules for the documentation.

Note

The old wiki uses dokuwiki. When importing text from there, consider the automated options that are provided in this page to convert it to reST.

Importing from DokuWiki to Sphinx

Use the following command to import a DokuWiki page (old wiki.analog.com):

pandoc imported.txt -f dokuwiki -t rst --columns=80 -s -o imported.rst --list-tables

The list-tables parameter requires pandoc-types >= 1.23, included in any recent pandoc release; if it is not an option, you shall remove it and export in the grid table format (see Tables for more information).

After converting, update it to better conform with the guidelines below, and make sure to use our directives and roles, for example, the Git.

To speed thing up, combine with wget:

wikifile=resources/eval/user-guides/adrv9009/adrv9009
outfile=output.rst

wget -O - https://wiki.analog.com/$wikifile?do=export_raw --no-verbose | \
  pandoc -f dokuwiki -t rst --columns=80 -s -o $outfile --list-tables

Also, consider recording macros in your favorite text editor to automate repetitive steps.

There is also the DokuWiki to Sphinx (bash.sh) to further help importing.

Directives and roles

Sphinx allows expanding functionality beyond the defaults directives and roles by loading extensions. Doctools implements custom directives and roles for common features, but third-party extensions can also be used.

Doctools extended directives and roles are provided at:

Third-party directives and roles

Repositories have the freedom to choose the third-party extensions they want to include in their documentation. They are listed on the requirements.tst file, and are installed with:

pip install -r requirements.txt

The recommendation is to avoid using extensions that render content on the client-side using JavaScript, because it requires:

Fetching third-party scripts at someone-else server most of the time; and
Makes exporting to other formats (like pdf) harder, because it is then necessary to patch and replace with another non-JavaScript renderer.
Requires managing these added scripts to ensure they apply on hot reload.

Support custom JavaScript in hot reload

Hot reload is a feature that reloads only parts of a webpage, allowing super fast content-loading, cool animations, and a better user experience.

However, for third-party client-side JavaScript, it is necessary to map the added globals to hot reload module.

On adi_doctools/theme/cosmic/scripts/hot_reload.js, extend the script_remove and script_add to include hooks for your particular added script.

For example, for MathJax, that is simply calling the global method MathJax.typeset() to process the new page (hot reload) if the page includes the script.

Here is a template:

export class HotReload {
  // ...
  script_remove (url) {
    switch(url) {
      case "https://example.com/npm/my_script.js":
        if (typeof MyScript !== 'undefined')
          // If there is a clean-up/deinit method, call it
          // to try to keep memory usage in check.
          MyScript.cleanup()
        break;
      // ...
    }
  }
  script_add (url) {
    // Get templated script node.
    const elem = this.js_script_memory.get(url)
    // Append to page to init loading.
    document.querySelector('head')?.append(elem)
    switch(url) {
      case "https://example.com/npm/my_script.js":
        if (typeof MyScript !== 'undefined')
          // On hot reload, already loaded, retrigger init.
          // You need to check the script documentation for
          // the exact methods names, calls.
          MyScript.init()
        else
          // On hot reload, first load, do
          elem.onload = () => {
            console.log("MyScript loaded!")
            // Do other things, if necessary, but in general those scripts
            // already do their thing on load.
          }
        break;
      // ...
    }
  }

Repository-specific extensions

Extensions with repository-specific functionalities can be added to the repository source code.

At conf.py, add:

import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join('.', 'ext'))

extensions = [
     ...
     "ext_repo_name"
]

Create the ext/ext_repo_name.py and implement your custom directives/roles. Replace repo_name with your repository name.

Tip

If you think your extension could be useful for other repositories, consider migrating it to Doctools source code.

If the extension relies on files at the repository (like XML files), and/or uses either env.found_docs or docnames to manage cache, check if env.config.monolithic exists and is true.

If yes, prefix:

The path to files with the relative path from the custom doc to the repository (always ../<repository> (e.g.,`../hdl`)); and
Doc-names with the repository name (e.g., hdl/).

This ensures compatibility with Custom Doc.

Here is a practical example:

Metadata to be parsed is at repo_name/builds/info.xml
The sphinx source dir env.srcdir values are:

Original doc (per repo): repo_name/doc/sphinx/source

Custom doc (always): _build.

So, if it is necessary to check env.config.monolithic to correct infer the target file path, for example:

if hasattr(env.config, 'monolithic') and env.config.monolithic:
     repo_path = path.join(env.srcdir, '..', 'repo_name')
else:
     repo_path = path.join(env.srcdir, '..', '..', '..')

artifact_path = path.abspath(path.join(repo_root, 'builds', 'info.xml'))

This is necessary because Sphinx has no way of knowing the repository root (without invoking a git shell), so hard-coding a relative path is necessary.

Note

If the extensions are reused by multiple repositories, infer the repository name from the first level (my_repo/) of the docname (my_repo/tutorial/index).

Custom doc copies env.srcdir(original) to env.srcdir(custom)/repo_name to avoid page conflicts when aggregating. So, sometimes it is necessary to prefix docnames with the repository name:

if hasattr(env.config, 'monolithic') and env.config.monolithic:
     prefix = "repo_name"
else:
     prefix = "."

_somedoc = path.join(prefix, _somedoc)

For intermediary files, for example a SQLite database, it is recommended to anchor to outdir/../managed:

dest_dir = path.abspath(path.join(env.app.builder.outdir, pardir, 'managed'))
sql_path = (dest_dir, "my_repo.sqlite")

Indentation

Directives are indented with 3 space, which is Sphinx’s default. At code directives, the code keeps its original indentation (e.g. 2 spaces for Verilog code), but is offset by 3 spaces at the beginning of every line, to instruct Sphinx the beginning and end of the code directive.

For extensive documentation with different topics, it makes sense to filter the toctree based on the current topic/toctree title. This is possible by setting the environment variable ADOC_FILTER_TOCTREE to 1. Alternatively, setting filter_toctree on conf.py has higher precedence than ADOC_FILTER_TOCTREE. And is supposed to be used alongside the topic field at lut.py to preserve high level links for each topic.

Enable this environment variable only on the release build, since writing pages with it enabled may be obnoxious and confusing prior the final structure/location of them.

Versioning

To avoid having the version set in multiple places or having to tweak conf.py to obtain it from somewhere else, continuous integration can set the environment variable ADOC_DOC_VERSION to set the version value.

The version value on conf.py has lower precedence, and will be overwritten if ADOC_DOC_VERSION is set.

The CI, in general, should set ADOC_DOC_VERSION as the current checkout branch in the pipeline (e.g. main, v1.0.0), for GitHub Actions, that can be ${{ github.ref_name}} [1].

Tip

If creating a branch or PR output, consider using GitHub short reference ${{ github.ref_name }}.

If both environment variable and version on conf.py are unset, it defaults to an empty string.

Exporting to PDF

The whole documentation can be exported to a PDF document for a more compact format using either rs2pdf or WeasyPrint. This is done by setting the environment variable called ADOC_MEDIA_PRINT (the value does not matter) and building the documentation.

For rst2pdf, add rst2pdf to conf.py extensions list, install rst2pdf, svglib (svg support) and matplotlib (LaTeX formulas) from pip and use:

sphinx-build -b pdf . _build/pdfbuild

In the output folder, you’ll find a PDF document named after the repository (e.g. Doctools.pdf). This document includes an auto-generated cover, followed by the remaining pages. Note that an HTML build of the documentation is not required for the PDF build.

Warning

The environment variable ADOC_MEDIA_PRINT should be unset when building the HTML pages of documentation. If not set, some components of the pages may not render properly.

For WeasyPrint, install weasyprint, and matplotlib (LaTeX formulas) from pip and use with Serve:

~/some_repository/docs$

adoc serve --directory . --builder pdf

The advantage of WeasyPrint is that the design styles (CSS stylesheet) is respected.

Inner working

Internally, ADOC_MEDIA_PRINT variable is set to app.config.media_print and should be used in scenarios where it is explicitly needed to compile the content in a different manner than for the hosted html. For example, to render content during build that would instead be rendered with third-party JavaScript libraries in the user browser.

Still, another approach is to patch the generated html, like is done at adi_doctools/cli/aux_print.py for Serve and Custom Doc with pdf builders.

Local references

References to labels have the format :ref:`context topic`, e.g. :ref:`role git` renders as Git.

Labels are created for any content with the syntax (dot-dot underscore<label>two-dots):

.. _context topic:

Hint

Add labels to any content that may be linked, locally or externally.

References to docs have the format :doc:`path/to/doc`, e.g. :doc:`docs_guidelines` for docs_guidelines.rst.

Attention

Do not break reference roles between lines! Even though Sphinx allows breaking line inside the reference role, it makes pattern matching hard.

Prefer hyphen separation - over underline _ for the “title” section, and always lower case, for example my_code control-interface instead of MY_CODE Control_Interface.

Numbered references

References can be numbered by using numref, for example, “see Figure 1”.

To use this feature, enable on the conf.py:

numfig = True

To customize the format, set numfig_format:

numfig_format = {'figure': 'Figure %s',
                 'table': 'Table %s',
                 'code-block': 'Listing %s',
                 'section': 'Section %s'}

Tip

By default enumeration is global, so if the toctree is not numbered to divide the pages in numbered sections (e.g. Figure 3.4.4), the numbering will “propagate” across page which may be counter-intuitive.

To have the numbering reset at every page, add numfig_per_doc = True to conf.py.

External references

External references to other Sphinx documentation are created using the built-in sphinx.ext.intersphinx extension.

To set up in-organization references read the section below, and for third-party docs, the section that follows.

For either, to create a reference to a label, use:

:external+inv:ref:`label`, where inv is a mapped source, for example, :external+hdl:ref:`spi_engine control-interface`.

To create a reference to a doc, use:

:external+inv:doc:`label`, where inv is a mapped source, for example, :external+hdl:doc:`library/spi_engine/index`.

As the other roles, it is possible to customize the text, e.g. :external+hdl:ref:`Custom text <spi_engine control-interface>`.

Tip

Pay attention to the log output, since warnings is thrown for each reference not found.

External references work with any kind of references, such as ref, doc, envvar, token, term, numref and keyword.

For references to labels it is possible to use the short form:

:ref-inv:`label` (equivalent to :external+inv:ref:`label`), but is discouraged.

Note

Sphinx 8 allows the syntax :ref:`<inv>:label`, which allows local references to have higher precedence than external refs, useful for generating custom docs like user guides. However, since some users may require Sphinx 7, use the former syntax, and let adoc patch it when necessary.

To show all links of an InterSphinx mapping file, use the built-in tool:

python3 -m sphinx.ext.intersphinx https://analogdevicesinc.github.io/hdl/objects.inv

In organization reference

To create references to Sphinx docs inside the organization add the repositories of interest to the conf.py file with the following format:

interref_repos = [external...]

For example:

interref_repos = ['hdl', 'no-OS', 'pyadi-iio/main']

Notice that in the example main suffixes pyadi-iio, this means that will look for the build at path main of this repo instead of at root. This can be used to target a specific version, if the target repository stores multiple, for example, v1.1, more about that Versioned.

Tip

For even more freedom, you can set up with an explicit path as an Outside organization Sphinx reference.

It is possible to customize the target URL with the interref_uri config or ADOC_INTERREF_URI environment variables. The default value is https://analogdevicesinc.github.io/ and can be set to a local path like ../../.

Beyond the main target dictated by interref_uri, by setting the config interref_local as true, a secondary target is inferred foreseeing a local copy of the target external documentation alongside the current repository:

/data/work
├─my-repo-0/doc/sources
│
├─my-repo-1/docs
│
└─my-repo-2/doc

The correct relative paths are resolved looking into the lut.py.

Outside organization Sphinx reference

To create references to third-party Sphinx documentations, add the mappings to the conf.py file with the following format:

intersphinx_mapping = {
    '<name>': ('<path/to/external>', None)
}

For example:

intersphinx_mapping = {
    'sphinx': ('https://www.sphinx-doc.org/en/master', None)
}

Text width

Each line must be less than 80 columns wide. You can use the fold command to break the lines of the imported text while respecting word-breaks:

cat imported.txt | fold -sw 80 > imported.rst

The header divider “---” shall be either 80 characters wide or end at the title character, that means, this is also valid:

My title
========

Tables

Prefer list-tables and imported csv-tables (using the file option), because they are faster to create, easier to maintain and the 80 column-width rule can be respected with list-tables.

Only use Grid Tables if strictly necessary, since they are hard to update.

To tune styling, the following classes are available:

bold-header: Make the header bold.
bold-first-column: Make the first column bold.

Lists

Unordered lists use * or - and ordered lists #..

Child items must be aligned with the first letter of the parent item, that means, 2 spaces for unordered list and 3 spaces for ordered lists, for example:

#. Parent ordered item.

   * Child unordered item.

     #. Child ordered item.
     #. Child ordered item.

Renders as:

Parent numbered item.
- Child unordered item.
  1. Child ordered item.
  2. Child ordered item.

Code

Prefer code-block to code directives, because code-blocks have more options, such as showing line numbers and emphasizing lines.

For example,

.. code-block:: python
   :linenos:
   :emphasize-lines: 2

   def hello_world():
       string = "Hello world"
       print(string)

Renders as:

def hello_world():
    string = "Hello world"
    print(string)

Images

Prefer the SVG format for images, and save it as Optimized SVG in inkscape to use less space.

Store them in a hierarchy, do not use images subdirectories. The idea is to have simpler relative paths, for example, e.g.:

.. image:: ad2234_sdz_schematic.svg

Or even simpler schematic.svg, if already in a ad2234 directory.

Instead of overcomplicated paths like:

.. image:: ../../project/images/ad2234_sdz/ad2234_sdz_schematic.svg

In general, this avoids dangling artifacts and keeps the documentation simple.

Git Large File Storage

Where applicable, Git Large File Storage (LFS) is used to replace large files with text pointers inside Git, reducing cloning time.

To setup, install from your package manager and init:

apt install git-lfs
git lfs install --skip-smudge

The files that will use Git LFS are tracked at .gitattributes, to add new files use a pattern at the repo root, for example:

git lfs track *.jpg

Or edit .gitattributes directly.

Common sections

HDL common sections

The More information and Support sections that are present in the HDL project documentation, are actually separate pages inserted as links. They’re located at hdl/projects/common/more_information.rst and /support.rst, and cannot be referenced here because they don’t have an ID at the beginning of the page, so not to have warnings when the documentation is rendered that they’re not included in any toctree.

They are inserted like this:

.. include:: ../common/more_information.rst

.. include:: ../common/support.rst

And will be rendered as sections of the page.

Dynamic elements

Dynamic elements refer to sections of the documentation that are updated using dynamically loaded metadata, scripts, and styles (modules).

The dynamic elements implemented:

doctools/metadata.json:
- The navigation bar at the top (repotoc entry).
- A banner at the top (banner entry).

The metadata/modules files are generated when core_repo is true in the conf.py. From the JavaScript side, it fetches first {content_root}[../versioned]/../doctools/[versioned]/metadata.json, which contains the basic metadata and the list of the extra scripts and styles.

Note

Path version is present and set if latest exists at {content_root}/../doctools and the stored version can be extracted.

Tip

To keep the documentation lean, the metadata file is cached on localStorage and is only fetched again after the threshold time expires.

To add new JavaScript modules and CSS styles:

Update ci/rollup.config.app.mjs.
Add the list of modules to adi_doctools/lut.py.

The rollup should always output to the static directory, the same as the base app.umd.js and app.min.css.

Example use cases/suggestions:

An ever improving search.umd.js, that follows the natural evolution of the doc (in progress).
A unified footer across releases, using DOM manipulation by a footer.umd.js.

Known issues

On Firefox on Windows, fetch requests can only happen to the same port (8000 -> 8000), so Serve daemon must be on port 8000

Documentation guidelines

Importing from DokuWiki to Sphinx

Directives and roles

Third-party directives and roles

Support custom JavaScript in hot reload

Repository-specific extensions

Indentation

Table of contents

Versioning

Exporting to PDF

Inner working

Local references

Numbered references

External references

In organization reference

Outside organization Sphinx reference

Text width

Tables

Lists

Code

Images

Git Large File Storage

Common sections

HDL common sections

Dynamic elements

Known issues