becarpenter-book6/utilities/utDoc.md

The book6 utilities

Usage note

All the utilities are written in Python 3 and should work on most common operating systems (but have mainly been tested on Windows 10). They mainly use tkinter to provide a nice user interface with options. If invoked on the command line with the book6 directory path as a parameter, they will run with default options. For example:

python3 makeBook.py

will get you a GUI, but

python3 makeBook.py ~/whatever/book6

will just run.

makeBook

This program is applied to the entire book6 directory whenever significant changes have been made. The reader is assumed to have carefully read the Chapter Template chapter.

makeBook's functions include:

• Update the global contents (Contents.md) to reflect the actual chapter contents.

• Update the individual chapter files (./ChapterName/ChapterName.md) to correspond to the global contents.

• Create any missing section files (./ChapterName/SectionName.md).

• Reconcile discrepancies between the global contents, the chapter contents, and the actual section files. (Not all discrepancies can be automatically reconciled - makeBook will issue a warning if it detects such a case).

• Resolve any raw citations ({{something}} or {{{something}}}) into a complete markdown link ([something](URL)).

• Create and insert internal links (from a chapter file to the individual sections, from each section to its predecessor and successor, etc.)

• Reformat markdown text if appropriate.

After updating a section, especially if adding a new citation with {{ }} or {{{ }}}, it's appropriate to run makeBook.py (Python 3; requires tkinter; uses mdformat if installed).

To make changes other than just updating an existing section, also see chapterReorg.

makeBook works roughly as follows, using a tkinter GUI, not standard input.

1. Request name of book6 directory from user. Normally this will be the local copy of the GitHub repo.

2. Determine whether automatic checking of RFC existence is possible. If makeBook cannot access a recent copy of the RFC index, this checking is impossible and it logs a warning.

3. Determine whether automatic checking of I-D existence is possible. If makeBook cannot access the current I-D index, this checking is impossible and it logs a warning.

4. Read in the existing main contents (Contents.md).

5. Convert any plain text chapter names to links.

6. Scan main contents and create any missing chapter directories and blank chapter files, build a chapter list, extract section lists for each chapter, and add any missing section names to the main contents.

7. For each chapter:

• 7.1 If there is a remaining discrepancy between the list of sections in the main contents and the section files actually present, log a warning that makeBook must be run again to resolve it. Also log a warning if there is an uppercase/lowercase discrepancy (which really needs manual fixing).

• 7.2 Create and initialize any missing section files.

• 7.3 If anything in the chapter file has changed during steps 7.1 and 7.2 , write it back.

• 7.4 For each section in the chapter:

  • 7.4.1 Read in the markdown file, update internal links if necessary, expand any {{ }} or {{{ }}} citations as markdown links (but log a warning if this fails).

  • 7.4.2. If anything in the section file has changed during step 7.4.1, write it back.

• 7.5. Read in the chapter markdown file (again), expand any {{ }} or {{{ }}} citations as markdown links (but log a warning if this fails), and write it back if necessary.

8. Finally, write back the main contents.

Note that expanding citations is done by the function expand_cites() which is straightforward but tedious to describe.

makeBook issues warnings to standard output but also to the file makeBook.log in the main book6 directory. This log should always be checked after a run!

indexBook

This program is applied to the entire book6 directory. It reads each section file in each chapter, and if one of the index terms in file ./utilities/index6.txt is present, an index entry is created or extended, pointing to the section.

Index terms are not case-sensitive, and each term can appear in several variants. The details of that are given in index.txt itself, but for example the line

link-local 'link local' link-locals

means that all three terms will be indexed under link-local.

The program also builds a citation index, showing which RFCs are indexed by which sections.

At the end, the collated and sorted results are written to ./Index.md and ./Citex.md`

indexBook issues warnings to standard output but also to the file indexBook.log in the main book6 directory. This log should always be checked after a run!

RFCbib6

This program creates a list of all current RFCs that mention IPv6 in their title or come from the major IPv6 working groups. Obsolete RFCs are not included. There are subsections for Standards, BCPs, Informational and Experimental RFCs. It does this by processing a recent copy of the RFC index (in XML format) according to the above criteria. The result is stored as a book6 chapter in ./20. Further Reading/RFC bibliography.md.

RFCbib6 issues warnings to standard output but also to the file RFCbib6.log in the main book6 directory. This log should always be checked after a run!

It's best to run makeBook afterwards. (Note that RFC bibliography.md is ignored by indexBook, to avoid noise in the citation index.)

bakeBook

This program concatenates the whole book into a single "baked" markdown file (called baked.md). This isn't intended for users, but as an intermediate step on the way to a PDF version of book6.

Its main job, apart from concatenating all relevant .md files, is to fix up internal references to match the way GitHub creates anchors for chapter and section titles in ## Title format. The code is very ad hoc and has to special-case a few details.

After baked.md is written, the code attempts to complete the conversion to baked.pdf, via pandoc and pdflatex. If this fails, the manual process is explained in the pdf directory.

The title page is at Title.md and is augmented during baking by adding a timestamp.

One gotcha is that all graphics files (.png, .jpg, etc.) needed to be copied into the main directory, and their names must be unique book-wide. Another gotcha is that section names must also be unique book-wide.

A necessary hack is that where a page break is needed, this is signaled by a dummy line: backslashpagebreak. This is converted later to a genuine pagebreak.

Conversion to an eBook format is under investigation.

Further info is in the pdf directory.