DevSecOps Engineer, Author, and Mentor

After writing Doxygen filters for Bash and AWK, Python looked as though it ought to be the straightforward one. Python already has documentation syntax, and Doxygen already understands Python, so the space between them appeared smaller than the problems I had solved for Bash and AWK. The interesting part turned out to be that both systems already had established ideas about what Python documentation should look like and how it should be interpreted.

With Bash and AWK, I was introducing documentation structures into languages where Doxygen needed substantial help understanding what I wanted documented. Python was different because it already had docstrings, PEP 257 conventions, type annotations, and established Sphinx/reStructuredText fields. I wanted to preserve that model rather than replace it with Doxygen-specific Python merely because Doxygen happened to be one of the publication targets. That distinction became the organizing principle for python-doxygen: the maintained source should remain ordinary Python, while the translation needed by Doxygen should happen at the Doxygen boundary.

A function could therefore remain documented in a form familiar to Python developers and Python-oriented tooling:

def load(path: str) -> str:
    """Load a value from ``path``.

    :param path: Path to load.
    :returns: The loaded value.
    :raises ValueError: The path is invalid.
    """

I did not want developers to maintain a second documentation block beside that docstring or learn a private documentation dialect for this one output format. The source documentation should remain useful to Python linters, IDEs, documentation tools, and anything inspecting __doc__. The problem was therefore less about inventing documentation syntax and more about reconciling the syntax Python already used with the structure Doxygen expected.

Read More

A few days ago, I wrote about bash-doxygen, a Doxygen filter I built for documenting Bash functions and variables. There is a mildly amusing detail buried inside that project: the core of the filter is itself an AWK script.

That is hardly unusual.

I have encountered AWK repeatedly over the years in places where Bash alone stopped being the right tool for a particular part of the job. A shell script might orchestrate commands, files, and processes, then reach a point where it needs to parse records, transform structured text, maintain state, or perform more substantial pattern matching. AWK fits that space well.

It is also one of those tools that seems to exist almost everywhere while receiving surprisingly little attention.

That combination creates an interesting maintenance problem.

AWK programs are often compact, capable, and stable enough that somebody writes one, gets it working, and then leaves it alone for a long time. Months or years later, another person opens the file and encounters an execution model that differs from the surrounding shell code, along with variables that spring into existence through use, pattern/action rules that behave unlike ordinary functions, and function parameters that may conventionally serve as local variables.

That is exactly the sort of code for which documentation earns its keep.

So I built awk-doxygen.

Like bash-doxygen, awk-doxygen is a documentation-led Doxygen filter. It takes explicitly documented AWK constructs and translates them into a small Doxygen-friendly pseudo-C++ representation. It does not attempt to turn AWK into C++, and it does not claim to be a complete AWK parser.

The maintained source remains AWK. The generated representation exists only so Doxygen has something it already knows how to index.

Read More

Last week, I wrote about mktext, a small library that came out of a larger Bash project and became useful enough to deserve its own boundary. bash-doxygen is another tool in that family, although it solves a very different problem: turning carefully documented Bash source into something Doxygen can understand.

I have used Doxygen for years because I like keeping implementation-level contracts close to the code they describe. Bash is awkward territory for that approach. Doxygen understands several programming languages directly, but Bash is not one of the languages for which it can reliably infer functions, parameters, variables, and declarations on its own.

One answer would be to build a Bash parser. That would be a much larger project than the problem justified.

Instead, bash-doxygen takes a narrower approach. It looks for Doxygen-style comment blocks immediately followed by recognizable Bash declarations, then emits a small pseudo-C++ representation that Doxygen can index. Undocumented helpers remain undocumented. Source that does not participate in the documentation contract is left alone.

Read More