DevSecOps Engineer, Author, and Mentor

Over the last several weeks, I have written about three Doxygen filters: bash-doxygen, awk-doxygen, and python-doxygen. Seen together, that sequence could suggest that I had decided to write a Doxygen filter for every language I use, although that was never the goal. I was trying to create a coherent documentation structure across the languages I use regularly while allowing the maintained source in each language to remain recognizable and appropriate to that language.

That distinction became more important as the work progressed. Bash needed a filter because Doxygen does not understand Bash well enough to infer the structures I wanted documented. AWK needed a different filter because its functions, globals, locals, and pattern/action rules do not map cleanly onto the Bash model. Python already had a mature documentation system, so its problem was less about inventing structure and more about translating Python-native docstrings without allowing Doxygen to become the authoring language. By the time I reached PHP, I had enough examples to ask a better question than “what should the next filter look like?” The more useful question was whether PHP needed a filter at all.

It did not. PHPDoc-style DocBlocks and Doxygen overlap closely enough that I could define a common subset and let Doxygen consume the maintained PHP directly. That result clarified the larger architecture for me: the consistency I wanted belonged in the documentation contract and the generated reference material, not in forcing every source language through identical syntax or identical tooling.

Read More

After writing Doxygen filters for Bash and AWK, Python looked as though it ought to be the straightforward one. Python already has documentation syntax, and Doxygen already understands Python, so the space between them appeared smaller than the problems I had solved for Bash and AWK. The interesting part turned out to be that both systems already had established ideas about what Python documentation should look like and how it should be interpreted.

With Bash and AWK, I was introducing documentation structures into languages where Doxygen needed substantial help understanding what I wanted documented. Python was different because it already had docstrings, PEP 257 conventions, type annotations, and established Sphinx/reStructuredText fields. I wanted to preserve that model rather than replace it with Doxygen-specific Python merely because Doxygen happened to be one of the publication targets. That distinction became the organizing principle for python-doxygen: the maintained source should remain ordinary Python, while the translation needed by Doxygen should happen at the Doxygen boundary.

A function could therefore remain documented in a form familiar to Python developers and Python-oriented tooling:

def load(path: str) -> str:
    """Load a value from ``path``.

    :param path: Path to load.
    :returns: The loaded value.
    :raises ValueError: The path is invalid.
    """

I did not want developers to maintain a second documentation block beside that docstring or learn a private documentation dialect for this one output format. The source documentation should remain useful to Python linters, IDEs, documentation tools, and anything inspecting __doc__. The problem was therefore less about inventing documentation syntax and more about reconciling the syntax Python already used with the structure Doxygen expected.

Read More

A few days ago, I wrote about bash-doxygen, a Doxygen filter I built for documenting Bash functions and variables. There is a mildly amusing detail buried inside that project: the core of the filter is itself an AWK script.

That is hardly unusual.

I have encountered AWK repeatedly over the years in places where Bash alone stopped being the right tool for a particular part of the job. A shell script might orchestrate commands, files, and processes, then reach a point where it needs to parse records, transform structured text, maintain state, or perform more substantial pattern matching. AWK fits that space well.

It is also one of those tools that seems to exist almost everywhere while receiving surprisingly little attention.

That combination creates an interesting maintenance problem.

AWK programs are often compact, capable, and stable enough that somebody writes one, gets it working, and then leaves it alone for a long time. Months or years later, another person opens the file and encounters an execution model that differs from the surrounding shell code, along with variables that spring into existence through use, pattern/action rules that behave unlike ordinary functions, and function parameters that may conventionally serve as local variables.

That is exactly the sort of code for which documentation earns its keep.

So I built awk-doxygen.

Like bash-doxygen, awk-doxygen is a documentation-led Doxygen filter. It takes explicitly documented AWK constructs and translates them into a small Doxygen-friendly pseudo-C++ representation. It does not attempt to turn AWK into C++, and it does not claim to be a complete AWK parser.

The maintained source remains AWK. The generated representation exists only so Doxygen has something it already knows how to index.

Read More