Bootstrap, an engineering exercise (part 2)
5 min read
The first part of this series talked about not wanting to write code and how the deliverable for this project needed to be immediately useful on a fresh, minimal system. This next part talks about engineering practices and common AI problems.
Engineering Practices
There is a lot of AI-generated code out there, often called "slop." It's often sloppy, poorly-conceived, obtuse, and bordering on a level of design craft so as to resemble obfuscation. In many ways, a lot of it stands in opposition to the very principles I hold to be inviolate.
This experiment was to be something other than the hastily-generated code that has become a plague to modern development.
Documentation
First and foremost, decisions needed to be documented. As the engineer on the project, I would make the decisions after examining the alternatives; those decisions would then be documented along with the reasoning I used. Each development step would start with asking the AI for any questions it may have and to lay out any assumptions it was making so that I could confirm or refute them.
When code was drafted, I used a documentation-first approach. That is, instead of a few Architecture Decision Records (ADRs), everything would be documented. As a result, there were dozens of ADRs before the first line of code was written.
Some may argue that this is more waterfall than agile in terms of project lifecycle. I disagree. While there were dozens of ADRs at the start, dozens more were made as the project grew. ADRs were updated as changes were needed.
Each file, class, method, function, and global variable was documented. I decided to use Doxygen structure so that not only would the particulars of the documentation be accessible by humans and machines, but so that I could render a documentation corpus for later perusal and reference. I wanted to quickly see where a function was defined, what it did, why it did what it did, how to call it, what it returned, what it required, and what assumptions it made.
Additionally, this documentation, because it captured the "why" and not just the "what" of the code, made updating that code a more straightforward process. I wanted to constrain and guide the AI down the roads that I wanted it to go, not whatever paths the model's training suggested would be most appropriate at that exact moment. I wanted the AI to have very definitive starting and ending points; I didn't want it to wander. I didn't want it to wonder. The less guessing the AI would do, the better.
On a recent project, I updated a little bit of code in one file to do a single thing. However, when reviewing the patch, I saw that it adjusted a table in a README into an ordered list. The documentation it updated had nothing to do with the code being changed. I don't know why it decided to do that. In fact, that same AI drafted the table only a few commits prior. For the new project, I wanted to avoid situations like that as best I could.
Changes to functionality, observations of broken code and failed logic, and instances where former assumptions were disproved needed to be documented in the form of issues on the project.
Modularity
Here's where things got fun. I realize that namespaces and plugins and discoverability and modularity aren't often associated with Bash scripting. My vision for the project, however, required a different perspective. I wanted to be able to drop a source code file in a directory and have it immediately integrated into the application. For example, the project's first path was to support Debian-based Linux distributions, such as Ubuntu. As a result, it relied on APT-based tooling. At the same time, I knew that I wanted to support Alpine Linux with its APK-based tooling. I didn't want to have to include some code somewhere that codified the logic of "if the system is using APT, do this; if it's using APK, then do that" in some kind of case statement or a bunch of if statements. Want to add APK support? Add the APK support file. DNF support? Add the DNF support file.
Well-designed interfaces, usable selection criteria, extensible logic, and a solid contract were absolute requirements. The interface needed to be well-documented so that another developer (i.e., another interaction with the LLM) could be able to pick right up where the last left off.
Common AI Problems
The more I work with AIs for generating and maintaining code, the more I've come to think of each interaction with an AI to be similar to working with a different developer. This is more obvious when starting a new chat, but it's still notable with each subsequent prompt. The AI builds context around what it's doing, but that context is plastic and malleable. Each interaction builds and maintains context, but it also looks at what it's doing fresh each time, just in the light of the continually-updated context.
Therefore, the more meaning and understanding and reasoning I can capture and store through documentation, comments, contracts, and ADRs, the richer the internal model can be and more aligned with my vision it will be.