Last night I shipped a major refactor of HAVOC, my spec-driven development framework. It was a lot of work, involving a Github app, several prod data migrations, and, hundreds of prompts steering it home.
I finished at 1 am, rubbed my eyes and felt strongly reminded of my old-school dev days, where I'd grind away at a problem late into the night, until I could put it down, quiet my brain, and sleep.
This morning, I'm taking a moment to reflect. Why was this so hard?
It wasn't the feature itself, I've built these features, with these tech stacks before. It wasn't the scope, or a lack of planning. It was three things;
- Existing feature/code complexity, which had to be maintained
- Existing prod data, which had to be protected, and migrated
- The integration of new touchpoints that add to both the code complexity, and the schema complexity
None of that's new. This isn't my first AI rodeo. But something did stand out to me this time.
Here’s the best way I can describe how AI dev feels to me, with zero data. 100% subjective opinion.
The pattern is consistent. The first prompt delivers massive progress... often a one-shot of the core features of your project. The second prompt delivers maybe half of that progress. The third prompt, one-third, the fourth, one-fourth, and so on.
It's a productivity power curve, and it's very Zipf-shaped.
At the end, it settles pretty much to a horizontal line, very much like traditional software engineering. At that point, I expect I'm still 2x more efficient than I was manually writing code, but it's still a slog, and sloggy in a different kind of way. Most of my efforts are around trying to understand what the AI has built, and how to fix or adjust it for my purposes.
So then. Perhaps the next frontier in AI-assisted software engineering is about AI debugging tools, visualizations, environment controllers. The things that help both devs and AI's understand and manage complexity,
And I'll add this. Visualizations are a game-changer. It's one thing to scroll through pages of code, or summaries about what the AI think it has built. It's another thing entirely to say "show me a sequence diagram of this custom oauth integration."
And to that end... mermaid is kicking markdown's butt.
