Agentic Engineering

Software engineering has completely changed. I spend a long time umming and ahhing at the extent to which that’s actually true. And what that actually means. Andrej Karpathy refers to it as a new programming paradigm in Software Is Changing (Again): we had low-level languages, high-level, and now we have Software 3.0, where the programming language is natural language. I think part of the reason I struggled to accept that at face value is because in my own experience I still felt as if I spent so much time living in the code.

When you see someone like Boris Cherny say that he just looks at his phone, has deleted his IDE, and just responds to a set of notifications about a whole host of tasks that are being done — I cannot relate.

Of course, part of this is a skill issue.

Another part, I suppose, must be the fact that my resources are so much more constrained. I still baulk at the thought of spending $200 per month on a Claude Code subscription. I remember going to an event and talking to someone who had about five subscriptions across different accounts, just so they could min/max the fact that the extra credits you get out of Max vs Pro live on some kind of curve.

I think at the time I wasn’t even hitting the limit on my Pro subscription.

Now, of course, things have changed. The Claude Pro plan feels so puny, a $200-per-month subscription so scary that OpenCode and its family of cheaper models are my favourite tools to use.

The way that I’ve been using agentic tools to write code has changed, and the project we’ll talk about in this article details the latest things I’ve been exploring.

Nurture Over Nature

Until recently my approach had essentially been: agent as flat-out code generator. We do a lot of planning together then I lean on the agent’s own parallelisation feature for any parallel work, and I stay embedded in the process the whole way through.

I took a lot of inspiration from The Hidden Layers of AI Systems: Memory, Retrieval, and Search, an MLOps Community London meetup featuring talks from Ewa Szyszka (Qdrant), Shashi Jagtap (Superagentic AI), Dmytro Yaroshenko (Factory AI) and Talha Sheikh (Checkout.com) on memory, retrieval and context for AI systems. My main takeaway was the idea that you should set your coding agents up for success by equipping them with the appropriate environment to work in.

The Project

The problem I pointed this at: PP/DS, a module inside SAP S/4HANA that decides what gets manufactured, on which machine, in what order, and when. PP/DS is to a factory what the Kubernetes scheduler is to a cluster (wink-wink-nudge-nudge at another project). And much like scheduler failures, PP/DS failures can be cryptic: the system tells you that something is wrong, rarely why, and the answer is buried somewhere across help pages, community threads and configuration screens.

So the demo is a copilot: a planner types a question in natural language and gets back a structured diagnosis with citations, and a graph they can click through to see how the system got there. The “only” missing piece is a connection to an actual PP/DS instance.

Why PP/DS?

Maybe I’d like to own a factory one day…

The Lab

In the ancient pre-coding-agent times, if I had a problem like “ingest these web pages with Docling and turn them into the kind of knowledge graph I want”, I’d probably sit down, go through the Docling documentation, form an idea of the docling-to-kb-graph transformation I wanted, and build it myself. The knowledge graph is central to what we’re doing here, but it felt nitty-gritty to me to have to be going through the Docling documentation to get at it when I had a vision of the graph I really wanted. Plus my agent can read the docs faster than me!

Instead I set the criteria I care about and let a set of parallel agents come up with good strategies that I can plug into the rest of the infrastructure. The criteria did change over time but settled around: is the graph fully connected and can the markdown we regenerate from it read very close to the original page. It probably warrants deeper analysis, but it was good enough.

The loop goes like this:

• An orchestrator — the agent I talk to — designs a batch of strategy hypotheses. Each hypothesis becomes a run directory containing a BRIEF.md: the hypothesis, what this strategy tries to solve; and an AGENT.md: the worker’s instructions and output contract.
• Cradle, the harness we built, scaffolds those directories and launches a worker agent into each one, non-interactively.
• Each worker implements a strategy.py — a build(doc) function that turns a parsed document into a typed graph — runs it over the whole corpus, writes its results, and writes a REPORT.md last of all. The report is the sentinel: cradle blocks until every sentinel appears, so a run isn’t “done” until the worker has written up what it built, what surprised it, and what it suggests trying next.
• A watcher picks up the artefacts, ingests them into a database, and scores every strategy centrally against that golden set of metrics. Workers are explicitly forbidden from scoring themselves.
• The orchestrator reads the leaderboard and the reports, forms new hypotheses, and spins up the next round.

Because the artefacts the individual agents produce land in a structured place, the orchestrator can see the results, see the strategies, and come up with new hypotheses without me in the middle. The lab is public on Codeberg if you want to poke around.

I also wanted to look at these graphs myself. Diagnostically, and honestly also because it’s entertaining watching a batch of labs churn out a bunch of different graphs. So there’s a gallery.

The graph explorer we used to reason about what the strategies were producing. The hairball is the “spine” I was referring to. This is where the different graphs get glued together.

There is an endless debate about whether all this actually makes you move faster. I definitely still should’ve just read the Docling documentation! But it certainly feels as if this would have taken longer without the agentic engineering tools. I think I am for sure sacrificing some awareness of the intricate parts of the system in favour of being able to architect larger solutions in less time.

And the workflow was always brittle; it needs whittling down into something smooth. But as a way of exploring agentic engineering, coming up with a system that produces code, rather than me being embedded throughout the process, I will be applying similar techniques to future projects.

Especially since I intend on making this a generalisable solution that can source information from a range of different places and automatically produce graphs with the structure I want.

Maybe a future system will try to ingest a new web page or some other kind of source, run it through our existing strategy, find that it does not perform well on our metrics, and then run a bunch of labs to ensure that those kinds of source documents can have better graph representations that are aligned with our standard, before being woven into the existing knowledge graph for an agent.

Building the Application

The lab was a tight loop with a very specific definition of good. Building the application around it was a wider, looser problem, and it needed a different mode.

Have a looong conversation with the agent, thanks Matt Pocock for the skills, split the work into streams with their own markdown file, and spin up an agent per stream. At times there were three to five agents working on different things. I open each agent in its own window and interact with it as it works, steering as it goes rather than letting OpenCode take care of this non-interactively. The lab was very specific about what we wanted; building an application in general has a lot more scope for different interpretations, and it requires that kind of guiding.

I keep an eye on them. I have the initial markdown files that we generated as plans for each work stream, so I know what each one is doing, and I have a picture of the wider system they’re all trying to create. The failure modes were fairly predictable. The classic: an agent kicks off a long-running command, gets stuck inside the call, and doesn’t even know it has failed. It’ll just sit there.

Old-friend tmux was there for the rescue. Long-running commands go in a tmux window; the agent hooks into that window and polls it so it can see what’s going on rather than getting trapped inside the call. The nice side effect is that I can open the same pane myself.

The Agents can still feel silly

Beyond stuck commands, the other thing I watch for is an agent doing something that looks suspicious. If something smells off, I just stop it and ask.

For example, we were adding embeddings to the graph database, and I could see each addition crawling by, one slow write at a time. I stopped it. What are you doing? What does the ingestion strategy look like? It turned out it was inserting embeddings in a way that tied each vector intimately to the existing schema, incrementally updating every table it was interested in.

So we changed the shape of it: one huge table with all the vectors and a bunch of references saying where each one belongs. Instead of incremental per-table updates, a handful of bulk uploads. This definitely felt as if it was something that should be obvious. And maybe partially why I still feel uncomfortable handing over the reins completely. What other silly mistakes did I not catch by not having an intimate relationship with the code? How much longer should the planning sessions have been to iron that out before it became an issue?

Conclusion

The mindset shift, as far as I can tell, does not have to be too severe. We’re still dealing with the same abstract problems we always dealt with in software engineering. Decomposition, parallelism, feedback loops. Before, if I had a problem with several plausible solution strategies, there was no real way to parallelise the thinking other than blood, sweat and tears. Now I can hand a harness to a set of agents, have them generate strategies in parallel against a gold standard, and let an orchestrator loop on the results. Same problem-solving instincts.

The learning journey is not over

I’m looking forward to getting better at agentic engineering as it does feel like a skill aside from traditional engineering. I will be honest, there’s a certain joy that I miss about wrangling syntax. But it is well balanced with the capability to build more complex systems quicker. There’s a joy to building things and a joy to just having them exist.

June’s OpenCode Zen bill, broken down by model. Spot the single Claude day.