AI SystemsAgents fail in the gaps between steps
Most AI agents do not fail at any single step. They fail in the gaps between them. Each tool call can work in isolation and pass its own test. Then the agent chains them: one tool's output becomes another's input, a retry takes a different path, context drifts across ten steps, and the result wanders from the intent. The failure lives in how the steps fit together, which is exactly what testing each one on its own will never catch.
When an agent goes wrong over a long run, how do you find where the chain actually broke?
ProductEvery API you expose is a product
The moment another team builds against an interface, it has users, expectations, and a support burden, no different from something shipped to customers. What it does not get is the versioning, the deprecation discipline, or the change communication a real product receives. So it ossifies: too depended-on to change freely, too informal to maintain well. The teams that handle this apply a fraction of their external product discipline internally: a named owner, a clear sense of who depends on the interface, a deliberate decision before a breaking change rather than an apology after one.
Which of your internal interfaces is quietly being treated as a product by everyone except the team that owns it?
AI SystemsConfident and wrong
The failure that matters in an AI system is not the one that throws an error. It is the answer that is fluent, confident, and wrong. A normal bug makes itself obvious: something breaks, an error shows up, someone notices. A wrong answer from a model looks exactly like a right one, well-formed and certain, and it flows straight into a decision while nothing objects. Everything you build to catch failures is looking for something that looks like a failure, and this never does.
How do you catch an AI output that is wrong but gives no sign of it?
EngineeringWhen the constraint stops being technical
Technical depth matters. At some point it stops being the primary constraint, and the problems that remain are not implementation problems. Resolving implementation problems quickly and knowing which ones are worth solving are different skills. When the constraint shifts, the role starts depending on the second one.
When did you notice the problems you were solving had changed?
EngineeringKnowing what a decision is worth
A team will spend a week choosing a logging library it could replace later at little cost, then lock in a public API it can never quietly change, almost without noticing the second was the bigger decision. Caution rarely lands where the stakes actually are. The reversible calls get over-thought, while the one-way doors get made in passing, by whoever happened to be in the room. The senior skill is not making good decisions. It is telling, before spending any energy, which kind of decision is in front of you.
How do you decide which calls deserve real time and which just need to be made?
AI SystemsThe eval set is the asset
When models turn over every few weeks, the model itself is the disposable part. What lasts is the way you measure it. A good evaluation set, the representative inputs and the bar for what counts as good, outlasts any single model. It is what lets a team adopt a new release in days instead of a quarter, because they can actually tell whether the new model is better for their case. Without it, every model change is a leap of faith dressed up as an upgrade.
What do you trust to tell you a new model is actually better for your product?
EngineeringWhat an incident review has to produce
Walk out of most incident reviews and you leave with a list of process changes: a new checklist, an extra approval, one more step in the runbook. What you rarely leave with is the ability to see the same failure coming next time, and the failure that recurs is almost never the one the new process was written to stop. Process is the easiest thing to add after an incident, and the easiest to mistake for learning.
Before you close an incident, what do you insist it leave behind?
EngineeringInterfaces as commitments
An interface is one of the few things a team builds that it cannot quietly take back. Once other people depend on it, every change becomes a negotiation with all of them at once. That is what makes interface design a leadership decision, not an implementation detail. The code behind it can be rewritten at will. The shape exposed to the world is a commitment that outlives the people who chose it.
Which interface are you still building around today, because changing it would cost more than living with it?
EngineeringThe metric that gets reported and the one that matters
The metrics that get reported up and the metrics that determine what happens in production are usually different. Deployment frequency is easy to track and easy to report. Recovery reliability shows itself only in the incident timeline. The first looks like delivery health. The second is delivery health. When these two diverge, the gap surfaces during the first incident that requires real coordination to recover from.
Where does that gap usually show up first?
AI SystemsModel switchability, not model chasing
New AI models arrive almost every week. Keeping up with every release is not the goal. What matters is how easily you can move to the next one. This is not about choosing the right model, it is about how the product is built around it. When a model is easy to swap, evaluating a new release becomes a business decision rather than an engineering exercise.
What signals help your team decide when a new model is actually worth adopting?