Agentic AI is changing how fast teams can build. The question worth asking before you start is still the same one it’s always been.
I’ve been in a few conversations recently with consultants selling agentic AI capabilities. The pitch is compelling. Agents can work autonomously, iterate on tasks, chain actions together, and operate at a scale no human team could match. The demos are impressive. The potential is real.
What’s missing from most of those conversations is a question.
Not “what problem are you trying to solve?” Not “what does your current process look like and where does it break?” Not “what would success look like and how would you measure it?” Just: here’s what agents can do, here’s how we’d deploy them, here’s what it would cost.
The capability is the answer. The problem is implied.
I’ve been thinking about that pattern since reading Andrew Ng’s recent letter in The Batch about agentic coding loops. His framing is worth engaging with directly because it’s honest and specific in a way that a lot of the current conversation isn’t. He’s describing a practice for building 0-to-1 prototypes in domains you don’t fully understand yet. Use a coding agent to build something quickly, react to what it produces, refine the spec, repeat. The argument is that seeing a working prototype is often the fastest way to discover what you actually want to build. Human judgment is scarce and expensive. Agent output is cheap. Use the cheap thing to sharpen the expensive thing.
That’s a legitimate practice for a specific context. It’s not a general theory of how to use agents.
When Exploration Becomes the Default
The pattern Ng describes works because it has a boundary. You’re exploring an unfamiliar domain. You don’t yet have enough knowledge to write a good spec. The agent output isn’t the product. It’s a tool for developing the spec that will eventually drive the product. The iteration is purposeful. You’re getting smarter with each loop.
The problem is when that exploratory posture gets extracted from its context and applied to everything. When “put agents on it and iterate” becomes the answer to problems that aren’t unfamiliar domains but rather well-understood organizational failures, poorly defined requirements, or processes nobody has bothered to map. When the iteration isn’t developing the spec. It’s substituting for it.
This is the thousand monkeys problem. An infinite number of agents iterating on an undefined problem will eventually produce something that looks like a solution. You’ll have burned significant compute to get there, you won’t be certain the output is right, and you’ll still need a human to evaluate it against a definition of success that was never written down. The eval problem hasn’t been automated. It’s been deferred.
What makes this harder to see than it sounds is that the iteration looks like progress. Things are being generated. Outputs are appearing. The agent is doing something. With human teams, the cost of spinning on an undefined problem was visible. It showed up in headcount, in sprint reviews, in frustrated engineers asking what they were actually building. With agents, that friction disappears. The inefficiency doesn’t. It scales.
Solutions Looking for Problems
This isn’t a new pattern. Every capability wave produces it. Someone gets good at building a thing. The thing is genuinely impressive. The capability becomes the product. And the question of whether the capability fits the problem gets answered implicitly because the capability is the answer.
The organizations that get the most value from new technology are consistently the ones that start with a problem worth solving and find the technology that fits it. The ones that start with the technology and work backward to the problem spend a lot of time discovering that the fit isn’t what they expected.
Agents are not exempt from this. They’re extraordinarily capable. That’s precisely what makes the skipped question more dangerous, not less. The more capable the tool, the easier it is to mistake activity for progress.
The Question That Should Come First
Ng’s framing contains the answer to the problem it doesn’t fully address. He says coming up with the spec, the evals, and the test set is one of the hardest tasks in agentic AI. Then he offers techniques for managing that difficulty once you’re already iterating. What he doesn’t dwell on, because his context is exploration where this is expected, is what happens when you start iterating before the spec exists in a context where it should.
The question that should come before any agent deployment isn’t “what can agents do?” It’s the same question that should precede any technology investment: what problem are we solving, and how will we know when we’ve solved it?
If the answer is clear, agents may be exactly the right tool. They can execute at a scale and speed that changes what’s possible. But if the answer is vague, or if the answer is “we’ll figure it out as we go,” then what you’re buying is expensive iteration on an undefined target. The agent will work very hard in a direction nobody has specified.
That’s not a technology problem. It’s the same problem it’s always been.
When a consultant walks in selling agents, the right first question isn’t about the technology. It’s about what breaks in your organization today that you believe agents would fix. If they can’t answer that, or if they’re surprised you asked, that’s useful information.
The problem should come before the answer. It always should have.
