The demo goes well. Someone types a question in a meeting, the agent answers in a paragraph that sounds like a competent colleague wrote it, and everyone leaves the room enthusiastic. Four months later there is a pilot nobody uses, a subscription nobody cancelled, and a quiet consensus that the technology isn't ready.
In most of those cases the technology was fine. What went wrong sits around it: the job was too broad, the agent had no access to the records that would have made its answers correct, and nobody agreed what it should do when it wasn't sure. Building AI agents for business is mostly those three decisions, made before anyone writes a prompt.
Pick the job by arithmetic, not by ambition
Most teams reach first for the interesting work. A better first target is the boring, high-volume task that three people already do by hand and nobody enjoys.
Do the sum out loud before committing. Take the number of times the task happens each week, multiply by the minutes it takes, and multiply by a loaded hourly cost. Sixty ticket triages a day at three minutes each is fifteen hours a week. Twelve invoices a month at eight minutes is not worth automating, however annoying those twelve invoices are.
Good candidates share a shape:
- It happens dozens of times a week, not dozens of times a year.
- A competent new hire could be taught it in a morning.
- The inputs arrive in a consistent form: an email, a PDF from the same five suppliers, a ticket in a queue.
- Being wrong occasionally is recoverable, because a person sees the output before it reaches a customer or a ledger.
That last one is doing more work than it looks. It is what makes the first project safe enough to ship in weeks rather than after a governance review that runs to Christmas.
Anthropic's engineering write-up on building effective agents makes a related point about architecture: start with the simplest thing that could work and add complexity only when it demonstrably improves the result. Most first projects that stall were multi-step, multi-tool systems where a single well-scoped step would have done.
Ground it in your own records
An agent answering from general knowledge is a plausible stranger. An agent answering from your help docs, your past tickets, your contracts and your product data is a colleague who has read the file.
Retrieval is what makes that difference, and it brings a second benefit that matters more over time: the answer can cite where it came from. A response with a link to the source paragraph is checkable in ten seconds. A response without one has to be trusted or verified from scratch, and people quickly stop doing the second thing.
Grounding also exposes an uncomfortable dependency. The agent is reading whatever your systems actually contain, which means the duplicate customer records, the three versions of the returns policy and the pricing sheet last updated by someone who left in 2024. Cleaning that up is not glamorous and it is frequently the majority of the project. It is the same discipline as building pipelines that don't break at 3am, applied to the documents rather than the tables.
A short honest audit before you start: which sources are current, who owns each one, and what happens when two of them disagree.
Design the handoff before the happy path
Every agent that survives contact with real users knows when to stop.
Decide these four things while the project is still on a whiteboard:
- The confidence threshold. Below what level of certainty does the agent stop answering and pass to a person?
- What "pass to a person" means in practice. A queue with the conversation attached, a named team, an expected response time. Not a dead end that says the assistant cannot help.
- What gets logged. Every question, every retrieved source, every answer, every escalation. You cannot improve what you cannot read back.
- What it is not allowed to do at all. Issue refunds, promise dates, quote prices outside a range, send anything externally without review.
The escalation path is what buys you permission to widen the scope later. An agent that handles 60% of a queue cleanly and hands over the rest with context is a colleague. One that attempts everything and is wrong 15% of the time gets switched off in a month, and the next proposal takes a year to get approved.
Measure it like a new hire
Week one is where most of the value gets decided, and most teams skip the only step that makes it measurable: writing down the current numbers before anything changes.
Get a baseline first. How long does the task take now, how many happen per week, what does the current error rate look like. Then watch four things after launch:
- Acceptance rate. How often does a person send the agent's output as-is?
- Edit distance. When they change it, do they tweak a sentence or rewrite the whole thing? Heavy editing means the agent is generating work rather than removing it.
- Escalation rate, and whether it falls. It should drop over the first month as the gaps in the source material get filled.
- Time to resolution on the tasks it touches, compared with the baseline.
Review it in week one, not quarter one. If acceptance is low, the fix is almost always narrower scope or better source material, and both are cheap to do early.
What stalls the ones that fail
A pattern repeats across the projects that quietly die:
- No owner. The agent belongs to a project, the project ends, and nobody is responsible for the answers it is still giving.
- Scope that grew before quality was proven. A ticket triager becomes a ticket triager that also drafts replies, quotes prices and updates the CRM, and now nothing is reliable enough to trust.
- No baseline, so nobody can say whether it helped. The conversation about renewal becomes a matter of opinion.
- Source material nobody maintains. The agent was right in March and is confidently out of date by September.
None of these are model problems, and none get better by upgrading to a larger model.
Where to start this week
Take the queue your team complains about most. Count how many times it happens in a week and how long each one takes, then write down the two facts an answer needs to be correct and where those facts currently live.
If the facts live somewhere a system can read, you have a first agent worth building. If they live in someone's head or in an inbox, you have a data problem to solve first, and solving it is worth doing whether or not an agent ever gets built. That sequencing is most of what we do when we build and integrate AI agents, and it is why the useful ones tend to look unremarkable from the outside.
