Agents vs. workflows: when an LLM belongs in the ops loop

“We want an AI agent for this” is the most common opening line we hear. About one time in five, an agent is the right answer. The rest of the time the client needs a workflow with one well-placed LLM call, and an agent would have cost three times as much to build and ten times as much to debug.

This is the framework we use to decide. It is not academic. It comes from shipping pipelines into ERP and CRM systems where a wrong write costs real money.

Three tools, not one

Deterministic workflow

Code. If-this-then-that, queues, retries, scheduled jobs. Same input, same output, every time. Boring, testable, cheap to run. Most of an ops loop should be this.

LLM step

A workflow that calls a model at one or two points to do something code cannot: read a messy PDF, classify a free-text email, map a supplier's column names to yours. The model returns structured output, validated against a schema, and the workflow carries on. The model never decides what happens next. Code does.

Autonomous agent

The model plans. It decides which tools to call, in what order, and when it is finished. This is powerful for open-ended tasks with no fixed path, like answering “why did margin drop in the west region last quarter” over an ERP database. It is the wrong tool for a task that has a known shape.

The decision table

Ask four questions about the task. The answers point at one of the three.

• Is the path through the task fixed? Yes: workflow. Yes, but one step needs judgement on unstructured input: LLM step. No, the path depends on what you find along the way: agent.
• Is the input structured? Fully: workflow. Partly (PDFs, emails, scans, free text): LLM step. Unknown until you look: agent.
• What does one wrong action cost? High and irreversible (posting to the ledger, sending to a customer): workflow or LLM step with a human gate. Low and reversible (a draft, a suggestion, a report): agent is acceptable.
• How often does it run? Thousands of times a day: workflow or LLM step, because agent latency and token cost compound. A few times a day by a human who is waiting for the answer: agent.
• Can you write the acceptance test? If you can list the expected outputs for fifty inputs: workflow or LLM step. If the best you can do is “a domain expert would say it is reasonable”: agent, with the expert in the loop.

Failure modes

Workflows fail loudly

A workflow hits an input it was not written for and throws. You get a stack trace, a dead-letter queue entry, a page. Annoying, but you know within minutes. The failure mode is brittleness, and the fix is a code change with a test.

LLM steps fail quietly

The model returns something plausible and wrong: a quantity of 100 instead of 1,000, a date in the wrong format, a product code that almost matches. Nothing throws. The defence is structural: validate every output against a strict schema (we use Zod-style validation on every field), attach a confidence score, and route anything below threshold to a human. Without that, quiet failures pile up until finance finds them at month end.

Agents fail expensively

An agent with a vague goal and a loose tool set will loop, call the same tool eleven times, or decide the task is done when it is not. Costs spike, the answer is late, and reproducing the failure is hard because the path was different every run. The defences are tight tool scopes, step budgets, allow-listed actions and a full trace of every call. And still, you keep agents away from writes unless a human approves.

Where the LLM sat in a 4-hour-to-8-minute automation

A logistics client was spending around four hours a day keying customer orders into their ERP. Orders arrived as email attachments: PDFs, scanned images, the odd spreadsheet, each in the customer's own layout. The request was “build us an agent that processes orders”.

We built a workflow. Five stages: ingest, extract, validate, reconcile, post. Four of the five are plain code. The model appears in exactly one place.

1. 01Ingest is a mailbox poller and a file-type sniffer. Code.
2. 02Extract runs OCR, then hands the raw text plus the page image to the model with a strict output schema: customer reference, line items, quantities, delivery date, each with a per-field confidence. This is the LLM step.
3. 03Validate checks the schema, checks quantities against pack sizes, checks the customer exists. Code.
4. 04Reconcile matches lines to SKUs and open contracts in the ERP with a deterministic matcher, falling back to fuzzy match only when exact match fails. Code.
5. 05Post writes the sales order through the ERP's own ORM layer, never raw SQL, and only when every field clears its confidence threshold. Otherwise it lands in a review queue. Code plus a human.

Median time from email arrival to posted order went from about 4 hours to 8 minutes. Manual entries dropped 80%. The model did the one thing code could not do, which was read a document it had never seen before. Everything around it stayed deterministic, so when something broke we knew which stage and which input.

If you can draw the flowchart, you do not need an agent. You need a workflow with a model in the box you could not draw.

— Jaimin Shah

When we do build agents

We build agents when the question is open-ended and the user is a person waiting for an answer. A conversational agent over an ERP database is the classic case: “which customers ordered less this quarter than last, and who owns those accounts” has no fixed path. The agent plans the queries, runs them through a read-only ORM-safe query builder, and explains the result.

Even there, the rules hold. Read-only by default. Tool scopes tied to the user's role. Every action allow-listed. Every call logged. Prompt-injection safeguards on anything that comes from the database, because customer notes are user input too. Autonomy inside a fence.

Start with the workflow. Add the LLM step where code cannot see. Reach for the agent only when the path itself is the unknown. That order saves money, and it saves the project.