From 4 hours to 8 minutes: anatomy of an order-processing automation

This is a teardown of one automation we shipped for a logistics client in Gujarat. It is not a hero story. The first version had bugs, week two was rough, and one design choice we were proud of turned out to be wrong. The numbers at the end are real, and so is everything before them.

The before state

The client moves freight for around 60 regular customers. Every customer sends orders their own way: PDF purchase orders, scanned and photographed order forms, spreadsheets, sometimes a plain email with the items typed in. Two people spent most of every morning reading those and keying them into the ERP.

We measured before we touched anything. Median time from email arrival to a posted sales order was a little over 4 hours. Around 4% of orders needed a correction after posting, usually a quantity or a delivery date. The two operators each handled about 90 orders a day and both said the same thing: the work was not hard, it was just endless.

The five stages

The pipeline is a plain queue-driven workflow. Each stage is a separate worker with its own retry policy, and each stage writes its output to a document store so we can replay any order from any point.

1. Ingest

A poller watches the orders mailbox, pulls new messages, splits attachments, sniffs the type, and stores everything with a content hash. The hash matters: customers resend the same PO more often than you would think, and duplicate detection at ingest removed a whole class of double-posted orders on day one.

2. Extract

Images and scanned PDFs go through OCR. Text PDFs and spreadsheets are parsed directly. Then the raw text, plus the page image where there is one, goes to the model with a strict output schema: customer reference, PO number, delivery date, and line items with SKU text, quantity and unit. Every field comes back with a confidence score between 0 and 1.

The model call is routed through a small abstraction that can switch providers. That is not a nice-to-have. When one provider's latency spiked for an afternoon in month three, the pipeline failed over and nobody noticed until they read the logs.

3. Validate

Pure code. Schema validation first, so a malformed response never reaches business logic. Then rules: quantity must be a positive integer, date must be in the future and within 90 days, customer reference must resolve to an active account. Each failed rule lowers the order's overall confidence rather than rejecting outright, because some failures are the model's fault and some are the customer's.

4. Reconcile

Line items are matched to SKUs and open contracts in the ERP. Exact match on customer part code first. Then a normalised match (strip punctuation, unify units). Only then a fuzzy match with a score. Fuzzy matches always carry lower confidence, and a fuzzy match below 0.85 is treated as unmatched.

5. Post

If every field clears its threshold, the order is written to the ERP through its ORM layer as a draft sales order and then confirmed. No raw SQL, ever. If anything is below threshold, the order lands in the review queue with the original document, the extracted fields, and the specific field that failed highlighted.

The human-in-the-loop gate

The thresholds are per field, not per order, because the cost of being wrong is different per field. A wrong quantity ships the wrong truck. A slightly off customer reference just needs a click.

# confidence thresholds, per field
auto_post_if_all_above:
  customer_ref: 0.95
  po_number: 0.90
  delivery_date: 0.92
  line.quantity: 0.97
  line.sku_match: 0.93
review_queue:
  sla_minutes: 30
  show_fields_below_threshold: true
kill_switch: ORDERS_AUTOPOST_ENABLED

The reviewer sees the document and the extraction side by side, fixes the field, and approves. Every correction is stored against the original extraction. That corpus is how we tuned thresholds in month two and how we caught the week-two problem below.

What broke in week two

Week one was quiet. Around 55% of orders auto-posted, the rest went to review, and the operators cleared the queue in under an hour. Then week two happened.

Three things went wrong at once. First, one large customer switched their PO template. The new layout put the delivery date in the header and the model, primed by weeks of their old layout, kept reading a print date instead. Confidence stayed high because the model was confident. Twelve orders auto-posted with the wrong date before an operator spotted the pattern.

Second, we had been proud of the fuzzy SKU matcher. It turned out that two of the customer's product families differed only by a suffix, and the matcher scored the wrong one at 0.88. Above our threshold. Six orders, wrong SKU.

Third, the review queue SLA alert was wired to the wrong channel, so nobody saw that the queue had backed up on a public holiday.

The fixes were unglamorous. We added a per-customer layout fingerprint: if a customer's document looks structurally different from their last twenty, the order goes to review regardless of confidence. We raised the fuzzy threshold to 0.93 and added a rule that any two candidate SKUs within 0.05 of each other force review. And we tested the alerting by actually breaking it.

Confidence is the model telling you how sure it is. It is not the model telling you it is right. Those are different numbers and only one of them is available.

— Jaimin Shah

Results

By month three, 78% of orders auto-post. Post-posting corrections dropped from 4% to under 1%, mostly because the review gate catches things a tired human at 11am did not. The two operators still exist and still matter. They handle the exceptions, chase customers about bad POs, and review the model's near misses. Their day got shorter and more interesting, which is the honest version of “AI made the team more productive”.

What we would do again

• Measure the before state for two weeks. Every number in this article exists because we did.
• Keep the model in one stage. Everything else is code, which means everything else is testable.
• Per-field thresholds, per-customer layout fingerprints, and a review queue with a real SLA and a tested alert.
• A kill switch that flips auto-posting off and routes everything to review. We used it once. It took 40 seconds.
• Store every correction. It is your evaluation set, your threshold tuning data, and your audit trail in one place.

The pattern generalises. Invoices, delivery notes, supplier confirmations, anything that arrives as a document and needs to land in a system. The stages stay the same, the schema changes, and the gate is always where the value is.