How 63,053 Trade Tickets Were Reclassified in 87 Hours With a Local LLM

A commodity trading firm needed to reclassify 63,053 trade booking and reconciliation records into a new root-cause taxonomy. The data had to stay on a local machine. Cloud APIs were not an option. The naive approach – calling an LLM for every record – would have taken roughly 14 days on a 7B-parameter model running locally. The business window closed before that run finished.

The implementation that completed the batch instead finished in 87 hours on a MacBook, unattended, according to a description of the project. The core insight was not better prompting. It was routing.

The firm's coding agent built a multi-layer waterfall that moved deterministic and similarity checks to the front of the pipeline and reserved the LLM for genuinely new cases. The records were text-heavy: tickets, incident reports, survey comments. A rules-based system lost accuracy over time as people described the same problems differently. Supervised ML was not an option because the 105-category labels did not exist yet.

So the team used a zero-shot LLM approach, but with reuse. The first time a pattern appeared, the LLM made the decision. After that, semantically similar records reused the same result at embedding speed, avoiding a second LLM call. The waterfall worked because it used the cheapest possible signal first: exact-match fingerprinting, then fuzzy string matching, then embedding similarity, and finally the LLM. Each layer covered a different class of similarity. The system only allowed reuse when guardrails passed – a minimum confidence score, a freshness check, and a prohibition on reusing a decision that itself had been reused more than once. If any guard failed, the pipeline fell through to the next layer.

Before the waterfall ran, a deterministic rule layer applied regex patterns to map known-certain phrases directly to categories, bypassing all caches and the LLM. The output recorded which strategy produced each classification and, when reuse occurred, the similarity score as match_confidence.

The hard part was keeping the pipeline correct and resumable while it ran for days. The system wrote output as append-only JSONL and stored progress in a checkpoint keyed by a deterministic row ID. At startup, it reconciled existing output files to avoid duplicates. The team also encountered a bug where a truthiness check on an empty cache object caused the pipeline to skip seeding on every row. The fix was explicit is not None checks.

A circuit breaker prevented a single transient timeout from shutting off embeddings for the rest of the run. The pipeline used a small retry budget, tripped only after consecutive failures, and auto-recovered on the first success. Telemetry tracked throughput by strategy, cache hit rates, and circuit breaker trips, allowing the coding agent to tune thresholds without ever reading the incident text.

The strategy mix from the resumed segment of 44,803 rows showed that the fingerprint cache handled 23.7% of records, the fuzzy string cache 11.4%, the semantic cache 5.2%, and the LLM 55.4%. The deterministic rule layer handled 4.3% outside the waterfall. The review queue – records where confidence fell below the threshold – contained 204 records, or 0.3% of the batch. The team said the pipeline produced a bounded worklist a human could actually review, instead of hiding uncertainty inside the bulk output.

One representative reuse output looked like this: row_hash matched a fingerprint at 1.0 confidence, strategy was 'fingerprint_cache', taxonomy category was 'Network_Outage'.

The project used a selection policy per run: the operator chose which text field to feed each layer, avoiding contradictory fields. A separate representation, stripped of HTML and variable identifiers, was used for matching only. That was what made two "different" tickets hash the same way.

The team noted that cache warming was not linear. The first slice of a cold run was always slowest because almost everything hit the LLM. As the fingerprint cache filled with the most common templates and the semantic cache started to recognise recurring problem descriptions, the hit rate climbed and the LLM handled a shrinking fraction of rows. The 87-hour runtime was measured on a resumed run where the caches were still cold at the start. In a steady-state run against a stable taxonomy, reuse layers would likely handle a much larger share.