Work · Product · 03

Werkbank

Which workshop should take this furniture order?

In 30 seconds

Problem
A furniture shop with three workshops routes every incoming order by hand, and misrouted jobs cost days.
Result
On 60 hand-written requests it routed 40% on its own at 95.8% accuracy and sent 23.3% to a review queue instead of guessing. Real-shop accuracy: pending.
My role
Sole developer: classifier, calibrated gate, review queue, app, auth and eval.
  • measured 95.8% accuracy on the 40% of hand-written requests it routed on its own
  • measured 23.3% of hand-written requests sent to the review queue instead of guessed
  • synthetic 96.8% top-1 accuracy after a new shop corrects 100 orders (from 75.5%)
  • pending — accuracy on real shop orders: no shop has used it yet
Type
Product
When
October 2026
Role
Sole developer: classifier, gate, app, auth, eval
Origin
A real client brief from a furniture shop with three workshops
Status
Live demo · EN and DE · 57 tests

01In plain language

A furniture shop with its own workshops gets requests like "sofa cushion torn, needs new foam" or "Tisch wackelt, Bein lose". Someone has to decide which bench each job goes to: carpentry, upholstery, polish and paint, or metalwork.

Werkbank reads the request the way the owner would write it on a job slip and sends it to the right workshop. It also labels the job type, urgency and effort. When it is not sure, it does not guess: the order waits in a review queue. Every correction the owner makes teaches that shop's model.

It started as a real brief from a furniture shop with three in-house workshops. I turned it into a product any shop can sign up for, in English and German.

02The question

How many orders can go straight to the right workshop without a person, and can the system tell which ones those are?

03Why the obvious approach fails

Keyword rules are easy to explain to a shop owner, but customers do not use the shop's words. "Recliner mechanism stuck" is a metalwork job with no metal word in it.

A classifier that always answers sends every unclear order somewhere. A wrong routing is worse than a slow one: the job sits at the wrong bench until someone notices.

An LLM on every order costs money per order, needs an API key in every shop, and still answers confidently when the request is vague.

04The gate

Two cheap layers run in under a millisecond: a keyword lexicon in English and German, and a naive Bayes model trained on seed data plus the shop's own corrections. Their blended score goes through a gate.

OutcomeCondition (balanced setting)
Auto-routeconfidence ≥ 0.85 and margin ≥ 0.5
Route, flagged "check"confidence ≥ 0.55
Review queueLower confidence, almost no known words, the two layers strongly disagree, or the shop has no workshop for the predicted job

Naive Bayes is known to be over-confident, so its scores are tempered by a temperature fitted on held-out phrasings (2.5). The interface never shows more than "99%+". Shops can move the gate between strict, balanced and relaxed.

An optional language model only sees orders that are already in review. It must quote the request word for word, or its answer is thrown away. An LLM decision is never marked as automatic.

05Thought process

Left to right, the path one order takes, from the owner's typing to the shop's next model.

  1. 01

    Read

    • Free text, English or German
    • Live preview while typing
    • Due date taken into account

    The owner writes the way they talk.

  2. 02

    Score

    • Keyword lexicon, EN + DE
    • Naive Bayes: seed + corrections
    • Temperature 2.5 calibration

    Two cheap opinions, in under a millisecond.

  3. 03

    Gate

    • Auto ≥ 0.85, margin ≥ 0.5
    • Check ≥ 0.55
    • Disagreement → review

    Sure, fairly sure, or ask.

  4. 04

    Review

    • Best guess shown to the owner
    • Optional LLM, quote required
    • Never marked automatic

    Unsure orders wait for a person.

  5. 05

    Learn

    • Every reassignment adds counts
    • No training job needed
    • Per-shop model

    Corrections make tomorrow easier.

The model predicts four job types, not workshop names. Each shop maps its own workshops onto them, so one model serves every shop.

06Decisions, and what they cost

ChosenNaive Bayes

One correction just adds counts. The shop's model updates instantly, with no training job.

Instead ofA model that needs retraining

Every correction would wait for a training run before the shop saw any benefit.

ChosenFix the gate before testing

Thresholds were set before looking at test results.

Instead ofSearch for the best thresholds

The search on validation data collapsed to the minimum threshold, so it was dropped and the thresholds were fixed by hand.

ChosenRules for urgency and effort

They depend on a few explicit words and the due date. "No rush" is checked before "rush".

Instead ofA third model

A shop owner can read a rule and see why a job was marked urgent.

ChosenTest on words it never trained on

The synthetic test set uses sentence templates and synonyms that never appear in training, plus 60 messy requests written by hand.

Instead ofA random split of the generated data

That would test whether it remembers its own templates.

07What came out

The Werkbank review queue in the demo shop. Three orders the system was unsure about, each with the reason in plain words, its best guesses with percentages, and one button per workshop.
The review queue in the demo shop. Each card says why the system held back: unknown words, rules and model disagreeing, or two workshops too close to call. The headboard is a fair catch: its top guess is wrong, which is why it waits for a person.
Test setAuto-routedAuto accuracyTo reviewAccuracy of all routed
Hand-written, messy (n = 60)40.0%95.8%23.3%93.5%
Synthetic, unseen phrasing (n = 800)18.9%95.4%36.1%84.0%

The gate holds: whatever skips review is right about 95% of the time on both sets. What changes is how much it dares to skip. On phrasing it has never seen, it sends more to review instead of guessing.

Strict or relaxed: the trade-off a shop chooses

Auto thresholdAuto-routedAuto accuracy
0.5071.8%82.8%
0.7043.6%86.5%
0.85 (default)18.9%95.4%
0.954.4%100.0%

The feedback loop

A simulated shop that phrases things in ways the seed model never saw corrects its first orders. Then the model is measured on 400 later orders:

CorrectionsTop-1 accuracyAuto-routedReview queue
075.5%20.8%34.8%
5092.0%63.7%12.5%
10096.8%82.5%6.3%
20098.5%87.5%4.0%

All from reports/eval.md, generated 26 Oct 2026. Seed model trained on 2,400 synthetic requests.

08What it refuses to claim

Real-world accuracy. There is no public dataset of furniture-shop tickets, so training data is synthetic, and synthetic test data is easier than real orders. The true number is unknown until a shop uses it.

The 400-correction result. It reaches 100%, but by then the test orders are close to what it trained on. The evaluation report says not to quote it, so it is not quoted here.

09What broke

Three hand-written orders went to the wrong bench

RequestRightRouted to
"need a study table built into the corner, 120x60, white laminate top"CarpentryFinishing (auto, 0.93)
"old teak chest, grandma's, wants it restored. finish is black and sticky"FinishingCarpentry (check, 0.74)
"recliner mechanism stuck, footrest won't come out"MetalworkUpholstery (check, 0.66)

Only the first one skipped review entirely. "White laminate top" pulled it towards finishing. The other two were routed but flagged for the owner to check.

German is the weaker language

The keyword lexicon is mostly English. In a small manual check, 3 of 5 German example sentences went to the review queue. They were not routed wrongly, but the shop gets less automation in German.

Job type is mediocre

Repair, custom, refurbish or assembly is right only 70% of the time on the synthetic set (75% hand-written). It is shown as a label, not used for routing.

10Still open

  • A real shop using it, and accuracy measured on real orders.
  • A bigger German lexicon.
  • Login rate limiting, and ending open sessions when a password is reset.
  • Payments: picking a plan records it, but billing is not connected.

11Reproduce it

Every number on this page comes from these commands, run in the repository.

git clone https://github.com/omkar-k0/werkbank && cd werkbank
npm install
npm test          # classifier, tenant isolation, split orders, auth, i18n
npm run model     # regenerate synthetic data, retrain, re-evaluate
npm run dev       # http://localhost:3000, Postgres in-process via PGlite
FileWhat it holds
reports/eval.mdAccuracy, routed share and review share per test set
reports/eval.jsonThe same numbers, machine-readable
scripts/eval.tsThe script that writes both reports

All training data is synthetic. The 60 hand-written requests were not used for training or thresholds.