Two Scorers and a Watchdog: Building Claims Triage and Underwriting Signals

Two claims land at first notice of loss a minute apart. One is a fender-tap with photos, a cooperative insured, and clean coverage — a clean fast-track payout that an adjuster will only slow down. The other looks just as ordinary, but it is quietly leaking subrogation dollars against a third party nobody has named yet, and three of its features sit exactly where your fraud population sits. Both will land in the same FNOL queue, in arrival order, and wait for whichever examiner picks them up next. The fast one will get over-handled; the expensive one will get under-handled. The build has to tell them apart before an adjuster ever opens either file.

That is the problem this playbook is for, and it is a routing problem before it is a modeling problem. If you run claims and underwriting systems and you are the engineer or systems lead asked to “add AI” without touching the book of record or inviting a bias complaint, this is your reference design: the two scoring pipelines, the gates that decide auto versus human, the reason-code store that makes every route explainable, and the fairness monitor that watches the whole thing for disparate impact. None of it adjudicates. All of it triages and signals.

Two pipelines over the same book of record

The first design decision is the one that keeps you out of trouble: neither pipeline owns a decision the law cares about. The policy-admin system, the claims core, and the rating engine stay authoritative. Each pipeline reads from them, computes a score and a set of reason codes, and hands a routing recommendation to a gate. The gate decides auto or human. The human, or a deterministic policy, still owns the disposition that touches money. Nothing in either scorer ever silently mutates the authoritative file.

Core systemspolicy admin · claims core · rating · billingauthoritative — read by the scorers, written only by the disposition

Two scorersclaims-triage scorer · underwriting-signal serviceleakage/subro/fraud features → route · risk features → adequacy flag

Gate + watchdogrouting gate · reason-code store · fairness monitordecides auto vs. human; logs why; watches for disparate impact

Two scoring pipelines, one shared governance plane. The claims scorer routes; the underwriting service flags; the same gate, reason-code store, and fairness monitor sit behind both.

Two scorers, one governance plane. That sharing is the whole economy of the build: you write the gate logic, the reason-code schema, and the fairness harness once, and both pipelines run on them. A competitor can rent the same models you do. What they cannot lift is the corpus you accumulate behind these scorers — the leakage features tuned to your own closed claims, the subrogation triggers that match your recovery economics, the risk features your loss development actually validated. That corpus is the asset; the model is the commodity.

The pipelines are coordinated, not merged, and the difference matters. They share governance but separate their gates, because being wrong means something different on each side. Misroute a claim and you waste an adjuster-hour or leak a settlement; the cost is bounded and visible the same week. Misprice a book and the damage compounds silently across an entire policy cohort, and you do not see it until loss development drifts a year later. One gate cannot serve both clocks.

The claims-triage scorer: features, reason codes, routes

At FNOL the claims scorer pulls what the core already knows — line of business, coverage, claimant, loss description, jurisdiction — and computes three feature families: leakage (does the proposed handling path drift above where comparable closed claims landed), subrogation (is there a recoverable third party that will evaporate if the file closes), and fraud/complexity (does this file sit near the patterns that warrant a specialist or an investigation). Each family yields a sub-score and, more importantly, the reason codes that explain it.

The output is a route, not a verdict: fast-track for clean, in-distribution, low-severity files; adjuster for normal handling; specialist for high-severity or coverage-ambiguous files; SIU when fraud features cross threshold. Every route carries the reason codes that produced it, so the adjuster who receives the file — or the auditor who reviews it later — sees exactly why it arrived where it did.

RULE ClaimTriageRoute
WHEN lob == "Auto" AND liability.third_party_at_fault_pct >= 50
THEN reason += "subro:third_party_negligence"
  AND route_hint = "adjuster"     // recovery right must not auto-close

WHEN fraud.score >= siu_threshold[lob]
THEN reason += "fraud:above_siu_threshold"
  AND route_hint = "SIU"

WHEN leakage.proposed_path > comparable_cohort.p90 AND comparable_cohort.n >= 30
THEN reason += "leakage:above_p90_cohort"
  AND route_hint = "specialist"

WHEN severity_band == 1 AND coverage.clear == true AND subro.flag == false
  AND fraud.score < auto_threshold[lob]
THEN reason += "clean:in_distribution_low_severity"
  AND route_hint = "fast_track"

DECISION: route = gate(route_hint, reason, autonomy_level[lob, severity_band])

The decision rule is deliberate about asymmetry. A subrogation feature, even a weak one, blocks the fast-track route, because a missed recovery is silent leakage no fraud model will ever flag — the file just closes a little light. Fast-track is reserved for files that are clean on all three families and inside the distribution the scorer was trained on. Out-of-distribution is itself a reason code: when a claim does not resemble anything in the comparables corpus, the route hint is “human,” never “auto.” Auto-routing an out-of-distribution claim onto fast-track is the failure mode that bleeds a book quietly, and the gate exists to refuse it.

The underwriting-signal service: pricing adequacy at quote

The second pipeline runs at a different moment with a different job. When a quote is built, the underwriting-signal service assembles internal risk features (prior loss history, exposure characteristics, the rating engine’s own factors) and external ones (permissible third-party signals for the line and state), and emits a single thing the underwriter actually needs: a pricing-adequacy flag — is this quote priced where comparable bound risks performed, or is it drifting thin against expected loss? It is a signal, not a price. The rating engine still owns the number; the service tells the underwriter when the number deserves a second look before it binds.

This is where the fairness obligation gets sharp. A pricing signal influences what a customer pays, so before it ships you must be able to prove the feature set did not drive a disparate price. That means two things the build has to enforce: the fairness check runs before the flag reaches the underwriter, and the feature set is versioned. When a rate filing or a market-conduct exam asks what drove pricing guidance in March, the only defensible answer is to replay the exact features, the exact thresholds, and the exact fairness result that were live then. A pricing signal you cannot reconstruct is a rate-filing problem waiting for a regulator to find it.

FNOL-to-route timemanual queue pickupscored at intakeCost / CX
Loss adjustment expense8–18% of incurred5–12%Cost
Subro recovery capturecaught after closeflagged at FNOLCost / leakage
Quote turnaround on adequate risks1–7 days10 min–2 daysRevenue
Disparate-impact detection lagfound in exammonitored continuouslyRisk / trust

Sample planning ranges — set against your own book, mix, and state constraints. Carry at least one revenue, one cost, and one risk measure into every monthly review.

The gates: auto versus human, and the bands that govern them

A score does not act. A gate acts on it, and the gate is the most important thing you build, because it is where autonomy is granted in small, defensible amounts. Each route or flag carries a tolerance band: how far inside the trained distribution it sits, and how reversible the action is. A sub-threshold, in-distribution, low-severity property claim can be auto-routed to fast-track because a wrong route there costs an adjuster two minutes to correct. An injury claim, a coverage ambiguity, a flagged subro right, or anything near the edge of the distribution stays at the gate under human review — not because the model is weak, but because the cost of being wrong is unbounded.

01Scorecompute features + sub-scores
02Explainemit reason codes per family
03Recommendroute hint / adequacy flag
04Gatecheck tolerance band · auto or human
05Auto-routeclean, in-distribution, reversible only
06Hand offhuman owns every consequential disposition

Autonomy is granted per route-class, inside a tolerance band, not switched on globally. The dashed gate is where today's threshold sits; it moves only as fairness and drift monitoring earn it.

The bands are explicit and tunable, and they are not the same for the two pipelines. The claims gate can hold a fairly generous auto-route band on routine property files, because misroutes surface and self-correct within the handling cycle. The underwriting gate holds a far tighter one: a pricing-adequacy flag is always a recommendation to a human, never an auto-action, because there is no cheap reversal for a mispriced cohort. Encoding that difference in the gate — not in tribal underwriting lore — is what makes the autonomy auditable.

The fairness monitor as a first-class component

Most teams bolt fairness on as a quarterly review. The build here treats it as a service that sits behind both pipelines and runs on the same events they emit. Every routing decision and every pricing flag streams into the monitor with its reason codes, its feature version, and the protected-class proxies the monitor is permitted to evaluate. The monitor computes disparate-impact statistics by cohort, continuously, and trips an alarm when a route distribution or a flag rate diverges across groups beyond the band you set with compliance.

This matters on both pipelines, for different reasons. On claims, a fast-track route that systematically skips one population is a fairness exposure dressed up as efficiency. On underwriting, a pricing-adequacy flag that fires harder on one group is the exact disparate-impact pattern a market-conduct exam exists to catch. Because the monitor reads the same versioned events the gates produce, it can answer the regulator’s question precisely: here is the route or flag, here is the feature version that drove it, here is the disparate-impact reading that was live at that moment. Shipping either scorer without this monitor is not a smaller build — it is an undefended one.

{
  "eventType": "RouteDecided",
  "eventVersion": "3.0",
  "claimId": "CLM-551992",
  "policyId": "POL-88371",
  "lob": "Auto",
  "state": "TX",
  "severityBand": 3,
  "subScores": { "leakage": 0.71, "subro": 0.88, "fraud": 0.22 },
  "reasonCodes": ["subro:third_party_negligence", "leakage:above_p90_cohort"],
  "route": "specialist",
  "gate": "human_required",
  "featureSetVersion": "claims-triage.fv-7",
  "fairnessCheck": { "status": "pass", "monitorVersion": "di-monitor.2.1" },
  "modelVersion": "triage-scorer-3.2.0",
  "correlationId": "ins-4091c2",
  "decidedAt": "2026-02-24T19:02:31Z"
}

The reason-code store is just a query plane over those events — the comparables for claims, the audit trail for fairness, the replay source for a rate filing, all the same table — and it lives close to the book of record so the high-volume scans never ship claimant or risk data to an external endpoint:

CREATE TABLE route_decision (
  claim_or_quote_id  TEXT PRIMARY KEY,
  pipeline           TEXT NOT NULL,   -- 'claims_triage' | 'underwriting_signal'
  lob                TEXT NOT NULL,
  cohort_key         TEXT NOT NULL,   -- lob.injury_or_risk.state.severity_band
  route_or_flag      TEXT NOT NULL,   -- fast_track|adjuster|specialist|SIU | adequacy_flag
  reason_codes       TEXT[] NOT NULL,
  feature_set_ver    TEXT NOT NULL,
  fairness_status    TEXT NOT NULL,   -- pass | review | trip
  decided_at         TIMESTAMPTZ NOT NULL,
  correlation_id     TEXT NOT NULL
);
CREATE INDEX idx_cohort_fairness ON route_decision(cohort_key, decided_at DESC);

Feedback: outcomes and loss development re-train the scorers

A scorer that never learns from how its routes turned out decays into yesterday’s intuition. Both pipelines close their loop against ground truth the business produces anyway. The claims scorer learns from closed-claim outcomes: did the fast-track files close clean, did the subro flags convert to recovery, did the SIU referrals validate? Mis-routes — a fast-track file that blew up, a missed subro a later audit caught — are the most valuable signal, and the disposition labels them for free. The underwriting service learns slower, from loss development: the adequacy flags it raised are graded by how the cohorts they touched actually developed, which is why it re-trains on a longer cadence. Wire the feedback so each disposition and each loss-development snapshot writes back a label keyed to the feature version that produced the decision — otherwise you cannot tell a model that drifted from one that was always wrong.

Runbook: shipping a new model without a rate-filing problem

The riskiest day in this build is the day you replace a scorer, because on the underwriting side a model change can become a rate change, and a rate change you cannot reconstruct is a filing problem. The runbook is built to make every model swap boring.

Before the swap — version and shadowPin the new feature set and model with a version stamp. Run it in shadow against live traffic, scoring but not acting, while the current model still drives routes and flags. Diff the route and flag distributions, and run the fairness monitor on the shadow output to confirm no new disparate impact.
At the swap — gate down, then earn backCut over with the autonomy gate tightened: more routes go to humans, fewer auto-act, until the new model has a clean fairness and drift record on real dispositions. For the underwriting service, treat the cutover as a potential rate impact — loop compliance in before the new flags influence a bound price.
After the swap — keep the old version replayableRetain the prior feature set, thresholds, and fairness results so any decision made under the old model can still be replayed exactly. A market-conduct exam can ask about last quarter; your answer is a replay, not a reconstruction from memory.

Ownership has to be named or the gates become a dashboard nobody acts on. The thin RACI that keeps both pipelines moving:

Role	RACI	Responsibility
Claims Director	A	Owns route accuracy, cycle time, and leakage
Adjuster / Examiner	R	Works routed files and disputes bad routes
Subrogation Lead	R	Pursues fired recovery triggers before SOL
Chief Actuary / Underwriting	A	Owns pricing-signal validity and feature versioning
Model Risk / Data Science	R	Re-trains scorers on outcomes and loss development
Fairness / Compliance	C	Sets disparate-impact bands; owns filing reconstruction

What we covered

The takeaways

Build two scorers, not one engine. A claims-triage scorer that routes at FNOL and an underwriting-signal service that flags adequacy at quote — coordinated on shared governance, separated at the gate.
Route on reason codes. Leakage, subro, and fraud features produce fast-track / adjuster / specialist / SIU routes, each carrying the codes that explain it.
Flag, never price. The underwriting service hands the underwriter a pricing-adequacy signal; the rating engine still owns the number.
Ship the fairness monitor on day one. Continuous disparate-impact watch over both pipelines, reading the same versioned events the gates emit.
Close the loop on ground truth. Closed-claim outcomes re-train the claims scorer; loss development re-trains the underwriting service — on the version that made each call.

The goal was never an autopilot for claims or quotes. It is two scorers that get the right file to the right desk and the right warning to the underwriter — explainable on every route, defensible on every price, and watched for bias the whole way — running beside the claims and policy systems you already trust, not on top of their wreckage.

References: ACORD claims event/data patterns · subrogation recovery and statute-of-limitations rules · state market-conduct and rate-filing requirements · NAIC model fairness / disparate-impact guidance · NIST AI RMF · NIST Privacy Framework · NIST CSF 2.0 · GDPR · CISA Secure by Design.