9NcjRFu6C4I
LangGraph agentic pipeline (av_langgraph_pipeline), Slurm job 386863 on an H100, 2026-07-22,
total 656 s — final status passed. Each card below shows one graph node
with its input and output. Badges: real model ran in this job,
cached real output reused from the previous job,
stand-in offline substitute, mock not yet implemented.
8 s, 24 fps train-simulator clip with audio — plus the preprocessed mono wav that feeds SAM-Audio.
In: input video+audio · Out: caption text
“A blue and yellow CSX locomotive travels along a track in a flat, grassy landscape. As it approaches a junction, the train derails and begins to flip over, lifting its front end high into the air while the rear remains on the ground. The train continues to tilt and rotate, eventually becoming almost vertical. The sound of a train horn is heard, followed by the clatter of wheels on the tracks.”
Generated by Qwen3-Omni in the first job of this session and reused here (identical node, cached to avoid ~10 min of recompute).
In: caption · Out: target object = train engine
Normally GPT (gpt-4o-mini) via the OpenAI API; the provided key had no quota (429 insufficient_quota),
so an offline rule returned “train engine” — the same prompt used by all earlier experiments on this clip, and the phrase SAM3
responds to far better than the bare “train” (first-frame ratio 0.095 vs 0.002).
In: input video + prompt “train engine” · Out: whole-video mask (+ overlay / masked previews)
first-frame / mask-area ratio: 0.0951
SAM3 tracks this synthetic (game-rendered) train in only ~9 of 202 frames — the mask goes empty after the opening frames. That weakness drives everything downstream on this clip and is why the demo gates below were lowered.
In: input video + SAM3 mask · Out: inpainted video (object visually removed)
With only ~9 masked frames, EffectErase's temporal context (train visible in every unmasked frame) largely repaints the train — an honest failure case for this clip, quantified by the next step.
In: inpainted video + original mask · Out: removal score (SAM3 re-segmentation ratio)
visual_removal_score: 0.4352 — passed the demo gate (0.40); the production gate is 0.80.
In: source audio + SAM3 mask video + prompt “train engine” ·
Out: 10 candidates (visual & text prompts × 5 seeds), winner picked by max ib_ta
(ImageBind text↔target), tiebreak min ib_ta_res (text↔residual, lower = cleaner removal).
| rank | prompt | seed | ib_ta ↑ | ib_ta_res ↓ | target (train engine) | residual (removed) |
|---|---|---|---|---|---|---|
| 1 ★ | visual | 1453635084 | 0.2335 | 0.0835 | ||
| 2 | visual | 240868205 | 0.1889 | 0.0220 | ||
| 3 | text | 1335522078 | 0.1781 | 0.1930 | ||
| 4 | visual | 1335522078 | 0.1727 | 0.1741 | ||
| 5 | text | 240868205 | 0.1397 | 0.0393 | ||
| 6 | text | 1453635084 | 0.0915 | 0.1489 | ||
| 7 | visual | 1132891577 | 0.0731 | 0.2736 | ||
| 8 | text | 1132891577 | 0.0588 | 0.2716 | ||
| 9 | text | 1778986134 | 0.0453 | 0.2587 | ||
| 10 | visual | 1778986134 | 0.0242 | 0.2637 |
Generation: SAM-Audio large, one model load, seeded per candidate (samaudio311 env).
Scoring: same calc_imagebind_score as the ranking_test eval (JavisDiT repo, javisdit env).
In: winner residual · Out: audio_removal_score = 0.9 (placeholder checker, passes the 0.80 gate)
Out: inpainted video paired with the winning object-removed audio.
| gate | production | this demo run | observed value |
|---|---|---|---|
| SAM3 first-frame ratio | 0.10 (dev; 0.80 target) | 0.05 | 0.0951 |
| whole-video mask area | 0.15 | 0.05 | 0.0951 |
| visual removal score | 0.80 | 0.40 | 0.4352 |
Lowered (via env vars, defaults unchanged in code) solely so this hard clip traverses every stage for visualization; at production thresholds the pipeline correctly discards it at the mask gate.