camera-3d-captions
> Captions living in 3D space around a talking head: a camera flies between the speaker and the words (whip-in, parallax truck, push), caption groups sit at different depths so camera moves pull them apart, hero words hide BEHIND the speaker through an alpha matte, a ring of words wraps round the speaker and turns in front of them, focus racks between depths, and text steps at 15 fps with ghost motion blur. Works in any type style (editorial serif, bold sans) or hand-drawn (p5.brush write-on). Trigger on: "3D text", "3D camera captions", "text around / behind the speaker", "camera moves through the text", "depth captions on my avatar video". Covers prep (portion, padded plate, matte, word clock, font metrics), the shot grammar, depth rules, QC, and limits.
1How It Works & Mechanism
The implementation path is 'rebuilding AE techniques into a frame-by-frame deterministic renderer': a single analytical clock renders the plate, matte, and each glyph simultaneously, with the entire composition avoiding CSS animations (so that the renderer supports arbitrary seeking).
One clock paints plate, matte and every glyph. Nothing is a CSS animation.
The camera is not true 3D, but an analytical projection: each shot defines cz (dolly along the axis) and px/py (screen shifts). All layers exist in 'reference-screen coordinates under a reference camera', projected via screen = C + (ref − C)·D0/(D0 − cz) + (px,py); text steps at 15 fps and samples the camera synchronously, overlaid with 7-pass ghost motion blur.
as a deterministic per-frame painter for HyperFrames. One analytic clock: CAM3D.render(t) writes every style.
The 'AE feel' does not rely on hardware capabilities, but on three hardcoded timing disciplines: text is POSTERIZED to 15 fps (sampled together with the camera it observes), 7-pass ghost motion blur is applied, and the plate and matte update every frame.
Text is posterized (camera AND own animation sampled at 15 fps) with 7-ghost motion blur; the plate/matte move every frame with a directional Gaussian from camera speed.
The timeline itself is NOT locked to a rigid template: the storyboard is dynamically split into 4–6 beats by the model based on spoken lines, splitting into a group per clause; what is locked is the render contract—15 fps grid, captions leading voice by 0.2 s, ENTRY displacement, and depth shadow/DOF formulas.
Pick 4–6 beats. Split each spoken clause into its own group.
The asset pipeline uses a one-time prep: cuts out the portion (audio source), plate-tall with reflected edge padding (prevents edge exposure during whip/push-pull camera moves), and the synchronized alpha matte (person.webm), then verifies all three item-by-item across dimensions/framerate/frame count, immediately exiting with code 1 upon mismatch—this is where 'timeline lock' actually takes effect.
[ "$tn" -eq "$pn" ] && [ "$mn" -eq "$pn" ] || fail "frame counts differ: portion $pn, plate $tn, matte $mn"
Font metrics are externalized into generatable data tables: font-metrics.py uses fontTools to extract each glyph's advance (em) and line-height 1 baseline, instantiating variable fonts at specified axes first; the engine then arranges each glyph individually (disabling CSS kerning), requiring variable axes to be pinned in CSS as well to prevent width drift from browser opsz scaling.
Writes `window.CAM3D_METRICS = {adv: {key: {char: em}}, base: {key: em}}`.
Camera motion curves are not fitted, but frame-by-frame measured lookup tables extracted from reference tutorials: tables.js stores camA (19-frame whip-in + crawl), camB (pull out/stabilize/push), wipe leading edge x, and remaining entry ratio, indexed at 30 fps frames and scaled by H/1080.
/* tables.js — measured motion tables (skill camera-3d-captions). Frame-indexed at 30 fps; px values are for a 1080-tall frame (scale by H/1080);
Hand-drawn style is reused across skills: leverages the installed p5-paint-animation engine to render each word into write-on sprite sheets (build-sprites.py invokes render-anim.mjs from that engine via subprocess, reading the frame directory and deleting it afterward); hence this style depends on the setup of another skill.
**Hand-drawn style (optional):** needs the `p5-paint-animation` skill installed and set up (its setup downloads pinned puppeteer, Chrome for Testing, p5 and p5.brush). `build-sprites.py` runs it headlessly; frames are written to that skill's `out/` and removed afterwards.
2Core Capabilities
3External Dependencies
| Kind | Dependency / Endpoint |
|---|---|
| cli | HyperFrames CLI（钉死 0.8.62，经 npx 从 npm 取） |
| cli | ffmpeg / ffprobe（切分、反射补边、探帧率与帧数、最后编码） |
| package | Python3 + fonttools/brotli/numpy/pillow（字体度量与手绘精灵图） |
| network | 抠像模型首次运行下载（人物分割权重 ~170 MB，本地推理） |
| network | GSAP 3.14.2（合成在预览/渲染时由页面加载；参考片 HTML 也引用同一 CDN） |
| package | 跨 skill 依赖：p5-paint-animation 的渲染引擎（仅手绘风格需要；用 render-anim.mjs 生成精灵图） |
| cli | 素材来源：用户自备 take 视频、自备字体（需自行确认嵌入许可）、以及用户自己本地跑的词级转录器 |
4Recommended Governance & Evidence
- scripts/: Executes local script or command: bash scripts/prep-take.sh <take> <start> <dur> <assets> (ffmpeg/ffprobe subprocesses;
- 网络端点: Declares external network access: registry.npmjs.org: fetches hyperframes@0.8.62 via npx per execution (pinned version; does not follow latest)
- 本地工作目录: File read/write and artifact output: inside project assets/: portion.mp4, plate-tall.mp4, person.webm, metrics.js, hand/*.png and
5Second-Pass Verification
- [ok] skill 自身是否读凭据（黄/橙分界的关键） — 全目录 process.env / os.environ 只命中 build-sprites.py 的 P5_ENGINE（p5 引擎目录路径）；无 API key、token、cookie、keychain、.env 读取。因此按范式的橙档第一条不成立，落黄。
- [ok] 「外部资源」逐条回查调用点 — hyperframes@0.8.62 在 SKILL.md 与 prep-take.sh（HF="hyperframes@0.8.62"）两处一致；ffmpeg/ffprobe 在 prep-take.sh 的 ffmpeg/probe() 内可验；fontTools 在 font-metrics.py import 段；GSAP CDN 在 worked-film.html 与 SKILL.md 的 Render time 条；p5 引擎调用在 build-sprites.py 的 subprocess.run(cwd=engine)。
- [ok] prep 阶段的三者对齐自检是否真会失败退出（不是装饰性文字） — prep-take.sh 定义 fail(){ echo ... >&2; exit 1; }，并对 plate 尺寸、matte 尺寸、三者的帧率与帧数各写了一条比较分支，任一不符即 exit 1；成功路径打印 'prep-take: plate and matte aligned with the portion ($pn frames @ $pr)'。
- [ok] 手绘风格是否真的复用 p5-paint-animation 引擎，且临时帧会被清理 — build-sprites.py 用 subprocess.run(['node','render-anim.mjs', ...], cwd=engine) 渲染临时 sketch（写在 tempfile 目录，落在自己的 temp 而非引擎内），读引擎 out/_frames_<name>.anim/ 的 PNG 后调用 shutil.rmtree(fdir, ignore_errors=True)；与 SKILL.md『frames are written to that skill's out/ and removed afterwards』一致。
- [discrepancy] worked-film.html 能否按 SKILL 的 Build 步骤原样打开（参考片的可运行性声明） — SKILL.md 只说 'its media is not included'，但参考片的 <head>/<body> 还引用了 assets/o2o-data.js（注释写 'window.O2O here = assets/kit/tables.js'）与 assets/Inter-Variable.woff2、assets/InterTight-900.woff2 等字体文件；o2o-data.js 与这两个 woff2 均不在包内（全量清单已确认）。按 SKILL 的『Copy assets/kit/{cam3d.js,tables.js,finish.js} and metrics.js into the project's assets/』操作仍缺 o2o-data.js，参考片不能原样运行——读者须自行把 tables.js 改名/桥接成 window.O2O 并自带字体。该出入对 pin commit 有效。
- [ok] 15 fps 网格与 0.2 s 提前量是否真在实现里（而不是只写在文档） — cam3d.js 顶部 var W = 1440, H = 1080, CX = 720, CY = 540, FPS = 30, POST = 15, K = 7;，并有 post(t) = Math.floor(t * POST + 1e-4) / POST；SKILL.md 给出 fr(t) = 2·round((t − 0.2)·15)（×2 即 30 fps 渲染下的 15 fps 映射），两处自洽。
- [unlocatable] 『带 grade 与多重影渲染约 2× 成本』的量化声明 — SKILL.md 的 Limits 段写 'Rendering with the grade and many ghost layers costs about 2× a plain render.'，包内没有基准脚本或计时数据可复现该倍数；本次为只读侦查且不得运行渲染，故仅作为文档声明记录，不作为已核事实使用。
6Verdict & Scenarios
70cf48e51fa89f4e…