Home / All Skills / Content & Media Creation / audio-sync-assembly
English (EN)简体中文 (ZH)日本語 (JA)
Content & Media Creation · saranambiar/hyperframes-video-agent-skills

audio-sync-assembly

Build and revise FFmpeg-based video assemblies from scene renders and narration, including audio probing, scene retiming, pauses, trim/concat, setpts, atempo, phrase-to-visual cue alignment, and final timeline manifests. Use when narration length, visual beats, product moments, or scene boundaries must sync precisely.

L4 High-Privilege · Graded auth & full auditV1 Commit-PinnedExecute CommandWrite FileDual-pass Audit
Author saranambiarGitHub saranambiar/hyperframes-video-agent-skills ↗Stars 48License Apache-2.0（仓库根 LICENSE 为 Apache License Version 2.0；GitHub API license.spdx_id = Apache-2.0）Commit f6b043d9fd
Control levels describe skill capabilities and suggested control measures; they do not represent quality or safety certifications, nor do they dictate usability. Usage decisions rest entirely with users and administrators. Different agent hosts apply varying constraints; do not assume hosts provide automatic protection. Third-party skill copyrights belong to their original authors. Use at your own risk.

1How It Works & Mechanism

The single source of truth for timing is strictly locked to the "latest approved narration audio + scene renders", explicitly prohibiting reverse-engineering timings from legacy final cuts—serving as the shared baseline for all retiming, subtitle, and music decisions across the entire workflow.

skills/audio-sync-assembly/SKILL.md
- Use the latest approved narration audio as timing source.
What this does: Establishes "what takes precedence" as the primary Source of Truth. It corresponds directly to the failure mode item "Do not infer timings from old final MP4s if source renders and audio clips exist." within the family checklist, preventing error accumulation caused by deriving timings from previous exports across iterations.

Emphasizes "minimal intervention": provides tiered strategies based on scene characteristics (full-segment setpts / segmented retiming / inserting silences with hold frames / minor atempo adjustments / retiming only the section preceding a product moment), rather than defaulting to global speed changes.

skills/audio-sync-assembly/SKILL.md
Choose the smallest timing intervention:
What this does: Defines the core production heuristic for this skill—first asking "what is the minimal timing intervention?" and designating "retiming the entire video" as the most common error (listed as the first item in Common Mistakes).

Probe before making any decisions: measure duration, frame rate, resolution, audio track presence, sample rate, and channels for every clip using ffprobe or the repository's media probe script.

skills/audio-sync-assembly/SKILL.md
- Duration. - Frame rate. - Resolution. - Audio stream presence.
What this does: Solidifies "measure before calculating" as the first operational step to avoid estimating setpts ratios, explicitly providing pathways for both ffprobe and "this repo's media probe script" in the same section.

Assembly sequence locks subtitles to the "music-free narration master": assemble the narration-only master first, then generate and burn subtitles, build the music bed, mix audio, and finally produce final exports and preview cuts.

skills/audio-sync-assembly/SKILL.md
3. Generate captions from narration-only master.
What this does: Eliminates the entire class of timing drift caused by "aligning subtitles against music-backed final cuts" through strict sequencing, drawing a clear line of responsibility with captions-and-music-bed (which requires a stable narration timeline as its prerequisite).

Segment split points must align with semantic events, explicitly prohibiting uniform slicing: scene transitions, product moments, problem reveals, chart milestones, title reveals, ending holds, and narration pauses.

skills/audio-sync-assembly/SKILL.md
Avoid splitting on arbitrary equal chunks.
What this does: Identifies "uniform split by seconds"—a seemingly fair approach that inevitably causes misalignments—as an anti-pattern; split points must strictly bind to narrative events.

Preview cuts are not full-video re-renders: generate clips only for sensitive areas affected by current changes (updated narration sections, product demo transitions, problem timings, subtitle styling blocks, final 10 seconds, and any modified scene boundaries).

skills/audio-sync-assembly/SKILL.md
Generate preview clips for:
What this does: Keeps verification costs proportional to the scope of edits, aligning with the "provide proof only at affected locations" principle in AGENTS.md.

2Core Capabilities

01FFmpeg command templates for full-segment/segmented retiming, hold frame pauses, audio tempo micro-adjustments, and explicit assembly
02Audio replacement decision rules: probe old vs. new durations → apply minor tempo adjustments to preserve downstream sync → reconstruct segment maps and regenerate subtitles when phrasing timings change substantially
03Phrase-level visual alignment: map spoken cue onsets to global timestamps (problem cards, product states, chart nodes, scene title reveals, CTA/logo landings)
04Timeline manifest field specifications: scene ID, source render path and duration, target duration, audio path and trim in/out, inserted silence, hold frames, segment retiming ratios, global scene start time, proof timestamps
05Pause handling heuristics: insert silence + hold or subtle motion, refrain from arbitrarily raising music levels, factor pause durations into downstream scene start offsets
06Regression defense checklist: retiming the entire video for a single-scene edit, audio replacements pushing downstream out of sync, forgetting to update subtitle timings, stream copies starting on non-keyframes, music cutting off before the video ends

3External Dependencies

KindDependency / Endpoint
cliffmpeg（setpts / trim / tpad / atempo / apad / atrim / concat / fps / scale / setsar），走 filter_complex 显式拼装
cliffprobe（探测时长/帧率/分辨率/音轨/采样率与声道）
package仓库根确定性脚本 scripts/probe_media.py 与 scripts/timeline_manifest_to_ffmpeg.py（本 skill 以「this repo's media probe script」泛指，非自身目录自带）
package模板 templates/timeline-manifest.example.json（记录目标时序的 JSON 结构）
package输入素材：最新批准的旁白音频与场景渲染（用户/项目本地提供，非外部服务）

4Recommended Governance & Evidence

Suggested Controls for Safe Operation
For Agent Runtime
Graded/per-use authorization, sandbox recommended
For Administrators
Approval workflow, rollback support, full audit
Evidence & Triggers (File Paths & Excerpts)
  • scripts/: Execute local script or command: (Executed by host agent) ffmpeg commands: setpts / trim / tpad / atempo / apad / atrim / concat
  • 本地工作目录: File read/write & artifact export: Scenes and final MP4s in project renders/, assembly scripts or command plans in assembly/, subtitle files in captions/
Audit Notes: This skill directory contains only SKILL.md and a references/ffmpeg-retiming.md reference (pure instructions, no executable files). Its operational surface consists strictly of local CLI tools (ffmpeg/ffprobe) and local file read/write (renders, manifests, subtitles, preview clips, proof frames), matching the standard profile for "calls local tools/CLI only, or performs only local file read/write". **No assets are sent to external services**: a full scan of this skill directory reveals no http(s):// endpoints, no fetch/requests, no token/env variable reading, and no API calls to upload or generation services; all audio and video processing occurs strictly within the local ffmpeg pipeline. Python scripts in the repository's root scripts/ directory only perform local ffprobe/ffmpeg calls and file I/O (verified via second pass). All nine sibling skills in this family share the same evaluation criteria: the directory contains only SKILL.md (+ optional references/*.md), actions are limited to local I/O and local CLIs, with zero endpoints, zero credentials, and zero external transmission of assets.

5Second-Pass Verification

  • [ok] 「外部依赖只有本地 CLI」是否成立（决定蓝档还是黄/橙档） — 本目录正则扫描无 http(s)://、无 fetch/requests、无 token/env/cookie；SKILL.md 只出现 ffprobe 与 this repo's media probe script，references 只出现 ffmpeg filter 片段。不存在素材外发或端点查询，故落蓝档。
  • [ok] ffmpeg 命令模板是否真能跑（不是示意伪代码） — references/ffmpeg-retiming.md 给的是标准 filtergraph 语法（setpts=factor*(PTS-STARTPTS)、tpad=stop_mode=clone:stop_duration、atempo、concat=n=...），可原样接进 -filter_complex；缩放/fps/format 同段给出，属可执行模板。
  • [discrepancy] SKILL.md 的时间线清单字段与 templates/timeline-manifest.example.json 是否一一对应 — SKILL.md 列了扁平 11 项（Scene ID / Source render path / Source duration / Target duration / Audio path / Audio trim start-end / Inserted silence / Hold frames / Segment retime ratios / Global scene start / Proof timestamps），模板 JSON 却是嵌套结构：project{name,width,height,fps,output} + segments[{id,video,source_duration,target_duration,audio,strategy,maps}] + captions{} + music{} + proofs[]。差异在于「插入静音/保持帧/分段重定比」被折叠进 strategy+maps（maps 里用 hold_source_time 表达保持帧），「全局场景起点」由脚本按目标时长累加算出而非显式字段，「证明时间戳」在 proofs[] 而非 segment 内。按 SKILL.md 的字段名手写 JSON 不会与模板同构，照抄者需以模板为准。
  • [discrepancy] 「this repo's media probe script once available」是否说明探测脚本当前不存在 — 措辞（once available）把探测脚本当成「将来可能有」，但 pin 内仓库根已存在 scripts/probe_media.py（ffprobe -print_format json 的封装，输出时长/体积/码率/视频编码与分辨率/帧率/像素格式/音频编码/采样率/声道），且 render-qa-and-surgical-changes 明确点名它。属文档措辞落后于仓库状态，不影响行为，但会让读者低估可用工具。
  • [ok] 与 captions-and-music-bed 的职责是否重叠冲突 — 本 skill 只把「生成字幕/建音乐床」写进组装顺序的第 3-6 步（顺序约束），字幕规则与 ducking 判据全部留给 captions-and-music-bed，无重复定义。

6Verdict & Scenarios

  • Codifies the timing source of truth and "minimal intervention" into checkable rules, directly eliminating the two most common rework triggers (reverse-engineering from old cuts, global retiming for single-scene edits)
  • Uses assembly sequence itself as a heuristic: subtitles must be generated from the narration-only master, and the music bed must follow subtitles—eliminating timing drift via workflow order
  • Command templates are executable and self-explanatory (ratio definitions, tpad syntax for hold frames, applicability conditions for atempo)
  • Standardized handoff formats (scene start maps / audio / tempo shifts / inserted pauses / preview paths / final durations), eliminating archaeology for downstream contributors
  • Pure local pipeline with zero network calls and zero credentials, making it the easiest skill in this family to audit
    • Best for: Best for: scenarios where one or more rendered scenes and approved narration tracks must be assembled into a precisely timed cut; cases where narration length does not match visual pacing, pauses or hold frames must be inserted, visual cues must align with specific spoken lines, or downstream timings must be recalculated after audio replacement. Also suitable as the assembly layer invoked during steps 3-6 of build-in-layers in end-to-end-video-playbook.
    Not for: Not suitable for: (1) Designing or generating scene visuals (use hyperframes-scene-builder / motion-design-systems); (2) Subtitle wording/styling or music track selection and ducking criteria (use captions-and-music-bed); (3) Small QA/surgical changes on approved final cuts (use render-qa-and-surgical-changes to avoid re-assembling the full cut); (4) Environments lacking ffmpeg/ffprobe—all heuristics in this skill depend entirely on a local ffmpeg pipeline with no pure-code fallback.
    Install Agent-Ready
    ① Verified Mirror 2026-09-28
    Option A · Direct DownloadDownload audio-sync-assembly.tar.gz
    sha256: 099bc62047c52f2e…
    Option B · Copy Guide for Agent
    install.json
    Agents automatically download and verify mirrors from this JSON guide.
    ② Upstream GitHub
    GitHub Repo ↗
    Pinned to commit f6b043d9fd.