Home / All Skills / Content & Media Creation / volcengine-video-understanding
English (EN)简体中文 (ZH)日本語 (JA)
Content & Media Creation · freestylefly/canghe-skills

volcengine-video-understanding

Volcano Video Understanding - Analyzes video content using the Volcano Engine Ark Video Understanding API. Uploads videos via the Files API (recommended) with support for large files (up to 512MB), suitable for video content analysis, object recognition, action understanding, and more. Activate this skill when users need to analyze videos, understand video content, or extract information from videos.

L4 High-Privilege · Graded auth & full auditV1 Commit-PinnedExecute CommandNetwork AccessWrite FileDual-pass Audit
Author freestyleflyGitHub freestylefly/canghe-skills ↗License MIT（仓库根 LICENSE）Commit dd0bf35595
Control levels describe skill capabilities and suggested control measures; they do not represent quality or safety certifications, nor do they dictate usability. Usage decisions rest entirely with users and administrators. Different agent hosts apply varying constraints; do not assume hosts provide automatic protection. Third-party skill copyrights belong to their original authors. Use at your own risk.

1How It Works & Mechanism

Sends local video to ByteDance Volcano Ark's Video Understanding API, leveraging Seed series models for in-depth understanding and analysis.

skills/canghe-volcengine-video-understanding/SKILL.md
使用字节跳动火山方舟视频理解 API（doubao-seed-2-0-pro-260215 等模型）对视频进行深度理解和分析。
What is happening here: Declares that the core functionality is a 'video -> external multimodal model' invocation, with only orchestration and result parsing handled locally.

The recommended workflow is a two-step process: upload via Files API + analyze via Responses API.

skills/canghe-volcengine-video-understanding/SKILL.md
**推荐方式**：Files API 上传 + Responses API 分析
What is happening here: First uploads the video as a file (supporting large files and reusability), then initiates the understanding task using file_id.

The upload uses multipart format and carries preprocessing FPS configuration.

skills/canghe-volcengine-video-understanding/scripts/video_understand.py
"preprocess_configs[video][fps]": str(fps)
What is happening here: Allows the server to preprocess the video based on the sampling frame rate, balancing token cost and detail retention.

Polls file status after upload, proceeding only when it reaches active/processed.

skills/canghe-volcengine-video-understanding/scripts/video_understand.py
if status in ["active", "processed"]:
What is happening here: Waits for video preprocessing to complete, preventing analysis from being initiated before the file is ready.

The understanding task uses the Responses API, submitting input_video(file_id) together with input_text(instruction).

skills/canghe-volcengine-video-understanding/scripts/video_understand.py
"type": "input_video",
What is happening here: Assembles the video reference and the user query into a single multimodal request.

Authentication relies on the environment variable ARK_API_KEY (system environment -> current directory .env -> user home directory .env).

skills/canghe-volcengine-video-understanding/scripts/video_understand.py
return require_env_key("ARK_API_KEY")
What is happening here: Reads the API key from host common/env_utils, then calls Volcano Ark endpoints using a Bearer header.

Results can be parsed as text and saved to disk as JSON via --output.

skills/canghe-volcengine-video-understanding/scripts/video_understand.py
with open(args.output, 'w', encoding='utf-8') as f:
What is happening here: Saves the raw API response or extracts output_text for convenient post-processing.

Default model and sampling frame rate are configurable (default: doubao-seed-2-0-pro-260215 / fps=1).

skills/canghe-volcengine-video-understanding/SKILL.md
| `--fps` | 1 | 视频采样帧率（预处理） |
What is happening here: Lower FPS saves tokens for slow-paced videos, while higher FPS preserves details for fast-action scenes.

2Core Capabilities

01Video content understanding (scene/character/action/emotion)
02Q&A based on video content
03Automatic generation of video descriptions and summaries
04Files API upload for large files (up to 512MB)
05Configurable model and sampling frame rate
06Polling to wait for preprocessing completion
07Saving results to disk as JSON / Providing Python API (analyze_video)

3External Dependencies

KindDependency / Endpoint
api火山方舟视频理解 API（Files / Responses），模型 doubao-seed-2-0-pro-260215 等
packagerequests（HTTP 客户端）
clipython3（运行 video_understand.py）

4Recommended Governance & Evidence

Suggested Controls for Safe Operation
For Agent Runtime
Graded/per-use authorization, sandbox recommended
For Administrators
Approval workflow, rollback support, full audit
Evidence & Triggers (File Paths & Excerpts)
  • scripts/: Executes local script or command: python3 scripts/video_understand.py <video> "<指令>" [--model ...] [--fps N]
  • 网络端点: Declares external network access: POST https://ark.cn-beijing.volces.com/api/v3/files (multipart upload video, with prepro
  • 本地工作目录: File read/write and artifact output: JSON result file specified by --output (with open(args.output, 'w', encoding='utf-8'))
Audit Notes: Two independent reasons: ① [Reads environment variable credentials] Script get_api_key() -> require_env_key("ARK_API_KEY") reads ARK_API_KEY and sends it via Bearer header to Volcano Ark endpoints; ② [Sends user assets to external service backend] Video is uploaded via Files API and analyzed via Responses API, which transmits local videos and analysis instructions to a third-party multimodal model and consumes account quota. According to the rating paradigm 'reading credentials/environment variables, or sending user assets to external generation/service backend' = Orange. Not escalated to Red: no reverse engineering, no impersonation, no bypassing protections, no security downgrades (no disabled TLS verification/sandbox found), no arbitrary code execution surface; script structure is clear, un-obfuscated, and contains no hardcoded keys.

5Second-Pass Verification

  • [ok] 读取 ARK_API_KEY 并外发 — 脚本 get_api_key() 用 require_env_key("ARK_API_KEY")；三处请求均带 "Authorization": f"Bearer {api_key}"。
  • [ok] 端点定位 — BASE_URL = "https://ark.cn-beijing.volces.com/api/v3"；风险扫描『URL/端點』与『網路請求』命中 files/responses 与脚本 requests.post/get。
  • [ok] 素材上传到外部服务 — upload_video_file() 以 requests.post 到 {BASE_URL}/files 上传视频；analyze_video() 以 file_id 调 /responses。
  • [ok] 无本地危险执行面 — 风险扫描『危險旗標/防護繞過』『沙箱/未受控執行』均无命中；脚本无 eval/exec、无 shell。
  • [ok] 唯一写盘为 --output — 风险扫描『寫盤/產生檔案』仅命中 open(args.output, 'w', encoding='utf-8')。
  • [ok] env_utils 为跨目录依赖 — 脚本 sys.path 插入 COMMON_DIR = Path(__file__).parent.parent.parent / "common"，导入 load_env/require_env_key；该模块位于 skill 目录之外，本报告仅作推测定位。
  • [ok] 档位复核 — 读 env 密钥 + 用户视频出网至生成型服务 → orange；无 red 触发项。
  • [ok] 元数据与 commit — commit=dd0bf355955b4c82b764740b4183c86a72ba0e0c，license=MIT（仓库根），installs=null。

6Verdict & Scenarios

  • Complete interface encapsulation: Upload -> Poll preprocessing -> Initiate understanding -> Parse results, all in a single CLI run, supporting large files (≤512MB).
  • Provides both CLI and Python API (analyze_video), easy to integrate into other workflows.
  • Configurable FPS preprocessing allows tradeoffs between cost and detail; supports multiple Seed video understanding models.
  • Clear error handling and limitation specifications (formats/sizes/7-day retention/timeouts).
    • Best for: Suitable for developers and content teams who have a Volcano Engine Ark account and need content understanding, Q&A, or summarization for local videos, especially when handling larger video files (≤512MB).
    Not for: Not suitable for environments where video/media transmission over the network is prohibited or with strict data compliance requirements; not suitable for users without an ARK_API_KEY or quota; also not suitable for users seeking offline local inference.
    Install Agent-Ready
    ① Verified Mirror 2026-10-10
    Option A · Direct DownloadDownload volcengine-video-understanding.tar.gz
    sha256: 5f1ee27839a37239…
    Option B · Copy Guide for Agent
    install.json
    Agents automatically download and verify mirrors from this JSON guide.
    ② Upstream GitHub
    GitHub Repo ↗
    Pinned to commit dd0bf35595.