Content & Media Creation · zarazhangrui/youtube-to-ebook
youtube-to-ebook
Transform YouTube videos into beautifully formatted ebook articles with transcripts
L4 High-Privilege · Graded auth & full auditV1 Commit-PinnedExecute CommandInstall DepsNetwork AccessWrite FileDual-pass Audit
Control levels describe skill capabilities and suggested control measures; they do not represent quality or safety certifications, nor do they dictate usability. Usage decisions rest entirely with users and administrators. Different agent hosts apply varying constraints; do not assume hosts provide automatic protection. Third-party skill copyrights belong to their original authors. Use at your own risk.
1How It Works & Mechanism
Four stages are linked into a single local pipeline (fetch videos → get transcripts → rewrite into magazine-style articles with Claude → package into EPUB and deliver via email); main.py simply invokes them sequentially without any branching logic.
main.py
from get_videos import main as fetch_videos from get_transcripts import get_transcripts_for_videos from write_articles import write_articles_for_videos from send_email import send_newsletter from video_tracker import filter_new_videos, mark_videos_processed, get_processed_count
Inside run(), STEP 1-5 are executed in sequence, proceeding only when each step yields non-empty output, and calling mark_videos_processed() after the final step succeeds. This is a verbatim implementation of the four-box flowchart in SKILL.md (Fetch Videos → Get Transcripts → Write Articles → Create EPUB).
Video discovery uses the YouTube Data API uploads playlist rather than search (bypassing the issue that 'search does not guarantee chronological order'), and relies on /shorts/ URL redirects to identify Shorts.
get_videos.py
# Get the 15 most recent videos from the uploads playlist # The uploads playlist is always in exact upload order (newest first)
Implementation details: get_channel_info uses channels().list(part="snippet,contentDetails", forHandle=handle) to retrieve relatedPlaylists.uploads; is_youtube_short uses requests.head('https://www.youtube.com/shorts/{id}', allow_redirects=True) to check if the final URL still contains /shorts/. SKILL.md lists these two as 'Known Pitfalls & Solutions' #1/#2.
Transcript retrieval practically relies on a third-party commercial API (Supadata) rather than YouTube's official API or open-source libraries—code comments explicitly state this is to 'avoid YouTube bans'.
get_transcripts.py
This script uses the Supadata API to fetch transcripts reliably. Unlike the youtube-transcript-api library, Supadata doesn't get blocked by YouTube.
Invocation format: requests.get(SUPADATA_TRANSCRIPT_URL, params={'url': youtube_url, 'text': 'true'}, headers={'x-api-key': SUPADATA_API_KEY}, timeout=60), endpoint https://api.supadata.ai/v1/transcript; rate limiting is implemented with a 1-second sleep between items ('if i < len(videos) - 1: time.sleep(1)').
'Magazine-style rewriting' uses a hardcoded, lengthy prompt: requiring title rewrites, targeting non-expert readers, explaining technical terms, preserving counterintuitive perspectives and memorable quotes, and explicitly mandating standalone articles readable without watching the video.
write_articles.py
- Do NOT include phrases like "In this video" - write it as a standalone article. Assume the reader has not watched the video and has zero context about it. This article is meant to be as a replacement, not complement, for watching the video.
The same prompt also mandates correcting transcription errors using the video title and description (SKILL.md Pitfall #5), hardcoding the model to claude-sonnet-4-20250514 and max_tokens=8000; dashboard.py's 'Writing Style' panel applies regex replacements to this prompt directly in the source file (PROMPT_FILE = PROJECT_DIR / "write_articles.py").
The final product is a genuine EPUB (ebooklib generates TOC + chapters + CSS + reading order) delivered as an attachment via Gmail SMTP_SSL on port 465, while archiving HTML/EPUB/JSON files to newsletters/.
send_email.py
# Create the ebook book = epub.EpubBook()
Sending logic: 'with smtplib.SMTP_SSL("smtp.gmail.com", 465) as server: server.login(GMAIL_ADDRESS, GMAIL_APP_PASSWORD); server.sendmail(...)'; upon successful sending, save_newsletter_archive() writes newsletters/newsletter_<ts>.{html,epub,json}, and os.remove(epub_path) cleans up temporary files. Filename format: youtube_digest_YYYYMMDD.epub.
Deduplication relies on a local JSON tracker: processed video IDs are persisted to processed_videos.json, processing only unseen new videos and marking them only after the email has been successfully sent.
video_tracker.py
TRACKER_FILE = os.path.join(os.path.dirname(__file__), "processed_videos.json")
In main.py, this corresponds to 'new_videos = filter_new_videos(videos)' and 'if success: mark_videos_processed(videos_with_transcripts)'; GitHub Actions uploads and downloads this file as an artifact (retention-days: 90), which serves as its sole mechanism for state persistence in CI environments.
Two optional automation channels are included in the repository: macOS launchd (plist + run_newsletter.sh with log redirection) and a weekly Wednesday GitHub Actions cron schedule (requiring 4 secrets).
.github/workflows/newsletter.yml
env: YOUTUBE_API_KEY: ${{ secrets.YOUTUBE_API_KEY }} ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} GMAIL_ADDRESS: ${{ secrets.GMAIL_ADDRESS }} GMAIL_APP_PASSWORD: ${{ secrets.GMAIL_APP_PASSWORD }} SUPADATA_API_KEY: ${{ secrets.SUPADATA_API_KEY }}
Triggers are 'cron: \'0 15 * * 3\'' (Wednesday 15:00 UTC) + workflow_dispatch; the local channel is defined in com.youtube.newsletter.plist and run_newsletter.sh ('python3 /Users/bytedance/youtube-newsletter/main.py >> …/logs/newsletter.log 2>&1'). Both channels coexist, whereas SKILL.md/README claim that 'cloud servers will get banned, please run locally'.
The bundled Streamlit dashboard is implemented by 'directly modifying source code files': the channel list edits get_videos.py, writing style modifies the prompt inside write_articles.py, and scheduling edits the plist under ~/Library/LaunchAgents followed by launchctl reload.
dashboard.py
CHANNELS_FILE = PROJECT_DIR / "get_videos.py" PROMPT_FILE = PROJECT_DIR / "write_articles.py"
Schedule persistence uses 'subprocess.run(["launchctl", "bootout", f"gui/{os.getuid()}", str(PLIST_FILE)], capture_output=True)' and bootstrap; pipeline triggering via button runs subprocess.run(["python3", str(PROJECT_DIR / "main.py")]). This means the dashboard only works on macOS with streamlit installed (streamlit is missing from requirements.txt).
2Core Capabilities
01Batch fetch the latest long-form videos by channel while automatically filtering out Shorts
02Reliably retrieve plain text YouTube transcripts via Supadata (including rate limiting and error code handling)
03Rewrite transcripts into standalone magazine-style long-form articles using Claude (including title rewriting, term explanations, and preserving memorable quotes)
04Generate standard EPUB ebooks complete with TOC, CSS, and chapter structures
05Deliver via Gmail (EPUB attachment + styled HTML body) with local archiving for later reading
06Incremental processing and deduplication: tracks sent videos in processed_videos.json to prevent duplicate delivery
07macOS scheduled automation (launchd plist running the local pipeline at fixed weekly times)
08Streamlit graphical dashboard: add/remove channels, edit writing styles, manually trigger pipelines, and browse newsletter archives and processed video logs
3External Dependencies
| Kind | Dependency / Endpoint |
|---|---|
| api | YouTube Data API v3（Google） |
| network | YouTube Shorts URL 探测（HEAD 请求判重定向） |
| api | Supadata 字幕 API（第三方商业服务） |
| api | Anthropic Claude API（文章改写） |
| network | Gmail SMTP（投递 EPUB 附件） |
| package | Python 依赖（含一个已不再被代码使用的库） |
| package | streamlit（dashboard 专用，未列入 requirements.txt） |
| cli | launchctl（macOS 排期，dashboard 内以子进程调用） |
| network | Google Fonts（dashboard 页面样式 @import，浏览器侧发起） |
4Recommended Governance & Evidence
Suggested Controls for Safe Operation
For Agent Runtime
Graded/per-use authorization, sandbox recommended
For Administrators
Approval workflow, rollback support, full audit
Evidence & Triggers (File Paths & Excerpts)
- scripts/: Local script/command execution: python main.py (main pipeline; entry point specified in README/SKILL.md)
- scripts/ 或 SKILL.md: Dynamic external language package dependency installation declared
- 网络端点: External network access declared: googleapis.com YouTube Data API v3 (channels.list / playlistItems.list, with dev key)
- 本地工作目录: File I/O and artifact output: processed_videos.json (tracks processed videos within project root)
Audit Notes: Standard classification rationale: This skill accesses 4-5 sets of credentials (YouTube API key, Anthropic key, Supadata key, Gmail account + App Password, and CI secrets), where the Gmail App Password represents a long-lived credential with write access to an email inbox. The workflow transmits transcripts and descriptions to external third parties (Supadata commercial API, Anthropic), falling under 'credential/environment variable access + third-party service dependency'. Since all external outbound targets are expected official or commercial endpoints, without TLS degradation, anti-scraping circumvention, or arbitrary code execution vectors (subprocesses strictly execute fixed argv for python3/launchctl), it is not flagged as Red. However, centralized credentials, email sending capabilities, and subprocess scheduling impart higher practical impact than typical text generation skills.
5Second-Pass Verification
- [ok] 四阶段流水线与去重逻辑 — main.py 五个 import 与方法体逐行核对；video_tracker.py 的 load/save/is_video_processed/mark_video_processed 与 processed_videos.json 路径均存在。
- [discrepancy] 字幕来自 Supadata（而非 SKILL.md 所述 youtube-transcript-api） — 代码侧唯一实现是 Supadata（get_transcripts.py 顶部注释与 SUPADATA_TRANSCRIPT_URL 常量）；但 SKILL.md『Known Pitfalls』#3 仍在教 'ytt_api = YouTubeTranscriptApi(); transcript = ytt_api.fetch(video_id)'，requirements.txt 也仍列 youtube-transcript-api——全仓 .py 无一处 import youtube_transcript_api。文档描述的是已被替换掉的旧实现。
- [discrepancy] 『云服务会被 YouTube 封，请本地跑』与仓库自带 Actions 工作流 — SKILL.md『### 6. Cloud Automation Blocked … Solution: Run automation locally on your Mac using launchd』、README『| Cloud servers blocked | Run locally, not GitHub Actions |』，但 .github/workflows/newsletter.yml 仍是完整的 ubuntu-latest 定时任务（cron '0 15 * * 3'），且注入 5 个 secrets。两种说法不能同时成立；从代码看 Supadata 正是为绕开该封锁而引入（'Supadata doesn't get blocked by YouTube'），故云路径更可能是『能跑但不保证字幕』。
- [discrepancy] 配置入口 channels.txt 与 `python main.py --channels` — SKILL.md 命令表写 '| `python main.py --channels` | Edit channel list |'、Key Files 与 README 结构表都列 channels.txt；实际 main.py 无 argparse（全仓 .py 无 argparse 引用），频道是 get_videos.py 的 CHANNELS 常量，dashboard 通过 CHANNELS_FILE = get_videos.py 改源码。channels.txt 在仓库中不存在。
- [discrepancy] SUPADATA_API_KEY 的文档覆盖 — .env.example 只有 YOUTUBE_API_KEY / ANTHROPIC_API_KEY / GMAIL_ADDRESS / GMAIL_APP_PASSWORD 四项，而 get_transcripts.py 在无该 key 时直接返回 None（『⚠ SUPADATA_API_KEY not set in .env file』）导致流水线无字幕。文档缺失这把必需 key（CI workflow 里反而有 secrets.SUPADATA_API_KEY）。
- [discrepancy] 许可声明一致性 — README 末段写 'MIT - Use freely, modify as needed.'，但 git ls-tree 无 LICENSE 文件，gh api license=null（spdx 未识别）。已在 meta.license 如实标注冲突，未按 MIT 硬写。
- [ok] 邮件投递与留档细节 — SMTP_SSL('smtp.gmail.com', 465) + server.login + sendmail；MIMEBase('application','epub+zip') 附件；save_newsletter_archive 写 html/epub/json 三件套；成功后 os.remove 临时 EPUB——与 SKILL.md『Email Delivery (Optional)』一致。
- [ok] dashboard 的破坏性写回（源码/plist） — save_channels 重写 get_videos.py 的 CHANNELS 块、风格保存重写 write_articles.py 内 prompt、save_schedule 重写 ~/Library/LaunchAgents plist 并 launchctl bootout/bootstrap —— 已在 file_writes 与 injection_surface 记入；另确认 requirements.txt 未含 streamlit（README 要求单独 pip install）。
6Verdict & Scenarios
Best for: Best for: Individuals or small teams who want to compile their subscribed technical channels into a weekly EPUB digest sent to themselves (the default recipient is GMAIL_ADDRESS); users comfortable managing 4-5 API keys/credentials, running scheduled jobs on macOS via launchd, and routing transcripts through a third-party commercial API; also serves as an excellent reference implementation for building 'YouTube → Content → Ebook' pipelines due to its clean and decoupled architecture.
Not for: Not recommended for: Compliance-sensitive environments that forbid sending transcript data to third-party commercial APIs; users reluctant to store long-lived credentials in local .env files (Gmail app password + 3 types of API keys); non-macOS users (plist, launchctl, and troubleshooting docs assume macOS, rendering schedule management unusable); users expecting turnkey operation solely from documentation (significant drift between docs and code requires verifying actual implementation); and scenarios requiring exact timestamps, multi-language transcripts, or high-volume channel monitoring (only fetches the single latest long-form video per channel, transcript quality depends entirely on Supadata).
Install Agent-Ready
① Verified Mirror 2026-09-15
Option A · Direct DownloadDownload youtube-to-ebook.tar.gz
sha256:
Option B · Copy Guide for Agent2f7c07f82fd3668b…Agents automatically download and verify mirrors from this JSON guide.
② Upstream GitHub
GitHub Repo ↗
Pinned to commit 7f67fd6b55.