Docs & Knowledge · daymade/claude-code-skills
qa-expert
This skill should be used when establishing comprehensive QA testing processes for any software project. Use when creating test strategies, writing test cases following Google Testing Standards, executing test plans, tracking bugs with P0-P4 classification, calculating quality metrics, or generating progress reports. Includes autonomous execution capability via master prompts and complete documentation templates for third-party QA team handoffs. Implements OWASP security testing and achieves 90% coverage targets.
L4 High-Privilege · Graded auth & full auditV1 Commit-PinnedExecute CommandWrite FileDual-pass Audit
Control levels describe skill capabilities and suggested control measures; they do not represent quality or safety certifications, nor do they dictate usability. Usage decisions rest entirely with users and administrators. Different agent hosts apply varying constraints; do not assume hosts provide automatic protection. Third-party skill copyrights belong to their original authors. Use at your own risk.
1How It Works & Mechanism
The project skeleton is generated entirely to disk by scripts/init_qa_project.py using standard library mkdir + open: 4 directory levels (tests/docs, tests/docs/templates, tests/e2e, tests/fixtures) are hardcoded lists with no reliance on template engines or third-party packages.
qa-expert/scripts/init_qa_project.py
dirs = [ "tests/docs", "tests/docs/templates", "tests/e2e", "tests/fixtures" ]
What is happening here: The init script is the only part of this skill that 'actually executes'—a single command writes the physical structure of the QA project (directories + two CSVs + four Markdown templates) into the target directory, while all other capabilities are driven purely by documentation and prompts.
The CSV column names act as an implicit data contract between the two scripts: init writes 13 columns, and calculate_metrics.py subsequently reads and aggregates them using that exact same set of column names (Test Case ID/Status/Result/Bug ID/Priority/Notes).
qa-expert/scripts/init_qa_project.py
headers = [ "Test Case ID", "Category", "Priority", "Test Name", "Estimated Time (min)", "Prerequisites", "Status", "Result", "Bug ID", "Execution Date", "Executed By", "Notes", "Screenshot/Log" ]
What is happening here: The 'test plan' is transformed into a machine-aggregatable spreadsheet; column names serve as the interface, and altering column names or changing languages will silently break the metrics calculation script (see verification item 4 for details).
Metric calculation criteria are hardcoded in the script: executed count = rows where Status is strictly 'Completed', passed count = rows where Result is strictly '✅ PASSED'; the execution rate denominator is all valid test case rows, and the pass rate denominator is executed rows.
qa-expert/scripts/calculate_metrics.py
executed = len([t for t in tests if t['Status'] == 'Completed']) passed = len([t for t in tests if t['Result'] == '✅ PASSED']) failed = len([t for t in tests if t['Result'] == '❌ FAILED'])
What is happening here: Exact string matching is used instead of enums or case normalization—the benefit is zero dependencies and transparent criteria, but the cost is that any casing or spelling variance (e.g., 'completed', 'PASSED') is silently treated as 0, distorting metrics without throwing an error.
Only 3 quality gates are implemented in the script (100% execution rate, ≥80% pass rate, P0=0), and the P0 determination relies on checking whether the Notes column contains the string 'P0', which represents weak-evidence matching.
qa-expert/scripts/calculate_metrics.py
gates = { "Test Execution ≥100%": execution_rate >= 100, "Pass Rate ≥80%": pass_rate >= 80, "P0 Bugs = 0": len([t for t in tests if t['Bug ID'].startswith('BUG') and 'P0' in t['Notes']]) == 0, }
What is happening here: The gate table in SKILL.md lists 6 gates (additionally including P1≤5, coverage ≥80%, and 90% OWASP security coverage), but the script computes only 3, leaving the remaining 3 to manual evaluation—the documentation promises more than the code delivers (see verification item 3 for details).
The 'Ground Truth Principle' establishes documentation as the sole source of truth while the CSV merely tracks execution status, forming the core discipline of this skill (addressing lessons learned from real incidents where the consistency rate between CSV and document test case IDs was only 3.2%).
qa-expert/SKILL.md
- **Test case documents** (e.g., `02-CLI-TEST-CASES.md`) = **authoritative source** for test steps
What is happening here: Prevents executing tests against out-of-sync or incorrect CSV versions—test steps reference only the documentation, and the CSV functions strictly as a status board. This discipline is elaborated into three principles (single source of truth, separation of concerns, explicit referencing) in references/ground_truth_principle.md.
'Autonomous execution' is not implemented via code, but through a pasteable master prompt: it instructs the host LLM to inspect the CSV for progress, execute test cases sequentially, and update the CSV immediately after each test, enabling resume-from-breakpoint execution.
qa-expert/references/master_qa_prompt.md
You are a senior QA engineer with 20+ years of experience at Google. Execute the QA test plan.
What is happening here: The role of the 'QA executor' is entirely delegated to the host model—scripts merely generate the scaffolding, while actual test execution is driven by the LLM and project documentation; SKILL.md claims this brings 100x speedup, zero human tracking error, and resumability from breakpoints.
The state recovery patch enables resumability: the master prompt instructs the LLM to read TEST-EXECUTION-TRACKING.csv to identify the last 'Completed' case and proceed from the next one, enforcing that 'updates must be made immediately after each test, strictly prohibiting batch updates'.
qa-expert/SKILL.md
- Auto-resume from last completed test (reads tracking CSV)
What is happening here: Uses the CSV as external memory to circumvent LLM context loss, paired with a manual stop-gate requiring immediate halting and escalation upon encountering P0 defects.
The test case system originates from templates rather than code generation: the test case structure is defined by assets/templates/TEST-CASE-TEMPLATE.md (AAA + TC-[CATEGORY]-[NUMBER] naming + P0-P4 priority).
qa-expert/assets/templates/TEST-CASE-TEMPLATE.md
### TC-[CATEGORY]-[NUMBER]: [Descriptive Title]
What is happening here: Enforces 'reproducibility' as a formatting constraint—preconditions must be verifiable, steps must contain exact commands, and expected results must be measurable; the template includes two concrete examples: CLI installation and security test cases.
A gap exists between the files actually generated on disk by init and those claimed in the documentation: the generated README/MASTER-QA-PROMPT references files such as QA-HANDOVER-INSTRUCTIONS.md and categorized test case documents that init does not actually create.
qa-expert/scripts/init_qa_project.py
1. Read `QA-HANDOVER-INSTRUCTIONS.md`
What is happening here: The README template instructs users to read handover documentation first, but the six create_directory_structure/create_* functions only write TEST-EXECUTION-TRACKING.csv, BUG-TRACKING-TEMPLATE.csv, BASELINE-METRICS.md, WEEKLY-PROGRESS-REPORT.md, MASTER-QA-PROMPT.md, and README.md—following the documentation directly will result in missing files (see verification item 2 for details).
2Core Capabilities
01One-click generation of QA project skeleton (directory structure + tracking CSVs + documentation templates)
02Writing reproducible test cases following AAA pattern and TC-[CATEGORY]-[NUMBER] convention with P0-P4 prioritization
03Execution tracking discipline: documentation as specification source, CSV as status board, prohibiting batch updates
04Bug logging: 19-column template covering severity, component, reproduction steps, environment, fix, and verification
05Quality metrics dashboard: execution rate, pass rate, unique defect count, P0-P3 distribution, and quality gate status
06Daily and weekly progress report templates with baseline comparisons for stakeholder reporting
07OWASP Top 10 security test checklist targeting 90% (9/10) coverage
08LLM autonomous execution master prompt supporting automated tracking, bug logging, reporting, and breakpoint resumption
3External Dependencies
| Kind | Dependency / Endpoint |
|---|---|
| cli | python3（执行自带脚本；仅标准库 os/sys/csv/pathlib/datetime/collections） |
| cli | git（master prompt 中克隆被测仓库的示例步骤） |
| cli | docker / npx supabase（Day 1 环境搭建示例：起本地数据库） |
| cli | pnpm / npm（Day 1 安装依赖示例） |
| cli | curl（诊断脚本：检查本地 dev server 是否响应） |
| network | 文档内参考外链（仅 Markdown 超链接，代码不访问） |
4Recommended Governance & Evidence
Suggested Controls for Safe Operation
For Agent Runtime
Graded/per-use authorization, sandbox recommended
For Administrators
Approval workflow, rollback support, full audit
Evidence & Triggers (File Paths & Excerpts)
- scripts/: Executes local script or command: python3 scripts/init_qa_project.py <project-name> [output-directory] (writes directories and files)
- 本地工作目录: File I/O and artifact output: creates tests/docs, tests/docs/templates, tests/e2e, and tests/fixtures 4-tier directory structure under the target directory (via mkdir)
Audit Notes: Contains only two local scripts using the standard library: init writes directories and template files into the user-specified folder (no deletions or overwriting outside user-designated project paths), and calculate_metrics is strictly read-only. A full scan of the directory reveals zero network calls (no occurrences of requests/urllib/socket/curl; all http references in SKILL.md are localhost examples or external doc links) and no reading of credentials or environment variables. Remaining caveats: (1) 'Autonomous execution' treats project documentation as an instruction source and contains executable command examples like git/docker/npx—the agent following them will execute real commands on the host machine, which is a host permission and document trust issue rather than hidden behavior of this skill; (2) Generated artifacts reference uncreated files (QA-HANDOVER-INSTRUCTIONS.md), which will interrupt workflow if followed blindly; (3) Marketing claims (100x speedup / 30+ prompts / 6 quality gates) exceed actual code implementation and should be verified as needed.
5Second-Pass Verification
- [ok] 无网络/无凭证：脚本执行面反查 — 对 qa-expert/ 全目录 grep subprocess|requests|urllib|socket|http|os.system|os.environ|getenv|eval|exec：脚本侧零命中，仅参考文献里的 localhost 示例、文档外链与 docker ps 等示例命令。两脚本 import 仅 os/sys/csv/pathlib/datetime/collections。第一遍『无网络、无凭证』结论成立。
- [discrepancy] init 落盘清单 vs SKILL.md/生成文档的自我引用 — init 只创建 6 个文件：TESTS-EXECUTION-TRACKING.csv、BUG-TRACKING-TEMPLATE.csv、BASELINE-METRICS.md、WEEKLY-PROGRESS-REPORT.md、MASTER-QA-PROMPT.md、README.md（均在 tests/docs 或 tests/docs/templates 下）。但生成的 README 正文写『1. Read `QA-HANDOVER-INSTRUCTIONS.md`』并索引 QA-HANDOVER-INSTRUCTIONS.md、01-[CATEGORY]-TEST-CASES.md、02-SECURITY-TEST-CASES.md，MASTER-QA-PROMPT.md 也要求先读 tests/docs/QA-HANDOVER-INSTRUCTIONS.md——这些文件 init 一律不生成，第三方按模板走会在第一步中断。SKILL.md『What gets created』同样未提示该缺口。
- [discrepancy] 质量门禁：SKILL.md 6 条 vs 脚本 3 条 — SKILL.md 门禁表列 Test Execution 100% / Pass Rate ≥80% / P0=0 / P1≤5 / Code Coverage ≥80% / Security 90% 六条并标注均为 Blocker；calculate_metrics.py 的 gates 字典只有前三条，且 P0 判定实现为『Bug ID 以 BUG 开头且 Notes 含 P0』这一启发式。P1≤5、覆盖率、OWASP 三项无任何自动判定。
- [ok] 指标口径与 CSV 键名契约 — calculate_metrics 读取的键 Test Case ID/Status/Result/Bug ID/Priority/Notes 均存在于 init 写出的 13 列表头中，跨脚本契约自洽。但口径依赖精确字符串：Status=='Completed'、Result=='✅ PASSED'、Notes 含 'P0'，大小写或写法的任何偏差都会静默计 0（无校验、无告警）——这是口径脆弱性而非事实错误。
- [discrepancy] llm_prompts_library『30+ prompts』宣称 — SKILL.md 两处写『30+ ready-to-use prompts』/『30+ ready-to-use reporting prompts』。独立计数（grep '^```' = 36 行 → 18 个围栏代码块；'### ' 小节 29 个，其中含 Best Practices/Common Mistakes/Troubleshooting 等非 prompt 小节）→ 可粘贴 prompt 块实为 18 个，最宽口径（含故障排查内的短示例）也不足 30。宣称偏高。
- [ok] 关键方法论声明与实际内容一致性（AAA/5 小时/OWASP 90%/P0-P4） — day1_onboarding.md 首行即『**Time**: 5 hours (with breaks)』，与 SKILL.md 的 5 小时时间线一致；google_testing_standards.md 确实给出 AAA 三段结构、TC-[CATEGORY]-[NUMBER] 命名与 P0-P4 定义；SKILL.md 的 OWASP A01-A07 清单与『**Target**: 90% OWASP coverage (9/10 threats mitigated)』自洽；模板文件 TEST-CASE-TEMPLATE.md 含 AAA 字段与实例。无夸大。
- [ok] Ground Truth 原则的双向核对 — SKILL.md『Test case documents … = authoritative source for test steps』与 ground_truth_principle.md『Ground Truth: 02-CLI-TEST-CASES.md (detailed test specifications)』/『Lesson Learned: … 3.2% consistency rate』表述一致，纪律有事故依据，非泛泛口号。
- [ok] pin commit 与目录资产面核对 — git rev-parse HEAD = d5c4678cb5d4fd6acc9c922690df035dbd33d247，提交时间 2026-09-09T20:31:25+08:00（= UTC 12:31:25Z），与任务给定一致。glob 全量（含隐藏文件）确认 qa-expert/ 仅 SKILL.md + scripts/2 + references/5 + assets/templates/1，无隐藏脚本、无二进制、无 .env。
6Verdict & Scenarios
Best for: Suitable for: teams building an auditable QA process from scratch for a project (especially hybrid Web+CLI+API architectures requiring handoffs to external QA teams or LLMs); users wishing to outsource test execution to an LLM while retaining CSV-level progress auditing; and projects needing an OWASP security testing checklist alongside P0-P4 defect grading criteria. In hosts such as Claude Code, run the init script first, fill in missing handover docs referenced in the README, and start execution with the master prompt.
Not for: Not suitable for: scenarios requiring automated generation and execution of test code—this skill does not output any test scripts, relying entirely on human or LLM execution; teams with mature CI test frameworks (e.g., Playwright/Jest/pytest+Allure) seeking automated code coverage and reporting, as it provides only documentation templates and spreadsheet schemas; unsupervised long-term autonomous runs, since quality gates only automate 3 rules, metric matching is brittle, and generated documentation references missing files requiring manual intervention; or compliance audit environments requiring strict privilege isolation (autonomous execution relies on host shell permissions).
Install Agent-Ready
① Verified Mirror 2026-09-15
Option A · Direct DownloadDownload qa-expert.tar.gz
sha256:
Option B · Copy Guide for Agenta79a23756c922164…Agents automatically download and verify mirrors from this JSON guide.
② Upstream GitHub
GitHub Repo ↗
Pinned to commit d5c4678cb5.