Merge pull request #265 from shimo4228/feat/skills/skill-stocktake

feat(skills): add skill-stocktake skill
2026-03-30 13:43:26 +08:00 · 2026-02-23 06:55:38 -08:00
parent 156b89ed30 91b7ccf56f
commit 3c833d8922
4 changed files with 488 additions and 0 deletions
--- a/skills/skill-stocktake/SKILL.md
+++ b/skills/skill-stocktake/SKILL.md
@@ -0,0 +1,175 @@
+---
+description: "Use when auditing Claude skills and commands for quality. Supports Quick Scan (changed skills only) and Full Stocktake modes with sequential subagent batch evaluation."
+---
+
+# skill-stocktake
+
+Slash command (`/skill-stocktake`) that audits all Claude skills and commands using a quality checklist + AI holistic judgment. Supports two modes: Quick Scan for recently changed skills, and Full Stocktake for a complete review.
+
+## Scope
+
+The command targets the following paths **relative to the directory where it is invoked**:
+
+| Path | Description |
+|------|-------------|
+| `~/.claude/skills/` | Global skills (all projects) |
+| `{cwd}/.claude/skills/` | Project-level skills (if the directory exists) |
+
+**At the start of Phase 1, the command explicitly lists which paths were found and scanned.**
+
+### Targeting a specific project
+
+To include project-level skills, run from that project's root directory:
+
+```bash
+cd ~/path/to/my-project
+/skill-stocktake
+```
+
+If the project has no `.claude/skills/` directory, only global skills and commands are evaluated.
+
+## Modes
+
+| Mode | Trigger | Duration |
+|------|---------|---------|
+| Quick Scan | `results.json` exists (default) | 5–10 min |
+| Full Stocktake | `results.json` absent, or `/skill-stocktake full` | 20–30 min |
+
+**Results cache:** `~/.claude/skills/skill-stocktake/results.json`
+
+## Quick Scan Flow
+
+Re-evaluate only skills that have changed since the last run (5–10 min).
+
+1. Read `~/.claude/skills/skill-stocktake/results.json`
+2. Run: `bash ~/.claude/skills/skill-stocktake/scripts/quick-diff.sh \
+         ~/.claude/skills/skill-stocktake/results.json`
+   (Project dir is auto-detected from `$PWD/.claude/skills`; pass it explicitly only if needed)
+3. If output is `[]`: report "No changes since last run." and stop
+4. Re-evaluate only those changed files using the same Phase 2 criteria
+5. Carry forward unchanged skills from previous results
+6. Output only the diff
+7. Run: `bash ~/.claude/skills/skill-stocktake/scripts/save-results.sh \
+         ~/.claude/skills/skill-stocktake/results.json <<< "$EVAL_RESULTS"`
+
+## Full Stocktake Flow
+
+### Phase 1 — Inventory
+
+Run: `bash ~/.claude/skills/skill-stocktake/scripts/scan.sh`
+
+The script enumerates skill files, extracts frontmatter, and collects UTC mtimes.
+Project dir is auto-detected from `$PWD/.claude/skills`; pass it explicitly only if needed.
+Present the scan summary and inventory table from the script output:
+
+```
+Scanning:
+  ✓ ~/.claude/skills/         (17 files)
+  ✗ {cwd}/.claude/skills/    (not found — global skills only)
+```
+
+| Skill | 7d use | 30d use | Description |
+|-------|--------|---------|-------------|
+
+### Phase 2 — Quality Evaluation
+
+Launch a Task tool subagent (**Explore agent, model: opus**) with the full inventory and checklist.
+The subagent reads each skill, applies the checklist, and returns per-skill JSON:
+
+`{ "verdict": "Keep"|"Improve"|"Update"|"Retire"|"Merge into [X]", "reason": "..." }`
+
+**Chunk guidance:** Process ~20 skills per subagent invocation to keep context manageable. Save intermediate results to `results.json` (`status: "in_progress"`) after each chunk.
+
+After all skills are evaluated: set `status: "completed"`, proceed to Phase 3.
+
+**Resume detection:** If `status: "in_progress"` is found on startup, resume from the first unevaluated skill.
+
+Each skill is evaluated against this checklist:
+
+```
+- [ ] Content overlap with other skills checked
+- [ ] Overlap with MEMORY.md / CLAUDE.md checked
+- [ ] Freshness of technical references verified (use WebSearch if tool names / CLI flags / APIs are present)
+- [ ] Usage frequency considered
+```
+
+Verdict criteria:
+
+| Verdict | Meaning |
+|---------|---------|
+| Keep | Useful and current |
+| Improve | Worth keeping, but specific improvements needed |
+| Update | Referenced technology is outdated (verify with WebSearch) |
+| Retire | Low quality, stale, or cost-asymmetric |
+| Merge into [X] | Substantial overlap with another skill; name the merge target |
+
+Evaluation is **holistic AI judgment** — not a numeric rubric. Guiding dimensions:
+- **Actionability**: code examples, commands, or steps that let you act immediately
+- **Scope fit**: name, trigger, and content are aligned; not too broad or narrow
+- **Uniqueness**: value not replaceable by MEMORY.md / CLAUDE.md / another skill
+- **Currency**: technical references work in the current environment
+
+**Reason quality requirements** — the `reason` field must be self-contained and decision-enabling:
+- Do NOT write "unchanged" alone — always restate the core evidence
+- For **Retire**: state (1) what specific defect was found, (2) what covers the same need instead
+  - Bad: `"Superseded"`
+  - Good: `"disable-model-invocation: true already set; superseded by continuous-learning-v2 which covers all the same patterns plus confidence scoring. No unique content remains."`
+- For **Merge**: name the target and describe what content to integrate
+  - Bad: `"Overlaps with X"`
+  - Good: `"42-line thin content; Step 4 of chatlog-to-article already covers the same workflow. Integrate the 'article angle' tip as a note in that skill."`
+- For **Improve**: describe the specific change needed (what section, what action, target size if relevant)
+  - Bad: `"Too long"`
+  - Good: `"276 lines; Section 'Framework Comparison' (L80–140) duplicates ai-era-architecture-principles; delete it to reach ~150 lines."`
+- For **Keep** (mtime-only change in Quick Scan): restate the original verdict rationale, do not write "unchanged"
+  - Bad: `"Unchanged"`
+  - Good: `"mtime updated but content unchanged. Unique Python reference explicitly imported by rules/python/; no overlap found."`
+
+### Phase 3 — Summary Table
+
+| Skill | 7d use | Verdict | Reason |
+|-------|--------|---------|--------|
+
+### Phase 4 — Consolidation
+
+1. **Retire / Merge**: present detailed justification per file before confirming with user:
+   - What specific problem was found (overlap, staleness, broken references, etc.)
+   - What alternative covers the same functionality (for Retire: which existing skill/rule; for Merge: the target file and what content to integrate)
+   - Impact of removal (any dependent skills, MEMORY.md references, or workflows affected)
+2. **Improve**: present specific improvement suggestions with rationale:
+   - What to change and why (e.g., "trim 430→200 lines because sections X/Y duplicate python-patterns")
+   - User decides whether to act
+3. **Update**: present updated content with sources checked
+4. Check MEMORY.md line count; propose compression if >100 lines
+
+## Results File Schema
+
+`~/.claude/skills/skill-stocktake/results.json`:
+
+**`evaluated_at`**: Must be set to the actual UTC time of evaluation completion.
+Obtain via Bash: `date -u +%Y-%m-%dT%H:%M:%SZ`. Never use a date-only approximation like `T00:00:00Z`.
+
+```json
+{
+  "evaluated_at": "2026-02-21T10:00:00Z",
+  "mode": "full",
+  "batch_progress": {
+    "total": 80,
+    "evaluated": 80,
+    "status": "completed"
+  },
+  "skills": {
+    "skill-name": {
+      "path": "~/.claude/skills/skill-name/SKILL.md",
+      "verdict": "Keep",
+      "reason": "Concrete, actionable, unique value for X workflow",
+      "mtime": "2026-01-15T08:30:00Z"
+    }
+  }
+}
+```
+
+## Notes
+
+- Evaluation is blind: the same checklist applies to all skills regardless of origin (ECC, self-authored, auto-extracted)
+- Archive / delete operations always require explicit user confirmation
+- No verdict branching by skill origin
--- a/skills/skill-stocktake/scripts/quick-diff.sh
+++ b/skills/skill-stocktake/scripts/quick-diff.sh
@@ -0,0 +1,87 @@
+#!/usr/bin/env bash
+# quick-diff.sh — compare skill file mtimes against results.json evaluated_at
+# Usage: quick-diff.sh RESULTS_JSON [CWD_SKILLS_DIR]
+# Output: JSON array of changed/new files to stdout (empty [] if no changes)
+#
+# When CWD_SKILLS_DIR is omitted, defaults to $PWD/.claude/skills so the
+# script always picks up project-level skills without relying on the caller.
+#
+# Environment:
+#   SKILL_STOCKTAKE_GLOBAL_DIR   Override ~/.claude/skills (for testing only;
+#                                do not set in production — intended for bats tests)
+#   SKILL_STOCKTAKE_PROJECT_DIR  Override project dir detection (for testing only)
+
+set -euo pipefail
+
+RESULTS_JSON="${1:-}"
+CWD_SKILLS_DIR="${SKILL_STOCKTAKE_PROJECT_DIR:-${2:-$PWD/.claude/skills}}"
+GLOBAL_DIR="${SKILL_STOCKTAKE_GLOBAL_DIR:-$HOME/.claude/skills}"
+
+if [[ -z "$RESULTS_JSON" || ! -f "$RESULTS_JSON" ]]; then
+  echo "Error: RESULTS_JSON not found: ${RESULTS_JSON:-<empty>}" >&2
+  exit 1
+fi
+
+# Validate CWD_SKILLS_DIR looks like a .claude/skills path (defense-in-depth).
+# Only warn when the path exists — a nonexistent path poses no traversal risk.
+if [[ -n "$CWD_SKILLS_DIR" && -d "$CWD_SKILLS_DIR" && "$CWD_SKILLS_DIR" != */.claude/skills* ]]; then
+  echo "Warning: CWD_SKILLS_DIR does not look like a .claude/skills path: $CWD_SKILLS_DIR" >&2
+fi
+
+evaluated_at=$(jq -r '.evaluated_at' "$RESULTS_JSON")
+
+# Fail fast on a missing or malformed evaluated_at rather than producing
+# unpredictable results from ISO 8601 string comparison against "null".
+if [[ ! "$evaluated_at" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$ ]]; then
+  echo "Error: invalid or missing evaluated_at in $RESULTS_JSON: $evaluated_at" >&2
+  exit 1
+fi
+
+# Pre-extract known paths from results.json once (O(1) lookup per file instead of O(n*m))
+known_paths=$(jq -r '.skills[].path' "$RESULTS_JSON" 2>/dev/null)
+
+tmpdir=$(mktemp -d)
+# Use a function to avoid embedding $tmpdir in a quoted string (prevents injection
+# if TMPDIR were crafted to contain shell metacharacters).
+_cleanup() { rm -rf "$tmpdir"; }
+trap _cleanup EXIT
+
+# Shared counter across process_dir calls — intentionally NOT local
+i=0
+
+process_dir() {
+  local dir="$1"
+  while IFS= read -r file; do
+    local mtime dp is_new
+    mtime=$(date -u -r "$file" +%Y-%m-%dT%H:%M:%SZ)
+    dp="${file/#$HOME/~}"
+
+    # Check if this file is known to results.json (exact whole-line match to
+    # avoid substring false-positives, e.g. "python-patterns" matching "python-patterns-v2").
+    if echo "$known_paths" | grep -qxF "$dp"; then
+      is_new="false"
+      # Known file: only emit if mtime changed (ISO 8601 string comparison is safe)
+      [[ "$mtime" > "$evaluated_at" ]] || continue
+    else
+      is_new="true"
+      # New file: always emit regardless of mtime
+    fi
+
+    jq -n \
+      --arg path "$dp" \
+      --arg mtime "$mtime" \
+      --argjson is_new "$is_new" \
+      '{path:$path,mtime:$mtime,is_new:$is_new}' \
+      > "$tmpdir/$i.json"
+    i=$((i+1))
+  done < <(find "$dir" -name "*.md" -type f 2>/dev/null | sort)
+}
+
+[[ -d "$GLOBAL_DIR" ]] && process_dir "$GLOBAL_DIR"
+[[ -n "$CWD_SKILLS_DIR" && -d "$CWD_SKILLS_DIR" ]] && process_dir "$CWD_SKILLS_DIR"
+
+if [[ $i -eq 0 ]]; then
+  echo "[]"
+else
+  jq -s '.' "$tmpdir"/*.json
+fi
--- a/skills/skill-stocktake/scripts/save-results.sh
+++ b/skills/skill-stocktake/scripts/save-results.sh
@@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+# save-results.sh — merge evaluated skills into results.json with correct UTC timestamp
+# Usage: save-results.sh RESULTS_JSON <<< "$EVAL_JSON"
+#
+# stdin format:
+#   { "skills": {...}, "mode"?: "full"|"quick", "batch_progress"?: {...} }
+#
+# Always sets evaluated_at to current UTC time via `date -u`.
+# Merges stdin .skills into existing results.json (new entries override old).
+# Optionally updates .mode and .batch_progress if present in stdin.
+
+set -euo pipefail
+
+RESULTS_JSON="${1:-}"
+
+if [[ -z "$RESULTS_JSON" ]]; then
+  echo "Error: RESULTS_JSON argument required" >&2
+  echo "Usage: save-results.sh RESULTS_JSON <<< \"\$EVAL_JSON\"" >&2
+  exit 1
+fi
+
+EVALUATED_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)
+
+# Read eval results from stdin and validate JSON before touching the results file
+input_json=$(cat)
+if ! echo "$input_json" | jq empty 2>/dev/null; then
+  echo "Error: stdin is not valid JSON" >&2
+  exit 1
+fi
+
+if [[ ! -f "$RESULTS_JSON" ]]; then
+  # Bootstrap: create new results.json from stdin JSON + current UTC timestamp
+  echo "$input_json" | jq --arg ea "$EVALUATED_AT" \
+    '. + { evaluated_at: $ea }' > "$RESULTS_JSON"
+  exit 0
+fi
+
+# Merge: new .skills override existing ones; old skills not in input_json are kept.
+# Optionally update .mode and .batch_progress if provided.
+#
+# Use mktemp for a collision-safe temp file (concurrent runs on the same RESULTS_JSON
+# would race on a predictable ".tmp" suffix; random suffix prevents silent overwrites).
+tmp=$(mktemp "${RESULTS_JSON}.XXXXXX")
+trap 'rm -f "$tmp"' EXIT
+
+jq -s \
+  --arg ea "$EVALUATED_AT" \
+  '.[0] as $existing | .[1] as $new |
+   $existing |
+   .evaluated_at = $ea |
+   .skills = ($existing.skills + ($new.skills // {})) |
+   if ($new | has("mode")) then .mode = $new.mode else . end |
+   if ($new | has("batch_progress")) then .batch_progress = $new.batch_progress else . end' \
+  "$RESULTS_JSON" <(echo "$input_json") > "$tmp"
+
+mv "$tmp" "$RESULTS_JSON"
--- a/skills/skill-stocktake/scripts/scan.sh
+++ b/skills/skill-stocktake/scripts/scan.sh
@@ -0,0 +1,170 @@
+#!/usr/bin/env bash
+# scan.sh — enumerate skill files, extract frontmatter and UTC mtime
+# Usage: scan.sh [CWD_SKILLS_DIR]
+# Output: JSON to stdout
+#
+# When CWD_SKILLS_DIR is omitted, defaults to $PWD/.claude/skills so the
+# script always picks up project-level skills without relying on the caller.
+#
+# Environment:
+#   SKILL_STOCKTAKE_GLOBAL_DIR   Override ~/.claude/skills (for testing only;
+#                                do not set in production — intended for bats tests)
+#   SKILL_STOCKTAKE_PROJECT_DIR  Override project dir detection (for testing only)
+
+set -euo pipefail
+
+GLOBAL_DIR="${SKILL_STOCKTAKE_GLOBAL_DIR:-$HOME/.claude/skills}"
+CWD_SKILLS_DIR="${SKILL_STOCKTAKE_PROJECT_DIR:-${1:-$PWD/.claude/skills}}"
+# Path to JSONL file containing tool-use observations (optional; used for usage frequency counts).
+# Override via SKILL_STOCKTAKE_OBSERVATIONS env var if your setup uses a different path.
+OBSERVATIONS="${SKILL_STOCKTAKE_OBSERVATIONS:-$HOME/.claude/observations.jsonl}"
+
+# Validate CWD_SKILLS_DIR looks like a .claude/skills path (defense-in-depth).
+# Only warn when the path exists — a nonexistent path poses no traversal risk.
+if [[ -n "$CWD_SKILLS_DIR" && -d "$CWD_SKILLS_DIR" && "$CWD_SKILLS_DIR" != */.claude/skills* ]]; then
+  echo "Warning: CWD_SKILLS_DIR does not look like a .claude/skills path: $CWD_SKILLS_DIR" >&2
+fi
+
+# Extract a frontmatter field (handles both quoted and unquoted single-line values).
+# Does NOT support multi-line YAML blocks (| or >) or nested YAML keys.
+extract_field() {
+  local file="$1" field="$2"
+  awk -v f="$field" '
+    BEGIN { fm=0 }
+    /^---$/ { fm++; next }
+    fm==1 {
+      n = length(f) + 2
+      if (substr($0, 1, n) == f ": ") {
+        val = substr($0, n+1)
+        gsub(/^"/, "", val)
+        gsub(/"$/, "", val)
+        print val
+        exit
+      }
+    }
+    fm>=2 { exit }
+  ' "$file"
+}
+
+# Get UTC timestamp N days ago (supports both macOS and GNU date)
+date_ago() {
+  local n="$1"
+  date -u -v-"${n}d" +%Y-%m-%dT%H:%M:%SZ 2>/dev/null ||
+  date -u -d "${n} days ago" +%Y-%m-%dT%H:%M:%SZ
+}
+
+# Count observations matching a file path since a cutoff timestamp
+count_obs() {
+  local file="$1" cutoff="$2"
+  if [[ ! -f "$OBSERVATIONS" ]]; then
+    echo 0
+    return
+  fi
+  jq -r --arg p "$file" --arg c "$cutoff" \
+    'select(.tool=="Read" and .path==$p and .timestamp>=$c) | 1' \
+    "$OBSERVATIONS" 2>/dev/null | wc -l | tr -d ' '
+}
+
+# Scan a directory and produce a JSON array of skill objects
+scan_dir_to_json() {
+  local dir="$1"
+  local c7 c30
+  c7=$(date_ago 7)
+  c30=$(date_ago 30)
+
+  local tmpdir
+  tmpdir=$(mktemp -d)
+  # Use a function to avoid embedding $tmpdir in a quoted string (prevents injection
+  # if TMPDIR were crafted to contain shell metacharacters).
+  local _scan_tmpdir="$tmpdir"
+  _scan_cleanup() { rm -rf "$_scan_tmpdir"; }
+  trap _scan_cleanup RETURN
+
+  # Pre-aggregate observation counts in two passes (one per window) instead of
+  # calling jq per-file — reduces from O(n*m) to O(n+m) jq invocations.
+  local obs_7d_counts obs_30d_counts
+  obs_7d_counts=""
+  obs_30d_counts=""
+  if [[ -f "$OBSERVATIONS" ]]; then
+    obs_7d_counts=$(jq -r --arg c "$c7" \
+      'select(.tool=="Read" and .timestamp>=$c) | .path' \
+      "$OBSERVATIONS" 2>/dev/null | sort | uniq -c)
+    obs_30d_counts=$(jq -r --arg c "$c30" \
+      'select(.tool=="Read" and .timestamp>=$c) | .path' \
+      "$OBSERVATIONS" 2>/dev/null | sort | uniq -c)
+  fi
+
+  local i=0
+  while IFS= read -r file; do
+    local name desc mtime u7 u30 dp
+    name=$(extract_field "$file" "name")
+    desc=$(extract_field "$file" "description")
+    mtime=$(date -u -r "$file" +%Y-%m-%dT%H:%M:%SZ)
+    # Use awk exact field match to avoid substring false-positives from grep -F.
+    # uniq -c output format: "   N /path/to/file" — path is always field 2.
+    u7=$(echo "$obs_7d_counts" | awk -v f="$file" '$2 == f {print $1}' | head -1)
+    u7="${u7:-0}"
+    u30=$(echo "$obs_30d_counts" | awk -v f="$file" '$2 == f {print $1}' | head -1)
+    u30="${u30:-0}"
+    dp="${file/#$HOME/~}"
+
+    jq -n \
+      --arg path "$dp" \
+      --arg name "$name" \
+      --arg description "$desc" \
+      --arg mtime "$mtime" \
+      --argjson use_7d "$u7" \
+      --argjson use_30d "$u30" \
+      '{path:$path,name:$name,description:$description,use_7d:$use_7d,use_30d:$use_30d,mtime:$mtime}' \
+      > "$tmpdir/$i.json"
+    i=$((i+1))
+  done < <(find "$dir" -name "*.md" -type f 2>/dev/null | sort)
+
+  if [[ $i -eq 0 ]]; then
+    echo "[]"
+  else
+    jq -s '.' "$tmpdir"/*.json
+  fi
+}
+
+# --- Main ---
+
+global_found="false"
+global_count=0
+global_skills="[]"
+
+if [[ -d "$GLOBAL_DIR" ]]; then
+  global_found="true"
+  global_skills=$(scan_dir_to_json "$GLOBAL_DIR")
+  global_count=$(echo "$global_skills" | jq 'length')
+fi
+
+project_found="false"
+project_path=""
+project_count=0
+project_skills="[]"
+
+if [[ -n "$CWD_SKILLS_DIR" && -d "$CWD_SKILLS_DIR" ]]; then
+  project_found="true"
+  project_path="$CWD_SKILLS_DIR"
+  project_skills=$(scan_dir_to_json "$CWD_SKILLS_DIR")
+  project_count=$(echo "$project_skills" | jq 'length')
+fi
+
+# Merge global + project skills into one array
+all_skills=$(jq -s 'add' <(echo "$global_skills") <(echo "$project_skills"))
+
+jq -n \
+  --arg global_found "$global_found" \
+  --argjson global_count "$global_count" \
+  --arg project_found "$project_found" \
+  --arg project_path "$project_path" \
+  --argjson project_count "$project_count" \
+  --argjson skills "$all_skills" \
+  '{
+    scan_summary: {
+      global: { found: ($global_found == "true"), count: $global_count },
+      project: { found: ($project_found == "true"), path: $project_path, count: $project_count }
+    },
+    skills: $skills
+  }'