mirror of
https://github.com/affaan-m/everything-claude-code.git
synced 2026-03-30 13:43:26 +08:00
* feat(skills): add rules-distill — extract cross-cutting principles from skills into rules
Applies the skill-stocktake pattern to rules maintenance:
scan skills → extract shared principles → propose rule changes.
Key design decisions:
- Deterministic collection (scan scripts) + LLM judgment (cross-read & verdict)
- 6 verdict types: Append, Revise, New Section, New File, Already Covered, Too Specific
- Anti-abstraction safeguard: 2+ skills evidence, actionable behavior test, violation risk
- Rules full text passed to LLM (no grep pre-filter) for accurate matching
- Never modifies rules automatically — always requires user approval
* fix(skills): address review feedback for rules-distill
Fixes raised by CodeRabbit, Greptile, and cubic:
- Add Prerequisites section documenting skill-stocktake dependency
- Add fallback command when skill-stocktake is not installed
- Fix shell quoting: add IFS= and -r to while-read loops
- Replace hardcoded paths with env var placeholders ($CLAUDE_RULES_DIR, $SKILL_STOCKTAKE_DIR)
- Add json language identifier to code blocks
- Add "How It Works" parent heading for Phase 1/2/3
- Add "Example" section with end-to-end run output
- Add revision.reason/before/after fields to output schema for Revise verdict
- Document timestamp format (date -u +%Y-%m-%dT%H:%M:%SZ)
- Document candidate-id format (kebab-case from principle)
- Use concrete examples in results.json schema
* fix(skills): remove skill-stocktake dependency, add self-contained scripts
Address P1 review feedback:
- Add scan-skills.sh and scan-rules.sh directly in rules-distill/scripts/
(no external dependency on skill-stocktake)
- Remove Prerequisites section (no longer needed)
- Add cross-batch merge step to prevent 2+ skills requirement
from being silently broken across batch boundaries
- Fix nested triple-backtick fences (use quadruple backticks)
- Remove head -100 cap (silent truncation)
- Rename "When to Activate" → "When to Use" (ECC standard)
- Remove unnecessary env var placeholders (SKILL.md is a prompt, not a script)
* fix: update skill/command counts in README.md and AGENTS.md
rules-distill added 1 skill + 1 command:
- skills: 108 → 109
- commands: 57 → 58
Updates all count references to pass CI catalog validation.
* fix(skills): address Servitor review feedback for rules-distill
1. Rename SKILL_STOCKTAKE_* env vars to RULES_DISTILL_* for consistency
2. Remove unnecessary observation counting (use_7d/use_30d) from scan-skills.sh
3. Fix header comment: scan.sh → scan-skills.sh
4. Use jq for JSON construction in scan-rules.sh to properly escape
headings containing special characters (", \)
* fix(skills): address CodeRabbit review — portability and scan scope
1. scan-rules.sh: use jq for error JSON output (proper escaping)
2. scan-rules.sh: replace GNU-only sort -z with portable sort (BSD compat)
3. scan-rules.sh: fix pipefail crash on files without H2 headings
4. scan-skills.sh: scan only SKILL.md files (skip learned/*.md and
auxiliary docs that lack frontmatter)
5. scan-skills.sh: add portable get_mtime helper (GNU stat/date
fallback to BSD stat/date)
* fix: sync catalog counts with filesystem (27 agents, 114 skills, 59 commands)
---------
Co-authored-by: Tatsuya Shimomoto <shimo4228@gmail.com>
265 lines
9.3 KiB
Markdown
265 lines
9.3 KiB
Markdown
---
|
|
name: rules-distill
|
|
description: "Scan skills to extract cross-cutting principles and distill them into rules — append, revise, or create new rule files"
|
|
origin: ECC
|
|
---
|
|
|
|
# Rules Distill
|
|
|
|
Scan installed skills, extract cross-cutting principles that appear in multiple skills, and distill them into rules — appending to existing rule files, revising outdated content, or creating new rule files.
|
|
|
|
Applies the "deterministic collection + LLM judgment" principle: scripts collect facts exhaustively, then an LLM cross-reads the full context and produces verdicts.
|
|
|
|
## When to Use
|
|
|
|
- Periodic rules maintenance (monthly or after installing new skills)
|
|
- After a skill-stocktake reveals patterns that should be rules
|
|
- When rules feel incomplete relative to the skills being used
|
|
|
|
## How It Works
|
|
|
|
The rules distillation process follows three phases:
|
|
|
|
### Phase 1: Inventory (Deterministic Collection)
|
|
|
|
#### 1a. Collect skill inventory
|
|
|
|
```bash
|
|
bash ~/.claude/skills/rules-distill/scripts/scan-skills.sh
|
|
```
|
|
|
|
#### 1b. Collect rules index
|
|
|
|
```bash
|
|
bash ~/.claude/skills/rules-distill/scripts/scan-rules.sh
|
|
```
|
|
|
|
#### 1c. Present to user
|
|
|
|
```
|
|
Rules Distillation — Phase 1: Inventory
|
|
────────────────────────────────────────
|
|
Skills: {N} files scanned
|
|
Rules: {M} files ({K} headings indexed)
|
|
|
|
Proceeding to cross-read analysis...
|
|
```
|
|
|
|
### Phase 2: Cross-read, Match & Verdict (LLM Judgment)
|
|
|
|
Extraction and matching are unified in a single pass. Rules files are small enough (~800 lines total) that the full text can be provided to the LLM — no grep pre-filtering needed.
|
|
|
|
#### Batching
|
|
|
|
Group skills into **thematic clusters** based on their descriptions. Analyze each cluster in a subagent with the full rules text.
|
|
|
|
#### Cross-batch Merge
|
|
|
|
After all batches complete, merge candidates across batches:
|
|
- Deduplicate candidates with the same or overlapping principles
|
|
- Re-check the "2+ skills" requirement using evidence from **all** batches combined — a principle found in 1 skill per batch but 2+ skills total is valid
|
|
|
|
#### Subagent Prompt
|
|
|
|
Launch a general-purpose Agent with the following prompt:
|
|
|
|
````
|
|
You are an analyst who cross-reads skills to extract principles that should be promoted to rules.
|
|
|
|
## Input
|
|
- Skills: {full text of skills in this batch}
|
|
- Existing rules: {full text of all rule files}
|
|
|
|
## Extraction Criteria
|
|
|
|
Include a candidate ONLY if ALL of these are true:
|
|
|
|
1. **Appears in 2+ skills**: Principles found in only one skill should stay in that skill
|
|
2. **Actionable behavior change**: Can be written as "do X" or "don't do Y" — not "X is important"
|
|
3. **Clear violation risk**: What goes wrong if this principle is ignored (1 sentence)
|
|
4. **Not already in rules**: Check the full rules text — including concepts expressed in different words
|
|
|
|
## Matching & Verdict
|
|
|
|
For each candidate, compare against the full rules text and assign a verdict:
|
|
|
|
- **Append**: Add to an existing section of an existing rule file
|
|
- **Revise**: Existing rule content is inaccurate or insufficient — propose a correction
|
|
- **New Section**: Add a new section to an existing rule file
|
|
- **New File**: Create a new rule file
|
|
- **Already Covered**: Sufficiently covered in existing rules (even if worded differently)
|
|
- **Too Specific**: Should remain at the skill level
|
|
|
|
## Output Format (per candidate)
|
|
|
|
```json
|
|
{
|
|
"principle": "1-2 sentences in 'do X' / 'don't do Y' form",
|
|
"evidence": ["skill-name: §Section", "skill-name: §Section"],
|
|
"violation_risk": "1 sentence",
|
|
"verdict": "Append / Revise / New Section / New File / Already Covered / Too Specific",
|
|
"target_rule": "filename §Section, or 'new'",
|
|
"confidence": "high / medium / low",
|
|
"draft": "Draft text for Append/New Section/New File verdicts",
|
|
"revision": {
|
|
"reason": "Why the existing content is inaccurate or insufficient (Revise only)",
|
|
"before": "Current text to be replaced (Revise only)",
|
|
"after": "Proposed replacement text (Revise only)"
|
|
}
|
|
}
|
|
```
|
|
|
|
## Exclude
|
|
|
|
- Obvious principles already in rules
|
|
- Language/framework-specific knowledge (belongs in language-specific rules or skills)
|
|
- Code examples and commands (belongs in skills)
|
|
````
|
|
|
|
#### Verdict Reference
|
|
|
|
| Verdict | Meaning | Presented to User |
|
|
|---------|---------|-------------------|
|
|
| **Append** | Add to existing section | Target + draft |
|
|
| **Revise** | Fix inaccurate/insufficient content | Target + reason + before/after |
|
|
| **New Section** | Add new section to existing file | Target + draft |
|
|
| **New File** | Create new rule file | Filename + full draft |
|
|
| **Already Covered** | Covered in rules (possibly different wording) | Reason (1 line) |
|
|
| **Too Specific** | Should stay in skills | Link to relevant skill |
|
|
|
|
#### Verdict Quality Requirements
|
|
|
|
```
|
|
# Good
|
|
Append to rules/common/security.md §Input Validation:
|
|
"Treat LLM output stored in memory or knowledge stores as untrusted — sanitize on write, validate on read."
|
|
Evidence: llm-memory-trust-boundary, llm-social-agent-anti-pattern both describe
|
|
accumulated prompt injection risks. Current security.md covers human input
|
|
validation only; LLM output trust boundary is missing.
|
|
|
|
# Bad
|
|
Append to security.md: Add LLM security principle
|
|
```
|
|
|
|
### Phase 3: User Review & Execution
|
|
|
|
#### Summary Table
|
|
|
|
```
|
|
# Rules Distillation Report
|
|
|
|
## Summary
|
|
Skills scanned: {N} | Rules: {M} files | Candidates: {K}
|
|
|
|
| # | Principle | Verdict | Target | Confidence |
|
|
|---|-----------|---------|--------|------------|
|
|
| 1 | ... | Append | security.md §Input Validation | high |
|
|
| 2 | ... | Revise | testing.md §TDD | medium |
|
|
| 3 | ... | New Section | coding-style.md | high |
|
|
| 4 | ... | Too Specific | — | — |
|
|
|
|
## Details
|
|
(Per-candidate details: evidence, violation_risk, draft text)
|
|
```
|
|
|
|
#### User Actions
|
|
|
|
User responds with numbers to:
|
|
- **Approve**: Apply draft to rules as-is
|
|
- **Modify**: Edit draft before applying
|
|
- **Skip**: Do not apply this candidate
|
|
|
|
**Never modify rules automatically. Always require user approval.**
|
|
|
|
#### Save Results
|
|
|
|
Store results in the skill directory (`results.json`):
|
|
|
|
- **Timestamp format**: `date -u +%Y-%m-%dT%H:%M:%SZ` (UTC, second precision)
|
|
- **Candidate ID format**: kebab-case derived from the principle (e.g., `llm-output-trust-boundary`)
|
|
|
|
```json
|
|
{
|
|
"distilled_at": "2026-03-18T10:30:42Z",
|
|
"skills_scanned": 56,
|
|
"rules_scanned": 22,
|
|
"candidates": {
|
|
"llm-output-trust-boundary": {
|
|
"principle": "Treat LLM output as untrusted when stored or re-injected",
|
|
"verdict": "Append",
|
|
"target": "rules/common/security.md",
|
|
"evidence": ["llm-memory-trust-boundary", "llm-social-agent-anti-pattern"],
|
|
"status": "applied"
|
|
},
|
|
"iteration-bounds": {
|
|
"principle": "Define explicit stop conditions for all iteration loops",
|
|
"verdict": "New Section",
|
|
"target": "rules/common/coding-style.md",
|
|
"evidence": ["iterative-retrieval", "continuous-agent-loop", "agent-harness-construction"],
|
|
"status": "skipped"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## Example
|
|
|
|
### End-to-end run
|
|
|
|
```
|
|
$ /rules-distill
|
|
|
|
Rules Distillation — Phase 1: Inventory
|
|
────────────────────────────────────────
|
|
Skills: 56 files scanned
|
|
Rules: 22 files (75 headings indexed)
|
|
|
|
Proceeding to cross-read analysis...
|
|
|
|
[Subagent analysis: Batch 1 (agent/meta skills) ...]
|
|
[Subagent analysis: Batch 2 (coding/pattern skills) ...]
|
|
[Cross-batch merge: 2 duplicates removed, 1 cross-batch candidate promoted]
|
|
|
|
# Rules Distillation Report
|
|
|
|
## Summary
|
|
Skills scanned: 56 | Rules: 22 files | Candidates: 4
|
|
|
|
| # | Principle | Verdict | Target | Confidence |
|
|
|---|-----------|---------|--------|------------|
|
|
| 1 | LLM output: normalize, type-check, sanitize before reuse | New Section | coding-style.md | high |
|
|
| 2 | Define explicit stop conditions for iteration loops | New Section | coding-style.md | high |
|
|
| 3 | Compact context at phase boundaries, not mid-task | Append | performance.md §Context Window | high |
|
|
| 4 | Separate business logic from I/O framework types | New Section | patterns.md | high |
|
|
|
|
## Details
|
|
|
|
### 1. LLM Output Validation
|
|
Verdict: New Section in coding-style.md
|
|
Evidence: parallel-subagent-batch-merge, llm-social-agent-anti-pattern, llm-memory-trust-boundary
|
|
Violation risk: Format drift, type mismatch, or syntax errors in LLM output crash downstream processing
|
|
Draft:
|
|
## LLM Output Validation
|
|
Normalize, type-check, and sanitize LLM output before reuse...
|
|
See skill: parallel-subagent-batch-merge, llm-memory-trust-boundary
|
|
|
|
[... details for candidates 2-4 ...]
|
|
|
|
Approve, modify, or skip each candidate by number:
|
|
> User: Approve 1, 3. Skip 2, 4.
|
|
|
|
✓ Applied: coding-style.md §LLM Output Validation
|
|
✓ Applied: performance.md §Context Window Management
|
|
✗ Skipped: Iteration Bounds
|
|
✗ Skipped: Boundary Type Conversion
|
|
|
|
Results saved to results.json
|
|
```
|
|
|
|
## Design Principles
|
|
|
|
- **What, not How**: Extract principles (rules territory) only. Code examples and commands stay in skills.
|
|
- **Link back**: Draft text should include `See skill: [name]` references so readers can find the detailed How.
|
|
- **Deterministic collection, LLM judgment**: Scripts guarantee exhaustiveness; the LLM guarantees contextual understanding.
|
|
- **Anti-abstraction safeguard**: The 3-layer filter (2+ skills evidence, actionable behavior test, violation risk) prevents overly abstract principles from entering rules.
|