Compare commits

...

9 Commits

Author SHA1 Message Date
Levi-Evan
2707ef2eb2 fix(hooks): cost-tracker reads usage from transcript instead of stdin
The Stop hook's stdin payload does not include `usage` or `model` —
those fields exist only in the session transcript JSONL (per-assistant-turn
`message.usage` blocks). The previous implementation looked for
`input.usage.input_tokens` on stdin and silently produced zero-filled rows
(observed: 2,340 rows captured with 0.0% non-zero token rate over 52 days).

Fix: read `transcript_path` from Stop stdin, scan the JSONL, sum usage
across all assistant turns. Use `message.model` to pick a rate tier
(haiku/sonnet/opus). Track cache tokens separately (cache_creation and
cache_read have distinct billing rates).

Behavior note: Stop fires per assistant response, so each row in
costs.jsonl represents the cumulative session total at that point.
To get per-session cost: take the last row per session_id.

Verified by reading a real Stop hook payload from
.claude/projects/*/<uuid>.jsonl — payload contains
{session_id, transcript_path, cwd, hook_event_name, ...} only,
matching the published Claude Code hook spec.
2026-05-14 16:35:12 -04:00
Affaan Mustafa
4423f10cfb docs: sync ECC Tools hosted output scoring (#1891) 2026-05-13 23:02:23 -04:00
Affaan Mustafa
3b12fb273f docs: sync ECC Tools hosted promotion readiness (#1890) 2026-05-13 22:39:01 -04:00
Affaan Mustafa
4fb80d8861 Sync ECC Tools status-aware depth plan roadmap (#1887) 2026-05-13 22:12:11 -04:00
Affaan Mustafa
a27831c13e Sync ECC Tools hosted status roadmap (#1886) 2026-05-13 21:49:42 -04:00
Affaan Mustafa
b24d762caa Sync ECC Tools hosted result history roadmap (#1885) 2026-05-13 21:31:08 -04:00
Affaan Mustafa
f94478e524 docs: sync roadmap after ECC-Tools hosted dispatch 2026-05-13 20:30:48 -04:00
Affaan Mustafa
6cdac19764 docs: sync roadmap after ECC-Tools depth-plan check 2026-05-13 20:10:38 -04:00
Affaan Mustafa
af3a206412 docs: sync roadmap after ECC-Tools team backlog job (#1880) 2026-05-13 19:44:49 -04:00
3 changed files with 204 additions and 32 deletions

View File

@@ -122,6 +122,55 @@ As of 2026-05-13:
billing/entitlement, cost-regression, and cost-policy evidence, excludes billing/entitlement, cost-regression, and cost-policy evidence, excludes
obvious secret-bearing paths from fetches, returns cost-control findings and obvious secret-bearing paths from fetches, returns cost-control findings and
next actions, and charges usage only after successful execution. next actions, and charges usage only after successful execution.
- ECC-Tools PR #62 merged as `781d6733e56f7556edb43fb96bdfb00b1f0a3aa6`
and added the sixth executable hosted analysis job:
`/api/analysis/jobs/team-backlog-routing` now gates on team handoff/project
tracking readiness, evaluates roadmap, runbook, handoff, release-plan,
issue-template, ownership, project-tracker, backlog, and follow-up evidence,
excludes obvious secret-bearing paths from fetches, returns team-routing
findings and next actions, and charges usage only after successful execution.
- ECC-Tools PR #63 merged as `fb9e4c5ceb9ccde50da74c7a69c3fa4bd321fc07`
and made the hosted execution plan operator-visible on queued PR analysis:
the queue now publishes a non-blocking `ECC Tools / Hosted Depth Plan`
check-run on the PR head SHA with ready/blocked hosted executor commands
and next action text, while keeping check-run publication best-effort so
bundle generation and analysis comments are not blocked.
- ECC-Tools PR #64 merged as `72020ef94db94840812977ea7ac37e9344036668`
and added PR-facing hosted job dispatch controls:
`/ecc-tools analyze --job ...` comments now queue hosted jobs against the
PR head SHA, execute them through the existing hosted readiness/evidence
gates, post artifacts/findings/next actions back to the PR, and scope
idempotency keys by job id so hosted jobs do not collide with bundle
analysis.
- ECC-Tools PR #65 merged as `bacd4adf6a3a629e8d403865456d15f127baaf4e`
and added hosted job result history/check-run summaries:
queued hosted jobs now cache both the latest result and immutable run records
for completed or blocked runs, then publish a non-blocking per-job check-run
on the PR head SHA with artifacts, findings, readiness blockers, and next
actions.
- ECC-Tools PR #66 merged as `4e1db48252d068ea5dcf4308b0bc11b0dfe0c9ce`
and added a read-only hosted status command:
`/ecc-tools analyze --job status` now reads the #65 latest-result cache for
the current PR head and posts a compact completed/blocked/not-run table with
the next hosted job command, without queueing work or billing usage.
- ECC-Tools PR #67 merged as `f20e6bec2b0bf49e4cc36e08b7285c795973b73d`
and made the hosted depth-plan check-run status-aware:
queued PR analysis now reads the #65/#66 latest-result cache when publishing
`ECC Tools / Hosted Depth Plan`, includes the latest hosted run status in
the plan table, and recommends the next unrun ready job before reruns.
- ECC-Tools PR #68 merged as `2cde524b5ef8f34ab7bb1af973248fe4be4359f8`
and added deterministic hosted promotion readiness:
opened/synchronized PRs now publish a non-blocking
`ECC Tools / Hosted Promotion Readiness` check-run that compares changed
files against the checked-in evaluator/RAG corpus, warns on missing
hosted-job promotion evidence, and can be disabled with
`PR_HOSTED_PROMOTION_READINESS_CHECK_MODE=off`.
- ECC-Tools PR #69 merged as `d0112dac7cef807ae27def41f057682ef0772cce`
and extended hosted promotion readiness with deterministic output scoring:
the check now reads cached completed hosted job results for the current PR
head, scores their artifacts and findings against evaluator/RAG corpus
expectations, and treats matching hosted artifacts as promotion evidence
before reporting a gap.
- Handoff `ecc-supply-chain-audit-20260513-0645.md` under - Handoff `ecc-supply-chain-audit-20260513-0645.md` under
`~/.cluster-swarm/handoffs/` `~/.cluster-swarm/handoffs/`
records the May 13 supply-chain sweep: no active lockfile/manifest hit for records the May 13 supply-chain sweep: no active lockfile/manifest hit for
@@ -335,6 +384,36 @@ As of 2026-05-13:
model-routing, token-budget, usage-limit, rate-limit, billing/entitlement, model-routing, token-budget, usage-limit, rate-limit, billing/entitlement,
cost-regression, and cost-policy evidence while avoiding obvious cost-regression, and cost-policy evidence while avoiding obvious
secret-bearing path fetches. secret-bearing path fetches.
- ECC-Tools PR #62 implemented the sixth job-specific hosted executor:
`/api/analysis/jobs/team-backlog-routing` applies the same hosted gates to
roadmap, runbook, handoff, release-plan, issue-template, ownership,
project-tracker, backlog, and follow-up evidence while avoiding obvious
secret-bearing path fetches.
- ECC-Tools PR #63 publishes the hosted depth-plan check-run after queued PR
analysis completes, making the six hosted executor commands visible on the
PR head SHA without turning the check into a merge blocker.
- ECC-Tools PR #64 wires those commands into the queue: maintainers can comment
`/ecc-tools analyze --job ci-diagnostics`, `security-evidence`,
`harness-compatibility`, `reference-set-evaluation`, `ai-routing-cost`, or
`team-backlog` on a PR and receive hosted job results in a PR comment.
- ECC-Tools PR #65 persists completed and blocked hosted job results to the
analysis cache for 30 days and publishes non-blocking `ECC Tools / Hosted
Job: ...` check-runs so maintainers can scan hosted outcomes from the PR
checks surface instead of rereading older comments.
- ECC-Tools PR #66 exposes the cached results from PR comments with
`/ecc-tools analyze --job status`, summarizing completed, blocked, and
not-yet-run hosted jobs for the PR head and recommending the next hosted job
command.
- ECC-Tools PR #67 feeds those cached results back into the hosted depth-plan
check-run so queued analysis recommends the next unrun ready hosted job from
cache state instead of repeating the static readiness order.
- ECC-Tools PR #68 adds the first evaluator-backed hosted promotion gate:
opened/synchronized PRs get a non-blocking Hosted Promotion Readiness
check-run that turns the evaluator/RAG corpus into warnings when changed
files match fixture scenarios without their expected evidence artifacts.
- ECC-Tools PR #69 extends that gate to score cached completed hosted job
outputs for the current PR head, so hosted artifacts can satisfy corpus
evidence expectations before the check reports a promotion gap.
- ECC PR #1803 landed the contributor Quarkus handling branch after maintainer - ECC PR #1803 landed the contributor Quarkus handling branch after maintainer
cleanup, current-`main` alignment, full local validation, and preservation of cleanup, current-`main` alignment, full local validation, and preservation of
the author's removal of incomplete ja-JP and zh-CN Quarkus translations. the author's removal of incomplete ja-JP and zh-CN Quarkus translations.
@@ -388,10 +467,10 @@ is not complete unless the evidence column exists and has been freshly verified.
| Claude and Codex plugin publication | Contact/submission path with required artifacts and status | Publication readiness, naming matrix, and May 12 dry-run evidence document plugin validation, clean-checkout Claude tag/install smoke, and Codex marketplace CLI shape | Needs explicit approval for real tag/push and marketplace submission | | Claude and Codex plugin publication | Contact/submission path with required artifacts and status | Publication readiness, naming matrix, and May 12 dry-run evidence document plugin validation, clean-checkout Claude tag/install smoke, and Codex marketplace CLI shape | Needs explicit approval for real tag/push and marketplace submission |
| Articles, tweets, and announcements | X thread, LinkedIn copy, GitHub release copy, push checklist | Draft launch collateral exists under rc.1 release docs | Needs URL-backed refresh | | Articles, tweets, and announcements | X thread, LinkedIn copy, GitHub release copy, push checklist | Draft launch collateral exists under rc.1 release docs | Needs URL-backed refresh |
| AgentShield enterprise iteration | Policy gates, SARIF, packs, provenance, corpus, HTML reports, exception lifecycle audit, baseline drift Action/CLI surfaces, evidence-pack redaction, harness adapter registry, enterprise research roadmap, supply-chain hardened release path, CI-safe baseline fingerprints, corpus accuracy recommendations, remediation workflow phases, env proxy hijack corpus coverage | PRs #53, #55-#64, #67-#69, and #78-#82 landed with test evidence; native PDF export deferred in favor of self-contained HTML plus print-to-PDF until explicit enterprise demand appears; `docs/architecture/agentshield-enterprise-research-roadmap.md` now has baseline drift, evidence-pack bundle, redaction, adapter-registry, supply-chain hardening, hashed baseline fingerprints, corpus accuracy recommendation, remediation workflow, and env proxy hijack corpus slices landed | Next hosted evidence-pack workflow depth | | AgentShield enterprise iteration | Policy gates, SARIF, packs, provenance, corpus, HTML reports, exception lifecycle audit, baseline drift Action/CLI surfaces, evidence-pack redaction, harness adapter registry, enterprise research roadmap, supply-chain hardened release path, CI-safe baseline fingerprints, corpus accuracy recommendations, remediation workflow phases, env proxy hijack corpus coverage | PRs #53, #55-#64, #67-#69, and #78-#82 landed with test evidence; native PDF export deferred in favor of self-contained HTML plus print-to-PDF until explicit enterprise demand appears; `docs/architecture/agentshield-enterprise-research-roadmap.md` now has baseline drift, evidence-pack bundle, redaction, adapter-registry, supply-chain hardening, hashed baseline fingerprints, corpus accuracy recommendation, remediation workflow, and env proxy hijack corpus slices landed | Next hosted evidence-pack workflow depth |
| ECC Tools next-level app | Billing audit, PR checks, deep analyzer, sync backlog, evaluator/RAG corpus, analysis-depth readiness, hosted execution planning, hosted CI diagnostics, hosted security evidence review, hosted harness compatibility audit, hosted reference-set evaluation, hosted AI routing/cost review | PRs #26-#43 plus #53-#61 landed with test evidence, including AgentShield evidence-pack gap routing, canonical bundle recognition, supply-chain signature gates, PR draft follow-up Linear tracking, evidence-backed/deep-ready repository classification, the `/api/analysis/depth-plan` hosted job plan, `/api/analysis/jobs/ci-diagnostics`, `/api/analysis/jobs/security-evidence-review`, `/api/analysis/jobs/harness-compatibility-audit`, `/api/analysis/jobs/reference-set-evaluation`, and `/api/analysis/jobs/ai-routing-cost-review` | Needs the remaining hosted worker executor for team backlog routing | | ECC Tools next-level app | Billing audit, PR checks, deep analyzer, sync backlog, evaluator/RAG corpus, analysis-depth readiness, hosted execution planning, hosted CI diagnostics, hosted security evidence review, hosted harness compatibility audit, hosted reference-set evaluation, hosted AI routing/cost review, hosted team backlog routing, hosted depth-plan check-run, PR-comment hosted job dispatch, hosted job result history/check-runs, hosted result status command, status-aware depth-plan recommendations, hosted promotion readiness, hosted promotion output scoring | PRs #26-#43 plus #53-#69 landed with test evidence, including AgentShield evidence-pack gap routing, canonical bundle recognition, supply-chain signature gates, PR draft follow-up Linear tracking, evidence-backed/deep-ready repository classification, the `/api/analysis/depth-plan` hosted job plan, `/api/analysis/jobs/ci-diagnostics`, `/api/analysis/jobs/security-evidence-review`, `/api/analysis/jobs/harness-compatibility-audit`, `/api/analysis/jobs/reference-set-evaluation`, `/api/analysis/jobs/ai-routing-cost-review`, `/api/analysis/jobs/team-backlog-routing`, the `ECC Tools / Hosted Depth Plan` check-run, `/ecc-tools analyze --job ...` PR-comment dispatch, non-blocking per-hosted-job result check-runs backed by 30-day result cache records, `/ecc-tools analyze --job status` cache lookup, cache-aware next-job recommendations in the depth-plan check-run, the `ECC Tools / Hosted Promotion Readiness` corpus-backed PR check-run, and deterministic hosted-output scoring against cached completed job artifacts/findings | Next work is retrieval/model-backed hosted promotion after deterministic output scoring |
| GitGuardian/Dependabot/CodeRabbit-style checks | Non-blocking taxonomy, deterministic follow-up checks, and local supply-chain gates | ECC-Tools risk taxonomy check plus follow-up signals landed, including Skill Quality, Deep Analyzer Evidence, Analyzer Corpus Evidence, RAG/Evaluator Evidence, PR Review/Salvage Evidence, and AgentShield evidence-pack evidence; #1846 added npm registry signature gates; #1848 added the supply-chain incident-response playbook and `pull_request_target` cache-poisoning validator guard; #1851 added the privileged checkout credential-persistence guard; AgentShield #78, JARVIS #13, and ECC-Tools #53 applied the same hardening outside trunk | Current supply-chain gate complete; deeper hosted review features remain future | | GitGuardian/Dependabot/CodeRabbit-style checks | Non-blocking taxonomy, deterministic follow-up checks, and local supply-chain gates | ECC-Tools risk taxonomy check plus follow-up signals landed, including Skill Quality, Deep Analyzer Evidence, Analyzer Corpus Evidence, RAG/Evaluator Evidence, PR Review/Salvage Evidence, and AgentShield evidence-pack evidence; #1846 added npm registry signature gates; #1848 added the supply-chain incident-response playbook and `pull_request_target` cache-poisoning validator guard; #1851 added the privileged checkout credential-persistence guard; AgentShield #78, JARVIS #13, and ECC-Tools #53 applied the same hardening outside trunk | Current supply-chain gate complete; deeper hosted review features remain future |
| Harness-agnostic learning system | Audit, adapter matrix, observability, traces, promotion loop | Audit/adapters/observability gates plus `docs/architecture/evaluator-rag-prototype.md`, `examples/evaluator-rag-prototype/`, and ECC-Tools PR #40 define read-only stale-salvage, billing-readiness, CI-failure-diagnosis, harness-config-quality, AgentShield policy-exception, skill-quality evidence, deep-analyzer evidence, and RAG/evaluator comparison scenarios with trace, report, playbook, verifier, and predictive-check artifacts | Local corpus complete; hosted integration remains future | | Harness-agnostic learning system | Audit, adapter matrix, observability, traces, promotion loop | Audit/adapters/observability gates plus `docs/architecture/evaluator-rag-prototype.md`, `examples/evaluator-rag-prototype/`, and ECC-Tools PR #40 define read-only stale-salvage, billing-readiness, CI-failure-diagnosis, harness-config-quality, AgentShield policy-exception, skill-quality evidence, deep-analyzer evidence, and RAG/evaluator comparison scenarios with trace, report, playbook, verifier, and predictive-check artifacts; ECC-Tools PRs #68/#69 now turn that corpus into a deterministic PR check-run gate with cached hosted-output scoring | Deterministic hosted PR check and cached output scoring integrated; hosted retrieval remains future |
| Linear roadmap is detailed | Linear project status plus repo mirror | Repo mirror exists; issue creation was retried on 2026-05-12 and remains blocked by the workspace free issue limit; this May 13 sync adds ECC #1860, AgentShield #78-#82, JARVIS #13, ECC-Tools #53-#61, resolved queue/discussion counts, and Linear project status updates through ECC-Tools #61 | Needs recurring status updates after each merge batch | | Linear roadmap is detailed | Linear project status plus repo mirror | Repo mirror exists; issue creation was retried on 2026-05-12 and remains blocked by the workspace free issue limit; this May 13 sync adds ECC #1860, AgentShield #78-#82, JARVIS #13, ECC-Tools #53-#69, resolved queue/discussion counts, and Linear project status updates through ECC-Tools #69 | Needs recurring status updates after each merge batch |
| Flow separation and progress tracking | Flow lanes with owner artifacts and update cadence | This roadmap defines lanes below and `docs/architecture/progress-sync-contract.md` makes GitHub/Linear/handoff/roadmap sync part of the readiness gate | Active | | Flow separation and progress tracking | Flow lanes with owner artifacts and update cadence | This roadmap defines lanes below and `docs/architecture/progress-sync-contract.md` makes GitHub/Linear/handoff/roadmap sync part of the readiness gate | Active |
| Realtime Linear sync | Project updates while issue limit is blocked; issues later | ECC-Tools #39 implements opt-in Linear API sync for deferred follow-up backlog items, and ECC-Tools #54 adds copy-ready PR drafts to that backlog when draft PR shells are not opened; `docs/architecture/progress-sync-contract.md` defines the local file-backed realtime boundary while issue capacity is blocked | Needs workspace capacity/config rollout | | Realtime Linear sync | Project updates while issue limit is blocked; issues later | ECC-Tools #39 implements opt-in Linear API sync for deferred follow-up backlog items, and ECC-Tools #54 adds copy-ready PR drafts to that backlog when draft PR shells are not opened; `docs/architecture/progress-sync-contract.md` defines the local file-backed realtime boundary while issue capacity is blocked | Needs workspace capacity/config rollout |
| Observability for self-use | Local readiness gate, traces, status snapshots, HUD/status contract, risk ledger, progress-sync contract | `npm run observability:ready` reports 21/21 | Complete for local gate | | Observability for self-use | Local readiness gate, traces, status snapshots, HUD/status contract, risk ledger, progress-sync contract | `npm run observability:ready` reports 21/21 | Complete for local gate |
@@ -410,9 +489,9 @@ repo evidence and merge commits.
| Queue hygiene and salvage | GitHub PR/issue state, salvage ledger | Append ledger entries for any future stale closures | Every cleanup batch | | Queue hygiene and salvage | GitHub PR/issue state, salvage ledger | Append ledger entries for any future stale closures | Every cleanup batch |
| Release and publication | rc.1 release docs, publication readiness doc | Naming matrix and plugin submission/contact checklist | Before any tag | | Release and publication | rc.1 release docs, publication readiness doc | Naming matrix and plugin submission/contact checklist | Before any tag |
| Harness OS core | Audit, adapter matrix, observability docs, `ecc2/` | HUD/session-control acceptance spec | Weekly until GA | | Harness OS core | Audit, adapter matrix, observability docs, `ecc2/` | HUD/session-control acceptance spec | Weekly until GA |
| Evaluation and RAG | Reference-set validation, harness audit, traces, ECC-Tools corpus | Read-only evaluator/RAG prototype plus stale-salvage, billing-readiness, CI-failure-diagnosis, harness-config-quality, AgentShield policy-exception, skill-quality evidence, deep-analyzer evidence, and RAG/evaluator comparison fixtures | Hosted retrieval/check-run automation plan | | Evaluation and RAG | Reference-set validation, harness audit, traces, ECC-Tools corpus | Read-only evaluator/RAG prototype plus stale-salvage, billing-readiness, CI-failure-diagnosis, harness-config-quality, AgentShield policy-exception, skill-quality evidence, deep-analyzer evidence, and RAG/evaluator comparison fixtures; ECC-Tools #68 publishes the corpus as a hosted promotion readiness check-run, and #69 scores cached hosted job outputs against the same corpus | Hosted retrieval/model-backed promotion plan |
| AgentShield enterprise | AgentShield PR evidence and roadmap notes | Remediation workflow depth or corpus expansion follow-up | Next implementation batch | | AgentShield enterprise | AgentShield PR evidence and roadmap notes | Remediation workflow depth or corpus expansion follow-up | Next implementation batch |
| ECC Tools app | ECC-Tools PR evidence, billing audit, risk taxonomy, evaluator/RAG corpus | ECC-Tools #53 published the supply-chain workflow hardening branch, #54 tracks copy-ready PR drafts in the Linear/project backlog, #55 classifies analysis-depth readiness, #56 exposes the hosted execution plan, #57 executes the first hosted CI diagnostics job, #58 executes the hosted security evidence review job, #59 executes the hosted harness compatibility audit, #60 executes the hosted reference-set evaluation, and #61 executes the hosted AI routing/cost review; next work is the team backlog routing executor | Next implementation batch | | ECC Tools app | ECC-Tools PR evidence, billing audit, risk taxonomy, evaluator/RAG corpus | ECC-Tools #53 published the supply-chain workflow hardening branch, #54 tracks copy-ready PR drafts in the Linear/project backlog, #55 classifies analysis-depth readiness, #56 exposes the hosted execution plan, #57 executes the first hosted CI diagnostics job, #58 executes the hosted security evidence review job, #59 executes the hosted harness compatibility audit, #60 executes the hosted reference-set evaluation, #61 executes the hosted AI routing/cost review, #62 executes hosted team backlog routing, #63 publishes the hosted depth-plan check-run, #64 dispatches hosted jobs from PR comments, #65 persists hosted result history/check-runs, #66 exposes hosted job status from PR comments, #67 makes depth-plan recommendations cache-aware, #68 publishes hosted promotion readiness from the evaluator/RAG corpus, and #69 scores cached hosted job outputs against that corpus; next work is retrieval/model-backed hosted promotion | Next implementation batch |
| Linear progress | Linear project status updates, `docs/architecture/progress-sync-contract.md`, and this mirror | Status update with queue/evidence/missing gates | Every significant merge batch | | Linear progress | Linear project status updates, `docs/architecture/progress-sync-contract.md`, and this mirror | Status update with queue/evidence/missing gates | Every significant merge batch |
The project status update should always include: The project status update should always include:
@@ -629,10 +708,9 @@ Acceptance:
PR #82 expanded corpus coverage for env proxy hijacks and out-of-band PR #82 expanded corpus coverage for env proxy hijacks and out-of-band
exfiltration; and ECC-Tools PRs #42/#43 now route and recognize evidence exfiltration; and ECC-Tools PRs #42/#43 now route and recognize evidence
packs. The next slice is hosted evidence-pack workflow depth. packs. The next slice is hosted evidence-pack workflow depth.
2. Extend the ECC-Tools hosted execution lane beyond #57's CI diagnostics, 2. Plan retrieval/model-backed hosted promotion on top of the #69 deterministic
#58's security evidence review, #59's harness compatibility audit, and hosted output scoring contract, keeping vector/model judgment behind fixture
#60's reference-set evaluation, and #61's AI routing/cost review into the evaluation until the retrieval contract is stable.
remaining depth-plan job: team backlog routing.
3. Enable/configure the merged Linear backlog sync path after workspace issue 3. Enable/configure the merged Linear backlog sync path after workspace issue
capacity clears or the Linear workspace is upgraded, then verify PR-draft capacity clears or the Linear workspace is upgraded, then verify PR-draft
salvage items land in the expected project. salvage items land in the expected project.

View File

@@ -1,63 +1,157 @@
#!/usr/bin/env node #!/usr/bin/env node
/** /**
* Cost Tracker Hook * Cost Tracker Hook (v2)
* *
* Appends lightweight session usage metrics to ~/.claude/metrics/costs.jsonl. * Reads transcript_path from Stop hook stdin, sums usage across all
* assistant turns in the session JSONL, and appends one row to
* ~/.claude/metrics/costs.jsonl.
*
* Stop hook stdin payload: { session_id, transcript_path, cwd, hook_event_name, ... }
* The Stop payload does NOT include `usage` or `model` directly. The previous
* version of this hook expected those fields and silently produced zero-filled
* rows (verified: 2,340 rows captured with 0.0% non-zero token rate over 52
* days). The fix is to read the transcript file Claude Code already passes us.
*
* JSONL assistant entry shape (per Claude Code):
* { type: "assistant", message: { model, usage: { input_tokens, output_tokens,
* cache_creation_input_tokens, cache_read_input_tokens } } }
*
* Cumulative behavior: Stop fires per assistant response, not per session.
* Each row therefore represents the cumulative session total up to that point.
* To get per-session cost, take the last row per session_id. To get per-day
* spend, aggregate.
*/ */
'use strict'; 'use strict';
const fs = require('fs');
const path = require('path'); const path = require('path');
const { ensureDir, appendFile, getClaudeDir } = require('../lib/utils'); const { ensureDir, appendFile, getClaudeDir } = require('../lib/utils');
const { estimateCost } = require('../lib/cost-estimate');
const { sanitizeSessionId } = require('../lib/session-bridge'); const { sanitizeSessionId } = require('../lib/session-bridge');
const MAX_STDIN = 1024 * 1024; // Approximate per-1M-token billing rates (USD).
let raw = ''; // Cache creation: 1.25x input rate. Cache read: 0.1x input rate.
const RATE_TABLE = {
haiku: { in: 0.80, out: 4.0, cacheWrite: 1.00, cacheRead: 0.08 },
sonnet: { in: 3.00, out: 15.0, cacheWrite: 3.75, cacheRead: 0.30 },
opus: { in: 15.00, out: 75.0, cacheWrite: 18.75, cacheRead: 1.50 }
};
function toNumber(value) { function getRates(model) {
const n = Number(value); const m = String(model || '').toLowerCase();
if (m.includes('haiku')) return RATE_TABLE.haiku;
if (m.includes('opus')) return RATE_TABLE.opus;
return RATE_TABLE.sonnet;
}
function toNumber(v) {
const n = Number(v);
return Number.isFinite(n) ? n : 0; return Number.isFinite(n) ? n : 0;
} }
/**
* Scan the session JSONL and sum token usage across all assistant turns.
* Returns { inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens, model }
* or null on read failure.
*/
function sumUsageFromTranscript(transcriptPath) {
let content;
try {
content = fs.readFileSync(transcriptPath, 'utf8');
} catch {
return null;
}
let inputTokens = 0;
let outputTokens = 0;
let cacheWriteTokens = 0;
let cacheReadTokens = 0;
let model = 'unknown';
for (const line of content.split('\n')) {
if (!line.trim()) continue;
let entry;
try { entry = JSON.parse(line); } catch { continue; }
if (entry.type !== 'assistant') continue;
const msg = entry.message;
if (!msg || !msg.usage) continue;
const u = msg.usage;
inputTokens += toNumber(u.input_tokens);
outputTokens += toNumber(u.output_tokens);
cacheWriteTokens += toNumber(u.cache_creation_input_tokens);
cacheReadTokens += toNumber(u.cache_read_input_tokens);
if (msg.model && msg.model !== 'unknown') model = msg.model;
}
return { inputTokens, outputTokens, cacheWriteTokens, cacheReadTokens, model };
}
const MAX_STDIN = 64 * 1024;
let raw = '';
process.stdin.setEncoding('utf8'); process.stdin.setEncoding('utf8');
process.stdin.on('data', chunk => { process.stdin.on('data', chunk => {
if (raw.length < MAX_STDIN) { if (raw.length < MAX_STDIN) raw += chunk.substring(0, MAX_STDIN - raw.length);
const remaining = MAX_STDIN - raw.length;
raw += chunk.substring(0, remaining);
}
}); });
process.stdin.on('end', () => { process.stdin.on('end', () => {
try { try {
const input = raw.trim() ? JSON.parse(raw) : {}; const input = raw.trim() ? JSON.parse(raw) : {};
const usage = input.usage || input.token_usage || {};
const inputTokens = toNumber(usage.input_tokens || usage.prompt_tokens || 0);
const outputTokens = toNumber(usage.output_tokens || usage.completion_tokens || 0);
const model = String(input.model || input._cursor?.model || process.env.CLAUDE_MODEL || 'unknown'); const transcriptPath = (typeof input.transcript_path === 'string' && input.transcript_path)
? input.transcript_path
: process.env.CLAUDE_TRANSCRIPT_PATH || null;
const sessionId = const sessionId =
sanitizeSessionId(input.session_id) || sanitizeSessionId(input.session_id) ||
sanitizeSessionId(process.env.ECC_SESSION_ID) || sanitizeSessionId(process.env.ECC_SESSION_ID) ||
sanitizeSessionId(process.env.CLAUDE_SESSION_ID) || sanitizeSessionId(process.env.CLAUDE_SESSION_ID) ||
'default'; 'default';
let usageTotals = null;
if (transcriptPath && fs.existsSync(transcriptPath)) {
usageTotals = sumUsageFromTranscript(transcriptPath);
}
const {
inputTokens = 0,
outputTokens = 0,
cacheWriteTokens = 0,
cacheReadTokens = 0,
model = 'unknown'
} = usageTotals || {};
const rates = getRates(model);
const estimatedCostUsd = Math.round((
(inputTokens / 1e6) * rates.in +
(outputTokens / 1e6) * rates.out +
(cacheWriteTokens / 1e6) * rates.cacheWrite +
(cacheReadTokens / 1e6) * rates.cacheRead
) * 1e6) / 1e6;
const metricsDir = path.join(getClaudeDir(), 'metrics'); const metricsDir = path.join(getClaudeDir(), 'metrics');
ensureDir(metricsDir); ensureDir(metricsDir);
const row = { const row = {
timestamp: new Date().toISOString(), timestamp: new Date().toISOString(),
session_id: sessionId, session_id: sessionId,
transcript_path: transcriptPath || '',
model, model,
input_tokens: inputTokens, input_tokens: inputTokens,
output_tokens: outputTokens, output_tokens: outputTokens,
estimated_cost_usd: estimateCost(model, inputTokens, outputTokens) cache_write_tokens: cacheWriteTokens,
cache_read_tokens: cacheReadTokens,
estimated_cost_usd: estimatedCostUsd
}; };
appendFile(path.join(metricsDir, 'costs.jsonl'), `${JSON.stringify(row)}\n`); appendFile(path.join(metricsDir, 'costs.jsonl'), `${JSON.stringify(row)}\n`);
} catch { } catch {
// Keep hook non-blocking. // Non-blocking — never fail the Stop hook.
} }
// Pass stdin through (required by ECC hook convention).
process.stdout.write(raw); process.stdout.write(raw);
}); });

View File

@@ -130,12 +130,12 @@ test('candidate playbook preserves stale-salvage operating rules', () => {
} }
}); });
test('roadmap points to the evaluator RAG prototype and keeps hosted integration open', () => { test('roadmap points to the evaluator RAG prototype and hosted PR check', () => {
const roadmap = read('docs/ECC-2.0-GA-ROADMAP.md'); const roadmap = read('docs/ECC-2.0-GA-ROADMAP.md');
assert.ok(roadmap.includes('docs/architecture/evaluator-rag-prototype.md')); assert.ok(roadmap.includes('docs/architecture/evaluator-rag-prototype.md'));
assert.ok(roadmap.includes('examples/evaluator-rag-prototype/')); assert.ok(roadmap.includes('examples/evaluator-rag-prototype/'));
assert.ok(roadmap.includes('Local corpus complete; hosted integration remains future')); assert.ok(roadmap.includes('Deterministic hosted PR check and cached output scoring integrated; hosted retrieval remains future'));
}); });
test('billing readiness scenario rejects launch copy overclaims', () => { test('billing readiness scenario rejects launch copy overclaims', () => {