address review: remove .cursor/ duplicate, use is not None checks

Changes based on CodeRabbit review feedback:

1. Remove entire .cursor/ directory — it was an identical copy of the
   main skills/commands/agents/rules, causing maintenance drift.
   Users of Cursor can reference the canonical files directly.

2. Use explicit `is not None` checks instead of truthiness for
   parsed['input'] and parsed['output']. Empty strings or empty
   dicts are valid values that should be preserved.
This commit is contained in:
tsli
2026-02-17 10:33:38 +08:00
parent cb4e4ca711
commit dbe737cc0b
119 changed files with 2 additions and 25946 deletions

View File

@@ -1,587 +0,0 @@
---
name: backend-patterns
description: Backend architecture patterns, API design, database optimization, and server-side best practices for Node.js, Express, and Next.js API routes.
---
# Backend Development Patterns
Backend architecture patterns and best practices for scalable server-side applications.
## API Design Patterns
### RESTful API Structure
```typescript
// ✅ Resource-based URLs
GET /api/markets # List resources
GET /api/markets/:id # Get single resource
POST /api/markets # Create resource
PUT /api/markets/:id # Replace resource
PATCH /api/markets/:id # Update resource
DELETE /api/markets/:id # Delete resource
// ✅ Query parameters for filtering, sorting, pagination
GET /api/markets?status=active&sort=volume&limit=20&offset=0
```
### Repository Pattern
```typescript
// Abstract data access logic
interface MarketRepository {
findAll(filters?: MarketFilters): Promise<Market[]>
findById(id: string): Promise<Market | null>
create(data: CreateMarketDto): Promise<Market>
update(id: string, data: UpdateMarketDto): Promise<Market>
delete(id: string): Promise<void>
}
class SupabaseMarketRepository implements MarketRepository {
async findAll(filters?: MarketFilters): Promise<Market[]> {
let query = supabase.from('markets').select('*')
if (filters?.status) {
query = query.eq('status', filters.status)
}
if (filters?.limit) {
query = query.limit(filters.limit)
}
const { data, error } = await query
if (error) throw new Error(error.message)
return data
}
// Other methods...
}
```
### Service Layer Pattern
```typescript
// Business logic separated from data access
class MarketService {
constructor(private marketRepo: MarketRepository) {}
async searchMarkets(query: string, limit: number = 10): Promise<Market[]> {
// Business logic
const embedding = await generateEmbedding(query)
const results = await this.vectorSearch(embedding, limit)
// Fetch full data
const markets = await this.marketRepo.findByIds(results.map(r => r.id))
// Sort by similarity
return markets.sort((a, b) => {
const scoreA = results.find(r => r.id === a.id)?.score || 0
const scoreB = results.find(r => r.id === b.id)?.score || 0
return scoreA - scoreB
})
}
private async vectorSearch(embedding: number[], limit: number) {
// Vector search implementation
}
}
```
### Middleware Pattern
```typescript
// Request/response processing pipeline
export function withAuth(handler: NextApiHandler): NextApiHandler {
return async (req, res) => {
const token = req.headers.authorization?.replace('Bearer ', '')
if (!token) {
return res.status(401).json({ error: 'Unauthorized' })
}
try {
const user = await verifyToken(token)
req.user = user
return handler(req, res)
} catch (error) {
return res.status(401).json({ error: 'Invalid token' })
}
}
}
// Usage
export default withAuth(async (req, res) => {
// Handler has access to req.user
})
```
## Database Patterns
### Query Optimization
```typescript
// ✅ GOOD: Select only needed columns
const { data } = await supabase
.from('markets')
.select('id, name, status, volume')
.eq('status', 'active')
.order('volume', { ascending: false })
.limit(10)
// ❌ BAD: Select everything
const { data } = await supabase
.from('markets')
.select('*')
```
### N+1 Query Prevention
```typescript
// ❌ BAD: N+1 query problem
const markets = await getMarkets()
for (const market of markets) {
market.creator = await getUser(market.creator_id) // N queries
}
// ✅ GOOD: Batch fetch
const markets = await getMarkets()
const creatorIds = markets.map(m => m.creator_id)
const creators = await getUsers(creatorIds) // 1 query
const creatorMap = new Map(creators.map(c => [c.id, c]))
markets.forEach(market => {
market.creator = creatorMap.get(market.creator_id)
})
```
### Transaction Pattern
```typescript
async function createMarketWithPosition(
marketData: CreateMarketDto,
positionData: CreatePositionDto
) {
// Use Supabase transaction
const { data, error } = await supabase.rpc('create_market_with_position', {
market_data: marketData,
position_data: positionData
})
if (error) throw new Error('Transaction failed')
return data
}
// SQL function in Supabase
CREATE OR REPLACE FUNCTION create_market_with_position(
market_data jsonb,
position_data jsonb
)
RETURNS jsonb
LANGUAGE plpgsql
AS $$
BEGIN
-- Start transaction automatically
INSERT INTO markets VALUES (market_data);
INSERT INTO positions VALUES (position_data);
RETURN jsonb_build_object('success', true);
EXCEPTION
WHEN OTHERS THEN
-- Rollback happens automatically
RETURN jsonb_build_object('success', false, 'error', SQLERRM);
END;
$$;
```
## Caching Strategies
### Redis Caching Layer
```typescript
class CachedMarketRepository implements MarketRepository {
constructor(
private baseRepo: MarketRepository,
private redis: RedisClient
) {}
async findById(id: string): Promise<Market | null> {
// Check cache first
const cached = await this.redis.get(`market:${id}`)
if (cached) {
return JSON.parse(cached)
}
// Cache miss - fetch from database
const market = await this.baseRepo.findById(id)
if (market) {
// Cache for 5 minutes
await this.redis.setex(`market:${id}`, 300, JSON.stringify(market))
}
return market
}
async invalidateCache(id: string): Promise<void> {
await this.redis.del(`market:${id}`)
}
}
```
### Cache-Aside Pattern
```typescript
async function getMarketWithCache(id: string): Promise<Market> {
const cacheKey = `market:${id}`
// Try cache
const cached = await redis.get(cacheKey)
if (cached) return JSON.parse(cached)
// Cache miss - fetch from DB
const market = await db.markets.findUnique({ where: { id } })
if (!market) throw new Error('Market not found')
// Update cache
await redis.setex(cacheKey, 300, JSON.stringify(market))
return market
}
```
## Error Handling Patterns
### Centralized Error Handler
```typescript
class ApiError extends Error {
constructor(
public statusCode: number,
public message: string,
public isOperational = true
) {
super(message)
Object.setPrototypeOf(this, ApiError.prototype)
}
}
export function errorHandler(error: unknown, req: Request): Response {
if (error instanceof ApiError) {
return NextResponse.json({
success: false,
error: error.message
}, { status: error.statusCode })
}
if (error instanceof z.ZodError) {
return NextResponse.json({
success: false,
error: 'Validation failed',
details: error.errors
}, { status: 400 })
}
// Log unexpected errors
console.error('Unexpected error:', error)
return NextResponse.json({
success: false,
error: 'Internal server error'
}, { status: 500 })
}
// Usage
export async function GET(request: Request) {
try {
const data = await fetchData()
return NextResponse.json({ success: true, data })
} catch (error) {
return errorHandler(error, request)
}
}
```
### Retry with Exponential Backoff
```typescript
async function fetchWithRetry<T>(
fn: () => Promise<T>,
maxRetries = 3
): Promise<T> {
let lastError: Error
for (let i = 0; i < maxRetries; i++) {
try {
return await fn()
} catch (error) {
lastError = error as Error
if (i < maxRetries - 1) {
// Exponential backoff: 1s, 2s, 4s
const delay = Math.pow(2, i) * 1000
await new Promise(resolve => setTimeout(resolve, delay))
}
}
}
throw lastError!
}
// Usage
const data = await fetchWithRetry(() => fetchFromAPI())
```
## Authentication & Authorization
### JWT Token Validation
```typescript
import jwt from 'jsonwebtoken'
interface JWTPayload {
userId: string
email: string
role: 'admin' | 'user'
}
export function verifyToken(token: string): JWTPayload {
try {
const payload = jwt.verify(token, process.env.JWT_SECRET!) as JWTPayload
return payload
} catch (error) {
throw new ApiError(401, 'Invalid token')
}
}
export async function requireAuth(request: Request) {
const token = request.headers.get('authorization')?.replace('Bearer ', '')
if (!token) {
throw new ApiError(401, 'Missing authorization token')
}
return verifyToken(token)
}
// Usage in API route
export async function GET(request: Request) {
const user = await requireAuth(request)
const data = await getDataForUser(user.userId)
return NextResponse.json({ success: true, data })
}
```
### Role-Based Access Control
```typescript
type Permission = 'read' | 'write' | 'delete' | 'admin'
interface User {
id: string
role: 'admin' | 'moderator' | 'user'
}
const rolePermissions: Record<User['role'], Permission[]> = {
admin: ['read', 'write', 'delete', 'admin'],
moderator: ['read', 'write', 'delete'],
user: ['read', 'write']
}
export function hasPermission(user: User, permission: Permission): boolean {
return rolePermissions[user.role].includes(permission)
}
export function requirePermission(permission: Permission) {
return (handler: (request: Request, user: User) => Promise<Response>) => {
return async (request: Request) => {
const user = await requireAuth(request)
if (!hasPermission(user, permission)) {
throw new ApiError(403, 'Insufficient permissions')
}
return handler(request, user)
}
}
}
// Usage - HOF wraps the handler
export const DELETE = requirePermission('delete')(
async (request: Request, user: User) => {
// Handler receives authenticated user with verified permission
return new Response('Deleted', { status: 200 })
}
)
```
## Rate Limiting
### Simple In-Memory Rate Limiter
```typescript
class RateLimiter {
private requests = new Map<string, number[]>()
async checkLimit(
identifier: string,
maxRequests: number,
windowMs: number
): Promise<boolean> {
const now = Date.now()
const requests = this.requests.get(identifier) || []
// Remove old requests outside window
const recentRequests = requests.filter(time => now - time < windowMs)
if (recentRequests.length >= maxRequests) {
return false // Rate limit exceeded
}
// Add current request
recentRequests.push(now)
this.requests.set(identifier, recentRequests)
return true
}
}
const limiter = new RateLimiter()
export async function GET(request: Request) {
const ip = request.headers.get('x-forwarded-for') || 'unknown'
const allowed = await limiter.checkLimit(ip, 100, 60000) // 100 req/min
if (!allowed) {
return NextResponse.json({
error: 'Rate limit exceeded'
}, { status: 429 })
}
// Continue with request
}
```
## Background Jobs & Queues
### Simple Queue Pattern
```typescript
class JobQueue<T> {
private queue: T[] = []
private processing = false
async add(job: T): Promise<void> {
this.queue.push(job)
if (!this.processing) {
this.process()
}
}
private async process(): Promise<void> {
this.processing = true
while (this.queue.length > 0) {
const job = this.queue.shift()!
try {
await this.execute(job)
} catch (error) {
console.error('Job failed:', error)
}
}
this.processing = false
}
private async execute(job: T): Promise<void> {
// Job execution logic
}
}
// Usage for indexing markets
interface IndexJob {
marketId: string
}
const indexQueue = new JobQueue<IndexJob>()
export async function POST(request: Request) {
const { marketId } = await request.json()
// Add to queue instead of blocking
await indexQueue.add({ marketId })
return NextResponse.json({ success: true, message: 'Job queued' })
}
```
## Logging & Monitoring
### Structured Logging
```typescript
interface LogContext {
userId?: string
requestId?: string
method?: string
path?: string
[key: string]: unknown
}
class Logger {
log(level: 'info' | 'warn' | 'error', message: string, context?: LogContext) {
const entry = {
timestamp: new Date().toISOString(),
level,
message,
...context
}
console.log(JSON.stringify(entry))
}
info(message: string, context?: LogContext) {
this.log('info', message, context)
}
warn(message: string, context?: LogContext) {
this.log('warn', message, context)
}
error(message: string, error: Error, context?: LogContext) {
this.log('error', message, {
...context,
error: error.message,
stack: error.stack
})
}
}
const logger = new Logger()
// Usage
export async function GET(request: Request) {
const requestId = crypto.randomUUID()
logger.info('Fetching markets', {
requestId,
method: 'GET',
path: '/api/markets'
})
try {
const markets = await fetchMarkets()
return NextResponse.json({ success: true, data: markets })
} catch (error) {
logger.error('Failed to fetch markets', error as Error, { requestId })
return NextResponse.json({ error: 'Internal error' }, { status: 500 })
}
}
```
**Remember**: Backend patterns enable scalable, maintainable server-side applications. Choose patterns that fit your complexity level.

View File

@@ -1,429 +0,0 @@
---
name: clickhouse-io
description: ClickHouse database patterns, query optimization, analytics, and data engineering best practices for high-performance analytical workloads.
---
# ClickHouse Analytics Patterns
ClickHouse-specific patterns for high-performance analytics and data engineering.
## Overview
ClickHouse is a column-oriented database management system (DBMS) for online analytical processing (OLAP). It's optimized for fast analytical queries on large datasets.
**Key Features:**
- Column-oriented storage
- Data compression
- Parallel query execution
- Distributed queries
- Real-time analytics
## Table Design Patterns
### MergeTree Engine (Most Common)
```sql
CREATE TABLE markets_analytics (
date Date,
market_id String,
market_name String,
volume UInt64,
trades UInt32,
unique_traders UInt32,
avg_trade_size Float64,
created_at DateTime
) ENGINE = MergeTree()
PARTITION BY toYYYYMM(date)
ORDER BY (date, market_id)
SETTINGS index_granularity = 8192;
```
### ReplacingMergeTree (Deduplication)
```sql
-- For data that may have duplicates (e.g., from multiple sources)
CREATE TABLE user_events (
event_id String,
user_id String,
event_type String,
timestamp DateTime,
properties String
) ENGINE = ReplacingMergeTree()
PARTITION BY toYYYYMM(timestamp)
ORDER BY (user_id, event_id, timestamp)
PRIMARY KEY (user_id, event_id);
```
### AggregatingMergeTree (Pre-aggregation)
```sql
-- For maintaining aggregated metrics
CREATE TABLE market_stats_hourly (
hour DateTime,
market_id String,
total_volume AggregateFunction(sum, UInt64),
total_trades AggregateFunction(count, UInt32),
unique_users AggregateFunction(uniq, String)
) ENGINE = AggregatingMergeTree()
PARTITION BY toYYYYMM(hour)
ORDER BY (hour, market_id);
-- Query aggregated data
SELECT
hour,
market_id,
sumMerge(total_volume) AS volume,
countMerge(total_trades) AS trades,
uniqMerge(unique_users) AS users
FROM market_stats_hourly
WHERE hour >= toStartOfHour(now() - INTERVAL 24 HOUR)
GROUP BY hour, market_id
ORDER BY hour DESC;
```
## Query Optimization Patterns
### Efficient Filtering
```sql
-- ✅ GOOD: Use indexed columns first
SELECT *
FROM markets_analytics
WHERE date >= '2025-01-01'
AND market_id = 'market-123'
AND volume > 1000
ORDER BY date DESC
LIMIT 100;
-- ❌ BAD: Filter on non-indexed columns first
SELECT *
FROM markets_analytics
WHERE volume > 1000
AND market_name LIKE '%election%'
AND date >= '2025-01-01';
```
### Aggregations
```sql
-- ✅ GOOD: Use ClickHouse-specific aggregation functions
SELECT
toStartOfDay(created_at) AS day,
market_id,
sum(volume) AS total_volume,
count() AS total_trades,
uniq(trader_id) AS unique_traders,
avg(trade_size) AS avg_size
FROM trades
WHERE created_at >= today() - INTERVAL 7 DAY
GROUP BY day, market_id
ORDER BY day DESC, total_volume DESC;
-- ✅ Use quantile for percentiles (more efficient than percentile)
SELECT
quantile(0.50)(trade_size) AS median,
quantile(0.95)(trade_size) AS p95,
quantile(0.99)(trade_size) AS p99
FROM trades
WHERE created_at >= now() - INTERVAL 1 HOUR;
```
### Window Functions
```sql
-- Calculate running totals
SELECT
date,
market_id,
volume,
sum(volume) OVER (
PARTITION BY market_id
ORDER BY date
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
) AS cumulative_volume
FROM markets_analytics
WHERE date >= today() - INTERVAL 30 DAY
ORDER BY market_id, date;
```
## Data Insertion Patterns
### Bulk Insert (Recommended)
```typescript
import { ClickHouse } from 'clickhouse'
const clickhouse = new ClickHouse({
url: process.env.CLICKHOUSE_URL,
port: 8123,
basicAuth: {
username: process.env.CLICKHOUSE_USER,
password: process.env.CLICKHOUSE_PASSWORD
}
})
// ✅ Batch insert (efficient)
async function bulkInsertTrades(trades: Trade[]) {
const values = trades.map(trade => `(
'${trade.id}',
'${trade.market_id}',
'${trade.user_id}',
${trade.amount},
'${trade.timestamp.toISOString()}'
)`).join(',')
await clickhouse.query(`
INSERT INTO trades (id, market_id, user_id, amount, timestamp)
VALUES ${values}
`).toPromise()
}
// ❌ Individual inserts (slow)
async function insertTrade(trade: Trade) {
// Don't do this in a loop!
await clickhouse.query(`
INSERT INTO trades VALUES ('${trade.id}', ...)
`).toPromise()
}
```
### Streaming Insert
```typescript
// For continuous data ingestion
import { createWriteStream } from 'fs'
import { pipeline } from 'stream/promises'
async function streamInserts() {
const stream = clickhouse.insert('trades').stream()
for await (const batch of dataSource) {
stream.write(batch)
}
await stream.end()
}
```
## Materialized Views
### Real-time Aggregations
```sql
-- Create materialized view for hourly stats
CREATE MATERIALIZED VIEW market_stats_hourly_mv
TO market_stats_hourly
AS SELECT
toStartOfHour(timestamp) AS hour,
market_id,
sumState(amount) AS total_volume,
countState() AS total_trades,
uniqState(user_id) AS unique_users
FROM trades
GROUP BY hour, market_id;
-- Query the materialized view
SELECT
hour,
market_id,
sumMerge(total_volume) AS volume,
countMerge(total_trades) AS trades,
uniqMerge(unique_users) AS users
FROM market_stats_hourly
WHERE hour >= now() - INTERVAL 24 HOUR
GROUP BY hour, market_id;
```
## Performance Monitoring
### Query Performance
```sql
-- Check slow queries
SELECT
query_id,
user,
query,
query_duration_ms,
read_rows,
read_bytes,
memory_usage
FROM system.query_log
WHERE type = 'QueryFinish'
AND query_duration_ms > 1000
AND event_time >= now() - INTERVAL 1 HOUR
ORDER BY query_duration_ms DESC
LIMIT 10;
```
### Table Statistics
```sql
-- Check table sizes
SELECT
database,
table,
formatReadableSize(sum(bytes)) AS size,
sum(rows) AS rows,
max(modification_time) AS latest_modification
FROM system.parts
WHERE active
GROUP BY database, table
ORDER BY sum(bytes) DESC;
```
## Common Analytics Queries
### Time Series Analysis
```sql
-- Daily active users
SELECT
toDate(timestamp) AS date,
uniq(user_id) AS daily_active_users
FROM events
WHERE timestamp >= today() - INTERVAL 30 DAY
GROUP BY date
ORDER BY date;
-- Retention analysis
SELECT
signup_date,
countIf(days_since_signup = 0) AS day_0,
countIf(days_since_signup = 1) AS day_1,
countIf(days_since_signup = 7) AS day_7,
countIf(days_since_signup = 30) AS day_30
FROM (
SELECT
user_id,
min(toDate(timestamp)) AS signup_date,
toDate(timestamp) AS activity_date,
dateDiff('day', signup_date, activity_date) AS days_since_signup
FROM events
GROUP BY user_id, activity_date
)
GROUP BY signup_date
ORDER BY signup_date DESC;
```
### Funnel Analysis
```sql
-- Conversion funnel
SELECT
countIf(step = 'viewed_market') AS viewed,
countIf(step = 'clicked_trade') AS clicked,
countIf(step = 'completed_trade') AS completed,
round(clicked / viewed * 100, 2) AS view_to_click_rate,
round(completed / clicked * 100, 2) AS click_to_completion_rate
FROM (
SELECT
user_id,
session_id,
event_type AS step
FROM events
WHERE event_date = today()
)
GROUP BY session_id;
```
### Cohort Analysis
```sql
-- User cohorts by signup month
SELECT
toStartOfMonth(signup_date) AS cohort,
toStartOfMonth(activity_date) AS month,
dateDiff('month', cohort, month) AS months_since_signup,
count(DISTINCT user_id) AS active_users
FROM (
SELECT
user_id,
min(toDate(timestamp)) OVER (PARTITION BY user_id) AS signup_date,
toDate(timestamp) AS activity_date
FROM events
)
GROUP BY cohort, month, months_since_signup
ORDER BY cohort, months_since_signup;
```
## Data Pipeline Patterns
### ETL Pattern
```typescript
// Extract, Transform, Load
async function etlPipeline() {
// 1. Extract from source
const rawData = await extractFromPostgres()
// 2. Transform
const transformed = rawData.map(row => ({
date: new Date(row.created_at).toISOString().split('T')[0],
market_id: row.market_slug,
volume: parseFloat(row.total_volume),
trades: parseInt(row.trade_count)
}))
// 3. Load to ClickHouse
await bulkInsertToClickHouse(transformed)
}
// Run periodically
setInterval(etlPipeline, 60 * 60 * 1000) // Every hour
```
### Change Data Capture (CDC)
```typescript
// Listen to PostgreSQL changes and sync to ClickHouse
import { Client } from 'pg'
const pgClient = new Client({ connectionString: process.env.DATABASE_URL })
pgClient.query('LISTEN market_updates')
pgClient.on('notification', async (msg) => {
const update = JSON.parse(msg.payload)
await clickhouse.insert('market_updates', [
{
market_id: update.id,
event_type: update.operation, // INSERT, UPDATE, DELETE
timestamp: new Date(),
data: JSON.stringify(update.new_data)
}
])
})
```
## Best Practices
### 1. Partitioning Strategy
- Partition by time (usually month or day)
- Avoid too many partitions (performance impact)
- Use DATE type for partition key
### 2. Ordering Key
- Put most frequently filtered columns first
- Consider cardinality (high cardinality first)
- Order impacts compression
### 3. Data Types
- Use smallest appropriate type (UInt32 vs UInt64)
- Use LowCardinality for repeated strings
- Use Enum for categorical data
### 4. Avoid
- SELECT * (specify columns)
- FINAL (merge data before query instead)
- Too many JOINs (denormalize for analytics)
- Small frequent inserts (batch instead)
### 5. Monitoring
- Track query performance
- Monitor disk usage
- Check merge operations
- Review slow query log
**Remember**: ClickHouse excels at analytical workloads. Design tables for your query patterns, batch inserts, and leverage materialized views for real-time aggregations.

View File

@@ -1,520 +0,0 @@
---
name: coding-standards
description: Universal coding standards, best practices, and patterns for TypeScript, JavaScript, React, and Node.js development.
---
# Coding Standards & Best Practices
Universal coding standards applicable across all projects.
## Code Quality Principles
### 1. Readability First
- Code is read more than written
- Clear variable and function names
- Self-documenting code preferred over comments
- Consistent formatting
### 2. KISS (Keep It Simple, Stupid)
- Simplest solution that works
- Avoid over-engineering
- No premature optimization
- Easy to understand > clever code
### 3. DRY (Don't Repeat Yourself)
- Extract common logic into functions
- Create reusable components
- Share utilities across modules
- Avoid copy-paste programming
### 4. YAGNI (You Aren't Gonna Need It)
- Don't build features before they're needed
- Avoid speculative generality
- Add complexity only when required
- Start simple, refactor when needed
## TypeScript/JavaScript Standards
### Variable Naming
```typescript
// ✅ GOOD: Descriptive names
const marketSearchQuery = 'election'
const isUserAuthenticated = true
const totalRevenue = 1000
// ❌ BAD: Unclear names
const q = 'election'
const flag = true
const x = 1000
```
### Function Naming
```typescript
// ✅ GOOD: Verb-noun pattern
async function fetchMarketData(marketId: string) { }
function calculateSimilarity(a: number[], b: number[]) { }
function isValidEmail(email: string): boolean { }
// ❌ BAD: Unclear or noun-only
async function market(id: string) { }
function similarity(a, b) { }
function email(e) { }
```
### Immutability Pattern (CRITICAL)
```typescript
// ✅ ALWAYS use spread operator
const updatedUser = {
...user,
name: 'New Name'
}
const updatedArray = [...items, newItem]
// ❌ NEVER mutate directly
user.name = 'New Name' // BAD
items.push(newItem) // BAD
```
### Error Handling
```typescript
// ✅ GOOD: Comprehensive error handling
async function fetchData(url: string) {
try {
const response = await fetch(url)
if (!response.ok) {
throw new Error(`HTTP ${response.status}: ${response.statusText}`)
}
return await response.json()
} catch (error) {
console.error('Fetch failed:', error)
throw new Error('Failed to fetch data')
}
}
// ❌ BAD: No error handling
async function fetchData(url) {
const response = await fetch(url)
return response.json()
}
```
### Async/Await Best Practices
```typescript
// ✅ GOOD: Parallel execution when possible
const [users, markets, stats] = await Promise.all([
fetchUsers(),
fetchMarkets(),
fetchStats()
])
// ❌ BAD: Sequential when unnecessary
const users = await fetchUsers()
const markets = await fetchMarkets()
const stats = await fetchStats()
```
### Type Safety
```typescript
// ✅ GOOD: Proper types
interface Market {
id: string
name: string
status: 'active' | 'resolved' | 'closed'
created_at: Date
}
function getMarket(id: string): Promise<Market> {
// Implementation
}
// ❌ BAD: Using 'any'
function getMarket(id: any): Promise<any> {
// Implementation
}
```
## React Best Practices
### Component Structure
```typescript
// ✅ GOOD: Functional component with types
interface ButtonProps {
children: React.ReactNode
onClick: () => void
disabled?: boolean
variant?: 'primary' | 'secondary'
}
export function Button({
children,
onClick,
disabled = false,
variant = 'primary'
}: ButtonProps) {
return (
<button
onClick={onClick}
disabled={disabled}
className={`btn btn-${variant}`}
>
{children}
</button>
)
}
// ❌ BAD: No types, unclear structure
export function Button(props) {
return <button onClick={props.onClick}>{props.children}</button>
}
```
### Custom Hooks
```typescript
// ✅ GOOD: Reusable custom hook
export function useDebounce<T>(value: T, delay: number): T {
const [debouncedValue, setDebouncedValue] = useState<T>(value)
useEffect(() => {
const handler = setTimeout(() => {
setDebouncedValue(value)
}, delay)
return () => clearTimeout(handler)
}, [value, delay])
return debouncedValue
}
// Usage
const debouncedQuery = useDebounce(searchQuery, 500)
```
### State Management
```typescript
// ✅ GOOD: Proper state updates
const [count, setCount] = useState(0)
// Functional update for state based on previous state
setCount(prev => prev + 1)
// ❌ BAD: Direct state reference
setCount(count + 1) // Can be stale in async scenarios
```
### Conditional Rendering
```typescript
// ✅ GOOD: Clear conditional rendering
{isLoading && <Spinner />}
{error && <ErrorMessage error={error} />}
{data && <DataDisplay data={data} />}
// ❌ BAD: Ternary hell
{isLoading ? <Spinner /> : error ? <ErrorMessage error={error} /> : data ? <DataDisplay data={data} /> : null}
```
## API Design Standards
### REST API Conventions
```
GET /api/markets # List all markets
GET /api/markets/:id # Get specific market
POST /api/markets # Create new market
PUT /api/markets/:id # Update market (full)
PATCH /api/markets/:id # Update market (partial)
DELETE /api/markets/:id # Delete market
# Query parameters for filtering
GET /api/markets?status=active&limit=10&offset=0
```
### Response Format
```typescript
// ✅ GOOD: Consistent response structure
interface ApiResponse<T> {
success: boolean
data?: T
error?: string
meta?: {
total: number
page: number
limit: number
}
}
// Success response
return NextResponse.json({
success: true,
data: markets,
meta: { total: 100, page: 1, limit: 10 }
})
// Error response
return NextResponse.json({
success: false,
error: 'Invalid request'
}, { status: 400 })
```
### Input Validation
```typescript
import { z } from 'zod'
// ✅ GOOD: Schema validation
const CreateMarketSchema = z.object({
name: z.string().min(1).max(200),
description: z.string().min(1).max(2000),
endDate: z.string().datetime(),
categories: z.array(z.string()).min(1)
})
export async function POST(request: Request) {
const body = await request.json()
try {
const validated = CreateMarketSchema.parse(body)
// Proceed with validated data
} catch (error) {
if (error instanceof z.ZodError) {
return NextResponse.json({
success: false,
error: 'Validation failed',
details: error.errors
}, { status: 400 })
}
}
}
```
## File Organization
### Project Structure
```
src/
├── app/ # Next.js App Router
│ ├── api/ # API routes
│ ├── markets/ # Market pages
│ └── (auth)/ # Auth pages (route groups)
├── components/ # React components
│ ├── ui/ # Generic UI components
│ ├── forms/ # Form components
│ └── layouts/ # Layout components
├── hooks/ # Custom React hooks
├── lib/ # Utilities and configs
│ ├── api/ # API clients
│ ├── utils/ # Helper functions
│ └── constants/ # Constants
├── types/ # TypeScript types
└── styles/ # Global styles
```
### File Naming
```
components/Button.tsx # PascalCase for components
hooks/useAuth.ts # camelCase with 'use' prefix
lib/formatDate.ts # camelCase for utilities
types/market.types.ts # camelCase with .types suffix
```
## Comments & Documentation
### When to Comment
```typescript
// ✅ GOOD: Explain WHY, not WHAT
// Use exponential backoff to avoid overwhelming the API during outages
const delay = Math.min(1000 * Math.pow(2, retryCount), 30000)
// Deliberately using mutation here for performance with large arrays
items.push(newItem)
// ❌ BAD: Stating the obvious
// Increment counter by 1
count++
// Set name to user's name
name = user.name
```
### JSDoc for Public APIs
```typescript
/**
* Searches markets using semantic similarity.
*
* @param query - Natural language search query
* @param limit - Maximum number of results (default: 10)
* @returns Array of markets sorted by similarity score
* @throws {Error} If OpenAI API fails or Redis unavailable
*
* @example
* ```typescript
* const results = await searchMarkets('election', 5)
* console.log(results[0].name) // "Trump vs Biden"
* ```
*/
export async function searchMarkets(
query: string,
limit: number = 10
): Promise<Market[]> {
// Implementation
}
```
## Performance Best Practices
### Memoization
```typescript
import { useMemo, useCallback } from 'react'
// ✅ GOOD: Memoize expensive computations
const sortedMarkets = useMemo(() => {
return markets.sort((a, b) => b.volume - a.volume)
}, [markets])
// ✅ GOOD: Memoize callbacks
const handleSearch = useCallback((query: string) => {
setSearchQuery(query)
}, [])
```
### Lazy Loading
```typescript
import { lazy, Suspense } from 'react'
// ✅ GOOD: Lazy load heavy components
const HeavyChart = lazy(() => import('./HeavyChart'))
export function Dashboard() {
return (
<Suspense fallback={<Spinner />}>
<HeavyChart />
</Suspense>
)
}
```
### Database Queries
```typescript
// ✅ GOOD: Select only needed columns
const { data } = await supabase
.from('markets')
.select('id, name, status')
.limit(10)
// ❌ BAD: Select everything
const { data } = await supabase
.from('markets')
.select('*')
```
## Testing Standards
### Test Structure (AAA Pattern)
```typescript
test('calculates similarity correctly', () => {
// Arrange
const vector1 = [1, 0, 0]
const vector2 = [0, 1, 0]
// Act
const similarity = calculateCosineSimilarity(vector1, vector2)
// Assert
expect(similarity).toBe(0)
})
```
### Test Naming
```typescript
// ✅ GOOD: Descriptive test names
test('returns empty array when no markets match query', () => { })
test('throws error when OpenAI API key is missing', () => { })
test('falls back to substring search when Redis unavailable', () => { })
// ❌ BAD: Vague test names
test('works', () => { })
test('test search', () => { })
```
## Code Smell Detection
Watch for these anti-patterns:
### 1. Long Functions
```typescript
// ❌ BAD: Function > 50 lines
function processMarketData() {
// 100 lines of code
}
// ✅ GOOD: Split into smaller functions
function processMarketData() {
const validated = validateData()
const transformed = transformData(validated)
return saveData(transformed)
}
```
### 2. Deep Nesting
```typescript
// ❌ BAD: 5+ levels of nesting
if (user) {
if (user.isAdmin) {
if (market) {
if (market.isActive) {
if (hasPermission) {
// Do something
}
}
}
}
}
// ✅ GOOD: Early returns
if (!user) return
if (!user.isAdmin) return
if (!market) return
if (!market.isActive) return
if (!hasPermission) return
// Do something
```
### 3. Magic Numbers
```typescript
// ❌ BAD: Unexplained numbers
if (retryCount > 3) { }
setTimeout(callback, 500)
// ✅ GOOD: Named constants
const MAX_RETRIES = 3
const DEBOUNCE_DELAY_MS = 500
if (retryCount > MAX_RETRIES) { }
setTimeout(callback, DEBOUNCE_DELAY_MS)
```
**Remember**: Code quality is not negotiable. Clear, maintainable code enables rapid development and confident refactoring.

View File

@@ -1,298 +0,0 @@
---
name: configure-ecc
description: Interactive installer for Everything Claude Code — guides users through selecting and installing skills and rules to user-level or project-level directories, verifies paths, and optionally optimizes installed files.
---
# Configure Everything Claude Code (ECC)
An interactive, step-by-step installation wizard for the Everything Claude Code project. Uses `AskUserQuestion` to guide users through selective installation of skills and rules, then verifies correctness and offers optimization.
## When to Activate
- User says "configure ecc", "install ecc", "setup everything claude code", or similar
- User wants to selectively install skills or rules from this project
- User wants to verify or fix an existing ECC installation
- User wants to optimize installed skills or rules for their project
## Prerequisites
This skill must be accessible to Claude Code before activation. Two ways to bootstrap:
1. **Via Plugin**: `/plugin install everything-claude-code` — the plugin loads this skill automatically
2. **Manual**: Copy only this skill to `~/.claude/skills/configure-ecc/SKILL.md`, then activate by saying "configure ecc"
---
## Step 0: Clone ECC Repository
Before any installation, clone the latest ECC source to `/tmp`:
```bash
rm -rf /tmp/everything-claude-code
git clone https://github.com/affaan-m/everything-claude-code.git /tmp/everything-claude-code
```
Set `ECC_ROOT=/tmp/everything-claude-code` as the source for all subsequent copy operations.
If the clone fails (network issues, etc.), use `AskUserQuestion` to ask the user to provide a local path to an existing ECC clone.
---
## Step 1: Choose Installation Level
Use `AskUserQuestion` to ask the user where to install:
```
Question: "Where should ECC components be installed?"
Options:
- "User-level (~/.claude/)" — "Applies to all your Claude Code projects"
- "Project-level (.claude/)" — "Applies only to the current project"
- "Both" — "Common/shared items user-level, project-specific items project-level"
```
Store the choice as `INSTALL_LEVEL`. Set the target directory:
- User-level: `TARGET=~/.claude`
- Project-level: `TARGET=.claude` (relative to current project root)
- Both: `TARGET_USER=~/.claude`, `TARGET_PROJECT=.claude`
Create the target directories if they don't exist:
```bash
mkdir -p $TARGET/skills $TARGET/rules
```
---
## Step 2: Select & Install Skills
### 2a: Choose Skill Categories
There are 27 skills organized into 4 categories. Use `AskUserQuestion` with `multiSelect: true`:
```
Question: "Which skill categories do you want to install?"
Options:
- "Framework & Language" — "Django, Spring Boot, Go, Python, Java, Frontend, Backend patterns"
- "Database" — "PostgreSQL, ClickHouse, JPA/Hibernate patterns"
- "Workflow & Quality" — "TDD, verification, learning, security review, compaction"
- "All skills" — "Install every available skill"
```
### 2b: Confirm Individual Skills
For each selected category, print the full list of skills below and ask the user to confirm or deselect specific ones. If the list exceeds 4 items, print the list as text and use `AskUserQuestion` with an "Install all listed" option plus "Other" for the user to paste specific names.
**Category: Framework & Language (16 skills)**
| Skill | Description |
|-------|-------------|
| `backend-patterns` | Backend architecture, API design, server-side best practices for Node.js/Express/Next.js |
| `coding-standards` | Universal coding standards for TypeScript, JavaScript, React, Node.js |
| `django-patterns` | Django architecture, REST API with DRF, ORM, caching, signals, middleware |
| `django-security` | Django security: auth, CSRF, SQL injection, XSS prevention |
| `django-tdd` | Django testing with pytest-django, factory_boy, mocking, coverage |
| `django-verification` | Django verification loop: migrations, linting, tests, security scans |
| `frontend-patterns` | React, Next.js, state management, performance, UI patterns |
| `golang-patterns` | Idiomatic Go patterns, conventions for robust Go applications |
| `golang-testing` | Go testing: table-driven tests, subtests, benchmarks, fuzzing |
| `java-coding-standards` | Java coding standards for Spring Boot: naming, immutability, Optional, streams |
| `python-patterns` | Pythonic idioms, PEP 8, type hints, best practices |
| `python-testing` | Python testing with pytest, TDD, fixtures, mocking, parametrization |
| `springboot-patterns` | Spring Boot architecture, REST API, layered services, caching, async |
| `springboot-security` | Spring Security: authn/authz, validation, CSRF, secrets, rate limiting |
| `springboot-tdd` | Spring Boot TDD with JUnit 5, Mockito, MockMvc, Testcontainers |
| `springboot-verification` | Spring Boot verification: build, static analysis, tests, security scans |
**Category: Database (3 skills)**
| Skill | Description |
|-------|-------------|
| `clickhouse-io` | ClickHouse patterns, query optimization, analytics, data engineering |
| `jpa-patterns` | JPA/Hibernate entity design, relationships, query optimization, transactions |
| `postgres-patterns` | PostgreSQL query optimization, schema design, indexing, security |
**Category: Workflow & Quality (8 skills)**
| Skill | Description |
|-------|-------------|
| `continuous-learning` | Auto-extract reusable patterns from sessions as learned skills |
| `continuous-learning-v2` | Instinct-based learning with confidence scoring, evolves into skills/commands/agents |
| `eval-harness` | Formal evaluation framework for eval-driven development (EDD) |
| `iterative-retrieval` | Progressive context refinement for subagent context problem |
| `security-review` | Security checklist: auth, input, secrets, API, payment features |
| `strategic-compact` | Suggests manual context compaction at logical intervals |
| `tdd-workflow` | Enforces TDD with 80%+ coverage: unit, integration, E2E |
| `verification-loop` | Verification and quality loop patterns |
**Standalone**
| Skill | Description |
|-------|-------------|
| `project-guidelines-example` | Template for creating project-specific skills |
### 2c: Execute Installation
For each selected skill, copy the entire skill directory:
```bash
cp -r $ECC_ROOT/skills/<skill-name> $TARGET/skills/
```
Note: `continuous-learning` and `continuous-learning-v2` have extra files (config.json, hooks, scripts) — ensure the entire directory is copied, not just SKILL.md.
---
## Step 3: Select & Install Rules
Use `AskUserQuestion` with `multiSelect: true`:
```
Question: "Which rule sets do you want to install?"
Options:
- "Common rules (Recommended)" — "Language-agnostic principles: coding style, git workflow, testing, security, etc. (8 files)"
- "TypeScript/JavaScript" — "TS/JS patterns, hooks, testing with Playwright (5 files)"
- "Python" — "Python patterns, pytest, black/ruff formatting (5 files)"
- "Go" — "Go patterns, table-driven tests, gofmt/staticcheck (5 files)"
```
Execute installation:
```bash
# Common rules (flat copy into rules/)
cp -r $ECC_ROOT/rules/common/* $TARGET/rules/
# Language-specific rules (flat copy into rules/)
cp -r $ECC_ROOT/rules/typescript/* $TARGET/rules/ # if selected
cp -r $ECC_ROOT/rules/python/* $TARGET/rules/ # if selected
cp -r $ECC_ROOT/rules/golang/* $TARGET/rules/ # if selected
```
**Important**: If the user selects any language-specific rules but NOT common rules, warn them:
> "Language-specific rules extend the common rules. Installing without common rules may result in incomplete coverage. Install common rules too?"
---
## Step 4: Post-Installation Verification
After installation, perform these automated checks:
### 4a: Verify File Existence
List all installed files and confirm they exist at the target location:
```bash
ls -la $TARGET/skills/
ls -la $TARGET/rules/
```
### 4b: Check Path References
Scan all installed `.md` files for path references:
```bash
grep -rn "~/.claude/" $TARGET/skills/ $TARGET/rules/
grep -rn "../common/" $TARGET/rules/
grep -rn "skills/" $TARGET/skills/
```
**For project-level installs**, flag any references to `~/.claude/` paths:
- If a skill references `~/.claude/settings.json` — this is usually fine (settings are always user-level)
- If a skill references `~/.claude/skills/` or `~/.claude/rules/` — this may be broken if installed only at project level
- If a skill references another skill by name — check that the referenced skill was also installed
### 4c: Check Cross-References Between Skills
Some skills reference others. Verify these dependencies:
- `django-tdd` may reference `django-patterns`
- `springboot-tdd` may reference `springboot-patterns`
- `continuous-learning-v2` references `~/.claude/homunculus/` directory
- `python-testing` may reference `python-patterns`
- `golang-testing` may reference `golang-patterns`
- Language-specific rules reference `common/` counterparts
### 4d: Report Issues
For each issue found, report:
1. **File**: The file containing the problematic reference
2. **Line**: The line number
3. **Issue**: What's wrong (e.g., "references ~/.claude/skills/python-patterns but python-patterns was not installed")
4. **Suggested fix**: What to do (e.g., "install python-patterns skill" or "update path to .claude/skills/")
---
## Step 5: Optimize Installed Files (Optional)
Use `AskUserQuestion`:
```
Question: "Would you like to optimize the installed files for your project?"
Options:
- "Optimize skills" — "Remove irrelevant sections, adjust paths, tailor to your tech stack"
- "Optimize rules" — "Adjust coverage targets, add project-specific patterns, customize tool configs"
- "Optimize both" — "Full optimization of all installed files"
- "Skip" — "Keep everything as-is"
```
### If optimizing skills:
1. Read each installed SKILL.md
2. Ask the user what their project's tech stack is (if not already known)
3. For each skill, suggest removals of irrelevant sections
4. Edit the SKILL.md files in-place at the installation target (NOT the source repo)
5. Fix any path issues found in Step 4
### If optimizing rules:
1. Read each installed rule .md file
2. Ask the user about their preferences:
- Test coverage target (default 80%)
- Preferred formatting tools
- Git workflow conventions
- Security requirements
3. Edit the rule files in-place at the installation target
**Critical**: Only modify files in the installation target (`$TARGET/`), NEVER modify files in the source ECC repository (`$ECC_ROOT/`).
---
## Step 6: Installation Summary
Clean up the cloned repository from `/tmp`:
```bash
rm -rf /tmp/everything-claude-code
```
Then print a summary report:
```
## ECC Installation Complete
### Installation Target
- Level: [user-level / project-level / both]
- Path: [target path]
### Skills Installed ([count])
- skill-1, skill-2, skill-3, ...
### Rules Installed ([count])
- common (8 files)
- typescript (5 files)
- ...
### Verification Results
- [count] issues found, [count] fixed
- [list any remaining issues]
### Optimizations Applied
- [list changes made, or "None"]
```
---
## Troubleshooting
### "Skills not being picked up by Claude Code"
- Verify the skill directory contains a `SKILL.md` file (not just loose .md files)
- For user-level: check `~/.claude/skills/<skill-name>/SKILL.md` exists
- For project-level: check `.claude/skills/<skill-name>/SKILL.md` exists
### "Rules not working"
- Rules are flat files, not in subdirectories: `$TARGET/rules/coding-style.md` (correct) vs `$TARGET/rules/common/coding-style.md` (incorrect for flat install)
- Restart Claude Code after installing rules
### "Path reference errors after project-level install"
- Some skills assume `~/.claude/` paths. Run Step 4 verification to find and fix these.
- For `continuous-learning-v2`, the `~/.claude/homunculus/` directory is always user-level — this is expected and not an error.

View File

@@ -1,284 +0,0 @@
---
name: continuous-learning-v2
description: Instinct-based learning system that observes sessions via hooks, creates atomic instincts with confidence scoring, and evolves them into skills/commands/agents.
version: 2.0.0
---
# Continuous Learning v2 - Instinct-Based Architecture
An advanced learning system that turns your Claude Code sessions into reusable knowledge through atomic "instincts" - small learned behaviors with confidence scoring.
## What's New in v2
| Feature | v1 | v2 |
|---------|----|----|
| Observation | Stop hook (session end) | PreToolUse/PostToolUse (100% reliable) |
| Analysis | Main context | Background agent (Haiku) |
| Granularity | Full skills | Atomic "instincts" |
| Confidence | None | 0.3-0.9 weighted |
| Evolution | Direct to skill | Instincts → cluster → skill/command/agent |
| Sharing | None | Export/import instincts |
## The Instinct Model
An instinct is a small learned behavior:
```yaml
---
id: prefer-functional-style
trigger: "when writing new functions"
confidence: 0.7
domain: "code-style"
source: "session-observation"
---
# Prefer Functional Style
## Action
Use functional patterns over classes when appropriate.
## Evidence
- Observed 5 instances of functional pattern preference
- User corrected class-based approach to functional on 2025-01-15
```
**Properties:**
- **Atomic** — one trigger, one action
- **Confidence-weighted** — 0.3 = tentative, 0.9 = near certain
- **Domain-tagged** — code-style, testing, git, debugging, workflow, etc.
- **Evidence-backed** — tracks what observations created it
## How It Works
```
Session Activity
│ Hooks capture prompts + tool use (100% reliable)
┌─────────────────────────────────────────┐
│ observations.jsonl │
│ (prompts, tool calls, outcomes) │
└─────────────────────────────────────────┘
│ Observer agent reads (background, Haiku)
┌─────────────────────────────────────────┐
│ PATTERN DETECTION │
│ • User corrections → instinct │
│ • Error resolutions → instinct │
│ • Repeated workflows → instinct │
└─────────────────────────────────────────┘
│ Creates/updates
┌─────────────────────────────────────────┐
│ instincts/personal/ │
│ • prefer-functional.md (0.7) │
│ • always-test-first.md (0.9) │
│ • use-zod-validation.md (0.6) │
└─────────────────────────────────────────┘
│ /evolve clusters
┌─────────────────────────────────────────┐
│ evolved/ │
│ • commands/new-feature.md │
│ • skills/testing-workflow.md │
│ • agents/refactor-specialist.md │
└─────────────────────────────────────────┘
```
## Quick Start
### 1. Enable Observation Hooks
Add to your `~/.claude/settings.json`.
**If installed as a plugin** (recommended):
```json
{
"hooks": {
"PreToolUse": [{
"matcher": "*",
"hooks": [{
"type": "command",
"command": "${CLAUDE_PLUGIN_ROOT}/skills/continuous-learning-v2/hooks/observe.sh pre"
}]
}],
"PostToolUse": [{
"matcher": "*",
"hooks": [{
"type": "command",
"command": "${CLAUDE_PLUGIN_ROOT}/skills/continuous-learning-v2/hooks/observe.sh post"
}]
}]
}
}
```
**If installed manually** to `~/.claude/skills`:
```json
{
"hooks": {
"PreToolUse": [{
"matcher": "*",
"hooks": [{
"type": "command",
"command": "~/.claude/skills/continuous-learning-v2/hooks/observe.sh pre"
}]
}],
"PostToolUse": [{
"matcher": "*",
"hooks": [{
"type": "command",
"command": "~/.claude/skills/continuous-learning-v2/hooks/observe.sh post"
}]
}]
}
}
```
### 2. Initialize Directory Structure
The Python CLI will create these automatically, but you can also create them manually:
```bash
mkdir -p ~/.claude/homunculus/{instincts/{personal,inherited},evolved/{agents,skills,commands}}
touch ~/.claude/homunculus/observations.jsonl
```
### 3. Use the Instinct Commands
```bash
/instinct-status # Show learned instincts with confidence scores
/evolve # Cluster related instincts into skills/commands
/instinct-export # Export instincts for sharing
/instinct-import # Import instincts from others
```
## Commands
| Command | Description |
|---------|-------------|
| `/instinct-status` | Show all learned instincts with confidence |
| `/evolve` | Cluster related instincts into skills/commands |
| `/instinct-export` | Export instincts for sharing |
| `/instinct-import <file>` | Import instincts from others |
## Configuration
Edit `config.json`:
```json
{
"version": "2.0",
"observation": {
"enabled": true,
"store_path": "~/.claude/homunculus/observations.jsonl",
"max_file_size_mb": 10,
"archive_after_days": 7
},
"instincts": {
"personal_path": "~/.claude/homunculus/instincts/personal/",
"inherited_path": "~/.claude/homunculus/instincts/inherited/",
"min_confidence": 0.3,
"auto_approve_threshold": 0.7,
"confidence_decay_rate": 0.05
},
"observer": {
"enabled": true,
"model": "haiku",
"run_interval_minutes": 5,
"patterns_to_detect": [
"user_corrections",
"error_resolutions",
"repeated_workflows",
"tool_preferences"
]
},
"evolution": {
"cluster_threshold": 3,
"evolved_path": "~/.claude/homunculus/evolved/"
}
}
```
## File Structure
```
~/.claude/homunculus/
├── identity.json # Your profile, technical level
├── observations.jsonl # Current session observations
├── observations.archive/ # Processed observations
├── instincts/
│ ├── personal/ # Auto-learned instincts
│ └── inherited/ # Imported from others
└── evolved/
├── agents/ # Generated specialist agents
├── skills/ # Generated skills
└── commands/ # Generated commands
```
## Integration with Skill Creator
When you use the [Skill Creator GitHub App](https://skill-creator.app), it now generates **both**:
- Traditional SKILL.md files (for backward compatibility)
- Instinct collections (for v2 learning system)
Instincts from repo analysis have `source: "repo-analysis"` and include the source repository URL.
## Confidence Scoring
Confidence evolves over time:
| Score | Meaning | Behavior |
|-------|---------|----------|
| 0.3 | Tentative | Suggested but not enforced |
| 0.5 | Moderate | Applied when relevant |
| 0.7 | Strong | Auto-approved for application |
| 0.9 | Near-certain | Core behavior |
**Confidence increases** when:
- Pattern is repeatedly observed
- User doesn't correct the suggested behavior
- Similar instincts from other sources agree
**Confidence decreases** when:
- User explicitly corrects the behavior
- Pattern isn't observed for extended periods
- Contradicting evidence appears
## Why Hooks vs Skills for Observation?
> "v1 relied on skills to observe. Skills are probabilistic—they fire ~50-80% of the time based on Claude's judgment."
Hooks fire **100% of the time**, deterministically. This means:
- Every tool call is observed
- No patterns are missed
- Learning is comprehensive
## Backward Compatibility
v2 is fully compatible with v1:
- Existing `~/.claude/skills/learned/` skills still work
- Stop hook still runs (but now also feeds into v2)
- Gradual migration path: run both in parallel
## Privacy
- Observations stay **local** on your machine
- Only **instincts** (patterns) can be exported
- No actual code or conversation content is shared
- You control what gets exported
## Related
- [Skill Creator](https://skill-creator.app) - Generate instincts from repo history
- [Homunculus](https://github.com/humanplane/homunculus) - Inspiration for v2 architecture
- [The Longform Guide](https://x.com/affaanmustafa/status/2014040193557471352) - Continuous learning section
---
*Instinct-based learning: teaching Claude your patterns, one observation at a time.*

View File

@@ -1,137 +0,0 @@
---
name: observer
description: Background agent that analyzes session observations to detect patterns and create instincts. Uses Haiku for cost-efficiency.
model: haiku
run_mode: background
---
# Observer Agent
A background agent that analyzes observations from Claude Code sessions to detect patterns and create instincts.
## When to Run
- After significant session activity (20+ tool calls)
- When user runs `/analyze-patterns`
- On a scheduled interval (configurable, default 5 minutes)
- When triggered by observation hook (SIGUSR1)
## Input
Reads observations from `~/.claude/homunculus/observations.jsonl`:
```jsonl
{"timestamp":"2025-01-22T10:30:00Z","event":"tool_start","session":"abc123","tool":"Edit","input":"..."}
{"timestamp":"2025-01-22T10:30:01Z","event":"tool_complete","session":"abc123","tool":"Edit","output":"..."}
{"timestamp":"2025-01-22T10:30:05Z","event":"tool_start","session":"abc123","tool":"Bash","input":"npm test"}
{"timestamp":"2025-01-22T10:30:10Z","event":"tool_complete","session":"abc123","tool":"Bash","output":"All tests pass"}
```
## Pattern Detection
Look for these patterns in observations:
### 1. User Corrections
When a user's follow-up message corrects Claude's previous action:
- "No, use X instead of Y"
- "Actually, I meant..."
- Immediate undo/redo patterns
→ Create instinct: "When doing X, prefer Y"
### 2. Error Resolutions
When an error is followed by a fix:
- Tool output contains error
- Next few tool calls fix it
- Same error type resolved similarly multiple times
→ Create instinct: "When encountering error X, try Y"
### 3. Repeated Workflows
When the same sequence of tools is used multiple times:
- Same tool sequence with similar inputs
- File patterns that change together
- Time-clustered operations
→ Create workflow instinct: "When doing X, follow steps Y, Z, W"
### 4. Tool Preferences
When certain tools are consistently preferred:
- Always uses Grep before Edit
- Prefers Read over Bash cat
- Uses specific Bash commands for certain tasks
→ Create instinct: "When needing X, use tool Y"
## Output
Creates/updates instincts in `~/.claude/homunculus/instincts/personal/`:
```yaml
---
id: prefer-grep-before-edit
trigger: "when searching for code to modify"
confidence: 0.65
domain: "workflow"
source: "session-observation"
---
# Prefer Grep Before Edit
## Action
Always use Grep to find the exact location before using Edit.
## Evidence
- Observed 8 times in session abc123
- Pattern: Grep → Read → Edit sequence
- Last observed: 2025-01-22
```
## Confidence Calculation
Initial confidence based on observation frequency:
- 1-2 observations: 0.3 (tentative)
- 3-5 observations: 0.5 (moderate)
- 6-10 observations: 0.7 (strong)
- 11+ observations: 0.85 (very strong)
Confidence adjusts over time:
- +0.05 for each confirming observation
- -0.1 for each contradicting observation
- -0.02 per week without observation (decay)
## Important Guidelines
1. **Be Conservative**: Only create instincts for clear patterns (3+ observations)
2. **Be Specific**: Narrow triggers are better than broad ones
3. **Track Evidence**: Always include what observations led to the instinct
4. **Respect Privacy**: Never include actual code snippets, only patterns
5. **Merge Similar**: If a new instinct is similar to existing, update rather than duplicate
## Example Analysis Session
Given observations:
```jsonl
{"event":"tool_start","tool":"Grep","input":"pattern: useState"}
{"event":"tool_complete","tool":"Grep","output":"Found in 3 files"}
{"event":"tool_start","tool":"Read","input":"src/hooks/useAuth.ts"}
{"event":"tool_complete","tool":"Read","output":"[file content]"}
{"event":"tool_start","tool":"Edit","input":"src/hooks/useAuth.ts..."}
```
Analysis:
- Detected workflow: Grep → Read → Edit
- Frequency: Seen 5 times this session
- Create instinct:
- trigger: "when modifying code"
- action: "Search with Grep, confirm with Read, then Edit"
- confidence: 0.6
- domain: "workflow"
## Integration with Skill Creator
When instincts are imported from Skill Creator (repo analysis), they have:
- `source: "repo-analysis"`
- `source_repo: "https://github.com/..."`
These should be treated as team/project conventions with higher initial confidence (0.7+).

View File

@@ -1,143 +0,0 @@
#!/bin/bash
# Continuous Learning v2 - Observer Agent Launcher
#
# Starts the background observer agent that analyzes observations
# and creates instincts. Uses Haiku model for cost efficiency.
#
# Usage:
# start-observer.sh # Start observer in background
# start-observer.sh stop # Stop running observer
# start-observer.sh status # Check if observer is running
set -e
CONFIG_DIR="${HOME}/.claude/homunculus"
PID_FILE="${CONFIG_DIR}/.observer.pid"
LOG_FILE="${CONFIG_DIR}/observer.log"
OBSERVATIONS_FILE="${CONFIG_DIR}/observations.jsonl"
mkdir -p "$CONFIG_DIR"
case "${1:-start}" in
stop)
if [ -f "$PID_FILE" ]; then
pid=$(cat "$PID_FILE")
if kill -0 "$pid" 2>/dev/null; then
echo "Stopping observer (PID: $pid)..."
kill "$pid"
rm -f "$PID_FILE"
echo "Observer stopped."
else
echo "Observer not running (stale PID file)."
rm -f "$PID_FILE"
fi
else
echo "Observer not running."
fi
exit 0
;;
status)
if [ -f "$PID_FILE" ]; then
pid=$(cat "$PID_FILE")
if kill -0 "$pid" 2>/dev/null; then
echo "Observer is running (PID: $pid)"
echo "Log: $LOG_FILE"
echo "Observations: $(wc -l < "$OBSERVATIONS_FILE" 2>/dev/null || echo 0) lines"
exit 0
else
echo "Observer not running (stale PID file)"
rm -f "$PID_FILE"
exit 1
fi
else
echo "Observer not running"
exit 1
fi
;;
start)
# Check if already running
if [ -f "$PID_FILE" ]; then
pid=$(cat "$PID_FILE")
if kill -0 "$pid" 2>/dev/null; then
echo "Observer already running (PID: $pid)"
exit 0
fi
rm -f "$PID_FILE"
fi
echo "Starting observer agent..."
# The observer loop
(
trap 'rm -f "$PID_FILE"; exit 0' TERM INT
analyze_observations() {
# Only analyze if observations file exists and has enough entries
if [ ! -f "$OBSERVATIONS_FILE" ]; then
return
fi
obs_count=$(wc -l < "$OBSERVATIONS_FILE" 2>/dev/null || echo 0)
if [ "$obs_count" -lt 10 ]; then
return
fi
echo "[$(date)] Analyzing $obs_count observations..." >> "$LOG_FILE"
# Use Claude Code with Haiku to analyze observations
# This spawns a quick analysis session
if command -v claude &> /dev/null; then
exit_code=0
claude --model haiku --max-turns 3 --print \
"Read $OBSERVATIONS_FILE and identify patterns. If you find 3+ occurrences of the same pattern, create an instinct file in $CONFIG_DIR/instincts/personal/ following the format in the observer agent spec. Be conservative - only create instincts for clear patterns." \
>> "$LOG_FILE" 2>&1 || exit_code=$?
if [ "$exit_code" -ne 0 ]; then
echo "[$(date)] Claude analysis failed (exit $exit_code)" >> "$LOG_FILE"
fi
else
echo "[$(date)] claude CLI not found, skipping analysis" >> "$LOG_FILE"
fi
# Archive processed observations
if [ -f "$OBSERVATIONS_FILE" ]; then
archive_dir="${CONFIG_DIR}/observations.archive"
mkdir -p "$archive_dir"
mv "$OBSERVATIONS_FILE" "$archive_dir/processed-$(date +%Y%m%d-%H%M%S).jsonl" 2>/dev/null || true
touch "$OBSERVATIONS_FILE"
fi
}
# Handle SIGUSR1 for on-demand analysis
trap 'analyze_observations' USR1
echo "$$" > "$PID_FILE"
echo "[$(date)] Observer started (PID: $$)" >> "$LOG_FILE"
while true; do
# Check every 5 minutes
sleep 300
analyze_observations
done
) &
disown
# Wait a moment for PID file
sleep 1
if [ -f "$PID_FILE" ]; then
echo "Observer started (PID: $(cat "$PID_FILE"))"
echo "Log: $LOG_FILE"
else
echo "Failed to start observer"
exit 1
fi
;;
*)
echo "Usage: $0 {start|stop|status}"
exit 1
;;
esac

View File

@@ -1,41 +0,0 @@
{
"version": "2.0",
"observation": {
"enabled": true,
"store_path": "~/.claude/homunculus/observations.jsonl",
"max_file_size_mb": 10,
"archive_after_days": 7,
"capture_tools": ["Edit", "Write", "Bash", "Read", "Grep", "Glob"],
"ignore_tools": ["TodoWrite"]
},
"instincts": {
"personal_path": "~/.claude/homunculus/instincts/personal/",
"inherited_path": "~/.claude/homunculus/instincts/inherited/",
"min_confidence": 0.3,
"auto_approve_threshold": 0.7,
"confidence_decay_rate": 0.02,
"max_instincts": 100
},
"observer": {
"enabled": false,
"model": "haiku",
"run_interval_minutes": 5,
"min_observations_to_analyze": 20,
"patterns_to_detect": [
"user_corrections",
"error_resolutions",
"repeated_workflows",
"tool_preferences",
"file_patterns"
]
},
"evolution": {
"cluster_threshold": 3,
"evolved_path": "~/.claude/homunculus/evolved/",
"auto_evolve": false
},
"integration": {
"skill_creator_api": "https://skill-creator.app/api",
"backward_compatible_v1": true
}
}

View File

@@ -1,164 +0,0 @@
#!/bin/bash
# Continuous Learning v2 - Observation Hook
#
# Captures tool use events for pattern analysis.
# Claude Code passes hook data via stdin as JSON.
#
# Hook config (in ~/.claude/settings.json):
#
# If installed as a plugin, use ${CLAUDE_PLUGIN_ROOT}:
# {
# "hooks": {
# "PreToolUse": [{
# "matcher": "*",
# "hooks": [{ "type": "command", "command": "${CLAUDE_PLUGIN_ROOT}/skills/continuous-learning-v2/hooks/observe.sh pre" }]
# }],
# "PostToolUse": [{
# "matcher": "*",
# "hooks": [{ "type": "command", "command": "${CLAUDE_PLUGIN_ROOT}/skills/continuous-learning-v2/hooks/observe.sh post" }]
# }]
# }
# }
#
# If installed manually to ~/.claude/skills:
# {
# "hooks": {
# "PreToolUse": [{
# "matcher": "*",
# "hooks": [{ "type": "command", "command": "~/.claude/skills/continuous-learning-v2/hooks/observe.sh pre" }]
# }],
# "PostToolUse": [{
# "matcher": "*",
# "hooks": [{ "type": "command", "command": "~/.claude/skills/continuous-learning-v2/hooks/observe.sh post" }]
# }]
# }
# }
set -e
# Hook phase from CLI argument: "pre" (PreToolUse) or "post" (PostToolUse)
HOOK_PHASE="${1:-post}"
CONFIG_DIR="${HOME}/.claude/homunculus"
OBSERVATIONS_FILE="${CONFIG_DIR}/observations.jsonl"
MAX_FILE_SIZE_MB=10
# Ensure directory exists
mkdir -p "$CONFIG_DIR"
# Skip if disabled
if [ -f "$CONFIG_DIR/disabled" ]; then
exit 0
fi
# Read JSON from stdin (Claude Code hook format)
INPUT_JSON=$(cat)
# Exit if no input
if [ -z "$INPUT_JSON" ]; then
exit 0
fi
# Parse using python via stdin pipe (safe for all JSON payloads)
# Pass HOOK_PHASE via env var since Claude Code does not include hook type in stdin JSON
PARSED=$(echo "$INPUT_JSON" | HOOK_PHASE="$HOOK_PHASE" python3 -c '
import json
import sys
import os
try:
data = json.load(sys.stdin)
# Determine event type from CLI argument passed via env var.
# Claude Code does NOT include a "hook_type" field in the stdin JSON,
# so we rely on the shell argument ("pre" or "post") instead.
hook_phase = os.environ.get("HOOK_PHASE", "post")
event = "tool_start" if hook_phase == "pre" else "tool_complete"
# Extract fields - Claude Code hook format
tool_name = data.get("tool_name", data.get("tool", "unknown"))
tool_input = data.get("tool_input", data.get("input", {}))
tool_output = data.get("tool_output", data.get("output", ""))
session_id = data.get("session_id", "unknown")
# Truncate large inputs/outputs
if isinstance(tool_input, dict):
tool_input_str = json.dumps(tool_input)[:5000]
else:
tool_input_str = str(tool_input)[:5000]
if isinstance(tool_output, dict):
tool_output_str = json.dumps(tool_output)[:5000]
else:
tool_output_str = str(tool_output)[:5000]
print(json.dumps({
"parsed": True,
"event": event,
"tool": tool_name,
"input": tool_input_str if event == "tool_start" else None,
"output": tool_output_str if event == "tool_complete" else None,
"session": session_id
}))
except Exception as e:
print(json.dumps({"parsed": False, "error": str(e)}))
')
# Check if parsing succeeded
PARSED_OK=$(echo "$PARSED" | python3 -c "import json,sys; print(json.load(sys.stdin).get('parsed', False))")
if [ "$PARSED_OK" != "True" ]; then
# Fallback: log raw input for debugging
timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
export TIMESTAMP="$timestamp"
echo "$INPUT_JSON" | python3 -c "
import json, sys, os
raw = sys.stdin.read()[:2000]
print(json.dumps({'timestamp': os.environ['TIMESTAMP'], 'event': 'parse_error', 'raw': raw}))
" >> "$OBSERVATIONS_FILE"
exit 0
fi
# Archive if file too large
if [ -f "$OBSERVATIONS_FILE" ]; then
file_size_mb=$(du -m "$OBSERVATIONS_FILE" 2>/dev/null | cut -f1)
if [ "${file_size_mb:-0}" -ge "$MAX_FILE_SIZE_MB" ]; then
archive_dir="${CONFIG_DIR}/observations.archive"
mkdir -p "$archive_dir"
mv "$OBSERVATIONS_FILE" "$archive_dir/observations-$(date +%Y%m%d-%H%M%S).jsonl"
fi
fi
# Build and write observation
timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
export TIMESTAMP="$timestamp"
echo "$PARSED" | python3 -c "
import json, sys, os
parsed = json.load(sys.stdin)
observation = {
'timestamp': os.environ['TIMESTAMP'],
'event': parsed['event'],
'tool': parsed['tool'],
'session': parsed['session']
}
if parsed['input']:
observation['input'] = parsed['input']
if parsed['output']:
observation['output'] = parsed['output']
print(json.dumps(observation))
" >> "$OBSERVATIONS_FILE"
# Signal observer if running
OBSERVER_PID_FILE="${CONFIG_DIR}/.observer.pid"
if [ -f "$OBSERVER_PID_FILE" ]; then
observer_pid=$(cat "$OBSERVER_PID_FILE")
if kill -0 "$observer_pid" 2>/dev/null; then
kill -USR1 "$observer_pid" 2>/dev/null || true
fi
fi
exit 0

View File

@@ -1,575 +0,0 @@
#!/usr/bin/env python3
"""
Instinct CLI - Manage instincts for Continuous Learning v2
Commands:
status - Show all instincts and their status
import - Import instincts from file or URL
export - Export instincts to file
evolve - Cluster instincts into skills/commands/agents
"""
import argparse
import json
import os
import sys
import re
import urllib.request
from pathlib import Path
from datetime import datetime
from collections import defaultdict
from typing import Optional
# ─────────────────────────────────────────────
# Configuration
# ─────────────────────────────────────────────
HOMUNCULUS_DIR = Path.home() / ".claude" / "homunculus"
INSTINCTS_DIR = HOMUNCULUS_DIR / "instincts"
PERSONAL_DIR = INSTINCTS_DIR / "personal"
INHERITED_DIR = INSTINCTS_DIR / "inherited"
EVOLVED_DIR = HOMUNCULUS_DIR / "evolved"
OBSERVATIONS_FILE = HOMUNCULUS_DIR / "observations.jsonl"
# Ensure directories exist
for d in [PERSONAL_DIR, INHERITED_DIR, EVOLVED_DIR / "skills", EVOLVED_DIR / "commands", EVOLVED_DIR / "agents"]:
d.mkdir(parents=True, exist_ok=True)
# ─────────────────────────────────────────────
# Instinct Parser
# ─────────────────────────────────────────────
def parse_instinct_file(content: str) -> list[dict]:
"""Parse YAML-like instinct file format."""
instincts = []
current = {}
in_frontmatter = False
content_lines = []
for line in content.split('\n'):
if line.strip() == '---':
if in_frontmatter:
# End of frontmatter - content comes next, don't append yet
in_frontmatter = False
else:
# Start of frontmatter
in_frontmatter = True
if current:
current['content'] = '\n'.join(content_lines).strip()
instincts.append(current)
current = {}
content_lines = []
elif in_frontmatter:
# Parse YAML-like frontmatter
if ':' in line:
key, value = line.split(':', 1)
key = key.strip()
value = value.strip().strip('"').strip("'")
if key == 'confidence':
current[key] = float(value)
else:
current[key] = value
else:
content_lines.append(line)
# Don't forget the last instinct
if current:
current['content'] = '\n'.join(content_lines).strip()
instincts.append(current)
return [i for i in instincts if i.get('id')]
def load_all_instincts() -> list[dict]:
"""Load all instincts from personal and inherited directories."""
instincts = []
for directory in [PERSONAL_DIR, INHERITED_DIR]:
if not directory.exists():
continue
yaml_files = sorted(
set(directory.glob("*.yaml"))
| set(directory.glob("*.yml"))
| set(directory.glob("*.md"))
)
for file in yaml_files:
try:
content = file.read_text()
parsed = parse_instinct_file(content)
for inst in parsed:
inst['_source_file'] = str(file)
inst['_source_type'] = directory.name
instincts.extend(parsed)
except Exception as e:
print(f"Warning: Failed to parse {file}: {e}", file=sys.stderr)
return instincts
# ─────────────────────────────────────────────
# Status Command
# ─────────────────────────────────────────────
def cmd_status(args):
"""Show status of all instincts."""
instincts = load_all_instincts()
if not instincts:
print("No instincts found.")
print(f"\nInstinct directories:")
print(f" Personal: {PERSONAL_DIR}")
print(f" Inherited: {INHERITED_DIR}")
return
# Group by domain
by_domain = defaultdict(list)
for inst in instincts:
domain = inst.get('domain', 'general')
by_domain[domain].append(inst)
# Print header
print(f"\n{'='*60}")
print(f" INSTINCT STATUS - {len(instincts)} total")
print(f"{'='*60}\n")
# Summary by source
personal = [i for i in instincts if i.get('_source_type') == 'personal']
inherited = [i for i in instincts if i.get('_source_type') == 'inherited']
print(f" Personal: {len(personal)}")
print(f" Inherited: {len(inherited)}")
print()
# Print by domain
for domain in sorted(by_domain.keys()):
domain_instincts = by_domain[domain]
print(f"## {domain.upper()} ({len(domain_instincts)})")
print()
for inst in sorted(domain_instincts, key=lambda x: -x.get('confidence', 0.5)):
conf = inst.get('confidence', 0.5)
conf_bar = '' * int(conf * 10) + '' * (10 - int(conf * 10))
trigger = inst.get('trigger', 'unknown trigger')
source = inst.get('source', 'unknown')
print(f" {conf_bar} {int(conf*100):3d}% {inst.get('id', 'unnamed')}")
print(f" trigger: {trigger}")
# Extract action from content
content = inst.get('content', '')
action_match = re.search(r'## Action\s*\n\s*(.+?)(?:\n\n|\n##|$)', content, re.DOTALL)
if action_match:
action = action_match.group(1).strip().split('\n')[0]
print(f" action: {action[:60]}{'...' if len(action) > 60 else ''}")
print()
# Observations stats
if OBSERVATIONS_FILE.exists():
obs_count = sum(1 for _ in open(OBSERVATIONS_FILE))
print(f"─────────────────────────────────────────────────────────")
print(f" Observations: {obs_count} events logged")
print(f" File: {OBSERVATIONS_FILE}")
print(f"\n{'='*60}\n")
# ─────────────────────────────────────────────
# Import Command
# ─────────────────────────────────────────────
def cmd_import(args):
"""Import instincts from file or URL."""
source = args.source
# Fetch content
if source.startswith('http://') or source.startswith('https://'):
print(f"Fetching from URL: {source}")
try:
with urllib.request.urlopen(source) as response:
content = response.read().decode('utf-8')
except Exception as e:
print(f"Error fetching URL: {e}", file=sys.stderr)
return 1
else:
path = Path(source).expanduser()
if not path.exists():
print(f"File not found: {path}", file=sys.stderr)
return 1
content = path.read_text()
# Parse instincts
new_instincts = parse_instinct_file(content)
if not new_instincts:
print("No valid instincts found in source.")
return 1
print(f"\nFound {len(new_instincts)} instincts to import.\n")
# Load existing
existing = load_all_instincts()
existing_ids = {i.get('id') for i in existing}
# Categorize
to_add = []
duplicates = []
to_update = []
for inst in new_instincts:
inst_id = inst.get('id')
if inst_id in existing_ids:
# Check if we should update
existing_inst = next((e for e in existing if e.get('id') == inst_id), None)
if existing_inst:
if inst.get('confidence', 0) > existing_inst.get('confidence', 0):
to_update.append(inst)
else:
duplicates.append(inst)
else:
to_add.append(inst)
# Filter by minimum confidence
min_conf = args.min_confidence or 0.0
to_add = [i for i in to_add if i.get('confidence', 0.5) >= min_conf]
to_update = [i for i in to_update if i.get('confidence', 0.5) >= min_conf]
# Display summary
if to_add:
print(f"NEW ({len(to_add)}):")
for inst in to_add:
print(f" + {inst.get('id')} (confidence: {inst.get('confidence', 0.5):.2f})")
if to_update:
print(f"\nUPDATE ({len(to_update)}):")
for inst in to_update:
print(f" ~ {inst.get('id')} (confidence: {inst.get('confidence', 0.5):.2f})")
if duplicates:
print(f"\nSKIP ({len(duplicates)} - already exists with equal/higher confidence):")
for inst in duplicates[:5]:
print(f" - {inst.get('id')}")
if len(duplicates) > 5:
print(f" ... and {len(duplicates) - 5} more")
if args.dry_run:
print("\n[DRY RUN] No changes made.")
return 0
if not to_add and not to_update:
print("\nNothing to import.")
return 0
# Confirm
if not args.force:
response = input(f"\nImport {len(to_add)} new, update {len(to_update)}? [y/N] ")
if response.lower() != 'y':
print("Cancelled.")
return 0
# Write to inherited directory
timestamp = datetime.now().strftime('%Y%m%d-%H%M%S')
source_name = Path(source).stem if not source.startswith('http') else 'web-import'
output_file = INHERITED_DIR / f"{source_name}-{timestamp}.yaml"
all_to_write = to_add + to_update
output_content = f"# Imported from {source}\n# Date: {datetime.now().isoformat()}\n\n"
for inst in all_to_write:
output_content += "---\n"
output_content += f"id: {inst.get('id')}\n"
output_content += f"trigger: \"{inst.get('trigger', 'unknown')}\"\n"
output_content += f"confidence: {inst.get('confidence', 0.5)}\n"
output_content += f"domain: {inst.get('domain', 'general')}\n"
output_content += f"source: inherited\n"
output_content += f"imported_from: \"{source}\"\n"
if inst.get('source_repo'):
output_content += f"source_repo: {inst.get('source_repo')}\n"
output_content += "---\n\n"
output_content += inst.get('content', '') + "\n\n"
output_file.write_text(output_content)
print(f"\n✅ Import complete!")
print(f" Added: {len(to_add)}")
print(f" Updated: {len(to_update)}")
print(f" Saved to: {output_file}")
return 0
# ─────────────────────────────────────────────
# Export Command
# ─────────────────────────────────────────────
def cmd_export(args):
"""Export instincts to file."""
instincts = load_all_instincts()
if not instincts:
print("No instincts to export.")
return 1
# Filter by domain if specified
if args.domain:
instincts = [i for i in instincts if i.get('domain') == args.domain]
# Filter by minimum confidence
if args.min_confidence:
instincts = [i for i in instincts if i.get('confidence', 0.5) >= args.min_confidence]
if not instincts:
print("No instincts match the criteria.")
return 1
# Generate output
output = f"# Instincts export\n# Date: {datetime.now().isoformat()}\n# Total: {len(instincts)}\n\n"
for inst in instincts:
output += "---\n"
for key in ['id', 'trigger', 'confidence', 'domain', 'source', 'source_repo']:
if inst.get(key):
value = inst[key]
if key == 'trigger':
output += f'{key}: "{value}"\n'
else:
output += f"{key}: {value}\n"
output += "---\n\n"
output += inst.get('content', '') + "\n\n"
# Write to file or stdout
if args.output:
Path(args.output).write_text(output)
print(f"Exported {len(instincts)} instincts to {args.output}")
else:
print(output)
return 0
# ─────────────────────────────────────────────
# Evolve Command
# ─────────────────────────────────────────────
def cmd_evolve(args):
"""Analyze instincts and suggest evolutions to skills/commands/agents."""
instincts = load_all_instincts()
if len(instincts) < 3:
print("Need at least 3 instincts to analyze patterns.")
print(f"Currently have: {len(instincts)}")
return 1
print(f"\n{'='*60}")
print(f" EVOLVE ANALYSIS - {len(instincts)} instincts")
print(f"{'='*60}\n")
# Group by domain
by_domain = defaultdict(list)
for inst in instincts:
domain = inst.get('domain', 'general')
by_domain[domain].append(inst)
# High-confidence instincts by domain (candidates for skills)
high_conf = [i for i in instincts if i.get('confidence', 0) >= 0.8]
print(f"High confidence instincts (>=80%): {len(high_conf)}")
# Find clusters (instincts with similar triggers)
trigger_clusters = defaultdict(list)
for inst in instincts:
trigger = inst.get('trigger', '')
# Normalize trigger
trigger_key = trigger.lower()
for keyword in ['when', 'creating', 'writing', 'adding', 'implementing', 'testing']:
trigger_key = trigger_key.replace(keyword, '').strip()
trigger_clusters[trigger_key].append(inst)
# Find clusters with 3+ instincts (good skill candidates)
skill_candidates = []
for trigger, cluster in trigger_clusters.items():
if len(cluster) >= 2:
avg_conf = sum(i.get('confidence', 0.5) for i in cluster) / len(cluster)
skill_candidates.append({
'trigger': trigger,
'instincts': cluster,
'avg_confidence': avg_conf,
'domains': list(set(i.get('domain', 'general') for i in cluster))
})
# Sort by cluster size and confidence
skill_candidates.sort(key=lambda x: (-len(x['instincts']), -x['avg_confidence']))
print(f"\nPotential skill clusters found: {len(skill_candidates)}")
if skill_candidates:
print(f"\n## SKILL CANDIDATES\n")
for i, cand in enumerate(skill_candidates[:5], 1):
print(f"{i}. Cluster: \"{cand['trigger']}\"")
print(f" Instincts: {len(cand['instincts'])}")
print(f" Avg confidence: {cand['avg_confidence']:.0%}")
print(f" Domains: {', '.join(cand['domains'])}")
print(f" Instincts:")
for inst in cand['instincts'][:3]:
print(f" - {inst.get('id')}")
print()
# Command candidates (workflow instincts with high confidence)
workflow_instincts = [i for i in instincts if i.get('domain') == 'workflow' and i.get('confidence', 0) >= 0.7]
if workflow_instincts:
print(f"\n## COMMAND CANDIDATES ({len(workflow_instincts)})\n")
for inst in workflow_instincts[:5]:
trigger = inst.get('trigger', 'unknown')
# Suggest command name
cmd_name = trigger.replace('when ', '').replace('implementing ', '').replace('a ', '')
cmd_name = cmd_name.replace(' ', '-')[:20]
print(f" /{cmd_name}")
print(f" From: {inst.get('id')}")
print(f" Confidence: {inst.get('confidence', 0.5):.0%}")
print()
# Agent candidates (complex multi-step patterns)
agent_candidates = [c for c in skill_candidates if len(c['instincts']) >= 3 and c['avg_confidence'] >= 0.75]
if agent_candidates:
print(f"\n## AGENT CANDIDATES ({len(agent_candidates)})\n")
for cand in agent_candidates[:3]:
agent_name = cand['trigger'].replace(' ', '-')[:20] + '-agent'
print(f" {agent_name}")
print(f" Covers {len(cand['instincts'])} instincts")
print(f" Avg confidence: {cand['avg_confidence']:.0%}")
print()
if args.generate:
generated = _generate_evolved(skill_candidates, workflow_instincts, agent_candidates)
if generated:
print(f"\n✅ Generated {len(generated)} evolved structures:")
for path in generated:
print(f" {path}")
else:
print("\nNo structures generated (need higher-confidence clusters).")
print(f"\n{'='*60}\n")
return 0
# ─────────────────────────────────────────────
# Generate Evolved Structures
# ─────────────────────────────────────────────
def _generate_evolved(skill_candidates: list, workflow_instincts: list, agent_candidates: list) -> list[str]:
"""Generate skill/command/agent files from analyzed instinct clusters."""
generated = []
# Generate skills from top candidates
for cand in skill_candidates[:5]:
trigger = cand['trigger'].strip()
if not trigger:
continue
name = re.sub(r'[^a-z0-9]+', '-', trigger.lower()).strip('-')[:30]
if not name:
continue
skill_dir = EVOLVED_DIR / "skills" / name
skill_dir.mkdir(parents=True, exist_ok=True)
content = f"# {name}\n\n"
content += f"Evolved from {len(cand['instincts'])} instincts "
content += f"(avg confidence: {cand['avg_confidence']:.0%})\n\n"
content += f"## When to Apply\n\n"
content += f"Trigger: {trigger}\n\n"
content += f"## Actions\n\n"
for inst in cand['instincts']:
inst_content = inst.get('content', '')
action_match = re.search(r'## Action\s*\n\s*(.+?)(?:\n\n|\n##|$)', inst_content, re.DOTALL)
action = action_match.group(1).strip() if action_match else inst.get('id', 'unnamed')
content += f"- {action}\n"
(skill_dir / "SKILL.md").write_text(content)
generated.append(str(skill_dir / "SKILL.md"))
# Generate commands from workflow instincts
for inst in workflow_instincts[:5]:
trigger = inst.get('trigger', 'unknown')
cmd_name = re.sub(r'[^a-z0-9]+', '-', trigger.lower().replace('when ', '').replace('implementing ', ''))
cmd_name = cmd_name.strip('-')[:20]
if not cmd_name:
continue
cmd_file = EVOLVED_DIR / "commands" / f"{cmd_name}.md"
content = f"# {cmd_name}\n\n"
content += f"Evolved from instinct: {inst.get('id', 'unnamed')}\n"
content += f"Confidence: {inst.get('confidence', 0.5):.0%}\n\n"
content += inst.get('content', '')
cmd_file.write_text(content)
generated.append(str(cmd_file))
# Generate agents from complex clusters
for cand in agent_candidates[:3]:
trigger = cand['trigger'].strip()
agent_name = re.sub(r'[^a-z0-9]+', '-', trigger.lower()).strip('-')[:20]
if not agent_name:
continue
agent_file = EVOLVED_DIR / "agents" / f"{agent_name}.md"
domains = ', '.join(cand['domains'])
instinct_ids = [i.get('id', 'unnamed') for i in cand['instincts']]
content = f"---\nmodel: sonnet\ntools: Read, Grep, Glob\n---\n"
content += f"# {agent_name}\n\n"
content += f"Evolved from {len(cand['instincts'])} instincts "
content += f"(avg confidence: {cand['avg_confidence']:.0%})\n"
content += f"Domains: {domains}\n\n"
content += f"## Source Instincts\n\n"
for iid in instinct_ids:
content += f"- {iid}\n"
agent_file.write_text(content)
generated.append(str(agent_file))
return generated
# ─────────────────────────────────────────────
# Main
# ─────────────────────────────────────────────
def main():
parser = argparse.ArgumentParser(description='Instinct CLI for Continuous Learning v2')
subparsers = parser.add_subparsers(dest='command', help='Available commands')
# Status
status_parser = subparsers.add_parser('status', help='Show instinct status')
# Import
import_parser = subparsers.add_parser('import', help='Import instincts')
import_parser.add_argument('source', help='File path or URL')
import_parser.add_argument('--dry-run', action='store_true', help='Preview without importing')
import_parser.add_argument('--force', action='store_true', help='Skip confirmation')
import_parser.add_argument('--min-confidence', type=float, help='Minimum confidence threshold')
# Export
export_parser = subparsers.add_parser('export', help='Export instincts')
export_parser.add_argument('--output', '-o', help='Output file')
export_parser.add_argument('--domain', help='Filter by domain')
export_parser.add_argument('--min-confidence', type=float, help='Minimum confidence')
# Evolve
evolve_parser = subparsers.add_parser('evolve', help='Analyze and evolve instincts')
evolve_parser.add_argument('--generate', action='store_true', help='Generate evolved structures')
args = parser.parse_args()
if args.command == 'status':
return cmd_status(args)
elif args.command == 'import':
return cmd_import(args)
elif args.command == 'export':
return cmd_export(args)
elif args.command == 'evolve':
return cmd_evolve(args)
else:
parser.print_help()
return 1
if __name__ == '__main__':
sys.exit(main() or 0)

View File

@@ -1,82 +0,0 @@
"""Tests for parse_instinct_file() — verifies content after frontmatter is preserved."""
import importlib.util
import os
# Load instinct-cli.py (hyphenated filename requires importlib)
_spec = importlib.util.spec_from_file_location(
"instinct_cli",
os.path.join(os.path.dirname(__file__), "instinct-cli.py"),
)
_mod = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(_mod)
parse_instinct_file = _mod.parse_instinct_file
MULTI_SECTION = """\
---
id: instinct-a
trigger: "when coding"
confidence: 0.9
domain: general
---
## Action
Do thing A.
## Examples
- Example A1
---
id: instinct-b
trigger: "when testing"
confidence: 0.7
domain: testing
---
## Action
Do thing B.
"""
def test_multiple_instincts_preserve_content():
result = parse_instinct_file(MULTI_SECTION)
assert len(result) == 2
assert "Do thing A." in result[0]["content"]
assert "Example A1" in result[0]["content"]
assert "Do thing B." in result[1]["content"]
def test_single_instinct_preserves_content():
content = """\
---
id: solo
trigger: "when reviewing"
confidence: 0.8
domain: review
---
## Action
Check for security issues.
## Evidence
Prevents vulnerabilities.
"""
result = parse_instinct_file(content)
assert len(result) == 1
assert "Check for security issues." in result[0]["content"]
assert "Prevents vulnerabilities." in result[0]["content"]
def test_empty_content_no_error():
content = """\
---
id: empty
trigger: "placeholder"
confidence: 0.5
domain: general
---
"""
result = parse_instinct_file(content)
assert len(result) == 1
assert result[0]["content"] == ""

View File

@@ -1,110 +0,0 @@
---
name: continuous-learning
description: Automatically extract reusable patterns from Claude Code sessions and save them as learned skills for future use.
---
# Continuous Learning Skill
Automatically evaluates Claude Code sessions on end to extract reusable patterns that can be saved as learned skills.
## How It Works
This skill runs as a **Stop hook** at the end of each session:
1. **Session Evaluation**: Checks if session has enough messages (default: 10+)
2. **Pattern Detection**: Identifies extractable patterns from the session
3. **Skill Extraction**: Saves useful patterns to `~/.claude/skills/learned/`
## Configuration
Edit `config.json` to customize:
```json
{
"min_session_length": 10,
"extraction_threshold": "medium",
"auto_approve": false,
"learned_skills_path": "~/.claude/skills/learned/",
"patterns_to_detect": [
"error_resolution",
"user_corrections",
"workarounds",
"debugging_techniques",
"project_specific"
],
"ignore_patterns": [
"simple_typos",
"one_time_fixes",
"external_api_issues"
]
}
```
## Pattern Types
| Pattern | Description |
|---------|-------------|
| `error_resolution` | How specific errors were resolved |
| `user_corrections` | Patterns from user corrections |
| `workarounds` | Solutions to framework/library quirks |
| `debugging_techniques` | Effective debugging approaches |
| `project_specific` | Project-specific conventions |
## Hook Setup
Add to your `~/.claude/settings.json`:
```json
{
"hooks": {
"Stop": [{
"matcher": "*",
"hooks": [{
"type": "command",
"command": "~/.claude/skills/continuous-learning/evaluate-session.sh"
}]
}]
}
}
```
## Why Stop Hook?
- **Lightweight**: Runs once at session end
- **Non-blocking**: Doesn't add latency to every message
- **Complete context**: Has access to full session transcript
## Related
- [The Longform Guide](https://x.com/affaanmustafa/status/2014040193557471352) - Section on continuous learning
- `/learn` command - Manual pattern extraction mid-session
---
## Comparison Notes (Research: Jan 2025)
### vs Homunculus (github.com/humanplane/homunculus)
Homunculus v2 takes a more sophisticated approach:
| Feature | Our Approach | Homunculus v2 |
|---------|--------------|---------------|
| Observation | Stop hook (end of session) | PreToolUse/PostToolUse hooks (100% reliable) |
| Analysis | Main context | Background agent (Haiku) |
| Granularity | Full skills | Atomic "instincts" |
| Confidence | None | 0.3-0.9 weighted |
| Evolution | Direct to skill | Instincts → cluster → skill/command/agent |
| Sharing | None | Export/import instincts |
**Key insight from homunculus:**
> "v1 relied on skills to observe. Skills are probabilistic—they fire ~50-80% of the time. v2 uses hooks for observation (100% reliable) and instincts as the atomic unit of learned behavior."
### Potential v2 Enhancements
1. **Instinct-based learning** - Smaller, atomic behaviors with confidence scoring
2. **Background observer** - Haiku agent analyzing in parallel
3. **Confidence decay** - Instincts lose confidence if contradicted
4. **Domain tagging** - code-style, testing, git, debugging, etc.
5. **Evolution path** - Cluster related instincts into skills/commands
See: `/Users/affoon/Documents/tasks/12-continuous-learning-v2.md` for full spec.

View File

@@ -1,18 +0,0 @@
{
"min_session_length": 10,
"extraction_threshold": "medium",
"auto_approve": false,
"learned_skills_path": "~/.claude/skills/learned/",
"patterns_to_detect": [
"error_resolution",
"user_corrections",
"workarounds",
"debugging_techniques",
"project_specific"
],
"ignore_patterns": [
"simple_typos",
"one_time_fixes",
"external_api_issues"
]
}

View File

@@ -1,69 +0,0 @@
#!/bin/bash
# Continuous Learning - Session Evaluator
# Runs on Stop hook to extract reusable patterns from Claude Code sessions
#
# Why Stop hook instead of UserPromptSubmit:
# - Stop runs once at session end (lightweight)
# - UserPromptSubmit runs every message (heavy, adds latency)
#
# Hook config (in ~/.claude/settings.json):
# {
# "hooks": {
# "Stop": [{
# "matcher": "*",
# "hooks": [{
# "type": "command",
# "command": "~/.claude/skills/continuous-learning/evaluate-session.sh"
# }]
# }]
# }
# }
#
# Patterns to detect: error_resolution, debugging_techniques, workarounds, project_specific
# Patterns to ignore: simple_typos, one_time_fixes, external_api_issues
# Extracted skills saved to: ~/.claude/skills/learned/
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CONFIG_FILE="$SCRIPT_DIR/config.json"
LEARNED_SKILLS_PATH="${HOME}/.claude/skills/learned"
MIN_SESSION_LENGTH=10
# Load config if exists
if [ -f "$CONFIG_FILE" ]; then
if ! command -v jq &>/dev/null; then
echo "[ContinuousLearning] jq is required to parse config.json but not installed, using defaults" >&2
else
MIN_SESSION_LENGTH=$(jq -r '.min_session_length // 10' "$CONFIG_FILE")
LEARNED_SKILLS_PATH=$(jq -r '.learned_skills_path // "~/.claude/skills/learned/"' "$CONFIG_FILE" | sed "s|~|$HOME|")
fi
fi
# Ensure learned skills directory exists
mkdir -p "$LEARNED_SKILLS_PATH"
# Get transcript path from stdin JSON (Claude Code hook input)
# Falls back to env var for backwards compatibility
stdin_data=$(cat)
transcript_path=$(echo "$stdin_data" | grep -o '"transcript_path":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -z "$transcript_path" ]; then
transcript_path="${CLAUDE_TRANSCRIPT_PATH:-}"
fi
if [ -z "$transcript_path" ] || [ ! -f "$transcript_path" ]; then
exit 0
fi
# Count messages in session
message_count=$(grep -c '"type":"user"' "$transcript_path" 2>/dev/null || echo "0")
# Skip short sessions
if [ "$message_count" -lt "$MIN_SESSION_LENGTH" ]; then
echo "[ContinuousLearning] Session too short ($message_count messages), skipping" >&2
exit 0
fi
# Signal to Claude that session should be evaluated for extractable patterns
echo "[ContinuousLearning] Session has $message_count messages - evaluate for extractable patterns" >&2
echo "[ContinuousLearning] Save learned skills to: $LEARNED_SKILLS_PATH" >&2

View File

@@ -1,722 +0,0 @@
---
name: cpp-coding-standards
description: C++ coding standards based on the C++ Core Guidelines (isocpp.github.io). Use when writing, reviewing, or refactoring C++ code to enforce modern, safe, and idiomatic practices.
---
# C++ Coding Standards (C++ Core Guidelines)
Comprehensive coding standards for modern C++ (C++17/20/23) derived from the [C++ Core Guidelines](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines). Enforces type safety, resource safety, immutability, and clarity.
## When to Use
- Writing new C++ code (classes, functions, templates)
- Reviewing or refactoring existing C++ code
- Making architectural decisions in C++ projects
- Enforcing consistent style across a C++ codebase
- Choosing between language features (e.g., `enum` vs `enum class`, raw pointer vs smart pointer)
### When NOT to Use
- Non-C++ projects
- Legacy C codebases that cannot adopt modern C++ features
- Embedded/bare-metal contexts where specific guidelines conflict with hardware constraints (adapt selectively)
## Cross-Cutting Principles
These themes recur across the entire guidelines and form the foundation:
1. **RAII everywhere** (P.8, R.1, E.6, CP.20): Bind resource lifetime to object lifetime
2. **Immutability by default** (P.10, Con.1-5, ES.25): Start with `const`/`constexpr`; mutability is the exception
3. **Type safety** (P.4, I.4, ES.46-49, Enum.3): Use the type system to prevent errors at compile time
4. **Express intent** (P.3, F.1, NL.1-2, T.10): Names, types, and concepts should communicate purpose
5. **Minimize complexity** (F.2-3, ES.5, Per.4-5): Simple code is correct code
6. **Value semantics over pointer semantics** (C.10, R.3-5, F.20, CP.31): Prefer returning by value and scoped objects
## Philosophy & Interfaces (P.*, I.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **P.1** | Express ideas directly in code |
| **P.3** | Express intent |
| **P.4** | Ideally, a program should be statically type safe |
| **P.5** | Prefer compile-time checking to run-time checking |
| **P.8** | Don't leak any resources |
| **P.10** | Prefer immutable data to mutable data |
| **I.1** | Make interfaces explicit |
| **I.2** | Avoid non-const global variables |
| **I.4** | Make interfaces precisely and strongly typed |
| **I.11** | Never transfer ownership by a raw pointer or reference |
| **I.23** | Keep the number of function arguments low |
### DO
```cpp
// P.10 + I.4: Immutable, strongly typed interface
struct Temperature {
double kelvin;
};
Temperature boil(const Temperature& water);
```
### DON'T
```cpp
// Weak interface: unclear ownership, unclear units
double boil(double* temp);
// Non-const global variable
int g_counter = 0; // I.2 violation
```
## Functions (F.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **F.1** | Package meaningful operations as carefully named functions |
| **F.2** | A function should perform a single logical operation |
| **F.3** | Keep functions short and simple |
| **F.4** | If a function might be evaluated at compile time, declare it `constexpr` |
| **F.6** | If your function must not throw, declare it `noexcept` |
| **F.8** | Prefer pure functions |
| **F.16** | For "in" parameters, pass cheaply-copied types by value and others by `const&` |
| **F.20** | For "out" values, prefer return values to output parameters |
| **F.21** | To return multiple "out" values, prefer returning a struct |
| **F.43** | Never return a pointer or reference to a local object |
### Parameter Passing
```cpp
// F.16: Cheap types by value, others by const&
void print(int x); // cheap: by value
void analyze(const std::string& data); // expensive: by const&
void transform(std::string s); // sink: by value (will move)
// F.20 + F.21: Return values, not output parameters
struct ParseResult {
std::string token;
int position;
};
ParseResult parse(std::string_view input); // GOOD: return struct
// BAD: output parameters
void parse(std::string_view input,
std::string& token, int& pos); // avoid this
```
### Pure Functions and constexpr
```cpp
// F.4 + F.8: Pure, constexpr where possible
constexpr int factorial(int n) noexcept {
return (n <= 1) ? 1 : n * factorial(n - 1);
}
static_assert(factorial(5) == 120);
```
### Anti-Patterns
- Returning `T&&` from functions (F.45)
- Using `va_arg` / C-style variadics (F.55)
- Capturing by reference in lambdas passed to other threads (F.53)
- Returning `const T` which inhibits move semantics (F.49)
## Classes & Class Hierarchies (C.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **C.2** | Use `class` if invariant exists; `struct` if data members vary independently |
| **C.9** | Minimize exposure of members |
| **C.20** | If you can avoid defining default operations, do (Rule of Zero) |
| **C.21** | If you define or `=delete` any copy/move/destructor, handle them all (Rule of Five) |
| **C.35** | Base class destructor: public virtual or protected non-virtual |
| **C.41** | A constructor should create a fully initialized object |
| **C.46** | Declare single-argument constructors `explicit` |
| **C.67** | A polymorphic class should suppress public copy/move |
| **C.128** | Virtual functions: specify exactly one of `virtual`, `override`, or `final` |
### Rule of Zero
```cpp
// C.20: Let the compiler generate special members
struct Employee {
std::string name;
std::string department;
int id;
// No destructor, copy/move constructors, or assignment operators needed
};
```
### Rule of Five
```cpp
// C.21: If you must manage a resource, define all five
class Buffer {
public:
explicit Buffer(std::size_t size)
: data_(std::make_unique<char[]>(size)), size_(size) {}
~Buffer() = default;
Buffer(const Buffer& other)
: data_(std::make_unique<char[]>(other.size_)), size_(other.size_) {
std::copy_n(other.data_.get(), size_, data_.get());
}
Buffer& operator=(const Buffer& other) {
if (this != &other) {
auto new_data = std::make_unique<char[]>(other.size_);
std::copy_n(other.data_.get(), other.size_, new_data.get());
data_ = std::move(new_data);
size_ = other.size_;
}
return *this;
}
Buffer(Buffer&&) noexcept = default;
Buffer& operator=(Buffer&&) noexcept = default;
private:
std::unique_ptr<char[]> data_;
std::size_t size_;
};
```
### Class Hierarchy
```cpp
// C.35 + C.128: Virtual destructor, use override
class Shape {
public:
virtual ~Shape() = default;
virtual double area() const = 0; // C.121: pure interface
};
class Circle : public Shape {
public:
explicit Circle(double r) : radius_(r) {}
double area() const override { return 3.14159 * radius_ * radius_; }
private:
double radius_;
};
```
### Anti-Patterns
- Calling virtual functions in constructors/destructors (C.82)
- Using `memset`/`memcpy` on non-trivial types (C.90)
- Providing different default arguments for virtual function and overrider (C.140)
- Making data members `const` or references, which suppresses move/copy (C.12)
## Resource Management (R.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **R.1** | Manage resources automatically using RAII |
| **R.3** | A raw pointer (`T*`) is non-owning |
| **R.5** | Prefer scoped objects; don't heap-allocate unnecessarily |
| **R.10** | Avoid `malloc()`/`free()` |
| **R.11** | Avoid calling `new` and `delete` explicitly |
| **R.20** | Use `unique_ptr` or `shared_ptr` to represent ownership |
| **R.21** | Prefer `unique_ptr` over `shared_ptr` unless sharing ownership |
| **R.22** | Use `make_shared()` to make `shared_ptr`s |
### Smart Pointer Usage
```cpp
// R.11 + R.20 + R.21: RAII with smart pointers
auto widget = std::make_unique<Widget>("config"); // unique ownership
auto cache = std::make_shared<Cache>(1024); // shared ownership
// R.3: Raw pointer = non-owning observer
void render(const Widget* w) { // does NOT own w
if (w) w->draw();
}
render(widget.get());
```
### RAII Pattern
```cpp
// R.1: Resource acquisition is initialization
class FileHandle {
public:
explicit FileHandle(const std::string& path)
: handle_(std::fopen(path.c_str(), "r")) {
if (!handle_) throw std::runtime_error("Failed to open: " + path);
}
~FileHandle() {
if (handle_) std::fclose(handle_);
}
FileHandle(const FileHandle&) = delete;
FileHandle& operator=(const FileHandle&) = delete;
FileHandle(FileHandle&& other) noexcept
: handle_(std::exchange(other.handle_, nullptr)) {}
FileHandle& operator=(FileHandle&& other) noexcept {
if (this != &other) {
if (handle_) std::fclose(handle_);
handle_ = std::exchange(other.handle_, nullptr);
}
return *this;
}
private:
std::FILE* handle_;
};
```
### Anti-Patterns
- Naked `new`/`delete` (R.11)
- `malloc()`/`free()` in C++ code (R.10)
- Multiple resource allocations in a single expression (R.13 -- exception safety hazard)
- `shared_ptr` where `unique_ptr` suffices (R.21)
## Expressions & Statements (ES.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **ES.5** | Keep scopes small |
| **ES.20** | Always initialize an object |
| **ES.23** | Prefer `{}` initializer syntax |
| **ES.25** | Declare objects `const` or `constexpr` unless modification is intended |
| **ES.28** | Use lambdas for complex initialization of `const` variables |
| **ES.45** | Avoid magic constants; use symbolic constants |
| **ES.46** | Avoid narrowing/lossy arithmetic conversions |
| **ES.47** | Use `nullptr` rather than `0` or `NULL` |
| **ES.48** | Avoid casts |
| **ES.50** | Don't cast away `const` |
### Initialization
```cpp
// ES.20 + ES.23 + ES.25: Always initialize, prefer {}, default to const
const int max_retries{3};
const std::string name{"widget"};
const std::vector<int> primes{2, 3, 5, 7, 11};
// ES.28: Lambda for complex const initialization
const auto config = [&] {
Config c;
c.timeout = std::chrono::seconds{30};
c.retries = max_retries;
c.verbose = debug_mode;
return c;
}();
```
### Anti-Patterns
- Uninitialized variables (ES.20)
- Using `0` or `NULL` as pointer (ES.47 -- use `nullptr`)
- C-style casts (ES.48 -- use `static_cast`, `const_cast`, etc.)
- Casting away `const` (ES.50)
- Magic numbers without named constants (ES.45)
- Mixing signed and unsigned arithmetic (ES.100)
- Reusing names in nested scopes (ES.12)
## Error Handling (E.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **E.1** | Develop an error-handling strategy early in a design |
| **E.2** | Throw an exception to signal that a function can't perform its assigned task |
| **E.6** | Use RAII to prevent leaks |
| **E.12** | Use `noexcept` when throwing is impossible or unacceptable |
| **E.14** | Use purpose-designed user-defined types as exceptions |
| **E.15** | Throw by value, catch by reference |
| **E.16** | Destructors, deallocation, and swap must never fail |
| **E.17** | Don't try to catch every exception in every function |
### Exception Hierarchy
```cpp
// E.14 + E.15: Custom exception types, throw by value, catch by reference
class AppError : public std::runtime_error {
public:
using std::runtime_error::runtime_error;
};
class NetworkError : public AppError {
public:
NetworkError(const std::string& msg, int code)
: AppError(msg), status_code(code) {}
int status_code;
};
void fetch_data(const std::string& url) {
// E.2: Throw to signal failure
throw NetworkError("connection refused", 503);
}
void run() {
try {
fetch_data("https://api.example.com");
} catch (const NetworkError& e) {
log_error(e.what(), e.status_code);
} catch (const AppError& e) {
log_error(e.what());
}
// E.17: Don't catch everything here -- let unexpected errors propagate
}
```
### Anti-Patterns
- Throwing built-in types like `int` or string literals (E.14)
- Catching by value (slicing risk) (E.15)
- Empty catch blocks that silently swallow errors
- Using exceptions for flow control (E.3)
- Error handling based on global state like `errno` (E.28)
## Constants & Immutability (Con.*)
### All Rules
| Rule | Summary |
|------|---------|
| **Con.1** | By default, make objects immutable |
| **Con.2** | By default, make member functions `const` |
| **Con.3** | By default, pass pointers and references to `const` |
| **Con.4** | Use `const` for values that don't change after construction |
| **Con.5** | Use `constexpr` for values computable at compile time |
```cpp
// Con.1 through Con.5: Immutability by default
class Sensor {
public:
explicit Sensor(std::string id) : id_(std::move(id)) {}
// Con.2: const member functions by default
const std::string& id() const { return id_; }
double last_reading() const { return reading_; }
// Only non-const when mutation is required
void record(double value) { reading_ = value; }
private:
const std::string id_; // Con.4: never changes after construction
double reading_{0.0};
};
// Con.3: Pass by const reference
void display(const Sensor& s) {
std::cout << s.id() << ": " << s.last_reading() << '\n';
}
// Con.5: Compile-time constants
constexpr double PI = 3.14159265358979;
constexpr int MAX_SENSORS = 256;
```
## Concurrency & Parallelism (CP.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **CP.2** | Avoid data races |
| **CP.3** | Minimize explicit sharing of writable data |
| **CP.4** | Think in terms of tasks, rather than threads |
| **CP.8** | Don't use `volatile` for synchronization |
| **CP.20** | Use RAII, never plain `lock()`/`unlock()` |
| **CP.21** | Use `std::scoped_lock` to acquire multiple mutexes |
| **CP.22** | Never call unknown code while holding a lock |
| **CP.42** | Don't wait without a condition |
| **CP.44** | Remember to name your `lock_guard`s and `unique_lock`s |
| **CP.100** | Don't use lock-free programming unless you absolutely have to |
### Safe Locking
```cpp
// CP.20 + CP.44: RAII locks, always named
class ThreadSafeQueue {
public:
void push(int value) {
std::lock_guard<std::mutex> lock(mutex_); // CP.44: named!
queue_.push(value);
cv_.notify_one();
}
int pop() {
std::unique_lock<std::mutex> lock(mutex_);
// CP.42: Always wait with a condition
cv_.wait(lock, [this] { return !queue_.empty(); });
const int value = queue_.front();
queue_.pop();
return value;
}
private:
std::mutex mutex_; // CP.50: mutex with its data
std::condition_variable cv_;
std::queue<int> queue_;
};
```
### Multiple Mutexes
```cpp
// CP.21: std::scoped_lock for multiple mutexes (deadlock-free)
void transfer(Account& from, Account& to, double amount) {
std::scoped_lock lock(from.mutex_, to.mutex_);
from.balance_ -= amount;
to.balance_ += amount;
}
```
### Anti-Patterns
- `volatile` for synchronization (CP.8 -- it's for hardware I/O only)
- Detaching threads (CP.26 -- lifetime management becomes nearly impossible)
- Unnamed lock guards: `std::lock_guard<std::mutex>(m);` destroys immediately (CP.44)
- Holding locks while calling callbacks (CP.22 -- deadlock risk)
- Lock-free programming without deep expertise (CP.100)
## Templates & Generic Programming (T.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **T.1** | Use templates to raise the level of abstraction |
| **T.2** | Use templates to express algorithms for many argument types |
| **T.10** | Specify concepts for all template arguments |
| **T.11** | Use standard concepts whenever possible |
| **T.13** | Prefer shorthand notation for simple concepts |
| **T.43** | Prefer `using` over `typedef` |
| **T.120** | Use template metaprogramming only when you really need to |
| **T.144** | Don't specialize function templates (overload instead) |
### Concepts (C++20)
```cpp
#include <concepts>
// T.10 + T.11: Constrain templates with standard concepts
template<std::integral T>
T gcd(T a, T b) {
while (b != 0) {
a = std::exchange(b, a % b);
}
return a;
}
// T.13: Shorthand concept syntax
void sort(std::ranges::random_access_range auto& range) {
std::ranges::sort(range);
}
// Custom concept for domain-specific constraints
template<typename T>
concept Serializable = requires(const T& t) {
{ t.serialize() } -> std::convertible_to<std::string>;
};
template<Serializable T>
void save(const T& obj, const std::string& path);
```
### Anti-Patterns
- Unconstrained templates in visible namespaces (T.47)
- Specializing function templates instead of overloading (T.144)
- Template metaprogramming where `constexpr` suffices (T.120)
- `typedef` instead of `using` (T.43)
## Standard Library (SL.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **SL.1** | Use libraries wherever possible |
| **SL.2** | Prefer the standard library to other libraries |
| **SL.con.1** | Prefer `std::array` or `std::vector` over C arrays |
| **SL.con.2** | Prefer `std::vector` by default |
| **SL.str.1** | Use `std::string` to own character sequences |
| **SL.str.2** | Use `std::string_view` to refer to character sequences |
| **SL.io.50** | Avoid `endl` (use `'\n'` -- `endl` forces a flush) |
```cpp
// SL.con.1 + SL.con.2: Prefer vector/array over C arrays
const std::array<int, 4> fixed_data{1, 2, 3, 4};
std::vector<std::string> dynamic_data;
// SL.str.1 + SL.str.2: string owns, string_view observes
std::string build_greeting(std::string_view name) {
return "Hello, " + std::string(name) + "!";
}
// SL.io.50: Use '\n' not endl
std::cout << "result: " << value << '\n';
```
## Enumerations (Enum.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **Enum.1** | Prefer enumerations over macros |
| **Enum.3** | Prefer `enum class` over plain `enum` |
| **Enum.5** | Don't use ALL_CAPS for enumerators |
| **Enum.6** | Avoid unnamed enumerations |
```cpp
// Enum.3 + Enum.5: Scoped enum, no ALL_CAPS
enum class Color { red, green, blue };
enum class LogLevel { debug, info, warning, error };
// BAD: plain enum leaks names, ALL_CAPS clashes with macros
enum { RED, GREEN, BLUE }; // Enum.3 + Enum.5 + Enum.6 violation
#define MAX_SIZE 100 // Enum.1 violation -- use constexpr
```
## Source Files & Naming (SF.*, NL.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **SF.1** | Use `.cpp` for code files and `.h` for interface files |
| **SF.7** | Don't write `using namespace` at global scope in a header |
| **SF.8** | Use `#include` guards for all `.h` files |
| **SF.11** | Header files should be self-contained |
| **NL.5** | Avoid encoding type information in names (no Hungarian notation) |
| **NL.8** | Use a consistent naming style |
| **NL.9** | Use ALL_CAPS for macro names only |
| **NL.10** | Prefer `underscore_style` names |
### Header Guard
```cpp
// SF.8: Include guard (or #pragma once)
#ifndef PROJECT_MODULE_WIDGET_H
#define PROJECT_MODULE_WIDGET_H
// SF.11: Self-contained -- include everything this header needs
#include <string>
#include <vector>
namespace project::module {
class Widget {
public:
explicit Widget(std::string name);
const std::string& name() const;
private:
std::string name_;
};
} // namespace project::module
#endif // PROJECT_MODULE_WIDGET_H
```
### Naming Conventions
```cpp
// NL.8 + NL.10: Consistent underscore_style
namespace my_project {
constexpr int max_buffer_size = 4096; // NL.9: not ALL_CAPS (it's not a macro)
class tcp_connection { // underscore_style class
public:
void send_message(std::string_view msg);
bool is_connected() const;
private:
std::string host_; // trailing underscore for members
int port_;
};
} // namespace my_project
```
### Anti-Patterns
- `using namespace std;` in a header at global scope (SF.7)
- Headers that depend on inclusion order (SF.10, SF.11)
- Hungarian notation like `strName`, `iCount` (NL.5)
- ALL_CAPS for anything other than macros (NL.9)
## Performance (Per.*)
### Key Rules
| Rule | Summary |
|------|---------|
| **Per.1** | Don't optimize without reason |
| **Per.2** | Don't optimize prematurely |
| **Per.6** | Don't make claims about performance without measurements |
| **Per.7** | Design to enable optimization |
| **Per.10** | Rely on the static type system |
| **Per.11** | Move computation from run time to compile time |
| **Per.19** | Access memory predictably |
### Guidelines
```cpp
// Per.11: Compile-time computation where possible
constexpr auto lookup_table = [] {
std::array<int, 256> table{};
for (int i = 0; i < 256; ++i) {
table[i] = i * i;
}
return table;
}();
// Per.19: Prefer contiguous data for cache-friendliness
std::vector<Point> points; // GOOD: contiguous
std::vector<std::unique_ptr<Point>> indirect_points; // BAD: pointer chasing
```
### Anti-Patterns
- Optimizing without profiling data (Per.1, Per.6)
- Choosing "clever" low-level code over clear abstractions (Per.4, Per.5)
- Ignoring data layout and cache behavior (Per.19)
## Quick Reference Checklist
Before marking C++ work complete:
- [ ] No raw `new`/`delete` -- use smart pointers or RAII (R.11)
- [ ] Objects initialized at declaration (ES.20)
- [ ] Variables are `const`/`constexpr` by default (Con.1, ES.25)
- [ ] Member functions are `const` where possible (Con.2)
- [ ] `enum class` instead of plain `enum` (Enum.3)
- [ ] `nullptr` instead of `0`/`NULL` (ES.47)
- [ ] No narrowing conversions (ES.46)
- [ ] No C-style casts (ES.48)
- [ ] Single-argument constructors are `explicit` (C.46)
- [ ] Rule of Zero or Rule of Five applied (C.20, C.21)
- [ ] Base class destructors are public virtual or protected non-virtual (C.35)
- [ ] Templates are constrained with concepts (T.10)
- [ ] No `using namespace` in headers at global scope (SF.7)
- [ ] Headers have include guards and are self-contained (SF.8, SF.11)
- [ ] Locks use RAII (`scoped_lock`/`lock_guard`) (CP.20)
- [ ] Exceptions are custom types, thrown by value, caught by reference (E.14, E.15)
- [ ] `'\n'` instead of `std::endl` (SL.io.50)
- [ ] No magic numbers (ES.45)

View File

@@ -1,322 +0,0 @@
---
name: cpp-testing
description: Use only when writing/updating/fixing C++ tests, configuring GoogleTest/CTest, diagnosing failing or flaky tests, or adding coverage/sanitizers.
---
# C++ Testing (Agent Skill)
Agent-focused testing workflow for modern C++ (C++17/20) using GoogleTest/GoogleMock with CMake/CTest.
## When to Use
- Writing new C++ tests or fixing existing tests
- Designing unit/integration test coverage for C++ components
- Adding test coverage, CI gating, or regression protection
- Configuring CMake/CTest workflows for consistent execution
- Investigating test failures or flaky behavior
- Enabling sanitizers for memory/race diagnostics
### When NOT to Use
- Implementing new product features without test changes
- Large-scale refactors unrelated to test coverage or failures
- Performance tuning without test regressions to validate
- Non-C++ projects or non-test tasks
## Core Concepts
- **TDD loop**: red → green → refactor (tests first, minimal fix, then cleanups).
- **Isolation**: prefer dependency injection and fakes over global state.
- **Test layout**: `tests/unit`, `tests/integration`, `tests/testdata`.
- **Mocks vs fakes**: mock for interactions, fake for stateful behavior.
- **CTest discovery**: use `gtest_discover_tests()` for stable test discovery.
- **CI signal**: run subset first, then full suite with `--output-on-failure`.
## TDD Workflow
Follow the RED → GREEN → REFACTOR loop:
1. **RED**: write a failing test that captures the new behavior
2. **GREEN**: implement the smallest change to pass
3. **REFACTOR**: clean up while tests stay green
```cpp
// tests/add_test.cpp
#include <gtest/gtest.h>
int Add(int a, int b); // Provided by production code.
TEST(AddTest, AddsTwoNumbers) { // RED
EXPECT_EQ(Add(2, 3), 5);
}
// src/add.cpp
int Add(int a, int b) { // GREEN
return a + b;
}
// REFACTOR: simplify/rename once tests pass
```
## Code Examples
### Basic Unit Test (gtest)
```cpp
// tests/calculator_test.cpp
#include <gtest/gtest.h>
int Add(int a, int b); // Provided by production code.
TEST(CalculatorTest, AddsTwoNumbers) {
EXPECT_EQ(Add(2, 3), 5);
}
```
### Fixture (gtest)
```cpp
// tests/user_store_test.cpp
// Pseudocode stub: replace UserStore/User with project types.
#include <gtest/gtest.h>
#include <memory>
#include <optional>
#include <string>
struct User { std::string name; };
class UserStore {
public:
explicit UserStore(std::string /*path*/) {}
void Seed(std::initializer_list<User> /*users*/) {}
std::optional<User> Find(const std::string &/*name*/) { return User{"alice"}; }
};
class UserStoreTest : public ::testing::Test {
protected:
void SetUp() override {
store = std::make_unique<UserStore>(":memory:");
store->Seed({{"alice"}, {"bob"}});
}
std::unique_ptr<UserStore> store;
};
TEST_F(UserStoreTest, FindsExistingUser) {
auto user = store->Find("alice");
ASSERT_TRUE(user.has_value());
EXPECT_EQ(user->name, "alice");
}
```
### Mock (gmock)
```cpp
// tests/notifier_test.cpp
#include <gmock/gmock.h>
#include <gtest/gtest.h>
#include <string>
class Notifier {
public:
virtual ~Notifier() = default;
virtual void Send(const std::string &message) = 0;
};
class MockNotifier : public Notifier {
public:
MOCK_METHOD(void, Send, (const std::string &message), (override));
};
class Service {
public:
explicit Service(Notifier &notifier) : notifier_(notifier) {}
void Publish(const std::string &message) { notifier_.Send(message); }
private:
Notifier &notifier_;
};
TEST(ServiceTest, SendsNotifications) {
MockNotifier notifier;
Service service(notifier);
EXPECT_CALL(notifier, Send("hello")).Times(1);
service.Publish("hello");
}
```
### CMake/CTest Quickstart
```cmake
# CMakeLists.txt (excerpt)
cmake_minimum_required(VERSION 3.20)
project(example LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
include(FetchContent)
# Prefer project-locked versions. If using a tag, use a pinned version per project policy.
set(GTEST_VERSION v1.17.0) # Adjust to project policy.
FetchContent_Declare(
googletest
URL https://github.com/google/googletest/archive/refs/tags/${GTEST_VERSION}.zip
)
FetchContent_MakeAvailable(googletest)
add_executable(example_tests
tests/calculator_test.cpp
src/calculator.cpp
)
target_link_libraries(example_tests GTest::gtest GTest::gmock GTest::gtest_main)
enable_testing()
include(GoogleTest)
gtest_discover_tests(example_tests)
```
```bash
cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build -j
ctest --test-dir build --output-on-failure
```
## Running Tests
```bash
ctest --test-dir build --output-on-failure
ctest --test-dir build -R ClampTest
ctest --test-dir build -R "UserStoreTest.*" --output-on-failure
```
```bash
./build/example_tests --gtest_filter=ClampTest.*
./build/example_tests --gtest_filter=UserStoreTest.FindsExistingUser
```
## Debugging Failures
1. Re-run the single failing test with gtest filter.
2. Add scoped logging around the failing assertion.
3. Re-run with sanitizers enabled.
4. Expand to full suite once the root cause is fixed.
## Coverage
Prefer target-level settings instead of global flags.
```cmake
option(ENABLE_COVERAGE "Enable coverage flags" OFF)
if(ENABLE_COVERAGE)
if(CMAKE_CXX_COMPILER_ID MATCHES "GNU")
target_compile_options(example_tests PRIVATE --coverage)
target_link_options(example_tests PRIVATE --coverage)
elseif(CMAKE_CXX_COMPILER_ID MATCHES "Clang")
target_compile_options(example_tests PRIVATE -fprofile-instr-generate -fcoverage-mapping)
target_link_options(example_tests PRIVATE -fprofile-instr-generate)
endif()
endif()
```
GCC + gcov + lcov:
```bash
cmake -S . -B build-cov -DENABLE_COVERAGE=ON
cmake --build build-cov -j
ctest --test-dir build-cov
lcov --capture --directory build-cov --output-file coverage.info
lcov --remove coverage.info '/usr/*' --output-file coverage.info
genhtml coverage.info --output-directory coverage
```
Clang + llvm-cov:
```bash
cmake -S . -B build-llvm -DENABLE_COVERAGE=ON -DCMAKE_CXX_COMPILER=clang++
cmake --build build-llvm -j
LLVM_PROFILE_FILE="build-llvm/default.profraw" ctest --test-dir build-llvm
llvm-profdata merge -sparse build-llvm/default.profraw -o build-llvm/default.profdata
llvm-cov report build-llvm/example_tests -instr-profile=build-llvm/default.profdata
```
## Sanitizers
```cmake
option(ENABLE_ASAN "Enable AddressSanitizer" OFF)
option(ENABLE_UBSAN "Enable UndefinedBehaviorSanitizer" OFF)
option(ENABLE_TSAN "Enable ThreadSanitizer" OFF)
if(ENABLE_ASAN)
add_compile_options(-fsanitize=address -fno-omit-frame-pointer)
add_link_options(-fsanitize=address)
endif()
if(ENABLE_UBSAN)
add_compile_options(-fsanitize=undefined -fno-omit-frame-pointer)
add_link_options(-fsanitize=undefined)
endif()
if(ENABLE_TSAN)
add_compile_options(-fsanitize=thread)
add_link_options(-fsanitize=thread)
endif()
```
## Flaky Tests Guardrails
- Never use `sleep` for synchronization; use condition variables or latches.
- Make temp directories unique per test and always clean them.
- Avoid real time, network, or filesystem dependencies in unit tests.
- Use deterministic seeds for randomized inputs.
## Best Practices
### DO
- Keep tests deterministic and isolated
- Prefer dependency injection over globals
- Use `ASSERT_*` for preconditions, `EXPECT_*` for multiple checks
- Separate unit vs integration tests in CTest labels or directories
- Run sanitizers in CI for memory and race detection
### DON'T
- Don't depend on real time or network in unit tests
- Don't use sleeps as synchronization when a condition variable can be used
- Don't over-mock simple value objects
- Don't use brittle string matching for non-critical logs
### Common Pitfalls
- **Using fixed temp paths** → Generate unique temp directories per test and clean them.
- **Relying on wall clock time** → Inject a clock or use fake time sources.
- **Flaky concurrency tests** → Use condition variables/latches and bounded waits.
- **Hidden global state** → Reset global state in fixtures or remove globals.
- **Over-mocking** → Prefer fakes for stateful behavior and only mock interactions.
- **Missing sanitizer runs** → Add ASan/UBSan/TSan builds in CI.
- **Coverage on debug-only builds** → Ensure coverage targets use consistent flags.
## Optional Appendix: Fuzzing / Property Testing
Only use if the project already supports LLVM/libFuzzer or a property-testing library.
- **libFuzzer**: best for pure functions with minimal I/O.
- **RapidCheck**: property-based tests to validate invariants.
Minimal libFuzzer harness (pseudocode: replace ParseConfig):
```cpp
#include <cstddef>
#include <cstdint>
#include <string>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
std::string input(reinterpret_cast<const char *>(data), size);
// ParseConfig(input); // project function
return 0;
}
```
## Alternatives to GoogleTest
- **Catch2**: header-only, expressive matchers
- **doctest**: lightweight, minimal compile overhead

View File

@@ -1,733 +0,0 @@
---
name: django-patterns
description: Django architecture patterns, REST API design with DRF, ORM best practices, caching, signals, middleware, and production-grade Django apps.
---
# Django Development Patterns
Production-grade Django architecture patterns for scalable, maintainable applications.
## When to Activate
- Building Django web applications
- Designing Django REST Framework APIs
- Working with Django ORM and models
- Setting up Django project structure
- Implementing caching, signals, middleware
## Project Structure
### Recommended Layout
```
myproject/
├── config/
│ ├── __init__.py
│ ├── settings/
│ │ ├── __init__.py
│ │ ├── base.py # Base settings
│ │ ├── development.py # Dev settings
│ │ ├── production.py # Production settings
│ │ └── test.py # Test settings
│ ├── urls.py
│ ├── wsgi.py
│ └── asgi.py
├── manage.py
└── apps/
├── __init__.py
├── users/
│ ├── __init__.py
│ ├── models.py
│ ├── views.py
│ ├── serializers.py
│ ├── urls.py
│ ├── permissions.py
│ ├── filters.py
│ ├── services.py
│ └── tests/
└── products/
└── ...
```
### Split Settings Pattern
```python
# config/settings/base.py
from pathlib import Path
BASE_DIR = Path(__file__).resolve().parent.parent.parent
SECRET_KEY = env('DJANGO_SECRET_KEY')
DEBUG = False
ALLOWED_HOSTS = []
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'rest_framework.authtoken',
'corsheaders',
# Local apps
'apps.users',
'apps.products',
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'corsheaders.middleware.CorsMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
ROOT_URLCONF = 'config.urls'
WSGI_APPLICATION = 'config.wsgi.application'
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': env('DB_NAME'),
'USER': env('DB_USER'),
'PASSWORD': env('DB_PASSWORD'),
'HOST': env('DB_HOST'),
'PORT': env('DB_PORT', default='5432'),
}
}
# config/settings/development.py
from .base import *
DEBUG = True
ALLOWED_HOSTS = ['localhost', '127.0.0.1']
DATABASES['default']['NAME'] = 'myproject_dev'
INSTALLED_APPS += ['debug_toolbar']
MIDDLEWARE += ['debug_toolbar.middleware.DebugToolbarMiddleware']
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
# config/settings/production.py
from .base import *
DEBUG = False
ALLOWED_HOSTS = env.list('ALLOWED_HOSTS')
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SECURE_HSTS_SECONDS = 31536000
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True
# Logging
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'WARNING',
'class': 'logging.FileHandler',
'filename': '/var/log/django/django.log',
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'WARNING',
'propagate': True,
},
},
}
```
## Model Design Patterns
### Model Best Practices
```python
from django.db import models
from django.contrib.auth.models import AbstractUser
from django.core.validators import MinValueValidator, MaxValueValidator
class User(AbstractUser):
"""Custom user model extending AbstractUser."""
email = models.EmailField(unique=True)
phone = models.CharField(max_length=20, blank=True)
birth_date = models.DateField(null=True, blank=True)
USERNAME_FIELD = 'email'
REQUIRED_FIELDS = ['username']
class Meta:
db_table = 'users'
verbose_name = 'user'
verbose_name_plural = 'users'
ordering = ['-date_joined']
def __str__(self):
return self.email
def get_full_name(self):
return f"{self.first_name} {self.last_name}".strip()
class Product(models.Model):
"""Product model with proper field configuration."""
name = models.CharField(max_length=200)
slug = models.SlugField(unique=True, max_length=250)
description = models.TextField(blank=True)
price = models.DecimalField(
max_digits=10,
decimal_places=2,
validators=[MinValueValidator(0)]
)
stock = models.PositiveIntegerField(default=0)
is_active = models.BooleanField(default=True)
category = models.ForeignKey(
'Category',
on_delete=models.CASCADE,
related_name='products'
)
tags = models.ManyToManyField('Tag', blank=True, related_name='products')
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
class Meta:
db_table = 'products'
ordering = ['-created_at']
indexes = [
models.Index(fields=['slug']),
models.Index(fields=['-created_at']),
models.Index(fields=['category', 'is_active']),
]
constraints = [
models.CheckConstraint(
check=models.Q(price__gte=0),
name='price_non_negative'
)
]
def __str__(self):
return self.name
def save(self, *args, **kwargs):
if not self.slug:
self.slug = slugify(self.name)
super().save(*args, **kwargs)
```
### QuerySet Best Practices
```python
from django.db import models
class ProductQuerySet(models.QuerySet):
"""Custom QuerySet for Product model."""
def active(self):
"""Return only active products."""
return self.filter(is_active=True)
def with_category(self):
"""Select related category to avoid N+1 queries."""
return self.select_related('category')
def with_tags(self):
"""Prefetch tags for many-to-many relationship."""
return self.prefetch_related('tags')
def in_stock(self):
"""Return products with stock > 0."""
return self.filter(stock__gt=0)
def search(self, query):
"""Search products by name or description."""
return self.filter(
models.Q(name__icontains=query) |
models.Q(description__icontains=query)
)
class Product(models.Model):
# ... fields ...
objects = ProductQuerySet.as_manager() # Use custom QuerySet
# Usage
Product.objects.active().with_category().in_stock()
```
### Manager Methods
```python
class ProductManager(models.Manager):
"""Custom manager for complex queries."""
def get_or_none(self, **kwargs):
"""Return object or None instead of DoesNotExist."""
try:
return self.get(**kwargs)
except self.model.DoesNotExist:
return None
def create_with_tags(self, name, price, tag_names):
"""Create product with associated tags."""
product = self.create(name=name, price=price)
tags = [Tag.objects.get_or_create(name=name)[0] for name in tag_names]
product.tags.set(tags)
return product
def bulk_update_stock(self, product_ids, quantity):
"""Bulk update stock for multiple products."""
return self.filter(id__in=product_ids).update(stock=quantity)
# In model
class Product(models.Model):
# ... fields ...
custom = ProductManager()
```
## Django REST Framework Patterns
### Serializer Patterns
```python
from rest_framework import serializers
from django.contrib.auth.password_validation import validate_password
from .models import Product, User
class ProductSerializer(serializers.ModelSerializer):
"""Serializer for Product model."""
category_name = serializers.CharField(source='category.name', read_only=True)
average_rating = serializers.FloatField(read_only=True)
discount_price = serializers.SerializerMethodField()
class Meta:
model = Product
fields = [
'id', 'name', 'slug', 'description', 'price',
'discount_price', 'stock', 'category_name',
'average_rating', 'created_at'
]
read_only_fields = ['id', 'slug', 'created_at']
def get_discount_price(self, obj):
"""Calculate discount price if applicable."""
if hasattr(obj, 'discount') and obj.discount:
return obj.price * (1 - obj.discount.percent / 100)
return obj.price
def validate_price(self, value):
"""Ensure price is non-negative."""
if value < 0:
raise serializers.ValidationError("Price cannot be negative.")
return value
class ProductCreateSerializer(serializers.ModelSerializer):
"""Serializer for creating products."""
class Meta:
model = Product
fields = ['name', 'description', 'price', 'stock', 'category']
def validate(self, data):
"""Custom validation for multiple fields."""
if data['price'] > 10000 and data['stock'] > 100:
raise serializers.ValidationError(
"Cannot have high-value products with large stock."
)
return data
class UserRegistrationSerializer(serializers.ModelSerializer):
"""Serializer for user registration."""
password = serializers.CharField(
write_only=True,
required=True,
validators=[validate_password],
style={'input_type': 'password'}
)
password_confirm = serializers.CharField(write_only=True, style={'input_type': 'password'})
class Meta:
model = User
fields = ['email', 'username', 'password', 'password_confirm']
def validate(self, data):
"""Validate passwords match."""
if data['password'] != data['password_confirm']:
raise serializers.ValidationError({
"password_confirm": "Password fields didn't match."
})
return data
def create(self, validated_data):
"""Create user with hashed password."""
validated_data.pop('password_confirm')
password = validated_data.pop('password')
user = User.objects.create(**validated_data)
user.set_password(password)
user.save()
return user
```
### ViewSet Patterns
```python
from rest_framework import viewsets, status, filters
from rest_framework.decorators import action
from rest_framework.response import Response
from rest_framework.permissions import IsAuthenticated, IsAdminUser
from django_filters.rest_framework import DjangoFilterBackend
from .models import Product
from .serializers import ProductSerializer, ProductCreateSerializer
from .permissions import IsOwnerOrReadOnly
from .filters import ProductFilter
from .services import ProductService
class ProductViewSet(viewsets.ModelViewSet):
"""ViewSet for Product model."""
queryset = Product.objects.select_related('category').prefetch_related('tags')
permission_classes = [IsAuthenticated, IsOwnerOrReadOnly]
filter_backends = [DjangoFilterBackend, filters.SearchFilter, filters.OrderingFilter]
filterset_class = ProductFilter
search_fields = ['name', 'description']
ordering_fields = ['price', 'created_at', 'name']
ordering = ['-created_at']
def get_serializer_class(self):
"""Return appropriate serializer based on action."""
if self.action == 'create':
return ProductCreateSerializer
return ProductSerializer
def perform_create(self, serializer):
"""Save with user context."""
serializer.save(created_by=self.request.user)
@action(detail=False, methods=['get'])
def featured(self, request):
"""Return featured products."""
featured = self.queryset.filter(is_featured=True)[:10]
serializer = self.get_serializer(featured, many=True)
return Response(serializer.data)
@action(detail=True, methods=['post'])
def purchase(self, request, pk=None):
"""Purchase a product."""
product = self.get_object()
service = ProductService()
result = service.purchase(product, request.user)
return Response(result, status=status.HTTP_201_CREATED)
@action(detail=False, methods=['get'], permission_classes=[IsAuthenticated])
def my_products(self, request):
"""Return products created by current user."""
products = self.queryset.filter(created_by=request.user)
page = self.paginate_queryset(products)
serializer = self.get_serializer(page, many=True)
return self.get_paginated_response(serializer.data)
```
### Custom Actions
```python
from rest_framework.decorators import api_view, permission_classes
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response
@api_view(['POST'])
@permission_classes([IsAuthenticated])
def add_to_cart(request):
"""Add product to user cart."""
product_id = request.data.get('product_id')
quantity = request.data.get('quantity', 1)
try:
product = Product.objects.get(id=product_id)
except Product.DoesNotExist:
return Response(
{'error': 'Product not found'},
status=status.HTTP_404_NOT_FOUND
)
cart, _ = Cart.objects.get_or_create(user=request.user)
CartItem.objects.create(
cart=cart,
product=product,
quantity=quantity
)
return Response({'message': 'Added to cart'}, status=status.HTTP_201_CREATED)
```
## Service Layer Pattern
```python
# apps/orders/services.py
from typing import Optional
from django.db import transaction
from .models import Order, OrderItem
class OrderService:
"""Service layer for order-related business logic."""
@staticmethod
@transaction.atomic
def create_order(user, cart: Cart) -> Order:
"""Create order from cart."""
order = Order.objects.create(
user=user,
total_price=cart.total_price
)
for item in cart.items.all():
OrderItem.objects.create(
order=order,
product=item.product,
quantity=item.quantity,
price=item.product.price
)
# Clear cart
cart.items.all().delete()
return order
@staticmethod
def process_payment(order: Order, payment_data: dict) -> bool:
"""Process payment for order."""
# Integration with payment gateway
payment = PaymentGateway.charge(
amount=order.total_price,
token=payment_data['token']
)
if payment.success:
order.status = Order.Status.PAID
order.save()
# Send confirmation email
OrderService.send_confirmation_email(order)
return True
return False
@staticmethod
def send_confirmation_email(order: Order):
"""Send order confirmation email."""
# Email sending logic
pass
```
## Caching Strategies
### View-Level Caching
```python
from django.views.decorators.cache import cache_page
from django.utils.decorators import method_decorator
@method_decorator(cache_page(60 * 15), name='dispatch') # 15 minutes
class ProductListView(generic.ListView):
model = Product
template_name = 'products/list.html'
context_object_name = 'products'
```
### Template Fragment Caching
```django
{% load cache %}
{% cache 500 sidebar %}
... expensive sidebar content ...
{% endcache %}
```
### Low-Level Caching
```python
from django.core.cache import cache
def get_featured_products():
"""Get featured products with caching."""
cache_key = 'featured_products'
products = cache.get(cache_key)
if products is None:
products = list(Product.objects.filter(is_featured=True))
cache.set(cache_key, products, timeout=60 * 15) # 15 minutes
return products
```
### QuerySet Caching
```python
from django.core.cache import cache
def get_popular_categories():
cache_key = 'popular_categories'
categories = cache.get(cache_key)
if categories is None:
categories = list(Category.objects.annotate(
product_count=Count('products')
).filter(product_count__gt=10).order_by('-product_count')[:20])
cache.set(cache_key, categories, timeout=60 * 60) # 1 hour
return categories
```
## Signals
### Signal Patterns
```python
# apps/users/signals.py
from django.db.models.signals import post_save
from django.dispatch import receiver
from django.contrib.auth import get_user_model
from .models import Profile
User = get_user_model()
@receiver(post_save, sender=User)
def create_user_profile(sender, instance, created, **kwargs):
"""Create profile when user is created."""
if created:
Profile.objects.create(user=instance)
@receiver(post_save, sender=User)
def save_user_profile(sender, instance, **kwargs):
"""Save profile when user is saved."""
instance.profile.save()
# apps/users/apps.py
from django.apps import AppConfig
class UsersConfig(AppConfig):
default_auto_field = 'django.db.models.BigAutoField'
name = 'apps.users'
def ready(self):
"""Import signals when app is ready."""
import apps.users.signals
```
## Middleware
### Custom Middleware
```python
# middleware/active_user_middleware.py
import time
from django.utils.deprecation import MiddlewareMixin
class ActiveUserMiddleware(MiddlewareMixin):
"""Middleware to track active users."""
def process_request(self, request):
"""Process incoming request."""
if request.user.is_authenticated:
# Update last active time
request.user.last_active = timezone.now()
request.user.save(update_fields=['last_active'])
class RequestLoggingMiddleware(MiddlewareMixin):
"""Middleware for logging requests."""
def process_request(self, request):
"""Log request start time."""
request.start_time = time.time()
def process_response(self, request, response):
"""Log request duration."""
if hasattr(request, 'start_time'):
duration = time.time() - request.start_time
logger.info(f'{request.method} {request.path} - {response.status_code} - {duration:.3f}s')
return response
```
## Performance Optimization
### N+1 Query Prevention
```python
# Bad - N+1 queries
products = Product.objects.all()
for product in products:
print(product.category.name) # Separate query for each product
# Good - Single query with select_related
products = Product.objects.select_related('category').all()
for product in products:
print(product.category.name)
# Good - Prefetch for many-to-many
products = Product.objects.prefetch_related('tags').all()
for product in products:
for tag in product.tags.all():
print(tag.name)
```
### Database Indexing
```python
class Product(models.Model):
name = models.CharField(max_length=200, db_index=True)
slug = models.SlugField(unique=True)
category = models.ForeignKey('Category', on_delete=models.CASCADE)
created_at = models.DateTimeField(auto_now_add=True)
class Meta:
indexes = [
models.Index(fields=['name']),
models.Index(fields=['-created_at']),
models.Index(fields=['category', 'created_at']),
]
```
### Bulk Operations
```python
# Bulk create
Product.objects.bulk_create([
Product(name=f'Product {i}', price=10.00)
for i in range(1000)
])
# Bulk update
products = Product.objects.all()[:100]
for product in products:
product.is_active = True
Product.objects.bulk_update(products, ['is_active'])
# Bulk delete
Product.objects.filter(stock=0).delete()
```
## Quick Reference
| Pattern | Description |
|---------|-------------|
| Split settings | Separate dev/prod/test settings |
| Custom QuerySet | Reusable query methods |
| Service Layer | Business logic separation |
| ViewSet | REST API endpoints |
| Serializer validation | Request/response transformation |
| select_related | Foreign key optimization |
| prefetch_related | Many-to-many optimization |
| Cache first | Cache expensive operations |
| Signals | Event-driven actions |
| Middleware | Request/response processing |
Remember: Django provides many shortcuts, but for production applications, structure and organization matter more than concise code. Build for maintainability.

View File

@@ -1,592 +0,0 @@
---
name: django-security
description: Django security best practices, authentication, authorization, CSRF protection, SQL injection prevention, XSS prevention, and secure deployment configurations.
---
# Django Security Best Practices
Comprehensive security guidelines for Django applications to protect against common vulnerabilities.
## When to Activate
- Setting up Django authentication and authorization
- Implementing user permissions and roles
- Configuring production security settings
- Reviewing Django application for security issues
- Deploying Django applications to production
## Core Security Settings
### Production Settings Configuration
```python
# settings/production.py
import os
DEBUG = False # CRITICAL: Never use True in production
ALLOWED_HOSTS = os.environ.get('ALLOWED_HOSTS', '').split(',')
# Security headers
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SECURE_HSTS_SECONDS = 31536000 # 1 year
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_BROWSER_XSS_FILTER = True
X_FRAME_OPTIONS = 'DENY'
# HTTPS and Cookies
SESSION_COOKIE_HTTPONLY = True
CSRF_COOKIE_HTTPONLY = True
SESSION_COOKIE_SAMESITE = 'Lax'
CSRF_COOKIE_SAMESITE = 'Lax'
# Secret key (must be set via environment variable)
SECRET_KEY = os.environ.get('DJANGO_SECRET_KEY')
if not SECRET_KEY:
raise ImproperlyConfigured('DJANGO_SECRET_KEY environment variable is required')
# Password validation
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
'OPTIONS': {
'min_length': 12,
}
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
```
## Authentication
### Custom User Model
```python
# apps/users/models.py
from django.contrib.auth.models import AbstractUser
from django.db import models
class User(AbstractUser):
"""Custom user model for better security."""
email = models.EmailField(unique=True)
phone = models.CharField(max_length=20, blank=True)
USERNAME_FIELD = 'email' # Use email as username
REQUIRED_FIELDS = ['username']
class Meta:
db_table = 'users'
verbose_name = 'User'
verbose_name_plural = 'Users'
def __str__(self):
return self.email
# settings/base.py
AUTH_USER_MODEL = 'users.User'
```
### Password Hashing
```python
# Django uses PBKDF2 by default. For stronger security:
PASSWORD_HASHERS = [
'django.contrib.auth.hashers.Argon2PasswordHasher',
'django.contrib.auth.hashers.PBKDF2PasswordHasher',
'django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher',
'django.contrib.auth.hashers.BCryptSHA256PasswordHasher',
]
```
### Session Management
```python
# Session configuration
SESSION_ENGINE = 'django.contrib.sessions.backends.cache' # Or 'db'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 3600 * 24 * 7 # 1 week
SESSION_SAVE_EVERY_REQUEST = False
SESSION_EXPIRE_AT_BROWSER_CLOSE = False # Better UX, but less secure
```
## Authorization
### Permissions
```python
# models.py
from django.db import models
from django.contrib.auth.models import Permission
class Post(models.Model):
title = models.CharField(max_length=200)
content = models.TextField()
author = models.ForeignKey(User, on_delete=models.CASCADE)
class Meta:
permissions = [
('can_publish', 'Can publish posts'),
('can_edit_others', 'Can edit posts of others'),
]
def user_can_edit(self, user):
"""Check if user can edit this post."""
return self.author == user or user.has_perm('app.can_edit_others')
# views.py
from django.contrib.auth.mixins import LoginRequiredMixin, PermissionRequiredMixin
from django.views.generic import UpdateView
class PostUpdateView(LoginRequiredMixin, PermissionRequiredMixin, UpdateView):
model = Post
permission_required = 'app.can_edit_others'
raise_exception = True # Return 403 instead of redirect
def get_queryset(self):
"""Only allow users to edit their own posts."""
return Post.objects.filter(author=self.request.user)
```
### Custom Permissions
```python
# permissions.py
from rest_framework import permissions
class IsOwnerOrReadOnly(permissions.BasePermission):
"""Allow only owners to edit objects."""
def has_object_permission(self, request, view, obj):
# Read permissions allowed for any request
if request.method in permissions.SAFE_METHODS:
return True
# Write permissions only for owner
return obj.author == request.user
class IsAdminOrReadOnly(permissions.BasePermission):
"""Allow admins to do anything, others read-only."""
def has_permission(self, request, view):
if request.method in permissions.SAFE_METHODS:
return True
return request.user and request.user.is_staff
class IsVerifiedUser(permissions.BasePermission):
"""Allow only verified users."""
def has_permission(self, request, view):
return request.user and request.user.is_authenticated and request.user.is_verified
```
### Role-Based Access Control (RBAC)
```python
# models.py
from django.contrib.auth.models import AbstractUser, Group
class User(AbstractUser):
ROLE_CHOICES = [
('admin', 'Administrator'),
('moderator', 'Moderator'),
('user', 'Regular User'),
]
role = models.CharField(max_length=20, choices=ROLE_CHOICES, default='user')
def is_admin(self):
return self.role == 'admin' or self.is_superuser
def is_moderator(self):
return self.role in ['admin', 'moderator']
# Mixins
class AdminRequiredMixin:
"""Mixin to require admin role."""
def dispatch(self, request, *args, **kwargs):
if not request.user.is_authenticated or not request.user.is_admin():
from django.core.exceptions import PermissionDenied
raise PermissionDenied
return super().dispatch(request, *args, **kwargs)
```
## SQL Injection Prevention
### Django ORM Protection
```python
# GOOD: Django ORM automatically escapes parameters
def get_user(username):
return User.objects.get(username=username) # Safe
# GOOD: Using parameters with raw()
def search_users(query):
return User.objects.raw('SELECT * FROM users WHERE username = %s', [query])
# BAD: Never directly interpolate user input
def get_user_bad(username):
return User.objects.raw(f'SELECT * FROM users WHERE username = {username}') # VULNERABLE!
# GOOD: Using filter with proper escaping
def get_users_by_email(email):
return User.objects.filter(email__iexact=email) # Safe
# GOOD: Using Q objects for complex queries
from django.db.models import Q
def search_users_complex(query):
return User.objects.filter(
Q(username__icontains=query) |
Q(email__icontains=query)
) # Safe
```
### Extra Security with raw()
```python
# If you must use raw SQL, always use parameters
User.objects.raw(
'SELECT * FROM users WHERE email = %s AND status = %s',
[user_input_email, status]
)
```
## XSS Prevention
### Template Escaping
```django
{# Django auto-escapes variables by default - SAFE #}
{{ user_input }} {# Escaped HTML #}
{# Explicitly mark safe only for trusted content #}
{{ trusted_html|safe }} {# Not escaped #}
{# Use template filters for safe HTML #}
{{ user_input|escape }} {# Same as default #}
{{ user_input|striptags }} {# Remove all HTML tags #}
{# JavaScript escaping #}
<script>
var username = {{ username|escapejs }};
</script>
```
### Safe String Handling
```python
from django.utils.safestring import mark_safe
from django.utils.html import escape
# BAD: Never mark user input as safe without escaping
def render_bad(user_input):
return mark_safe(user_input) # VULNERABLE!
# GOOD: Escape first, then mark safe
def render_good(user_input):
return mark_safe(escape(user_input))
# GOOD: Use format_html for HTML with variables
from django.utils.html import format_html
def greet_user(username):
return format_html('<span class="user">{}</span>', escape(username))
```
### HTTP Headers
```python
# settings.py
SECURE_CONTENT_TYPE_NOSNIFF = True # Prevent MIME sniffing
SECURE_BROWSER_XSS_FILTER = True # Enable XSS filter
X_FRAME_OPTIONS = 'DENY' # Prevent clickjacking
# Custom middleware
from django.conf import settings
class SecurityHeaderMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
response['X-Content-Type-Options'] = 'nosniff'
response['X-Frame-Options'] = 'DENY'
response['X-XSS-Protection'] = '1; mode=block'
response['Content-Security-Policy'] = "default-src 'self'"
return response
```
## CSRF Protection
### Default CSRF Protection
```python
# settings.py - CSRF is enabled by default
CSRF_COOKIE_SECURE = True # Only send over HTTPS
CSRF_COOKIE_HTTPONLY = True # Prevent JavaScript access
CSRF_COOKIE_SAMESITE = 'Lax' # Prevent CSRF in some cases
CSRF_TRUSTED_ORIGINS = ['https://example.com'] # Trusted domains
# Template usage
<form method="post">
{% csrf_token %}
{{ form.as_p }}
<button type="submit">Submit</button>
</form>
# AJAX requests
function getCookie(name) {
let cookieValue = null;
if (document.cookie && document.cookie !== '') {
const cookies = document.cookie.split(';');
for (let i = 0; i < cookies.length; i++) {
const cookie = cookies[i].trim();
if (cookie.substring(0, name.length + 1) === (name + '=')) {
cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
break;
}
}
}
return cookieValue;
}
fetch('/api/endpoint/', {
method: 'POST',
headers: {
'X-CSRFToken': getCookie('csrftoken'),
'Content-Type': 'application/json',
},
body: JSON.stringify(data)
});
```
### Exempting Views (Use Carefully)
```python
from django.views.decorators.csrf import csrf_exempt
@csrf_exempt # Only use when absolutely necessary!
def webhook_view(request):
# Webhook from external service
pass
```
## File Upload Security
### File Validation
```python
import os
from django.core.exceptions import ValidationError
def validate_file_extension(value):
"""Validate file extension."""
ext = os.path.splitext(value.name)[1]
valid_extensions = ['.jpg', '.jpeg', '.png', '.gif', '.pdf']
if not ext.lower() in valid_extensions:
raise ValidationError('Unsupported file extension.')
def validate_file_size(value):
"""Validate file size (max 5MB)."""
filesize = value.size
if filesize > 5 * 1024 * 1024:
raise ValidationError('File too large. Max size is 5MB.')
# models.py
class Document(models.Model):
file = models.FileField(
upload_to='documents/',
validators=[validate_file_extension, validate_file_size]
)
```
### Secure File Storage
```python
# settings.py
MEDIA_ROOT = '/var/www/media/'
MEDIA_URL = '/media/'
# Use a separate domain for media in production
MEDIA_DOMAIN = 'https://media.example.com'
# Don't serve user uploads directly
# Use whitenoise or a CDN for static files
# Use a separate server or S3 for media files
```
## API Security
### Rate Limiting
```python
# settings.py
REST_FRAMEWORK = {
'DEFAULT_THROTTLE_CLASSES': [
'rest_framework.throttling.AnonRateThrottle',
'rest_framework.throttling.UserRateThrottle'
],
'DEFAULT_THROTTLE_RATES': {
'anon': '100/day',
'user': '1000/day',
'upload': '10/hour',
}
}
# Custom throttle
from rest_framework.throttling import UserRateThrottle
class BurstRateThrottle(UserRateThrottle):
scope = 'burst'
rate = '60/min'
class SustainedRateThrottle(UserRateThrottle):
scope = 'sustained'
rate = '1000/day'
```
### Authentication for APIs
```python
# settings.py
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': [
'rest_framework.authentication.TokenAuthentication',
'rest_framework.authentication.SessionAuthentication',
'rest_framework_simplejwt.authentication.JWTAuthentication',
],
'DEFAULT_PERMISSION_CLASSES': [
'rest_framework.permissions.IsAuthenticated',
],
}
# views.py
from rest_framework.decorators import api_view, permission_classes
from rest_framework.permissions import IsAuthenticated
@api_view(['GET', 'POST'])
@permission_classes([IsAuthenticated])
def protected_view(request):
return Response({'message': 'You are authenticated'})
```
## Security Headers
### Content Security Policy
```python
# settings.py
CSP_DEFAULT_SRC = "'self'"
CSP_SCRIPT_SRC = "'self' https://cdn.example.com"
CSP_STYLE_SRC = "'self' 'unsafe-inline'"
CSP_IMG_SRC = "'self' data: https:"
CSP_CONNECT_SRC = "'self' https://api.example.com"
# Middleware
class CSPMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
response['Content-Security-Policy'] = (
f"default-src {CSP_DEFAULT_SRC}; "
f"script-src {CSP_SCRIPT_SRC}; "
f"style-src {CSP_STYLE_SRC}; "
f"img-src {CSP_IMG_SRC}; "
f"connect-src {CSP_CONNECT_SRC}"
)
return response
```
## Environment Variables
### Managing Secrets
```python
# Use python-decouple or django-environ
import environ
env = environ.Env(
# set casting, default value
DEBUG=(bool, False)
)
# reading .env file
environ.Env.read_env()
SECRET_KEY = env('DJANGO_SECRET_KEY')
DATABASE_URL = env('DATABASE_URL')
ALLOWED_HOSTS = env.list('ALLOWED_HOSTS')
# .env file (never commit this)
DEBUG=False
SECRET_KEY=your-secret-key-here
DATABASE_URL=postgresql://user:password@localhost:5432/dbname
ALLOWED_HOSTS=example.com,www.example.com
```
## Logging Security Events
```python
# settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'WARNING',
'class': 'logging.FileHandler',
'filename': '/var/log/django/security.log',
},
'console': {
'level': 'INFO',
'class': 'logging.StreamHandler',
},
},
'loggers': {
'django.security': {
'handlers': ['file', 'console'],
'level': 'WARNING',
'propagate': True,
},
'django.request': {
'handlers': ['file'],
'level': 'ERROR',
'propagate': False,
},
},
}
```
## Quick Security Checklist
| Check | Description |
|-------|-------------|
| `DEBUG = False` | Never run with DEBUG in production |
| HTTPS only | Force SSL, secure cookies |
| Strong secrets | Use environment variables for SECRET_KEY |
| Password validation | Enable all password validators |
| CSRF protection | Enabled by default, don't disable |
| XSS prevention | Django auto-escapes, don't use `&#124;safe` with user input |
| SQL injection | Use ORM, never concatenate strings in queries |
| File uploads | Validate file type and size |
| Rate limiting | Throttle API endpoints |
| Security headers | CSP, X-Frame-Options, HSTS |
| Logging | Log security events |
| Updates | Keep Django and dependencies updated |
Remember: Security is a process, not a product. Regularly review and update your security practices.

View File

@@ -1,728 +0,0 @@
---
name: django-tdd
description: Django testing strategies with pytest-django, TDD methodology, factory_boy, mocking, coverage, and testing Django REST Framework APIs.
---
# Django Testing with TDD
Test-driven development for Django applications using pytest, factory_boy, and Django REST Framework.
## When to Activate
- Writing new Django applications
- Implementing Django REST Framework APIs
- Testing Django models, views, and serializers
- Setting up testing infrastructure for Django projects
## TDD Workflow for Django
### Red-Green-Refactor Cycle
```python
# Step 1: RED - Write failing test
def test_user_creation():
user = User.objects.create_user(email='test@example.com', password='testpass123')
assert user.email == 'test@example.com'
assert user.check_password('testpass123')
assert not user.is_staff
# Step 2: GREEN - Make test pass
# Create User model or factory
# Step 3: REFACTOR - Improve while keeping tests green
```
## Setup
### pytest Configuration
```ini
# pytest.ini
[pytest]
DJANGO_SETTINGS_MODULE = config.settings.test
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts =
--reuse-db
--nomigrations
--cov=apps
--cov-report=html
--cov-report=term-missing
--strict-markers
markers =
slow: marks tests as slow
integration: marks tests as integration tests
```
### Test Settings
```python
# config/settings/test.py
from .base import *
DEBUG = True
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': ':memory:',
}
}
# Disable migrations for speed
class DisableMigrations:
def __contains__(self, item):
return True
def __getitem__(self, item):
return None
MIGRATION_MODULES = DisableMigrations()
# Faster password hashing
PASSWORD_HASHERS = [
'django.contrib.auth.hashers.MD5PasswordHasher',
]
# Email backend
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
# Celery always eager
CELERY_TASK_ALWAYS_EAGER = True
CELERY_TASK_EAGER_PROPAGATES = True
```
### conftest.py
```python
# tests/conftest.py
import pytest
from django.utils import timezone
from django.contrib.auth import get_user_model
User = get_user_model()
@pytest.fixture(autouse=True)
def timezone_settings(settings):
"""Ensure consistent timezone."""
settings.TIME_ZONE = 'UTC'
@pytest.fixture
def user(db):
"""Create a test user."""
return User.objects.create_user(
email='test@example.com',
password='testpass123',
username='testuser'
)
@pytest.fixture
def admin_user(db):
"""Create an admin user."""
return User.objects.create_superuser(
email='admin@example.com',
password='adminpass123',
username='admin'
)
@pytest.fixture
def authenticated_client(client, user):
"""Return authenticated client."""
client.force_login(user)
return client
@pytest.fixture
def api_client():
"""Return DRF API client."""
from rest_framework.test import APIClient
return APIClient()
@pytest.fixture
def authenticated_api_client(api_client, user):
"""Return authenticated API client."""
api_client.force_authenticate(user=user)
return api_client
```
## Factory Boy
### Factory Setup
```python
# tests/factories.py
import factory
from factory import fuzzy
from datetime import datetime, timedelta
from django.contrib.auth import get_user_model
from apps.products.models import Product, Category
User = get_user_model()
class UserFactory(factory.django.DjangoModelFactory):
"""Factory for User model."""
class Meta:
model = User
email = factory.Sequence(lambda n: f"user{n}@example.com")
username = factory.Sequence(lambda n: f"user{n}")
password = factory.PostGenerationMethodCall('set_password', 'testpass123')
first_name = factory.Faker('first_name')
last_name = factory.Faker('last_name')
is_active = True
class CategoryFactory(factory.django.DjangoModelFactory):
"""Factory for Category model."""
class Meta:
model = Category
name = factory.Faker('word')
slug = factory.LazyAttribute(lambda obj: obj.name.lower())
description = factory.Faker('text')
class ProductFactory(factory.django.DjangoModelFactory):
"""Factory for Product model."""
class Meta:
model = Product
name = factory.Faker('sentence', nb_words=3)
slug = factory.LazyAttribute(lambda obj: obj.name.lower().replace(' ', '-'))
description = factory.Faker('text')
price = fuzzy.FuzzyDecimal(10.00, 1000.00, 2)
stock = fuzzy.FuzzyInteger(0, 100)
is_active = True
category = factory.SubFactory(CategoryFactory)
created_by = factory.SubFactory(UserFactory)
@factory.post_generation
def tags(self, create, extracted, **kwargs):
"""Add tags to product."""
if not create:
return
if extracted:
for tag in extracted:
self.tags.add(tag)
```
### Using Factories
```python
# tests/test_models.py
import pytest
from tests.factories import ProductFactory, UserFactory
def test_product_creation():
"""Test product creation using factory."""
product = ProductFactory(price=100.00, stock=50)
assert product.price == 100.00
assert product.stock == 50
assert product.is_active is True
def test_product_with_tags():
"""Test product with tags."""
tags = [TagFactory(name='electronics'), TagFactory(name='new')]
product = ProductFactory(tags=tags)
assert product.tags.count() == 2
def test_multiple_products():
"""Test creating multiple products."""
products = ProductFactory.create_batch(10)
assert len(products) == 10
```
## Model Testing
### Model Tests
```python
# tests/test_models.py
import pytest
from django.core.exceptions import ValidationError
from tests.factories import UserFactory, ProductFactory
class TestUserModel:
"""Test User model."""
def test_create_user(self, db):
"""Test creating a regular user."""
user = UserFactory(email='test@example.com')
assert user.email == 'test@example.com'
assert user.check_password('testpass123')
assert not user.is_staff
assert not user.is_superuser
def test_create_superuser(self, db):
"""Test creating a superuser."""
user = UserFactory(
email='admin@example.com',
is_staff=True,
is_superuser=True
)
assert user.is_staff
assert user.is_superuser
def test_user_str(self, db):
"""Test user string representation."""
user = UserFactory(email='test@example.com')
assert str(user) == 'test@example.com'
class TestProductModel:
"""Test Product model."""
def test_product_creation(self, db):
"""Test creating a product."""
product = ProductFactory()
assert product.id is not None
assert product.is_active is True
assert product.created_at is not None
def test_product_slug_generation(self, db):
"""Test automatic slug generation."""
product = ProductFactory(name='Test Product')
assert product.slug == 'test-product'
def test_product_price_validation(self, db):
"""Test price cannot be negative."""
product = ProductFactory(price=-10)
with pytest.raises(ValidationError):
product.full_clean()
def test_product_manager_active(self, db):
"""Test active manager method."""
ProductFactory.create_batch(5, is_active=True)
ProductFactory.create_batch(3, is_active=False)
active_count = Product.objects.active().count()
assert active_count == 5
def test_product_stock_management(self, db):
"""Test stock management."""
product = ProductFactory(stock=10)
product.reduce_stock(5)
product.refresh_from_db()
assert product.stock == 5
with pytest.raises(ValueError):
product.reduce_stock(10) # Not enough stock
```
## View Testing
### Django View Testing
```python
# tests/test_views.py
import pytest
from django.urls import reverse
from tests.factories import ProductFactory, UserFactory
class TestProductViews:
"""Test product views."""
def test_product_list(self, client, db):
"""Test product list view."""
ProductFactory.create_batch(10)
response = client.get(reverse('products:list'))
assert response.status_code == 200
assert len(response.context['products']) == 10
def test_product_detail(self, client, db):
"""Test product detail view."""
product = ProductFactory()
response = client.get(reverse('products:detail', kwargs={'slug': product.slug}))
assert response.status_code == 200
assert response.context['product'] == product
def test_product_create_requires_login(self, client, db):
"""Test product creation requires authentication."""
response = client.get(reverse('products:create'))
assert response.status_code == 302
assert response.url.startswith('/accounts/login/')
def test_product_create_authenticated(self, authenticated_client, db):
"""Test product creation as authenticated user."""
response = authenticated_client.get(reverse('products:create'))
assert response.status_code == 200
def test_product_create_post(self, authenticated_client, db, category):
"""Test creating a product via POST."""
data = {
'name': 'Test Product',
'description': 'A test product',
'price': '99.99',
'stock': 10,
'category': category.id,
}
response = authenticated_client.post(reverse('products:create'), data)
assert response.status_code == 302
assert Product.objects.filter(name='Test Product').exists()
```
## DRF API Testing
### Serializer Testing
```python
# tests/test_serializers.py
import pytest
from rest_framework.exceptions import ValidationError
from apps.products.serializers import ProductSerializer
from tests.factories import ProductFactory
class TestProductSerializer:
"""Test ProductSerializer."""
def test_serialize_product(self, db):
"""Test serializing a product."""
product = ProductFactory()
serializer = ProductSerializer(product)
data = serializer.data
assert data['id'] == product.id
assert data['name'] == product.name
assert data['price'] == str(product.price)
def test_deserialize_product(self, db):
"""Test deserializing product data."""
data = {
'name': 'Test Product',
'description': 'Test description',
'price': '99.99',
'stock': 10,
'category': 1,
}
serializer = ProductSerializer(data=data)
assert serializer.is_valid()
product = serializer.save()
assert product.name == 'Test Product'
assert float(product.price) == 99.99
def test_price_validation(self, db):
"""Test price validation."""
data = {
'name': 'Test Product',
'price': '-10.00',
'stock': 10,
}
serializer = ProductSerializer(data=data)
assert not serializer.is_valid()
assert 'price' in serializer.errors
def test_stock_validation(self, db):
"""Test stock cannot be negative."""
data = {
'name': 'Test Product',
'price': '99.99',
'stock': -5,
}
serializer = ProductSerializer(data=data)
assert not serializer.is_valid()
assert 'stock' in serializer.errors
```
### API ViewSet Testing
```python
# tests/test_api.py
import pytest
from rest_framework.test import APIClient
from rest_framework import status
from django.urls import reverse
from tests.factories import ProductFactory, UserFactory
class TestProductAPI:
"""Test Product API endpoints."""
@pytest.fixture
def api_client(self):
"""Return API client."""
return APIClient()
def test_list_products(self, api_client, db):
"""Test listing products."""
ProductFactory.create_batch(10)
url = reverse('api:product-list')
response = api_client.get(url)
assert response.status_code == status.HTTP_200_OK
assert response.data['count'] == 10
def test_retrieve_product(self, api_client, db):
"""Test retrieving a product."""
product = ProductFactory()
url = reverse('api:product-detail', kwargs={'pk': product.id})
response = api_client.get(url)
assert response.status_code == status.HTTP_200_OK
assert response.data['id'] == product.id
def test_create_product_unauthorized(self, api_client, db):
"""Test creating product without authentication."""
url = reverse('api:product-list')
data = {'name': 'Test Product', 'price': '99.99'}
response = api_client.post(url, data)
assert response.status_code == status.HTTP_401_UNAUTHORIZED
def test_create_product_authorized(self, authenticated_api_client, db):
"""Test creating product as authenticated user."""
url = reverse('api:product-list')
data = {
'name': 'Test Product',
'description': 'Test',
'price': '99.99',
'stock': 10,
}
response = authenticated_api_client.post(url, data)
assert response.status_code == status.HTTP_201_CREATED
assert response.data['name'] == 'Test Product'
def test_update_product(self, authenticated_api_client, db):
"""Test updating a product."""
product = ProductFactory(created_by=authenticated_api_client.user)
url = reverse('api:product-detail', kwargs={'pk': product.id})
data = {'name': 'Updated Product'}
response = authenticated_api_client.patch(url, data)
assert response.status_code == status.HTTP_200_OK
assert response.data['name'] == 'Updated Product'
def test_delete_product(self, authenticated_api_client, db):
"""Test deleting a product."""
product = ProductFactory(created_by=authenticated_api_client.user)
url = reverse('api:product-detail', kwargs={'pk': product.id})
response = authenticated_api_client.delete(url)
assert response.status_code == status.HTTP_204_NO_CONTENT
def test_filter_products_by_price(self, api_client, db):
"""Test filtering products by price."""
ProductFactory(price=50)
ProductFactory(price=150)
url = reverse('api:product-list')
response = api_client.get(url, {'price_min': 100})
assert response.status_code == status.HTTP_200_OK
assert response.data['count'] == 1
def test_search_products(self, api_client, db):
"""Test searching products."""
ProductFactory(name='Apple iPhone')
ProductFactory(name='Samsung Galaxy')
url = reverse('api:product-list')
response = api_client.get(url, {'search': 'Apple'})
assert response.status_code == status.HTTP_200_OK
assert response.data['count'] == 1
```
## Mocking and Patching
### Mocking External Services
```python
# tests/test_views.py
from unittest.mock import patch, Mock
import pytest
class TestPaymentView:
"""Test payment view with mocked payment gateway."""
@patch('apps.payments.services.stripe')
def test_successful_payment(self, mock_stripe, client, user, product):
"""Test successful payment with mocked Stripe."""
# Configure mock
mock_stripe.Charge.create.return_value = {
'id': 'ch_123',
'status': 'succeeded',
'amount': 9999,
}
client.force_login(user)
response = client.post(reverse('payments:process'), {
'product_id': product.id,
'token': 'tok_visa',
})
assert response.status_code == 302
mock_stripe.Charge.create.assert_called_once()
@patch('apps.payments.services.stripe')
def test_failed_payment(self, mock_stripe, client, user, product):
"""Test failed payment."""
mock_stripe.Charge.create.side_effect = Exception('Card declined')
client.force_login(user)
response = client.post(reverse('payments:process'), {
'product_id': product.id,
'token': 'tok_visa',
})
assert response.status_code == 302
assert 'error' in response.url
```
### Mocking Email Sending
```python
# tests/test_email.py
from django.core import mail
from django.test import override_settings
@override_settings(EMAIL_BACKEND='django.core.mail.backends.locmem.EmailBackend')
def test_order_confirmation_email(db, order):
"""Test order confirmation email."""
order.send_confirmation_email()
assert len(mail.outbox) == 1
assert order.user.email in mail.outbox[0].to
assert 'Order Confirmation' in mail.outbox[0].subject
```
## Integration Testing
### Full Flow Testing
```python
# tests/test_integration.py
import pytest
from django.urls import reverse
from tests.factories import UserFactory, ProductFactory
class TestCheckoutFlow:
"""Test complete checkout flow."""
def test_guest_to_purchase_flow(self, client, db):
"""Test complete flow from guest to purchase."""
# Step 1: Register
response = client.post(reverse('users:register'), {
'email': 'test@example.com',
'password': 'testpass123',
'password_confirm': 'testpass123',
})
assert response.status_code == 302
# Step 2: Login
response = client.post(reverse('users:login'), {
'email': 'test@example.com',
'password': 'testpass123',
})
assert response.status_code == 302
# Step 3: Browse products
product = ProductFactory(price=100)
response = client.get(reverse('products:detail', kwargs={'slug': product.slug}))
assert response.status_code == 200
# Step 4: Add to cart
response = client.post(reverse('cart:add'), {
'product_id': product.id,
'quantity': 1,
})
assert response.status_code == 302
# Step 5: Checkout
response = client.get(reverse('checkout:review'))
assert response.status_code == 200
assert product.name in response.content.decode()
# Step 6: Complete purchase
with patch('apps.checkout.services.process_payment') as mock_payment:
mock_payment.return_value = True
response = client.post(reverse('checkout:complete'))
assert response.status_code == 302
assert Order.objects.filter(user__email='test@example.com').exists()
```
## Testing Best Practices
### DO
- **Use factories**: Instead of manual object creation
- **One assertion per test**: Keep tests focused
- **Descriptive test names**: `test_user_cannot_delete_others_post`
- **Test edge cases**: Empty inputs, None values, boundary conditions
- **Mock external services**: Don't depend on external APIs
- **Use fixtures**: Eliminate duplication
- **Test permissions**: Ensure authorization works
- **Keep tests fast**: Use `--reuse-db` and `--nomigrations`
### DON'T
- **Don't test Django internals**: Trust Django to work
- **Don't test third-party code**: Trust libraries to work
- **Don't ignore failing tests**: All tests must pass
- **Don't make tests dependent**: Tests should run in any order
- **Don't over-mock**: Mock only external dependencies
- **Don't test private methods**: Test public interface
- **Don't use production database**: Always use test database
## Coverage
### Coverage Configuration
```bash
# Run tests with coverage
pytest --cov=apps --cov-report=html --cov-report=term-missing
# Generate HTML report
open htmlcov/index.html
```
### Coverage Goals
| Component | Target Coverage |
|-----------|-----------------|
| Models | 90%+ |
| Serializers | 85%+ |
| Views | 80%+ |
| Services | 90%+ |
| Utilities | 80%+ |
| Overall | 80%+ |
## Quick Reference
| Pattern | Usage |
|---------|-------|
| `@pytest.mark.django_db` | Enable database access |
| `client` | Django test client |
| `api_client` | DRF API client |
| `factory.create_batch(n)` | Create multiple objects |
| `patch('module.function')` | Mock external dependencies |
| `override_settings` | Temporarily change settings |
| `force_authenticate()` | Bypass authentication in tests |
| `assertRedirects` | Check for redirects |
| `assertTemplateUsed` | Verify template usage |
| `mail.outbox` | Check sent emails |
Remember: Tests are documentation. Good tests explain how your code should work. Keep them simple, readable, and maintainable.

View File

@@ -1,460 +0,0 @@
---
name: django-verification
description: "Verification loop for Django projects: migrations, linting, tests with coverage, security scans, and deployment readiness checks before release or PR."
---
# Django Verification Loop
Run before PRs, after major changes, and pre-deploy to ensure Django application quality and security.
## Phase 1: Environment Check
```bash
# Verify Python version
python --version # Should match project requirements
# Check virtual environment
which python
pip list --outdated
# Verify environment variables
python -c "import os; import environ; print('DJANGO_SECRET_KEY set' if os.environ.get('DJANGO_SECRET_KEY') else 'MISSING: DJANGO_SECRET_KEY')"
```
If environment is misconfigured, stop and fix.
## Phase 2: Code Quality & Formatting
```bash
# Type checking
mypy . --config-file pyproject.toml
# Linting with ruff
ruff check . --fix
# Formatting with black
black . --check
black . # Auto-fix
# Import sorting
isort . --check-only
isort . # Auto-fix
# Django-specific checks
python manage.py check --deploy
```
Common issues:
- Missing type hints on public functions
- PEP 8 formatting violations
- Unsorted imports
- Debug settings left in production configuration
## Phase 3: Migrations
```bash
# Check for unapplied migrations
python manage.py showmigrations
# Create missing migrations
python manage.py makemigrations --check
# Dry-run migration application
python manage.py migrate --plan
# Apply migrations (test environment)
python manage.py migrate
# Check for migration conflicts
python manage.py makemigrations --merge # Only if conflicts exist
```
Report:
- Number of pending migrations
- Any migration conflicts
- Model changes without migrations
## Phase 4: Tests + Coverage
```bash
# Run all tests with pytest
pytest --cov=apps --cov-report=html --cov-report=term-missing --reuse-db
# Run specific app tests
pytest apps/users/tests/
# Run with markers
pytest -m "not slow" # Skip slow tests
pytest -m integration # Only integration tests
# Coverage report
open htmlcov/index.html
```
Report:
- Total tests: X passed, Y failed, Z skipped
- Overall coverage: XX%
- Per-app coverage breakdown
Coverage targets:
| Component | Target |
|-----------|--------|
| Models | 90%+ |
| Serializers | 85%+ |
| Views | 80%+ |
| Services | 90%+ |
| Overall | 80%+ |
## Phase 5: Security Scan
```bash
# Dependency vulnerabilities
pip-audit
safety check --full-report
# Django security checks
python manage.py check --deploy
# Bandit security linter
bandit -r . -f json -o bandit-report.json
# Secret scanning (if gitleaks is installed)
gitleaks detect --source . --verbose
# Environment variable check
python -c "from django.core.exceptions import ImproperlyConfigured; from django.conf import settings; settings.DEBUG"
```
Report:
- Vulnerable dependencies found
- Security configuration issues
- Hardcoded secrets detected
- DEBUG mode status (should be False in production)
## Phase 6: Django Management Commands
```bash
# Check for model issues
python manage.py check
# Collect static files
python manage.py collectstatic --noinput --clear
# Create superuser (if needed for tests)
echo "from apps.users.models import User; User.objects.create_superuser('admin@example.com', 'admin')" | python manage.py shell
# Database integrity
python manage.py check --database default
# Cache verification (if using Redis)
python -c "from django.core.cache import cache; cache.set('test', 'value', 10); print(cache.get('test'))"
```
## Phase 7: Performance Checks
```bash
# Django Debug Toolbar output (check for N+1 queries)
# Run in dev mode with DEBUG=True and access a page
# Look for duplicate queries in SQL panel
# Query count analysis
django-admin debugsqlshell # If django-debug-sqlshell installed
# Check for missing indexes
python manage.py shell << EOF
from django.db import connection
with connection.cursor() as cursor:
cursor.execute("SELECT table_name, index_name FROM information_schema.statistics WHERE table_schema = 'public'")
print(cursor.fetchall())
EOF
```
Report:
- Number of queries per page (should be < 50 for typical pages)
- Missing database indexes
- Duplicate queries detected
## Phase 8: Static Assets
```bash
# Check for npm dependencies (if using npm)
npm audit
npm audit fix
# Build static files (if using webpack/vite)
npm run build
# Verify static files
ls -la staticfiles/
python manage.py findstatic css/style.css
```
## Phase 9: Configuration Review
```python
# Run in Python shell to verify settings
python manage.py shell << EOF
from django.conf import settings
import os
# Critical checks
checks = {
'DEBUG is False': not settings.DEBUG,
'SECRET_KEY set': bool(settings.SECRET_KEY and len(settings.SECRET_KEY) > 30),
'ALLOWED_HOSTS set': len(settings.ALLOWED_HOSTS) > 0,
'HTTPS enabled': getattr(settings, 'SECURE_SSL_REDIRECT', False),
'HSTS enabled': getattr(settings, 'SECURE_HSTS_SECONDS', 0) > 0,
'Database configured': settings.DATABASES['default']['ENGINE'] != 'django.db.backends.sqlite3',
}
for check, result in checks.items():
status = '' if result else ''
print(f"{status} {check}")
EOF
```
## Phase 10: Logging Configuration
```bash
# Test logging output
python manage.py shell << EOF
import logging
logger = logging.getLogger('django')
logger.warning('Test warning message')
logger.error('Test error message')
EOF
# Check log files (if configured)
tail -f /var/log/django/django.log
```
## Phase 11: API Documentation (if DRF)
```bash
# Generate schema
python manage.py generateschema --format openapi-json > schema.json
# Validate schema
# Check if schema.json is valid JSON
python -c "import json; json.load(open('schema.json'))"
# Access Swagger UI (if using drf-yasg)
# Visit http://localhost:8000/swagger/ in browser
```
## Phase 12: Diff Review
```bash
# Show diff statistics
git diff --stat
# Show actual changes
git diff
# Show changed files
git diff --name-only
# Check for common issues
git diff | grep -i "todo\|fixme\|hack\|xxx"
git diff | grep "print(" # Debug statements
git diff | grep "DEBUG = True" # Debug mode
git diff | grep "import pdb" # Debugger
```
Checklist:
- No debugging statements (print, pdb, breakpoint())
- No TODO/FIXME comments in critical code
- No hardcoded secrets or credentials
- Database migrations included for model changes
- Configuration changes documented
- Error handling present for external calls
- Transaction management where needed
## Output Template
```
DJANGO VERIFICATION REPORT
==========================
Phase 1: Environment Check
✓ Python 3.11.5
✓ Virtual environment active
✓ All environment variables set
Phase 2: Code Quality
✓ mypy: No type errors
✗ ruff: 3 issues found (auto-fixed)
✓ black: No formatting issues
✓ isort: Imports properly sorted
✓ manage.py check: No issues
Phase 3: Migrations
✓ No unapplied migrations
✓ No migration conflicts
✓ All models have migrations
Phase 4: Tests + Coverage
Tests: 247 passed, 0 failed, 5 skipped
Coverage:
Overall: 87%
users: 92%
products: 89%
orders: 85%
payments: 91%
Phase 5: Security Scan
✗ pip-audit: 2 vulnerabilities found (fix required)
✓ safety check: No issues
✓ bandit: No security issues
✓ No secrets detected
✓ DEBUG = False
Phase 6: Django Commands
✓ collectstatic completed
✓ Database integrity OK
✓ Cache backend reachable
Phase 7: Performance
✓ No N+1 queries detected
✓ Database indexes configured
✓ Query count acceptable
Phase 8: Static Assets
✓ npm audit: No vulnerabilities
✓ Assets built successfully
✓ Static files collected
Phase 9: Configuration
✓ DEBUG = False
✓ SECRET_KEY configured
✓ ALLOWED_HOSTS set
✓ HTTPS enabled
✓ HSTS enabled
✓ Database configured
Phase 10: Logging
✓ Logging configured
✓ Log files writable
Phase 11: API Documentation
✓ Schema generated
✓ Swagger UI accessible
Phase 12: Diff Review
Files changed: 12
+450, -120 lines
✓ No debug statements
✓ No hardcoded secrets
✓ Migrations included
RECOMMENDATION: ⚠️ Fix pip-audit vulnerabilities before deploying
NEXT STEPS:
1. Update vulnerable dependencies
2. Re-run security scan
3. Deploy to staging for final testing
```
## Pre-Deployment Checklist
- [ ] All tests passing
- [ ] Coverage ≥ 80%
- [ ] No security vulnerabilities
- [ ] No unapplied migrations
- [ ] DEBUG = False in production settings
- [ ] SECRET_KEY properly configured
- [ ] ALLOWED_HOSTS set correctly
- [ ] Database backups enabled
- [ ] Static files collected and served
- [ ] Logging configured and working
- [ ] Error monitoring (Sentry, etc.) configured
- [ ] CDN configured (if applicable)
- [ ] Redis/cache backend configured
- [ ] Celery workers running (if applicable)
- [ ] HTTPS/SSL configured
- [ ] Environment variables documented
## Continuous Integration
### GitHub Actions Example
```yaml
# .github/workflows/django-verification.yml
name: Django Verification
on: [push, pull_request]
jobs:
verify:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:14
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Cache pip
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install ruff black mypy pytest pytest-django pytest-cov bandit safety pip-audit
- name: Code quality checks
run: |
ruff check .
black . --check
isort . --check-only
mypy .
- name: Security scan
run: |
bandit -r . -f json -o bandit-report.json
safety check --full-report
pip-audit
- name: Run tests
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/test
DJANGO_SECRET_KEY: test-secret-key
run: |
pytest --cov=apps --cov-report=xml --cov-report=term-missing
- name: Upload coverage
uses: codecov/codecov-action@v3
```
## Quick Reference
| Check | Command |
|-------|---------|
| Environment | `python --version` |
| Type checking | `mypy .` |
| Linting | `ruff check .` |
| Formatting | `black . --check` |
| Migrations | `python manage.py makemigrations --check` |
| Tests | `pytest --cov=apps` |
| Security | `pip-audit && bandit -r .` |
| Django check | `python manage.py check --deploy` |
| Collectstatic | `python manage.py collectstatic --noinput` |
| Diff stats | `git diff --stat` |
Remember: Automated verification catches common issues but doesn't replace manual code review and testing in staging environment.

View File

@@ -1,227 +0,0 @@
---
name: eval-harness
description: Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles
tools: Read, Write, Edit, Bash, Grep, Glob
---
# Eval Harness Skill
A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.
## Philosophy
Eval-Driven Development treats evals as the "unit tests of AI development":
- Define expected behavior BEFORE implementation
- Run evals continuously during development
- Track regressions with each change
- Use pass@k metrics for reliability measurement
## Eval Types
### Capability Evals
Test if Claude can do something it couldn't before:
```markdown
[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] Criterion 3
Expected Output: Description of expected result
```
### Regression Evals
Ensure changes don't break existing functionality:
```markdown
[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
- existing-test-1: PASS/FAIL
- existing-test-2: PASS/FAIL
- existing-test-3: PASS/FAIL
Result: X/Y passed (previously Y/Y)
```
## Grader Types
### 1. Code-Based Grader
Deterministic checks using code:
```bash
# Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
# Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
# Check if build succeeds
npm run build && echo "PASS" || echo "FAIL"
```
### 2. Model-Based Grader
Use Claude to evaluate open-ended outputs:
```markdown
[MODEL GRADER PROMPT]
Evaluate the following code change:
1. Does it solve the stated problem?
2. Is it well-structured?
3. Are edge cases handled?
4. Is error handling appropriate?
Score: 1-5 (1=poor, 5=excellent)
Reasoning: [explanation]
```
### 3. Human Grader
Flag for manual review:
```markdown
[HUMAN REVIEW REQUIRED]
Change: Description of what changed
Reason: Why human review is needed
Risk Level: LOW/MEDIUM/HIGH
```
## Metrics
### pass@k
"At least one success in k attempts"
- pass@1: First attempt success rate
- pass@3: Success within 3 attempts
- Typical target: pass@3 > 90%
### pass^k
"All k trials succeed"
- Higher bar for reliability
- pass^3: 3 consecutive successes
- Use for critical paths
## Eval Workflow
### 1. Define (Before Coding)
```markdown
## EVAL DEFINITION: feature-xyz
### Capability Evals
1. Can create new user account
2. Can validate email format
3. Can hash password securely
### Regression Evals
1. Existing login still works
2. Session management unchanged
3. Logout flow intact
### Success Metrics
- pass@3 > 90% for capability evals
- pass^3 = 100% for regression evals
```
### 2. Implement
Write code to pass the defined evals.
### 3. Evaluate
```bash
# Run capability evals
[Run each capability eval, record PASS/FAIL]
# Run regression evals
npm test -- --testPathPattern="existing"
# Generate report
```
### 4. Report
```markdown
EVAL REPORT: feature-xyz
========================
Capability Evals:
create-user: PASS (pass@1)
validate-email: PASS (pass@2)
hash-password: PASS (pass@1)
Overall: 3/3 passed
Regression Evals:
login-flow: PASS
session-mgmt: PASS
logout-flow: PASS
Overall: 3/3 passed
Metrics:
pass@1: 67% (2/3)
pass@3: 100% (3/3)
Status: READY FOR REVIEW
```
## Integration Patterns
### Pre-Implementation
```
/eval define feature-name
```
Creates eval definition file at `.claude/evals/feature-name.md`
### During Implementation
```
/eval check feature-name
```
Runs current evals and reports status
### Post-Implementation
```
/eval report feature-name
```
Generates full eval report
## Eval Storage
Store evals in project:
```
.claude/
evals/
feature-xyz.md # Eval definition
feature-xyz.log # Eval run history
baseline.json # Regression baselines
```
## Best Practices
1. **Define evals BEFORE coding** - Forces clear thinking about success criteria
2. **Run evals frequently** - Catch regressions early
3. **Track pass@k over time** - Monitor reliability trends
4. **Use code graders when possible** - Deterministic > probabilistic
5. **Human review for security** - Never fully automate security checks
6. **Keep evals fast** - Slow evals don't get run
7. **Version evals with code** - Evals are first-class artifacts
## Example: Adding Authentication
```markdown
## EVAL: add-authentication
### Phase 1: Define (10 min)
Capability Evals:
- [ ] User can register with email/password
- [ ] User can login with valid credentials
- [ ] Invalid credentials rejected with proper error
- [ ] Sessions persist across page reloads
- [ ] Logout clears session
Regression Evals:
- [ ] Public routes still accessible
- [ ] API responses unchanged
- [ ] Database schema compatible
### Phase 2: Implement (varies)
[Write code]
### Phase 3: Evaluate
Run: /eval check add-authentication
### Phase 4: Report
EVAL REPORT: add-authentication
==============================
Capability: 5/5 passed (pass@3: 100%)
Regression: 3/3 passed (pass^3: 100%)
Status: SHIP IT
```

View File

@@ -1,631 +0,0 @@
---
name: frontend-patterns
description: Frontend development patterns for React, Next.js, state management, performance optimization, and UI best practices.
---
# Frontend Development Patterns
Modern frontend patterns for React, Next.js, and performant user interfaces.
## Component Patterns
### Composition Over Inheritance
```typescript
// ✅ GOOD: Component composition
interface CardProps {
children: React.ReactNode
variant?: 'default' | 'outlined'
}
export function Card({ children, variant = 'default' }: CardProps) {
return <div className={`card card-${variant}`}>{children}</div>
}
export function CardHeader({ children }: { children: React.ReactNode }) {
return <div className="card-header">{children}</div>
}
export function CardBody({ children }: { children: React.ReactNode }) {
return <div className="card-body">{children}</div>
}
// Usage
<Card>
<CardHeader>Title</CardHeader>
<CardBody>Content</CardBody>
</Card>
```
### Compound Components
```typescript
interface TabsContextValue {
activeTab: string
setActiveTab: (tab: string) => void
}
const TabsContext = createContext<TabsContextValue | undefined>(undefined)
export function Tabs({ children, defaultTab }: {
children: React.ReactNode
defaultTab: string
}) {
const [activeTab, setActiveTab] = useState(defaultTab)
return (
<TabsContext.Provider value={{ activeTab, setActiveTab }}>
{children}
</TabsContext.Provider>
)
}
export function TabList({ children }: { children: React.ReactNode }) {
return <div className="tab-list">{children}</div>
}
export function Tab({ id, children }: { id: string, children: React.ReactNode }) {
const context = useContext(TabsContext)
if (!context) throw new Error('Tab must be used within Tabs')
return (
<button
className={context.activeTab === id ? 'active' : ''}
onClick={() => context.setActiveTab(id)}
>
{children}
</button>
)
}
// Usage
<Tabs defaultTab="overview">
<TabList>
<Tab id="overview">Overview</Tab>
<Tab id="details">Details</Tab>
</TabList>
</Tabs>
```
### Render Props Pattern
```typescript
interface DataLoaderProps<T> {
url: string
children: (data: T | null, loading: boolean, error: Error | null) => React.ReactNode
}
export function DataLoader<T>({ url, children }: DataLoaderProps<T>) {
const [data, setData] = useState<T | null>(null)
const [loading, setLoading] = useState(true)
const [error, setError] = useState<Error | null>(null)
useEffect(() => {
fetch(url)
.then(res => res.json())
.then(setData)
.catch(setError)
.finally(() => setLoading(false))
}, [url])
return <>{children(data, loading, error)}</>
}
// Usage
<DataLoader<Market[]> url="/api/markets">
{(markets, loading, error) => {
if (loading) return <Spinner />
if (error) return <Error error={error} />
return <MarketList markets={markets!} />
}}
</DataLoader>
```
## Custom Hooks Patterns
### State Management Hook
```typescript
export function useToggle(initialValue = false): [boolean, () => void] {
const [value, setValue] = useState(initialValue)
const toggle = useCallback(() => {
setValue(v => !v)
}, [])
return [value, toggle]
}
// Usage
const [isOpen, toggleOpen] = useToggle()
```
### Async Data Fetching Hook
```typescript
interface UseQueryOptions<T> {
onSuccess?: (data: T) => void
onError?: (error: Error) => void
enabled?: boolean
}
export function useQuery<T>(
key: string,
fetcher: () => Promise<T>,
options?: UseQueryOptions<T>
) {
const [data, setData] = useState<T | null>(null)
const [error, setError] = useState<Error | null>(null)
const [loading, setLoading] = useState(false)
const refetch = useCallback(async () => {
setLoading(true)
setError(null)
try {
const result = await fetcher()
setData(result)
options?.onSuccess?.(result)
} catch (err) {
const error = err as Error
setError(error)
options?.onError?.(error)
} finally {
setLoading(false)
}
}, [fetcher, options])
useEffect(() => {
if (options?.enabled !== false) {
refetch()
}
}, [key, refetch, options?.enabled])
return { data, error, loading, refetch }
}
// Usage
const { data: markets, loading, error, refetch } = useQuery(
'markets',
() => fetch('/api/markets').then(r => r.json()),
{
onSuccess: data => console.log('Fetched', data.length, 'markets'),
onError: err => console.error('Failed:', err)
}
)
```
### Debounce Hook
```typescript
export function useDebounce<T>(value: T, delay: number): T {
const [debouncedValue, setDebouncedValue] = useState<T>(value)
useEffect(() => {
const handler = setTimeout(() => {
setDebouncedValue(value)
}, delay)
return () => clearTimeout(handler)
}, [value, delay])
return debouncedValue
}
// Usage
const [searchQuery, setSearchQuery] = useState('')
const debouncedQuery = useDebounce(searchQuery, 500)
useEffect(() => {
if (debouncedQuery) {
performSearch(debouncedQuery)
}
}, [debouncedQuery])
```
## State Management Patterns
### Context + Reducer Pattern
```typescript
interface State {
markets: Market[]
selectedMarket: Market | null
loading: boolean
}
type Action =
| { type: 'SET_MARKETS'; payload: Market[] }
| { type: 'SELECT_MARKET'; payload: Market }
| { type: 'SET_LOADING'; payload: boolean }
function reducer(state: State, action: Action): State {
switch (action.type) {
case 'SET_MARKETS':
return { ...state, markets: action.payload }
case 'SELECT_MARKET':
return { ...state, selectedMarket: action.payload }
case 'SET_LOADING':
return { ...state, loading: action.payload }
default:
return state
}
}
const MarketContext = createContext<{
state: State
dispatch: Dispatch<Action>
} | undefined>(undefined)
export function MarketProvider({ children }: { children: React.ReactNode }) {
const [state, dispatch] = useReducer(reducer, {
markets: [],
selectedMarket: null,
loading: false
})
return (
<MarketContext.Provider value={{ state, dispatch }}>
{children}
</MarketContext.Provider>
)
}
export function useMarkets() {
const context = useContext(MarketContext)
if (!context) throw new Error('useMarkets must be used within MarketProvider')
return context
}
```
## Performance Optimization
### Memoization
```typescript
// ✅ useMemo for expensive computations
const sortedMarkets = useMemo(() => {
return markets.sort((a, b) => b.volume - a.volume)
}, [markets])
// ✅ useCallback for functions passed to children
const handleSearch = useCallback((query: string) => {
setSearchQuery(query)
}, [])
// ✅ React.memo for pure components
export const MarketCard = React.memo<MarketCardProps>(({ market }) => {
return (
<div className="market-card">
<h3>{market.name}</h3>
<p>{market.description}</p>
</div>
)
})
```
### Code Splitting & Lazy Loading
```typescript
import { lazy, Suspense } from 'react'
// ✅ Lazy load heavy components
const HeavyChart = lazy(() => import('./HeavyChart'))
const ThreeJsBackground = lazy(() => import('./ThreeJsBackground'))
export function Dashboard() {
return (
<div>
<Suspense fallback={<ChartSkeleton />}>
<HeavyChart data={data} />
</Suspense>
<Suspense fallback={null}>
<ThreeJsBackground />
</Suspense>
</div>
)
}
```
### Virtualization for Long Lists
```typescript
import { useVirtualizer } from '@tanstack/react-virtual'
export function VirtualMarketList({ markets }: { markets: Market[] }) {
const parentRef = useRef<HTMLDivElement>(null)
const virtualizer = useVirtualizer({
count: markets.length,
getScrollElement: () => parentRef.current,
estimateSize: () => 100, // Estimated row height
overscan: 5 // Extra items to render
})
return (
<div ref={parentRef} style={{ height: '600px', overflow: 'auto' }}>
<div
style={{
height: `${virtualizer.getTotalSize()}px`,
position: 'relative'
}}
>
{virtualizer.getVirtualItems().map(virtualRow => (
<div
key={virtualRow.index}
style={{
position: 'absolute',
top: 0,
left: 0,
width: '100%',
height: `${virtualRow.size}px`,
transform: `translateY(${virtualRow.start}px)`
}}
>
<MarketCard market={markets[virtualRow.index]} />
</div>
))}
</div>
</div>
)
}
```
## Form Handling Patterns
### Controlled Form with Validation
```typescript
interface FormData {
name: string
description: string
endDate: string
}
interface FormErrors {
name?: string
description?: string
endDate?: string
}
export function CreateMarketForm() {
const [formData, setFormData] = useState<FormData>({
name: '',
description: '',
endDate: ''
})
const [errors, setErrors] = useState<FormErrors>({})
const validate = (): boolean => {
const newErrors: FormErrors = {}
if (!formData.name.trim()) {
newErrors.name = 'Name is required'
} else if (formData.name.length > 200) {
newErrors.name = 'Name must be under 200 characters'
}
if (!formData.description.trim()) {
newErrors.description = 'Description is required'
}
if (!formData.endDate) {
newErrors.endDate = 'End date is required'
}
setErrors(newErrors)
return Object.keys(newErrors).length === 0
}
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault()
if (!validate()) return
try {
await createMarket(formData)
// Success handling
} catch (error) {
// Error handling
}
}
return (
<form onSubmit={handleSubmit}>
<input
value={formData.name}
onChange={e => setFormData(prev => ({ ...prev, name: e.target.value }))}
placeholder="Market name"
/>
{errors.name && <span className="error">{errors.name}</span>}
{/* Other fields */}
<button type="submit">Create Market</button>
</form>
)
}
```
## Error Boundary Pattern
```typescript
interface ErrorBoundaryState {
hasError: boolean
error: Error | null
}
export class ErrorBoundary extends React.Component<
{ children: React.ReactNode },
ErrorBoundaryState
> {
state: ErrorBoundaryState = {
hasError: false,
error: null
}
static getDerivedStateFromError(error: Error): ErrorBoundaryState {
return { hasError: true, error }
}
componentDidCatch(error: Error, errorInfo: React.ErrorInfo) {
console.error('Error boundary caught:', error, errorInfo)
}
render() {
if (this.state.hasError) {
return (
<div className="error-fallback">
<h2>Something went wrong</h2>
<p>{this.state.error?.message}</p>
<button onClick={() => this.setState({ hasError: false })}>
Try again
</button>
</div>
)
}
return this.props.children
}
}
// Usage
<ErrorBoundary>
<App />
</ErrorBoundary>
```
## Animation Patterns
### Framer Motion Animations
```typescript
import { motion, AnimatePresence } from 'framer-motion'
// ✅ List animations
export function AnimatedMarketList({ markets }: { markets: Market[] }) {
return (
<AnimatePresence>
{markets.map(market => (
<motion.div
key={market.id}
initial={{ opacity: 0, y: 20 }}
animate={{ opacity: 1, y: 0 }}
exit={{ opacity: 0, y: -20 }}
transition={{ duration: 0.3 }}
>
<MarketCard market={market} />
</motion.div>
))}
</AnimatePresence>
)
}
// ✅ Modal animations
export function Modal({ isOpen, onClose, children }: ModalProps) {
return (
<AnimatePresence>
{isOpen && (
<>
<motion.div
className="modal-overlay"
initial={{ opacity: 0 }}
animate={{ opacity: 1 }}
exit={{ opacity: 0 }}
onClick={onClose}
/>
<motion.div
className="modal-content"
initial={{ opacity: 0, scale: 0.9, y: 20 }}
animate={{ opacity: 1, scale: 1, y: 0 }}
exit={{ opacity: 0, scale: 0.9, y: 20 }}
>
{children}
</motion.div>
</>
)}
</AnimatePresence>
)
}
```
## Accessibility Patterns
### Keyboard Navigation
```typescript
export function Dropdown({ options, onSelect }: DropdownProps) {
const [isOpen, setIsOpen] = useState(false)
const [activeIndex, setActiveIndex] = useState(0)
const handleKeyDown = (e: React.KeyboardEvent) => {
switch (e.key) {
case 'ArrowDown':
e.preventDefault()
setActiveIndex(i => Math.min(i + 1, options.length - 1))
break
case 'ArrowUp':
e.preventDefault()
setActiveIndex(i => Math.max(i - 1, 0))
break
case 'Enter':
e.preventDefault()
onSelect(options[activeIndex])
setIsOpen(false)
break
case 'Escape':
setIsOpen(false)
break
}
}
return (
<div
role="combobox"
aria-expanded={isOpen}
aria-haspopup="listbox"
onKeyDown={handleKeyDown}
>
{/* Dropdown implementation */}
</div>
)
}
```
### Focus Management
```typescript
export function Modal({ isOpen, onClose, children }: ModalProps) {
const modalRef = useRef<HTMLDivElement>(null)
const previousFocusRef = useRef<HTMLElement | null>(null)
useEffect(() => {
if (isOpen) {
// Save currently focused element
previousFocusRef.current = document.activeElement as HTMLElement
// Focus modal
modalRef.current?.focus()
} else {
// Restore focus when closing
previousFocusRef.current?.focus()
}
}, [isOpen])
return isOpen ? (
<div
ref={modalRef}
role="dialog"
aria-modal="true"
tabIndex={-1}
onKeyDown={e => e.key === 'Escape' && onClose()}
>
{children}
</div>
) : null
}
```
**Remember**: Modern frontend patterns enable maintainable, performant user interfaces. Choose patterns that fit your project complexity.

View File

@@ -1,673 +0,0 @@
---
name: golang-patterns
description: Idiomatic Go patterns, best practices, and conventions for building robust, efficient, and maintainable Go applications.
---
# Go Development Patterns
Idiomatic Go patterns and best practices for building robust, efficient, and maintainable applications.
## When to Activate
- Writing new Go code
- Reviewing Go code
- Refactoring existing Go code
- Designing Go packages/modules
## Core Principles
### 1. Simplicity and Clarity
Go favors simplicity over cleverness. Code should be obvious and easy to read.
```go
// Good: Clear and direct
func GetUser(id string) (*User, error) {
user, err := db.FindUser(id)
if err != nil {
return nil, fmt.Errorf("get user %s: %w", id, err)
}
return user, nil
}
// Bad: Overly clever
func GetUser(id string) (*User, error) {
return func() (*User, error) {
if u, e := db.FindUser(id); e == nil {
return u, nil
} else {
return nil, e
}
}()
}
```
### 2. Make the Zero Value Useful
Design types so their zero value is immediately usable without initialization.
```go
// Good: Zero value is useful
type Counter struct {
mu sync.Mutex
count int // zero value is 0, ready to use
}
func (c *Counter) Inc() {
c.mu.Lock()
c.count++
c.mu.Unlock()
}
// Good: bytes.Buffer works with zero value
var buf bytes.Buffer
buf.WriteString("hello")
// Bad: Requires initialization
type BadCounter struct {
counts map[string]int // nil map will panic
}
```
### 3. Accept Interfaces, Return Structs
Functions should accept interface parameters and return concrete types.
```go
// Good: Accepts interface, returns concrete type
func ProcessData(r io.Reader) (*Result, error) {
data, err := io.ReadAll(r)
if err != nil {
return nil, err
}
return &Result{Data: data}, nil
}
// Bad: Returns interface (hides implementation details unnecessarily)
func ProcessData(r io.Reader) (io.Reader, error) {
// ...
}
```
## Error Handling Patterns
### Error Wrapping with Context
```go
// Good: Wrap errors with context
func LoadConfig(path string) (*Config, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("load config %s: %w", path, err)
}
var cfg Config
if err := json.Unmarshal(data, &cfg); err != nil {
return nil, fmt.Errorf("parse config %s: %w", path, err)
}
return &cfg, nil
}
```
### Custom Error Types
```go
// Define domain-specific errors
type ValidationError struct {
Field string
Message string
}
func (e *ValidationError) Error() string {
return fmt.Sprintf("validation failed on %s: %s", e.Field, e.Message)
}
// Sentinel errors for common cases
var (
ErrNotFound = errors.New("resource not found")
ErrUnauthorized = errors.New("unauthorized")
ErrInvalidInput = errors.New("invalid input")
)
```
### Error Checking with errors.Is and errors.As
```go
func HandleError(err error) {
// Check for specific error
if errors.Is(err, sql.ErrNoRows) {
log.Println("No records found")
return
}
// Check for error type
var validationErr *ValidationError
if errors.As(err, &validationErr) {
log.Printf("Validation error on field %s: %s",
validationErr.Field, validationErr.Message)
return
}
// Unknown error
log.Printf("Unexpected error: %v", err)
}
```
### Never Ignore Errors
```go
// Bad: Ignoring error with blank identifier
result, _ := doSomething()
// Good: Handle or explicitly document why it's safe to ignore
result, err := doSomething()
if err != nil {
return err
}
// Acceptable: When error truly doesn't matter (rare)
_ = writer.Close() // Best-effort cleanup, error logged elsewhere
```
## Concurrency Patterns
### Worker Pool
```go
func WorkerPool(jobs <-chan Job, results chan<- Result, numWorkers int) {
var wg sync.WaitGroup
for i := 0; i < numWorkers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for job := range jobs {
results <- process(job)
}
}()
}
wg.Wait()
close(results)
}
```
### Context for Cancellation and Timeouts
```go
func FetchWithTimeout(ctx context.Context, url string) ([]byte, error) {
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
if err != nil {
return nil, fmt.Errorf("create request: %w", err)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, fmt.Errorf("fetch %s: %w", url, err)
}
defer resp.Body.Close()
return io.ReadAll(resp.Body)
}
```
### Graceful Shutdown
```go
func GracefulShutdown(server *http.Server) {
quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
<-quit
log.Println("Shutting down server...")
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := server.Shutdown(ctx); err != nil {
log.Fatalf("Server forced to shutdown: %v", err)
}
log.Println("Server exited")
}
```
### errgroup for Coordinated Goroutines
```go
import "golang.org/x/sync/errgroup"
func FetchAll(ctx context.Context, urls []string) ([][]byte, error) {
g, ctx := errgroup.WithContext(ctx)
results := make([][]byte, len(urls))
for i, url := range urls {
i, url := i, url // Capture loop variables
g.Go(func() error {
data, err := FetchWithTimeout(ctx, url)
if err != nil {
return err
}
results[i] = data
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
return results, nil
}
```
### Avoiding Goroutine Leaks
```go
// Bad: Goroutine leak if context is cancelled
func leakyFetch(ctx context.Context, url string) <-chan []byte {
ch := make(chan []byte)
go func() {
data, _ := fetch(url)
ch <- data // Blocks forever if no receiver
}()
return ch
}
// Good: Properly handles cancellation
func safeFetch(ctx context.Context, url string) <-chan []byte {
ch := make(chan []byte, 1) // Buffered channel
go func() {
data, err := fetch(url)
if err != nil {
return
}
select {
case ch <- data:
case <-ctx.Done():
}
}()
return ch
}
```
## Interface Design
### Small, Focused Interfaces
```go
// Good: Single-method interfaces
type Reader interface {
Read(p []byte) (n int, err error)
}
type Writer interface {
Write(p []byte) (n int, err error)
}
type Closer interface {
Close() error
}
// Compose interfaces as needed
type ReadWriteCloser interface {
Reader
Writer
Closer
}
```
### Define Interfaces Where They're Used
```go
// In the consumer package, not the provider
package service
// UserStore defines what this service needs
type UserStore interface {
GetUser(id string) (*User, error)
SaveUser(user *User) error
}
type Service struct {
store UserStore
}
// Concrete implementation can be in another package
// It doesn't need to know about this interface
```
### Optional Behavior with Type Assertions
```go
type Flusher interface {
Flush() error
}
func WriteAndFlush(w io.Writer, data []byte) error {
if _, err := w.Write(data); err != nil {
return err
}
// Flush if supported
if f, ok := w.(Flusher); ok {
return f.Flush()
}
return nil
}
```
## Package Organization
### Standard Project Layout
```text
myproject/
├── cmd/
│ └── myapp/
│ └── main.go # Entry point
├── internal/
│ ├── handler/ # HTTP handlers
│ ├── service/ # Business logic
│ ├── repository/ # Data access
│ └── config/ # Configuration
├── pkg/
│ └── client/ # Public API client
├── api/
│ └── v1/ # API definitions (proto, OpenAPI)
├── testdata/ # Test fixtures
├── go.mod
├── go.sum
└── Makefile
```
### Package Naming
```go
// Good: Short, lowercase, no underscores
package http
package json
package user
// Bad: Verbose, mixed case, or redundant
package httpHandler
package json_parser
package userService // Redundant 'Service' suffix
```
### Avoid Package-Level State
```go
// Bad: Global mutable state
var db *sql.DB
func init() {
db, _ = sql.Open("postgres", os.Getenv("DATABASE_URL"))
}
// Good: Dependency injection
type Server struct {
db *sql.DB
}
func NewServer(db *sql.DB) *Server {
return &Server{db: db}
}
```
## Struct Design
### Functional Options Pattern
```go
type Server struct {
addr string
timeout time.Duration
logger *log.Logger
}
type Option func(*Server)
func WithTimeout(d time.Duration) Option {
return func(s *Server) {
s.timeout = d
}
}
func WithLogger(l *log.Logger) Option {
return func(s *Server) {
s.logger = l
}
}
func NewServer(addr string, opts ...Option) *Server {
s := &Server{
addr: addr,
timeout: 30 * time.Second, // default
logger: log.Default(), // default
}
for _, opt := range opts {
opt(s)
}
return s
}
// Usage
server := NewServer(":8080",
WithTimeout(60*time.Second),
WithLogger(customLogger),
)
```
### Embedding for Composition
```go
type Logger struct {
prefix string
}
func (l *Logger) Log(msg string) {
fmt.Printf("[%s] %s\n", l.prefix, msg)
}
type Server struct {
*Logger // Embedding - Server gets Log method
addr string
}
func NewServer(addr string) *Server {
return &Server{
Logger: &Logger{prefix: "SERVER"},
addr: addr,
}
}
// Usage
s := NewServer(":8080")
s.Log("Starting...") // Calls embedded Logger.Log
```
## Memory and Performance
### Preallocate Slices When Size is Known
```go
// Bad: Grows slice multiple times
func processItems(items []Item) []Result {
var results []Result
for _, item := range items {
results = append(results, process(item))
}
return results
}
// Good: Single allocation
func processItems(items []Item) []Result {
results := make([]Result, 0, len(items))
for _, item := range items {
results = append(results, process(item))
}
return results
}
```
### Use sync.Pool for Frequent Allocations
```go
var bufferPool = sync.Pool{
New: func() interface{} {
return new(bytes.Buffer)
},
}
func ProcessRequest(data []byte) []byte {
buf := bufferPool.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
bufferPool.Put(buf)
}()
buf.Write(data)
// Process...
return buf.Bytes()
}
```
### Avoid String Concatenation in Loops
```go
// Bad: Creates many string allocations
func join(parts []string) string {
var result string
for _, p := range parts {
result += p + ","
}
return result
}
// Good: Single allocation with strings.Builder
func join(parts []string) string {
var sb strings.Builder
for i, p := range parts {
if i > 0 {
sb.WriteString(",")
}
sb.WriteString(p)
}
return sb.String()
}
// Best: Use standard library
func join(parts []string) string {
return strings.Join(parts, ",")
}
```
## Go Tooling Integration
### Essential Commands
```bash
# Build and run
go build ./...
go run ./cmd/myapp
# Testing
go test ./...
go test -race ./...
go test -cover ./...
# Static analysis
go vet ./...
staticcheck ./...
golangci-lint run
# Module management
go mod tidy
go mod verify
# Formatting
gofmt -w .
goimports -w .
```
### Recommended Linter Configuration (.golangci.yml)
```yaml
linters:
enable:
- errcheck
- gosimple
- govet
- ineffassign
- staticcheck
- unused
- gofmt
- goimports
- misspell
- unconvert
- unparam
linters-settings:
errcheck:
check-type-assertions: true
govet:
check-shadowing: true
issues:
exclude-use-default: false
```
## Quick Reference: Go Idioms
| Idiom | Description |
|-------|-------------|
| Accept interfaces, return structs | Functions accept interface params, return concrete types |
| Errors are values | Treat errors as first-class values, not exceptions |
| Don't communicate by sharing memory | Use channels for coordination between goroutines |
| Make the zero value useful | Types should work without explicit initialization |
| A little copying is better than a little dependency | Avoid unnecessary external dependencies |
| Clear is better than clever | Prioritize readability over cleverness |
| gofmt is no one's favorite but everyone's friend | Always format with gofmt/goimports |
| Return early | Handle errors first, keep happy path unindented |
## Anti-Patterns to Avoid
```go
// Bad: Naked returns in long functions
func process() (result int, err error) {
// ... 50 lines ...
return // What is being returned?
}
// Bad: Using panic for control flow
func GetUser(id string) *User {
user, err := db.Find(id)
if err != nil {
panic(err) // Don't do this
}
return user
}
// Bad: Passing context in struct
type Request struct {
ctx context.Context // Context should be first param
ID string
}
// Good: Context as first parameter
func ProcessRequest(ctx context.Context, id string) error {
// ...
}
// Bad: Mixing value and pointer receivers
type Counter struct{ n int }
func (c Counter) Value() int { return c.n } // Value receiver
func (c *Counter) Increment() { c.n++ } // Pointer receiver
// Pick one style and be consistent
```
**Remember**: Go code should be boring in the best way - predictable, consistent, and easy to understand. When in doubt, keep it simple.

View File

@@ -1,719 +0,0 @@
---
name: golang-testing
description: Go testing patterns including table-driven tests, subtests, benchmarks, fuzzing, and test coverage. Follows TDD methodology with idiomatic Go practices.
---
# Go Testing Patterns
Comprehensive Go testing patterns for writing reliable, maintainable tests following TDD methodology.
## When to Activate
- Writing new Go functions or methods
- Adding test coverage to existing code
- Creating benchmarks for performance-critical code
- Implementing fuzz tests for input validation
- Following TDD workflow in Go projects
## TDD Workflow for Go
### The RED-GREEN-REFACTOR Cycle
```
RED → Write a failing test first
GREEN → Write minimal code to pass the test
REFACTOR → Improve code while keeping tests green
REPEAT → Continue with next requirement
```
### Step-by-Step TDD in Go
```go
// Step 1: Define the interface/signature
// calculator.go
package calculator
func Add(a, b int) int {
panic("not implemented") // Placeholder
}
// Step 2: Write failing test (RED)
// calculator_test.go
package calculator
import "testing"
func TestAdd(t *testing.T) {
got := Add(2, 3)
want := 5
if got != want {
t.Errorf("Add(2, 3) = %d; want %d", got, want)
}
}
// Step 3: Run test - verify FAIL
// $ go test
// --- FAIL: TestAdd (0.00s)
// panic: not implemented
// Step 4: Implement minimal code (GREEN)
func Add(a, b int) int {
return a + b
}
// Step 5: Run test - verify PASS
// $ go test
// PASS
// Step 6: Refactor if needed, verify tests still pass
```
## Table-Driven Tests
The standard pattern for Go tests. Enables comprehensive coverage with minimal code.
```go
func TestAdd(t *testing.T) {
tests := []struct {
name string
a, b int
expected int
}{
{"positive numbers", 2, 3, 5},
{"negative numbers", -1, -2, -3},
{"zero values", 0, 0, 0},
{"mixed signs", -1, 1, 0},
{"large numbers", 1000000, 2000000, 3000000},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := Add(tt.a, tt.b)
if got != tt.expected {
t.Errorf("Add(%d, %d) = %d; want %d",
tt.a, tt.b, got, tt.expected)
}
})
}
}
```
### Table-Driven Tests with Error Cases
```go
func TestParseConfig(t *testing.T) {
tests := []struct {
name string
input string
want *Config
wantErr bool
}{
{
name: "valid config",
input: `{"host": "localhost", "port": 8080}`,
want: &Config{Host: "localhost", Port: 8080},
},
{
name: "invalid JSON",
input: `{invalid}`,
wantErr: true,
},
{
name: "empty input",
input: "",
wantErr: true,
},
{
name: "minimal config",
input: `{}`,
want: &Config{}, // Zero value config
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ParseConfig(tt.input)
if tt.wantErr {
if err == nil {
t.Error("expected error, got nil")
}
return
}
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("got %+v; want %+v", got, tt.want)
}
})
}
}
```
## Subtests and Sub-benchmarks
### Organizing Related Tests
```go
func TestUser(t *testing.T) {
// Setup shared by all subtests
db := setupTestDB(t)
t.Run("Create", func(t *testing.T) {
user := &User{Name: "Alice"}
err := db.CreateUser(user)
if err != nil {
t.Fatalf("CreateUser failed: %v", err)
}
if user.ID == "" {
t.Error("expected user ID to be set")
}
})
t.Run("Get", func(t *testing.T) {
user, err := db.GetUser("alice-id")
if err != nil {
t.Fatalf("GetUser failed: %v", err)
}
if user.Name != "Alice" {
t.Errorf("got name %q; want %q", user.Name, "Alice")
}
})
t.Run("Update", func(t *testing.T) {
// ...
})
t.Run("Delete", func(t *testing.T) {
// ...
})
}
```
### Parallel Subtests
```go
func TestParallel(t *testing.T) {
tests := []struct {
name string
input string
}{
{"case1", "input1"},
{"case2", "input2"},
{"case3", "input3"},
}
for _, tt := range tests {
tt := tt // Capture range variable
t.Run(tt.name, func(t *testing.T) {
t.Parallel() // Run subtests in parallel
result := Process(tt.input)
// assertions...
_ = result
})
}
}
```
## Test Helpers
### Helper Functions
```go
func setupTestDB(t *testing.T) *sql.DB {
t.Helper() // Marks this as a helper function
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatalf("failed to open database: %v", err)
}
// Cleanup when test finishes
t.Cleanup(func() {
db.Close()
})
// Run migrations
if _, err := db.Exec(schema); err != nil {
t.Fatalf("failed to create schema: %v", err)
}
return db
}
func assertNoError(t *testing.T, err error) {
t.Helper()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
}
func assertEqual[T comparable](t *testing.T, got, want T) {
t.Helper()
if got != want {
t.Errorf("got %v; want %v", got, want)
}
}
```
### Temporary Files and Directories
```go
func TestFileProcessing(t *testing.T) {
// Create temp directory - automatically cleaned up
tmpDir := t.TempDir()
// Create test file
testFile := filepath.Join(tmpDir, "test.txt")
err := os.WriteFile(testFile, []byte("test content"), 0644)
if err != nil {
t.Fatalf("failed to create test file: %v", err)
}
// Run test
result, err := ProcessFile(testFile)
if err != nil {
t.Fatalf("ProcessFile failed: %v", err)
}
// Assert...
_ = result
}
```
## Golden Files
Testing against expected output files stored in `testdata/`.
```go
var update = flag.Bool("update", false, "update golden files")
func TestRender(t *testing.T) {
tests := []struct {
name string
input Template
}{
{"simple", Template{Name: "test"}},
{"complex", Template{Name: "test", Items: []string{"a", "b"}}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := Render(tt.input)
golden := filepath.Join("testdata", tt.name+".golden")
if *update {
// Update golden file: go test -update
err := os.WriteFile(golden, got, 0644)
if err != nil {
t.Fatalf("failed to update golden file: %v", err)
}
}
want, err := os.ReadFile(golden)
if err != nil {
t.Fatalf("failed to read golden file: %v", err)
}
if !bytes.Equal(got, want) {
t.Errorf("output mismatch:\ngot:\n%s\nwant:\n%s", got, want)
}
})
}
}
```
## Mocking with Interfaces
### Interface-Based Mocking
```go
// Define interface for dependencies
type UserRepository interface {
GetUser(id string) (*User, error)
SaveUser(user *User) error
}
// Production implementation
type PostgresUserRepository struct {
db *sql.DB
}
func (r *PostgresUserRepository) GetUser(id string) (*User, error) {
// Real database query
}
// Mock implementation for tests
type MockUserRepository struct {
GetUserFunc func(id string) (*User, error)
SaveUserFunc func(user *User) error
}
func (m *MockUserRepository) GetUser(id string) (*User, error) {
return m.GetUserFunc(id)
}
func (m *MockUserRepository) SaveUser(user *User) error {
return m.SaveUserFunc(user)
}
// Test using mock
func TestUserService(t *testing.T) {
mock := &MockUserRepository{
GetUserFunc: func(id string) (*User, error) {
if id == "123" {
return &User{ID: "123", Name: "Alice"}, nil
}
return nil, ErrNotFound
},
}
service := NewUserService(mock)
user, err := service.GetUserProfile("123")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if user.Name != "Alice" {
t.Errorf("got name %q; want %q", user.Name, "Alice")
}
}
```
## Benchmarks
### Basic Benchmarks
```go
func BenchmarkProcess(b *testing.B) {
data := generateTestData(1000)
b.ResetTimer() // Don't count setup time
for i := 0; i < b.N; i++ {
Process(data)
}
}
// Run: go test -bench=BenchmarkProcess -benchmem
// Output: BenchmarkProcess-8 10000 105234 ns/op 4096 B/op 10 allocs/op
```
### Benchmark with Different Sizes
```go
func BenchmarkSort(b *testing.B) {
sizes := []int{100, 1000, 10000, 100000}
for _, size := range sizes {
b.Run(fmt.Sprintf("size=%d", size), func(b *testing.B) {
data := generateRandomSlice(size)
b.ResetTimer()
for i := 0; i < b.N; i++ {
// Make a copy to avoid sorting already sorted data
tmp := make([]int, len(data))
copy(tmp, data)
sort.Ints(tmp)
}
})
}
}
```
### Memory Allocation Benchmarks
```go
func BenchmarkStringConcat(b *testing.B) {
parts := []string{"hello", "world", "foo", "bar", "baz"}
b.Run("plus", func(b *testing.B) {
for i := 0; i < b.N; i++ {
var s string
for _, p := range parts {
s += p
}
_ = s
}
})
b.Run("builder", func(b *testing.B) {
for i := 0; i < b.N; i++ {
var sb strings.Builder
for _, p := range parts {
sb.WriteString(p)
}
_ = sb.String()
}
})
b.Run("join", func(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = strings.Join(parts, "")
}
})
}
```
## Fuzzing (Go 1.18+)
### Basic Fuzz Test
```go
func FuzzParseJSON(f *testing.F) {
// Add seed corpus
f.Add(`{"name": "test"}`)
f.Add(`{"count": 123}`)
f.Add(`[]`)
f.Add(`""`)
f.Fuzz(func(t *testing.T, input string) {
var result map[string]interface{}
err := json.Unmarshal([]byte(input), &result)
if err != nil {
// Invalid JSON is expected for random input
return
}
// If parsing succeeded, re-encoding should work
_, err = json.Marshal(result)
if err != nil {
t.Errorf("Marshal failed after successful Unmarshal: %v", err)
}
})
}
// Run: go test -fuzz=FuzzParseJSON -fuzztime=30s
```
### Fuzz Test with Multiple Inputs
```go
func FuzzCompare(f *testing.F) {
f.Add("hello", "world")
f.Add("", "")
f.Add("abc", "abc")
f.Fuzz(func(t *testing.T, a, b string) {
result := Compare(a, b)
// Property: Compare(a, a) should always equal 0
if a == b && result != 0 {
t.Errorf("Compare(%q, %q) = %d; want 0", a, b, result)
}
// Property: Compare(a, b) and Compare(b, a) should have opposite signs
reverse := Compare(b, a)
if (result > 0 && reverse >= 0) || (result < 0 && reverse <= 0) {
if result != 0 || reverse != 0 {
t.Errorf("Compare(%q, %q) = %d, Compare(%q, %q) = %d; inconsistent",
a, b, result, b, a, reverse)
}
}
})
}
```
## Test Coverage
### Running Coverage
```bash
# Basic coverage
go test -cover ./...
# Generate coverage profile
go test -coverprofile=coverage.out ./...
# View coverage in browser
go tool cover -html=coverage.out
# View coverage by function
go tool cover -func=coverage.out
# Coverage with race detection
go test -race -coverprofile=coverage.out ./...
```
### Coverage Targets
| Code Type | Target |
|-----------|--------|
| Critical business logic | 100% |
| Public APIs | 90%+ |
| General code | 80%+ |
| Generated code | Exclude |
### Excluding Generated Code from Coverage
```go
//go:generate mockgen -source=interface.go -destination=mock_interface.go
// In coverage profile, exclude with build tags:
// go test -cover -tags=!generate ./...
```
## HTTP Handler Testing
```go
func TestHealthHandler(t *testing.T) {
// Create request
req := httptest.NewRequest(http.MethodGet, "/health", nil)
w := httptest.NewRecorder()
// Call handler
HealthHandler(w, req)
// Check response
resp := w.Result()
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Errorf("got status %d; want %d", resp.StatusCode, http.StatusOK)
}
body, _ := io.ReadAll(resp.Body)
if string(body) != "OK" {
t.Errorf("got body %q; want %q", body, "OK")
}
}
func TestAPIHandler(t *testing.T) {
tests := []struct {
name string
method string
path string
body string
wantStatus int
wantBody string
}{
{
name: "get user",
method: http.MethodGet,
path: "/users/123",
wantStatus: http.StatusOK,
wantBody: `{"id":"123","name":"Alice"}`,
},
{
name: "not found",
method: http.MethodGet,
path: "/users/999",
wantStatus: http.StatusNotFound,
},
{
name: "create user",
method: http.MethodPost,
path: "/users",
body: `{"name":"Bob"}`,
wantStatus: http.StatusCreated,
},
}
handler := NewAPIHandler()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var body io.Reader
if tt.body != "" {
body = strings.NewReader(tt.body)
}
req := httptest.NewRequest(tt.method, tt.path, body)
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
if w.Code != tt.wantStatus {
t.Errorf("got status %d; want %d", w.Code, tt.wantStatus)
}
if tt.wantBody != "" && w.Body.String() != tt.wantBody {
t.Errorf("got body %q; want %q", w.Body.String(), tt.wantBody)
}
})
}
}
```
## Testing Commands
```bash
# Run all tests
go test ./...
# Run tests with verbose output
go test -v ./...
# Run specific test
go test -run TestAdd ./...
# Run tests matching pattern
go test -run "TestUser/Create" ./...
# Run tests with race detector
go test -race ./...
# Run tests with coverage
go test -cover -coverprofile=coverage.out ./...
# Run short tests only
go test -short ./...
# Run tests with timeout
go test -timeout 30s ./...
# Run benchmarks
go test -bench=. -benchmem ./...
# Run fuzzing
go test -fuzz=FuzzParse -fuzztime=30s ./...
# Count test runs (for flaky test detection)
go test -count=10 ./...
```
## Best Practices
**DO:**
- Write tests FIRST (TDD)
- Use table-driven tests for comprehensive coverage
- Test behavior, not implementation
- Use `t.Helper()` in helper functions
- Use `t.Parallel()` for independent tests
- Clean up resources with `t.Cleanup()`
- Use meaningful test names that describe the scenario
**DON'T:**
- Test private functions directly (test through public API)
- Use `time.Sleep()` in tests (use channels or conditions)
- Ignore flaky tests (fix or remove them)
- Mock everything (prefer integration tests when possible)
- Skip error path testing
## Integration with CI/CD
```yaml
# GitHub Actions example
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.22'
- name: Run tests
run: go test -race -coverprofile=coverage.out ./...
- name: Check coverage
run: |
go tool cover -func=coverage.out | grep total | awk '{print $3}' | \
awk -F'%' '{if ($1 < 80) exit 1}'
```
**Remember**: Tests are documentation. They show how your code is meant to be used. Write them clearly and keep them up to date.

View File

@@ -1,202 +0,0 @@
---
name: iterative-retrieval
description: Pattern for progressively refining context retrieval to solve the subagent context problem
---
# Iterative Retrieval Pattern
Solves the "context problem" in multi-agent workflows where subagents don't know what context they need until they start working.
## The Problem
Subagents are spawned with limited context. They don't know:
- Which files contain relevant code
- What patterns exist in the codebase
- What terminology the project uses
Standard approaches fail:
- **Send everything**: Exceeds context limits
- **Send nothing**: Agent lacks critical information
- **Guess what's needed**: Often wrong
## The Solution: Iterative Retrieval
A 4-phase loop that progressively refines context:
```
┌─────────────────────────────────────────────┐
│ │
│ ┌──────────┐ ┌──────────┐ │
│ │ DISPATCH │─────▶│ EVALUATE │ │
│ └──────────┘ └──────────┘ │
│ ▲ │ │
│ │ ▼ │
│ ┌──────────┐ ┌──────────┐ │
│ │ LOOP │◀─────│ REFINE │ │
│ └──────────┘ └──────────┘ │
│ │
│ Max 3 cycles, then proceed │
└─────────────────────────────────────────────┘
```
### Phase 1: DISPATCH
Initial broad query to gather candidate files:
```javascript
// Start with high-level intent
const initialQuery = {
patterns: ['src/**/*.ts', 'lib/**/*.ts'],
keywords: ['authentication', 'user', 'session'],
excludes: ['*.test.ts', '*.spec.ts']
};
// Dispatch to retrieval agent
const candidates = await retrieveFiles(initialQuery);
```
### Phase 2: EVALUATE
Assess retrieved content for relevance:
```javascript
function evaluateRelevance(files, task) {
return files.map(file => ({
path: file.path,
relevance: scoreRelevance(file.content, task),
reason: explainRelevance(file.content, task),
missingContext: identifyGaps(file.content, task)
}));
}
```
Scoring criteria:
- **High (0.8-1.0)**: Directly implements target functionality
- **Medium (0.5-0.7)**: Contains related patterns or types
- **Low (0.2-0.4)**: Tangentially related
- **None (0-0.2)**: Not relevant, exclude
### Phase 3: REFINE
Update search criteria based on evaluation:
```javascript
function refineQuery(evaluation, previousQuery) {
return {
// Add new patterns discovered in high-relevance files
patterns: [...previousQuery.patterns, ...extractPatterns(evaluation)],
// Add terminology found in codebase
keywords: [...previousQuery.keywords, ...extractKeywords(evaluation)],
// Exclude confirmed irrelevant paths
excludes: [...previousQuery.excludes, ...evaluation
.filter(e => e.relevance < 0.2)
.map(e => e.path)
],
// Target specific gaps
focusAreas: evaluation
.flatMap(e => e.missingContext)
.filter(unique)
};
}
```
### Phase 4: LOOP
Repeat with refined criteria (max 3 cycles):
```javascript
async function iterativeRetrieve(task, maxCycles = 3) {
let query = createInitialQuery(task);
let bestContext = [];
for (let cycle = 0; cycle < maxCycles; cycle++) {
const candidates = await retrieveFiles(query);
const evaluation = evaluateRelevance(candidates, task);
// Check if we have sufficient context
const highRelevance = evaluation.filter(e => e.relevance >= 0.7);
if (highRelevance.length >= 3 && !hasCriticalGaps(evaluation)) {
return highRelevance;
}
// Refine and continue
query = refineQuery(evaluation, query);
bestContext = mergeContext(bestContext, highRelevance);
}
return bestContext;
}
```
## Practical Examples
### Example 1: Bug Fix Context
```
Task: "Fix the authentication token expiry bug"
Cycle 1:
DISPATCH: Search for "token", "auth", "expiry" in src/**
EVALUATE: Found auth.ts (0.9), tokens.ts (0.8), user.ts (0.3)
REFINE: Add "refresh", "jwt" keywords; exclude user.ts
Cycle 2:
DISPATCH: Search refined terms
EVALUATE: Found session-manager.ts (0.95), jwt-utils.ts (0.85)
REFINE: Sufficient context (2 high-relevance files)
Result: auth.ts, tokens.ts, session-manager.ts, jwt-utils.ts
```
### Example 2: Feature Implementation
```
Task: "Add rate limiting to API endpoints"
Cycle 1:
DISPATCH: Search "rate", "limit", "api" in routes/**
EVALUATE: No matches - codebase uses "throttle" terminology
REFINE: Add "throttle", "middleware" keywords
Cycle 2:
DISPATCH: Search refined terms
EVALUATE: Found throttle.ts (0.9), middleware/index.ts (0.7)
REFINE: Need router patterns
Cycle 3:
DISPATCH: Search "router", "express" patterns
EVALUATE: Found router-setup.ts (0.8)
REFINE: Sufficient context
Result: throttle.ts, middleware/index.ts, router-setup.ts
```
## Integration with Agents
Use in agent prompts:
```markdown
When retrieving context for this task:
1. Start with broad keyword search
2. Evaluate each file's relevance (0-1 scale)
3. Identify what context is still missing
4. Refine search criteria and repeat (max 3 cycles)
5. Return files with relevance >= 0.7
```
## Best Practices
1. **Start broad, narrow progressively** - Don't over-specify initial queries
2. **Learn codebase terminology** - First cycle often reveals naming conventions
3. **Track what's missing** - Explicit gap identification drives refinement
4. **Stop at "good enough"** - 3 high-relevance files beats 10 mediocre ones
5. **Exclude confidently** - Low-relevance files won't become relevant
## Related
- [The Longform Guide](https://x.com/affaanmustafa/status/2014040193557471352) - Subagent orchestration section
- `continuous-learning` skill - For patterns that improve over time
- Agent definitions in `~/.claude/agents/`

View File

@@ -1,138 +0,0 @@
---
name: java-coding-standards
description: "Java coding standards for Spring Boot services: naming, immutability, Optional usage, streams, exceptions, generics, and project layout."
---
# Java Coding Standards
Standards for readable, maintainable Java (17+) code in Spring Boot services.
## Core Principles
- Prefer clarity over cleverness
- Immutable by default; minimize shared mutable state
- Fail fast with meaningful exceptions
- Consistent naming and package structure
## Naming
```java
// ✅ Classes/Records: PascalCase
public class MarketService {}
public record Money(BigDecimal amount, Currency currency) {}
// ✅ Methods/fields: camelCase
private final MarketRepository marketRepository;
public Market findBySlug(String slug) {}
// ✅ Constants: UPPER_SNAKE_CASE
private static final int MAX_PAGE_SIZE = 100;
```
## Immutability
```java
// ✅ Favor records and final fields
public record MarketDto(Long id, String name, MarketStatus status) {}
public class Market {
private final Long id;
private final String name;
// getters only, no setters
}
```
## Optional Usage
```java
// ✅ Return Optional from find* methods
Optional<Market> market = marketRepository.findBySlug(slug);
// ✅ Map/flatMap instead of get()
return market
.map(MarketResponse::from)
.orElseThrow(() -> new EntityNotFoundException("Market not found"));
```
## Streams Best Practices
```java
// ✅ Use streams for transformations, keep pipelines short
List<String> names = markets.stream()
.map(Market::name)
.filter(Objects::nonNull)
.toList();
// ❌ Avoid complex nested streams; prefer loops for clarity
```
## Exceptions
- Use unchecked exceptions for domain errors; wrap technical exceptions with context
- Create domain-specific exceptions (e.g., `MarketNotFoundException`)
- Avoid broad `catch (Exception ex)` unless rethrowing/logging centrally
```java
throw new MarketNotFoundException(slug);
```
## Generics and Type Safety
- Avoid raw types; declare generic parameters
- Prefer bounded generics for reusable utilities
```java
public <T extends Identifiable> Map<Long, T> indexById(Collection<T> items) { ... }
```
## Project Structure (Maven/Gradle)
```
src/main/java/com/example/app/
config/
controller/
service/
repository/
domain/
dto/
util/
src/main/resources/
application.yml
src/test/java/... (mirrors main)
```
## Formatting and Style
- Use 2 or 4 spaces consistently (project standard)
- One public top-level type per file
- Keep methods short and focused; extract helpers
- Order members: constants, fields, constructors, public methods, protected, private
## Code Smells to Avoid
- Long parameter lists → use DTO/builders
- Deep nesting → early returns
- Magic numbers → named constants
- Static mutable state → prefer dependency injection
- Silent catch blocks → log and act or rethrow
## Logging
```java
private static final Logger log = LoggerFactory.getLogger(MarketService.class);
log.info("fetch_market slug={}", slug);
log.error("failed_fetch_market slug={}", slug, ex);
```
## Null Handling
- Accept `@Nullable` only when unavoidable; otherwise use `@NonNull`
- Use Bean Validation (`@NotNull`, `@NotBlank`) on inputs
## Testing Expectations
- JUnit 5 + AssertJ for fluent assertions
- Mockito for mocking; avoid partial mocks where possible
- Favor deterministic tests; no hidden sleeps
**Remember**: Keep code intentional, typed, and observable. Optimize for maintainability over micro-optimizations unless proven necessary.

View File

@@ -1,141 +0,0 @@
---
name: jpa-patterns
description: JPA/Hibernate patterns for entity design, relationships, query optimization, transactions, auditing, indexing, pagination, and pooling in Spring Boot.
---
# JPA/Hibernate Patterns
Use for data modeling, repositories, and performance tuning in Spring Boot.
## Entity Design
```java
@Entity
@Table(name = "markets", indexes = {
@Index(name = "idx_markets_slug", columnList = "slug", unique = true)
})
@EntityListeners(AuditingEntityListener.class)
public class MarketEntity {
@Id @GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
@Column(nullable = false, length = 200)
private String name;
@Column(nullable = false, unique = true, length = 120)
private String slug;
@Enumerated(EnumType.STRING)
private MarketStatus status = MarketStatus.ACTIVE;
@CreatedDate private Instant createdAt;
@LastModifiedDate private Instant updatedAt;
}
```
Enable auditing:
```java
@Configuration
@EnableJpaAuditing
class JpaConfig {}
```
## Relationships and N+1 Prevention
```java
@OneToMany(mappedBy = "market", cascade = CascadeType.ALL, orphanRemoval = true)
private List<PositionEntity> positions = new ArrayList<>();
```
- Default to lazy loading; use `JOIN FETCH` in queries when needed
- Avoid `EAGER` on collections; use DTO projections for read paths
```java
@Query("select m from MarketEntity m left join fetch m.positions where m.id = :id")
Optional<MarketEntity> findWithPositions(@Param("id") Long id);
```
## Repository Patterns
```java
public interface MarketRepository extends JpaRepository<MarketEntity, Long> {
Optional<MarketEntity> findBySlug(String slug);
@Query("select m from MarketEntity m where m.status = :status")
Page<MarketEntity> findByStatus(@Param("status") MarketStatus status, Pageable pageable);
}
```
- Use projections for lightweight queries:
```java
public interface MarketSummary {
Long getId();
String getName();
MarketStatus getStatus();
}
Page<MarketSummary> findAllBy(Pageable pageable);
```
## Transactions
- Annotate service methods with `@Transactional`
- Use `@Transactional(readOnly = true)` for read paths to optimize
- Choose propagation carefully; avoid long-running transactions
```java
@Transactional
public Market updateStatus(Long id, MarketStatus status) {
MarketEntity entity = repo.findById(id)
.orElseThrow(() -> new EntityNotFoundException("Market"));
entity.setStatus(status);
return Market.from(entity);
}
```
## Pagination
```java
PageRequest page = PageRequest.of(pageNumber, pageSize, Sort.by("createdAt").descending());
Page<MarketEntity> markets = repo.findByStatus(MarketStatus.ACTIVE, page);
```
For cursor-like pagination, include `id > :lastId` in JPQL with ordering.
## Indexing and Performance
- Add indexes for common filters (`status`, `slug`, foreign keys)
- Use composite indexes matching query patterns (`status, created_at`)
- Avoid `select *`; project only needed columns
- Batch writes with `saveAll` and `hibernate.jdbc.batch_size`
## Connection Pooling (HikariCP)
Recommended properties:
```
spring.datasource.hikari.maximum-pool-size=20
spring.datasource.hikari.minimum-idle=5
spring.datasource.hikari.connection-timeout=30000
spring.datasource.hikari.validation-timeout=5000
```
For PostgreSQL LOB handling, add:
```
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
```
## Caching
- 1st-level cache is per EntityManager; avoid keeping entities across transactions
- For read-heavy entities, consider second-level cache cautiously; validate eviction strategy
## Migrations
- Use Flyway or Liquibase; never rely on Hibernate auto DDL in production
- Keep migrations idempotent and additive; avoid dropping columns without plan
## Testing Data Access
- Prefer `@DataJpaTest` with Testcontainers to mirror production
- Assert SQL efficiency using logs: set `logging.level.org.hibernate.SQL=DEBUG` and `logging.level.org.hibernate.orm.jdbc.bind=TRACE` for parameter values
**Remember**: Keep entities lean, queries intentional, and transactions short. Prevent N+1 with fetch strategies and projections, and index for your read/write paths.

View File

@@ -1,165 +0,0 @@
---
name: nutrient-document-processing
description: Process, convert, OCR, extract, redact, sign, and fill documents using the Nutrient DWS API. Works with PDFs, DOCX, XLSX, PPTX, HTML, and images.
---
# Nutrient Document Processing
Process documents with the [Nutrient DWS Processor API](https://www.nutrient.io/api/). Convert formats, extract text and tables, OCR scanned documents, redact PII, add watermarks, digitally sign, and fill PDF forms.
## Setup
Get a free API key at **[nutrient.io](https://dashboard.nutrient.io/sign_up/?product=processor)**
```bash
export NUTRIENT_API_KEY="pdf_live_..."
```
All requests go to `https://api.nutrient.io/build` as multipart POST with an `instructions` JSON field.
## Operations
### Convert Documents
```bash
# DOCX to PDF
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.docx=@document.docx" \
-F 'instructions={"parts":[{"file":"document.docx"}]}' \
-o output.pdf
# PDF to DOCX
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"output":{"type":"docx"}}' \
-o output.docx
# HTML to PDF
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "index.html=@index.html" \
-F 'instructions={"parts":[{"html":"index.html"}]}' \
-o output.pdf
```
Supported inputs: PDF, DOCX, XLSX, PPTX, DOC, XLS, PPT, PPS, PPSX, ODT, RTF, HTML, JPG, PNG, TIFF, HEIC, GIF, WebP, SVG, TGA, EPS.
### Extract Text and Data
```bash
# Extract plain text
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"output":{"type":"text"}}' \
-o output.txt
# Extract tables as Excel
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"output":{"type":"xlsx"}}' \
-o tables.xlsx
```
### OCR Scanned Documents
```bash
# OCR to searchable PDF (supports 100+ languages)
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "scanned.pdf=@scanned.pdf" \
-F 'instructions={"parts":[{"file":"scanned.pdf"}],"actions":[{"type":"ocr","language":"english"}]}' \
-o searchable.pdf
```
Languages: Supports 100+ languages via ISO 639-2 codes (e.g., `eng`, `deu`, `fra`, `spa`, `jpn`, `kor`, `chi_sim`, `chi_tra`, `ara`, `hin`, `rus`). Full language names like `english` or `german` also work. See the [complete OCR language table](https://www.nutrient.io/guides/document-engine/ocr/language-support/) for all supported codes.
### Redact Sensitive Information
```bash
# Pattern-based (SSN, email)
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"actions":[{"type":"redaction","strategy":"preset","strategyOptions":{"preset":"social-security-number"}},{"type":"redaction","strategy":"preset","strategyOptions":{"preset":"email-address"}}]}' \
-o redacted.pdf
# Regex-based
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"actions":[{"type":"redaction","strategy":"regex","strategyOptions":{"regex":"\\b[A-Z]{2}\\d{6}\\b"}}]}' \
-o redacted.pdf
```
Presets: `social-security-number`, `email-address`, `credit-card-number`, `international-phone-number`, `north-american-phone-number`, `date`, `time`, `url`, `ipv4`, `ipv6`, `mac-address`, `us-zip-code`, `vin`.
### Add Watermarks
```bash
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"actions":[{"type":"watermark","text":"CONFIDENTIAL","fontSize":72,"opacity":0.3,"rotation":-45}]}' \
-o watermarked.pdf
```
### Digital Signatures
```bash
# Self-signed CMS signature
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"actions":[{"type":"sign","signatureType":"cms"}]}' \
-o signed.pdf
```
### Fill PDF Forms
```bash
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "form.pdf=@form.pdf" \
-F 'instructions={"parts":[{"file":"form.pdf"}],"actions":[{"type":"fillForm","formFields":{"name":"Jane Smith","email":"jane@example.com","date":"2026-02-06"}}]}' \
-o filled.pdf
```
## MCP Server (Alternative)
For native tool integration, use the MCP server instead of curl:
```json
{
"mcpServers": {
"nutrient-dws": {
"command": "npx",
"args": ["-y", "@nutrient-sdk/dws-mcp-server"],
"env": {
"NUTRIENT_DWS_API_KEY": "YOUR_API_KEY",
"SANDBOX_PATH": "/path/to/working/directory"
}
}
}
}
```
## When to Use
- Converting documents between formats (PDF, DOCX, XLSX, PPTX, HTML, images)
- Extracting text, tables, or key-value pairs from PDFs
- OCR on scanned documents or images
- Redacting PII before sharing documents
- Adding watermarks to drafts or confidential documents
- Digitally signing contracts or agreements
- Filling PDF forms programmatically
## Links
- [API Playground](https://dashboard.nutrient.io/processor-api/playground/)
- [Full API Docs](https://www.nutrient.io/guides/dws-processor/)
- [Agent Skill Repo](https://github.com/PSPDFKit-labs/nutrient-agent-skill)
- [npm MCP Server](https://www.npmjs.com/package/@nutrient-sdk/dws-mcp-server)

View File

@@ -1,146 +0,0 @@
---
name: postgres-patterns
description: PostgreSQL database patterns for query optimization, schema design, indexing, and security. Based on Supabase best practices.
---
# PostgreSQL Patterns
Quick reference for PostgreSQL best practices. For detailed guidance, use the `database-reviewer` agent.
## When to Activate
- Writing SQL queries or migrations
- Designing database schemas
- Troubleshooting slow queries
- Implementing Row Level Security
- Setting up connection pooling
## Quick Reference
### Index Cheat Sheet
| Query Pattern | Index Type | Example |
|--------------|------------|---------|
| `WHERE col = value` | B-tree (default) | `CREATE INDEX idx ON t (col)` |
| `WHERE col > value` | B-tree | `CREATE INDEX idx ON t (col)` |
| `WHERE a = x AND b > y` | Composite | `CREATE INDEX idx ON t (a, b)` |
| `WHERE jsonb @> '{}'` | GIN | `CREATE INDEX idx ON t USING gin (col)` |
| `WHERE tsv @@ query` | GIN | `CREATE INDEX idx ON t USING gin (col)` |
| Time-series ranges | BRIN | `CREATE INDEX idx ON t USING brin (col)` |
### Data Type Quick Reference
| Use Case | Correct Type | Avoid |
|----------|-------------|-------|
| IDs | `bigint` | `int`, random UUID |
| Strings | `text` | `varchar(255)` |
| Timestamps | `timestamptz` | `timestamp` |
| Money | `numeric(10,2)` | `float` |
| Flags | `boolean` | `varchar`, `int` |
### Common Patterns
**Composite Index Order:**
```sql
-- Equality columns first, then range columns
CREATE INDEX idx ON orders (status, created_at);
-- Works for: WHERE status = 'pending' AND created_at > '2024-01-01'
```
**Covering Index:**
```sql
CREATE INDEX idx ON users (email) INCLUDE (name, created_at);
-- Avoids table lookup for SELECT email, name, created_at
```
**Partial Index:**
```sql
CREATE INDEX idx ON users (email) WHERE deleted_at IS NULL;
-- Smaller index, only includes active users
```
**RLS Policy (Optimized):**
```sql
CREATE POLICY policy ON orders
USING ((SELECT auth.uid()) = user_id); -- Wrap in SELECT!
```
**UPSERT:**
```sql
INSERT INTO settings (user_id, key, value)
VALUES (123, 'theme', 'dark')
ON CONFLICT (user_id, key)
DO UPDATE SET value = EXCLUDED.value;
```
**Cursor Pagination:**
```sql
SELECT * FROM products WHERE id > $last_id ORDER BY id LIMIT 20;
-- O(1) vs OFFSET which is O(n)
```
**Queue Processing:**
```sql
UPDATE jobs SET status = 'processing'
WHERE id = (
SELECT id FROM jobs WHERE status = 'pending'
ORDER BY created_at LIMIT 1
FOR UPDATE SKIP LOCKED
) RETURNING *;
```
### Anti-Pattern Detection
```sql
-- Find unindexed foreign keys
SELECT conrelid::regclass, a.attname
FROM pg_constraint c
JOIN pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = ANY(c.conkey)
WHERE c.contype = 'f'
AND NOT EXISTS (
SELECT 1 FROM pg_index i
WHERE i.indrelid = c.conrelid AND a.attnum = ANY(i.indkey)
);
-- Find slow queries
SELECT query, mean_exec_time, calls
FROM pg_stat_statements
WHERE mean_exec_time > 100
ORDER BY mean_exec_time DESC;
-- Check table bloat
SELECT relname, n_dead_tup, last_vacuum
FROM pg_stat_user_tables
WHERE n_dead_tup > 1000
ORDER BY n_dead_tup DESC;
```
### Configuration Template
```sql
-- Connection limits (adjust for RAM)
ALTER SYSTEM SET max_connections = 100;
ALTER SYSTEM SET work_mem = '8MB';
-- Timeouts
ALTER SYSTEM SET idle_in_transaction_session_timeout = '30s';
ALTER SYSTEM SET statement_timeout = '30s';
-- Monitoring
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
-- Security defaults
REVOKE ALL ON SCHEMA public FROM public;
SELECT pg_reload_conf();
```
## Related
- Agent: `database-reviewer` - Full database review workflow
- Skill: `clickhouse-io` - ClickHouse analytics patterns
- Skill: `backend-patterns` - API and backend patterns
---
*Based on [Supabase Agent Skills](https://github.com/supabase/agent-skills) (MIT License)*

View File

@@ -1,348 +0,0 @@
---
name: project-guidelines-example
description: "Example project-specific skill template based on a real production application."
---
# Project Guidelines Skill (Example)
This is an example of a project-specific skill. Use this as a template for your own projects.
Based on a real production application: [Zenith](https://zenith.chat) - AI-powered customer discovery platform.
## When to Use
Reference this skill when working on the specific project it's designed for. Project skills contain:
- Architecture overview
- File structure
- Code patterns
- Testing requirements
- Deployment workflow
---
## Architecture Overview
**Tech Stack:**
- **Frontend**: Next.js 15 (App Router), TypeScript, React
- **Backend**: FastAPI (Python), Pydantic models
- **Database**: Supabase (PostgreSQL)
- **AI**: Claude API with tool calling and structured output
- **Deployment**: Google Cloud Run
- **Testing**: Playwright (E2E), pytest (backend), React Testing Library
**Services:**
```
┌─────────────────────────────────────────────────────────────┐
│ Frontend │
│ Next.js 15 + TypeScript + TailwindCSS │
│ Deployed: Vercel / Cloud Run │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Backend │
│ FastAPI + Python 3.11 + Pydantic │
│ Deployed: Cloud Run │
└─────────────────────────────────────────────────────────────┘
┌───────────────┼───────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Supabase │ │ Claude │ │ Redis │
│ Database │ │ API │ │ Cache │
└──────────┘ └──────────┘ └──────────┘
```
---
## File Structure
```
project/
├── frontend/
│ └── src/
│ ├── app/ # Next.js app router pages
│ │ ├── api/ # API routes
│ │ ├── (auth)/ # Auth-protected routes
│ │ └── workspace/ # Main app workspace
│ ├── components/ # React components
│ │ ├── ui/ # Base UI components
│ │ ├── forms/ # Form components
│ │ └── layouts/ # Layout components
│ ├── hooks/ # Custom React hooks
│ ├── lib/ # Utilities
│ ├── types/ # TypeScript definitions
│ └── config/ # Configuration
├── backend/
│ ├── routers/ # FastAPI route handlers
│ ├── models.py # Pydantic models
│ ├── main.py # FastAPI app entry
│ ├── auth_system.py # Authentication
│ ├── database.py # Database operations
│ ├── services/ # Business logic
│ └── tests/ # pytest tests
├── deploy/ # Deployment configs
├── docs/ # Documentation
└── scripts/ # Utility scripts
```
---
## Code Patterns
### API Response Format (FastAPI)
```python
from pydantic import BaseModel
from typing import Generic, TypeVar, Optional
T = TypeVar('T')
class ApiResponse(BaseModel, Generic[T]):
success: bool
data: Optional[T] = None
error: Optional[str] = None
@classmethod
def ok(cls, data: T) -> "ApiResponse[T]":
return cls(success=True, data=data)
@classmethod
def fail(cls, error: str) -> "ApiResponse[T]":
return cls(success=False, error=error)
```
### Frontend API Calls (TypeScript)
```typescript
interface ApiResponse<T> {
success: boolean
data?: T
error?: string
}
async function fetchApi<T>(
endpoint: string,
options?: RequestInit
): Promise<ApiResponse<T>> {
try {
const response = await fetch(`/api${endpoint}`, {
...options,
headers: {
'Content-Type': 'application/json',
...options?.headers,
},
})
if (!response.ok) {
return { success: false, error: `HTTP ${response.status}` }
}
return await response.json()
} catch (error) {
return { success: false, error: String(error) }
}
}
```
### Claude AI Integration (Structured Output)
```python
from anthropic import Anthropic
from pydantic import BaseModel
class AnalysisResult(BaseModel):
summary: str
key_points: list[str]
confidence: float
async def analyze_with_claude(content: str) -> AnalysisResult:
client = Anthropic()
response = client.messages.create(
model="claude-sonnet-4-5-20250514",
max_tokens=1024,
messages=[{"role": "user", "content": content}],
tools=[{
"name": "provide_analysis",
"description": "Provide structured analysis",
"input_schema": AnalysisResult.model_json_schema()
}],
tool_choice={"type": "tool", "name": "provide_analysis"}
)
# Extract tool use result
tool_use = next(
block for block in response.content
if block.type == "tool_use"
)
return AnalysisResult(**tool_use.input)
```
### Custom Hooks (React)
```typescript
import { useState, useCallback } from 'react'
interface UseApiState<T> {
data: T | null
loading: boolean
error: string | null
}
export function useApi<T>(
fetchFn: () => Promise<ApiResponse<T>>
) {
const [state, setState] = useState<UseApiState<T>>({
data: null,
loading: false,
error: null,
})
const execute = useCallback(async () => {
setState(prev => ({ ...prev, loading: true, error: null }))
const result = await fetchFn()
if (result.success) {
setState({ data: result.data!, loading: false, error: null })
} else {
setState({ data: null, loading: false, error: result.error! })
}
}, [fetchFn])
return { ...state, execute }
}
```
---
## Testing Requirements
### Backend (pytest)
```bash
# Run all tests
poetry run pytest tests/
# Run with coverage
poetry run pytest tests/ --cov=. --cov-report=html
# Run specific test file
poetry run pytest tests/test_auth.py -v
```
**Test structure:**
```python
import pytest
from httpx import AsyncClient
from main import app
@pytest.fixture
async def client():
async with AsyncClient(app=app, base_url="http://test") as ac:
yield ac
@pytest.mark.asyncio
async def test_health_check(client: AsyncClient):
response = await client.get("/health")
assert response.status_code == 200
assert response.json()["status"] == "healthy"
```
### Frontend (React Testing Library)
```bash
# Run tests
npm run test
# Run with coverage
npm run test -- --coverage
# Run E2E tests
npm run test:e2e
```
**Test structure:**
```typescript
import { render, screen, fireEvent } from '@testing-library/react'
import { WorkspacePanel } from './WorkspacePanel'
describe('WorkspacePanel', () => {
it('renders workspace correctly', () => {
render(<WorkspacePanel />)
expect(screen.getByRole('main')).toBeInTheDocument()
})
it('handles session creation', async () => {
render(<WorkspacePanel />)
fireEvent.click(screen.getByText('New Session'))
expect(await screen.findByText('Session created')).toBeInTheDocument()
})
})
```
---
## Deployment Workflow
### Pre-Deployment Checklist
- [ ] All tests passing locally
- [ ] `npm run build` succeeds (frontend)
- [ ] `poetry run pytest` passes (backend)
- [ ] No hardcoded secrets
- [ ] Environment variables documented
- [ ] Database migrations ready
### Deployment Commands
```bash
# Build and deploy frontend
cd frontend && npm run build
gcloud run deploy frontend --source .
# Build and deploy backend
cd backend
gcloud run deploy backend --source .
```
### Environment Variables
```bash
# Frontend (.env.local)
NEXT_PUBLIC_API_URL=https://api.example.com
NEXT_PUBLIC_SUPABASE_URL=https://xxx.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJ...
# Backend (.env)
DATABASE_URL=postgresql://...
ANTHROPIC_API_KEY=sk-ant-...
SUPABASE_URL=https://xxx.supabase.co
SUPABASE_KEY=eyJ...
```
---
## Critical Rules
1. **No emojis** in code, comments, or documentation
2. **Immutability** - never mutate objects or arrays
3. **TDD** - write tests before implementation
4. **80% coverage** minimum
5. **Many small files** - 200-400 lines typical, 800 max
6. **No console.log** in production code
7. **Proper error handling** with try/catch
8. **Input validation** with Pydantic/Zod
---
## Related Skills
- `coding-standards.md` - General coding best practices
- `backend-patterns.md` - API and database patterns
- `frontend-patterns.md` - React and Next.js patterns
- `tdd-workflow/` - Test-driven development methodology

View File

@@ -1,749 +0,0 @@
---
name: python-patterns
description: Pythonic idioms, PEP 8 standards, type hints, and best practices for building robust, efficient, and maintainable Python applications.
---
# Python Development Patterns
Idiomatic Python patterns and best practices for building robust, efficient, and maintainable applications.
## When to Activate
- Writing new Python code
- Reviewing Python code
- Refactoring existing Python code
- Designing Python packages/modules
## Core Principles
### 1. Readability Counts
Python prioritizes readability. Code should be obvious and easy to understand.
```python
# Good: Clear and readable
def get_active_users(users: list[User]) -> list[User]:
"""Return only active users from the provided list."""
return [user for user in users if user.is_active]
# Bad: Clever but confusing
def get_active_users(u):
return [x for x in u if x.a]
```
### 2. Explicit is Better Than Implicit
Avoid magic; be clear about what your code does.
```python
# Good: Explicit configuration
import logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
# Bad: Hidden side effects
import some_module
some_module.setup() # What does this do?
```
### 3. EAFP - Easier to Ask Forgiveness Than Permission
Python prefers exception handling over checking conditions.
```python
# Good: EAFP style
def get_value(dictionary: dict, key: str) -> Any:
try:
return dictionary[key]
except KeyError:
return default_value
# Bad: LBYL (Look Before You Leap) style
def get_value(dictionary: dict, key: str) -> Any:
if key in dictionary:
return dictionary[key]
else:
return default_value
```
## Type Hints
### Basic Type Annotations
```python
from typing import Optional, List, Dict, Any
def process_user(
user_id: str,
data: Dict[str, Any],
active: bool = True
) -> Optional[User]:
"""Process a user and return the updated User or None."""
if not active:
return None
return User(user_id, data)
```
### Modern Type Hints (Python 3.9+)
```python
# Python 3.9+ - Use built-in types
def process_items(items: list[str]) -> dict[str, int]:
return {item: len(item) for item in items}
# Python 3.8 and earlier - Use typing module
from typing import List, Dict
def process_items(items: List[str]) -> Dict[str, int]:
return {item: len(item) for item in items}
```
### Type Aliases and TypeVar
```python
from typing import TypeVar, Union
# Type alias for complex types
JSON = Union[dict[str, Any], list[Any], str, int, float, bool, None]
def parse_json(data: str) -> JSON:
return json.loads(data)
# Generic types
T = TypeVar('T')
def first(items: list[T]) -> T | None:
"""Return the first item or None if list is empty."""
return items[0] if items else None
```
### Protocol-Based Duck Typing
```python
from typing import Protocol
class Renderable(Protocol):
def render(self) -> str:
"""Render the object to a string."""
def render_all(items: list[Renderable]) -> str:
"""Render all items that implement the Renderable protocol."""
return "\n".join(item.render() for item in items)
```
## Error Handling Patterns
### Specific Exception Handling
```python
# Good: Catch specific exceptions
def load_config(path: str) -> Config:
try:
with open(path) as f:
return Config.from_json(f.read())
except FileNotFoundError as e:
raise ConfigError(f"Config file not found: {path}") from e
except json.JSONDecodeError as e:
raise ConfigError(f"Invalid JSON in config: {path}") from e
# Bad: Bare except
def load_config(path: str) -> Config:
try:
with open(path) as f:
return Config.from_json(f.read())
except:
return None # Silent failure!
```
### Exception Chaining
```python
def process_data(data: str) -> Result:
try:
parsed = json.loads(data)
except json.JSONDecodeError as e:
# Chain exceptions to preserve the traceback
raise ValueError(f"Failed to parse data: {data}") from e
```
### Custom Exception Hierarchy
```python
class AppError(Exception):
"""Base exception for all application errors."""
pass
class ValidationError(AppError):
"""Raised when input validation fails."""
pass
class NotFoundError(AppError):
"""Raised when a requested resource is not found."""
pass
# Usage
def get_user(user_id: str) -> User:
user = db.find_user(user_id)
if not user:
raise NotFoundError(f"User not found: {user_id}")
return user
```
## Context Managers
### Resource Management
```python
# Good: Using context managers
def process_file(path: str) -> str:
with open(path, 'r') as f:
return f.read()
# Bad: Manual resource management
def process_file(path: str) -> str:
f = open(path, 'r')
try:
return f.read()
finally:
f.close()
```
### Custom Context Managers
```python
from contextlib import contextmanager
@contextmanager
def timer(name: str):
"""Context manager to time a block of code."""
start = time.perf_counter()
yield
elapsed = time.perf_counter() - start
print(f"{name} took {elapsed:.4f} seconds")
# Usage
with timer("data processing"):
process_large_dataset()
```
### Context Manager Classes
```python
class DatabaseTransaction:
def __init__(self, connection):
self.connection = connection
def __enter__(self):
self.connection.begin_transaction()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is None:
self.connection.commit()
else:
self.connection.rollback()
return False # Don't suppress exceptions
# Usage
with DatabaseTransaction(conn):
user = conn.create_user(user_data)
conn.create_profile(user.id, profile_data)
```
## Comprehensions and Generators
### List Comprehensions
```python
# Good: List comprehension for simple transformations
names = [user.name for user in users if user.is_active]
# Bad: Manual loop
names = []
for user in users:
if user.is_active:
names.append(user.name)
# Complex comprehensions should be expanded
# Bad: Too complex
result = [x * 2 for x in items if x > 0 if x % 2 == 0]
# Good: Use a generator function
def filter_and_transform(items: Iterable[int]) -> list[int]:
result = []
for x in items:
if x > 0 and x % 2 == 0:
result.append(x * 2)
return result
```
### Generator Expressions
```python
# Good: Generator for lazy evaluation
total = sum(x * x for x in range(1_000_000))
# Bad: Creates large intermediate list
total = sum([x * x for x in range(1_000_000)])
```
### Generator Functions
```python
def read_large_file(path: str) -> Iterator[str]:
"""Read a large file line by line."""
with open(path) as f:
for line in f:
yield line.strip()
# Usage
for line in read_large_file("huge.txt"):
process(line)
```
## Data Classes and Named Tuples
### Data Classes
```python
from dataclasses import dataclass, field
from datetime import datetime
@dataclass
class User:
"""User entity with automatic __init__, __repr__, and __eq__."""
id: str
name: str
email: str
created_at: datetime = field(default_factory=datetime.now)
is_active: bool = True
# Usage
user = User(
id="123",
name="Alice",
email="alice@example.com"
)
```
### Data Classes with Validation
```python
@dataclass
class User:
email: str
age: int
def __post_init__(self):
# Validate email format
if "@" not in self.email:
raise ValueError(f"Invalid email: {self.email}")
# Validate age range
if self.age < 0 or self.age > 150:
raise ValueError(f"Invalid age: {self.age}")
```
### Named Tuples
```python
from typing import NamedTuple
class Point(NamedTuple):
"""Immutable 2D point."""
x: float
y: float
def distance(self, other: 'Point') -> float:
return ((self.x - other.x) ** 2 + (self.y - other.y) ** 2) ** 0.5
# Usage
p1 = Point(0, 0)
p2 = Point(3, 4)
print(p1.distance(p2)) # 5.0
```
## Decorators
### Function Decorators
```python
import functools
import time
def timer(func: Callable) -> Callable:
"""Decorator to time function execution."""
@functools.wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = func(*args, **kwargs)
elapsed = time.perf_counter() - start
print(f"{func.__name__} took {elapsed:.4f}s")
return result
return wrapper
@timer
def slow_function():
time.sleep(1)
# slow_function() prints: slow_function took 1.0012s
```
### Parameterized Decorators
```python
def repeat(times: int):
"""Decorator to repeat a function multiple times."""
def decorator(func: Callable) -> Callable:
@functools.wraps(func)
def wrapper(*args, **kwargs):
results = []
for _ in range(times):
results.append(func(*args, **kwargs))
return results
return wrapper
return decorator
@repeat(times=3)
def greet(name: str) -> str:
return f"Hello, {name}!"
# greet("Alice") returns ["Hello, Alice!", "Hello, Alice!", "Hello, Alice!"]
```
### Class-Based Decorators
```python
class CountCalls:
"""Decorator that counts how many times a function is called."""
def __init__(self, func: Callable):
functools.update_wrapper(self, func)
self.func = func
self.count = 0
def __call__(self, *args, **kwargs):
self.count += 1
print(f"{self.func.__name__} has been called {self.count} times")
return self.func(*args, **kwargs)
@CountCalls
def process():
pass
# Each call to process() prints the call count
```
## Concurrency Patterns
### Threading for I/O-Bound Tasks
```python
import concurrent.futures
import threading
def fetch_url(url: str) -> str:
"""Fetch a URL (I/O-bound operation)."""
import urllib.request
with urllib.request.urlopen(url) as response:
return response.read().decode()
def fetch_all_urls(urls: list[str]) -> dict[str, str]:
"""Fetch multiple URLs concurrently using threads."""
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
future_to_url = {executor.submit(fetch_url, url): url for url in urls}
results = {}
for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future]
try:
results[url] = future.result()
except Exception as e:
results[url] = f"Error: {e}"
return results
```
### Multiprocessing for CPU-Bound Tasks
```python
def process_data(data: list[int]) -> int:
"""CPU-intensive computation."""
return sum(x ** 2 for x in data)
def process_all(datasets: list[list[int]]) -> list[int]:
"""Process multiple datasets using multiple processes."""
with concurrent.futures.ProcessPoolExecutor() as executor:
results = list(executor.map(process_data, datasets))
return results
```
### Async/Await for Concurrent I/O
```python
import asyncio
async def fetch_async(url: str) -> str:
"""Fetch a URL asynchronously."""
import aiohttp
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
return await response.text()
async def fetch_all(urls: list[str]) -> dict[str, str]:
"""Fetch multiple URLs concurrently."""
tasks = [fetch_async(url) for url in urls]
results = await asyncio.gather(*tasks, return_exceptions=True)
return dict(zip(urls, results))
```
## Package Organization
### Standard Project Layout
```
myproject/
├── src/
│ └── mypackage/
│ ├── __init__.py
│ ├── main.py
│ ├── api/
│ │ ├── __init__.py
│ │ └── routes.py
│ ├── models/
│ │ ├── __init__.py
│ │ └── user.py
│ └── utils/
│ ├── __init__.py
│ └── helpers.py
├── tests/
│ ├── __init__.py
│ ├── conftest.py
│ ├── test_api.py
│ └── test_models.py
├── pyproject.toml
├── README.md
└── .gitignore
```
### Import Conventions
```python
# Good: Import order - stdlib, third-party, local
import os
import sys
from pathlib import Path
import requests
from fastapi import FastAPI
from mypackage.models import User
from mypackage.utils import format_name
# Good: Use isort for automatic import sorting
# pip install isort
```
### __init__.py for Package Exports
```python
# mypackage/__init__.py
"""mypackage - A sample Python package."""
__version__ = "1.0.0"
# Export main classes/functions at package level
from mypackage.models import User, Post
from mypackage.utils import format_name
__all__ = ["User", "Post", "format_name"]
```
## Memory and Performance
### Using __slots__ for Memory Efficiency
```python
# Bad: Regular class uses __dict__ (more memory)
class Point:
def __init__(self, x: float, y: float):
self.x = x
self.y = y
# Good: __slots__ reduces memory usage
class Point:
__slots__ = ['x', 'y']
def __init__(self, x: float, y: float):
self.x = x
self.y = y
```
### Generator for Large Data
```python
# Bad: Returns full list in memory
def read_lines(path: str) -> list[str]:
with open(path) as f:
return [line.strip() for line in f]
# Good: Yields lines one at a time
def read_lines(path: str) -> Iterator[str]:
with open(path) as f:
for line in f:
yield line.strip()
```
### Avoid String Concatenation in Loops
```python
# Bad: O(n²) due to string immutability
result = ""
for item in items:
result += str(item)
# Good: O(n) using join
result = "".join(str(item) for item in items)
# Good: Using StringIO for building
from io import StringIO
buffer = StringIO()
for item in items:
buffer.write(str(item))
result = buffer.getvalue()
```
## Python Tooling Integration
### Essential Commands
```bash
# Code formatting
black .
isort .
# Linting
ruff check .
pylint mypackage/
# Type checking
mypy .
# Testing
pytest --cov=mypackage --cov-report=html
# Security scanning
bandit -r .
# Dependency management
pip-audit
safety check
```
### pyproject.toml Configuration
```toml
[project]
name = "mypackage"
version = "1.0.0"
requires-python = ">=3.9"
dependencies = [
"requests>=2.31.0",
"pydantic>=2.0.0",
]
[project.optional-dependencies]
dev = [
"pytest>=7.4.0",
"pytest-cov>=4.1.0",
"black>=23.0.0",
"ruff>=0.1.0",
"mypy>=1.5.0",
]
[tool.black]
line-length = 88
target-version = ['py39']
[tool.ruff]
line-length = 88
select = ["E", "F", "I", "N", "W"]
[tool.mypy]
python_version = "3.9"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "--cov=mypackage --cov-report=term-missing"
```
## Quick Reference: Python Idioms
| Idiom | Description |
|-------|-------------|
| EAFP | Easier to Ask Forgiveness than Permission |
| Context managers | Use `with` for resource management |
| List comprehensions | For simple transformations |
| Generators | For lazy evaluation and large datasets |
| Type hints | Annotate function signatures |
| Dataclasses | For data containers with auto-generated methods |
| `__slots__` | For memory optimization |
| f-strings | For string formatting (Python 3.6+) |
| `pathlib.Path` | For path operations (Python 3.4+) |
| `enumerate` | For index-element pairs in loops |
## Anti-Patterns to Avoid
```python
# Bad: Mutable default arguments
def append_to(item, items=[]):
items.append(item)
return items
# Good: Use None and create new list
def append_to(item, items=None):
if items is None:
items = []
items.append(item)
return items
# Bad: Checking type with type()
if type(obj) == list:
process(obj)
# Good: Use isinstance
if isinstance(obj, list):
process(obj)
# Bad: Comparing to None with ==
if value == None:
process()
# Good: Use is
if value is None:
process()
# Bad: from module import *
from os.path import *
# Good: Explicit imports
from os.path import join, exists
# Bad: Bare except
try:
risky_operation()
except:
pass
# Good: Specific exception
try:
risky_operation()
except SpecificError as e:
logger.error(f"Operation failed: {e}")
```
__Remember__: Python code should be readable, explicit, and follow the principle of least surprise. When in doubt, prioritize clarity over cleverness.

View File

@@ -1,815 +0,0 @@
---
name: python-testing
description: Python testing strategies using pytest, TDD methodology, fixtures, mocking, parametrization, and coverage requirements.
---
# Python Testing Patterns
Comprehensive testing strategies for Python applications using pytest, TDD methodology, and best practices.
## When to Activate
- Writing new Python code (follow TDD: red, green, refactor)
- Designing test suites for Python projects
- Reviewing Python test coverage
- Setting up testing infrastructure
## Core Testing Philosophy
### Test-Driven Development (TDD)
Always follow the TDD cycle:
1. **RED**: Write a failing test for the desired behavior
2. **GREEN**: Write minimal code to make the test pass
3. **REFACTOR**: Improve code while keeping tests green
```python
# Step 1: Write failing test (RED)
def test_add_numbers():
result = add(2, 3)
assert result == 5
# Step 2: Write minimal implementation (GREEN)
def add(a, b):
return a + b
# Step 3: Refactor if needed (REFACTOR)
```
### Coverage Requirements
- **Target**: 80%+ code coverage
- **Critical paths**: 100% coverage required
- Use `pytest --cov` to measure coverage
```bash
pytest --cov=mypackage --cov-report=term-missing --cov-report=html
```
## pytest Fundamentals
### Basic Test Structure
```python
import pytest
def test_addition():
"""Test basic addition."""
assert 2 + 2 == 4
def test_string_uppercase():
"""Test string uppercasing."""
text = "hello"
assert text.upper() == "HELLO"
def test_list_append():
"""Test list append."""
items = [1, 2, 3]
items.append(4)
assert 4 in items
assert len(items) == 4
```
### Assertions
```python
# Equality
assert result == expected
# Inequality
assert result != unexpected
# Truthiness
assert result # Truthy
assert not result # Falsy
assert result is True # Exactly True
assert result is False # Exactly False
assert result is None # Exactly None
# Membership
assert item in collection
assert item not in collection
# Comparisons
assert result > 0
assert 0 <= result <= 100
# Type checking
assert isinstance(result, str)
# Exception testing (preferred approach)
with pytest.raises(ValueError):
raise ValueError("error message")
# Check exception message
with pytest.raises(ValueError, match="invalid input"):
raise ValueError("invalid input provided")
# Check exception attributes
with pytest.raises(ValueError) as exc_info:
raise ValueError("error message")
assert str(exc_info.value) == "error message"
```
## Fixtures
### Basic Fixture Usage
```python
import pytest
@pytest.fixture
def sample_data():
"""Fixture providing sample data."""
return {"name": "Alice", "age": 30}
def test_sample_data(sample_data):
"""Test using the fixture."""
assert sample_data["name"] == "Alice"
assert sample_data["age"] == 30
```
### Fixture with Setup/Teardown
```python
@pytest.fixture
def database():
"""Fixture with setup and teardown."""
# Setup
db = Database(":memory:")
db.create_tables()
db.insert_test_data()
yield db # Provide to test
# Teardown
db.close()
def test_database_query(database):
"""Test database operations."""
result = database.query("SELECT * FROM users")
assert len(result) > 0
```
### Fixture Scopes
```python
# Function scope (default) - runs for each test
@pytest.fixture
def temp_file():
with open("temp.txt", "w") as f:
yield f
os.remove("temp.txt")
# Module scope - runs once per module
@pytest.fixture(scope="module")
def module_db():
db = Database(":memory:")
db.create_tables()
yield db
db.close()
# Session scope - runs once per test session
@pytest.fixture(scope="session")
def shared_resource():
resource = ExpensiveResource()
yield resource
resource.cleanup()
```
### Fixture with Parameters
```python
@pytest.fixture(params=[1, 2, 3])
def number(request):
"""Parameterized fixture."""
return request.param
def test_numbers(number):
"""Test runs 3 times, once for each parameter."""
assert number > 0
```
### Using Multiple Fixtures
```python
@pytest.fixture
def user():
return User(id=1, name="Alice")
@pytest.fixture
def admin():
return User(id=2, name="Admin", role="admin")
def test_user_admin_interaction(user, admin):
"""Test using multiple fixtures."""
assert admin.can_manage(user)
```
### Autouse Fixtures
```python
@pytest.fixture(autouse=True)
def reset_config():
"""Automatically runs before every test."""
Config.reset()
yield
Config.cleanup()
def test_without_fixture_call():
# reset_config runs automatically
assert Config.get_setting("debug") is False
```
### Conftest.py for Shared Fixtures
```python
# tests/conftest.py
import pytest
@pytest.fixture
def client():
"""Shared fixture for all tests."""
app = create_app(testing=True)
with app.test_client() as client:
yield client
@pytest.fixture
def auth_headers(client):
"""Generate auth headers for API testing."""
response = client.post("/api/login", json={
"username": "test",
"password": "test"
})
token = response.json["token"]
return {"Authorization": f"Bearer {token}"}
```
## Parametrization
### Basic Parametrization
```python
@pytest.mark.parametrize("input,expected", [
("hello", "HELLO"),
("world", "WORLD"),
("PyThOn", "PYTHON"),
])
def test_uppercase(input, expected):
"""Test runs 3 times with different inputs."""
assert input.upper() == expected
```
### Multiple Parameters
```python
@pytest.mark.parametrize("a,b,expected", [
(2, 3, 5),
(0, 0, 0),
(-1, 1, 0),
(100, 200, 300),
])
def test_add(a, b, expected):
"""Test addition with multiple inputs."""
assert add(a, b) == expected
```
### Parametrize with IDs
```python
@pytest.mark.parametrize("input,expected", [
("valid@email.com", True),
("invalid", False),
("@no-domain.com", False),
], ids=["valid-email", "missing-at", "missing-domain"])
def test_email_validation(input, expected):
"""Test email validation with readable test IDs."""
assert is_valid_email(input) is expected
```
### Parametrized Fixtures
```python
@pytest.fixture(params=["sqlite", "postgresql", "mysql"])
def db(request):
"""Test against multiple database backends."""
if request.param == "sqlite":
return Database(":memory:")
elif request.param == "postgresql":
return Database("postgresql://localhost/test")
elif request.param == "mysql":
return Database("mysql://localhost/test")
def test_database_operations(db):
"""Test runs 3 times, once for each database."""
result = db.query("SELECT 1")
assert result is not None
```
## Markers and Test Selection
### Custom Markers
```python
# Mark slow tests
@pytest.mark.slow
def test_slow_operation():
time.sleep(5)
# Mark integration tests
@pytest.mark.integration
def test_api_integration():
response = requests.get("https://api.example.com")
assert response.status_code == 200
# Mark unit tests
@pytest.mark.unit
def test_unit_logic():
assert calculate(2, 3) == 5
```
### Run Specific Tests
```bash
# Run only fast tests
pytest -m "not slow"
# Run only integration tests
pytest -m integration
# Run integration or slow tests
pytest -m "integration or slow"
# Run tests marked as unit but not slow
pytest -m "unit and not slow"
```
### Configure Markers in pytest.ini
```ini
[pytest]
markers =
slow: marks tests as slow
integration: marks tests as integration tests
unit: marks tests as unit tests
django: marks tests as requiring Django
```
## Mocking and Patching
### Mocking Functions
```python
from unittest.mock import patch, Mock
@patch("mypackage.external_api_call")
def test_with_mock(api_call_mock):
"""Test with mocked external API."""
api_call_mock.return_value = {"status": "success"}
result = my_function()
api_call_mock.assert_called_once()
assert result["status"] == "success"
```
### Mocking Return Values
```python
@patch("mypackage.Database.connect")
def test_database_connection(connect_mock):
"""Test with mocked database connection."""
connect_mock.return_value = MockConnection()
db = Database()
db.connect()
connect_mock.assert_called_once_with("localhost")
```
### Mocking Exceptions
```python
@patch("mypackage.api_call")
def test_api_error_handling(api_call_mock):
"""Test error handling with mocked exception."""
api_call_mock.side_effect = ConnectionError("Network error")
with pytest.raises(ConnectionError):
api_call()
api_call_mock.assert_called_once()
```
### Mocking Context Managers
```python
@patch("builtins.open", new_callable=mock_open)
def test_file_reading(mock_file):
"""Test file reading with mocked open."""
mock_file.return_value.read.return_value = "file content"
result = read_file("test.txt")
mock_file.assert_called_once_with("test.txt", "r")
assert result == "file content"
```
### Using Autospec
```python
@patch("mypackage.DBConnection", autospec=True)
def test_autospec(db_mock):
"""Test with autospec to catch API misuse."""
db = db_mock.return_value
db.query("SELECT * FROM users")
# This would fail if DBConnection doesn't have query method
db_mock.assert_called_once()
```
### Mock Class Instances
```python
class TestUserService:
@patch("mypackage.UserRepository")
def test_create_user(self, repo_mock):
"""Test user creation with mocked repository."""
repo_mock.return_value.save.return_value = User(id=1, name="Alice")
service = UserService(repo_mock.return_value)
user = service.create_user(name="Alice")
assert user.name == "Alice"
repo_mock.return_value.save.assert_called_once()
```
### Mock Property
```python
@pytest.fixture
def mock_config():
"""Create a mock with a property."""
config = Mock()
type(config).debug = PropertyMock(return_value=True)
type(config).api_key = PropertyMock(return_value="test-key")
return config
def test_with_mock_config(mock_config):
"""Test with mocked config properties."""
assert mock_config.debug is True
assert mock_config.api_key == "test-key"
```
## Testing Async Code
### Async Tests with pytest-asyncio
```python
import pytest
@pytest.mark.asyncio
async def test_async_function():
"""Test async function."""
result = await async_add(2, 3)
assert result == 5
@pytest.mark.asyncio
async def test_async_with_fixture(async_client):
"""Test async with async fixture."""
response = await async_client.get("/api/users")
assert response.status_code == 200
```
### Async Fixture
```python
@pytest.fixture
async def async_client():
"""Async fixture providing async test client."""
app = create_app()
async with app.test_client() as client:
yield client
@pytest.mark.asyncio
async def test_api_endpoint(async_client):
"""Test using async fixture."""
response = await async_client.get("/api/data")
assert response.status_code == 200
```
### Mocking Async Functions
```python
@pytest.mark.asyncio
@patch("mypackage.async_api_call")
async def test_async_mock(api_call_mock):
"""Test async function with mock."""
api_call_mock.return_value = {"status": "ok"}
result = await my_async_function()
api_call_mock.assert_awaited_once()
assert result["status"] == "ok"
```
## Testing Exceptions
### Testing Expected Exceptions
```python
def test_divide_by_zero():
"""Test that dividing by zero raises ZeroDivisionError."""
with pytest.raises(ZeroDivisionError):
divide(10, 0)
def test_custom_exception():
"""Test custom exception with message."""
with pytest.raises(ValueError, match="invalid input"):
validate_input("invalid")
```
### Testing Exception Attributes
```python
def test_exception_with_details():
"""Test exception with custom attributes."""
with pytest.raises(CustomError) as exc_info:
raise CustomError("error", code=400)
assert exc_info.value.code == 400
assert "error" in str(exc_info.value)
```
## Testing Side Effects
### Testing File Operations
```python
import tempfile
import os
def test_file_processing():
"""Test file processing with temp file."""
with tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.txt') as f:
f.write("test content")
temp_path = f.name
try:
result = process_file(temp_path)
assert result == "processed: test content"
finally:
os.unlink(temp_path)
```
### Testing with pytest's tmp_path Fixture
```python
def test_with_tmp_path(tmp_path):
"""Test using pytest's built-in temp path fixture."""
test_file = tmp_path / "test.txt"
test_file.write_text("hello world")
result = process_file(str(test_file))
assert result == "hello world"
# tmp_path automatically cleaned up
```
### Testing with tmpdir Fixture
```python
def test_with_tmpdir(tmpdir):
"""Test using pytest's tmpdir fixture."""
test_file = tmpdir.join("test.txt")
test_file.write("data")
result = process_file(str(test_file))
assert result == "data"
```
## Test Organization
### Directory Structure
```
tests/
├── conftest.py # Shared fixtures
├── __init__.py
├── unit/ # Unit tests
│ ├── __init__.py
│ ├── test_models.py
│ ├── test_utils.py
│ └── test_services.py
├── integration/ # Integration tests
│ ├── __init__.py
│ ├── test_api.py
│ └── test_database.py
└── e2e/ # End-to-end tests
├── __init__.py
└── test_user_flow.py
```
### Test Classes
```python
class TestUserService:
"""Group related tests in a class."""
@pytest.fixture(autouse=True)
def setup(self):
"""Setup runs before each test in this class."""
self.service = UserService()
def test_create_user(self):
"""Test user creation."""
user = self.service.create_user("Alice")
assert user.name == "Alice"
def test_delete_user(self):
"""Test user deletion."""
user = User(id=1, name="Bob")
self.service.delete_user(user)
assert not self.service.user_exists(1)
```
## Best Practices
### DO
- **Follow TDD**: Write tests before code (red-green-refactor)
- **Test one thing**: Each test should verify a single behavior
- **Use descriptive names**: `test_user_login_with_invalid_credentials_fails`
- **Use fixtures**: Eliminate duplication with fixtures
- **Mock external dependencies**: Don't depend on external services
- **Test edge cases**: Empty inputs, None values, boundary conditions
- **Aim for 80%+ coverage**: Focus on critical paths
- **Keep tests fast**: Use marks to separate slow tests
### DON'T
- **Don't test implementation**: Test behavior, not internals
- **Don't use complex conditionals in tests**: Keep tests simple
- **Don't ignore test failures**: All tests must pass
- **Don't test third-party code**: Trust libraries to work
- **Don't share state between tests**: Tests should be independent
- **Don't catch exceptions in tests**: Use `pytest.raises`
- **Don't use print statements**: Use assertions and pytest output
- **Don't write tests that are too brittle**: Avoid over-specific mocks
## Common Patterns
### Testing API Endpoints (FastAPI/Flask)
```python
@pytest.fixture
def client():
app = create_app(testing=True)
return app.test_client()
def test_get_user(client):
response = client.get("/api/users/1")
assert response.status_code == 200
assert response.json["id"] == 1
def test_create_user(client):
response = client.post("/api/users", json={
"name": "Alice",
"email": "alice@example.com"
})
assert response.status_code == 201
assert response.json["name"] == "Alice"
```
### Testing Database Operations
```python
@pytest.fixture
def db_session():
"""Create a test database session."""
session = Session(bind=engine)
session.begin_nested()
yield session
session.rollback()
session.close()
def test_create_user(db_session):
user = User(name="Alice", email="alice@example.com")
db_session.add(user)
db_session.commit()
retrieved = db_session.query(User).filter_by(name="Alice").first()
assert retrieved.email == "alice@example.com"
```
### Testing Class Methods
```python
class TestCalculator:
@pytest.fixture
def calculator(self):
return Calculator()
def test_add(self, calculator):
assert calculator.add(2, 3) == 5
def test_divide_by_zero(self, calculator):
with pytest.raises(ZeroDivisionError):
calculator.divide(10, 0)
```
## pytest Configuration
### pytest.ini
```ini
[pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts =
--strict-markers
--disable-warnings
--cov=mypackage
--cov-report=term-missing
--cov-report=html
markers =
slow: marks tests as slow
integration: marks tests as integration tests
unit: marks tests as unit tests
```
### pyproject.toml
```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
addopts = [
"--strict-markers",
"--cov=mypackage",
"--cov-report=term-missing",
"--cov-report=html",
]
markers = [
"slow: marks tests as slow",
"integration: marks tests as integration tests",
"unit: marks tests as unit tests",
]
```
## Running Tests
```bash
# Run all tests
pytest
# Run specific file
pytest tests/test_utils.py
# Run specific test
pytest tests/test_utils.py::test_function
# Run with verbose output
pytest -v
# Run with coverage
pytest --cov=mypackage --cov-report=html
# Run only fast tests
pytest -m "not slow"
# Run until first failure
pytest -x
# Run and stop on N failures
pytest --maxfail=3
# Run last failed tests
pytest --lf
# Run tests with pattern
pytest -k "test_user"
# Run with debugger on failure
pytest --pdb
```
## Quick Reference
| Pattern | Usage |
|---------|-------|
| `pytest.raises()` | Test expected exceptions |
| `@pytest.fixture()` | Create reusable test fixtures |
| `@pytest.mark.parametrize()` | Run tests with multiple inputs |
| `@pytest.mark.slow` | Mark slow tests |
| `pytest -m "not slow"` | Skip slow tests |
| `@patch()` | Mock functions and classes |
| `tmp_path` fixture | Automatic temp directory |
| `pytest --cov` | Generate coverage report |
| `assert` | Simple and readable assertions |
**Remember**: Tests are code too. Keep them clean, readable, and maintainable. Good tests catch bugs; great tests prevent them.

View File

@@ -1,494 +0,0 @@
---
name: security-review
description: Use this skill when adding authentication, handling user input, working with secrets, creating API endpoints, or implementing payment/sensitive features. Provides comprehensive security checklist and patterns.
---
# Security Review Skill
This skill ensures all code follows security best practices and identifies potential vulnerabilities.
## When to Activate
- Implementing authentication or authorization
- Handling user input or file uploads
- Creating new API endpoints
- Working with secrets or credentials
- Implementing payment features
- Storing or transmitting sensitive data
- Integrating third-party APIs
## Security Checklist
### 1. Secrets Management
#### ❌ NEVER Do This
```typescript
const apiKey = "sk-proj-xxxxx" // Hardcoded secret
const dbPassword = "password123" // In source code
```
#### ✅ ALWAYS Do This
```typescript
const apiKey = process.env.OPENAI_API_KEY
const dbUrl = process.env.DATABASE_URL
// Verify secrets exist
if (!apiKey) {
throw new Error('OPENAI_API_KEY not configured')
}
```
#### Verification Steps
- [ ] No hardcoded API keys, tokens, or passwords
- [ ] All secrets in environment variables
- [ ] `.env.local` in .gitignore
- [ ] No secrets in git history
- [ ] Production secrets in hosting platform (Vercel, Railway)
### 2. Input Validation
#### Always Validate User Input
```typescript
import { z } from 'zod'
// Define validation schema
const CreateUserSchema = z.object({
email: z.string().email(),
name: z.string().min(1).max(100),
age: z.number().int().min(0).max(150)
})
// Validate before processing
export async function createUser(input: unknown) {
try {
const validated = CreateUserSchema.parse(input)
return await db.users.create(validated)
} catch (error) {
if (error instanceof z.ZodError) {
return { success: false, errors: error.errors }
}
throw error
}
}
```
#### File Upload Validation
```typescript
function validateFileUpload(file: File) {
// Size check (5MB max)
const maxSize = 5 * 1024 * 1024
if (file.size > maxSize) {
throw new Error('File too large (max 5MB)')
}
// Type check
const allowedTypes = ['image/jpeg', 'image/png', 'image/gif']
if (!allowedTypes.includes(file.type)) {
throw new Error('Invalid file type')
}
// Extension check
const allowedExtensions = ['.jpg', '.jpeg', '.png', '.gif']
const extension = file.name.toLowerCase().match(/\.[^.]+$/)?.[0]
if (!extension || !allowedExtensions.includes(extension)) {
throw new Error('Invalid file extension')
}
return true
}
```
#### Verification Steps
- [ ] All user inputs validated with schemas
- [ ] File uploads restricted (size, type, extension)
- [ ] No direct use of user input in queries
- [ ] Whitelist validation (not blacklist)
- [ ] Error messages don't leak sensitive info
### 3. SQL Injection Prevention
#### ❌ NEVER Concatenate SQL
```typescript
// DANGEROUS - SQL Injection vulnerability
const query = `SELECT * FROM users WHERE email = '${userEmail}'`
await db.query(query)
```
#### ✅ ALWAYS Use Parameterized Queries
```typescript
// Safe - parameterized query
const { data } = await supabase
.from('users')
.select('*')
.eq('email', userEmail)
// Or with raw SQL
await db.query(
'SELECT * FROM users WHERE email = $1',
[userEmail]
)
```
#### Verification Steps
- [ ] All database queries use parameterized queries
- [ ] No string concatenation in SQL
- [ ] ORM/query builder used correctly
- [ ] Supabase queries properly sanitized
### 4. Authentication & Authorization
#### JWT Token Handling
```typescript
// ❌ WRONG: localStorage (vulnerable to XSS)
localStorage.setItem('token', token)
// ✅ CORRECT: httpOnly cookies
res.setHeader('Set-Cookie',
`token=${token}; HttpOnly; Secure; SameSite=Strict; Max-Age=3600`)
```
#### Authorization Checks
```typescript
export async function deleteUser(userId: string, requesterId: string) {
// ALWAYS verify authorization first
const requester = await db.users.findUnique({
where: { id: requesterId }
})
if (requester.role !== 'admin') {
return NextResponse.json(
{ error: 'Unauthorized' },
{ status: 403 }
)
}
// Proceed with deletion
await db.users.delete({ where: { id: userId } })
}
```
#### Row Level Security (Supabase)
```sql
-- Enable RLS on all tables
ALTER TABLE users ENABLE ROW LEVEL SECURITY;
-- Users can only view their own data
CREATE POLICY "Users view own data"
ON users FOR SELECT
USING (auth.uid() = id);
-- Users can only update their own data
CREATE POLICY "Users update own data"
ON users FOR UPDATE
USING (auth.uid() = id);
```
#### Verification Steps
- [ ] Tokens stored in httpOnly cookies (not localStorage)
- [ ] Authorization checks before sensitive operations
- [ ] Row Level Security enabled in Supabase
- [ ] Role-based access control implemented
- [ ] Session management secure
### 5. XSS Prevention
#### Sanitize HTML
```typescript
import DOMPurify from 'isomorphic-dompurify'
// ALWAYS sanitize user-provided HTML
function renderUserContent(html: string) {
const clean = DOMPurify.sanitize(html, {
ALLOWED_TAGS: ['b', 'i', 'em', 'strong', 'p'],
ALLOWED_ATTR: []
})
return <div dangerouslySetInnerHTML={{ __html: clean }} />
}
```
#### Content Security Policy
```typescript
// next.config.js
const securityHeaders = [
{
key: 'Content-Security-Policy',
value: `
default-src 'self';
script-src 'self' 'unsafe-eval' 'unsafe-inline';
style-src 'self' 'unsafe-inline';
img-src 'self' data: https:;
font-src 'self';
connect-src 'self' https://api.example.com;
`.replace(/\s{2,}/g, ' ').trim()
}
]
```
#### Verification Steps
- [ ] User-provided HTML sanitized
- [ ] CSP headers configured
- [ ] No unvalidated dynamic content rendering
- [ ] React's built-in XSS protection used
### 6. CSRF Protection
#### CSRF Tokens
```typescript
import { csrf } from '@/lib/csrf'
export async function POST(request: Request) {
const token = request.headers.get('X-CSRF-Token')
if (!csrf.verify(token)) {
return NextResponse.json(
{ error: 'Invalid CSRF token' },
{ status: 403 }
)
}
// Process request
}
```
#### SameSite Cookies
```typescript
res.setHeader('Set-Cookie',
`session=${sessionId}; HttpOnly; Secure; SameSite=Strict`)
```
#### Verification Steps
- [ ] CSRF tokens on state-changing operations
- [ ] SameSite=Strict on all cookies
- [ ] Double-submit cookie pattern implemented
### 7. Rate Limiting
#### API Rate Limiting
```typescript
import rateLimit from 'express-rate-limit'
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // 100 requests per window
message: 'Too many requests'
})
// Apply to routes
app.use('/api/', limiter)
```
#### Expensive Operations
```typescript
// Aggressive rate limiting for searches
const searchLimiter = rateLimit({
windowMs: 60 * 1000, // 1 minute
max: 10, // 10 requests per minute
message: 'Too many search requests'
})
app.use('/api/search', searchLimiter)
```
#### Verification Steps
- [ ] Rate limiting on all API endpoints
- [ ] Stricter limits on expensive operations
- [ ] IP-based rate limiting
- [ ] User-based rate limiting (authenticated)
### 8. Sensitive Data Exposure
#### Logging
```typescript
// ❌ WRONG: Logging sensitive data
console.log('User login:', { email, password })
console.log('Payment:', { cardNumber, cvv })
// ✅ CORRECT: Redact sensitive data
console.log('User login:', { email, userId })
console.log('Payment:', { last4: card.last4, userId })
```
#### Error Messages
```typescript
// ❌ WRONG: Exposing internal details
catch (error) {
return NextResponse.json(
{ error: error.message, stack: error.stack },
{ status: 500 }
)
}
// ✅ CORRECT: Generic error messages
catch (error) {
console.error('Internal error:', error)
return NextResponse.json(
{ error: 'An error occurred. Please try again.' },
{ status: 500 }
)
}
```
#### Verification Steps
- [ ] No passwords, tokens, or secrets in logs
- [ ] Error messages generic for users
- [ ] Detailed errors only in server logs
- [ ] No stack traces exposed to users
### 9. Blockchain Security (Solana)
#### Wallet Verification
```typescript
import { verify } from '@solana/web3.js'
async function verifyWalletOwnership(
publicKey: string,
signature: string,
message: string
) {
try {
const isValid = verify(
Buffer.from(message),
Buffer.from(signature, 'base64'),
Buffer.from(publicKey, 'base64')
)
return isValid
} catch (error) {
return false
}
}
```
#### Transaction Verification
```typescript
async function verifyTransaction(transaction: Transaction) {
// Verify recipient
if (transaction.to !== expectedRecipient) {
throw new Error('Invalid recipient')
}
// Verify amount
if (transaction.amount > maxAmount) {
throw new Error('Amount exceeds limit')
}
// Verify user has sufficient balance
const balance = await getBalance(transaction.from)
if (balance < transaction.amount) {
throw new Error('Insufficient balance')
}
return true
}
```
#### Verification Steps
- [ ] Wallet signatures verified
- [ ] Transaction details validated
- [ ] Balance checks before transactions
- [ ] No blind transaction signing
### 10. Dependency Security
#### Regular Updates
```bash
# Check for vulnerabilities
npm audit
# Fix automatically fixable issues
npm audit fix
# Update dependencies
npm update
# Check for outdated packages
npm outdated
```
#### Lock Files
```bash
# ALWAYS commit lock files
git add package-lock.json
# Use in CI/CD for reproducible builds
npm ci # Instead of npm install
```
#### Verification Steps
- [ ] Dependencies up to date
- [ ] No known vulnerabilities (npm audit clean)
- [ ] Lock files committed
- [ ] Dependabot enabled on GitHub
- [ ] Regular security updates
## Security Testing
### Automated Security Tests
```typescript
// Test authentication
test('requires authentication', async () => {
const response = await fetch('/api/protected')
expect(response.status).toBe(401)
})
// Test authorization
test('requires admin role', async () => {
const response = await fetch('/api/admin', {
headers: { Authorization: `Bearer ${userToken}` }
})
expect(response.status).toBe(403)
})
// Test input validation
test('rejects invalid input', async () => {
const response = await fetch('/api/users', {
method: 'POST',
body: JSON.stringify({ email: 'not-an-email' })
})
expect(response.status).toBe(400)
})
// Test rate limiting
test('enforces rate limits', async () => {
const requests = Array(101).fill(null).map(() =>
fetch('/api/endpoint')
)
const responses = await Promise.all(requests)
const tooManyRequests = responses.filter(r => r.status === 429)
expect(tooManyRequests.length).toBeGreaterThan(0)
})
```
## Pre-Deployment Security Checklist
Before ANY production deployment:
- [ ] **Secrets**: No hardcoded secrets, all in env vars
- [ ] **Input Validation**: All user inputs validated
- [ ] **SQL Injection**: All queries parameterized
- [ ] **XSS**: User content sanitized
- [ ] **CSRF**: Protection enabled
- [ ] **Authentication**: Proper token handling
- [ ] **Authorization**: Role checks in place
- [ ] **Rate Limiting**: Enabled on all endpoints
- [ ] **HTTPS**: Enforced in production
- [ ] **Security Headers**: CSP, X-Frame-Options configured
- [ ] **Error Handling**: No sensitive data in errors
- [ ] **Logging**: No sensitive data logged
- [ ] **Dependencies**: Up to date, no vulnerabilities
- [ ] **Row Level Security**: Enabled in Supabase
- [ ] **CORS**: Properly configured
- [ ] **File Uploads**: Validated (size, type)
- [ ] **Wallet Signatures**: Verified (if blockchain)
## Resources
- [OWASP Top 10](https://owasp.org/www-project-top-ten/)
- [Next.js Security](https://nextjs.org/docs/security)
- [Supabase Security](https://supabase.com/docs/guides/auth)
- [Web Security Academy](https://portswigger.net/web-security)
---
**Remember**: Security is not optional. One vulnerability can compromise the entire platform. When in doubt, err on the side of caution.

View File

@@ -1,361 +0,0 @@
| name | description |
|------|-------------|
| cloud-infrastructure-security | Use this skill when deploying to cloud platforms, configuring infrastructure, managing IAM policies, setting up logging/monitoring, or implementing CI/CD pipelines. Provides cloud security checklist aligned with best practices. |
# Cloud & Infrastructure Security Skill
This skill ensures cloud infrastructure, CI/CD pipelines, and deployment configurations follow security best practices and comply with industry standards.
## When to Activate
- Deploying applications to cloud platforms (AWS, Vercel, Railway, Cloudflare)
- Configuring IAM roles and permissions
- Setting up CI/CD pipelines
- Implementing infrastructure as code (Terraform, CloudFormation)
- Configuring logging and monitoring
- Managing secrets in cloud environments
- Setting up CDN and edge security
- Implementing disaster recovery and backup strategies
## Cloud Security Checklist
### 1. IAM & Access Control
#### Principle of Least Privilege
```yaml
# ✅ CORRECT: Minimal permissions
iam_role:
permissions:
- s3:GetObject # Only read access
- s3:ListBucket
resources:
- arn:aws:s3:::my-bucket/* # Specific bucket only
# ❌ WRONG: Overly broad permissions
iam_role:
permissions:
- s3:* # All S3 actions
resources:
- "*" # All resources
```
#### Multi-Factor Authentication (MFA)
```bash
# ALWAYS enable MFA for root/admin accounts
aws iam enable-mfa-device \
--user-name admin \
--serial-number arn:aws:iam::123456789:mfa/admin \
--authentication-code1 123456 \
--authentication-code2 789012
```
#### Verification Steps
- [ ] No root account usage in production
- [ ] MFA enabled for all privileged accounts
- [ ] Service accounts use roles, not long-lived credentials
- [ ] IAM policies follow least privilege
- [ ] Regular access reviews conducted
- [ ] Unused credentials rotated or removed
### 2. Secrets Management
#### Cloud Secrets Managers
```typescript
// ✅ CORRECT: Use cloud secrets manager
import { SecretsManager } from '@aws-sdk/client-secrets-manager';
const client = new SecretsManager({ region: 'us-east-1' });
const secret = await client.getSecretValue({ SecretId: 'prod/api-key' });
const apiKey = JSON.parse(secret.SecretString).key;
// ❌ WRONG: Hardcoded or in environment variables only
const apiKey = process.env.API_KEY; // Not rotated, not audited
```
#### Secrets Rotation
```bash
# Set up automatic rotation for database credentials
aws secretsmanager rotate-secret \
--secret-id prod/db-password \
--rotation-lambda-arn arn:aws:lambda:region:account:function:rotate \
--rotation-rules AutomaticallyAfterDays=30
```
#### Verification Steps
- [ ] All secrets stored in cloud secrets manager (AWS Secrets Manager, Vercel Secrets)
- [ ] Automatic rotation enabled for database credentials
- [ ] API keys rotated at least quarterly
- [ ] No secrets in code, logs, or error messages
- [ ] Audit logging enabled for secret access
### 3. Network Security
#### VPC and Firewall Configuration
```terraform
# ✅ CORRECT: Restricted security group
resource "aws_security_group" "app" {
name = "app-sg"
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["10.0.0.0/16"] # Internal VPC only
}
egress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"] # Only HTTPS outbound
}
}
# ❌ WRONG: Open to the internet
resource "aws_security_group" "bad" {
ingress {
from_port = 0
to_port = 65535
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"] # All ports, all IPs!
}
}
```
#### Verification Steps
- [ ] Database not publicly accessible
- [ ] SSH/RDP ports restricted to VPN/bastion only
- [ ] Security groups follow least privilege
- [ ] Network ACLs configured
- [ ] VPC flow logs enabled
### 4. Logging & Monitoring
#### CloudWatch/Logging Configuration
```typescript
// ✅ CORRECT: Comprehensive logging
import { CloudWatchLogsClient, CreateLogStreamCommand } from '@aws-sdk/client-cloudwatch-logs';
const logSecurityEvent = async (event: SecurityEvent) => {
await cloudwatch.putLogEvents({
logGroupName: '/aws/security/events',
logStreamName: 'authentication',
logEvents: [{
timestamp: Date.now(),
message: JSON.stringify({
type: event.type,
userId: event.userId,
ip: event.ip,
result: event.result,
// Never log sensitive data
})
}]
});
};
```
#### Verification Steps
- [ ] CloudWatch/logging enabled for all services
- [ ] Failed authentication attempts logged
- [ ] Admin actions audited
- [ ] Log retention configured (90+ days for compliance)
- [ ] Alerts configured for suspicious activity
- [ ] Logs centralized and tamper-proof
### 5. CI/CD Pipeline Security
#### Secure Pipeline Configuration
```yaml
# ✅ CORRECT: Secure GitHub Actions workflow
name: Deploy
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
permissions:
contents: read # Minimal permissions
steps:
- uses: actions/checkout@v4
# Scan for secrets
- name: Secret scanning
uses: trufflesecurity/trufflehog@main
# Dependency audit
- name: Audit dependencies
run: npm audit --audit-level=high
# Use OIDC, not long-lived tokens
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789:role/GitHubActionsRole
aws-region: us-east-1
```
#### Supply Chain Security
```json
// package.json - Use lock files and integrity checks
{
"scripts": {
"install": "npm ci", // Use ci for reproducible builds
"audit": "npm audit --audit-level=moderate",
"check": "npm outdated"
}
}
```
#### Verification Steps
- [ ] OIDC used instead of long-lived credentials
- [ ] Secrets scanning in pipeline
- [ ] Dependency vulnerability scanning
- [ ] Container image scanning (if applicable)
- [ ] Branch protection rules enforced
- [ ] Code review required before merge
- [ ] Signed commits enforced
### 6. Cloudflare & CDN Security
#### Cloudflare Security Configuration
```typescript
// ✅ CORRECT: Cloudflare Workers with security headers
export default {
async fetch(request: Request): Promise<Response> {
const response = await fetch(request);
// Add security headers
const headers = new Headers(response.headers);
headers.set('X-Frame-Options', 'DENY');
headers.set('X-Content-Type-Options', 'nosniff');
headers.set('Referrer-Policy', 'strict-origin-when-cross-origin');
headers.set('Permissions-Policy', 'geolocation=(), microphone=()');
return new Response(response.body, {
status: response.status,
headers
});
}
};
```
#### WAF Rules
```bash
# Enable Cloudflare WAF managed rules
# - OWASP Core Ruleset
# - Cloudflare Managed Ruleset
# - Rate limiting rules
# - Bot protection
```
#### Verification Steps
- [ ] WAF enabled with OWASP rules
- [ ] Rate limiting configured
- [ ] Bot protection active
- [ ] DDoS protection enabled
- [ ] Security headers configured
- [ ] SSL/TLS strict mode enabled
### 7. Backup & Disaster Recovery
#### Automated Backups
```terraform
# ✅ CORRECT: Automated RDS backups
resource "aws_db_instance" "main" {
allocated_storage = 20
engine = "postgres"
backup_retention_period = 30 # 30 days retention
backup_window = "03:00-04:00"
maintenance_window = "mon:04:00-mon:05:00"
enabled_cloudwatch_logs_exports = ["postgresql"]
deletion_protection = true # Prevent accidental deletion
}
```
#### Verification Steps
- [ ] Automated daily backups configured
- [ ] Backup retention meets compliance requirements
- [ ] Point-in-time recovery enabled
- [ ] Backup testing performed quarterly
- [ ] Disaster recovery plan documented
- [ ] RPO and RTO defined and tested
## Pre-Deployment Cloud Security Checklist
Before ANY production cloud deployment:
- [ ] **IAM**: Root account not used, MFA enabled, least privilege policies
- [ ] **Secrets**: All secrets in cloud secrets manager with rotation
- [ ] **Network**: Security groups restricted, no public databases
- [ ] **Logging**: CloudWatch/logging enabled with retention
- [ ] **Monitoring**: Alerts configured for anomalies
- [ ] **CI/CD**: OIDC auth, secrets scanning, dependency audits
- [ ] **CDN/WAF**: Cloudflare WAF enabled with OWASP rules
- [ ] **Encryption**: Data encrypted at rest and in transit
- [ ] **Backups**: Automated backups with tested recovery
- [ ] **Compliance**: GDPR/HIPAA requirements met (if applicable)
- [ ] **Documentation**: Infrastructure documented, runbooks created
- [ ] **Incident Response**: Security incident plan in place
## Common Cloud Security Misconfigurations
### S3 Bucket Exposure
```bash
# ❌ WRONG: Public bucket
aws s3api put-bucket-acl --bucket my-bucket --acl public-read
# ✅ CORRECT: Private bucket with specific access
aws s3api put-bucket-acl --bucket my-bucket --acl private
aws s3api put-bucket-policy --bucket my-bucket --policy file://policy.json
```
### RDS Public Access
```terraform
# ❌ WRONG
resource "aws_db_instance" "bad" {
publicly_accessible = true # NEVER do this!
}
# ✅ CORRECT
resource "aws_db_instance" "good" {
publicly_accessible = false
vpc_security_group_ids = [aws_security_group.db.id]
}
```
## Resources
- [AWS Security Best Practices](https://aws.amazon.com/security/best-practices/)
- [CIS AWS Foundations Benchmark](https://www.cisecurity.org/benchmark/amazon_web_services)
- [Cloudflare Security Documentation](https://developers.cloudflare.com/security/)
- [OWASP Cloud Security](https://owasp.org/www-project-cloud-security/)
- [Terraform Security Best Practices](https://www.terraform.io/docs/cloud/guides/recommended-practices/)
**Remember**: Cloud misconfigurations are the leading cause of data breaches. A single exposed S3 bucket or overly permissive IAM policy can compromise your entire infrastructure. Always follow the principle of least privilege and defense in depth.

View File

@@ -1,164 +0,0 @@
---
name: security-scan
description: Scan your Claude Code configuration (.claude/ directory) for security vulnerabilities, misconfigurations, and injection risks using AgentShield. Checks CLAUDE.md, settings.json, MCP servers, hooks, and agent definitions.
---
# Security Scan Skill
Audit your Claude Code configuration for security issues using [AgentShield](https://github.com/affaan-m/agentshield).
## When to Activate
- Setting up a new Claude Code project
- After modifying `.claude/settings.json`, `CLAUDE.md`, or MCP configs
- Before committing configuration changes
- When onboarding to a new repository with existing Claude Code configs
- Periodic security hygiene checks
## What It Scans
| File | Checks |
|------|--------|
| `CLAUDE.md` | Hardcoded secrets, auto-run instructions, prompt injection patterns |
| `settings.json` | Overly permissive allow lists, missing deny lists, dangerous bypass flags |
| `mcp.json` | Risky MCP servers, hardcoded env secrets, npx supply chain risks |
| `hooks/` | Command injection via interpolation, data exfiltration, silent error suppression |
| `agents/*.md` | Unrestricted tool access, prompt injection surface, missing model specs |
## Prerequisites
AgentShield must be installed. Check and install if needed:
```bash
# Check if installed
npx ecc-agentshield --version
# Install globally (recommended)
npm install -g ecc-agentshield
# Or run directly via npx (no install needed)
npx ecc-agentshield scan .
```
## Usage
### Basic Scan
Run against the current project's `.claude/` directory:
```bash
# Scan current project
npx ecc-agentshield scan
# Scan a specific path
npx ecc-agentshield scan --path /path/to/.claude
# Scan with minimum severity filter
npx ecc-agentshield scan --min-severity medium
```
### Output Formats
```bash
# Terminal output (default) — colored report with grade
npx ecc-agentshield scan
# JSON — for CI/CD integration
npx ecc-agentshield scan --format json
# Markdown — for documentation
npx ecc-agentshield scan --format markdown
# HTML — self-contained dark-theme report
npx ecc-agentshield scan --format html > security-report.html
```
### Auto-Fix
Apply safe fixes automatically (only fixes marked as auto-fixable):
```bash
npx ecc-agentshield scan --fix
```
This will:
- Replace hardcoded secrets with environment variable references
- Tighten wildcard permissions to scoped alternatives
- Never modify manual-only suggestions
### Opus 4.6 Deep Analysis
Run the adversarial three-agent pipeline for deeper analysis:
```bash
# Requires ANTHROPIC_API_KEY
export ANTHROPIC_API_KEY=your-key
npx ecc-agentshield scan --opus --stream
```
This runs:
1. **Attacker (Red Team)** — finds attack vectors
2. **Defender (Blue Team)** — recommends hardening
3. **Auditor (Final Verdict)** — synthesizes both perspectives
### Initialize Secure Config
Scaffold a new secure `.claude/` configuration from scratch:
```bash
npx ecc-agentshield init
```
Creates:
- `settings.json` with scoped permissions and deny list
- `CLAUDE.md` with security best practices
- `mcp.json` placeholder
### GitHub Action
Add to your CI pipeline:
```yaml
- uses: affaan-m/agentshield@v1
with:
path: '.'
min-severity: 'medium'
fail-on-findings: true
```
## Severity Levels
| Grade | Score | Meaning |
|-------|-------|---------|
| A | 90-100 | Secure configuration |
| B | 75-89 | Minor issues |
| C | 60-74 | Needs attention |
| D | 40-59 | Significant risks |
| F | 0-39 | Critical vulnerabilities |
## Interpreting Results
### Critical Findings (fix immediately)
- Hardcoded API keys or tokens in config files
- `Bash(*)` in the allow list (unrestricted shell access)
- Command injection in hooks via `${file}` interpolation
- Shell-running MCP servers
### High Findings (fix before production)
- Auto-run instructions in CLAUDE.md (prompt injection vector)
- Missing deny lists in permissions
- Agents with unnecessary Bash access
### Medium Findings (recommended)
- Silent error suppression in hooks (`2>/dev/null`, `|| true`)
- Missing PreToolUse security hooks
- `npx -y` auto-install in MCP server configs
### Info Findings (awareness)
- Missing descriptions on MCP servers
- Prohibitive instructions correctly flagged as good practice
## Links
- **GitHub**: [github.com/affaan-m/agentshield](https://github.com/affaan-m/agentshield)
- **npm**: [npmjs.com/package/ecc-agentshield](https://www.npmjs.com/package/ecc-agentshield)

View File

@@ -1,304 +0,0 @@
---
name: springboot-patterns
description: Spring Boot architecture patterns, REST API design, layered services, data access, caching, async processing, and logging. Use for Java Spring Boot backend work.
---
# Spring Boot Development Patterns
Spring Boot architecture and API patterns for scalable, production-grade services.
## REST API Structure
```java
@RestController
@RequestMapping("/api/markets")
@Validated
class MarketController {
private final MarketService marketService;
MarketController(MarketService marketService) {
this.marketService = marketService;
}
@GetMapping
ResponseEntity<Page<MarketResponse>> list(
@RequestParam(defaultValue = "0") int page,
@RequestParam(defaultValue = "20") int size) {
Page<Market> markets = marketService.list(PageRequest.of(page, size));
return ResponseEntity.ok(markets.map(MarketResponse::from));
}
@PostMapping
ResponseEntity<MarketResponse> create(@Valid @RequestBody CreateMarketRequest request) {
Market market = marketService.create(request);
return ResponseEntity.status(HttpStatus.CREATED).body(MarketResponse.from(market));
}
}
```
## Repository Pattern (Spring Data JPA)
```java
public interface MarketRepository extends JpaRepository<MarketEntity, Long> {
@Query("select m from MarketEntity m where m.status = :status order by m.volume desc")
List<MarketEntity> findActive(@Param("status") MarketStatus status, Pageable pageable);
}
```
## Service Layer with Transactions
```java
@Service
public class MarketService {
private final MarketRepository repo;
public MarketService(MarketRepository repo) {
this.repo = repo;
}
@Transactional
public Market create(CreateMarketRequest request) {
MarketEntity entity = MarketEntity.from(request);
MarketEntity saved = repo.save(entity);
return Market.from(saved);
}
}
```
## DTOs and Validation
```java
public record CreateMarketRequest(
@NotBlank @Size(max = 200) String name,
@NotBlank @Size(max = 2000) String description,
@NotNull @FutureOrPresent Instant endDate,
@NotEmpty List<@NotBlank String> categories) {}
public record MarketResponse(Long id, String name, MarketStatus status) {
static MarketResponse from(Market market) {
return new MarketResponse(market.id(), market.name(), market.status());
}
}
```
## Exception Handling
```java
@ControllerAdvice
class GlobalExceptionHandler {
@ExceptionHandler(MethodArgumentNotValidException.class)
ResponseEntity<ApiError> handleValidation(MethodArgumentNotValidException ex) {
String message = ex.getBindingResult().getFieldErrors().stream()
.map(e -> e.getField() + ": " + e.getDefaultMessage())
.collect(Collectors.joining(", "));
return ResponseEntity.badRequest().body(ApiError.validation(message));
}
@ExceptionHandler(AccessDeniedException.class)
ResponseEntity<ApiError> handleAccessDenied() {
return ResponseEntity.status(HttpStatus.FORBIDDEN).body(ApiError.of("Forbidden"));
}
@ExceptionHandler(Exception.class)
ResponseEntity<ApiError> handleGeneric(Exception ex) {
// Log unexpected errors with stack traces
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(ApiError.of("Internal server error"));
}
}
```
## Caching
Requires `@EnableCaching` on a configuration class.
```java
@Service
public class MarketCacheService {
private final MarketRepository repo;
public MarketCacheService(MarketRepository repo) {
this.repo = repo;
}
@Cacheable(value = "market", key = "#id")
public Market getById(Long id) {
return repo.findById(id)
.map(Market::from)
.orElseThrow(() -> new EntityNotFoundException("Market not found"));
}
@CacheEvict(value = "market", key = "#id")
public void evict(Long id) {}
}
```
## Async Processing
Requires `@EnableAsync` on a configuration class.
```java
@Service
public class NotificationService {
@Async
public CompletableFuture<Void> sendAsync(Notification notification) {
// send email/SMS
return CompletableFuture.completedFuture(null);
}
}
```
## Logging (SLF4J)
```java
@Service
public class ReportService {
private static final Logger log = LoggerFactory.getLogger(ReportService.class);
public Report generate(Long marketId) {
log.info("generate_report marketId={}", marketId);
try {
// logic
} catch (Exception ex) {
log.error("generate_report_failed marketId={}", marketId, ex);
throw ex;
}
return new Report();
}
}
```
## Middleware / Filters
```java
@Component
public class RequestLoggingFilter extends OncePerRequestFilter {
private static final Logger log = LoggerFactory.getLogger(RequestLoggingFilter.class);
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response,
FilterChain filterChain) throws ServletException, IOException {
long start = System.currentTimeMillis();
try {
filterChain.doFilter(request, response);
} finally {
long duration = System.currentTimeMillis() - start;
log.info("req method={} uri={} status={} durationMs={}",
request.getMethod(), request.getRequestURI(), response.getStatus(), duration);
}
}
}
```
## Pagination and Sorting
```java
PageRequest page = PageRequest.of(pageNumber, pageSize, Sort.by("createdAt").descending());
Page<Market> results = marketService.list(page);
```
## Error-Resilient External Calls
```java
public <T> T withRetry(Supplier<T> supplier, int maxRetries) {
int attempts = 0;
while (true) {
try {
return supplier.get();
} catch (Exception ex) {
attempts++;
if (attempts >= maxRetries) {
throw ex;
}
try {
Thread.sleep((long) Math.pow(2, attempts) * 100L);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
throw ex;
}
}
}
}
```
## Rate Limiting (Filter + Bucket4j)
**Security Note**: The `X-Forwarded-For` header is untrusted by default because clients can spoof it.
Only use forwarded headers when:
1. Your app is behind a trusted reverse proxy (nginx, AWS ALB, etc.)
2. You have registered `ForwardedHeaderFilter` as a bean
3. You have configured `server.forward-headers-strategy=NATIVE` or `FRAMEWORK` in application properties
4. Your proxy is configured to overwrite (not append to) the `X-Forwarded-For` header
When `ForwardedHeaderFilter` is properly configured, `request.getRemoteAddr()` will automatically
return the correct client IP from the forwarded headers. Without this configuration, use
`request.getRemoteAddr()` directly—it returns the immediate connection IP, which is the only
trustworthy value.
```java
@Component
public class RateLimitFilter extends OncePerRequestFilter {
private final Map<String, Bucket> buckets = new ConcurrentHashMap<>();
/*
* SECURITY: This filter uses request.getRemoteAddr() to identify clients for rate limiting.
*
* If your application is behind a reverse proxy (nginx, AWS ALB, etc.), you MUST configure
* Spring to handle forwarded headers properly for accurate client IP detection:
*
* 1. Set server.forward-headers-strategy=NATIVE (for cloud platforms) or FRAMEWORK in
* application.properties/yaml
* 2. If using FRAMEWORK strategy, register ForwardedHeaderFilter:
*
* @Bean
* ForwardedHeaderFilter forwardedHeaderFilter() {
* return new ForwardedHeaderFilter();
* }
*
* 3. Ensure your proxy overwrites (not appends) the X-Forwarded-For header to prevent spoofing
* 4. Configure server.tomcat.remoteip.trusted-proxies or equivalent for your container
*
* Without this configuration, request.getRemoteAddr() returns the proxy IP, not the client IP.
* Do NOT read X-Forwarded-For directly—it is trivially spoofable without trusted proxy handling.
*/
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response,
FilterChain filterChain) throws ServletException, IOException {
// Use getRemoteAddr() which returns the correct client IP when ForwardedHeaderFilter
// is configured, or the direct connection IP otherwise. Never trust X-Forwarded-For
// headers directly without proper proxy configuration.
String clientIp = request.getRemoteAddr();
Bucket bucket = buckets.computeIfAbsent(clientIp,
k -> Bucket.builder()
.addLimit(Bandwidth.classic(100, Refill.greedy(100, Duration.ofMinutes(1))))
.build());
if (bucket.tryConsume(1)) {
filterChain.doFilter(request, response);
} else {
response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value());
}
}
}
```
## Background Jobs
Use Springs `@Scheduled` or integrate with queues (e.g., Kafka, SQS, RabbitMQ). Keep handlers idempotent and observable.
## Observability
- Structured logging (JSON) via Logback encoder
- Metrics: Micrometer + Prometheus/OTel
- Tracing: Micrometer Tracing with OpenTelemetry or Brave backend
## Production Defaults
- Prefer constructor injection, avoid field injection
- Enable `spring.mvc.problemdetails.enabled=true` for RFC 7807 errors (Spring Boot 3+)
- Configure HikariCP pool sizes for workload, set timeouts
- Use `@Transactional(readOnly = true)` for queries
- Enforce null-safety via `@NonNull` and `Optional` where appropriate
**Remember**: Keep controllers thin, services focused, repositories simple, and errors handled centrally. Optimize for maintainability and testability.

View File

@@ -1,261 +0,0 @@
---
name: springboot-security
description: Spring Security best practices for authn/authz, validation, CSRF, secrets, headers, rate limiting, and dependency security in Java Spring Boot services.
---
# Spring Boot Security Review
Use when adding auth, handling input, creating endpoints, or dealing with secrets.
## Authentication
- Prefer stateless JWT or opaque tokens with revocation list
- Use `httpOnly`, `Secure`, `SameSite=Strict` cookies for sessions
- Validate tokens with `OncePerRequestFilter` or resource server
```java
@Component
public class JwtAuthFilter extends OncePerRequestFilter {
private final JwtService jwtService;
public JwtAuthFilter(JwtService jwtService) {
this.jwtService = jwtService;
}
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response,
FilterChain chain) throws ServletException, IOException {
String header = request.getHeader(HttpHeaders.AUTHORIZATION);
if (header != null && header.startsWith("Bearer ")) {
String token = header.substring(7);
Authentication auth = jwtService.authenticate(token);
SecurityContextHolder.getContext().setAuthentication(auth);
}
chain.doFilter(request, response);
}
}
```
## Authorization
- Enable method security: `@EnableMethodSecurity`
- Use `@PreAuthorize("hasRole('ADMIN')")` or `@PreAuthorize("@authz.canEdit(#id)")`
- Deny by default; expose only required scopes
```java
@RestController
@RequestMapping("/api/admin")
public class AdminController {
@PreAuthorize("hasRole('ADMIN')")
@GetMapping("/users")
public List<UserDto> listUsers() {
return userService.findAll();
}
@PreAuthorize("@authz.isOwner(#id, authentication)")
@DeleteMapping("/users/{id}")
public ResponseEntity<Void> deleteUser(@PathVariable Long id) {
userService.delete(id);
return ResponseEntity.noContent().build();
}
}
```
## Input Validation
- Use Bean Validation with `@Valid` on controllers
- Apply constraints on DTOs: `@NotBlank`, `@Email`, `@Size`, custom validators
- Sanitize any HTML with a whitelist before rendering
```java
// BAD: No validation
@PostMapping("/users")
public User createUser(@RequestBody UserDto dto) {
return userService.create(dto);
}
// GOOD: Validated DTO
public record CreateUserDto(
@NotBlank @Size(max = 100) String name,
@NotBlank @Email String email,
@NotNull @Min(0) @Max(150) Integer age
) {}
@PostMapping("/users")
public ResponseEntity<UserDto> createUser(@Valid @RequestBody CreateUserDto dto) {
return ResponseEntity.status(HttpStatus.CREATED)
.body(userService.create(dto));
}
```
## SQL Injection Prevention
- Use Spring Data repositories or parameterized queries
- For native queries, use `:param` bindings; never concatenate strings
```java
// BAD: String concatenation in native query
@Query(value = "SELECT * FROM users WHERE name = '" + name + "'", nativeQuery = true)
// GOOD: Parameterized native query
@Query(value = "SELECT * FROM users WHERE name = :name", nativeQuery = true)
List<User> findByName(@Param("name") String name);
// GOOD: Spring Data derived query (auto-parameterized)
List<User> findByEmailAndActiveTrue(String email);
```
## Password Encoding
- Always hash passwords with BCrypt or Argon2 — never store plaintext
- Use `PasswordEncoder` bean, not manual hashing
```java
@Bean
public PasswordEncoder passwordEncoder() {
return new BCryptPasswordEncoder(12); // cost factor 12
}
// In service
public User register(CreateUserDto dto) {
String hashedPassword = passwordEncoder.encode(dto.password());
return userRepository.save(new User(dto.email(), hashedPassword));
}
```
## CSRF Protection
- For browser session apps, keep CSRF enabled; include token in forms/headers
- For pure APIs with Bearer tokens, disable CSRF and rely on stateless auth
```java
http
.csrf(csrf -> csrf.disable())
.sessionManagement(sm -> sm.sessionCreationPolicy(SessionCreationPolicy.STATELESS));
```
## Secrets Management
- No secrets in source; load from env or vault
- Keep `application.yml` free of credentials; use placeholders
- Rotate tokens and DB credentials regularly
```yaml
# BAD: Hardcoded in application.yml
spring:
datasource:
password: mySecretPassword123
# GOOD: Environment variable placeholder
spring:
datasource:
password: ${DB_PASSWORD}
# GOOD: Spring Cloud Vault integration
spring:
cloud:
vault:
uri: https://vault.example.com
token: ${VAULT_TOKEN}
```
## Security Headers
```java
http
.headers(headers -> headers
.contentSecurityPolicy(csp -> csp
.policyDirectives("default-src 'self'"))
.frameOptions(HeadersConfigurer.FrameOptionsConfig::sameOrigin)
.xssProtection(Customizer.withDefaults())
.referrerPolicy(rp -> rp.policy(ReferrerPolicyHeaderWriter.ReferrerPolicy.NO_REFERRER)));
```
## CORS Configuration
- Configure CORS at the security filter level, not per-controller
- Restrict allowed origins — never use `*` in production
```java
@Bean
public CorsConfigurationSource corsConfigurationSource() {
CorsConfiguration config = new CorsConfiguration();
config.setAllowedOrigins(List.of("https://app.example.com"));
config.setAllowedMethods(List.of("GET", "POST", "PUT", "DELETE"));
config.setAllowedHeaders(List.of("Authorization", "Content-Type"));
config.setAllowCredentials(true);
config.setMaxAge(3600L);
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
source.registerCorsConfiguration("/api/**", config);
return source;
}
// In SecurityFilterChain:
http.cors(cors -> cors.configurationSource(corsConfigurationSource()));
```
## Rate Limiting
- Apply Bucket4j or gateway-level limits on expensive endpoints
- Log and alert on bursts; return 429 with retry hints
```java
// Using Bucket4j for per-endpoint rate limiting
@Component
public class RateLimitFilter extends OncePerRequestFilter {
private final Map<String, Bucket> buckets = new ConcurrentHashMap<>();
private Bucket createBucket() {
return Bucket.builder()
.addLimit(Bandwidth.classic(100, Refill.intervally(100, Duration.ofMinutes(1))))
.build();
}
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response,
FilterChain chain) throws ServletException, IOException {
String clientIp = request.getRemoteAddr();
Bucket bucket = buckets.computeIfAbsent(clientIp, k -> createBucket());
if (bucket.tryConsume(1)) {
chain.doFilter(request, response);
} else {
response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value());
response.getWriter().write("{\"error\": \"Rate limit exceeded\"}");
}
}
}
```
## Dependency Security
- Run OWASP Dependency Check / Snyk in CI
- Keep Spring Boot and Spring Security on supported versions
- Fail builds on known CVEs
## Logging and PII
- Never log secrets, tokens, passwords, or full PAN data
- Redact sensitive fields; use structured JSON logging
## File Uploads
- Validate size, content type, and extension
- Store outside web root; scan if required
## Checklist Before Release
- [ ] Auth tokens validated and expired correctly
- [ ] Authorization guards on every sensitive path
- [ ] All inputs validated and sanitized
- [ ] No string-concatenated SQL
- [ ] CSRF posture correct for app type
- [ ] Secrets externalized; none committed
- [ ] Security headers configured
- [ ] Rate limiting on APIs
- [ ] Dependencies scanned and up to date
- [ ] Logs free of sensitive data
**Remember**: Deny by default, validate inputs, least privilege, and secure-by-configuration first.

View File

@@ -1,157 +0,0 @@
---
name: springboot-tdd
description: Test-driven development for Spring Boot using JUnit 5, Mockito, MockMvc, Testcontainers, and JaCoCo. Use when adding features, fixing bugs, or refactoring.
---
# Spring Boot TDD Workflow
TDD guidance for Spring Boot services with 80%+ coverage (unit + integration).
## When to Use
- New features or endpoints
- Bug fixes or refactors
- Adding data access logic or security rules
## Workflow
1) Write tests first (they should fail)
2) Implement minimal code to pass
3) Refactor with tests green
4) Enforce coverage (JaCoCo)
## Unit Tests (JUnit 5 + Mockito)
```java
@ExtendWith(MockitoExtension.class)
class MarketServiceTest {
@Mock MarketRepository repo;
@InjectMocks MarketService service;
@Test
void createsMarket() {
CreateMarketRequest req = new CreateMarketRequest("name", "desc", Instant.now(), List.of("cat"));
when(repo.save(any())).thenAnswer(inv -> inv.getArgument(0));
Market result = service.create(req);
assertThat(result.name()).isEqualTo("name");
verify(repo).save(any());
}
}
```
Patterns:
- Arrange-Act-Assert
- Avoid partial mocks; prefer explicit stubbing
- Use `@ParameterizedTest` for variants
## Web Layer Tests (MockMvc)
```java
@WebMvcTest(MarketController.class)
class MarketControllerTest {
@Autowired MockMvc mockMvc;
@MockBean MarketService marketService;
@Test
void returnsMarkets() throws Exception {
when(marketService.list(any())).thenReturn(Page.empty());
mockMvc.perform(get("/api/markets"))
.andExpect(status().isOk())
.andExpect(jsonPath("$.content").isArray());
}
}
```
## Integration Tests (SpringBootTest)
```java
@SpringBootTest
@AutoConfigureMockMvc
@ActiveProfiles("test")
class MarketIntegrationTest {
@Autowired MockMvc mockMvc;
@Test
void createsMarket() throws Exception {
mockMvc.perform(post("/api/markets")
.contentType(MediaType.APPLICATION_JSON)
.content("""
{"name":"Test","description":"Desc","endDate":"2030-01-01T00:00:00Z","categories":["general"]}
"""))
.andExpect(status().isCreated());
}
}
```
## Persistence Tests (DataJpaTest)
```java
@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Import(TestContainersConfig.class)
class MarketRepositoryTest {
@Autowired MarketRepository repo;
@Test
void savesAndFinds() {
MarketEntity entity = new MarketEntity();
entity.setName("Test");
repo.save(entity);
Optional<MarketEntity> found = repo.findByName("Test");
assertThat(found).isPresent();
}
}
```
## Testcontainers
- Use reusable containers for Postgres/Redis to mirror production
- Wire via `@DynamicPropertySource` to inject JDBC URLs into Spring context
## Coverage (JaCoCo)
Maven snippet:
```xml
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.14</version>
<executions>
<execution>
<goals><goal>prepare-agent</goal></goals>
</execution>
<execution>
<id>report</id>
<phase>verify</phase>
<goals><goal>report</goal></goals>
</execution>
</executions>
</plugin>
```
## Assertions
- Prefer AssertJ (`assertThat`) for readability
- For JSON responses, use `jsonPath`
- For exceptions: `assertThatThrownBy(...)`
## Test Data Builders
```java
class MarketBuilder {
private String name = "Test";
MarketBuilder withName(String name) { this.name = name; return this; }
Market build() { return new Market(null, name, MarketStatus.ACTIVE); }
}
```
## CI Commands
- Maven: `mvn -T 4 test` or `mvn verify`
- Gradle: `./gradlew test jacocoTestReport`
**Remember**: Keep tests fast, isolated, and deterministic. Test behavior, not implementation details.

View File

@@ -1,222 +0,0 @@
---
name: springboot-verification
description: "Verification loop for Spring Boot projects: build, static analysis, tests with coverage, security scans, and diff review before release or PR."
---
# Spring Boot Verification Loop
Run before PRs, after major changes, and pre-deploy.
## Phase 1: Build
```bash
mvn -T 4 clean verify -DskipTests
# or
./gradlew clean assemble -x test
```
If build fails, stop and fix.
## Phase 2: Static Analysis
Maven (common plugins):
```bash
mvn -T 4 spotbugs:check pmd:check checkstyle:check
```
Gradle (if configured):
```bash
./gradlew checkstyleMain pmdMain spotbugsMain
```
## Phase 3: Tests + Coverage
```bash
mvn -T 4 test
mvn jacoco:report # verify 80%+ coverage
# or
./gradlew test jacocoTestReport
```
Report:
- Total tests, passed/failed
- Coverage % (lines/branches)
### Unit Tests
Test service logic in isolation with mocked dependencies:
```java
@ExtendWith(MockitoExtension.class)
class UserServiceTest {
@Mock private UserRepository userRepository;
@InjectMocks private UserService userService;
@Test
void createUser_validInput_returnsUser() {
var dto = new CreateUserDto("Alice", "alice@example.com");
var expected = new User(1L, "Alice", "alice@example.com");
when(userRepository.save(any(User.class))).thenReturn(expected);
var result = userService.create(dto);
assertThat(result.name()).isEqualTo("Alice");
verify(userRepository).save(any(User.class));
}
@Test
void createUser_duplicateEmail_throwsException() {
var dto = new CreateUserDto("Alice", "existing@example.com");
when(userRepository.existsByEmail(dto.email())).thenReturn(true);
assertThatThrownBy(() -> userService.create(dto))
.isInstanceOf(DuplicateEmailException.class);
}
}
```
### Integration Tests with Testcontainers
Test against a real database instead of H2:
```java
@SpringBootTest
@Testcontainers
class UserRepositoryIntegrationTest {
@Container
static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:16-alpine")
.withDatabaseName("testdb");
@DynamicPropertySource
static void configureProperties(DynamicPropertyRegistry registry) {
registry.add("spring.datasource.url", postgres::getJdbcUrl);
registry.add("spring.datasource.username", postgres::getUsername);
registry.add("spring.datasource.password", postgres::getPassword);
}
@Autowired private UserRepository userRepository;
@Test
void findByEmail_existingUser_returnsUser() {
userRepository.save(new User("Alice", "alice@example.com"));
var found = userRepository.findByEmail("alice@example.com");
assertThat(found).isPresent();
assertThat(found.get().getName()).isEqualTo("Alice");
}
}
```
### API Tests with MockMvc
Test controller layer with full Spring context:
```java
@WebMvcTest(UserController.class)
class UserControllerTest {
@Autowired private MockMvc mockMvc;
@MockBean private UserService userService;
@Test
void createUser_validInput_returns201() throws Exception {
var user = new UserDto(1L, "Alice", "alice@example.com");
when(userService.create(any())).thenReturn(user);
mockMvc.perform(post("/api/users")
.contentType(MediaType.APPLICATION_JSON)
.content("""
{"name": "Alice", "email": "alice@example.com"}
"""))
.andExpect(status().isCreated())
.andExpect(jsonPath("$.name").value("Alice"));
}
@Test
void createUser_invalidEmail_returns400() throws Exception {
mockMvc.perform(post("/api/users")
.contentType(MediaType.APPLICATION_JSON)
.content("""
{"name": "Alice", "email": "not-an-email"}
"""))
.andExpect(status().isBadRequest());
}
}
```
## Phase 4: Security Scan
```bash
# Dependency CVEs
mvn org.owasp:dependency-check-maven:check
# or
./gradlew dependencyCheckAnalyze
# Secrets in source
grep -rn "password\s*=\s*\"" src/ --include="*.java" --include="*.yml" --include="*.properties"
grep -rn "sk-\|api_key\|secret" src/ --include="*.java" --include="*.yml"
# Secrets (git history)
git secrets --scan # if configured
```
### Common Security Findings
```
# Check for System.out.println (use logger instead)
grep -rn "System\.out\.print" src/main/ --include="*.java"
# Check for raw exception messages in responses
grep -rn "e\.getMessage()" src/main/ --include="*.java"
# Check for wildcard CORS
grep -rn "allowedOrigins.*\*" src/main/ --include="*.java"
```
## Phase 5: Lint/Format (optional gate)
```bash
mvn spotless:apply # if using Spotless plugin
./gradlew spotlessApply
```
## Phase 6: Diff Review
```bash
git diff --stat
git diff
```
Checklist:
- No debugging logs left (`System.out`, `log.debug` without guards)
- Meaningful errors and HTTP statuses
- Transactions and validation present where needed
- Config changes documented
## Output Template
```
VERIFICATION REPORT
===================
Build: [PASS/FAIL]
Static: [PASS/FAIL] (spotbugs/pmd/checkstyle)
Tests: [PASS/FAIL] (X/Y passed, Z% coverage)
Security: [PASS/FAIL] (CVE findings: N)
Diff: [X files changed]
Overall: [READY / NOT READY]
Issues to Fix:
1. ...
2. ...
```
## Continuous Mode
- Re-run phases on significant changes or every 3060 minutes in long sessions
- Keep a short loop: `mvn -T 4 test` + spotbugs for quick feedback
**Remember**: Fast feedback beats late surprises. Keep the gate strict—treat warnings as defects in production systems.

View File

@@ -1,63 +0,0 @@
---
name: strategic-compact
description: Suggests manual context compaction at logical intervals to preserve context through task phases rather than arbitrary auto-compaction.
---
# Strategic Compact Skill
Suggests manual `/compact` at strategic points in your workflow rather than relying on arbitrary auto-compaction.
## Why Strategic Compaction?
Auto-compaction triggers at arbitrary points:
- Often mid-task, losing important context
- No awareness of logical task boundaries
- Can interrupt complex multi-step operations
Strategic compaction at logical boundaries:
- **After exploration, before execution** - Compact research context, keep implementation plan
- **After completing a milestone** - Fresh start for next phase
- **Before major context shifts** - Clear exploration context before different task
## How It Works
The `suggest-compact.sh` script runs on PreToolUse (Edit/Write) and:
1. **Tracks tool calls** - Counts tool invocations in session
2. **Threshold detection** - Suggests at configurable threshold (default: 50 calls)
3. **Periodic reminders** - Reminds every 25 calls after threshold
## Hook Setup
Add to your `~/.claude/settings.json`:
```json
{
"hooks": {
"PreToolUse": [{
"matcher": "tool == \"Edit\" || tool == \"Write\"",
"hooks": [{
"type": "command",
"command": "~/.claude/skills/strategic-compact/suggest-compact.sh"
}]
}]
}
}
```
## Configuration
Environment variables:
- `COMPACT_THRESHOLD` - Tool calls before first suggestion (default: 50)
## Best Practices
1. **Compact after planning** - Once plan is finalized, compact to start fresh
2. **Compact after debugging** - Clear error-resolution context before continuing
3. **Don't compact mid-implementation** - Preserve context for related changes
4. **Read the suggestion** - The hook tells you *when*, you decide *if*
## Related
- [The Longform Guide](https://x.com/affaanmustafa/status/2014040193557471352) - Token optimization section
- Memory persistence hooks - For state that survives compaction

View File

@@ -1,80 +0,0 @@
#!/usr/bin/env node
/**
* Strategic Compact Suggester
*
* Cross-platform (Windows, macOS, Linux)
*
* Runs on PreToolUse or periodically to suggest manual compaction at logical intervals
*
* Why manual over auto-compact:
* - Auto-compact happens at arbitrary points, often mid-task
* - Strategic compacting preserves context through logical phases
* - Compact after exploration, before execution
* - Compact after completing a milestone, before starting next
*/
const fs = require('fs');
const path = require('path');
const {
getTempDir,
writeFile,
log
} = require('../lib/utils');
async function main() {
// Track tool call count (increment in a temp file)
// Use a session-specific counter file based on session ID from environment
// or parent PID as fallback
const sessionId = process.env.CLAUDE_SESSION_ID || 'default';
const counterFile = path.join(getTempDir(), `claude-tool-count-${sessionId}`);
const rawThreshold = parseInt(process.env.COMPACT_THRESHOLD || '50', 10);
const threshold = Number.isFinite(rawThreshold) && rawThreshold > 0 && rawThreshold <= 10000
? rawThreshold
: 50;
let count = 1;
// Read existing count or start at 1
// Use fd-based read+write to reduce (but not eliminate) race window
// between concurrent hook invocations
try {
const fd = fs.openSync(counterFile, 'a+');
try {
const buf = Buffer.alloc(64);
const bytesRead = fs.readSync(fd, buf, 0, 64, 0);
if (bytesRead > 0) {
const parsed = parseInt(buf.toString('utf8', 0, bytesRead).trim(), 10);
// Clamp to reasonable range — corrupted files could contain huge values
// that pass Number.isFinite() (e.g., parseInt('9'.repeat(30)) => 1e+29)
count = (Number.isFinite(parsed) && parsed > 0 && parsed <= 1000000)
? parsed + 1
: 1;
}
// Truncate and write new value
fs.ftruncateSync(fd, 0);
fs.writeSync(fd, String(count), 0);
} finally {
fs.closeSync(fd);
}
} catch {
// Fallback: just use writeFile if fd operations fail
writeFile(counterFile, String(count));
}
// Suggest compact after threshold tool calls
if (count === threshold) {
log(`[StrategicCompact] ${threshold} tool calls reached - consider /compact if transitioning phases`);
}
// Suggest at regular intervals after threshold (every 25 calls from threshold)
if (count > threshold && (count - threshold) % 25 === 0) {
log(`[StrategicCompact] ${count} tool calls - good checkpoint for /compact if context is stale`);
}
process.exit(0);
}
main().catch(err => {
console.error('[StrategicCompact] Error:', err.message);
process.exit(0);
});

View File

@@ -1,54 +0,0 @@
#!/bin/bash
# Strategic Compact Suggester
# Runs on PreToolUse or periodically to suggest manual compaction at logical intervals
#
# Why manual over auto-compact:
# - Auto-compact happens at arbitrary points, often mid-task
# - Strategic compacting preserves context through logical phases
# - Compact after exploration, before execution
# - Compact after completing a milestone, before starting next
#
# Hook config (in ~/.claude/settings.json):
# {
# "hooks": {
# "PreToolUse": [{
# "matcher": "Edit|Write",
# "hooks": [{
# "type": "command",
# "command": "~/.claude/skills/strategic-compact/suggest-compact.sh"
# }]
# }]
# }
# }
#
# Criteria for suggesting compact:
# - Session has been running for extended period
# - Large number of tool calls made
# - Transitioning from research/exploration to implementation
# - Plan has been finalized
# Track tool call count (increment in a temp file)
# Use CLAUDE_SESSION_ID for session-specific counter (not $$ which changes per invocation)
SESSION_ID="${CLAUDE_SESSION_ID:-${PPID:-default}}"
COUNTER_FILE="/tmp/claude-tool-count-${SESSION_ID}"
THRESHOLD=${COMPACT_THRESHOLD:-50}
# Initialize or increment counter
if [ -f "$COUNTER_FILE" ]; then
count=$(cat "$COUNTER_FILE")
count=$((count + 1))
echo "$count" > "$COUNTER_FILE"
else
echo "1" > "$COUNTER_FILE"
count=1
fi
# Suggest compact after threshold tool calls
if [ "$count" -eq "$THRESHOLD" ]; then
echo "[StrategicCompact] $THRESHOLD tool calls reached - consider /compact if transitioning phases" >&2
fi
# Suggest at regular intervals after threshold
if [ "$count" -gt "$THRESHOLD" ] && [ $((count % 25)) -eq 0 ]; then
echo "[StrategicCompact] $count tool calls - good checkpoint for /compact if context is stale" >&2
fi

View File

@@ -1,409 +0,0 @@
---
name: tdd-workflow
description: Use this skill when writing new features, fixing bugs, or refactoring code. Enforces test-driven development with 80%+ coverage including unit, integration, and E2E tests.
---
# Test-Driven Development Workflow
This skill ensures all code development follows TDD principles with comprehensive test coverage.
## When to Activate
- Writing new features or functionality
- Fixing bugs or issues
- Refactoring existing code
- Adding API endpoints
- Creating new components
## Core Principles
### 1. Tests BEFORE Code
ALWAYS write tests first, then implement code to make tests pass.
### 2. Coverage Requirements
- Minimum 80% coverage (unit + integration + E2E)
- All edge cases covered
- Error scenarios tested
- Boundary conditions verified
### 3. Test Types
#### Unit Tests
- Individual functions and utilities
- Component logic
- Pure functions
- Helpers and utilities
#### Integration Tests
- API endpoints
- Database operations
- Service interactions
- External API calls
#### E2E Tests (Playwright)
- Critical user flows
- Complete workflows
- Browser automation
- UI interactions
## TDD Workflow Steps
### Step 1: Write User Journeys
```
As a [role], I want to [action], so that [benefit]
Example:
As a user, I want to search for markets semantically,
so that I can find relevant markets even without exact keywords.
```
### Step 2: Generate Test Cases
For each user journey, create comprehensive test cases:
```typescript
describe('Semantic Search', () => {
it('returns relevant markets for query', async () => {
// Test implementation
})
it('handles empty query gracefully', async () => {
// Test edge case
})
it('falls back to substring search when Redis unavailable', async () => {
// Test fallback behavior
})
it('sorts results by similarity score', async () => {
// Test sorting logic
})
})
```
### Step 3: Run Tests (They Should Fail)
```bash
npm test
# Tests should fail - we haven't implemented yet
```
### Step 4: Implement Code
Write minimal code to make tests pass:
```typescript
// Implementation guided by tests
export async function searchMarkets(query: string) {
// Implementation here
}
```
### Step 5: Run Tests Again
```bash
npm test
# Tests should now pass
```
### Step 6: Refactor
Improve code quality while keeping tests green:
- Remove duplication
- Improve naming
- Optimize performance
- Enhance readability
### Step 7: Verify Coverage
```bash
npm run test:coverage
# Verify 80%+ coverage achieved
```
## Testing Patterns
### Unit Test Pattern (Jest/Vitest)
```typescript
import { render, screen, fireEvent } from '@testing-library/react'
import { Button } from './Button'
describe('Button Component', () => {
it('renders with correct text', () => {
render(<Button>Click me</Button>)
expect(screen.getByText('Click me')).toBeInTheDocument()
})
it('calls onClick when clicked', () => {
const handleClick = jest.fn()
render(<Button onClick={handleClick}>Click</Button>)
fireEvent.click(screen.getByRole('button'))
expect(handleClick).toHaveBeenCalledTimes(1)
})
it('is disabled when disabled prop is true', () => {
render(<Button disabled>Click</Button>)
expect(screen.getByRole('button')).toBeDisabled()
})
})
```
### API Integration Test Pattern
```typescript
import { NextRequest } from 'next/server'
import { GET } from './route'
describe('GET /api/markets', () => {
it('returns markets successfully', async () => {
const request = new NextRequest('http://localhost/api/markets')
const response = await GET(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(data.success).toBe(true)
expect(Array.isArray(data.data)).toBe(true)
})
it('validates query parameters', async () => {
const request = new NextRequest('http://localhost/api/markets?limit=invalid')
const response = await GET(request)
expect(response.status).toBe(400)
})
it('handles database errors gracefully', async () => {
// Mock database failure
const request = new NextRequest('http://localhost/api/markets')
// Test error handling
})
})
```
### E2E Test Pattern (Playwright)
```typescript
import { test, expect } from '@playwright/test'
test('user can search and filter markets', async ({ page }) => {
// Navigate to markets page
await page.goto('/')
await page.click('a[href="/markets"]')
// Verify page loaded
await expect(page.locator('h1')).toContainText('Markets')
// Search for markets
await page.fill('input[placeholder="Search markets"]', 'election')
// Wait for debounce and results
await page.waitForTimeout(600)
// Verify search results displayed
const results = page.locator('[data-testid="market-card"]')
await expect(results).toHaveCount(5, { timeout: 5000 })
// Verify results contain search term
const firstResult = results.first()
await expect(firstResult).toContainText('election', { ignoreCase: true })
// Filter by status
await page.click('button:has-text("Active")')
// Verify filtered results
await expect(results).toHaveCount(3)
})
test('user can create a new market', async ({ page }) => {
// Login first
await page.goto('/creator-dashboard')
// Fill market creation form
await page.fill('input[name="name"]', 'Test Market')
await page.fill('textarea[name="description"]', 'Test description')
await page.fill('input[name="endDate"]', '2025-12-31')
// Submit form
await page.click('button[type="submit"]')
// Verify success message
await expect(page.locator('text=Market created successfully')).toBeVisible()
// Verify redirect to market page
await expect(page).toHaveURL(/\/markets\/test-market/)
})
```
## Test File Organization
```
src/
├── components/
│ ├── Button/
│ │ ├── Button.tsx
│ │ ├── Button.test.tsx # Unit tests
│ │ └── Button.stories.tsx # Storybook
│ └── MarketCard/
│ ├── MarketCard.tsx
│ └── MarketCard.test.tsx
├── app/
│ └── api/
│ └── markets/
│ ├── route.ts
│ └── route.test.ts # Integration tests
└── e2e/
├── markets.spec.ts # E2E tests
├── trading.spec.ts
└── auth.spec.ts
```
## Mocking External Services
### Supabase Mock
```typescript
jest.mock('@/lib/supabase', () => ({
supabase: {
from: jest.fn(() => ({
select: jest.fn(() => ({
eq: jest.fn(() => Promise.resolve({
data: [{ id: 1, name: 'Test Market' }],
error: null
}))
}))
}))
}
}))
```
### Redis Mock
```typescript
jest.mock('@/lib/redis', () => ({
searchMarketsByVector: jest.fn(() => Promise.resolve([
{ slug: 'test-market', similarity_score: 0.95 }
])),
checkRedisHealth: jest.fn(() => Promise.resolve({ connected: true }))
}))
```
### OpenAI Mock
```typescript
jest.mock('@/lib/openai', () => ({
generateEmbedding: jest.fn(() => Promise.resolve(
new Array(1536).fill(0.1) // Mock 1536-dim embedding
))
}))
```
## Test Coverage Verification
### Run Coverage Report
```bash
npm run test:coverage
```
### Coverage Thresholds
```json
{
"jest": {
"coverageThresholds": {
"global": {
"branches": 80,
"functions": 80,
"lines": 80,
"statements": 80
}
}
}
}
```
## Common Testing Mistakes to Avoid
### ❌ WRONG: Testing Implementation Details
```typescript
// Don't test internal state
expect(component.state.count).toBe(5)
```
### ✅ CORRECT: Test User-Visible Behavior
```typescript
// Test what users see
expect(screen.getByText('Count: 5')).toBeInTheDocument()
```
### ❌ WRONG: Brittle Selectors
```typescript
// Breaks easily
await page.click('.css-class-xyz')
```
### ✅ CORRECT: Semantic Selectors
```typescript
// Resilient to changes
await page.click('button:has-text("Submit")')
await page.click('[data-testid="submit-button"]')
```
### ❌ WRONG: No Test Isolation
```typescript
// Tests depend on each other
test('creates user', () => { /* ... */ })
test('updates same user', () => { /* depends on previous test */ })
```
### ✅ CORRECT: Independent Tests
```typescript
// Each test sets up its own data
test('creates user', () => {
const user = createTestUser()
// Test logic
})
test('updates user', () => {
const user = createTestUser()
// Update logic
})
```
## Continuous Testing
### Watch Mode During Development
```bash
npm test -- --watch
# Tests run automatically on file changes
```
### Pre-Commit Hook
```bash
# Runs before every commit
npm test && npm run lint
```
### CI/CD Integration
```yaml
# GitHub Actions
- name: Run Tests
run: npm test -- --coverage
- name: Upload Coverage
uses: codecov/codecov-action@v3
```
## Best Practices
1. **Write Tests First** - Always TDD
2. **One Assert Per Test** - Focus on single behavior
3. **Descriptive Test Names** - Explain what's tested
4. **Arrange-Act-Assert** - Clear test structure
5. **Mock External Dependencies** - Isolate unit tests
6. **Test Edge Cases** - Null, undefined, empty, large
7. **Test Error Paths** - Not just happy paths
8. **Keep Tests Fast** - Unit tests < 50ms each
9. **Clean Up After Tests** - No side effects
10. **Review Coverage Reports** - Identify gaps
## Success Metrics
- 80%+ code coverage achieved
- All tests passing (green)
- No skipped or disabled tests
- Fast test execution (< 30s for unit tests)
- E2E tests cover critical user flows
- Tests catch bugs before production
---
**Remember**: Tests are not optional. They are the safety net that enables confident refactoring, rapid development, and production reliability.

View File

@@ -1,125 +0,0 @@
---
name: verification-loop
description: "A comprehensive verification system for Claude Code sessions."
---
# Verification Loop Skill
A comprehensive verification system for Claude Code sessions.
## When to Use
Invoke this skill:
- After completing a feature or significant code change
- Before creating a PR
- When you want to ensure quality gates pass
- After refactoring
## Verification Phases
### Phase 1: Build Verification
```bash
# Check if project builds
npm run build 2>&1 | tail -20
# OR
pnpm build 2>&1 | tail -20
```
If build fails, STOP and fix before continuing.
### Phase 2: Type Check
```bash
# TypeScript projects
npx tsc --noEmit 2>&1 | head -30
# Python projects
pyright . 2>&1 | head -30
```
Report all type errors. Fix critical ones before continuing.
### Phase 3: Lint Check
```bash
# JavaScript/TypeScript
npm run lint 2>&1 | head -30
# Python
ruff check . 2>&1 | head -30
```
### Phase 4: Test Suite
```bash
# Run tests with coverage
npm run test -- --coverage 2>&1 | tail -50
# Check coverage threshold
# Target: 80% minimum
```
Report:
- Total tests: X
- Passed: X
- Failed: X
- Coverage: X%
### Phase 5: Security Scan
```bash
# Check for secrets
grep -rn "sk-" --include="*.ts" --include="*.js" . 2>/dev/null | head -10
grep -rn "api_key" --include="*.ts" --include="*.js" . 2>/dev/null | head -10
# Check for console.log
grep -rn "console.log" --include="*.ts" --include="*.tsx" src/ 2>/dev/null | head -10
```
### Phase 6: Diff Review
```bash
# Show what changed
git diff --stat
git diff HEAD~1 --name-only
```
Review each changed file for:
- Unintended changes
- Missing error handling
- Potential edge cases
## Output Format
After running all phases, produce a verification report:
```
VERIFICATION REPORT
==================
Build: [PASS/FAIL]
Types: [PASS/FAIL] (X errors)
Lint: [PASS/FAIL] (X warnings)
Tests: [PASS/FAIL] (X/Y passed, Z% coverage)
Security: [PASS/FAIL] (X issues)
Diff: [X files changed]
Overall: [READY/NOT READY] for PR
Issues to Fix:
1. ...
2. ...
```
## Continuous Mode
For long sessions, run verification every 15 minutes or after major changes:
```markdown
Set a mental checkpoint:
- After completing each function
- After finishing a component
- Before moving to next task
Run: /verify
```
## Integration with Hooks
This skill complements PostToolUse hooks but provides deeper verification.
Hooks catch issues immediately; this skill provides comprehensive review.