evals/evals.json
[
{
"id": 1,
"name": "context-background-in-handler",
"description": "Tests whether the model propagates r.Context() instead of creating context.Background() inside an HTTP handler",
"prompt": "Write a Go HTTP handler for GET /orders/:id that fetches an order from a database and calls an external payment service to get payment status. Both operations should be cancellable if the client disconnects. Use database/sql and net/http.",
"trap": "Model might create context.Background() inside the handler instead of using r.Context(), breaking cancellation chain",
"assertions": [
{"id": "1.1", "text": "Uses r.Context() to obtain the request context, NOT context.Background() inside the handler"},
{"id": "1.2", "text": "Passes the same context (or a derived child) to the database query (QueryRowContext or similar *Context variant)"},
{"id": "1.3", "text": "Passes the same context to the external HTTP call via http.NewRequestWithContext"},
{"id": "1.4", "text": "Does NOT use http.NewRequest (without context) for the external service call"},
{"id": "1.5", "text": "Checks ctx.Err() or handles context cancellation when the client disconnects"}
]
},
{
"id": 2,
"name": "cancel-leak-timeout",
"description": "Tests whether the model defers cancel() immediately after WithTimeout to prevent resource leaks",
"prompt": "Write a Go function that retries an HTTP request up to 3 times with a 5-second timeout per attempt. Each attempt should have its own independent timeout. Return the response body as []byte or the last error.",
"trap": "Model might forget to defer cancel() after WithTimeout, or create a single timeout for all retries instead of per-attempt",
"assertions": [
{"id": "2.1", "text": "Creates a new context.WithTimeout for each retry attempt (not one timeout for all retries)"},
{"id": "2.2", "text": "Calls defer cancel() (or cancel() before next iteration) for every WithTimeout call"},
{"id": "2.3", "text": "Does NOT discard the cancel function with _ (e.g., ctx, _ = context.WithTimeout(...))"},
{"id": "2.4", "text": "Uses http.NewRequestWithContext to attach the per-attempt timeout context"},
{"id": "2.5", "text": "Accepts a parent context parameter and derives timeouts from it"}
]
},
{
"id": 3,
"name": "context-value-key-type",
"description": "Tests whether the model uses unexported key types for context values instead of string keys",
"prompt": "Write Go middleware that extracts a tenant ID from the X-Tenant-ID header and makes it available to downstream handlers. Also write a helper function to retrieve the tenant ID from the context. Other packages in the codebase will import and use this helper.",
"trap": "Model might use a plain string key like context.WithValue(ctx, \"tenant_id\", ...) which causes namespace collisions across packages",
"assertions": [
{"id": "3.1", "text": "Uses an unexported type for the context key (e.g., type contextKey string or type tenantKey struct{})"},
{"id": "3.2", "text": "Does NOT use a plain string as the context key (e.g., context.WithValue(ctx, \"tenant_id\", ...))"},
{"id": "3.3", "text": "Provides a typed getter function (e.g., TenantIDFromContext) that returns the value with proper type assertion"},
{"id": "3.4", "text": "Provides a setter function or the middleware injects the value using the unexported key"},
{"id": "3.5", "text": "The getter handles the case where the value is missing from the context (returns zero value + bool or error)"}
]
},
{
"id": 4,
"name": "context-in-struct-trap",
"description": "Tests whether the model avoids storing context.Context in a struct field",
"prompt": "Design a Go Worker struct that processes jobs from a channel. The worker should support graceful shutdown — when told to stop, it finishes the current job and exits. Write NewWorker, Start, and Stop methods.",
"trap": "Model might store ctx context.Context as a struct field for shutdown signaling instead of passing it through function parameters",
"assertions": [
{"id": "4.1", "text": "Does NOT store context.Context as a field in the Worker struct"},
{"id": "4.2", "text": "Passes context as a parameter to Start() or Run() method (e.g., Start(ctx context.Context))"},
{"id": "4.3", "text": "Uses context cancellation or a done channel for graceful shutdown signaling"},
{"id": "4.4", "text": "Listens to ctx.Done() in a select statement to detect shutdown"},
{"id": "4.5", "text": "ctx is the first parameter where it appears, named ctx context.Context"}
]
},
{
"id": 5,
"name": "without-cancel-background-work",
"description": "Tests whether the model uses context.WithoutCancel for background work that must outlive the request",
"prompt": "Write a Go HTTP handler that processes a payment. After successfully charging the customer, it must send an audit log event to an external audit service asynchronously. The audit log MUST complete even if the client disconnects immediately after receiving the 200 response. The audit log needs the trace_id from the request context for correlation. Target Go 1.21+.",
"trap": "Model might use context.Background() (losing trace_id) or pass r.Context() directly to the goroutine (cancelled when handler returns)",
"assertions": [
{"id": "5.1", "text": "Uses context.WithoutCancel to create a context for the audit goroutine"},
{"id": "5.2", "text": "Does NOT pass r.Context() directly to the background audit goroutine (it gets cancelled when the handler returns)"},
{"id": "5.3", "text": "Does NOT use context.Background() for the audit goroutine (that would lose trace_id and other values)"},
{"id": "5.4", "text": "The audit goroutine preserves request-scoped values (trace_id) from the original context"},
{"id": "5.5", "text": "Launches the audit as a separate goroutine (go keyword) so the handler can return immediately"}
]
},
{
"id": 6,
"name": "nested-timeout-shorter-wins",
"description": "Tests understanding that nested timeouts always use the shorter deadline",
"prompt": "Write a Go service method that calls two downstream services sequentially. The overall operation has a 10-second timeout. The first call (to a fast cache) should have a 2-second timeout. The second call (to a slow database) should have an 8-second timeout. If the cache is slow, the database should still get its full 8 seconds. Implement this timeout hierarchy.",
"trap": "Model might naively nest WithTimeout(parentCtx, 8s) under a parent that already has 10s-minus-elapsed, not realizing that if the first call takes 2 seconds the parent only has 8s left — or worse, create child with longer timeout than parent remaining",
"assertions": [
{"id": "6.1", "text": "Creates the overall 10-second timeout from the parent context"},
{"id": "6.2", "text": "Creates the cache timeout as a child of the overall context (so the 2s cache timeout is bounded by the 10s overall)"},
{"id": "6.3", "text": "Acknowledges or handles that the database timeout is bounded by whatever time remains on the parent (not a fresh 8 seconds independent of the parent)"},
{"id": "6.4", "text": "Defers cancel() for every WithTimeout call"},
{"id": "6.5", "text": "Does NOT create independent context.Background() timeouts that bypass the overall deadline"}
]
},
{
"id": 7,
"name": "context-todo-vs-background",
"description": "Tests correct usage of context.TODO() vs context.Background()",
"prompt": "I'm refactoring a Go codebase to add context support. Many functions don't accept context yet. Write a migration plan showing how to incrementally add context.Context to this call chain: main() -> runServer() -> handleRequest() -> processOrder() -> saveToDatabase(). Some functions already accept context, others don't yet. Show intermediate steps with code.",
"trap": "Model might use context.Background() everywhere as a placeholder. Skill teaches context.TODO() is the correct placeholder when a function needs a context but doesn't have one yet",
"assertions": [
{"id": "7.1", "text": "Uses context.TODO() (not context.Background()) as the temporary placeholder in functions not yet fully migrated"},
{"id": "7.2", "text": "Uses context.Background() only at the true top level (main function or test setup)"},
{"id": "7.3", "text": "Shows a migration path where context.TODO() is gradually replaced as callers are updated"},
{"id": "7.4", "text": "Context is always the first parameter, named ctx context.Context"},
{"id": "7.5", "text": "Does NOT pass nil as a context value at any point in the migration"}
]
},
{
"id": 8,
"name": "db-context-variants",
"description": "db.BeginTx (not db.Begin) is required for cancellable transactions; context must flow to every db call including transaction start",
"prompt": "Review this Go repository method and identify all context-related issues:\n\n```go\nfunc (r *UserRepository) TransferCredits(ctx context.Context, fromID, toID string, amount int) error {\n tx, err := r.db.Begin()\n if err != nil {\n return fmt.Errorf(\"begin tx: %w\", err)\n }\n defer tx.Rollback()\n\n _, err = tx.ExecContext(ctx, \"UPDATE users SET credits = credits - $1 WHERE id = $2\", amount, fromID)\n if err != nil {\n return fmt.Errorf(\"debit: %w\", err)\n }\n\n _, err = tx.ExecContext(ctx, \"UPDATE users SET credits = credits + $1 WHERE id = $2\", amount, toID)\n if err != nil {\n return fmt.Errorf(\"credit: %w\", err)\n }\n\n return tx.Commit()\n}\n```\n\nWhat is wrong with how context is used? Fix it.",
"trap": "The model sees ExecContext being used and may think context usage is correct. The bug: db.Begin() starts the transaction without a context — if ctx is already cancelled when Begin() is called, the transaction still starts. Also, tx.Commit() should be tx.CommitContext(ctx) if available, but the main issue is db.BeginTx(ctx, nil) instead of db.Begin().",
"assertions": [
{"id": "8.1", "text": "Identifies db.Begin() as the bug — it starts a transaction ignoring the context; if ctx is cancelled before Begin() returns, the transaction still starts"},
{"id": "8.2", "text": "Replaces db.Begin() with db.BeginTx(ctx, nil) — BeginTx respects context cancellation and won't start a transaction if ctx is already done"},
{"id": "8.3", "text": "Keeps ExecContext calls passing ctx — these are already correct"},
{"id": "8.4", "text": "Does not add a redundant ctx check before Begin() as the fix — the correct fix is BeginTx, not a manual ctx.Err() guard"},
{"id": "8.5", "text": "Each method accepts ctx context.Context as its first parameter and it flows through all DB operations"}
]
},
{
"id": 9,
"name": "context-values-abuse",
"description": "Tests that the model does not abuse context values for function parameters",
"prompt": "Write a Go service that processes orders. The service needs: a database connection, a logger, a user ID (from auth), a trace ID (from middleware), and the order details. Design the ProcessOrder function signature and show how to pass all these dependencies.",
"trap": "Model might stuff the database connection, logger, or order details into context values instead of passing them as explicit parameters or struct fields",
"assertions": [
{"id": "9.1", "text": "Database connection is passed as a struct field or explicit parameter, NOT via context value"},
{"id": "9.2", "text": "Logger is passed as a struct field or explicit parameter, NOT via context value"},
{"id": "9.3", "text": "Order details are passed as an explicit function parameter, NOT via context value"},
{"id": "9.4", "text": "User ID and/or trace ID are stored in context values (these are request-scoped metadata)"},
{"id": "9.5", "text": "Distinguishes between infrastructure dependencies (explicit) and request-scoped metadata (context values)"}
]
},
{
"id": 10,
"name": "afterfunc-cleanup",
"description": "Tests awareness of context.AfterFunc for cleanup callbacks (Go 1.21+)",
"prompt": "Write a Go function that opens a temporary file for processing and must clean it up when the request context is cancelled. The file should be deleted asynchronously when the context is done, without blocking the main processing flow. The main function should continue processing immediately after registering the cleanup. Target Go 1.21+.",
"trap": "Model might use a goroutine with <-ctx.Done() manually or use defer (which doesn't handle context cancellation). Skill teaches context.AfterFunc for non-blocking cleanup callbacks",
"assertions": [
{"id": "10.1", "text": "Uses context.AfterFunc to register the cleanup callback"},
{"id": "10.2", "text": "Does NOT block the main flow waiting for context cancellation (no <-ctx.Done() in the main goroutine for cleanup purposes)"},
{"id": "10.3", "text": "The cleanup function removes the temporary file"},
{"id": "10.4", "text": "Captures the stop function returned by AfterFunc for potential cancellation of the callback"},
{"id": "10.5", "text": "Also includes a defer-based cleanup as a safety net (AfterFunc + defer for belt-and-suspenders)"}
]
}
]
references/cancellation.md
# Cancellation, Timeouts & Deadlines
## Cancellation
`context.WithCancel` returns a derived context and a `cancel` function. When `cancel()` is called, the context's `Done()` channel is closed, signaling all listeners to stop.
```go
func processItems(ctx context.Context, items []Item) error {
ctx, cancel := context.WithCancel(ctx)
defer cancel() // always defer cancel to free resources
errCh := make(chan error, len(items))
for _, item := range items {
go func(item Item) {
errCh <- processOne(ctx, item)
}(item)
}
for range items {
if err := <-errCh; err != nil {
cancel() // cancel remaining goroutines on first error
return fmt.Errorf("processing items: %w", err)
}
}
return nil
}
```
### Why `defer cancel()` matters
Every `WithCancel`, `WithTimeout`, and `WithDeadline` creates cancellation state; timeout/deadline contexts also use timer resources. `cancel()` MUST be called on all control-flow paths unless the function explicitly returns or transfers ownership of both the context and cancel function. In ordinary scoped work, defer cancel immediately.
```go
// ✗ Bad — cancel is never called, resources leak
func fetch(ctx context.Context) error {
ctx, _ = context.WithTimeout(ctx, 5*time.Second)
return doWork(ctx)
}
// ✓ Good — scoped work, defer cancel immediately
func fetch(ctx context.Context) error {
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
return doWork(ctx)
}
```
## Timeouts and Deadlines
### `context.WithTimeout` — relative duration
```go
func (s *UserService) GetUser(ctx context.Context, id string) (*User, error) {
ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
defer cancel()
return s.repo.FindByID(ctx, id)
}
```
### `context.WithDeadline` — absolute point in time
```go
func (s *BatchService) ProcessBatch(ctx context.Context, batch Batch) error {
// The batch must complete by its SLA deadline
ctx, cancel := context.WithDeadline(ctx, batch.SLADeadline)
defer cancel()
for _, item := range batch.Items {
if err := s.process(ctx, item); err != nil {
return fmt.Errorf("processing batch item %s: %w", item.ID, err)
}
}
return nil
}
```
### Nested timeouts take the shorter deadline
If a parent context has a 5s timeout and you create a child with 10s, the child still expires at 5s. The shorter deadline always wins.
```go
// Parent has 2s timeout — child's 10s is effectively ignored
parentCtx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()
childCtx, childCancel := context.WithTimeout(parentCtx, 10*time.Second)
defer childCancel()
// childCtx expires after 2s, not 10s
```
## Listening for Cancellation
### The `select` pattern
Use `ctx.Done()` in a `select` statement to react to cancellation alongside other work:
```go
func poll(ctx context.Context, interval time.Duration) error {
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return ctx.Err() // context.Canceled or context.DeadlineExceeded
case <-ticker.C:
if err := doWork(ctx); err != nil {
return fmt.Errorf("polling: %w", err)
}
}
}
}
```
### Checking cancellation in loops
For CPU-bound work, periodically check `ctx.Err()`:
```go
func processLargeDataset(ctx context.Context, items []Item) error {
for i, item := range items {
if ctx.Err() != nil {
return fmt.Errorf("processing interrupted after %d/%d items: %w", i, len(items), ctx.Err())
}
process(item)
}
return nil
}
```
## `context.AfterFunc` (Go 1.21+)
Registers a callback that runs in its own goroutine when the context is cancelled. Useful for cleanup without blocking the main flow.
```go
func watchResource(ctx context.Context, res *Resource) {
stop := context.AfterFunc(ctx, func() {
// Runs in a new goroutine when ctx is cancelled
res.Release()
})
// If you no longer need the callback, cancel it:
// stop() returns true if the callback was successfully cancelled
_ = stop
}
```
## `context.WithoutCancel` (Go 1.21+)
Creates a child context that is not cancelled when the parent is. Use this for background work that must continue after the request completes — like async logging, audit trails, or enqueuing follow-up tasks.
```go
func (h *Handler) CreateOrder(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
order, err := h.orderService.Create(ctx, req)
if err != nil {
// handle error
return
}
// Audit log must complete even if the client disconnects.
// WithoutCancel preserves context values (trace_id) but detaches cancellation.
auditCtx := context.WithoutCancel(ctx)
go h.auditService.LogOrderCreated(auditCtx, order)
w.WriteHeader(http.StatusCreated)
}
```
Without `WithoutCancel`, you'd have to choose between `ctx` (which gets cancelled when the handler returns, killing your background work) and `context.Background()` (which loses trace_id and other values). `WithoutCancel` gives you the best of both: values are preserved, but cancellation is detached.
references/http-services.md
# Context in HTTP Servers & Service Calls
## Context in HTTP Servers
`http.Request` carries a context that is cancelled when the client disconnects or the request handler returns. MUST use `r.Context()` — NEVER create a new `context.Background()` inside a handler.
```go
func (h *Handler) GetOrder(w http.ResponseWriter, r *http.Request) {
ctx := r.Context() // this context is cancelled if the client disconnects
order, err := h.orderService.Get(ctx, r.PathValue("id"))
if err != nil {
if ctx.Err() != nil {
// Client disconnected, no point writing a response
return
}
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
json.NewEncoder(w).Encode(order)
}
```
## Middleware enriching context
Middleware injects request-scoped values before handlers run. Use unexported key types to prevent collisions:
```go
// Helpers for trace propagation
type contextKey string
const (
traceIDKey contextKey = "trace_id"
spanIDKey contextKey = "span_id"
)
func TracingMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
traceID := r.Header.Get("X-Trace-ID")
if traceID == "" {
traceID = generateTraceID()
}
spanID := r.Header.Get("X-Span-ID")
if spanID == "" {
spanID = generateSpanID()
}
ctx := context.WithValue(r.Context(), traceIDKey, traceID)
ctx = context.WithValue(ctx, spanIDKey, spanID)
w.Header().Set("X-Trace-ID", traceID)
w.Header().Set("X-Span-ID", spanID)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
// Propagate trace context to downstream services
func (c *HTTPClient) Do(ctx context.Context, method, url string, body io.Reader) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, method, url, body)
if err != nil {
return nil, fmt.Errorf("creating request: %w", err)
}
if traceID, ok := ctx.Value(traceIDKey).(string); ok {
req.Header.Set("X-Trace-ID", traceID)
}
if spanID, ok := ctx.Value(spanIDKey).(string); ok {
req.Header.Set("X-Span-ID", spanID)
}
return c.client.Do(req)
}
```
## Context in Calls to Other Services
Context MUST be propagated to all HTTP clients and databases using context-aware APIs: `http.NewRequestWithContext`, `QueryContext`, `ExecContext`, and `QueryRowContext`. This ensures that client disconnections cancel all downstream operations.
```go
// ✗ Bad — downstream calls ignore the request context
func (c *PaymentClient) Charge(ctx context.Context, amount int) error {
req, _ := http.NewRequest("POST", c.url+"/charge", body)
return c.client.Do(req) // not context-aware
}
// ✓ Good — all downstream operations respect the context
func (c *PaymentClient) Charge(ctx context.Context, amount int) error {
req, err := http.NewRequestWithContext(ctx, "POST", c.url+"/charge", body)
if err != nil {
return fmt.Errorf("creating request: %w", err)
}
return c.client.Do(req)
}
```
```go
// ✗ Bad — downstream calls ignore the request context
func (r *UserRepo) FindByID(ctx context.Context, id string) (*User, error) {
row := r.db.QueryRow("SELECT * FROM users WHERE id = $1", id)
// ...
}
// ✓ Good — all downstream operations respect the context
func (r *UserRepo) FindByID(ctx context.Context, id string) (*User, error) {
row := r.db.QueryRowContext(ctx, "SELECT * FROM users WHERE id = $1", id)
// ...
}
```
references/values-tracing.md
# Context Values & Cross-Service Tracing
## Using context values correctly
Context values carry request-scoped metadata that crosses API boundaries — not function parameters, configuration, or optional arguments. Good candidates: trace IDs, span IDs, request IDs, authenticated user info, correlation IDs.
Always use an unexported type as the key to prevent collisions between packages:
```go
// ✓ Good — unexported key type prevents collisions
type contextKey string
const (
traceIDKey contextKey = "trace_id"
requestIDKey contextKey = "request_id"
)
func WithTraceID(ctx context.Context, traceID string) context.Context {
return context.WithValue(ctx, traceIDKey, traceID)
}
func TraceIDFromContext(ctx context.Context) (string, bool) {
traceID, ok := ctx.Value(traceIDKey).(string)
return traceID, ok
}
```
```go
// ✗ Bad — string keys collide across packages
ctx = context.WithValue(ctx, "trace_id", traceID) // another package could use the same key
```
## What belongs in context values vs function parameters
| Data | Context value? | Why |
| --- | --- | --- |
| trace_id, span_id, request_id | Yes | Request-scoped metadata for observability |
| Authenticated user/tenant | Yes | Request-scoped, crosses API boundaries |
| Database connection | No | Infrastructure dependency, pass explicitly |
| Feature flags | No | Configuration, pass explicitly or inject |
| Function arguments (user ID, order data) | No | Business logic parameters, pass as arguments |
| Logger | Depends | OK if enriched with request-scoped fields (trace_id); otherwise pass explicitly |
## Trace propagation between services
In a microservices architecture, `context.Context` is the vehicle for trace propagation. When Service A calls Service B, the trace_id and span_id travel through context values and are injected into outgoing HTTP headers (typically via OpenTelemetry). This creates a connected trace across the entire request path.
```go
// Middleware injects trace_id from incoming request headers into context
func TracingMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
traceID := r.Header.Get("X-Trace-ID")
if traceID == "" {
traceID = generateTraceID()
}
ctx := WithTraceID(r.Context(), traceID)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
// When making outbound HTTP calls, inject trace_id from context into headers
func (c *HTTPClient) Do(ctx context.Context, method, url string, body io.Reader) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, method, url, body)
if err != nil {
return nil, fmt.Errorf("creating request: %w", err)
}
// Propagate trace_id to downstream service
if traceID, ok := TraceIDFromContext(ctx); ok {
req.Header.Set("X-Trace-ID", traceID)
}
return c.client.Do(req)
}
```
With OpenTelemetry, this propagation is handled automatically through the `otel` SDK and `propagation.TraceContext`, but the mechanism is the same: context carries the trace state, and it must be propagated through every layer.