assets/operations/scalability-checklist.md
# System Scalability Checklist
Use this checklist when designing for horizontal scalability and high availability.
## Load Estimation
- [ ] **Current load:** [N] requests/second, [N] concurrent users
- [ ] **Peak load:** [N] requests/second (expected during [event/time])
- [ ] **Growth projection:** [X]% yearly growth
- [ ] **Target capacity:** Support [N]x current load
## Scalability Dimensions
### Horizontal Scaling (Preferred)
- [ ] **Stateless services:** All application servers are stateless
- [ ] **Session storage:** Use Redis/Memcached for distributed sessions
- [ ] **File storage:** Use object storage (S3/GCS) instead of local filesystem
- [ ] **Auto-scaling:** Configure based on CPU/memory/RPS metrics
- [ ] **Load balancer:** Layer 7 (application-aware) load balancing
### Vertical Scaling (Limited)
- [ ] **Database instance:** Right-sized for current + 6 months growth
- [ ] **Cache instance:** Sized for working set + 20% headroom
- [ ] **Max capacity:** Identified vertical scaling limit
## Database Scalability
### Read Scaling
- [ ] **Read replicas:** [N] replicas for read-heavy workloads
- [ ] **Read/write splitting:** Route reads to replicas, writes to primary
- [ ] **Connection pooling:** PgBouncer/ProxySQL to limit connections
- [ ] **Query optimization:** Queries < [10ms] with proper indexing
### Write Scaling
- [ ] **Sharding strategy:** [Hash-based / Range-based / Geographic]
- Shard key: [user_id / tenant_id / region]
- Number of shards: [N] (plan for [10x] growth)
- [ ] **Write-ahead logging:** Asynchronous replication for replicas
- [ ] **Bulk operations:** Batch inserts/updates to reduce round trips
### Caching Strategy
- [ ] **Cache layers:**
- L1: In-memory application cache ([Caffeine / Guava])
- L2: Distributed cache ([Redis / Memcached])
- L3: CDN for static assets ([CloudFront / Fastly])
- [ ] **Cache hit ratio:** Target > [90]%
- [ ] **TTL strategy:** Balance freshness vs load ([5min] for hot data, [1h] for warm)
- [ ] **Cache eviction:** LRU/LFU policy configured
- [ ] **Cache warming:** Pre-populate on deployment
- [ ] **Cache invalidation:** Event-driven invalidation for updates
## API Gateway
- [ ] **Rate limiting:**
- Per-user: [N] req/min
- Per-IP: [N] req/min
- Burst allowance: [N] requests
- [ ] **Request throttling:** Queue requests during spikes
- [ ] **Response compression:** Gzip/Brotli enabled
- [ ] **API versioning:** Support [N] concurrent versions
## Asynchronous Processing
- [ ] **Message queue:** [Kafka / RabbitMQ / AWS SQS]
- Throughput: [N] messages/second
- Retention: [X] days
- [ ] **Worker pools:** [N] workers per queue
- [ ] **Backpressure:** Reject requests when queue length > [N]
- [ ] **Dead letter queue:** For failed message handling
## Content Delivery
- [ ] **CDN:** CloudFront/Fastly for static assets
- [ ] **Edge caching:** Cache-Control headers configured
- [ ] **Image optimization:** WebP format, lazy loading
- [ ] **Asset bundling:** Minified and bundled CSS/JS
## Data Storage Patterns
### Hot/Warm/Cold Data
- [ ] **Hot data:** Last [7] days in primary DB (fast access)
- [ ] **Warm data:** Last [30] days in read replicas
- [ ] **Cold data:** Older than [30] days archived to S3/Glacier
- [ ] **Archival strategy:** Automated data lifecycle policies
### Data Partitioning
- [ ] **Time-based partitioning:** Partition by month/year for time-series data
- [ ] **Hash partitioning:** Distribute by hash(user_id) for even distribution
- [ ] **List partitioning:** Partition by region/tenant for isolation
## Connection Management
- [ ] **Database connection pool:**
- Min connections: [N]
- Max connections: [N]
- Connection timeout: [Xms]
- [ ] **HTTP keep-alive:** Reuse connections to upstream services
- [ ] **Circuit breaker:** Prevent cascade failures
## Observability for Scalability
### Key Metrics
- [ ] **Golden signals:**
- Latency: p50, p95, p99 response times
- Traffic: Requests per second
- Errors: Error rate (4xx, 5xx)
- Saturation: CPU, memory, disk, network usage
- [ ] **Capacity metrics:**
- Database connections used/available
- Queue depth and processing rate
- Cache hit/miss ratio
- Thread pool utilization
### Alerts
- [ ] CPU > [70]% for [5] minutes → Scale out
- [ ] Memory > [80]% for [5] minutes → Investigate memory leak
- [ ] Database connections > [80]% → Add replicas
- [ ] Queue depth > [1000] → Add workers
- [ ] Error rate > [1]% → Page on-call
## Load Testing
- [ ] **Baseline test:** Measure current performance under typical load
- [ ] **Stress test:** Identify breaking point (max capacity before failure)
- [ ] **Spike test:** Test behavior under sudden 10x traffic spike
- [ ] **Soak test:** Run at [2x] typical load for [24] hours
- [ ] **Tools:** [k6 / JMeter / Gatling / Locust]
### Load Test Scenarios
- [ ] Scenario 1: [N] users browsing product catalog
- [ ] Scenario 2: [N] users checking out simultaneously
- [ ] Scenario 3: [N] API clients polling for updates
- [ ] Target performance: p95 < [200ms], error rate < [0.1]%
## Cost Optimization
- [ ] **Right-sizing:** Use smallest instance that meets SLA
- [ ] **Reserved instances:** [70]% reserved, [30]% on-demand/spot
- [ ] **Auto-scaling policies:**
- Scale up: When CPU > [70]% for [2] minutes
- Scale down: When CPU < [30]% for [10] minutes
- Min instances: [N], Max instances: [N]
- [ ] **Database optimization:** Remove unused indexes, optimize slow queries
- [ ] **CDN optimization:** Increase cache TTL where possible
## Geographic Distribution
- [ ] **Multi-region deployment:**
- Primary region: [us-east-1]
- Secondary region: [eu-west-1]
- Failover: Automatic DNS failover
- [ ] **Data locality:** Store user data in nearest region (GDPR compliance)
- [ ] **Latency optimization:** < [100ms] within region, < [300ms] cross-region
## Disaster Recovery
- [ ] **RTO (Recovery Time Objective):** [X] hours
- [ ] **RPO (Recovery Point Objective):** [X] minutes
- [ ] **Backup frequency:** Database backup every [X] hours
- [ ] **Failover testing:** Quarterly DR drills
- [ ] **Data replication:** Asynchronous cross-region replication
## Security at Scale
- [ ] **DDoS protection:** CloudFlare/AWS Shield enabled
- [ ] **Rate limiting:** Per-IP and per-user limits
- [ ] **WAF rules:** Block common attack patterns
- [ ] **Certificate management:** Auto-renewal with Let's Encrypt/ACM
## Deployment Strategy
- [ ] **Blue-green deployment:** Zero-downtime releases
- [ ] **Canary releases:** [10]% traffic to new version initially
- [ ] **Feature flags:** Toggle features without deployment
- [ ] **Rollback plan:** Automated rollback if error rate > [X]%
## Checklist Before Production
- [ ] Load tested at [3x] expected peak traffic
- [ ] Auto-scaling policies validated
- [ ] Database read replicas configured
- [ ] Caching strategy implemented and tested
- [ ] Monitoring and alerts configured
- [ ] Disaster recovery plan documented and tested
- [ ] Cost monitoring and budgets set
- [ ] Security review completed
- [ ] Runbooks created for common incidents
- [ ] On-call rotation established
assets/patterns/event-driven-template.md
# Event-Driven Architecture Template
Use this template when designing event-driven systems with asynchronous communication.
## Event Schema Definition
### Event: [EventName]
```json
{
"eventId": "uuid",
"eventType": "[EventName]",
"eventVersion": "1.0",
"timestamp": "ISO8601",
"source": "[ServiceName]",
"correlationId": "uuid",
"payload": {
// Event-specific data
},
"metadata": {
"userId": "string",
"traceId": "string"
}
}
```
**Trigger:** [When is this event published?]
**Consumers:** [Which services consume this event?]
**Retention:** [How long to keep in event store?]
## Event Catalog
| Event Name | Version | Producer | Consumers | Schema Registry |
|------------|---------|----------|-----------|-----------------|
| OrderPlaced | 1.0 | OrderService | PaymentService, InventoryService | schema-registry/order-placed-v1.json |
| PaymentProcessed | 1.0 | PaymentService | OrderService, NotificationService | schema-registry/payment-processed-v1.json |
## Event Broker Configuration
- **Platform:** [Kafka / RabbitMQ / AWS EventBridge / Google Pub/Sub]
- **Topics/Queues:**
- `orders.placed` - Order creation events
- `payments.processed` - Payment completion events
- `inventory.updated` - Stock level changes
- **Partitioning strategy:** [By customer ID / By order ID / Round-robin]
- **Replication factor:** [3 for production]
- **Retention period:** [7 days for event replay]
## Producer Configuration
### Service: [ProducerServiceName]
**Events Published:**
- **[EventName]**: Published when [business event occurs]
- Partition key: [customer_id / entity_id]
- Ordering guarantee: [Yes/No]
- Retry policy: [3 retries with exponential backoff]
**Reliability Patterns:**
- **Outbox pattern:** Write to database + outbox table atomically
- **At-least-once delivery:** Idempotent event production
- **Schema validation:** Validate against schema registry before publishing
- **Failure handling:** Store failed events in dead letter queue
**Code Example:**
```python
async def publish_event(event: OrderPlaced):
# Transactional outbox pattern
async with db.transaction():
await db.orders.create(event.order)
await db.outbox.insert({
"event_id": event.id,
"event_type": "OrderPlaced",
"payload": event.to_json(),
"status": "pending"
})
# Asynchronous event publishing (handled by outbox processor)
await event_broker.publish(
topic="orders.placed",
key=event.customer_id,
value=event.to_json(),
headers={"trace_id": trace_id}
)
```
## Consumer Configuration
### Service: [ConsumerServiceName]
**Events Consumed:**
- **[EventName]**: From [ProducerService]
- Consumer group: `[service-name]-[event-name]`
- Concurrency: [N parallel workers]
- Max retries: [5 with exponential backoff]
- Dead letter queue: `[event-name].dlq`
**Processing Patterns:**
- **Idempotency:** Check event ID before processing
- **Ordering:** Process events in order within partition
- **Error handling:** Retry transient errors, DLQ for permanent failures
- **Acknowledgment:** Acknowledge only after successful processing
**Code Example:**
```python
@consumer(topic="orders.placed", group="payment-service-orders")
async def process_order_placed(event: OrderPlaced):
# Idempotency check
if await db.processed_events.exists(event.id):
logger.info(f"Event {event.id} already processed")
return
try:
# Business logic
payment = await payment_service.process_payment(event.order)
# Mark as processed
await db.processed_events.insert(event.id)
# Publish downstream event
await publish_event(PaymentProcessed(payment))
except RetryableError as e:
logger.warning(f"Retrying event {event.id}: {e}")
raise # Will be retried
except PermanentError as e:
logger.error(f"Sending event {event.id} to DLQ: {e}")
await dlq.send(event, error=str(e))
```
## Event Sourcing
**Use when:** Need complete audit trail and event replay capabilities.
### Aggregate: [AggregateName]
**Command Handlers:**
- **CreateOrder**: Validates and produces OrderCreated event
- **UpdateOrderStatus**: Produces OrderStatusUpdated event
**Event Store:**
- **Storage:** [PostgreSQL event_store table / EventStoreDB / Kafka topic]
- **Schema:**
```sql
CREATE TABLE events (
event_id UUID PRIMARY KEY,
aggregate_id UUID NOT NULL,
aggregate_type VARCHAR(100),
event_type VARCHAR(100),
event_data JSONB,
event_version INT,
created_at TIMESTAMP,
INDEX(aggregate_id, event_version)
);
```
**Snapshot Strategy:**
- Create snapshot every [100] events
- Store in separate `snapshots` table
- Load latest snapshot + subsequent events for aggregate rebuild
## Saga Pattern (Distributed Transactions)
### Saga: [SagaName]
**Orchestration Approach:** [Choreography / Orchestrator]
**Steps:**
1. **OrderService** → Publish `OrderPlaced`
2. **PaymentService** → Process payment → Publish `PaymentProcessed` or `PaymentFailed`
3. **InventoryService** → Reserve stock → Publish `InventoryReserved` or `InventoryUnavailable`
4. **ShippingService** → Schedule shipment → Publish `ShipmentScheduled`
**Compensation (Rollback):**
- If `PaymentFailed` → Publish `OrderCancelled`
- If `InventoryUnavailable` → Publish `PaymentRefunded` → `OrderCancelled`
**State Machine:**
```
[Order Created] → [Payment Processing] → [Inventory Reserved] → [Shipment Scheduled] → [Order Completed]
↓ ↓ ↓
[Order Cancelled] ← [Payment Failed] ← [Inventory Unavailable]
```
## Schema Evolution
**Versioning Strategy:** [Backward compatible / Forward compatible / Full compatibility]
**Schema Registry:** [Confluent Schema Registry / AWS Glue Schema Registry]
**Version Migration:**
- **v1 → v2 Changes:**
- Added optional field: `customer_email`
- Deprecated field: `customer_address` (use `shipping_address` instead)
- **Consumer compatibility:** Old consumers can read v2 events (ignore new fields)
- **Producer compatibility:** New producers can emit v1 events if needed
## Monitoring & Observability
**Metrics:**
- Event publish rate and latency
- Consumer lag (events behind current offset)
- Processing errors and retry count
- Dead letter queue size
**Alerts:**
- Consumer lag > [1000 events]
- Error rate > [5%]
- DLQ messages > [100]
- Event processing latency p99 > [500ms]
**Distributed Tracing:**
- Propagate trace IDs across events
- Track event flow: Producer → Broker → Consumer → Downstream events
## Testing
**Unit Tests:**
- Event schema validation
- Idempotency logic
- Compensation logic (saga rollback)
**Integration Tests:**
- End-to-end event flow
- Consumer error handling and retries
- Dead letter queue behavior
**Chaos Testing:**
- Simulate broker downtime
- Duplicate event delivery
- Out-of-order event delivery
- Consumer crash mid-processing
## Security
- **Encryption:** TLS for data in transit, encryption at rest for event store
- **Authorization:** ACLs for topic access (producers/consumers)
- **Audit:** Log all event publications and consumptions
- **PII handling:** Encrypt sensitive fields in event payload
## Cost Optimization
- **Retention policy:** Delete events older than [X days]
- **Compaction:** Use log compaction for entity snapshots
- **Resource allocation:** Right-size broker and consumer resources
- **Batch processing:** Batch consume events to reduce overhead
## Disaster Recovery
- **Event replay:** Re-process events from timestamp/offset
- **Backup:** Regular snapshots of event store
- **Cross-region replication:** Mirror events to DR region
- **RTO/RPO:** Recovery time objective [X hours], Recovery point objective [X minutes]
assets/patterns/microservices-template.md
# Microservices Architecture Template
Use this template when designing a microservices-based system.
## Service Definition
- **Service name:** [ServiceName]
- **Bounded context:** [Domain boundary this service owns]
- **Team owner:** [Team responsible for this service]
## Service Responsibilities
- **Core capabilities:**
- [Primary business capability 1]
- [Primary business capability 2]
- **Data ownership:**
- [Entities this service owns]
- [Events this service publishes]
## API Contract
### REST Endpoints
```http
GET /api/v1/[resource]
POST /api/v1/[resource]
PUT /api/v1/[resource]/{id}
DELETE /api/v1/[resource]/{id}
```
### Events Published
- **[EventName]**: Published when [trigger condition]
- **[EventName]**: Published when [trigger condition]
### Events Consumed
- **[EventName]**: From [SourceService], triggers [action]
## Dependencies
### Upstream Services (Calls)
- **[ServiceName]**: For [purpose], timeout: [Xms], circuit breaker threshold: [N failures]
### Downstream Services (Called by)
- **[ServiceName]**: Expects [SLA], provides [data/functionality]
## Data Storage
- **Database type:** [PostgreSQL / MongoDB / Cassandra / etc.]
- **Schema approach:** [Database-per-service / Shared tables / etc.]
- **Replication:** [Primary-replica / Multi-master]
- **Backup strategy:** [Automated daily / Point-in-time recovery]
## Resilience
- **Timeouts:**
- Database queries: [Xms]
- External API calls: [Xms]
- Internal service calls: [Xms]
- **Retry policy:** Exponential backoff, max [N] retries
- **Circuit breaker:** Open after [N] failures, half-open after [X] seconds
- **Rate limiting:** [N] requests per second per client
- **Fallback behavior:** [Return cached data / Default response / Graceful degradation]
## Observability
- **Metrics:**
- Request rate, latency (p50, p95, p99)
- Error rate (4xx, 5xx)
- Dependency health
- Business metrics: [specific to service]
- **Distributed tracing:** Jaeger/OpenTelemetry with trace IDs
- **Logging:**
- Structured JSON logs
- Log level: INFO in production, DEBUG in dev
- Key fields: trace_id, user_id, service_name, timestamp
- **Health checks:**
- Liveness: `/health/live` (basic ping)
- Readiness: `/health/ready` (dependencies check)
## Deployment
- **Container:** Docker image, registry: [ECR / Docker Hub / etc.]
- **Orchestration:** Kubernetes
- **Scaling policy:**
- Min replicas: [N]
- Max replicas: [N]
- Scale trigger: CPU > [X]% or Memory > [X]% or RPS > [N]
- **Deployment strategy:** Rolling update / Canary / Blue-green
- **Rollback plan:** Automated rollback if error rate > [X]%
## Security
- **Authentication:** JWT tokens, validated via [Auth service / API Gateway]
- **Authorization:** Role-based access control (RBAC)
- **Service-to-service auth:** mTLS via service mesh
- **Secrets:** Stored in [AWS Secrets Manager / Vault / Kubernetes Secrets]
- **Input validation:** All user inputs validated and sanitized
- **Rate limiting:** Per-user and per-IP limits
## Testing
- **Unit tests:** Coverage target: 80%+
- **Integration tests:** Test API contracts and database interactions
- **Contract tests:** Pact/Spring Cloud Contract for upstream/downstream
- **Load tests:** Target [N] RPS at p95 < [Xms]
- **Chaos testing:** Simulate dependency failures, network latency
## Communication Patterns
- **Synchronous:** REST/gRPC for request-response
- **Asynchronous:** Kafka/RabbitMQ for events
- **Idempotency:** All write operations support idempotency keys
- **Message format:** JSON for REST, Protobuf for gRPC, Avro for Kafka
## Cost Optimization
- **Resource allocation:**
- CPU request: [Xm], limit: [Xm]
- Memory request: [XMi], limit: [XMi]
- **Auto-scaling:** Scale down during off-peak hours
- **Data retention:** [X days] for logs, [X days] for metrics
## Migration Plan
- **Phase 1:** [Extract service from monolith / Build new service]
- **Phase 2:** [Gradual traffic migration / Feature flag rollout]
- **Phase 3:** [Full cutover / Decommission old system]
- **Rollback criteria:** [Error rate / Latency / Business metrics]
## ADR References
- [ADR-001]: [Decision about technology choice]
- [ADR-002]: [Decision about data model]
assets/planning/adr-template.md
# Architecture Decision Record (ADR)
**ADR ID**: ADR-YYYY-MM-DD-XXX
**Title**: [Short decision title]
**Status**: Proposed | Accepted | Rejected | Superseded (link)
**Date**: YYYY-MM-DD
**Owner**: [Team / person]
**Deciders**: [Names / roles]
**Scope**: [Service / domain / product area]
---
## Core
### 1. Context
**Problem statement**: What are we deciding, and why now?
**Goals**:
- [Goal 1]
- [Goal 2]
**Non-goals**:
- [Non-goal 1]
- [Non-goal 2]
**Constraints** (REQUIRED):
- Regulatory/compliance:
- Latency/SLO:
- Data residency:
- Platform/runtime:
- Team/operational maturity:
**Assumptions** (mark unvalidated):
- [Assumption] ([Inference] if not verified)
### 2. Decision Drivers (What Matters Most)
Rank the drivers to make tradeoffs explicit.
| Priority | Driver | Why it matters | How we measure |
|---:|---|---|---|
| 1 | Reliability | | SLO, error budget |
| 2 | Security | | Threat model, control coverage |
| 3 | Cost | | Unit cost, infra spend |
| 4 | Delivery speed | | Lead time, deployment frequency |
| 5 | Operability | | On-call load, MTTR |
### 3. Options Considered
| Option | Summary | Pros | Cons | Reversibility |
|---|---|---|---|---|
| A | | | | Easy / Medium / Hard |
| B | | | | Easy / Medium / Hard |
| C | | | | Easy / Medium / Hard |
### 4. Decision
**We choose**: [Option X]
**Why** (1–5 bullets max):
- [Reason]
### 5. Architecture Impact (Implementation-Ready)
**Boundaries and contracts**
- Public APIs/contracts affected:
- Backward compatibility plan:
- Schema evolution strategy:
**Data and consistency**
- Source of truth:
- Consistency model (strong/eventual/mixed):
- Migration strategy (expand/contract, dual writes, backfill):
**Failure modes and resilience**
- Known failure modes:
- Timeouts/retries/backoff policy:
- Idempotency strategy:
- Degradation plan (what still works when dependencies fail):
**Security**
- Threat model summary:
- AuthN/AuthZ model:
- Secret and key management:
- Audit logging requirements:
**Observability**
- SLIs/SLOs:
- Metrics/traces/logs to add:
- Dashboards and alerts:
**Cost and capacity**
- Expected traffic/load:
- Cost model (drivers, main spend areas):
- Capacity plan (limits, scaling triggers):
### 6. Rollout, Validation, and Rollback
**Rollout plan**
- Feature flag / staged rollout:
- Data migration steps:
- Runbook updates:
**Validation plan**
- Tests to add (unit/integration/contract):
- Load/perf tests:
- Chaos/failure injection (if applicable):
**Rollback plan**
- How to revert code:
- How to revert data (or forward-fix):
- Timebox for rollback decision:
### 7. Consequences
**Positive**
- [Benefit]
**Negative / tradeoffs**
- [Cost or risk]
**Follow-ups**
- [Task] (owner, due date)
### 8. Links
- Design doc:
- Diagram(s):
- Tickets/epics:
- Related ADRs:
---
## Optional: AI/Automation
Include only if this ADR affects AI/automation features or AI-assisted workflows.
### AI Risk and Safety
- User impact of wrong output (harm model):
- Human override path and audit trail:
- Data handling (PII/PHI, retention, residency):
- Abuse cases (prompt injection, data exfiltration) [Inference]
### Evaluation and Quality Gates
- Offline evaluation set definition:
- Online metrics and guardrails:
- Rollback triggers for quality regression:
### Operations and Cost
- Latency/cost targets:
- Rate limiting and quotas:
- Observability for model/tool calls:
---
## References
- ADR concept and template rationale: https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions
assets/planning/architecture-blueprint.md
# Service Architecture Blueprint
Use this when drafting a new service or major redesign.
- **Service name / domain:** Bounded context, upstream/downstream dependencies
- **Non-negotiable requirements:** Latency/throughput targets, availability %, data retention, compliance
- **Architecture diagram link:** C4 level 2/3 diagrams, sequence for critical paths
- **Data flows:** Read/write paths, schemas, partitioning/replication strategy
- **Integration contracts:** APIs/events with SLAs, versioning, idempotency rules, consumers/providers
- **Resilience:** Timeouts, retries, circuit breakers, bulkheads, backpressure, graceful degradation
- **Security:** AuthN/AuthZ model, secrets handling, ingress/egress policies, audit requirements
- **Deployability:** CI/CD steps, canary/blue-green, rollback, migration plan
- **Observability:** Golden signals, dashboards, alert rules, trace spans, log fields
- **Risks & assumptions:** Unknowns, kill criteria, contingency plans
data/sources.json
{
"metadata": {
"skill": "software-architecture-design",
"description": "Curated resources for system design, architecture patterns, and scalability",
"last_updated": "2026-01-17",
"total_sources": 60
},
"architecture_patterns": [
{
"name": "Microservices.io - Pattern Catalog",
"url": "https://microservices.io/patterns/index.html",
"description": "Comprehensive microservices pattern catalog by Chris Richardson",
"add_as_web_search": true
},
{
"name": "Microsoft Azure - Architecture Center",
"url": "https://learn.microsoft.com/en-us/azure/architecture/",
"description": "Cloud architecture patterns, best practices, and design guidance",
"add_as_web_search": true
},
{
"name": "AWS Well-Architected Framework",
"url": "https://docs.aws.amazon.com/wellarchitected/latest/framework/welcome.html",
"description": "AWS architecture best practices across six pillars",
"add_as_web_search": true
},
{
"name": "Google Cloud Architecture Framework",
"url": "https://cloud.google.com/architecture/framework",
"description": "Google Cloud system design principles and patterns",
"add_as_web_search": true
},
{
"name": "Martin Fowler - Architecture",
"url": "https://martinfowler.com/architecture/",
"description": "Classic architecture patterns and refactoring guides",
"add_as_web_search": false
},
{
"name": "C4 Model",
"url": "https://c4model.com/",
"description": "Lightweight architecture diagramming model optimized for communication and maintenance",
"add_as_web_search": true
},
{
"name": "The Twelve-Factor App",
"url": "https://12factor.net/",
"description": "Operational principles for cloud-native apps (config, logs, disposability, concurrency)",
"add_as_web_search": true
}
],
"distributed_systems": [
{
"name": "ByteByteGo - Scalability Patterns",
"url": "https://blog.bytebytego.com/p/scalability-patterns-for-modern-distributed",
"description": "Scalability patterns for distributed systems with diagrams",
"add_as_web_search": true
},
{
"name": "Designing Distributed Systems (Book)",
"url": "https://info.microsoft.com/rs/157-GQE-382/images/EN-CNTNT-eBook-DesigningDistributedSystems.pdf",
"description": "Brendan Burns - Patterns using Kubernetes and containers",
"add_as_web_search": false
},
{
"name": "Distributed Systems Patterns",
"url": "https://dev.to/somadevtoo/9-software-architecture-patterns-for-distributed-systems-2o86",
"description": "Essential patterns for scalability and resilience",
"add_as_web_search": true
},
{
"name": "System Design Scalability Guide",
"url": "https://www.geeksforgeeks.org/guide-for-designing-highly-scalable-systems/",
"description": "Comprehensive guide to designing scalable systems",
"add_as_web_search": true
}
],
"adr_templates": [
{
"name": "ADR GitHub Organization",
"url": "https://adr.github.io/",
"description": "Official ADR homepage with multiple template formats",
"add_as_web_search": false
},
{
"name": "Documenting Architecture Decisions (Michael Nygard)",
"url": "https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions",
"description": "The canonical ADR write-up: when to write ADRs and what to include",
"add_as_web_search": true
},
{
"name": "ADR Template Repository",
"url": "https://github.com/joelparkerhenderson/architecture-decision-record",
"description": "200+ ADR examples and template variations",
"add_as_web_search": false
},
{
"name": "AWS ADR Best Practices",
"url": "https://aws.amazon.com/blogs/architecture/master-architecture-decision-records-adrs-best-practices-for-effective-decision-making/",
"description": "AWS guidance on effective ADR processes",
"add_as_web_search": true
},
{
"name": "Microsoft ADR Guide",
"url": "https://learn.microsoft.com/en-us/azure/well-architected/architect-role/architecture-decision-record",
"description": "Azure Well-Architected ADR practices",
"add_as_web_search": true
},
{
"name": "MADR - Markdown ADR",
"url": "https://adr.github.io/madr/",
"description": "Markdown ADR format with pros/cons analysis",
"add_as_web_search": false
}
],
"scalability_reliability": [
{
"name": "Google SRE Book",
"url": "https://sre.google/sre-book/table-of-contents/",
"description": "Site Reliability Engineering practices from Google",
"add_as_web_search": false
},
{
"name": "Google SRE Workbook",
"url": "https://sre.google/workbook/table-of-contents/",
"description": "Practical SRE implementation guide",
"add_as_web_search": false
},
{
"name": "CAP Theorem Explained",
"url": "https://www.ibm.com/topics/cap-theorem",
"description": "Understanding consistency, availability, partition tolerance",
"add_as_web_search": false
},
{
"name": "Database Scaling Patterns",
"url": "https://learn.microsoft.com/en-us/azure/architecture/patterns/sharding",
"description": "Sharding, replication, and partitioning strategies",
"add_as_web_search": true
},
{
"name": "Circuit Breaker Pattern",
"url": "https://learn.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker",
"description": "Prevent cascading failures in distributed systems",
"add_as_web_search": false
},
{
"name": "Retry Pattern",
"url": "https://learn.microsoft.com/en-us/azure/architecture/patterns/retry",
"description": "Exponential backoff and retry strategies",
"add_as_web_search": false
},
{
"name": "Bulkhead Pattern",
"url": "https://learn.microsoft.com/en-us/azure/architecture/patterns/bulkhead",
"description": "Isolate resources to prevent total failure",
"add_as_web_search": false
}
],
"event_driven": [
{
"name": "Event-Driven Architecture Guide",
"url": "https://aws.amazon.com/event-driven-architecture/",
"description": "AWS patterns for event-driven systems",
"add_as_web_search": true
},
{
"name": "Event Sourcing Pattern",
"url": "https://martinfowler.com/eaaDev/EventSourcing.html",
"description": "Martin Fowler's event sourcing guide",
"add_as_web_search": false
},
{
"name": "CQRS Pattern",
"url": "https://martinfowler.com/bliki/CQRS.html",
"description": "Command Query Responsibility Segregation",
"add_as_web_search": false
},
{
"name": "Saga Pattern",
"url": "https://microservices.io/patterns/data/saga.html",
"description": "Distributed transaction management",
"add_as_web_search": false
}
],
"observability": [
{
"name": "OpenTelemetry",
"url": "https://opentelemetry.io/docs/",
"description": "Observability framework for cloud-native software",
"add_as_web_search": true
},
{
"name": "Distributed Tracing Best Practices",
"url": "https://opentelemetry.io/docs/concepts/observability-primer/",
"description": "Understanding traces, metrics, and logs",
"add_as_web_search": true
},
{
"name": "SLI/SLO/SLA Guide",
"url": "https://sre.google/sre-book/service-level-objectives/",
"description": "Service level objectives and error budgets",
"add_as_web_search": false
}
],
"api_design": [
{
"name": "REST API Best Practices",
"url": "https://learn.microsoft.com/en-us/azure/architecture/best-practices/api-design",
"description": "Microsoft API design guidelines",
"add_as_web_search": true
},
{
"name": "RFC 9457 - Problem Details for HTTP APIs",
"url": "https://www.rfc-editor.org/rfc/rfc9457",
"description": "Standard error response format for HTTP APIs (updates RFC 7807)",
"add_as_web_search": true
},
{
"name": "GraphQL Best Practices",
"url": "https://graphql.org/learn/best-practices/",
"description": "Official GraphQL patterns and conventions",
"add_as_web_search": true
},
{
"name": "gRPC Design Guide",
"url": "https://grpc.io/docs/guides/",
"description": "High-performance RPC framework patterns",
"add_as_web_search": true
},
{
"name": "API Gateway Pattern",
"url": "https://microservices.io/patterns/apigateway.html",
"description": "Single entry point for microservices",
"add_as_web_search": false
}
],
"security_architecture": [
{
"name": "OWASP Architecture Cheat Sheet",
"url": "https://cheatsheetseries.owasp.org/cheatsheets/Microservices_Security_Cheat_Sheet.html",
"description": "Security patterns for microservices",
"add_as_web_search": true
},
{
"name": "Zero Trust Architecture",
"url": "https://www.nist.gov/publications/zero-trust-architecture",
"description": "NIST zero trust security model",
"add_as_web_search": false
},
{
"name": "Service Mesh Security",
"url": "https://istio.io/latest/docs/concepts/security/",
"description": "mTLS and service-to-service authentication",
"add_as_web_search": true
}
],
"books_resources": [
{
"name": "Designing Data-Intensive Applications",
"url": "https://dataintensive.net/",
"description": "Martin Kleppmann - Distributed systems bible",
"add_as_web_search": false
},
{
"name": "Building Microservices (2nd Edition)",
"url": "https://www.oreilly.com/library/view/building-microservices-2nd/9781492034018/",
"description": "Sam Newman - Microservices patterns",
"add_as_web_search": false
},
{
"name": "Release It! (2nd Edition)",
"url": "https://pragprog.com/titles/mnee2/release-it-second-edition/",
"description": "Michael Nygard - Production-ready patterns",
"add_as_web_search": false
}
],
"modern_architecture_2025": [
{
"name": "InfoQ Architecture Trends 2025",
"url": "https://www.infoq.com/articles/architecture-trends-2025/",
"description": "Annual architecture trends: data mesh, composable, edge computing, AI integration",
"add_as_web_search": true
},
{
"name": "Data Mesh Architecture - Zhamak Dehghani",
"url": "https://www.datamesh-architecture.com/",
"description": "Canonical data mesh principles: domain ownership, data as product, federated governance",
"add_as_web_search": true
},
{
"name": "CNCF Platform Engineering Maturity Model",
"url": "https://tag-app-delivery.cncf.io/whitepapers/platforms/",
"description": "Internal developer platforms, golden paths, self-service infrastructure",
"add_as_web_search": true
},
{
"name": "Continuous Architecture in Practice",
"url": "https://continuousarchitecture.com/",
"description": "Iterative architecture planning, fitness functions, just-enough upfront design",
"add_as_web_search": false
},
{
"name": "Edge Computing Patterns - AWS",
"url": "https://aws.amazon.com/edge/",
"description": "Edge computing patterns for IoT, latency-sensitive applications",
"add_as_web_search": true
},
{
"name": "Composable Architecture - Gartner",
"url": "https://www.gartner.com/en/information-technology/glossary/composable-business",
"description": "Packaged business capabilities (PBCs), API-first composition",
"add_as_web_search": false
}
],
"optional_ai_architecture": [
{
"name": "RAG Architecture Patterns - Anthropic",
"url": "https://docs.anthropic.com/en/docs/build-with-claude/retrieval-augmented-generation",
"description": "Retrieval-augmented generation: vector stores, retrievers, context packing",
"add_as_web_search": true,
"optional": true
},
{
"name": "Building Effective Agents - Anthropic",
"url": "https://www.anthropic.com/research/building-effective-agents",
"description": "Multi-agent architecture: single, hierarchical, decentralized patterns",
"add_as_web_search": true,
"optional": true
},
{
"name": "LangChain Architecture",
"url": "https://python.langchain.com/docs/concepts/",
"description": "LLM application architecture: chains, agents, memory, retrieval",
"add_as_web_search": true,
"optional": true
},
{
"name": "LlamaIndex RAG Patterns",
"url": "https://docs.llamaindex.ai/en/stable/",
"description": "Data indexing, retrieval strategies, response synthesis",
"add_as_web_search": true,
"optional": true
},
{
"name": "Google's Eight Multi-Agent Design Patterns",
"url": "https://www.infoq.com/news/2026/01/multi-agent-design-patterns/",
"description": "Eight foundational multi-agent architectures: sequential, loop, parallel, hierarchical, bidding, human-in-loop",
"add_as_web_search": true,
"optional": true
},
{
"name": "Google Cloud - Agent Design Pattern Guide",
"url": "https://docs.cloud.google.com/architecture/choose-design-pattern-agentic-ai-system",
"description": "Official Google guidance on agentic AI architecture patterns and A2A protocol",
"add_as_web_search": true,
"optional": true
},
{
"name": "MCP Architecture Patterns - Speakeasy",
"url": "https://www.speakeasy.com/mcp/using-mcp/ai-agents/architecture-patterns",
"description": "Model Context Protocol integration patterns for AI agents",
"add_as_web_search": true,
"optional": true
},
{
"name": "Salesforce - Agentic Enterprise IT Architecture",
"url": "https://architect.salesforce.com/fundamentals/agentic-enterprise-it-architecture",
"description": "Enterprise architecture for AI-powered applications and headless app re-engineering",
"add_as_web_search": true,
"optional": true
}
],
"platform_engineering_2026": [
{
"name": "Platform Engineering Predictions 2026",
"url": "https://platformengineering.org/blog/10-platform-engineering-predictions-for-2026",
"description": "AI-platform convergence, FinOps integration, internal developer platforms, unified delivery pipelines",
"add_as_web_search": true
},
{
"name": "AI Merging with Platform Engineering - The New Stack",
"url": "https://thenewstack.io/in-2026-ai-is-merging-with-platform-engineering-are-you-ready/",
"description": "AI agents as first-class platform citizens, unified ML and app delivery",
"add_as_web_search": true
},
{
"name": "Backstage - Spotify Internal Developer Portal",
"url": "https://backstage.io/docs/overview/what-is-backstage",
"description": "Widely used IDP framework: service catalog, templates (golden paths), and developer portal patterns",
"add_as_web_search": true
},
{
"name": "Modular Monolith Guide - ByteByteGo",
"url": "https://blog.bytebytego.com/p/monolith-vs-microservices-vs-modular",
"description": "Monolith vs microservices vs modular monolith comparison with industry data",
"add_as_web_search": true
}
]
}
references/api-gateway-service-mesh.md
# API Gateway & Service Mesh Patterns
Deep reference for API gateway architectures, service mesh implementation (Istio, Linkerd, Envoy), sidecar patterns, and service-to-service communication. Use when designing inter-service communication, implementing traffic management, or choosing between gateway, mesh, or hybrid topologies.
## Contents
- [API Gateway Patterns](#api-gateway-patterns)
- [Service Mesh Architecture](#service-mesh-architecture)
- [Technology Comparison](#technology-comparison)
- [mTLS and Service Identity](#mtls-and-service-identity)
- [Observability Through Mesh](#observability-through-mesh)
- [Gateway vs Mesh vs Both](#gateway-vs-mesh-vs-both)
- [Implementation Patterns](#implementation-patterns)
- [Anti-Patterns](#anti-patterns)
- [Decision Framework](#decision-framework)
- [Cross-References](#cross-references)
---
## API Gateway Patterns
### Core Gateway Responsibilities
| Function | Description | Example |
|----------|-------------|---------|
| Routing | Route requests to backend services | `/api/orders/*` -> orders-service |
| Authentication | Validate tokens, API keys | JWT verification, OAuth introspection |
| Rate limiting | Throttle requests per client/endpoint | 100 req/min per API key |
| Request transformation | Modify headers, body, path | Add correlation IDs, strip internal headers |
| Response aggregation | Combine multiple backend responses | BFF pattern for mobile clients |
| Load balancing | Distribute traffic across instances | Round-robin, least connections, weighted |
| Caching | Cache responses at the edge | Cache GET responses with TTL |
| Circuit breaking | Fail fast when backend is unhealthy | Open circuit after 5 consecutive failures |
| TLS termination | Handle HTTPS at the gateway | Offload TLS from backend services |
### Gateway Topology Patterns
**Pattern 1: Single Gateway**
```text
┌────────┐ ┌──────────────┐ ┌──────────┐
│ Client │────▶│ Gateway │────▶│ Service A│
└────────┘ │ │────▶│ Service B│
│ │────▶│ Service C│
└──────────────┘ └──────────┘
```
Best for: Small teams, <10 services, uniform client needs.
**Pattern 2: Backend-for-Frontend (BFF)**
```text
┌──────────┐ ┌───────────────┐
│ Mobile │────▶│ Mobile BFF │───▶ Services
└──────────┘ └───────────────┘
┌──────────┐ ┌───────────────┐
│ Web │────▶│ Web BFF │───▶ Services
└──────────┘ └───────────────┘
┌──────────┐ ┌───────────────┐
│ Partner │────▶│ Partner API │───▶ Services
│ API │ │ Gateway │
└──────────┘ └───────────────┘
```
Best for: Different client types with distinct data needs.
**Pattern 3: Federated Gateway**
```text
┌────────┐ ┌───────────────┐ ┌──────────────┐
│ Client │────▶│ Edge Gateway │────▶│ Team A GW │───▶ Team A services
└────────┘ │ (auth, rate │────▶│ Team B GW │───▶ Team B services
│ limiting) │────▶│ Team C GW │───▶ Team C services
└───────────────┘ └──────────────┘
```
Best for: Large organizations, multiple teams owning their own gateway configuration.
### Rate Limiting Patterns
```typescript
// Token bucket rate limiting (typical gateway configuration)
// Kong rate-limiting plugin configuration
const rateLimitConfig = {
plugin: 'rate-limiting',
config: {
minute: 60, // 60 requests per minute
hour: 1000, // 1000 requests per hour
policy: 'redis', // Use Redis for distributed counting
fault_tolerant: true, // Allow traffic if Redis is down
hide_client_headers: false,
redis_host: 'redis',
redis_port: 6379,
},
};
// Response headers
// X-RateLimit-Limit: 60
// X-RateLimit-Remaining: 45
// X-RateLimit-Reset: 1640000000
```
| Algorithm | Description | Use When |
|-----------|-------------|----------|
| Token bucket | Tokens replenish at fixed rate, consumed per request | Burst-tolerant, smooth throttling |
| Sliding window | Count requests in a rolling time window | Precise limits, no burst |
| Fixed window | Count requests per fixed time interval | Simple, slight burst at window edges |
| Leaky bucket | Process requests at fixed rate, queue excess | Smooth output rate |
### Request Aggregation
```typescript
// BFF aggregation pattern — single client request, multiple backend calls
app.get('/api/dashboard', async (req, res) => {
const userId = req.user.id;
// Parallel fetch from multiple services
const [profile, orders, notifications, recommendations] = await Promise.all([
userService.getProfile(userId),
orderService.getRecent(userId, { limit: 5 }),
notificationService.getUnread(userId),
recommendationService.getForUser(userId),
]);
// Aggregate into client-optimized response
res.json({
user: { name: profile.name, avatar: profile.avatar },
recentOrders: orders.map(o => ({ id: o.id, status: o.status, total: o.total })),
unreadCount: notifications.length,
recommendations: recommendations.slice(0, 3),
});
});
```
---
## Service Mesh Architecture
### Core Concepts
A service mesh is a dedicated infrastructure layer for service-to-service communication. It uses sidecar proxies deployed alongside each service to handle networking concerns.
```text
┌─────────────────────────────────────────────────┐
│ Control Plane │
│ (Configuration, certificates, policies) │
└───────────┬──────────────┬──────────────┬────────┘
│ │ │
┌──────▼──────┐ ┌────▼────────┐ ┌──▼──────────┐
│ Service A │ │ Service B │ │ Service C │
│ ┌────────┐ │ │ ┌────────┐ │ │ ┌────────┐ │
│ │ App │ │ │ │ App │ │ │ │ App │ │
│ └───┬────┘ │ │ └───┬────┘ │ │ └───┬────┘ │
│ ┌───▼────┐ │ │ ┌───▼────┐ │ │ ┌───▼────┐ │
│ │Sidecar │◄├─┼─▶│Sidecar │◄├─┼─▶│Sidecar │ │
│ │Proxy │ │ │ │Proxy │ │ │ │Proxy │ │
│ └────────┘ │ │ └────────┘ │ │ └────────┘ │
└─────────────┘ └────────────┘ └──────────────┘
Data Plane (proxies handle all traffic)
```
### Sidecar Proxy Responsibilities
| Responsibility | Description |
|---------------|-------------|
| Traffic routing | Route requests based on rules (headers, weight, path) |
| Load balancing | Distribute traffic across service instances |
| mTLS encryption | Encrypt all service-to-service traffic |
| Circuit breaking | Prevent cascading failures |
| Retry and timeout | Automatic retry with configurable backoff |
| Observability | Emit metrics, traces, and access logs |
| Health checking | Active and passive health checks |
| Rate limiting | Per-service or per-route limits |
| Access control | Authorization policies between services |
### Control Plane vs Data Plane
| Component | Control Plane | Data Plane |
|-----------|--------------|------------|
| Role | Configuration and policy distribution | Request processing |
| Components | Istiod, Linkerd control plane | Envoy proxy, Linkerd proxy |
| Scaling | Single instance or HA pair | One per service instance (sidecar) |
| Failure impact | No new config updates, existing config works | Service-to-service traffic affected |
| Resource usage | Low (control only) | Per-pod overhead (CPU, memory, latency) |
---
## Technology Comparison
### Gateway Comparison
| Feature | Kong | AWS API Gateway | Envoy (standalone) | Traefik | NGINX |
|---------|------|----------------|--------------------|---------| ------|
| Deployment | Self-hosted / Cloud | Managed | Self-hosted | Self-hosted | Self-hosted |
| Plugin ecosystem | Large (Lua, Go) | AWS integrations | Filters (C++, Wasm) | Middleware | Modules |
| Rate limiting | Built-in | Built-in | Filter | Middleware | Module |
| Auth | JWT, OAuth, OIDC | IAM, Cognito, Lambda auth | ext_authz filter | ForwardAuth | Auth module |
| Observability | Prometheus, Datadog | CloudWatch | Prometheus, Zipkin | Prometheus | Stub status |
| gRPC support | Yes | Yes | Native | Yes | Yes |
| WebSocket | Yes | Yes | Yes | Yes | Yes |
| Best for | Multi-cloud, plugin needs | AWS-native workloads | High performance, mesh ingress | Docker/K8s auto-discovery | Simple, proven |
### Service Mesh Comparison
| Feature | Istio | Linkerd | AWS App Mesh | Cilium Service Mesh |
|---------|-------|---------|--------------|---------------------|
| Proxy | Envoy | linkerd2-proxy (Rust) | Envoy | eBPF (no sidecar) |
| Complexity | High | Low | Medium | Medium |
| Resource overhead | Higher (Envoy sidecar) | Lower (lightweight proxy) | Medium | Lowest (kernel-level) |
| mTLS | Automatic | Automatic | Manual config | Automatic |
| Multi-cluster | Yes | Yes (limited) | Cross-account | Yes |
| Traffic management | Advanced (fault injection, mirroring) | Basic (split, retry) | Basic | Advanced |
| Observability | Rich (Kiali, Jaeger, Prometheus) | Built-in dashboard | CloudWatch, X-Ray | Hubble |
| Learning curve | Steep | Gentle | Moderate | Moderate |
| Best for | Complex mesh, advanced traffic | Simple mesh, low overhead | AWS-native workloads | High performance, eBPF |
### Selection Quick Guide
```text
Service mesh selection:
├─ < 10 services → No mesh needed. Use library-based patterns.
├─ 10-50 services, want simplicity → Linkerd
├─ 10-50 services, need advanced traffic mgmt → Istio
├─ AWS-native, managed preference → App Mesh
├─ Performance-critical, eBPF available → Cilium
└─ > 50 services, multi-cluster → Istio (with careful tuning)
```
---
## mTLS and Service Identity
### Mutual TLS in Service Mesh
```text
Service A Service B
┌──────────┐ ┌──────────┐
│ │── 1. TLS handshake (client cert) ──▶│ │
│ App │◀─ 2. TLS handshake (server cert) ──│ App │
│ │── 3. Encrypted traffic ────────────▶│ │
└──────────┘ └──────────┘
Both sides present certificates issued by the mesh CA.
Identity is based on service account, not network address.
```
### Identity-Based Authorization
```yaml
# Istio AuthorizationPolicy
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
name: orders-policy
namespace: production
spec:
selector:
matchLabels:
app: orders-service
rules:
# Allow only payment-service and api-gateway to call orders
- from:
- source:
principals:
- cluster.local/ns/production/sa/payment-service
- cluster.local/ns/production/sa/api-gateway
to:
- operation:
methods: ["GET", "POST"]
paths: ["/api/orders/*"]
# Deny everything else (implicit deny)
```
### Certificate Management
| Approach | Description | Complexity |
|----------|-------------|------------|
| Mesh-managed CA | Mesh control plane issues and rotates certs | Low (automatic) |
| External CA (Vault) | HashiCorp Vault issues certs, mesh distributes | Medium |
| SPIFFE/SPIRE | Standard identity framework, pluggable CAs | Medium-High |
| Manual certs | Team manages certs manually | High (do not do this) |
**Recommended:** Use mesh-managed CA for most deployments. Integrate with external CA (Vault, AWS ACM PCA) for enterprise compliance requirements.
---
## Observability Through Mesh
### Distributed Tracing
Service mesh proxies automatically inject trace headers and emit spans.
```text
Client → [Gateway span] → [Service A span] → [Service B span] → [Database span]
│
[Service C span]
Each proxy adds its own span without application code changes.
```
**Trace header propagation:**
| Header | Standard | Used By |
|--------|----------|---------|
| `traceparent` | W3C Trace Context | OpenTelemetry, modern systems |
| `x-request-id` | De facto | Envoy, Istio |
| `x-b3-traceid` | Zipkin B3 | Zipkin, older Istio |
### Mesh-Level Metrics
Standard metrics emitted by sidecar proxies (no application instrumentation needed):
| Metric | Description | Alert On |
|--------|-------------|----------|
| `request_count` | Total requests per service/route | Unexpected traffic drops |
| `request_duration` | Latency histogram (P50, P95, P99) | P99 > SLO threshold |
| `response_code` | Count by status code (2xx, 4xx, 5xx) | 5xx rate > 0.1% |
| `tcp_connections` | Active TCP connections | Approaching connection limits |
| `retry_count` | Automatic retries triggered | High retry rate = unhealthy upstream |
### Golden Signals Dashboard
```text
For each service, track:
1. Latency — P50, P95, P99 response time
2. Traffic — Requests per second
3. Errors — 5xx rate as percentage of total
4. Saturation — CPU, memory, connection pool usage
Service mesh provides signals 1-3 automatically.
Signal 4 requires application-level metrics.
```
### Service Topology Visualization
Mesh provides automatic service dependency mapping:
```text
Tools:
- Kiali (Istio) — Service graph, health status, traffic flow
- Linkerd Viz — Dashboard with golden metrics per service
- Hubble (Cilium) — Network flow visibility
- Jaeger/Tempo — Distributed trace visualization
```
---
## Gateway vs Mesh vs Both
### Comparison Matrix
| Concern | API Gateway | Service Mesh | Both (Recommended) |
|---------|-------------|-------------|-------------------|
| North-south traffic (client → service) | Primary role | Not designed for | Gateway handles |
| East-west traffic (service → service) | Not designed for | Primary role | Mesh handles |
| External authentication | Yes | No | Gateway handles |
| Service-to-service auth (mTLS) | No | Yes | Mesh handles |
| Public rate limiting | Yes | No | Gateway handles |
| Internal circuit breaking | Limited | Yes | Mesh handles |
| External API versioning | Yes | No | Gateway handles |
| Internal traffic splitting | No | Yes | Mesh handles |
| TLS termination (external) | Yes | No | Gateway handles |
| mTLS (internal) | No | Yes | Mesh handles |
### Recommended Architecture
```text
┌──────────────┐
│ Internet │
└──────┬───────┘
│
┌──────▼───────┐ ← North-south boundary
│ API Gateway │ Auth, rate limiting, TLS termination, routing
│ (Kong/Envoy) │
└──────┬───────┘
│
┌──────▼───────────────────────────────┐
│ Service Mesh (Istio/Linkerd) │ ← East-west boundary
│ │ mTLS, retries, circuit breaking,
│ ┌─────┐ ┌─────┐ ┌─────┐ │ observability, traffic management
│ │Svc A│◄─▶│Svc B│◄─▶│Svc C│ │
│ └─────┘ └─────┘ └─────┘ │
└──────────────────────────────────────┘
```
### When You Do NOT Need a Service Mesh
| Scenario | Why No Mesh |
|----------|-------------|
| < 10 services | Overhead exceeds benefit |
| Single team, single repo | Library-based patterns suffice |
| Serverless architecture | Functions are too short-lived for sidecars |
| Low traffic internal tools | Complexity not justified |
| Early-stage startup | Focus on product, not infrastructure |
**Alternatives to mesh for small deployments:**
- Library-based retries and circuit breaking (p-retry, opossum)
- Application-level mTLS (cert-manager + application config)
- OpenTelemetry SDK for observability (no mesh needed)
---
## Implementation Patterns
### Canary Deployment via Mesh
```yaml
# Istio: Route 95% to v1, 5% to v2
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: orders
spec:
hosts:
- orders
http:
- route:
- destination:
host: orders
subset: v1
weight: 95
- destination:
host: orders
subset: v2
weight: 5
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: orders
spec:
host: orders
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
```
### Retry and Timeout Configuration
```yaml
# Istio retry policy
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: orders
spec:
hosts:
- orders
http:
- timeout: 3s
retries:
attempts: 3
perTryTimeout: 1s
retryOn: 5xx,reset,connect-failure,retriable-4xx
route:
- destination:
host: orders
```
### Circuit Breaker Configuration
```yaml
# Istio circuit breaker
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: orders
spec:
host: orders
trafficPolicy:
connectionPool:
tcp:
maxConnections: 100
http:
http1MaxPendingRequests: 100
http2MaxRequests: 1000
maxRequestsPerConnection: 10
outlierDetection:
consecutive5xxErrors: 5
interval: 10s
baseEjectionTime: 30s
maxEjectionPercent: 50
```
---
## Anti-Patterns
### 1. Mesh for Everything
```text
[FAIL] Deploying a service mesh for 3 services because "everyone uses Istio"
→ Massive operational overhead for minimal benefit
[PASS] Start with library-based patterns. Adopt mesh when you have
10+ services and measurable networking pain.
```
### 2. Gateway as Service Bus
```text
[FAIL] Putting business logic, data transformation, and orchestration in the gateway
→ Gateway becomes a bottleneck and single point of failure for logic
[PASS] Gateway handles cross-cutting concerns only (auth, rate limiting, routing).
Business logic stays in services.
```
### 3. Ignoring Sidecar Resource Overhead
```text
[FAIL] Deploying mesh without accounting for sidecar CPU/memory per pod
→ Resource exhaustion, unexpected costs, latency increase
[PASS] Budget 50-128MB memory and 0.1-0.5 CPU per sidecar proxy.
Monitor mesh overhead as a first-class metric.
```
### 4. No Mesh Bypass for Debugging
```text
[FAIL] No way to bypass the mesh for troubleshooting
→ Cannot isolate whether issues are mesh-related or application-related
[PASS] Maintain ability to disable sidecar injection per namespace or pod.
Have runbooks for mesh-bypass debugging.
```
### 5. mTLS Without Identity Policies
```text
[FAIL] Enabling mTLS but no authorization policies → all services can call all services
→ mTLS only encrypts traffic, it does not authorize
[PASS] Pair mTLS with explicit AuthorizationPolicy rules.
Default deny, explicit allow per service pair.
```
---
## Decision Framework
### Do You Need an API Gateway?
```text
1. Do you expose APIs to external clients? [Yes → You need a gateway]
2. Do you need centralized auth for external traffic? [Yes → Gateway]
3. Do you need rate limiting for public APIs? [Yes → Gateway]
4. Do you have multiple BFF needs (mobile, web, partner)? [Yes → Multiple gateways]
5. Internal-only services? [Gateway optional, mesh sufficient]
```
### Do You Need a Service Mesh?
```text
1. How many services? [< 10 → No, 10-50 → Maybe, > 50 → Yes]
2. Do you need mTLS between services? [Yes → +2]
3. Do you need advanced traffic management? [Yes → +2]
4. Is observability a gap today? [Yes → +1]
5. Does the team have K8s experience? [Yes → +1, No → -2]
6. Are you running on Kubernetes? [Yes → +1, No → -2]
Score: 0-2 → Library-based patterns
Score: 3-4 → Linkerd (simpler mesh)
Score: 5+ → Istio or Cilium (full mesh)
```
### Gateway Selection
```text
Choosing a gateway:
├─ AWS-native, managed preference → AWS API Gateway
├─ Multi-cloud, plugin extensibility → Kong
├─ Performance-critical, already using Envoy → Envoy as gateway
├─ Kubernetes auto-discovery → Traefik
├─ Simple reverse proxy, proven → NGINX
└─ GraphQL federation → Apollo Router
```
---
## Cross-References
- [modern-patterns.md](modern-patterns.md) — Service mesh and microservices overview
- [scalability-reliability-guide.md](scalability-reliability-guide.md) — Load balancing, circuit breakers, resilience
- [migration-modernization-guide.md](migration-modernization-guide.md) — Strangler fig pattern with gateway routing
- [data-architecture-patterns.md](data-architecture-patterns.md) — Service-to-service data patterns
- [../software-backend/SKILL.md](../software-backend/SKILL.md) — Backend service implementation
- [../../ops-devops-platform/SKILL.md](../../ops-devops-platform/SKILL.md) — Kubernetes, CI/CD, deployment strategies
- [../../software-security-appsec/SKILL.md](../../software-security-appsec/SKILL.md) — Zero-trust security, mTLS
references/architecture-trends-2026.md
# Architecture Trends (2026)
Use this reference when the user explicitly asks for "current" or "2026" guidance, or when the system design depends on ecosystem maturity (managed services, tooling, compliance, cost).
## Platform Engineering and Internal Developer Platforms (IDPs)
Goal: reduce cognitive load and standardize delivery via self-service "golden paths".
Common building blocks:
- Service catalog and ownership (systems, components, dependencies)
- Templates/scaffolding ("paved roads") for new services and common workflows
- Self-service provisioning (IaC APIs, opinionated modules)
- Policy as code (security, compliance, FinOps guardrails)
- Built-in observability defaults (logs/metrics/traces, dashboards, alerts)
When to use:
- Multiple product teams with recurring platform needs
- Frequent service creation or consistent compliance requirements
- High operational overhead and inconsistent delivery practices
Avoid:
- Building a portal without paved roads (catalog without outcomes)
- Platform team as a ticket queue (no true self-service)
## Data Mesh (Analytics and Data Product Architecture)
Goal: scale analytics by shifting ownership to domain teams and standardizing interoperability.
Core ideas:
- Domain-owned data products with SLAs (freshness, latency, schema stability)
- Federated governance (standards + tooling, not a central bottleneck)
- Contracts and versioning for schemas and semantic definitions
When to use:
- Cross-domain analytics is slowed by central data bottlenecks
- Multiple domains need to publish reliable datasets to many consumers
Avoid:
- Rebranding a data lake as data mesh without ownership and contracts
- Uncontrolled schema changes without consumer communication
## Composable Architecture (Packaged Business Capabilities)
Goal: assemble business capabilities quickly via well-defined contracts.
Typical characteristics:
- API-first capability components with clear ownership
- Event-driven coordination for cross-capability workflows
- Composition layer (workflow engine, orchestration, or integration platform)
When to use:
- You need to rapidly combine capabilities across products or teams
- You have a stable set of reusable domain capabilities
Avoid:
- Tight coupling through shared databases or shared internal libraries
## Continuous Architecture and Fitness Functions
Goal: keep architecture aligned with reality through automation and regular review.
Practices:
- "Just-enough" upfront design, iterate based on feedback and risk
- Fitness functions: automated checks that enforce architectural constraints (dependency rules, SLO budgets, cost gates)
- ADRs for irreversible tradeoffs, revisited when assumptions change
When to use:
- Any long-lived product where architectural drift is a risk
- Systems with explicit constraints (latency, compliance, cost)
## Edge-First and Hybrid Edge/Cloud
Goal: meet latency, bandwidth, or offline requirements via local processing.
Common patterns:
- Edge caching and request shaping (CDN, edge gateways)
- Edge validation and filtering (reduce bandwidth to cloud)
- Hybrid pipelines (edge aggregation, cloud analytics and long-term storage)
When to use:
- Real-time UX needs, constrained networks, IoT/OT environments
Avoid:
- Splitting logic across edge/cloud without clear data ownership and observability
## AI-Native System Architecture (RAG, Tools, Agents)
Use when LLMs are part of the product or internal platform.
RAG and tool patterns:
- Retrieval as a bounded subsystem (indexing, access control, evaluation)
- Tool gateway layer (rate limits, authZ, auditing, allowlists)
- Async orchestration for slow and failure-prone steps (queues, workflows)
Production requirements that are easy to miss:
- Evaluation and regression testing (golden sets, drift checks)
- Observability tailored to AI (prompt/response logging policy, safety filters, cost tracking)
- Security (prompt injection, data exfiltration, tool abuse, multi-tenant isolation)
Anti-patterns:
- Using a vector store as the source of truth for business data
- Shipping agents without termination conditions and without audit logs
references/data-architecture-patterns.md
# Data Architecture Patterns
Comprehensive guide to data architecture patterns for distributed systems: CQRS, event sourcing, data mesh, polyglot persistence, saga patterns, and consistency models. Use when designing data flows across service boundaries, choosing storage strategies, or managing distributed state.
## Contents
- [CQRS Implementation Patterns](#cqrs-implementation-patterns)
- [Event Sourcing](#event-sourcing)
- [Data Mesh](#data-mesh)
- [Polyglot Persistence](#polyglot-persistence)
- [Saga Patterns](#saga-patterns)
- [Consistency Models](#consistency-models)
- [Anti-Patterns](#anti-patterns)
- [Decision Framework](#decision-framework)
- [Cross-References](#cross-references)
---
## CQRS Implementation Patterns
### When to Use CQRS
| Indicator | Strength |
|-----------|----------|
| Read/write ratio > 10:1 | Strong signal |
| Read and write models differ structurally | Strong signal |
| Independent scaling of reads vs writes needed | Strong signal |
| Simple CRUD with uniform access | Do NOT use CQRS |
| Small team, single database | Do NOT use CQRS |
| Prototype or MVP stage | Do NOT use CQRS |
### CQRS Variants
**Variant 1: Same Database, Separate Models**
Simplest form. Single database with distinct read/write code paths.
```typescript
// Write side — normalized, validated
class OrderCommandHandler {
async createOrder(cmd: CreateOrderCommand): Promise<void> {
const order = Order.create(cmd.customerId, cmd.items);
await this.writeRepo.save(order); // normalized tables
await this.eventBus.publish(order.events); // domain events
}
}
// Read side — denormalized, optimized for queries
class OrderQueryHandler {
async getCustomerDashboard(customerId: string): Promise<Dashboard> {
// Single query against a denormalized view or materialized table
return this.readRepo.getDashboard(customerId);
}
}
```
**Variant 2: Separate Databases**
Write DB (normalized, ACID) + Read DB (denormalized, optimized). Sync via events.
```text
┌──────────────┐ events ┌──────────────────┐
│ Write DB │ ──────────────▶│ Read DB │
│ (Postgres) │ │ (Elasticsearch / │
│ normalized │ │ Redis / Mongo) │
└──────────────┘ └──────────────────┘
```
**Variant 3: CQRS + Event Sourcing**
Write side stores events as source of truth. Read side projects events into queryable models.
### Synchronization Strategies
| Strategy | Latency | Complexity | Use When |
|----------|---------|------------|----------|
| Synchronous (same transaction) | Zero | Low | Same-DB variant, simple reads |
| Async via domain events | Seconds | Medium | Separate DBs, eventual consistency OK |
| Change Data Capture (CDC) | Sub-second | Medium | Existing DB, no event bus yet |
| Polling projectors | Seconds–minutes | Low | Batch reporting, analytics |
### Projection Patterns
```typescript
// Event handler that maintains a read model
class OrderProjection {
async handleOrderCreated(event: OrderCreatedEvent): Promise<void> {
await this.readDb.customerOrders.upsert({
customerId: event.customerId,
orderId: event.orderId,
status: 'created',
total: event.total,
itemCount: event.items.length,
createdAt: event.timestamp,
});
}
async handleOrderShipped(event: OrderShippedEvent): Promise<void> {
await this.readDb.customerOrders.update(
{ orderId: event.orderId },
{ status: 'shipped', shippedAt: event.timestamp }
);
}
// Rebuild: replay all events from the event store
async rebuild(): Promise<void> {
await this.readDb.customerOrders.truncate();
const events = await this.eventStore.readAll('Order');
for (const event of events) {
await this.handle(event);
}
}
}
```
---
## Event Sourcing
### Core Concepts
Event sourcing stores every state change as an immutable event. Current state is derived by replaying events.
```text
Event Store (append-only log):
[OrderCreated] → [ItemAdded] → [ItemAdded] → [OrderPaid] → [OrderShipped]
Current State = fold(initialState, events)
```
### When to Use Event Sourcing
| Good Fit | Poor Fit |
|----------|----------|
| Audit trail is a business requirement | Simple CRUD with no audit needs |
| Need to answer "how did we get here?" | High-throughput writes with no read-back |
| Complex domain with temporal queries | Team unfamiliar with event-driven design |
| Event-driven architecture already in place | Tight latency requirements on reads (without CQRS) |
| Regulatory compliance (financial, healthcare) | Schema changes are frequent and unpredictable |
### Event Store Design
```typescript
interface DomainEvent {
eventId: string; // UUID, globally unique
aggregateId: string; // Entity this event belongs to
aggregateType: string; // e.g., 'Order', 'Account'
eventType: string; // e.g., 'OrderCreated', 'ItemAdded'
version: number; // Monotonic per aggregate
timestamp: string; // ISO 8601
data: Record<string, unknown>; // Event payload
metadata: {
correlationId: string; // Trace through the system
causationId: string; // Which command caused this
userId?: string; // Who triggered it
};
}
```
**Storage options:**
| Store | Strengths | Trade-offs |
|-------|-----------|------------|
| EventStoreDB | Purpose-built, projections, subscriptions | Operational overhead of specialized DB |
| PostgreSQL (append-only table) | Familiar, ACID, good tooling | Must build projection infrastructure |
| DynamoDB / Cosmos DB | Managed, scalable | Ordering guarantees require careful key design |
| Kafka (as event store) | High throughput, retention | Not a true event store (no per-aggregate reads) |
### Snapshots
Snapshots prevent replaying thousands of events for every state reconstruction.
```typescript
class OrderAggregate {
private version = 0;
private state: OrderState;
private static SNAPSHOT_INTERVAL = 100;
async load(aggregateId: string): Promise<void> {
// 1. Load latest snapshot (if any)
const snapshot = await this.snapshotStore.getLatest(aggregateId);
if (snapshot) {
this.state = snapshot.state;
this.version = snapshot.version;
}
// 2. Replay events AFTER the snapshot
const events = await this.eventStore.readFrom(
aggregateId,
this.version + 1
);
for (const event of events) {
this.apply(event);
}
}
async save(newEvents: DomainEvent[]): Promise<void> {
await this.eventStore.append(this.aggregateId, newEvents, this.version);
this.version += newEvents.length;
// 3. Create snapshot periodically
if (this.version % OrderAggregate.SNAPSHOT_INTERVAL === 0) {
await this.snapshotStore.save({
aggregateId: this.aggregateId,
version: this.version,
state: this.state,
});
}
}
}
```
### Schema Evolution for Events
Events are immutable once stored. Handle schema changes with:
1. **Upcasting** — Transform old events to new shape at read time
2. **Versioned event types** — `OrderCreated_v2` with migration logic
3. **Weak schema** — Keep events loosely typed, validate at projection
```typescript
// Upcaster example
function upcast(event: DomainEvent): DomainEvent {
if (event.eventType === 'OrderCreated' && !event.data.currency) {
return { ...event, data: { ...event.data, currency: 'USD' } };
}
return event;
}
```
---
## Data Mesh
### Principles
Data mesh treats data as a product, owned by domain teams rather than a centralized data team.
| Principle | Description |
|-----------|-------------|
| Domain ownership | Each domain team owns, produces, and serves its data |
| Data as a product | Data has SLOs, documentation, discoverability, quality guarantees |
| Self-serve platform | Central platform provides tooling, not data pipelines |
| Federated governance | Standards are global, enforcement is local |
### Data Product Structure
```text
orders-domain/
├── data-products/
│ ├── order-events/ # Real-time event stream
│ │ ├── schema.avro
│ │ ├── slo.yaml # Freshness, completeness, accuracy
│ │ └── README.md # Discovery documentation
│ ├── order-metrics/ # Aggregated metrics
│ │ ├── schema.sql
│ │ └── slo.yaml
│ └── order-snapshots/ # Periodic full snapshots
│ ├── schema.parquet
│ └── slo.yaml
├── pipelines/ # Domain-owned transformations
└── contracts/ # Input/output contracts
```
### Data Product Quality Checklist
- [ ] Schema is versioned and published to a registry
- [ ] SLOs defined: freshness (<N minutes), completeness (>99%), accuracy
- [ ] Documentation: purpose, owner, schema, access patterns, known limitations
- [ ] Discoverability: registered in data catalog
- [ ] Access control: RBAC or attribute-based access
- [ ] Monitoring: alerting on SLO violations
### Federated Governance Model
| Global Standards (Central) | Local Standards (Domain) |
|---------------------------|-------------------------|
| Naming conventions | Schema design choices |
| Security and access policies | Transformation logic |
| Data classification (PII, etc.) | Refresh cadence |
| Interoperability formats | Internal storage format |
| Quality SLO minimums | Quality SLO targets above minimum |
---
## Polyglot Persistence
### Choosing the Right Database Per Service
| Use Case | Recommended Store | Why |
|----------|-------------------|-----|
| Transactional business data | PostgreSQL / MySQL | ACID, relational integrity, mature tooling |
| Document-oriented, flexible schema | MongoDB / DynamoDB | Schema flexibility, horizontal scaling |
| Full-text search | Elasticsearch / Typesense | Inverted index, relevance scoring |
| Caching, sessions, real-time | Redis / Valkey / Dragonfly | Sub-ms latency, pub/sub, TTL |
| Time-series metrics | TimescaleDB / InfluxDB | Time-partitioned storage, rollups |
| Graph relationships | Neo4j / Amazon Neptune | Traversal queries, relationship-first |
| Event streams | Kafka / Redpanda | Ordered append-only log, high throughput |
| Analytics / OLAP | ClickHouse / BigQuery / Snowflake | Columnar storage, fast aggregations |
| Binary/object storage | S3 / GCS / R2 | Cheap, durable, scalable blob storage |
### Decision Criteria
```text
Choosing a database:
├─ What is the access pattern?
│ ├─ Key-value lookups → Redis, DynamoDB
│ ├─ Complex joins → PostgreSQL
│ ├─ Free-text search → Elasticsearch
│ └─ Graph traversal → Neo4j
├─ What are the consistency needs?
│ ├─ Strong ACID → PostgreSQL, MySQL
│ └─ Eventual consistency OK → DynamoDB, Cassandra
├─ What is the write volume?
│ ├─ < 10K writes/sec → PostgreSQL handles it
│ └─ > 100K writes/sec → Kafka, Cassandra, DynamoDB
└─ What is the data lifecycle?
├─ Short-lived (cache) → Redis with TTL
├─ Append-only (logs) → Kafka, ClickHouse
└─ Long-lived (records) → PostgreSQL, S3
```
### Operational Considerations
| Factor | Impact |
|--------|--------|
| Backup and restore | Each DB has different tooling; automate all |
| Connection management | Pooling differs per store; budget connections |
| Monitoring | Unified dashboards across heterogeneous stores |
| Schema migrations | Coordinate across services during deploys |
| Team expertise | Each new DB adds cognitive load |
**Rule of thumb:** Start with one database (usually PostgreSQL). Add specialized stores only when PostgreSQL demonstrably cannot meet a specific access pattern or performance requirement.
---
## Saga Patterns
### Orchestration vs Choreography
| Aspect | Orchestration | Choreography |
|--------|---------------|--------------|
| Coordination | Central orchestrator directs steps | Each service listens and reacts |
| Coupling | Services coupled to orchestrator | Services coupled to event schema |
| Visibility | Single place to see the flow | Flow is distributed across services |
| Error handling | Orchestrator manages compensation | Each service manages its own rollback |
| Best for | Complex, multi-step business processes | Simple, loosely coupled workflows |
| Debugging | Easier — single flow view | Harder — distributed trace needed |
### Orchestration Example
```typescript
// Saga orchestrator — manages the multi-step order process
class OrderSaga {
async execute(orderId: string): Promise<SagaResult> {
const steps: SagaStep[] = [
{
name: 'reserveInventory',
execute: () => this.inventoryService.reserve(orderId),
compensate: () => this.inventoryService.release(orderId),
},
{
name: 'processPayment',
execute: () => this.paymentService.charge(orderId),
compensate: () => this.paymentService.refund(orderId),
},
{
name: 'shipOrder',
execute: () => this.shippingService.schedule(orderId),
compensate: () => this.shippingService.cancel(orderId),
},
];
const completedSteps: SagaStep[] = [];
for (const step of steps) {
try {
await step.execute();
completedSteps.push(step);
} catch (error) {
// Compensate in reverse order
for (const completed of completedSteps.reverse()) {
await completed.compensate();
}
return { status: 'failed', failedStep: step.name, error };
}
}
return { status: 'completed' };
}
}
```
### Choreography Example
```typescript
// Each service subscribes to events and publishes next steps
// Inventory service
class InventoryEventHandler {
async handleOrderCreated(event: OrderCreatedEvent): Promise<void> {
try {
await this.inventoryRepo.reserve(event.orderId, event.items);
await this.eventBus.publish(new InventoryReservedEvent(event.orderId));
} catch (error) {
await this.eventBus.publish(new InventoryReservationFailedEvent(
event.orderId, error.message
));
}
}
// Compensation
async handlePaymentFailed(event: PaymentFailedEvent): Promise<void> {
await this.inventoryRepo.release(event.orderId);
}
}
// Payment service
class PaymentEventHandler {
async handleInventoryReserved(event: InventoryReservedEvent): Promise<void> {
try {
await this.paymentProcessor.charge(event.orderId);
await this.eventBus.publish(new PaymentProcessedEvent(event.orderId));
} catch (error) {
await this.eventBus.publish(new PaymentFailedEvent(
event.orderId, error.message
));
}
}
}
```
### Saga State Machine
```text
OrderSaga States:
[Pending] → ReserveInventory
├─ Success → [InventoryReserved] → ProcessPayment
│ ├─ Success → [PaymentProcessed] → ScheduleShipping
│ │ ├─ Success → [Completed]
│ │ └─ Failure → RefundPayment → ReleaseInventory → [Failed]
│ └─ Failure → ReleaseInventory → [Failed]
└─ Failure → [Failed]
```
---
## Consistency Models
### Comparison
| Model | Guarantee | Latency | Use When |
|-------|-----------|---------|----------|
| Strong (linearizable) | Latest write always visible | Higher | Financial transactions, inventory counts |
| Sequential | Operations appear in agreed order | Medium | Distributed locks, leader election |
| Causal | Cause-and-effect order preserved | Medium | Chat messages, comment threads |
| Read-your-writes | Writer sees own writes immediately | Low | User profile updates, settings |
| Eventual | All replicas converge given time | Lowest | Social feeds, analytics, caches |
| Monotonic reads | Reader never sees older data after newer | Low | Dashboard displays, reporting |
### Read-Your-Writes Implementation
```typescript
// Pattern: Write to primary, read from primary for the writing user
class UserProfileService {
async updateProfile(userId: string, data: ProfileData): Promise<void> {
await this.primaryDb.users.update(userId, data);
// Set a "read-from-primary" marker with short TTL
await this.cache.set(`read-primary:${userId}`, '1', 'EX', 5);
}
async getProfile(userId: string, requestingUserId: string): Promise<Profile> {
// If the requesting user just wrote, read from primary
const readFromPrimary = await this.cache.get(`read-primary:${requestingUserId}`);
if (readFromPrimary || userId === requestingUserId) {
return this.primaryDb.users.findById(userId);
}
// Otherwise, read from replica (faster, eventually consistent)
return this.replicaDb.users.findById(userId);
}
}
```
### Conflict Resolution Strategies
| Strategy | Description | Use When |
|----------|-------------|----------|
| Last-write-wins (LWW) | Timestamp determines winner | Low-conflict data, user preferences |
| Merge (CRDTs) | Automatic conflict-free merge | Collaborative editing, counters |
| Application-level | Custom business logic resolves | Shopping carts, inventory |
| Manual | Flag conflict for human review | Legal documents, financial records |
---
## Anti-Patterns
### 1. Shared Database Across Services
```text
[FAIL] Service A and Service B both read/write to the same tables
→ Tight coupling, schema changes break both services, no independent deployment
[PASS] Each service owns its tables/schema; share data via APIs or events
```
### 2. Event Sourcing Everything
```text
[FAIL] Using event sourcing for simple CRUD entities (user profiles, settings)
→ Unnecessary complexity, painful schema evolution
[PASS] Use event sourcing for domains with complex state transitions, audit needs,
or temporal queries. Use plain CRUD elsewhere.
```
### 3. Distributed Transactions (2PC)
```text
[FAIL] Two-phase commit across microservices
→ Blocks on slowest participant, single point of failure, poor availability
[PASS] Use sagas with compensating transactions for cross-service consistency
```
### 4. CQRS Without Justification
```text
[FAIL] Applying CQRS to a simple CRUD app with uniform read/write patterns
→ Doubles the code surface, adds sync complexity for no benefit
[PASS] Start with a single model. Split only when read/write patterns diverge
or independent scaling is needed.
```
### 5. Ignoring Event Ordering
```text
[FAIL] Consuming events without partition keys → out-of-order processing
→ Inventory goes negative, payments double-charged
[PASS] Use aggregate ID as partition key. Process events per-partition in order.
Handle idempotency for at-least-once delivery.
```
### 6. Fat Events
```text
[FAIL] Events carrying the entire entity state (100+ fields)
→ Tight coupling, bandwidth waste, hard to evolve
[PASS] Events carry only what changed plus correlation IDs.
Consumers query for additional data if needed.
```
---
## Decision Framework
### Should You Use CQRS?
```text
1. Are read and write models structurally different? [Yes → +2, No → 0]
2. Is read:write ratio > 10:1? [Yes → +2, No → 0]
3. Do you need independent scaling of reads vs writes? [Yes → +2, No → 0]
4. Is the team experienced with event-driven systems? [Yes → +1, No → -1]
5. Is this a greenfield project? [Yes → +1, No → -1]
Score: 0–3 → Stick with single model
Score: 4–6 → Consider CQRS (same-DB variant first)
Score: 7+ → Strong candidate for full CQRS
```
### Should You Use Event Sourcing?
```text
1. Is audit trail a legal/business requirement? [Yes → +3, No → 0]
2. Do you need temporal queries ("state at time T")? [Yes → +2, No → 0]
3. Is the domain event-driven by nature? [Yes → +2, No → 0]
4. Can the team handle eventual consistency? [Yes → +1, No → -2]
5. Are events the natural language of the domain? [Yes → +1, No → 0]
Score: 0–2 → Use traditional CRUD + audit log
Score: 3–5 → Consider event sourcing for key aggregates
Score: 6+ → Strong candidate for event sourcing
```
### Orchestration vs Choreography
```text
Process involves:
├─ 2–3 services, simple flow → Choreography
├─ 4+ services, complex branching → Orchestration
├─ Need single view of process state → Orchestration
├─ Services owned by different teams → Choreography (less central control)
└─ Strict SLA on completion time → Orchestration (easier to monitor)
```
---
## Cross-References
- [modern-patterns.md](modern-patterns.md) — Architecture pattern overview including CQRS and event-driven
- [scalability-reliability-guide.md](scalability-reliability-guide.md) — CAP theorem, database scaling, caching strategies
- [../software-backend/references/database-patterns.md](../software-backend/references/database-patterns.md) — PostgreSQL-specific patterns, connection pooling, migrations
- [../software-backend/references/message-queues-background-jobs.md](../software-backend/references/message-queues-background-jobs.md) — BullMQ, Kafka, message broker comparison
- [../../assets/patterns/event-driven-template.md](../../assets/patterns/event-driven-template.md) — Event-driven architecture template with saga patterns
references/migration-modernization-guide.md
# Migration & Modernization Guide
Step-by-step patterns for migrating from monoliths to microservices, decomposing databases, and incrementally modernizing legacy systems. Use when planning major refactors, strangler fig migrations, or database decomposition strategies.
## Contents
- [Strangler Fig Pattern](#strangler-fig-pattern)
- [Database Decomposition](#database-decomposition)
- [Feature Flags for Migration](#feature-flags-for-migration)
- [Risk Assessment Framework](#risk-assessment-framework)
- [Parallel Running and Shadow Traffic](#parallel-running-and-shadow-traffic)
- [Migration Path: Monolith to Microservices](#migration-path-monolith-to-microservices)
- [Anti-Patterns](#anti-patterns)
- [Decision Framework](#decision-framework)
- [Cross-References](#cross-references)
---
## Strangler Fig Pattern
### Core Concept
Incrementally replace pieces of a legacy system by routing traffic to new implementations while the old system remains operational. Named after strangler fig trees that grow around a host tree.
```text
Phase 1: Identify Phase 2: Intercept Phase 3: Replace
┌────────────────┐ ┌────────────────┐ ┌────────────────┐
│ Monolith │ │ Facade/Proxy │ │ Facade/Proxy │
│ ┌──────────┐ │ │ │ │ │ │ │
│ │ Feature A │ │ │ ┌───┴───┐ │ │ ┌───┴───┐ │
│ │ Feature B │ │ │ │ │ │ │ │ │ │
│ │ Feature C │ │ │ Old A New B │ │ New A New B │
│ └──────────┘ │ │ Old C │ │ New C │
└────────────────┘ └────────────────┘ └────────────────┘
```
### Step-by-Step Implementation
**Step 1: Add a routing facade**
```typescript
// API Gateway or reverse proxy routes requests
// Start with 100% traffic to monolith
// nginx.conf or API gateway config
// location /api/orders {
// proxy_pass http://monolith:3000; # All traffic to monolith initially
// }
```
**Step 2: Identify extraction candidates**
Rank modules by these criteria:
| Criterion | Weight | Description |
|-----------|--------|-------------|
| Business value of independent deployment | High | Revenue impact, release frequency needs |
| Coupling to other modules | High | Fewer dependencies = easier extraction |
| Team ownership clarity | Medium | Clear owner = better extraction outcome |
| Data isolation feasibility | High | Shared tables make extraction hard |
| Change frequency | Medium | Frequently changed code benefits most |
| Performance isolation needs | Medium | One module's load affecting another |
**Step 3: Build the new service**
```typescript
// New service implements the SAME interface as the monolith feature
// This is critical — callers should not know about the migration
// New orders-service (standalone)
app.post('/api/orders', async (req, res) => {
// New implementation with same API contract
const order = await createOrder(req.body);
res.json(order);
});
```
**Step 4: Gradual traffic shift**
```yaml
# Canary routing: shift traffic incrementally
# Week 1: 5% to new service
# Week 2: 25% to new service
# Week 3: 50% to new service
# Week 4: 100% to new service (if metrics are green)
# Example: Istio VirtualService
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: orders
spec:
hosts:
- orders.example.com
http:
- route:
- destination:
host: orders-new
weight: 25
- destination:
host: monolith
weight: 75
```
**Step 5: Decommission the old code**
After 100% traffic runs on the new service with stable metrics for at least 2 weeks:
- [ ] Remove old code path from monolith
- [ ] Remove old database tables (after data migration verification)
- [ ] Update documentation and runbooks
- [ ] Remove feature flags related to migration
---
## Database Decomposition
### Shared Database Problem
```text
[FAIL] Multiple services sharing one database:
┌─────────┐ ┌─────────┐ ┌─────────┐
│Service A│ │Service B│ │Service C│
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└────────────┼────────────┘
│
┌───────▼───────┐
│ Shared DB │
│ (everything) │
└───────────────┘
Problems:
- Schema changes break multiple services
- No independent deployment
- Performance contention
- Unclear data ownership
```
### Decomposition Strategies
**Strategy 1: Database View Layer**
Intermediate step. Create views that simulate per-service schemas.
```sql
-- Service A sees only its data through a view
CREATE VIEW service_a_orders AS
SELECT id, customer_id, total, status, created_at
FROM orders
WHERE department = 'retail';
-- Service B sees different columns
CREATE VIEW service_b_orders AS
SELECT id, warehouse_id, shipping_status, tracking_number
FROM orders
WHERE shipping_status IS NOT NULL;
```
**Strategy 2: Schema-Per-Service (Same Instance)**
Each service gets its own schema within the same database instance.
```sql
-- Separate schemas, same PostgreSQL instance
CREATE SCHEMA orders_service;
CREATE SCHEMA inventory_service;
CREATE SCHEMA payments_service;
-- Each service's migrations target its own schema
-- Cross-schema access is explicitly forbidden (enforce via permissions)
REVOKE ALL ON SCHEMA orders_service FROM inventory_user;
```
**Strategy 3: Database-Per-Service (Full Separation)**
Each service has its own database instance.
```text
┌─────────┐ ┌─────────┐ ┌─────────┐
│Service A│ │Service B│ │Service C│
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
┌────▼────┐ ┌────▼────┐ ┌────▼────┐
│ DB A │ │ DB B │ │ DB C │
└─────────┘ └─────────┘ └─────────┘
```
### Data Synchronization During Decomposition
| Approach | Description | When |
|----------|-------------|------|
| Dual writes | Write to both old and new DB | Short transition periods only |
| CDC (Change Data Capture) | Stream changes from old to new | Gradual migration, minimal code changes |
| ETL batch sync | Periodic bulk sync | Non-real-time data, analytics |
| API calls | New service fetches data via API | Loosely coupled, eventually consistent |
**CDC Example with Debezium:**
```json
{
"name": "orders-connector",
"config": {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"database.hostname": "monolith-db",
"database.port": "5432",
"database.dbname": "monolith",
"table.include.list": "public.orders,public.order_items",
"topic.prefix": "migration",
"slot.name": "orders_migration"
}
}
```
### Join Elimination Strategies
When decomposing a shared database, you lose cross-table joins. Solutions:
| Pattern | Use When | Trade-off |
|---------|----------|-----------|
| API composition | Low-frequency queries, few services | Latency from multiple API calls |
| Data denormalization | Read-heavy, stale data OK | Storage duplication, sync complexity |
| Event-driven sync | Real-time needs, owned data changes | Eventual consistency |
| Materialized views in read DB | Complex reporting queries | Additional infrastructure |
---
## Feature Flags for Migration
### Migration-Specific Flag Patterns
```typescript
// Flag: route traffic to new vs old implementation
const flags = {
'orders-service-v2': {
type: 'percentage',
value: 25, // 25% of traffic to new service
metadata: {
migration: 'orders-extraction',
startDate: '2026-01-15',
targetCompletion: '2026-03-01',
rollbackPlan: 'Set to 0%, revert proxy config',
},
},
};
// Usage in routing layer
async function handleOrderRequest(req: Request): Promise<Response> {
if (featureFlags.isEnabled('orders-service-v2', { userId: req.userId })) {
return newOrdersService.handle(req);
}
return monolith.handle(req);
}
```
### Flag Lifecycle for Migrations
```text
Phase 1: Create flag (default: OFF)
Phase 2: Enable for internal users (testing)
Phase 3: Enable for 5% of traffic (canary)
Phase 4: Ramp to 25%, 50%, 75%, 100%
Phase 5: Remove flag + old code path (cleanup)
IMPORTANT: Set a cleanup deadline. Stale flags are tech debt.
```
### Comparison Read Pattern
```typescript
// During migration: read from BOTH, compare results, return old
async function getOrder(orderId: string): Promise<Order> {
const oldResult = await monolith.getOrder(orderId);
if (featureFlags.isEnabled('orders-comparison-reads')) {
try {
const newResult = await newService.getOrder(orderId);
if (!deepEqual(oldResult, newResult)) {
logger.warn('Migration mismatch', {
orderId,
diff: generateDiff(oldResult, newResult),
});
metrics.increment('migration.comparison.mismatch');
} else {
metrics.increment('migration.comparison.match');
}
} catch (error) {
metrics.increment('migration.comparison.error');
}
}
return oldResult; // Always return the trusted source during migration
}
```
---
## Risk Assessment Framework
### Migration Risk Matrix
| Risk Factor | Low Risk | Medium Risk | High Risk |
|-------------|----------|-------------|-----------|
| Data coupling | No shared tables | Shared lookup tables | Shared mutable tables with joins |
| Traffic volume | <100 req/s | 100–1000 req/s | >1000 req/s |
| Consistency requirements | Eventual OK | Read-your-writes needed | Strong ACID required |
| Team experience | Done migrations before | Some distributed systems experience | First microservice extraction |
| Rollback complexity | Stateless, easy revert | Some state to reconcile | Data divergence makes rollback hard |
| Business criticality | Internal tool | Customer-facing, non-revenue | Payment, checkout, auth |
### Risk Mitigation Checklist
- [ ] **Runbook** written for rollback (with specific steps, not "revert changes")
- [ ] **Monitoring** dashboards ready before migration starts
- [ ] **Alerts** configured for error rate increase, latency spikes
- [ ] **Data reconciliation** script ready to compare old vs new
- [ ] **Communication plan** for stakeholders (downtime windows, expected behavior changes)
- [ ] **Rollback tested** in staging environment
- [ ] **Feature flag** wired for instant traffic revert
- [ ] **Incremental rollout** plan with go/no-go criteria at each stage
### Go/No-Go Criteria Per Phase
| Metric | Green (Go) | Yellow (Pause) | Red (Rollback) |
|--------|------------|----------------|-----------------|
| Error rate | <0.1% increase | 0.1–1% increase | >1% increase |
| P95 latency | <10% increase | 10–50% increase | >50% increase |
| Data mismatches | <0.01% | 0.01–0.1% | >0.1% |
| Business metric (conversion, etc.) | No change | <2% drop | >2% drop |
---
## Parallel Running and Shadow Traffic
### Shadow Traffic Pattern
Route a copy of production traffic to the new service without affecting users.
```text
┌────────┐ ┌──────────────┐ ┌────────────┐
│ Client │────▶│ Proxy/LB │────▶│ Monolith │──▶ Response to client
└────────┘ └──────┬───────┘ └────────────┘
│ (async copy)
└──────────────▶┌─────────────┐
│ New Service │──▶ Response discarded
└─────────────┘ (logged + compared)
```
### Implementation
```typescript
// Shadow traffic middleware
async function shadowTraffic(req: Request, res: Response, next: NextFunction) {
// 1. Process the real request normally
next();
// 2. Asynchronously send a copy to the new service (fire-and-forget)
if (featureFlags.isEnabled('shadow-traffic-orders')) {
setImmediate(async () => {
try {
const shadowStart = Date.now();
const shadowResponse = await newService.mirror(req);
const shadowLatency = Date.now() - shadowStart;
metrics.histogram('shadow.latency', shadowLatency);
// Compare responses (logged, not returned to client)
if (res.locals.responseBody) {
const match = deepEqual(res.locals.responseBody, shadowResponse);
metrics.increment(match ? 'shadow.match' : 'shadow.mismatch');
}
} catch (error) {
metrics.increment('shadow.error');
// Failures in shadow traffic NEVER affect the real response
}
});
}
}
```
### Parallel Running Checklist
- [ ] Shadow traffic does NOT mutate production data
- [ ] Shadow requests are clearly marked (header: `X-Shadow: true`)
- [ ] New service has separate database/storage from production
- [ ] Comparison results are aggregated into dashboards
- [ ] Shadow traffic can be turned off instantly via feature flag
- [ ] Load from shadow traffic is accounted for in capacity planning
---
## Migration Path: Monolith to Microservices
### Recommended Progression
```text
Stage 1: Modular Monolith
└─ Enforce module boundaries within the monolith
└─ Define clear APIs between modules
└─ Separate schemas per module (same DB)
└─ Duration: 2-6 months
Stage 2: Extract First Service
└─ Pick the module with least coupling
└─ Use strangler fig + shadow traffic
└─ Establish service infrastructure (CI/CD, monitoring, service mesh)
└─ Duration: 1-3 months
Stage 3: Extract Core Services
└─ Extract 2-4 more services in parallel (different teams)
└─ Introduce async messaging for cross-service communication
└─ Decompose database per service
└─ Duration: 3-9 months
Stage 4: Steady State
└─ New features built as services by default
└─ Remaining monolith handles only legacy features
└─ Monolith shrinks over time (or stays as a module)
```
### Case Study Pattern: E-Commerce Migration
| Phase | Extract | Why First | Risk Level |
|-------|---------|-----------|------------|
| 1 | Notifications | No writes to core data, read-only | Low |
| 2 | Search/Catalog | Heavy reads, independent index | Low-Medium |
| 3 | Inventory | Clear bounded context, event-driven | Medium |
| 4 | Orders | Core domain, complex state machine | High |
| 5 | Payments | Regulatory, compliance needs isolation | High |
| 6 | User/Auth | Shared dependency, extract last | Very High |
**Key principle:** Extract the easiest services first to build team confidence and infrastructure. Save the hardest (most coupled, most critical) for last.
---
## Migration Plan Binding
Migration plans are only executable when architecture, controls, and rollout mechanics are bound together in a single coherent plan:
- **Target-state decisions** must be connected to pilot-provider rollout sequencing, degraded modes, cutover controls, readiness checks, and decommission sequencing.
- Architecture proposals alone are not enough — the plan needs explicit stages, open-gap tracking, and clear acknowledgement of what is still unresolved.
- Roadmaps, trackers, and architecture docs must stay synchronized in the same delivery cycle. When they drift, they become contradictory instructions for the next implementation session.
- Distinguish verified current state, selected target state, and unresolved gaps in all migration documentation. Flattening them together makes docs unsafe to trust.
### Migration Plan Checklist
- [ ] Target architecture documented with explicit boundaries
- [ ] Pilot-provider or canary rollout sequence defined
- [ ] Degraded mode behavior specified (what happens when new path fails)
- [ ] Cutover controls identified (feature flags, traffic routing, kill switches)
- [ ] Readiness checks listed (what must be true before each phase advances)
- [ ] Decommission sequencing planned (what gets removed, when, and by whom)
- [ ] Open gaps explicitly tracked with owners
- [ ] Roadmap, architecture doc, and gap tracker synchronized
---
## Anti-Patterns
### 1. Big Bang Rewrite
```text
[FAIL] "Let's rewrite the entire monolith as microservices over 6 months"
→ Delayed value delivery, high risk, second-system effect, team burnout
[PASS] Incremental extraction with strangler fig. Ship value every 2-4 weeks.
Each extraction is independently valuable and rollback-safe.
```
### 2. Premature Decomposition
```text
[FAIL] Splitting into 20 microservices before understanding domain boundaries
→ Distributed monolith, wrong service boundaries, expensive to fix
[PASS] Start with a modular monolith. Let boundaries emerge from real usage.
Extract services only when you have clear, stable bounded contexts.
```
### 3. Shared Database Migration
```text
[FAIL] Extracting services but keeping the shared database
→ Services are coupled at the data layer, no independent deployment
[PASS] Database decomposition is part of the migration plan, not an afterthought.
Use the view layer or schema-per-service as intermediate steps.
```
### 4. No Rollback Plan
```text
[FAIL] "We'll figure out rollback if something goes wrong"
→ Panic during incidents, data loss, extended outages
[PASS] Write the rollback runbook BEFORE starting migration.
Test rollback in staging. Include data reconciliation steps.
```
### 5. Ignoring Data Migration
```text
[FAIL] Extracting the service but leaving historical data in the monolith
→ Split brain, queries return partial results, reporting breaks
[PASS] Plan data migration as a first-class concern. Include:
- Historical data transfer
- Data format transformation
- Validation and reconciliation
- Cutover strategy
```
### 6. Migrating Without Observability
```text
[FAIL] Extracting services without distributed tracing or unified logging
→ Impossible to debug issues across service boundaries
[PASS] Set up observability BEFORE the first extraction:
- Distributed tracing (OpenTelemetry)
- Centralized logging with correlation IDs
- Service-level dashboards and alerts
```
---
## Decision Framework
### Should You Migrate at All?
```text
1. Are deployment bottlenecks hurting business velocity? [Yes → +2, No → 0]
2. Do different parts of the system need different scaling? [Yes → +2, No → 0]
3. Are multiple teams blocked by monolith coupling? [Yes → +2, No → 0]
4. Is the monolith technically healthy (tests, CI, docs)? [Yes → +1, No → -1]
5. Does the team have distributed systems experience? [Yes → +1, No → -2]
6. Is there executive buy-in for a multi-quarter effort? [Yes → +1, No → -2]
Score: 0–3 → Improve the monolith (modularize, add tests, fix CI)
Score: 4–6 → Start with modular monolith, plan first extraction
Score: 7+ → Begin migration with strangler fig approach
```
### Extraction Order Prioritization
Score each module (1-5 per criterion), extract in descending total order:
| Module | Coupling (inverse) | Change Frequency | Business Value | Team Readiness | Total |
|--------|-------------------|-----------------|----------------|----------------|-------|
| Module A | ? | ? | ? | ? | ? |
---
## Cross-References
- [modern-patterns.md](modern-patterns.md) — Architecture patterns overview (microservices, modular monolith)
- [data-architecture-patterns.md](data-architecture-patterns.md) — CQRS, event sourcing, saga patterns for distributed data
- [scalability-reliability-guide.md](scalability-reliability-guide.md) — Scaling strategies post-migration
- [api-gateway-service-mesh.md](api-gateway-service-mesh.md) — Service mesh and gateway patterns for microservices
- [../software-backend/SKILL.md](../software-backend/SKILL.md) — Service-level implementation patterns
- [../../ops-devops-platform/SKILL.md](../../ops-devops-platform/SKILL.md) — CI/CD and deployment for microservices
references/modern-patterns.md
# Modern Software Architecture Patterns
Comprehensive guide to contemporary architecture patterns based on industry trends and practices.
## Top Architecture Patterns
### 1. Microservices Architecture
**When to use**:
- Multiple independent teams
- Need independent deployment and scaling
- Different technologies for different services
- Clear bounded contexts
**Structure**:
```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Service │ │ Service │ │ Service │
│ A │ │ B │ │ C │
│ (Node.js) │ │ (Python) │ │ (Go) │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
└─────────────────┴─────────────────┘
│
┌────────▼────────┐
│ API Gateway │
│ (Kong/Nginx) │
└─────────────────┘
```
**Best practices**:
- **Service discovery**: Use Consul, Eureka, or Kubernetes DNS
- **API Gateway**: Single entry point, authentication, rate limiting
- **Communication**: REST for synchronous, message queues for async
- **Data**: Each service owns its database (no shared DB)
- **Deployment**: Containerization (Docker) + orchestration (Kubernetes)
**Challenges**:
- Distributed system complexity
- Network latency and failures
- Data consistency across services
- Testing and debugging
- Operational overhead
**Mitigation**:
```yaml
# Service mesh (Istio/Linkerd) for:
- Service-to-service authentication
- Load balancing
- Circuit breaking
- Distributed tracing
- Metrics collection
```
### 2. Event-Driven Architecture (EDA)
**When to use**:
- Real-time data processing
- Asynchronous workflows
- Decoupled systems
- High scalability requirements
**Structure**:
```
┌──────────┐ ┌──────────────┐ ┌──────────┐
│ Producer │──────▶│ Event Broker │──────▶│ Consumer │
│ │ │ (Kafka/RabbitMQ) │ │
└──────────┘ └──────────────┘ └──────────┘
│
├──────▶ Consumer 2
└──────▶ Consumer 3
```
**Event patterns**:
**Event Notification**:
```json
{
"eventType": "OrderPlaced",
"orderId": "12345",
"timestamp": "2023-06-15T10:30:00Z"
}
```
**Event-Carried State Transfer**:
```json
{
"eventType": "OrderPlaced",
"orderId": "12345",
"customer": {"id": "C123", "name": "John"},
"items": [{"id": "P456", "qty": 2}],
"total": 99.99,
"timestamp": "2023-06-15T10:30:00Z"
}
```
**Event Sourcing**:
```json
[
{"event": "OrderCreated", "orderId": "12345", "seq": 1},
{"event": "ItemAdded", "orderId": "12345", "itemId": "P456", "seq": 2},
{"event": "OrderPaid", "orderId": "12345", "amount": 99.99, "seq": 3}
]
```
**Best practices**:
- **Idempotency**: Handle duplicate events gracefully
- **Schema evolution**: Use versioned event schemas
- **Error handling**: Dead letter queues for failed events
- **Monitoring**: Track event lag and processing times
- **Ordering**: Use partition keys for ordered processing
**Tools**:
- Apache Kafka - High-throughput distributed streaming
- RabbitMQ - Flexible message broker
- AWS EventBridge - Serverless event bus
- Google Pub/Sub - Global messaging service
### 3. Serverless Architecture
**When to use**:
- Variable/unpredictable load
- Event-driven workloads
- Rapid development and deployment
- Cost optimization (pay per use)
**Structure**:
```
┌─────────┐ ┌──────────────┐ ┌─────────┐
│ Event │─────▶│ Function │─────▶│ Store │
│ Source │ │ (Lambda/CF) │ │ (DynamoDB)
└─────────┘ └──────────────┘ └─────────┘
Event Sources:
- API Gateway (HTTP)
- S3 (file upload)
- DynamoDB Streams
- EventBridge (scheduled)
- SQS/SNS (messaging)
```
**Best practices**:
- **Cold start mitigation**: Keep functions warm with provisioned concurrency
- **Stateless design**: Use external state stores (Redis, DynamoDB)
- **Granular functions**: Single responsibility (≤300 LOC)
- **Resource limits**: Configure memory and timeout appropriately
- **Observability**: Use X-Ray, CloudWatch, or DataDog
**Example - AWS Lambda**:
```javascript
// Optimized function structure
export const handler = async (event) => {
// Input validation
const { userId, action } = JSON.parse(event.body);
// Business logic
const result = await processUserAction(userId, action);
// Response
return {
statusCode: 200,
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(result)
};
};
// Keep external connections alive (outside handler)
const db = initDatabase();
```
**Cost optimization**:
- Use ARM-based functions (Graviton) - 20% cheaper
- Right-size memory allocation
- Use step functions for orchestration
- Implement caching to reduce invocations
### 4. Layered (N-Tier) Architecture
**When to use**:
- Monolithic applications
- Clear separation of concerns needed
- Team familiar with traditional patterns
- Moderate complexity
**Classic layers**:
```
┌──────────────────────────┐
│ Presentation Layer │ ← Controllers, Views, API endpoints
├──────────────────────────┤
│ Business Logic Layer │ ← Services, Domain models
├──────────────────────────┤
│ Data Access Layer │ ← Repositories, ORM
├──────────────────────────┤
│ Database Layer │ ← PostgreSQL, MongoDB
└──────────────────────────┘
```
**Dependency rule**: Outer layers depend on inner layers only
**Example structure**:
```
src/
├── controllers/ # HTTP request handlers
│ └── userController.js
├── services/ # Business logic
│ └── userService.js
├── repositories/ # Data access
│ └── userRepository.js
├── models/ # Domain models
│ └── user.js
└── database/ # DB configuration
└── connection.js
```
**Best practices**:
- **Dependency injection**: Pass dependencies, don't hardcode
- **Interface segregation**: Define clear contracts between layers
- **Error propagation**: Handle errors at appropriate layer
- **Transaction management**: Handle at service layer
### 5. Hexagonal Architecture (Ports & Adapters)
**When to use**:
- Need high testability
- Multiple interfaces (REST, GraphQL, CLI)
- Business logic must be technology-agnostic
- Long-term maintainability priority
**Structure**:
```
┌─────────────────────────┐
│ Application Core │
│ (Business Logic) │
│ │
│ ┌─────────────────┐ │
│ │ Domain Model │ │
│ └─────────────────┘ │
└────────┬──────┬──────────┘
│ │
┌────────────┘ └────────────┐
│ │
┌───▼──────┐ ┌─────▼────┐
│ Ports │ │ Ports │
│ (Input) │ │ (Output) │
└───┬──────┘ └─────┬────┘
│ │
┌───▼──────────┐ ┌────────▼─────┐
│ Adapters │ │ Adapters │
│ REST, GraphQL│ │ DB, External │
└──────────────┘ └──────────────┘
```
**Implementation**:
```typescript
// Core domain (technology-agnostic)
interface UserRepository {
findById(id: string): Promise<User>;
save(user: User): Promise<void>;
}
class UserService {
constructor(private userRepo: UserRepository) {}
async activateUser(id: string): Promise<User> {
const user = await this.userRepo.findById(id);
user.activate(); // Business logic
await this.userRepo.save(user);
return user;
}
}
// Adapters (technology-specific)
class PostgresUserRepository implements UserRepository {
async findById(id: string): Promise<User> {
const row = await db.query('SELECT * FROM users WHERE id = $1', [id]);
return User.fromDatabase(row);
}
async save(user: User): Promise<void> {
await db.query('UPDATE users SET ...', user.toDatabase());
}
}
class RestAdapter {
constructor(private userService: UserService) {}
async handleActivateUser(req, res) {
const user = await this.userService.activateUser(req.params.id);
res.json(user);
}
}
```
### 6. CQRS (Command Query Responsibility Segregation)
**When to use**:
- Read and write patterns are very different
- High read:write ratio
- Complex reporting requirements
- Need independent scaling of reads and writes
**Structure**:
```
┌─────────────┐
│ Command │
│ (Write) │
└──────┬──────┘
│
┌─────────▼──────────┐
│ Write Database │
│ (Normalized) │
└─────────┬──────────┘
│ (sync/async)
┌─────────▼──────────┐
│ Read Database │
│ (Denormalized) │
└─────────┬──────────┘
│
┌──────▼──────┐
│ Query │
│ (Read) │
└─────────────┘
```
**Example**:
```typescript
// Command (Write)
class CreateOrderCommand {
constructor(
public customerId: string,
public items: OrderItem[]
) {}
}
class OrderCommandHandler {
async handle(cmd: CreateOrderCommand) {
const order = new Order(cmd.customerId, cmd.items);
await writeDb.orders.save(order);
// Publish event for read model update
await eventBus.publish(new OrderCreatedEvent(order));
}
}
// Query (Read)
class GetCustomerOrdersQuery {
constructor(public customerId: string) {}
}
class OrderQueryHandler {
async handle(query: GetCustomerOrdersQuery) {
// Read from optimized read model
return await readDb.customerOrders.find({
customerId: query.customerId
});
}
}
// Event handler to sync read model
class OrderCreatedEventHandler {
async handle(event: OrderCreatedEvent) {
// Update denormalized read model
await readDb.customerOrders.insert({
customerId: event.customerId,
orderId: event.orderId,
total: event.total,
// ... optimized for reads
});
}
}
```
### 7. Modular Monolith
**When to use**:
- Team size 5-30 developers
- Want clear boundaries without microservices overhead
- Need faster development than microservices
- Shared domain concepts across modules
**Structure**:
```
monolith/
├── modules/
│ ├── orders/
│ │ ├── api/ # Public interface
│ │ ├── domain/ # Business logic (private)
│ │ └── infrastructure/ # DB, external services (private)
│ ├── payments/
│ │ ├── api/
│ │ ├── domain/
│ │ └── infrastructure/
│ └── shipping/
│ ├── api/
│ ├── domain/
│ └── infrastructure/
└── shared/
├── database/
└── messaging/
```
**Module boundaries**:
```typescript
// orders/api/OrdersModule.ts (public API)
export class OrdersModule {
static async createOrder(data: CreateOrderDTO): Promise<Order> {
// Implementation hidden
}
static async getOrder(id: string): Promise<Order> {
// Implementation hidden
}
}
// payments/PaymentsService.ts
import { OrdersModule } from '../orders/api/OrdersModule';
class PaymentsService {
async processPayment(orderId: string) {
// Use public API only, no direct access to orders internals
const order = await OrdersModule.getOrder(orderId);
// ...
}
}
```
**Advantages over microservices**:
- Single deployment (simpler CI/CD)
- No network latency between modules
- Shared transactions possible
- Easier refactoring (can extract to microservice later)
### 8. Micro-Frontend Architecture
**When to use**:
- Multiple teams working on different features
- Different technology stacks for different parts
- Independent deployment of UI components
- Large-scale front-end applications
**Approaches**:
**A) Server-side composition (SSR)**:
```nginx
# Nginx routes different paths to different apps
location /products {
proxy_pass http://products-frontend:3000;
}
location /checkout {
proxy_pass http://checkout-frontend:3001;
}
```
**B) Build-time composition (Module Federation)**:
```javascript
// Webpack Module Federation
module.exports = {
plugins: [
new ModuleFederationPlugin({
name: 'products',
filename: 'remoteEntry.js',
exposes: {
'./ProductList': './src/components/ProductList'
},
shared: ['react', 'react-dom']
})
]
};
// Host app imports remote component
const ProductList = React.lazy(() => import('products/ProductList'));
```
**C) Runtime composition (Single-SPA)**:
```javascript
import { registerApplication, start } from 'single-spa';
registerApplication({
name: 'products',
app: () => import('./products/main.js'),
activeWhen: location => location.pathname.startsWith('/products')
});
registerApplication({
name: 'checkout',
app: () => import('./checkout/main.js'),
activeWhen: '/checkout'
});
start();
```
### 9. Service Mesh Architecture
**When to use**:
- Microservices at scale (10+ services)
- Need advanced traffic management
- Security and observability are critical
- Polyglot microservices
**Structure**:
```
Service A ──▶ Sidecar Proxy (Envoy)
│ ──▶ Sidecar Proxy ──▶ Service B
└─ Control Plane (Istio)
│
├─ Traffic management
├─ Security (mTLS)
└─ Observability
```
**Features**:
- **Traffic management**: Load balancing, circuit breaking, retries
- **Security**: Mutual TLS, authorization policies
- **Observability**: Distributed tracing, metrics, logging
**Example - Istio**:
```yaml
# Virtual Service (traffic routing)
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- match:
- headers:
user-agent:
regex: '.*Chrome.*'
route:
- destination:
host: reviews
subset: v2
- route:
- destination:
host: reviews
subset: v1
# Circuit breaker
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews
trafficPolicy:
connectionPool:
tcp:
maxConnections: 100
http:
http1MaxPendingRequests: 1
maxRequestsPerConnection: 2
outlierDetection:
consecutive5xxErrors: 5
interval: 30s
baseEjectionTime: 30s
```
### 10. Edge Computing Architecture
**When to use**:
- Need ultra-low latency
- IoT applications
- Content delivery
- Real-time processing
**Structure**:
```
┌─────────────────────────────────────────┐
│ Cloud (Central) │
│ - Data aggregation │
│ - ML model training │
│ - Long-term storage │
└────────────┬────────────────────────────┘
│
┌────────┴────────┐
│ │
┌───▼──────┐ ┌─────▼────┐
│ Edge │ │ Edge │
│ Node 1 │ │ Node 2 │
│ - Process│ │ - Process│
│ - Cache │ │ - Cache │
│ - Filter │ │ - Filter │
└───┬──────┘ └─────┬────┘
│ │
┌───▼──┐ ┌───▼──┐
│ IoT │ │ IoT │
│Device│ │Device│
└──────┘ └──────┘
```
**Use cases**:
- CDN edge workers (Cloudflare Workers, Lambda@Edge)
- Smart city sensors
- Industrial IoT
- Autonomous vehicles
**Example - Cloudflare Worker**:
```javascript
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request));
});
async function handleRequest(request) {
// Process at edge (near user)
const cache = caches.default;
let response = await cache.match(request);
if (!response) {
// Fetch from origin if not cached
response = await fetch(request);
// Cache at edge
event.waitUntil(cache.put(request, response.clone()));
}
return response;
}
```
## Architecture Selection Decision Tree
```
Start: What are you building?
├─ Simple CRUD app
│ └─ Use: Layered Architecture
│
├─ Need independent team scaling?
│ ├─ Yes → Need independent deployments?
│ │ ├─ Yes → Use: Microservices
│ │ └─ No → Use: Modular Monolith
│ └─ No → Use: Layered or Hexagonal
│
├─ Event-driven requirements?
│ ├─ Primary pattern → Use: Event-Driven Architecture
│ └─ Secondary pattern → Add messaging to chosen architecture
│
├─ Unpredictable/variable load?
│ └─ Use: Serverless
│
├─ Different read/write patterns?
│ └─ Use: CQRS + Event Sourcing
│
└─ Multiple UI teams?
└─ Use: Micro-Frontends
```
## Anti-Patterns to Avoid
### 1. Distributed Monolith
Microservices that are tightly coupled:
```
[FAIL] Service A calls Service B, which calls Service C, which calls Service A
[OK] Use message queues or events to decouple
```
### 2. God Service
One service that does everything:
```
[FAIL] UserOrderPaymentShippingService
[OK] UserService, OrderService, PaymentService, ShippingService
```
### 3. Anemic Domain Model
Models with no behavior, just getters/setters:
```typescript
[FAIL] // Anemic
class Order {
items: OrderItem[];
getItems() { return this.items; }
setItems(items) { this.items = items; }
}
[OK] // Rich domain model
class Order {
private items: OrderItem[];
addItem(item: OrderItem) {
this.validateItem(item);
this.items.push(item);
this.recalculateTotal();
}
canBeCancelled(): boolean {
return this.status === 'pending' && !this.isPaid;
}
}
```
### 4. Chatty APIs
Too many network calls:
```
[FAIL] GET /users/1, GET /users/1/orders, GET /orders/1/items
[OK] GET /users/1?include=orders.items
```
## Resources
- Martin Fowler - Architecture Patterns
- Microsoft Azure - Architecture Center
- AWS - Well-Architected Framework
- Google Cloud - Architecture Framework
- Microservices.io - Pattern catalog
references/operational-playbook.md
## Core Architecture Questions
Use these questions to frame any design discussion:
- **Domain:**
- What problem is this system solving?
- What are the core domain concepts and invariants?
- **Boundaries:**
- How should responsibilities be split across services or modules?
- What must stay together for strong consistency?
- **Data:**
- What data is stored, where, and in what shape?
- What are the consistency and durability requirements?
- **Workload:**
- What are the expected read/write patterns?
- How does traffic scale over time (burst vs steady)?
- **Failure:**
- What happens when dependencies fail or degrade?
- What are the recovery paths and fallbacks?
---
## Pattern: Layered Architecture
Use when the system is not yet highly distributed, or when clarity and separation of concerns are more important than aggressive scalability.
Typical layering:
- Presentation/API layer: HTTP or messaging interfaces, authentication, request validation.
- Application/service layer: orchestration, use cases, workflows.
- Domain layer: core business logic and invariants.
- Infrastructure layer: databases, queues, external services, file storage.
Checklist:
- Clear direction of dependencies (outer layers depend on inner, not vice versa).
- Domain code does not depend on transport or storage details.
- Cross-cutting concerns (logging, metrics, security) handled via composition, not duplication.
---
## Pattern: Service Decomposition
Use when deciding between monolith, modular monolith, and microservices.
- **Monolith:**
- Use when the team is small and the problem space is still evolving
- Focus on modular boundaries inside a single deployable
- **Modular monolith:**
- Use when you need strong internal boundaries and clear ownership but still want a single deployment unit
- **Microservices:**
- Use when you have clear bounded contexts, strong ownership boundaries, and operational maturity
**Decomposition heuristics:**
- Group code by domain boundary, not by technical layer only
- Avoid splitting entities that must be updated transactionally
- Prefer a small number of well-designed services over many tiny ones
---
## Pattern: Data and Consistency
Use when designing storage and consistency behavior.
- Choose the primary source of truth for each piece of data
- **Decide consistency model:**
- **Strong:** transactions and immediate consistency; fewer consumers, higher coupling
- **Eventual:** asynchronous updates; requires idempotency and reconciliation
- **Avoid writing to multiple sources in a single request without a clear strategy:**
- Use a "single writer" or orchestrator service
- Use outbox patterns for publishing events reliably
**Checklist:**
- Data ownership is clear for each service
- Failure modes for partial writes are understood and handled
- Migrations and schema evolution have a plan (backwards compatibility where needed)
---
## Pattern: Request-Driven vs Event-Driven
Use this pattern to decide between synchronous and asynchronous flows.
- **Request-driven (synchronous):**
- Good for user-facing APIs needing immediate feedback
- Latency and availability of dependencies directly affect the caller
- **Event-driven (asynchronous):**
- Good for decoupling producers and consumers
- Suited to background work, aggregations, notifications
**Guidelines:**
- Keep request paths shallow for user interactions; offload heavy work via events or queues
- In event-driven flows, design idempotent handlers and clear retry behaviors
---
## Pattern: Security Architecture
**Use when:** Designing secure system architectures with proper security layers and controls.
**IMPORTANT:** For comprehensive security patterns, see [../software-security-appsec/SKILL.md](../../software-security-appsec/SKILL.md) which covers:
- Defense in depth and security boundaries
- Zero trust architecture
- Threat modeling (STRIDE framework)
- Authentication & Authorization architecture
- Data protection at rest and in transit
- Secure design principles
**Architecture-Specific Security Patterns:**
- **Security Boundaries**: Define trust boundaries between layers (client → API → services → data)
- **Defense in Depth**: Layer security controls (authentication → authorization → validation → encryption)
- **Least Privilege**: Each service should have minimum necessary permissions
- **Fail Securely**: Systems should default to deny, not allow on errors
- **API Gateway Pattern**: Centralized authentication, rate limiting, logging
- **Service Mesh**: mTLS between services, zero trust networking
- **Secret Management**: Centralized secret storage (AWS Secrets Manager, Vault)
**Quick Example - Security Layers:**
```text
Client
↓ HTTPS + JWT
API Gateway (Auth, Rate Limit, Logging)
↓ Service-to-Service Auth
Backend Services (Authorization, Validation)
↓ Encrypted Connection
Database (Encrypted at Rest, Row-Level Security)
```
**Checklist:**
- Authentication at entry points
- Authorization on every service call
- Input validation at boundaries
- Encryption in transit (TLS)
- Encryption at rest for sensitive data
- Centralized logging for security events
- Rate limiting and DDoS protection
- Regular security audits and penetration testing
---
## External Resources
See [data/sources.json](../data/sources.json) for 42 curated references on:
- Architecture pattern catalogs (AWS, Azure, Google Cloud, Martin Fowler, microservices.io)
- Distributed systems and scalability guides (ByteByteGo, System Design)
- ADR templates and best practices (GitHub ADR org, AWS, Microsoft)
- Scalability and reliability (Google SRE Book, CAP theorem, resilience patterns)
- Event-driven architecture (AWS, Martin Fowler, CQRS, Saga patterns)
- Observability (OpenTelemetry, distributed tracing, SLI/SLO/SLA)
- API design (REST, GraphQL, gRPC, API Gateway patterns)
- Security architecture (OWASP, Zero Trust, Service Mesh security)
- Essential books (Designing Data-Intensive Applications, Building Microservices, Release It!)
references/scalability-reliability-guide.md
# Scalability & Reliability Architecture Guide
## Core Principles
### CAP Theorem
You can only achieve 2 out of 3:
- **C**onsistency - All nodes see the same data
- **A**vailability - Every request receives a response
- **P**artition tolerance - System continues despite network failures
**Practical choices**:
- **CP**: Traditional databases (PostgreSQL, MongoDB with strong consistency)
- **AP**: NoSQL databases (Cassandra, DynamoDB), eventual consistency
- **Reality**: Partition tolerance is mandatory (networks fail), so choose CA or AP
### Scalability Dimensions
**Vertical scaling** (scale up):
- Add more CPU, RAM, storage to single machine
- Limits: Hardware maximums (~1TB RAM, ~128 CPU cores)
- Use case: Databases, monolithic apps
**Horizontal scaling** (scale out):
- Add more machines
- Limits: Architecture complexity
- Use case: Stateless services, distributed systems
## Scalability Patterns
### 1. Database Scaling
**Read replicas**:
```
┌───────────┐
│ Primary │───┐ (writes)
│ (Write) │ │
└───────────┘ │
├──▶ Replica 1 (reads)
├──▶ Replica 2 (reads)
└──▶ Replica 3 (reads)
```
**Sharding** (horizontal partitioning):
```
User IDs 1-1000000 → Shard 1
User IDs 1000001-2000000 → Shard 2
User IDs 2000001-3000000 → Shard 3
Sharding strategies:
- Hash-based: hash(userId) % num_shards
- Range-based: users 1-1M on shard 1, etc.
- Geo-based: US users on shard 1, EU users on shard 2
```
**Example - PostgreSQL**:
```sql
-- Partition by range
CREATE TABLE orders (
id BIGSERIAL,
created_at TIMESTAMP,
customer_id BIGINT,
total DECIMAL
) PARTITION BY RANGE (created_at);
CREATE TABLE orders_q1 PARTITION OF orders
FOR VALUES FROM ('2023-01-01') TO ('2023-04-01');
CREATE TABLE orders_q2 PARTITION OF orders
FOR VALUES FROM ('2023-04-01') TO ('2023-07-01');
```
**CQRS for read scaling**:
```typescript
// Write model (normalized)
class OrderWriteModel {
async createOrder(data: OrderData) {
await db.orders.insert(data);
await eventBus.publish(new OrderCreated(data));
}
}
// Read model (denormalized for performance)
class OrderReadModel {
async getCustomerOrders(customerId: string) {
// Optimized read-only view
return await readDb.customerOrderSummary.find({ customerId });
}
}
```
### 2. Caching Strategies
**Cache hierarchy**:
```
Request
│
├─▶ L1: In-memory cache (milliseconds)
│ └─ Hit? Return
│
├─▶ L2: Distributed cache (Redis/Memcached) (5-10ms)
│ └─ Hit? Return and populate L1
│
└─▶ L3: Database (50-100ms+)
└─ Populate L2 and L1
```
**Cache patterns**:
**Cache-Aside** (lazy loading):
```typescript
async function getUser(id: string): Promise<User> {
// 1. Try cache
const cached = await cache.get(`user:${id}`);
if (cached) return JSON.parse(cached);
// 2. Cache miss - fetch from DB
const user = await db.users.findById(id);
// 3. Populate cache
await cache.set(`user:${id}`, JSON.stringify(user), { ttl: 3600 });
return user;
}
```
**Write-Through**:
```typescript
async function updateUser(id: string, data: Partial<User>) {
// 1. Write to database
const user = await db.users.update(id, data);
// 2. Write to cache
await cache.set(`user:${id}`, JSON.stringify(user), { ttl: 3600 });
return user;
}
```
**Write-Behind** (async):
```typescript
async function updateUser(id: string, data: Partial<User>) {
// 1. Write to cache immediately (fast response)
const user = { ...existingUser, ...data };
await cache.set(`user:${id}`, JSON.stringify(user), { ttl: 3600 });
// 2. Queue database write (async)
await queue.publish('user.update', { id, data });
return user;
}
```
**Cache invalidation strategies**:
```typescript
// 1. TTL (Time To Live)
cache.set('key', value, { ttl: 3600 }); // 1 hour
// 2. Event-based invalidation
eventBus.on('user.updated', async (userId) => {
await cache.delete(`user:${userId}`);
});
// 3. Cache tags
cache.set('user:123', user, { tags: ['users', 'active-users'] });
cache.invalidateTag('active-users'); // Clear all with this tag
```
### 3. Load Balancing
**Algorithms**:
**Round Robin** (simplest):
```
Request 1 → Server A
Request 2 → Server B
Request 3 → Server C
Request 4 → Server A (cycle repeats)
```
**Least Connections**:
```
Server A: 5 connections
Server B: 3 connections ← Route here
Server C: 8 connections
```
**Consistent Hashing** (for stateful sessions):
```typescript
function getServer(userId: string, servers: Server[]): Server {
const hash = hashFunction(userId);
const index = hash % servers.length;
return servers[index];
}
// Same user always goes to same server
```
**Nginx configuration**:
```nginx
upstream backend {
least_conn; # Algorithm
server backend1.example.com weight=3;
server backend2.example.com weight=2;
server backend3.example.com weight=1;
server backend4.example.com backup; # Only used if others fail
}
server {
location / {
proxy_pass http://backend;
proxy_next_upstream error timeout invalid_header http_500;
proxy_connect_timeout 2s;
}
}
```
### 4. Asynchronous Processing
**Message Queue Pattern**:
```
┌────────┐ ┌───────┐ ┌────────┐
│ API │──────▶│ Queue │──────▶│ Worker │
│ │ │ (SQS) │ │ Pool │
└────────┘ └───────┘ └────────┘
│ │
└─ Immediate response └─ Process async
Benefits:
- Decoupled components
- Handle traffic spikes (queue buffers)
- Retry failed jobs
- Scale workers independently
```
**Implementation**:
```typescript
// Producer (API)
app.post('/process-video', async (req, res) => {
const videoId = req.body.videoId;
// Queue the work (don't process synchronously)
await queue.send('video-processing', {
videoId,
priority: req.body.priority || 'normal'
});
// Immediate response
res.json({ status: 'queued', videoId });
});
// Consumer (Worker)
queue.on('video-processing', async (job) => {
try {
await processVideo(job.data.videoId);
await job.complete();
} catch (error) {
// Retry with exponential backoff
await job.retry({ delay: Math.pow(2, job.attemptsMade) * 1000 });
}
});
```
### 5. Circuit Breaker Pattern
Prevent cascade failures when dependencies fail:
```typescript
class CircuitBreaker {
private failureCount = 0;
private lastFailureTime = 0;
private state: 'closed' | 'open' | 'half-open' = 'closed';
constructor(
private threshold = 5, // Failures before opening
private timeout = 60000, // Time to stay open (ms)
private resetTimeout = 30000 // Time to try half-open
) {}
async execute<T>(fn: () => Promise<T>): Promise<T> {
if (this.state === 'open') {
if (Date.now() - this.lastFailureTime > this.resetTimeout) {
this.state = 'half-open';
} else {
throw new Error('Circuit breaker is OPEN');
}
}
try {
const result = await fn();
this.onSuccess();
return result;
} catch (error) {
this.onFailure();
throw error;
}
}
private onSuccess() {
this.failureCount = 0;
this.state = 'closed';
}
private onFailure() {
this.failureCount++;
this.lastFailureTime = Date.now();
if (this.failureCount >= this.threshold) {
this.state = 'open';
}
}
}
// Usage
const breaker = new CircuitBreaker();
app.get('/external-api', async (req, res) => {
try {
const data = await breaker.execute(() =>
fetch('https://external-api.com/data')
);
res.json(data);
} catch (error) {
res.status(503).json({ error: 'Service temporarily unavailable' });
}
});
```
## Reliability Patterns
### 1. Health Checks
**Kubernetes liveness & readiness**:
```yaml
apiVersion: v1
kind: Pod
spec:
containers:
- name: app
image: myapp:1.0
livenessProbe:
httpGet:
path: /health/live
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 3
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 2
```
**Health check endpoint**:
```typescript
app.get('/health/live', (req, res) => {
// Basic check: is the app running?
res.json({ status: 'ok' });
});
app.get('/health/ready', async (req, res) => {
// Comprehensive check: can the app serve traffic?
const checks = await Promise.all([
checkDatabase(),
checkRedis(),
checkMessageQueue()
]);
const healthy = checks.every(c => c.healthy);
res.status(healthy ? 200 : 503).json({
status: healthy ? 'ready' : 'not ready',
checks
});
});
async function checkDatabase(): Promise<HealthCheck> {
try {
await db.raw('SELECT 1');
return { name: 'database', healthy: true };
} catch (error) {
return { name: 'database', healthy: false, error: error.message };
}
}
```
### 2. Retry with Exponential Backoff
```typescript
async function retryWithBackoff<T>(
fn: () => Promise<T>,
maxRetries = 3,
baseDelay = 1000
): Promise<T> {
for (let attempt = 0; attempt <= maxRetries; attempt++) {
try {
return await fn();
} catch (error) {
if (attempt === maxRetries) {
throw error;
}
// Exponential backoff with jitter
const delay = baseDelay * Math.pow(2, attempt);
const jitter = Math.random() * 1000;
await sleep(delay + jitter);
logger.warn(`Retry attempt ${attempt + 1}/${maxRetries}`, { error });
}
}
throw new Error('Max retries exceeded');
}
// Usage
const data = await retryWithBackoff(() =>
fetch('https://api.example.com/data')
);
```
### 3. Rate Limiting
**Token bucket algorithm**:
```typescript
class RateLimiter {
private tokens: Map<string, { count: number; lastRefill: number }>;
constructor(
private maxTokens = 100,
private refillRate = 10, // tokens per second
) {
this.tokens = new Map();
}
async isAllowed(key: string): Promise<boolean> {
const now = Date.now();
const bucket = this.tokens.get(key) || { count: this.maxTokens, lastRefill: now };
// Refill tokens based on time elapsed
const elapsed = (now - bucket.lastRefill) / 1000;
bucket.count = Math.min(
this.maxTokens,
bucket.count + elapsed * this.refillRate
);
bucket.lastRefill = now;
if (bucket.count >= 1) {
bucket.count -= 1;
this.tokens.set(key, bucket);
return true;
}
this.tokens.set(key, bucket);
return false;
}
}
// Middleware
const limiter = new RateLimiter();
app.use(async (req, res, next) => {
const key = req.ip;
const allowed = await limiter.isAllowed(key);
if (!allowed) {
return res.status(429).json({ error: 'Too many requests' });
}
next();
});
```
**Redis-based distributed rate limiting**:
```typescript
async function isRateLimited(userId: string): Promise<boolean> {
const key = `rate-limit:${userId}`;
const limit = 100; // requests
const window = 60; // seconds
const current = await redis.incr(key);
if (current === 1) {
await redis.expire(key, window);
}
return current > limit;
}
```
### 4. Graceful Degradation
**Feature flags**:
```typescript
const features = {
recommendations: { enabled: true, fallback: 'popular' },
search: { enabled: true, fallback: 'cached' },
analytics: { enabled: true, fallback: 'disabled' }
};
async function getRecommendations(userId: string) {
if (!features.recommendations.enabled) {
return getPopularItems(); // Fallback
}
try {
return await mlService.getPersonalizedRecommendations(userId);
} catch (error) {
logger.error('Recommendations service failed', { error });
return getPopularItems(); // Graceful degradation
}
}
```
### 5. Bulkhead Pattern
Isolate resources to prevent total system failure:
```typescript
// Separate connection pools for different operations
const pools = {
critical: new Pool({ max: 50 }), // Always available
analytics: new Pool({ max: 20 }), // Can fail without affecting critical
background: new Pool({ max: 10 }) // Lowest priority
};
// Critical operation (user authentication)
async function authenticateUser(credentials: Credentials) {
return pools.critical.query('SELECT * FROM users WHERE email = $1', [credentials.email]);
}
// Analytics (can fail without breaking core functionality)
async function trackEvent(event: Event) {
try {
return pools.analytics.query('INSERT INTO analytics ...', event);
} catch (error) {
logger.error('Analytics failed', { error }); // Log but don't throw
}
}
```
## Monitoring & Observability
### Key Metrics (SRE)
**Golden Signals**:
1. **Latency** - Time to serve a request
2. **Traffic** - Requests per second
3. **Errors** - Rate of failed requests
4. **Saturation** - Resource utilization
**SLI/SLO/SLA**:
```yaml
# Service Level Indicator (SLI)
availability: "percentage of successful requests"
latency_p99: "99th percentile response time"
# Service Level Objective (SLO)
availability_target: 99.9% # "Three nines"
latency_p99_target: 200ms
# Service Level Agreement (SLA) - Customer commitment
availability_commitment: 99.5%
penalty: "10% credit if violated"
```
**Error budget**:
```
Monthly error budget = (1 - SLO) × total requests
If SLO = 99.9%, error budget = 0.1% = 43.2 minutes/month downtime
Use budget for:
- Deployments
- Experiments
- Maintenance
If budget exhausted:
- Freeze feature releases
- Focus on reliability
```
### Distributed Tracing
**OpenTelemetry example**:
```typescript
import { trace } from '@opentelemetry/api';
const tracer = trace.getTracer('my-service');
async function handleRequest(req, res) {
const span = tracer.startSpan('handle-request', {
attributes: {
'http.method': req.method,
'http.url': req.url,
'user.id': req.user?.id
}
});
try {
const user = await getUserWithTracing(req.user.id);
const orders = await getOrdersWithTracing(user.id);
span.setStatus({ code: SpanStatusCode.OK });
res.json({ user, orders });
} catch (error) {
span.recordException(error);
span.setStatus({ code: SpanStatusCode.ERROR, message: error.message });
throw error;
} finally {
span.end();
}
}
async function getUserWithTracing(userId: string) {
return tracer.startActiveSpan('get-user', async (span) => {
span.setAttribute('user.id', userId);
const user = await db.users.findById(userId);
span.end();
return user;
});
}
```
## Performance Benchmarks
**Target latencies (95th percentile)**:
- Database query: <10ms
- Cache hit: <1ms
- Internal API call: <50ms
- External API call: <200ms
- Page load: <2s
- API response: <100ms
**Capacity planning**:
```
Required capacity = (Peak RPS × Average latency) / Target CPU utilization
Example:
Peak RPS = 10,000
Average latency = 50ms = 0.05s
Target CPU = 70%
Capacity = (10,000 × 0.05) / 0.70 = 714 concurrent requests
≈ 15-20 servers (assuming ~40 concurrent requests per server)
```
## Resources
- Google SRE Book
- AWS Well-Architected Framework
- Microsoft Azure Architecture Center
- Designing Data-Intensive Applications (Martin Kleppmann)
- Release It! (Michael Nygard)