[00:12] Tail Latency Amplification in Microservice Fan-Out
When staging tests run under low load, single-hop tail latency stays below 14ms. Once an aggregator fans out to 20 backend shards concurrently, overall request completion waits for the slowest shard.
KEY POINT
At a 1% per-leaf tail probability across 20 parallel leaf RPCs, end-to-end probability of hitting slow p99 surges from 1% to 18.2%.
[04:18] Hedged Requests vs Tied Requests Benchmarks
| Strategy | Extra Traffic | Production p99 Latency | Best Topology Fit |
|---|---|---|---|
| Synchronous Baseline | 0% | 580 ms | Single-hop calls |
| Hedged Reads (p95 wait) | +4.8% | 31 ms | Replicated read caches |
| Tied Requests w/ Queue Token | +2.1% | 34 ms | Observable server queues |
"Production reliability stems from bounding worst-case queues, never hoping averages stay low."
[13:00] Zero-Omission Protocol Configuration
grpc_deadline_ms=120 hedge_delay_ms=18 max_retry_budget_ratio=0.05 ./gateway-server
- Propagate caller remaining deadline across every downstream RPC header [13:00].
- Enforce circuit-breaker backoff whenever retry consumption approaches 5%.