hardA queue absorbs a burst only up to what still finishes inside the timeout. Past that: move capacity to the burst if you know when it comes, and refuse fast if you do not.
load balancer
service
bursts and queues
scheduled scaling
autoscaling lag
load shedding
0 of 5 done
1How big the buffer really is
client_web
Web client
load: 100–140 requests a minute; 4 a second for 6 s after the newslettertimeout: gives up after 10 s
load_balancer
Load balancer
health check: GET /health every 2s, out after 3 failures
service×3
Export service
instances: 3latency: 1 s per request, one at a time per instancequeue: up to 12 waiting per instance
Press enter or space to select a node. You can then use the arrow keys to move the node around. Press delete to remove it and escape to cancel.
Press enter or space to select an edge. You can then press delete to remove it or escape to cancel.
Three instances behind the balancer. Each handles one request a second, one at a time, and queues up to 12. Clients give up after 10 seconds. Beyond what the three are already doing, how many extra requests can their queues hold before requests start timing out?