Scheduling and Resources: Requests Are Not Limits

The scheduler is not a special subsystem. It is a controller whose definition of correct is "every pod has a node assigned", and its reconciliation is to pick one and write it into the API.

It runs in two phases.

Filtering eliminates nodes that cannot host the pod: insufficient unreserved capacity, a taint the pod does not tolerate, a node selector that does not match, a required port already taken.

Scoring ranks the survivors, favouring nodes with more free room, nodes that already have the image cached, and spreading across failure domains.

The highest-scoring node wins and its name is written to the pod's spec. The scheduler then stops thinking about that pod entirely, and the kubelet on that node, watching for pods assigned to it, picks it up.

The crucial detail for everything that follows: filtering uses requests, not actual usage. A node running at 5% CPU whose pods have collectively requested all of its capacity is full as far as the scheduler is concerned. Capacity in Kubernetes means unrequested capacity, not idle capacity.

Every container may declare two values per resource, and they are used by different components for different purposes.

resources:
  requests:
    cpu: "250m"        # 0.25 of a core
    memory: "256Mi"
  limits:
    cpu: "1000m"       # 1 core
    memory: "512Mi"

Requests are a reservation. The scheduler subtracts them from a node's capacity when deciding what fits, and it guarantees this much is available. Nothing enforces them at run time: a container may use more if the node has spare.

Limits are a ceiling, enforced by cgroups on the node, exactly as in the containers cursus. Nothing about scheduling uses them.

So the pairing is: requests determine where the pod goes and what it is promised; limits determine what happens if it asks for more.

That separation is the most consequential thing to understand here, and getting it wrong produces two symmetric failures. Requests too high wastes money, because reserved-but-unused capacity blocks other pods from being scheduled. Requests too low overcommits the node, so pods land somewhere that cannot actually feed them and performance collapses under load.

The asymmetry is the single most important operational fact in this lesson, and it follows from physics rather than policy.

CPU is compressible. Time can be taken away and given back. A container over its CPU limit is throttled: the cgroup quota stops scheduling it for the rest of each period. It runs slower and it keeps running.

Memory is not compressible. Bytes already written cannot be reclaimed by asking. A container over its memory limit is killed by the kernel's out-of-memory handler, and restarted.

Two consequences follow directly.

A CPU limit is a latency risk, not a safety feature. Throttling is invisible in most dashboards: the pod is healthy, CPU usage sits neatly at the limit, and response times are terrible. It is a common cause of unexplained tail latency, and the diagnosis is to look at throttling metrics rather than utilisation.

A memory limit is a kill switch. Setting it too close to real usage means periodic restarts during traffic peaks, appearing as OOM kills on a host with free memory, because the limit that was hit was the container's.

The practical guidance many teams converge on: always set memory limits, and be cautious with CPU limits, since an unthrottled pod that occasionally uses spare CPU is usually better than a predictably slow one.

Kubernetes assigns each pod a QoS class automatically, from how its requests and limits relate. You never set it directly.

Guaranteed. Every container specifies both requests and limits, and they are equal, for both CPU and memory.

Burstable. Requests are set and are lower than limits, so the pod can exceed its reservation when the node has room.

BestEffort. No requests or limits at all.

The class determines survival under node pressure. When a node runs short, the kubelet evicts in order: BestEffort first, then Burstable, and Guaranteed last. Within Burstable, the kubelet also considers how far over its request each pod is: one using 300% of its request is evicted before one using 110%.

That last detail is the useful one, because it makes the incentive concrete. A pod with an honest request is protected in proportion to that honesty, and a pod that requested almost nothing and grew large is the first to go.

So the practical instruction is not to chase a class. It is to set requests close to real usage. That produces sensible scheduling, sensible eviction ordering and sensible cost, and the QoS class follows from it.

Guaranteed is worth targeting deliberately for genuinely latency-critical workloads, accepting that equal requests and limits means paying for the peak all the time.

Because limits can exceed requests, the sum of limits on a node routinely exceeds its capacity. That is not a misconfiguration; it is the point.

Most workloads use far less than their peak most of the time. Reserving the peak for every pod would leave most of the fleet idle, so the system lets pods request their typical usage and burst into whatever is spare.

The gamble is that they do not all peak together. When they do, the node runs out and the eviction ordering above decides who loses.

That is a deliberate trade of utilisation against reliability, and the ratio is a real decision. A cluster whose requests sum to the node capacity is safe and expensive. One heavily overcommitted is cheap and will shed pods under correlated load.

The honest framing is that overcommitment is a statistical bet, and the assumption behind it is that peaks are independent. That assumption is exactly the one that fails in the situations that matter, for the same reason correlations converge under stress in the lesson When the Distribution Lies.

A traffic spike hits every replica of a service at once, which is precisely when the spare capacity you were counting on is being claimed by everyone simultaneously.

The scheduler's default is reasonable, and several mechanisms override it when placement actually matters.

nodeSelector requires matching node labels. Simple and absolute: no matching node means the pod stays pending.

Affinity is the expressive version, and its value is in offering preferences as well as requirements. Preferred rules influence scoring without blocking scheduling, which avoids the failure mode where a strict constraint leaves pods unschedulable.

Pod anti-affinity places pods apart from each other, which is how you stop three replicas of a service landing on one node and being lost together. This is worth setting on anything whose availability you claim.

Taints and tolerations work in reverse: a taint on a node repels pods unless they explicitly tolerate it. That is how nodes are reserved, for instance keeping GPU machines free of general workloads.

Topology spread constraints state the goal directly, such as spreading replicas evenly across availability zones, which is usually what anti-affinity was being used to approximate.

The caution across all of them is the same. Every constraint reduces the set of viable nodes, and over-constrained pods sit in Pending forever with an event explaining that no node satisfies them. Constraints are cheap to add and easy to accumulate past the point where anything fits.

Three autoscalers operate at different levels, and they interact in ways worth anticipating.

Horizontal pod autoscaling changes the replica count based on a metric, usually CPU relative to requests. Note the denominator: it scales on utilisation against the request, so a wrong request makes the autoscaler wrong in the same proportion.

Vertical pod autoscaling changes requests and limits based on observed usage. It solves the sizing problem the whole lesson has been circling, and historically it required recreating pods to apply changes, which is disruptive for the workloads that most need right-sizing.

Cluster autoscaling adds and removes nodes, watching for pods stuck in Pending because nothing has room.

The interaction to plan for is the chain: load rises, horizontal scaling adds pods, no node has capacity, pods go pending, cluster scaling adds a node, the node joins, images pull, pods start. Each stage takes time, and the total is minutes rather than seconds.

That is fine for a gradual ramp and inadequate for a step change, which is why headroom still matters. Autoscaling adjusts capacity; it does not respond instantly, and treating it as instant is how launches fail.

Running horizontal and vertical autoscaling on the same CPU metric is also a known conflict, since one changes the numerator and the other the denominator.

The row to internalise is the middle pair, because they fail in opposite ways and only one is visible. A memory limit hit produces a restart and an event; a CPU limit hit produces nothing except slowness.

The practical routine is unglamorous and effective: measure actual usage, set requests near the typical value, set memory limits with real headroom, and treat CPU limits as a deliberate choice rather than a default.

The deeper point is that these numbers are the interface between your workload and the system's scheduling, eviction and autoscaling decisions all at once. They are usually copied from an example and then never revisited, and they are simultaneously the largest driver of cluster cost and the most common cause of unexplained latency.

What has not been discussed is what happens when the reconciliation model itself makes things harder, which is the last lesson.