The Compute Spectrum: Machines, Containers, Functions

The compute options look like a menu of alternatives. They are better understood as one axis, ordered by how much of the machine you still think about.

At one end you rent a virtual machine and it is yours: you choose the operating system, install what you like, patch it, and it runs until you stop it. At the other end you upload a function and the platform decides when to run it, how many copies exist, and on what.

Every step along that axis makes the same trade in the same direction:

Key idea: there is no "best" point on this spectrum, and moving right is not progress. Each step buys operational relief and costs flexibility, so the right position is the furthest right you can go without losing something the workload genuinely needs.

The virtual machine is the oldest cloud primitive and the least fashionable, which obscures how often it remains correct.

What it gives you is total control: any operating system, any kernel parameter, any daemon, any piece of licensed software that expects a real host, any GPU driver stack. What it costs you is that everything on that machine is now your responsibility, from patching to monitoring to replacing it when it fails.

Two aspects reward understanding early:

• Instance families are specialised. Instance types are grouped by what they optimise, general purpose, compute optimised, memory optimised, storage optimised, and accelerated with GPUs. Picking a family that matches the workload's actual bottleneck is usually a larger win than picking a bigger instance in the wrong family.
• Purchasing modes change the price enormously. On-demand charges per unit time with no commitment. Reserved capacity and savings plans discount heavily for a one or three year commitment. Spot instances offer spare capacity at a steep discount, with the condition that AWS can reclaim them at short notice.

In practice: spot capacity is the most under-used lever in cloud cost. Any workload that can checkpoint and resume, batch processing, CI runners, model training, rendering, can often run on interruptible capacity for a fraction of on-demand, and the engineering it requires, tolerating a killed worker, is engineering you wanted anyway.

A container packages your application with its dependencies so it runs the same everywhere. The catalogue covers container mechanics in depth elsewhere; what matters here is that running them in a cloud raises two separate questions people routinely conflate.

Who schedules the containers? Something must decide which container runs where, restart failures, and roll out new versions. AWS offers its own orchestrator, ECS, and a managed Kubernetes, EKS. The trade is familiar: the native option is simpler and integrates tightly with the rest of AWS, while managed Kubernetes brings a large portable ecosystem and correspondingly more complexity to operate.

Whose servers do they run on? Independently of the orchestrator, the containers need machines. Either you run a pool of virtual machines and manage their capacity, patching and scaling, or you use Fargate, where AWS provides the compute per container and there is no cluster of servers to look after.

Gotcha: teams reach for managed Kubernetes because it is the industry standard and then discover the managed part covers the control plane, not the operational burden. Node upgrades, autoscaler tuning, networking plugins, ingress and the rest remain yours. Choose it for the ecosystem and portability, which are real; do not choose it expecting it to be less work than the simpler option.

Serverless functions invert the model. Instead of a process you keep running that waits for work, you supply code and the platform creates an execution environment when work arrives.

The consequence that shapes every design decision is the cold start. If no environment is warm, the platform must provision one, load your code and dependencies, and run any initialisation before your handler executes. That is the cold path. A subsequent request arriving while an environment is still warm skips all of it and runs the handler directly.

The practical rules follow directly from the diagram's two paths. Work done outside the handler happens once per environment, not once per request, which is why database connections and clients belong there. Large dependency bundles lengthen every cold start, so keeping the package small is a latency optimisation. And because environments are frozen between invocations rather than destroyed, background work started but not awaited may simply stop mid-flight and resume unpredictably later.

Scaling is the flip side: concurrency is achieved by running many environments at once, one per concurrent request, which is why serverless absorbs spikes gracefully and why a downstream database can be overwhelmed by a burst it never expected.

Serverless is excellent for event-driven, spiky, stateless work, and it is oversold. The honest list of poor fits, each for a structural reason rather than a fixable one:

• Sustained high-volume traffic. Per-invocation pricing is superb when usage is bursty and idle much of the time. A service running flat out around the clock is usually cheaper on reserved machines, because you are no longer paying a premium for elasticity you never use.
• Long-running work. Functions have an execution ceiling. Work that legitimately runs for hours belongs in a container or batch job rather than being contorted into chunks to fit.
• Latency-critical paths with strict tails. Cold starts put a fat tail on the latency distribution. Provisioned concurrency mitigates it by keeping environments warm, at which point you are paying for idle capacity, which was the thing serverless promised to avoid.
• Anything needing local state or persistent connections. No in-memory cache survives reliably, and connection-per-environment behaviour is exactly what exhausts database connection pools during a scaling burst.

The decision is usually settled by four properties of the workload rather than by preference.

That last row deserves emphasis, because it is the one most often decided by fashion instead of arithmetic. Operating a cluster is a real ongoing job. A team of four shipping a product will get more from the managed option's constraints than from the flexibility of infrastructure nobody has time to tend.

In practice: mixing is normal and correct. A typical system runs its API on containers, its image processing on serverless functions triggered by uploads, its nightly aggregation as a batch job on spot capacity, and one virtual machine for the vendor appliance that refuses to be containerised. Uniformity is not a goal; fit is.

One design property matters more than the position you pick, and it is what makes the position changeable later: whether your compute is disposable.

Disposable means any running instance can be destroyed and replaced without ceremony, because it holds nothing that matters. No local state that is not also elsewhere, no manual configuration applied by hand, no uploaded files living only on that disk, nothing whose loss requires a person.

That property is what enables everything teams want from cloud infrastructure:

• Autoscaling, which is just creation and destruction on demand.
• Rolling deploys and instant rollback, which replace instances rather than mutating them.
• Spot capacity, which is only safe if losing a worker is uneventful.
• Recovery, since rebuilding from a definition beats repairing a machine nobody remembers configuring.

Key idea: the machine that cannot be destroyed is the one that eventually causes the outage, because everyone is afraid to touch it and its configuration exists only in its running state. Serverless enforces disposability by construction; on virtual machines it is a discipline you choose, expressed as infrastructure-as-code plus state kept in the storage services of the next lesson.