Privacy Attacks, Real Guarantees, and Open Models

The premise of federated learning is that sending model updates instead of data protects the data. That premise needs qualification, because an update is a function of the data that produced it.

Consider the simplest case. A client computes a gradient on a single example. For many architectures the gradient with respect to the first layer's weights is directly related to that input. The relationship is close enough that reconstruction attacks have recovered recognisable training images from gradients, and the general phenomenon is well established in the literature.

Larger batches and more local steps make this harder, because the update aggregates many examples and mixes them. But harder is not impossible, and the attack surface does not close.

A weaker but broadly applicable attack is membership inference: rather than reconstructing an example, determine whether a specific known record was in someone's training set. In a medical setting that alone can be the disclosure that matters, since membership in a cohort may reveal a diagnosis.

So the accurate framing is that federation provides data minimisation, which is genuinely valuable and often what regulation demands, but not a quantified privacy guarantee.

The first mechanism addresses a specific threat: the server seeing any individual client's update.

Secure aggregation is a cryptographic protocol that lets the server learn the sum of client updates while learning nothing about any single one.

The idea is elegantly simple in outline. Clients pair up and agree on shared random masks. Each client adds its masks to its update before sending, arranged so that within any pair, one adds a mask and the other subtracts the identical mask. Individually the masked updates are indistinguishable from noise. Summed, the masks cancel exactly and the true total emerges.

The protocol must also handle dropouts, since a client vanishing mid-round leaves its partner's mask uncancelled, and real implementations use secret sharing so surviving clients can reconstruct the missing masks.

What this buys and what it does not is worth being exact about. It removes the server as a party that sees individual contributions, which addresses the trust problem from the decentralization lesson. It does not protect against what the aggregate itself reveals, and with few participants an aggregate can still be revealing.

The second mechanism gives the guarantee secure aggregation cannot: a bound on what the output reveals.

Differential privacy makes a precise promise. The trained model would be nearly as likely to arise if any single individual's data were removed. Since the outcome barely depends on any one person, observing the outcome tells you almost nothing about any one person, regardless of what auxiliary information an attacker holds.

Achieving it in training requires two operations, and both matter.

Clip. Bound each contribution's norm, so no single client or example can move the model much.

Add noise. Perturb the aggregate with calibrated random noise, so the result is not a deterministic function of any individual's data.

The strength is controlled by a parameter conventionally written epsilon, where smaller means stronger privacy and more noise.

The trade-off is unavoidable and should be stated plainly: privacy costs accuracy. Noise sufficient for a strong guarantee degrades the model, and the damage falls hardest on rare patterns, which are exactly what a model learns from underrepresented groups. Strong privacy and equitable performance can be in tension.

The three mechanisms address different adversaries and compose rather than substitute. Choosing among them is really a question of which threat you face, and a system defending against all three pays all three costs.

Privacy asks what the system leaks about clients. The opposite question is what a malicious client can do to the system, and federation is structurally exposed here.

The server never sees client data, so it cannot verify that an update reflects honest training. That is the same property that provides privacy.

Model poisoning submits a crafted update designed to damage the global model or install specific behaviour. A backdoor attack is the sharper version: train so the model behaves normally except on inputs carrying a chosen trigger, then submit that update. If it survives aggregation the backdoor persists in the model everyone receives.

The standard defence is robust aggregation: instead of averaging, use a statistic that resists outliers, such as a coordinate-wise median or trimmed mean, or filter updates far from the bulk.

Here is the uncomfortable interaction, and it is the reason this deserves its own step. Robust aggregation and privacy are in direct conflict. Detecting anomalous updates requires inspecting individual updates, which secure aggregation exists to prevent. Strengthening one weakens the other, and there is no configuration that maximises both.

Federated learning keeps data private while sharing a model. It is worth closing with the opposite arrangement, which the same institution pursued.

On 2 September 2025, EPFL, ETH Zurich, and the Swiss National Supercomputing Centre released Apertus, described as Switzerland's first large-scale open multilingual language model. The name is Latin for "open", and the claim is unusually strong: the entire development process, including architecture, model weights, training data and recipes, is openly accessible and fully documented.

The family comprises 8B and 70B parameter models, in pretrained and instruction-tuned variants, with support described as covering over 1,000 languages, and developed with attention to Swiss copyright and data protection law and the EU AI Act.

The contrast with federation is instructive rather than contradictory. Federation addresses data that cannot be shared. Apertus addresses models whose construction cannot be inspected, where weights may be downloadable but the data and process behind them are not.

These are complementary answers to one underlying question: what has to be visible, and to whom, for a machine learning system to be trustworthy?

A checklist for anyone considering this, since federation carries real costs and is sometimes chosen for the wrong reason.

The first row is the one people skip. Federation is markedly harder than centralised training: slower convergence, more system complexity, harder debugging, and usually some accuracy cost. If you are legally permitted to pool the data, pool it.

Two closing gotchas. Do not report only global accuracy in a federated deployment, because a global average can conceal that the model fails on a subset of clients who are, in a hospital consortium, a real population. And privacy parameters are not decorative: a large epsilon provides a guarantee weak enough to be close to meaningless, so the number has to be justified rather than tuned until accuracy looks acceptable.