Insurance AI Under the Rules: Unfair Discrimination and Proxies

Here is the tension that makes insurance regulation unlike any other AI domain. Insurance is a business whose core function is discrimination, in the literal, neutral sense: distinguishing between risks and charging them differently. Lesson 1 showed why it must. Adverse selection punishes an insurer that fails to separate risks, so risk classification is not an abuse of the model, it is the model.

At the same time, the law forbids unfair discrimination: distinguishing on the basis of protected characteristics such as race, or on grounds with no legitimate actuarial basis.

So insurance regulation does not ask "is this model discriminating?" It obviously is. It asks a harder question: is this distinction one the law permits? The traditional answer is the principle of actuarial justification, a rating factor is permissible if it genuinely predicts loss and is not a prohibited characteristic. Charging more for a sports car than a minivan is fine; charging more by race is not, no matter what the data says.

That framework worked reasonably well when models had a dozen hand-chosen variables an actuary could defend one at a time. Modern AI breaks it, not by being evil, but by being complicated, and the rest of this lesson is about how regulators responded.

The central problem has a name: proxy discrimination. An insurer does not need to use race as a variable to produce race-based pricing. It only needs variables that correlate with race, and a model flexible enough to find the combination.

This is not hypothetical, it is a predictable consequence of how the technology works. Geography correlates with race in many countries because of historical segregation. Shopping patterns, occupation, education, credit history, and countless behavioral traces carry demographic signal. A model told only "minimize prediction error" will use whatever correlates with the outcome, and it has no concept that some correlations are legally radioactive. Remove the protected variable and the model reconstructs it from the remainder.

This defeats the traditional compliance approach in a specific way. The classic test was inspect the variable list: no race variable, no race discrimination. That test is now nearly worthless. A model with a hundred innocuous-sounding features can encode a protected characteristic more effectively than a model that used it directly, while passing an input-based review perfectly.

The regulatory response follows logically from this diagnosis. If you cannot certify fairness by looking at the inputs, you must test the outcomes. That single shift, from inspecting variables to quantitatively testing results, is what every framework in the rest of this lesson has in common.

In the United States, insurance is regulated state by state, which risks fifty different AI regimes. The National Association of Insurance Commissioners (NAIC) addressed this by adopting, in December 2023, its Model Bulletin on the Use of Artificial Intelligence Systems by Insurers. It is a model that states adopt individually, and by mid-2026 more than 20 US jurisdictions had adopted it in full or substantially similar form, making it the de facto American baseline.

The bulletin is principles-based rather than prescriptive: it does not ban techniques or mandate thresholds. Instead it requires insurers to be able to show their work. Its centerpiece is a requirement to maintain a written AI Systems Program (AIS Program) governing AI that makes or supports decisions in regulated insurance practices. The expectations include:

• Accountability at the top: senior management and board responsibility, not a task delegated to a modeling team.
• Risk controls and model validation: testing for errors, bias, and unfair discrimination, before and during deployment.
• Third-party oversight: if you buy a model or data from a vendor, you remain responsible. Insurers are expected to assess vendor data and systems and secure contractual protections such as audit rights and cooperation with regulators.
• Consumer transparency: notifying consumers when AI systems are in use, and giving them appropriate information about how those systems may affect decisions about them.

And it tells insurers what regulators may demand: the bulletin sets out the information a state may request about an insurer's AI during an examination or investigation. The message is documentation, not prohibition.

Two states went further than principles, and their approaches show what outcome-testing looks like in law.

Colorado, SB21-169 ('Protecting Consumers from Unfair Discrimination in Insurance Practices'). It targets external consumer data and information sources (ECDIS), data about consumers from outside the insurer, along with the algorithms and predictive models that use it. Its scope is broad, reaching marketing, underwriting, pricing, utilization management, reimbursement methodologies, and claims management. Its core demand is the shift named earlier: insurers must quantitatively test whether their ECDIS, algorithms, and models produce unfairly discriminatory outcomes, and demonstrate that testing to the Division of Insurance. The Division has proceeded line by line, exposing a proposed regulation for quantitative testing in life insurance underwriting in September 2023.

New York, NYDFS Insurance Circular Letter No. 7 (2024), adopted 11 July 2024, covering AI systems and ECDIS in underwriting and pricing. Its distinctive requirement is the proxy assessment: insurers must assess whether an ECDIS, or individual data fields within it, correlates with membership of a protected class in a way that could produce unfair or unlawful discrimination.

Read that requirement against the previous step and its logic is exact. NYDFS is not asking "did you use a protected variable?" It is asking "did you use anything that stands in for one?" The regulator has named proxy discrimination and made testing for it an obligation.

Europe took a different route: rather than an insurance-specific rule, insurance is named inside a general AI law, and lands in its most demanding tier.

The EU AI Act classifies AI by risk. Annex III, point 5(c) designates as high-risk: AI systems intended to be used for risk assessment and pricing in relation to natural persons in the case of life and health insurance. That places insurer pricing models alongside the Act's most scrutinized uses. Draft guidance has read the phrase broadly, extending to areas such as private long-term care insurance (a health insurance service), certain personal pension products, and credit life insurance attached to a mortgage where it functions as a payment-protection obligation. The full obligations for Annex III high-risk systems apply from 2 August 2026.

High-risk classification is consequential: it brings duties around risk management, data governance, technical documentation, record-keeping, transparency, human oversight, and accuracy and robustness.

Two things are worth noticing. First, the Act's test is what the system does, whether it evaluates and classifies natural persons for risk assessment, not what it is called. Second, the scope is deliberate: life and health pricing is singled out because being mispriced or refused for health cover has a severity that pricing a warehouse does not. This is not AI regulation that happened to catch insurance; the drafters named insurance on purpose.

Four frameworks, two continents, different legal machinery, and a strikingly consistent set of demands.

Strip away the drafting and four principles recur:

• Test outcomes, not inputs. Because proxies defeat input inspection, fairness must be demonstrated in results.
• Own your vendors. Buying the model does not outsource the responsibility.
• Keep a human accountable. Senior accountability, human oversight, and the ability to explain a decision to the person it affected.
• Document everything. Regulators expect to be shown the work on examination.

The convergence is the signal. Independent regulators, reasoning from the same facts about how these models behave, arrived at nearly the same answer, which suggests these obligations reflect the technology's actual failure modes rather than any one jurisdiction's politics.

Close with what remains hard, because the rules do not dissolve the difficulties.

• Fairness testing needs protected-class data you may not have. To test whether outcomes differ by race, you need to know applicants' race, which insurers often do not collect and may be restricted from collecting. Regulators and insurers therefore lean on inference methods to estimate it, which introduces its own error and its own controversy. This is a genuine, unresolved tension: you may need the data to prove you are not misusing the data.
• There is no single definition of fair. Different statistical fairness criteria conflict mathematically, so satisfying one can violate another. "Quantitatively test for unfairly discriminatory outcomes" leaves real interpretive work.
• Accuracy and explainability trade off. The flexible models that lift the loss ratio are the hardest to explain, and Lesson 2 noted the lift is often a few points. An insurer may face a real choice between a better model and a defensible one.
• Feedback loops are invisible. Decline a group, never observe their claims, and your data never learns you were wrong. The model's own decisions shape the data that trains its successor.
• The promise is the product. Recall Lesson 1: insurance sells a contingent promise, and Lesson 2 ended at claims. A wrongly automated denial is not a bad recommendation, it is a broken promise at the moment of need.

The fair verdict: AI in insurance is a strong technical fit for a genuine prediction business, delivering real gains in speed, expense, and accuracy, operating under rules that exist because the failure modes are real and land on people who cannot appeal them.

Dropping a protected variable does not stop proxy discrimination, because correlated features let a model reconstruct it, so input inspection fails and regulators converge on testing outcomes, assessing proxies, and requiring documented human accountability.