Where LLMs Actually Fit in a Trading Firm

A trading firm's numerical data has been machine-readable for decades. Prices, volumes, fundamentals, positions: all structured, all cheap to process, all mined exhaustively by everyone.

The text has not been. Regulatory filings, earnings call transcripts, broker research, central bank statements, contracts, news, prospectuses. Every one of these contains information that moves prices, and every one required a person to read it.

That asymmetry shaped the industry. Analyst coverage concentrated on large companies because reading is expensive per name, which is why small caps are under-covered rather than because they are less interesting. Systematic strategies leaned on numerical data because that is what could be processed at scale.

So the honest framing of what changed is not that a model can now predict returns. It is that reading became cheap, and a constraint that shaped what was economical for forty years relaxed.

Everything worth saying about LLMs in trading follows from asking what that relaxation actually enables, rather than from asking whether a chatbot can pick stocks.

The distinction that separates working systems from demonstrations is what the model is asked to produce.

Extraction asks the model to find something stated in a document and return it in a structured form. What guidance range did management give? Which risk factors are new since last year? Which counterparties are named? Was the tone of the analyst Q&A more defensive than the prepared remarks?

Each of those has a ground truth in the text. The answer is checkable by a human reading the same document, and a wrong answer is a wrong answer rather than a difference of opinion.

Judgement asks the model for a conclusion the text does not contain. Will this stock go up? Is this company a good investment?

The difference matters for one specific reason: extraction can be evaluated and judgement cannot. You can build a labelled set of documents with correct extractions and measure accuracy. You cannot build a labelled set of correct opinions about future prices, because the label is a noisy realisation of something largely unpredictable, and the next lesson shows the evaluation is contaminated as well.

Systems that survive contact with production are almost always doing extraction, with the judgement left where it was.

The top branch is a hard exclusion and it is worth being blunt about, because it is where most of the marketing implies these systems live.

The lesson Market Design: Ticks, Venues, and the Speed Race establishes that competition in continuous markets is a race measured in microseconds. An LLM producing tokens over hundreds of milliseconds is not slow for that race; it is not in it. No amount of optimisation closes a gap of five or six orders of magnitude.

The second branch is a subtler exclusion. An execution algorithm making per-slice decisions cannot wait for a model, but it can consume a score computed earlier. The distinction is precomputed input versus in-the-loop call, and it is the difference between a system that works and one that times out.

The bottom two are where the fit is real. Research operates on hours to weeks, where a second of latency is irrelevant and coverage is the binding constraint. Compliance and surveillance are document-processing problems by nature.

So the honest map is: LLMs belong upstream of the trade, not inside it.

The economics are worth making concrete, because the value is a throughput argument rather than an accuracy one.

A human analyst covers perhaps fifteen to thirty companies properly. Reading one annual report carefully is a day; a quarter's earnings calls across a sector is a week. That cost per name is why coverage thins with market capitalisation, and why the same handful of large companies are analysed by everyone.

Processing the same documents by model changes the cost per name by orders of magnitude, and the consequence is not better analysis of the covered names. It is coverage of names nobody was covering, and consistency across them.

That second part is underrated. Twenty analysts applying their own judgement to twenty companies produce twenty incomparable assessments. One model applying one rubric produces a cross-sectionally comparable measurement, which is what a systematic strategy needs and what human coverage structurally cannot provide.

There is a pointed connection to the previous cursus here. Lopez-Lira and Tang found the predictive content of their signal concentrated in small stocks, which is exactly where human coverage is thinnest.

That is consistent with the mechanism being information that was always there and nobody was reading, rather than the model having insight. It is also, as the third lesson shows, exactly where the signal is hardest to trade.

An obvious question is whether a domain-specific model beats a general one, and there is a well-documented data point.

Bloomberg described BloombergGPT in "BloombergGPT: A Large Language Model for Finance" (2023). It is a 50-billion-parameter model trained on roughly 700 billion tokens, about half from FinPile, a corpus of financial documents drawn from Bloomberg's archives, with the remainder from general-purpose sources.

On public financial benchmarks it recorded an average score of about 62.5%, against roughly 51.9% to 54.4% for comparable open models of similar size, while remaining competitive on general benchmarks.

That is a genuine result and it needs reading in context. It compares against contemporaneous models of similar size, and the frontier moved substantially afterwards. The broader pattern across the field has been that general models improved faster than the domain gap could be maintained, which is why most deployed systems now use a strong general model with domain knowledge supplied through retrieval and prompting rather than through pretraining.

The transferable point is about where domain knowledge should live. Baked into weights it is expensive to build and immediately stale. Supplied at inference from a document store it is cheap to update and, crucially, auditable, which the last lesson shows is not optional in this industry.

A consideration specific to this industry, and one that generic AI discussion misses entirely.

The lesson Why a Spread Exists at All establishes that trading profits come from having information others do not, and that a trade is evidence precisely because information is asymmetric.

Now consider a signal produced by a widely available model, from public news, using a prompt that is the obvious one. Every participant with the same idea gets approximately the same output. The information is public, the transformation is public, and the result is therefore not private information in the sense that pays.

What happens next is what always happens: everyone trades it, the price moves to reflect it faster, and the horizon over which it predicts anything shortens toward zero. That is the market working correctly, and it is fatal to a strategy built on the obvious application.

So the durable edge is not in the model. It is in things the model does not supply: proprietary documents nobody else can process, a non-obvious question asked of common documents, speed of ingestion for a signal that decays, or integration into a decision the model does not make.

That is a familiar shape rather than a new one. A public dataset with a public method has always been competed away, and a general-purpose model is the most public method yet invented.

Several general LLM weaknesses become specifically dangerous in this domain.

Fabricated numbers. A model that invents a plausible figure is a nuisance in most applications and a direct financial loss here. Any number extracted must be traceable to a span in a source document, which is an architectural requirement rather than a quality improvement.

Non-determinism. The same input can produce different outputs. For a system whose decisions must be explained to a risk committee or a regulator, unreproducible outputs are a compliance problem regardless of whether they were correct.

Confident errors. Output carries no calibrated uncertainty by default. A wrong extraction reads exactly like a right one, so nothing in the response signals that a human should look.

Prompt injection. This one is unusually serious here. The inputs are documents from outside the firm, some written by parties with an interest in how they are read. A filing or a press release containing text crafted to steer a model is an adversarial input arriving through the front door, and the lesson LLM Guardrails and Red-Teaming treats the general problem.

The recurring theme is that this domain removes the tolerance normally available. Elsewhere an occasional wrong answer is absorbed by a user noticing; here it is capital at risk, and often nobody is reading.

The pattern in the middle column is that the strong rows all convert text into structure, and the weak rows all ask for a conclusion.

That is the summary of the lesson, and it survives model improvements, because it is a statement about what can be checked rather than about what a model can do.

The obvious objection is that a paper in the Journal of Financial Economics reported an LLM predicting returns from headlines, which appears to contradict the weak row directly. That paper is real and its findings are more interesting than either the enthusiastic or the dismissive reading suggests.

Before it can be read properly, though, one problem has to be dealt with, because it makes almost every published result in this area harder to interpret than it looks. The model was trained on text from the period the backtest covers, and it may already know what happened next.