The Five Analyses That Answer Most Business Questions

Business questions feel varied and they mostly reduce to five analytical patterns. Recognising which one you need is most of the skill, because the pattern determines the calculation, the chart and the caveats.

Trend. How has this changed over time? Answers whether something is getting better or worse.

Breakdown. How does this differ across groups? Answers where a problem is concentrated, which is almost always more useful than the overall figure.

Funnel. Where do people drop out of a sequence? Answers which step to fix when a process has stages.

Cohort. How do groups defined by when they started behave over time? Answers whether things are actually improving, as opposed to looking like they are.

Distribution. What does the spread look like? Answers whether an average means anything.

Most real questions combine two. Trend plus breakdown, which is trend by segment, is probably the single most useful analysis in business, because it separates a general movement from something happening in one place.

What is not on this list is prediction, which is a different discipline, and causal attribution, which this catalogue's cursus on knowing what actually works covers properly. This lesson is about description: establishing what is happening, accurately. That is the foundation, and organisations that skip it in favour of sophistication generally do not know what is happening.

The rest of this lesson takes each pattern, what it is for, and how it misleads.

Plotting a measure over time is the most common analysis and it has a specific set of ways to mislead.

Seasonality. Retail sales rise in December every year. A December-to-January fall is not a decline; it is the calendar. The fix is comparing to the same period last year rather than to the previous period, which is why year-on-year comparison is the default in most businesses.

Unequal periods. Months have different numbers of days and different numbers of working days, and a four-week month against a five-week month is a twenty-five percent difference in exposure that has nothing to do with performance. Comparing daily averages rather than totals removes it.

Definition changes. A metric that jumps sharply on a specific date usually means someone changed how it was calculated, not that the business changed. This should be the first hypothesis for any step change, and it is usually the correct one.

Composition changes. If your customer mix shifted, an aggregate can move without any individual group changing, which the storytelling cursus covered as Simpson's paradox.

And the start date. Beginning a chart at a low point makes any subsequent movement look like growth. Choosing the start deliberately is legitimate; choosing it because it flatters is the manipulation this catalogue warns about.

The practical protections. Always plot enough history to see the normal variation, ideally two full seasonal cycles. Compare like periods. Check whether a step change coincides with a system or definition change before explaining it commercially. And state the start date's rationale if it is not obvious.

A trend chart with two years of history and year-on-year comparison is a genuinely informative object. One with six months and a convenient start is an argument dressed as evidence.

Splitting a measure by group is the most consistently valuable analysis available, because aggregates hide almost everything that matters.

What it does. An overall conversion rate of three percent tells you nothing about what to fix. The same figure split by traffic source, device, product and customer type will usually show one segment far worse than the rest, and that segment is the thing to work on.

The general finding, which holds across most businesses: problems are concentrated rather than uniform. A retention problem is usually a retention problem in one segment. A cost overrun is usually one category. Averages spread a specific problem across everything and make it look like a general condition requiring a general fix, which is why general fixes so often achieve nothing.

How to do it well.

Split by the dimensions where you could act differently. Splitting by something you cannot change produces an interesting fact and no decision.

Show the size of each group alongside the rate. A segment with a terrible conversion rate and eleven visitors is not your priority, and rate-only breakdowns hide this constantly.

Go one level deeper than feels necessary. The finding is often at the second split rather than the first.

And beware of finding a pattern by trying many splits. If you test fifteen dimensions, one will look striking by chance. Treat a segment discovered by exhaustive splitting as a hypothesis to check on new data rather than a finding, which is the same multiple-comparisons problem the A/B testing cursus covers.

The practical instruction. When any aggregate looks wrong or interesting, break it down before explaining it. Most explanations offered for aggregate movements dissolve on contact with a segment breakdown.

A funnel counts how many people reach each stage of a sequence, and it answers which step to fix. It also misleads in three specific ways.

The structure. People arrive, some proceed to the next stage, some drop out, and the sequence continues. The largest drop is usually where attention goes.

The first trap is that the largest drop is not necessarily the biggest opportunity. A stage losing eighty percent of people may be an inherent filter, where those people were never going to buy. A stage losing fifteen percent late in the sequence may represent people who were ready and hit an obstacle, and recovering them is worth far more per person.

The second is that funnels imply an order that users do not follow. Real behaviour includes going backwards, skipping stages, arriving in the middle from a link, and returning three weeks later on a different device. A strict funnel counts these as drop-outs, which overstates loss at every stage.

The third is the time window. If you count people entering and leaving within the same day, everyone with a longer decision cycle appears as a drop-out. Long-consideration purchases look catastrophic in a same-day funnel and fine in a thirty-day one.

So the useful practice. Report the funnel with a stated window matched to your actual decision cycle. Distinguish stages that filter from stages that obstruct. And check whether the people who dropped out came back later, because a large share of apparent losses are delays.

Cohort analysis is the least used of these patterns and frequently the most revealing, because it separates changes in your product from changes in your mix of customers.

What it is. Group customers by when they started, then track each group's behaviour over the months since they joined. A cohort is everyone who signed up in March; you follow their retention at month one, month two, and so on, and compare against the April cohort at the same ages.

Why this matters more than an overall retention figure. Overall retention mixes together customers of every age. If you grow quickly, your average is dominated by new customers, who behave differently from established ones. Overall retention can fall while every cohort improves, simply because you acquired a lot of new people. And it can rise while every cohort worsens, if acquisition slows and your base ages.

That means overall retention answers almost nothing about whether the product is getting better, which is usually the question being asked.

What a cohort view shows that nothing else does.

Whether recent customers behave better than older ones at the same age, which is the actual test of whether changes worked.

Where in the lifecycle people leave, which is frequently concentrated in the first period and implies onboarding rather than product.

And whether retention curves flatten, which tells you whether you have a stable base at all.

The practical caution. Cohorts get small quickly when split further, and a cohort of forty people produces noisy percentages. State the cohort sizes alongside the rates, which is the same discipline as the breakdown step.

Every pattern in this lesson is descriptive, and the most common failure is treating a description as a cause.

What these analyses establish. That something changed. Where it is concentrated. At which stage people leave. How groups differ. What the spread looks like. All of that is factual and useful.

What they do not establish. Why. A breakdown showing that mobile converts worse than desktop does not tell you that mobile is the problem. Mobile users may be different people, arriving from different sources, with different intent. The device is correlated with the outcome and may cause none of it.

The general form of the error is familiar and worth restating in this specific context. Two things vary together for three possible reasons: the first causes the second, the second causes the first, or something else causes both. Descriptive analysis cannot distinguish them, and the third case is far more common in business data than people expect, because almost everything is correlated with customer type, acquisition channel and time.

What you can honestly say from these patterns. Retention is lower in this segment. Drop-out concentrates at this stage. Recent cohorts behave differently from older ones. Each of those is a fact about your data.

What you should then do. Treat it as a hypothesis about a cause, and either test it properly, which is the subject of this catalogue's cursus on knowing what actually works, or act on it while explicitly acknowledging you are acting on a correlation.

That acknowledgement is not weakness. An organisation that knows it is acting on a plausible correlation will check whether the action worked. One that believes it established a cause will not.

A step that precedes all of this and receives almost no attention: deciding which measures an organisation watches.

The common failure is measuring what is easy to collect. Web analytics produce dozens of metrics automatically, so organisations report page views and sessions, which almost never correspond to anything anyone can act on. Meanwhile the thing that matters, whether customers got what they came for, is not instrumented because it would require deciding what that means.

Some principles that produce better measure selection.

Prefer measures where you can name the action a change would trigger. If nothing would happen differently at any value, it is a number rather than a metric.

Prefer measures closer to the outcome you actually care about, even when they are harder. Revenue per customer is closer to the point than clicks, and closer measures resist gaming better.

Pair any measure that can be gamed with one that constrains it. Speed of resolution paired with reopened tickets. Volume of output paired with a quality measure. This is the practical response to Goodhart's law, which the nonprofits cursus covered: a single measure as a target degrades, and a pair is much harder to game in both directions at once.

Keep the set small and stable. Twenty metrics means nobody watches any of them, and changing definitions destroys your ability to see trends, which are the most valuable thing you have.

And instrument what you are about to change, before you change it. The most common analytical regret is not having a baseline, and it is entirely preventable by spending ten minutes before the launch rather than a week after it.

What someone should actually take from this cursus.

The question comes first, restated as population, measure, period and comparison. Then find out what your data means, because it was collected for an operational purpose and every field carries that purpose's assumptions. Then look at it, before summarising it.

The technical layer is small. Tidy data, pivot tables, joins with row counts checked before and after, and lookups with exact matching and match rates verified. That covers most business analysis, and the errors in it are silent rather than loud, which is why the checks have to be habitual.

The analytical layer is five patterns. Trend, watching for seasonality, unequal periods and definition changes. Breakdown, which is where problems actually live and which should be run before explaining any aggregate. Funnel, with a stated time window and a distinction between filtering and obstructing. Cohort, which is the only one of these that tells you whether things are genuinely improving. And distribution, which tells you whether an average means anything at all.

And the interpretive limit. All of it is descriptive. It establishes what is happening, not why, and treating a correlation as a cause is the failure that turns good analysis into confident wrong action.

The honest summary of the whole cursus. Nothing here is sophisticated, and that is the point. The gap between organisations that make good use of their data and those that do not is almost never a gap in technique. It is that one of them restates the question, checks the row counts, looks at the segments, and says what it does not know, and the other produces numbers.