Where AI Fits in Product Management, and Where It Does Not

A product manager is accountable for building the right thing, which in practice means a bundle of quite different activities (the course on becoming a product manager covers the role itself):

• Discovery: interviewing customers, reading support tickets, sales notes and reviews to find problems worth solving.
• Synthesis: turning that pile of evidence into a few defensible claims about what users need.
• Specification: writing the documents that tell design and engineering what to build and why.
• Analysis: reading usage data and experiment results to judge whether it worked.
• Communication: updates, roadmaps and reviews, each rewritten for a different audience.
• Decision: choosing what not to build, and owning the outcome.

The team and the executives experience all of this as one thing. For deciding where AI helps, it has to be taken apart, because each piece responds to AI differently.

Mapping the bundle against what current language models actually change:

The pattern is the same one that runs through every profession in this series. The visible artefacts, documents and decks and summaries, get cheap. The parts that were never about the artefact, knowing the customer and owning the call, do not. A PM whose value sat in producing documents has a problem; one whose value sat in judgement gets time back.

The clearest field evidence comes from a pre-registered experiment with Boston Consulting Group, reported by Dell'Acqua, Mollick, Lakhani and colleagues in a Harvard Business School working paper. 758 consultants, about 7% of the firm's individual contributors, were randomly assigned to work with or without GPT-4 on realistic tasks.

The authors call the boundary a jagged technological frontier: tasks that look equally difficult to a person can fall on opposite sides of it.

Key idea: The danger is not that AI fails. It is that it fails on tasks that look just like the ones where it succeeds, and it fails fluently, so the output gives no warning.

The task that fell outside the frontier in that study asked for a recommendation to a CEO, drawing on interviews with company insiders and a spreadsheet of financial data. Getting it right required combining the two, and consultants who leaned on the model's fluent analysis were more likely to reach the wrong answer.

That shape, quantitative data plus qualitative context plus a recommendation, is a fair description of a product decision. So the product manager's job sits unusually close to the edge of the frontier:

• Inside it: summarising a long thread, drafting a release note, restructuring a spec, turning one update into three audiences.
• On the edge: synthesising twenty interviews, explaining a metric movement, recommending a priority.
• Outside it: anything that depends on context that is not in the documents you gave the model, such as a promise sales made, a constraint engineering mentioned in a hallway, or a customer's tone.

Gotcha: The model cannot tell you which side a task is on. You find out by checking its output against evidence it did not see, which is exactly the step that feels unnecessary when the output reads well.

The same study observed two patterns among consultants who used AI successfully, which the authors named after the way the work was divided:

Neither pattern is "let the model do it and check the result at the end". Both keep the person's judgement engaged throughout, either by owning whole sub-tasks or by steering continuously.

In practice: For each recurring task, decide in advance which mode it gets. The failures cluster in tasks that drift from centaur delegation into unexamined acceptance, because the delegation started working and the checking quietly stopped.

A second field experiment by Fabrizio Dell'Acqua, titled Falling Asleep at the Wheel, gave 181 professional recruiters 44 job applications to assess for a skill that was not obvious from the CV. Some worked with a high-quality AI recommendation, some with a deliberately weaker one, some with none.

The recruiters with the better AI spent less time per application, followed its recommendations more blindly, and made worse decisions than those given the weaker AI, who stayed alert, questioned it and improved over the course of the task.

The mechanism generalises to product work. When an assistant is usually right, checking it feels like wasted effort, so effort drops exactly where it was needed: on the cases the assistant gets wrong.

Gotcha: Your checking habit is strongest in the first weeks, when the tool is new and you distrust it. Build the checks into the workflow then, as fixed steps, because by the time the tool has earned your trust you will no longer notice skipping them.

A product manager handles material that rarely feels confidential in the moment and often is.

The consent point is the one teams miss. A participant who agreed to be recorded for your research did not necessarily agree to have their words processed by another company's model.

Key idea: The tool your company has approved, under a business agreement, is a different product from the consumer version of the same model, even when the interface looks identical.

Start where the frontier is well inside and the cost of a mistake is visible and small, then move toward the edge only with checks in place.

1. Communication: meeting summaries, stakeholder updates rewritten per audience, release notes. Errors are easy to spot and cheap to fix.
2. Spec critique: ask a model to attack a spec you wrote, listing missing edge cases and unstated assumptions. You stay the author.
3. Research synthesis, with traceability: only once every theme can be traced to the quotes behind it, as the next lesson sets out.
4. Analytics questions: only against defined metrics and with the query shown, as the final lesson covers.
5. Prioritisation and decisions: use AI to argue the other side, never to make the call.

Try it: For one week, log every task where you used AI and mark it inside, edge or outside the frontier after the fact, based on whether its output needed a correction you could only make with context it did not have. The log becomes your personal map of the frontier.