Machine Learning Engineer Interview Prep: Questions and a Mock Test

The framing that best predicts a strong candidate is treating the model as one component inside a system that has to keep working. That framing has an authoritative source. The 2015 NIPS paper Hidden Technical Debt in Machine Learning Systems, by Sculley and colleagues at Google, argues that "it is dangerous to think of these quick wins as coming for free" and that "it is common to incur massive ongoing maintenance costs in real-world ML systems".

The paper names the specific risk factors, and the list is close to a syllabus for the systems half of an ML interview: "boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues, changes in the external world, and a variety of system-level anti-patterns".

Two of those come up constantly. Entanglement is the observation that changing one input changes the meaning of all the others, so no feature is genuinely independent and no change is genuinely local. Undeclared consumers are the teams silently reading your model's output, who will be broken by a change you thought was yours to make. Being able to name these and give an example from your own work is a strong signal, because it shows you have maintained a model rather than only shipped one.

Google's Rules of Machine Learning is short, free, and quoted often enough by interviewers that it is worth reading before a loop. Its opening position is deliberately deflationary. Rule 1 is "Don't be afraid to launch a product without machine learning", with the reasoning that "machine learning is cool, but it requires data". Rule 4 is "Keep the first model simple and get the infrastructure right", because "the first model provides the biggest boost to your product, so it doesn't need to be fancy".

This is a common trap in system design rounds. Asked to build a recommendation or ranking feature, many candidates reach immediately for a deep model. The answer that scores better usually starts with a heuristic or a simple baseline, spends its energy on the data path, logging, and evaluation, and treats model sophistication as the thing you buy once the infrastructure can tell you whether it helped.

The corollary matters too. If you propose a simple baseline, expect to be pushed on when you would move past it, and the answer should be tied to measurement rather than to fashion.

Skew is the gap between the features a model saw in training and the features it sees in production, and it is the single most common cause of a model that looked excellent offline and disappointed in production. Interviewers use it because it separates people who have deployed a model from people who have only trained one.

Rules of ML gives an unusually concrete prescription. Rule 29 states that "the best way to make sure that you train like you serve is to save the set of features used at serving time, and then pipe those features to a log to use them at training time". In other words, do not recompute training features from a warehouse using code that merely resembles the serving path; log what serving actually used. Rule 37 asks you to measure the skew explicitly, distinguishing performance on training versus holdout data, holdout versus next-day data, and next-day versus live traffic.

A strong answer names the three usual sources: different code computing the same feature in two places, different data sources with different freshness, and feedback loops where the model's own output alters the distribution it is later trained on. Mentioning a feature store as one structural answer is fine, provided you can say what problem it solves rather than treating it as a product name.

Evaluation questions are where interviewers find out whether you can be trusted with a number. Several failures recur.

The first is accuracy on an imbalanced problem. At one percent positives, a model that predicts negative always scores 99 percent, so accuracy is not merely a weak metric here, it is actively misleading. Precision, recall, F1, and the tradeoff between them are the expected vocabulary, along with the ability to say which error is more expensive in the specific business context you were given.

The second is leakage. If a feature encodes information unavailable at prediction time, the offline score is fiction. Time-based splits rather than random splits, and grouping by entity so the same user cannot appear in both train and test, are the standard defences.

The third is confusing threshold-free and threshold-dependent measures. ROC AUC summarises ranking quality across all thresholds and is famously optimistic under heavy class imbalance, because the false positive rate is diluted by an enormous negative class. Precision-recall AUC is the better summary there. And whatever the offline metric, the senior move is to connect it to an online one and to say how you would run the experiment that establishes the link.

Google Cloud's MLOps architecture guidance is a useful frame for the operational round, because it names the maturity levels explicitly. Level 0 is a manual, script-driven, interactive process where data analysis, preparation, training and validation are all done by hand. Level 1 automates the pipeline to achieve continuous training, which requires automated data and model validation, pipeline triggers and metadata management. Level 2 adds CI/CD for the pipeline itself.

The distinction the guidance draws is worth carrying into an answer. Continuous integration here is "not only about testing and validating code and components, but also testing and validating data, data schemas, and models". Continuous delivery delivers a system, an ML training pipeline, that itself deploys another service. Continuous training is the property unique to ML systems.

On drift, be precise about which kind. Data drift is a change in the input distribution. Concept drift is a change in the relationship between inputs and the target, which is worse because inputs can look entirely normal while the model quietly becomes wrong. The monitoring answer follows from that: track input distributions and prediction distributions continuously, because labels usually arrive late or not at all, and treat a change in prediction distribution as a signal to investigate rather than as proof of anything.