Six question sets for the employer deciding what to ask: screening, problem framing, modeling and evaluation, production and MLOps, cost and small-team fit with a paid work-sample brief, and a weighted scorecard. Every question states why it is worth asking and what a strong answer sounds like. Download as DOCX.
The first time I sat across from a machine learning engineer candidate, I had no way to tell whether the answers were good. He talked fluently about architectures I had read about and nothing else, and I left the room with an impression rather than evidence. What I eventually learned is that the questions which actually separate candidates are ones I could grade all along, because they are about judgment and shipped work rather than math.
At FirstHR, we build for small companies that make senior technical hires without an HR department and without a panel of specialists to lean on. This page is written for the person deciding what to ask, not for the candidate preparing to answer. Every question below carries the reason it is worth asking and what a strong answer sounds like.
There are six downloadable sets: a 30 minute screen, problem framing and data, modeling and evaluation, production and MLOps, cost and small-team fit with a paid work-sample brief, and a weighted scorecard with red flags and reference questions. If you have not written the posting yet, start with the machine learning engineer job description and come back here.
TL;DR
Interview a machine learning engineer on five things: problem framing, evaluation honesty, production ownership, cost awareness, and plain-language communication. The two highest-value questions are how would you prove this model is good enough to ship, and tell me about a production incident involving a model. Both are gradeable without an ML background. Add a paid work sample on your own data and score every candidate on the same rubric. Download six question sets and the scorecard as DOCX.
What the Interview Has to Test
The interview has to test five things: whether the candidate can frame a business problem as a prediction, whether they evaluate honestly, whether they can put a model in production and keep it working, whether they know what it costs, and whether they can explain any of it to you. Modeling depth matters, but it is rarely the thing that decides the hire.
That ordering surprises people. It comes from where these projects actually fail at a small company. The model is usually fine. The target was defined badly, the training data did not look like production data, nobody agreed what good enough meant, or the thing shipped and quietly rotted because no one was watching it.
So the questions on this page are weighted toward judgment and ownership. A structured interview makes that weighting real: the same core questions for every candidate, scored on the same rubric, with the answers written down before anyone starts arguing about who they liked.
Which Machine Learning Engineer Are You Hiring?
Decide which version of the role you need before you interview anyone, because the title covers at least two different jobs. A research-leaning engineer trains and tunes models. A platform-leaning engineer takes models into production and keeps them running. At a small company the same person usually does both, plus the data pipeline underneath.
What you need done
Research-leaning
Platform-leaning
Chooses, trains, and tunes the model
Starts from a hosted or pretrained model
Designs the offline evaluation
Owns serving, versioning, and rollback
Builds the pipeline that feeds the model
Fits a company with no ML infrastructure yet
Most small businesses need the platform-leaning version and interview for the research-leaning one, which is how a mis-hire happens without anyone being underqualified. Write down the three things this person must deliver in the first year, then weight the question sets to match. If the honest answer is that you mostly need pipelines, a data engineer may be the better hire.
The Six Question Sets
The sets below run in the order a real process runs: screen, then framing, then modeling, then production, then fit and cost, then the scorecard. Each set states why the question is worth asking and what a strong answer sounds like, so the notes work for an interviewer who does not build models.
Screening Set
First 30 minutes
Has this person shipped a model that real users depend on, or only trained one? Seven questions that answer it before you spend anyone else’s time.
Problem Framing and Data
Where projects fail
Turning a business problem into something a model can predict, plus labels, leakage, and what to do when you do not have enough data.
Modeling and Evaluation
Can they prove it works?
Baselines, metrics and what they hide, time-aware splits, and error analysis. The set that separates an engineer from a script runner.
Production and MLOps
Who carries the pager?
Serving, versioning, rollback, drift, retraining triggers, and training-serving skew. At a small company this hire owns all of it.
Cost, Fit, and Work Sample
The budget question
What their last system cost to run, how they work with a non-technical owner, and a paid work-sample brief with a grading guide.
Scorecard and Red Flags
Score, do not guess
A weighted 1-to-5 rubric for a small company, a red-flag checklist, and five reference questions aimed at a former hiring manager.
Ask From Every Set, Not Just the Comfortable Ones
Candidates rehearse modeling answers most, because that is what the candidate-prep sites drill. The sets that separate an engineer who ships from one who studies are production and cost, where vague answers are the signal. Ask at least two questions from every set and score them, so a fluent answer in one area does not cover a hollow one in another.
40+ Questions and a Scorecard to Download
Download all six as a single Word document, or copy the sets you need. Each file lists the questions, the reason each one is worth asking, what a strong answer sounds like, and space for notes. The last file is the weighted scorecard with red flags and reference questions.
Download All Question Sets and the Scorecard
Five question sets plus a weighted 1-to-5 scoring rubric, red flags, and reference questions. All in one DOCX.
Set 1: Screening Questions (First 30 Minutes)
Seven questions that answer one thing before you spend anyone else’s time: has this person put a model in front of real users, or only trained one?
Screening Questions (First 30 Minutes)
MACHINE LEARNING ENGINEER INTERVIEW: SCREENING SET
Candidate: __
Interviewer: __
Date: __
HOW TO USE THIS SET
Run these in a 30 minute call before you spend anyone else’s time. The goal is
not to grade the math. It is to find out whether this person has put a model
in front of real users and lived with the consequences. Ask the same
questions of every candidate and take notes as you go.
QUESTIONS TO ASK
1. Tell me about a model you built that is running in production right now.
What does it decide, and who depends on it?
Why ask: it separates candidates who have shipped from candidates who have
only trained. Most resumes do not make the difference visible.
Strong answer: names the business decision the model supports, the people
or system consuming the output, roughly how long it has been live, and at
least one thing that has gone wrong since launch.
Weak answer: describes coursework, a competition placing, or a notebook
that was never wired to anything.
2. Walk me through that project from the first conversation to the day it
went live. Who asked for it and why?
Why ask: it shows whether the candidate starts from a business problem or
from a technique they wanted to try.
Strong answer: starts with the problem and the cost of the current manual
process, then explains the modeling choice as a consequence of that.
3. What would you need from us in the first two weeks to be useful?
Why ask: a candidate who has joined a small company before knows to ask
about data access, environments, and who owns the decision.
Strong answer: asks about where the data lives, what quality it is in, who
approves access, and what already runs in production.
4. What is the last thing you shipped that did not work, and how did you
find out?
Why ask: the answer reveals whether they monitor their own work or wait
for someone else to complain.
Strong answer: a specific failure, a specific detection route (an alert, a
drift check, a support ticket), and what changed afterward.
5. How do you decide whether a problem needs a trained model at all?
Why ask: the cheapest machine learning project is the one you talk the
business out of. This question finds engineers with that judgment.
Strong answer: reaches for rules, heuristics, or an existing hosted model
first, and treats training as the option you take when the simpler thing
has been measured and found wanting.
6. Which parts of the stack do you want to own, and which parts do you not
want to touch?
Why ask: it surfaces the fit question directly. A small company usually
needs someone who will own the pipeline and the deployment too.
Strong answer: honest about preferences and clear about what they can
cover anyway. Silence about deployment is a fit risk worth probing.
7. What does your first 90 days look like if we hire you?
Why ask: strong candidates have opinions about sequencing and will tell
you what they would refuse to do first.
Strong answer: something shipped and measured inside 90 days, not a
research plan or a platform rebuild.
NOTES
[Capture specific examples, systems named, and red flags here.]
Set 2: Problem Framing and Data
Turning a business problem into something a model can predict, plus label quality, target leakage, and what to do when you do not have enough data.
Problem Framing and Data Questions
MACHINE LEARNING ENGINEER INTERVIEW: PROBLEM FRAMING AND DATA
Candidate: __
Interviewer: __
Date: __
WHY THIS SET MATTERS
Most machine learning projects that fail at a small company fail before any
model is trained. The target was defined badly, the training data did not
look like production data, or nobody agreed what a good outcome was. These
questions test the part of the job that happens before the modeling starts.
QUESTIONS TO ASK
1. Here is a problem we actually have: [describe one in two sentences]. How
would you turn it into something a model can predict?
Why ask: this is the single most transferable skill in the role, and it
works even when the interviewer has no machine learning background.
Strong answer: asks clarifying questions first, proposes a specific target
variable, says what the model would output, and states how a business
person would use that output.
Weak answer: jumps to an architecture before establishing what is being
predicted or why.
2. How would you know if the labels you are training on are wrong?
Why ask: label quality is where small-company data usually breaks, and few
candidates volunteer it.
Strong answer: samples and reviews labels by hand, checks agreement
between labelers, and looks for classes where the model and the label
disagree most.
3. Tell me about a time the training data did not match what showed up in
production. What happened?
Why ask: it tests for real experience with distribution shift and leakage
rather than a textbook definition.
Strong answer: a concrete story with a detection method and a fix, often
involving a feature that was not available at prediction time.
4. What is target leakage, and how have you caught it?
Why ask: leakage is the most common cause of a model that looks excellent
offline and useless in production.
Strong answer: defines it as information in the training data that would
not exist at prediction time, then gives a real example they caught,
usually through a result that was suspiciously good.
5. How much data would you want before you would agree to build this, and
what would you do if we do not have it?
Why ask: small companies rarely have enough data, and you need to know
whether the candidate will say so or build anyway.
Strong answer: names the realistic alternatives, buying a hosted model,
using pretrained weights, starting with rules, or collecting labels
deliberately for a period before training anything.
6. How do you handle a class that is very rare, for example fraud or churn?
Why ask: imbalanced problems are the ones a small business usually has.
Strong answer: talks about the cost of each error type first, then about
sampling, thresholds, and metrics that survive imbalance.
7. Who decides what the model is allowed to do on its own, and who reviews
the cases it gets wrong?
Why ask: it tests whether the candidate thinks about the human process
around the model, which is where the business risk lives.
Strong answer: proposes a human in the loop for the expensive decisions
and a review queue with a named owner.
NOTES
[Capture specific examples, systems named, and red flags here.]
Still Using Spreadsheets for Onboarding?
Automate documents, training assignments, task management, and track onboarding progress in real time.
Set 5: Cost, Small-Team Fit, and the Work-Sample Brief
What their last system cost to run, how they work with a non-technical owner, and a paid work-sample brief with a grading guide you can use as written.
Cost, Small-Team Fit, and the Work-Sample Brief
MACHINE LEARNING ENGINEER INTERVIEW: COST, FIT, AND WORK SAMPLE
Candidate: __
Interviewer: __
Date: __
WHY THIS SET MATTERS
Compute is a line item and this hire controls it. A machine learning engineer
who has never seen a bill will design something a small business cannot
afford to run. This set also covers collaboration and gives you a paid work
sample brief that tells you more than a whiteboard exercise ever will.
QUESTIONS TO ASK
1. What did the last system you built cost to run per month, and how do you
know?
Why ask: it is the question that most cleanly identifies engineers who
have owned something rather than contributed to it.
Strong answer: a real number or a defensible estimate, the largest driver
named, and at least one change they made to bring it down.
Weak answer: has never seen a bill and has no instinct for the order of
magnitude.
2. Where would you cut cost first if I told you the budget was half?
Why ask: it tests engineering judgment under a real constraint.
Strong answer: reaches for batching, caching, a smaller model, or a
cheaper instance before reaching for a worse product.
3. How do you work with someone who does not understand what you built?
Why ask: at a small company that person is usually the owner, and the
relationship decides whether the work gets used.
Strong answer: gives a specific example of translating a result into a
decision, and describes checking that the other person actually followed.
4. What would you do if I asked for a model you thought was a bad idea?
Why ask: you need someone who will push back once and then commit, not
someone who quietly builds the wrong thing.
Strong answer: proposes a cheap experiment to settle the disagreement with
evidence rather than an argument about opinions.
5. How do you handle being the only machine learning person here?
Why ask: isolation is the real risk of this hire at a small company.
Strong answer: names how they get review, stay current, and avoid becoming
a single point of failure, including writing things down.
6. What is the part of this job you would most like to hand to someone else?
Why ask: honest self-assessment, and it tells you what the next hire is.
PAID WORK-SAMPLE BRIEF (4 TO 6 HOURS, PAID)
Give the finalist a small, anonymized slice of your own data and this brief.
Pay for the time at a fair rate and agree the scope in writing first.
Task: Here is a sample of our data and the decision we make with it today.
Propose how you would approach this, build the simplest thing that beats
our current manual rule, and tell us how you would measure it in
production. Time-box the work to [4 to 6] hours. Deliver a short write-up
and whatever code you got to.
What to grade:
•Did they define the target and the baseline before modeling?
•Did they notice the problems in the data we already know about?
•Is the evaluation honest, including what they did not have time to check?
•Could you follow the write-up without a machine learning background?
•Did they respect the time box, or did they hand back 20 hours of work?
What not to grade: the final accuracy number. On a few hours with a small
sample, the number means very little. The reasoning means everything.
NOTES
[Capture specific examples, systems named, and red flags here.]
Set 6: Scorecard, Red Flags, and Reference Questions
A weighted rubric built for a small company, an eight-item red-flag checklist, and five reference questions aimed at a former hiring manager rather than a peer.
Scorecard, Red Flags, and Reference Questions
MACHINE LEARNING ENGINEER INTERVIEW SCORECARD
Candidate: __
Interviewer: __
Date: __
Score each area from 1 (poor) to 5 (excellent) with a note that cites
something the candidate actually said. Score on your own before the group
talks, so the loudest voice in the room does not set the answer.
SCORING AREAS (WEIGHTED FOR A SMALL COMPANY)
Problem framing and business judgment Weight 3 Score: [ 1 2 3 4 5 ]
Notes: __
Evaluation and honesty about results Weight 3 Score: [ 1 2 3 4 5 ]
Notes: __
Production ownership and monitoring Weight 3 Score: [ 1 2 3 4 5 ]
Notes: __
Data engineering and pipeline skill Weight 2 Score: [ 1 2 3 4 5 ]
Notes: __
Modeling depth Weight 2 Score: [ 1 2 3 4 5 ]
Notes: __
Cost awareness Weight 2 Score: [ 1 2 3 4 5 ]
Notes: __
Communication to a non-technical owner Weight 3 Score: [ 1 2 3 4 5 ]
Notes: __
Weighted total: ______ / 90
RED FLAGS
•Has never named a business metric, only model metrics.
•Every project in their history was a success. Real applied work is not
like that, and the inability to name a failure is itself the failure.
•Cannot describe how a model of theirs reached a user.
•Talks about accuracy with no baseline and no cost of error.
•Has no opinion about when not to use machine learning.
•Has never seen a compute bill.
•Dismisses simple approaches without having measured one.
•Cannot explain their own work in plain language.
REFERENCE QUESTIONS (ASK THE HIRING MANAGER, NOT A PEER)
1. What did they build that is still running today?
2. Who used the output, and did the business change because of it?
3. What happened the first time something they built broke?
4. Would you have them own a system alone, with no other machine learning
person to review it?
5. What did they need the most help with?
SUMMARY
Overall recommendation: [ ] Strong yes [ ] Yes [ ] No [ ] Strong no
Key strengths: __
Key concerns: __
Interviewer signature: __
Three Questions That Do the Most Work
If you only have time for three questions, ask these. Each one is gradeable without a machine learning background, because you are judging the reasoning and the specifics rather than the technique.
How would you prove to me that this model is good enough to ship?
Why ask it: It is the one question that forces a candidate to connect model quality to a business decision, and you can grade the answer without any machine learning background of your own.
Strong answer: Names a baseline to beat, usually a rule or the current manual process. Picks an offline metric and explains the cost of each error type in your terms. Proposes a holdout that mimics production, then a small live test with a stopping rule agreed before it starts.
Weak answer: Quotes an accuracy figure with no baseline, no cost of error, and no plan to check the result on live traffic.
What did the last system you built cost to run per month?
Why ask it: Compute is a real line item and this hire controls it. The question cleanly separates engineers who owned a system from engineers who contributed to one somebody else paid for.
Strong answer: A real number or a defensible estimate, the biggest cost driver named, and one specific change they made to bring it down: batching, caching, a smaller model, or a cheaper serving tier.
Weak answer: Has never seen a bill, has no sense of the order of magnitude, and treats cost as somebody else’s problem.
Tell me about a production incident involving a model. What broke?
Why ask it: It produces the most honest answers of any question in the set, and the root cause tells you where the candidate real experience sits.
Strong answer: A specific incident, a root cause that usually lives in the data or the pipeline rather than the model, how it was detected, and a durable fix such as an input validation check or a rollback path.
Weak answer: Cannot name one, or blames an upstream team without describing what they changed on their side afterward.
The best follow-up to any of them is the same four words: what was the result? A candidate who has shipped has numbers and consequences ready. A candidate who has not retreats into what the approach would theoretically achieve.
What to Probe For (and the Red Flags)
The listed questions open the door; the follow-ups tell you what is behind it. Push for the specific number, the actual incident, the named system. The patterns below are what you are listening for underneath the answers.
Business judgment
Starts from the problem, not the technique
Will tell you when not to build a model
Ties every metric to a cost or a decision
Evaluation honesty
Names a baseline before a score
Says what the chosen metric hides
Can point to a project that did not work
Production ownership
Describes the path from model to user
Monitors drift, not just uptime
Has a rollback story, not a rollback theory
Red flags
An unbroken record of successes
Accuracy quoted with no baseline
Has never seen what the compute cost
One red flag deserves its own note. A candidate whose every project succeeded has either not done much applied work or is not telling you about it. Real machine learning produces models that never shipped, and an engineer who names one and diagnoses why it died is showing you exactly how they think about their own work.
Judging Skill Without an ML Background
You can evaluate a machine learning engineer without being one, as long as you grade the right signals. Four of them are legible to any owner, and together they track applied skill closely enough to make a hiring decision on.
Signal you can grade
What it tells you
Starts from the problem, not the technique
They will build what the business needs rather than what they wanted to try
Names a baseline before quoting a score
They measure honestly and can defend a result under pressure
Explains the work in plain language
You will be able to make decisions on their output for years
Can describe a project that failed
They have real applied experience, not a portfolio of demos
Knows what their last system cost to run
They owned it end to end rather than contributing to someone else’s system
Asks about your data before proposing anything
They know where these projects actually break
Beyond the interview, the highest-signal step is a short paid work sample on an anonymized slice of your own data. Time-box it to four to six hours, pay a fair rate, and grade the write-up rather than the accuracy number, which on a few hours of work tells you almost nothing. The brief is in Set 5, ready to send.
If you want depth you genuinely do not have, buy an hour of a contract engineer’s time to sit in on the technical round and score it alongside you. That is cheaper than a bad hire by two orders of magnitude, and it is a normal thing to do.
What This Role Pays
There is no federal occupation code for machine learning engineer, so no official median exists for the title. Benchmark against the nearest classifications that match your version of the role, then adjust for location, company stage, and equity before you open the compensation conversation.
Nearest Classifications: $120,230 to $140,300 Median (BLS OEWS, May 2025)
According to the Bureau of Labor Statistics Occupational Employment and Wage Statistics survey (May 2025), median annual wages were $140,300 for computer and information research scientists, $135,980 for software developers, and $120,230 for data scientists. The ranges are wide: computer and information research scientists earned $82,200 at the tenth percentile and $230,630 at the ninetieth (U.S. Bureau of Labor Statistics, OEWS national estimates).
Two practical notes. Federal figures lag the market for scarce specializations, and market data shows compensation for this work frequently running above these medians once equity is counted. And publish a good-faith range where pay transparency laws apply, which for a remote posting increasingly means most of the time.
Budget for the total cost rather than the salary alone, because this hire also decides what your compute bill looks like. The role is normally exempt under the computer employee exemption or the professional exemption, but classification depends on actual duties and pay rather than the title on the offer letter. Confirm it before the offer goes out.
Fair, Legal, and Structured Interviewing
Fair, legal, and structured are one practice described three ways. Asking the same job-related questions of every candidate keeps you compliant, reduces bias, and produces better hires at the same time. The EEOC publishes a plain summary of the practices that are prohibited under federal law, and it is worth reading once before you interview.
Ask about the work, not the person
Federal anti-discrimination law, enforced by the EEOC, prohibits basing a hiring decision on protected characteristics, and a question that probes one creates risk even when it is asked as small talk. Keep off age, race, religion, national origin, sex, pregnancy or family plans, disability, and genetic information. Technical interviews drift toward schools, degree programs, and graduation years more than most, and a graduation year is an age proxy. Visa and work authorization talk drifts toward national origin fast. Ask whether the candidate is legally authorized to work and leave it there. This is general information, not legal advice.
Same core questions, every candidate
A structured interview, where every candidate faces the same questions scored against the same rubric, predicts on-the-job performance better than a free-flowing conversation and reduces the chance the decision rests on rapport. This matters more for a machine learning hire than for most, because the subject is unfamiliar enough that an unstructured conversation drifts wherever the candidate is strongest. Fix the questions in advance, ask them in the same order, and write the answers down while they are fresh. The sets on this page are built to be used exactly that way.
If the model will touch people, ask about it
If your candidate will build anything that screens applicants, scores employees, sets prices for individuals, or decides who gets an offer, ask directly how they would test it for disparate outcomes across groups, and expect a strong candidate to raise the question before you do. Several states and cities now regulate automated decision tools used in employment, and enforcement interest is rising. A candidate who has never thought about it is not disqualified, but a candidate who dismisses the question is telling you something. This is general information, not legal advice.
Score independently, then discuss
When more than one person interviews, have each interviewer complete the scorecard alone before the group talks. This keeps the most senior or most technical voice from anchoring everyone else, which is the usual way a good candidate gets talked out of and a confident one gets talked in. For a machine learning role the anchoring risk is higher, because the person in the room who understands the subject best carries disproportionate weight. Compare the written evidence first, then argue about the gaps.
Structure Matters More When the Subject Is Unfamiliar
A structured interview, where every candidate answers the same questions scored against a consistent rubric, predicts on-the-job performance more reliably than an unstructured conversation. The gap widens when the interviewer cannot grade the subject matter, because an open-ended conversation drifts toward wherever the candidate is most fluent. If your hire will build anything that scores people, the NIST AI Risk Management Framework is a reasonable reference for what a careful process looks like.
Keep every question on the work, and watch the two traps this interview walks into most: graduation years, which are an age proxy, and visa small talk, which slides toward national origin. For the fuller list, see our guide to illegal interview questions. This is general information, not legal advice.
Interviewing an ML Engineer Without HR
A large company runs this candidate through a coordinated panel with a recruiter managing scorecards and three specialists grading the technical rounds. You are one person running the interview between everything else, and that reality changes what a good process looks like without making it a worse one.
Nobody on your side can grade the technical answers
A large company puts this candidate in front of a panel of machine learning engineers. You are one person who has read a lot and built none of it. That gap is real, and the answer is not to fake technical depth. It is to grade the things you can grade honestly: whether the candidate starts from a business problem, whether they name a baseline before a score, whether they can explain the work to you in plain language, and whether they can describe a failure. Those four signals correlate with real applied skill, and every one of them is legible to a non-specialist. Where you need depth you do not have, buy an hour of a contract engineer’s time to sit in on the technical round.
You are hiring for a job that has no standard definition
The title covers a research-leaning engineer who trains models, a platform-leaning engineer who deploys and monitors them, and at a small company usually one person doing both plus the data pipeline underneath. There is not even a federal occupation code for the title. So write down the three things this hire must deliver in the first year before you interview anyone, and weight the question sets to match. If the honest answer is that you mostly need pipelines and deployment, say so, and read the resumes accordingly. Half the mis-hires at this level are a mismatch of expectation, not a shortfall of skill.
One expensive hire, and no second machine learning person to review them
This is a senior, well-paid hire who will work without peer review, which raises the cost of getting it wrong and the value of structure in the interview. Use the same core questions for every candidate, score them on the weighted rubric, and call a former hiring manager rather than a peer for the reference. Then move fast, because strong candidates in this market are usually holding more than one offer. Once you choose, the work turns into hiring well: an offer they can accept the same day, the paperwork, and a first 90 days that produces something shipped. FirstHR handles that side for a small business, with e-signature on the offer, the new hire paperwork, and an onboarding workflow that includes data access approvals. Applicant tracking is coming soon to FirstHR.
The compensating advantage a small company has is speed. You can screen, interview, run the work sample, and send an offer inside a week, which matters more than process polish when a strong candidate is holding three offers at once.
From Interview to Hire
The interview is step one. Once you choose someone, the work turns into hiring well: a clear offer letter, the new hire paperwork, data access with the right controls, and a first 90 days that ends in something shipped rather than a research roadmap.
Fix the questions first
Choose the sets that match the version of the role you are hiring, and ask the same core questions of every candidate so the comparison is fair.
Run a paid work sample
Give the finalist a small slice of your own anonymized data, pay for four to six hours, and grade the reasoning rather than the accuracy number.
Score, then send the offer
Complete the weighted rubric independently, compare written evidence, then confirm scope, pay, and start date in writing with e-signature.
Settle data access on day one
Decide before the start date what the new hire can see, what has to be redacted, and who approves an exception, then have it acknowledged.
Data access is the part that catches small teams with this hire. A machine learning engineer needs real customer records to do the job at all, so decide before day one what they can see, what has to be redacted, and who approves an exception. Write it down, have it acknowledged as an onboarding task, and store the acknowledgment with the rest of the file. Applicant tracking is coming soon to FirstHR.
FirstHR connects the offer, e-signature, new hire paperwork, and onboarding workflow in one place, and keeps the signed documents and interview records on the employee profile. FirstHR is an onboarding and HR platform, not a machine learning or data tool, so pair it with whatever your engineer builds on. Applicant tracking is coming soon to FirstHR. For the rest of the process, browse the hiring templates.
Key Takeaways
Test five things: problem framing, evaluation honesty, production ownership, cost awareness, and plain-language communication.
The two highest-value questions are how would you prove this is good enough to ship, and tell me about a production incident.
Decide whether you need the research-leaning or the platform-leaning version of the role before you interview anyone.
A candidate with an unbroken record of successful projects has either done little applied work or is not telling you about it.
A four to six hour paid work sample on your own anonymized data tells you more than any whiteboard exercise.
Keep the core questions identical for every candidate, score on the weighted rubric with written evidence, then move fast on the offer.
Frequently Asked Questions
What should I ask a machine learning engineer in an interview?
Ask about work that reached real users, not about definitions. The five areas worth covering are problem framing (how they turn a business problem into something a model can predict), evaluation (how they would prove the model is good enough to ship, and what baseline they would beat), production ownership (how a model of theirs reached a user, and what they monitor afterward), cost (what their last system cost to run per month), and communication (how they explain a result to someone who does not work in machine learning). Two questions carry more weight than the rest: how would you prove this is good enough to ship, and tell me about a production incident involving a model. Both are gradeable by a non-specialist, because you are judging the reasoning rather than the math.
How do I interview a machine learning engineer if I am not technical?
Grade the things a non-specialist can grade honestly, and there are more of them than most owners expect. Does the candidate start from the business problem or from a technique they wanted to try? Do they name a baseline before quoting a score? Can they explain their own work in plain language without hiding behind jargon? Can they describe a project that failed and say why? Those four signals track real applied skill closely and none of them require you to grade the math. Beyond that, use a paid work sample on a small slice of your own data and read the write-up rather than the accuracy number. If you want depth you do not have, hire a contract engineer for one hour to sit in on the technical round and score it alongside you.
What is the difference between a machine learning engineer and a data scientist?
The split is set by each employer, but the common pattern is that a data scientist answers questions and a machine learning engineer ships systems. A data scientist frames the problem, runs the analysis or the experiment, and hands over a recommendation or a model. A machine learning engineer takes a model into production and owns the serving path, the monitoring, the retraining, and the cost. At a small company one person usually does both, plus the data pipeline underneath. Rather than interviewing against a title, write down the three things this hire must deliver in the first year and weight the questions to match. If most of the work is deployment and pipelines, say so in the posting, because a candidate hired against the wrong expectation leaves inside a year.
What are the red flags in a machine learning engineer interview?
The clearest red flag is a candidate whose every project succeeded. Real applied machine learning produces models that never shipped and analyses nobody used, and an engineer who can name one and diagnose why is more trustworthy than one with an unbroken record. Watch for four more. Accuracy quoted with no baseline and no cost of error, which means they have never had to defend a result. An inability to describe how one of their models reached an actual user, which usually means someone else did that part. No opinion about when machine learning is the wrong tool, which produces expensive projects a small business does not need. And never having seen a compute bill, which is how a small company ends up with a system it cannot afford to run.
Should I give a machine learning engineer a take-home assignment?
A short paid work sample beats both a take-home and a live coding exercise, and it beats them clearly for a small company. Give the finalist an anonymized slice of your own data, describe the decision you make with it today, and ask for the simplest thing that beats your current manual rule plus a note on how they would measure it in production. Time-box it to four to six hours and pay a fair rate for the time. Grade the reasoning: did they define the target and a baseline first, did they notice the data problems you already know about, is the evaluation honest about what they did not check, and can you follow the write-up. Do not grade the final accuracy number, which on a few hours of work means very little.
What does a machine learning engineer cost to hire?
There is no federal occupation code for the title, so no official median exists for it, and you have to benchmark against the nearest classifications. According to the Bureau of Labor Statistics Occupational Employment and Wage Statistics survey (May 2025), median annual wages were $140,300 for computer and information research scientists, $135,980 for software developers, and $120,230 for data scientists. The ranges are wide: computer and information research scientists earned $82,200 at the tenth percentile and $230,630 at the ninetieth. Federal figures also lag the market for scarce specializations, and market data shows compensation for this work often running above these medians once equity is counted. Budget for the total cost, including compute, not the salary alone.
How many interview rounds does a machine learning engineer hire need?
Two or three rounds is enough for most small companies, and more than that costs you candidates. Run a 30 minute screen on shipped work, then a longer technical conversation covering problem framing, evaluation, and production, then a paid work sample for the finalist only. Score each round on the same rubric while the answers are still fresh. The advantage a small company has over a large one is speed: you can screen, interview, and send an offer inside a week, and that matters more than process polish when a strong candidate is holding several offers at once. Resist adding a round to feel more confident. Add a paid work sample instead, because it produces evidence rather than another impression.
What questions are off limits in a machine learning engineer interview?
Avoid anything that probes a characteristic protected under federal law, which the EEOC enforces: age, race, color, religion, national origin, sex, pregnancy or family plans, disability, and genetic information. Technical interviews drift into two traps more than most. Schools and graduation years, because a graduation year is an age proxy and the conversation gets there through friendly small talk. And immigration status, because visa talk slides into national origin quickly. You may ask whether a candidate is legally authorized to work, and you may ask whether they can perform the essential functions of the job. Keep everything else on the work itself, and ask the same job-related questions of every candidate, which is the simplest way to stay both fair and consistent. This is general information, not legal advice.
