Take-home tests are failing everyone: completion, drop-off and signal quality

The take-home exercise was supposed to be the fair option. It removes whiteboard anxiety, it lets people work in their own environment, and it produces an artefact you can actually read. Every argument for it is reasonable.

We analysed 4,180 issued take-home assignments across 76 companies, tracking who accepted, who started, who submitted, and how the submissions eventually predicted performance on the job. The format is not delivering what it promises.

Nearly half never come back

Of assignments issued, 46% were never submitted. That is not a completion problem at the margins; it is the central fact about the format.

When we surveyed non-submitters, the most common reason by a wide margin was time, not difficulty, not offence at being asked, not another offer. They intended to do it, the week did not allow it, and by the time it did the process had moved on.

Figure 1 shows where people leave. Note that the sharpest narrowing happens between invitation and submission, which is to say before anyone has evaluated any work at all. Whatever this stage is filtering for, it is doing most of its filtering before the assessment begins.

The stated estimate is fiction

Completion rates track the stated time estimate almost linearly. A brief promising "about 4 hours" got 71% completion. A brief promising "about 8 hours" got 38%.

More damning: actual median time spent exceeded the stated estimate in 84% of assignments. A four-hour brief took a median of 6.4 hours. An eight-hour brief took 13.5. Candidates are consistently being asked for roughly 60% more time than they agreed to, and the ones who cannot absorb that overrun are the ones who quietly disappear.

We do not think this is dishonesty on the part of hiring teams. It is the standard planning error, made worse because the person writing the brief already knows the codebase and the intended solution.

Whatever this stage is filtering for, it is doing most of its filtering before the assessment begins.

Section 1: Nearly half never come back

Who gets filtered out

This is the part of the analysis that changed our own practice. Drop-off was 2.4 times higher among candidates who reported caregiving responsibilities and 1.9 times higher among candidates currently employed full-time.

Both groups are, on average, no less capable. They have less discretionary evening time. A stage that demands six to thirteen unpaid hours in a week selects powerfully for people with spare capacity, which correlates with being early-career, unattached, unemployed, or supported by someone else.

The format that was adopted to make hiring fairer turns out to have a sharply regressive filter built into it. It is just an invisible one, because the people it excludes never appear in your candidate data. They are a gap in the funnel, not a rejection you can review.

And the signal is mediocre

If take-homes bought exceptional predictive power, there would be a real trade-off to argue about. They do not. Take-home scores correlated with six-month performance at r = 0.29 in our matched subsample.

A structured 90-minute pairing session on a realistic problem hit r = 0.44 while asking for a quarter of the time. A written design memo, roughly an hour, hit 0.38. Figure 3 plots signal against candidate time cost, and the take-home sits in the worst quadrant: expensive and mediocre.

Our reading of why is that a take-home measures the artefact and not the process. You see the final commit, not the reasoning, the dead ends, or how the person responded to a constraint changing halfway through. Those are the things that predict senior performance, and they are exactly what a pairing session surfaces.

If you are going to run one, pay for it

Paid take-homes in our sample had an 89% completion rate against 54% unpaid, at a median cost of $180 per completed assignment. The candidate pool that completed them was measurably broader on every dimension we could measure.

Against the mis-hire cost modelled in ER-2026-05, $180 is not a real number. If a take-home is genuinely the right instrument for your role (and for some, such as long-horizon architectural work, it may be) pay for the time and shorten it to three hours. If you cannot justify paying for it, that is useful information about how much you actually valued the signal.