A job recommender, end to end

You cannot recommend from data you never captured, and the signals worth having are rarely captured by default. Clicks are logged everywhere; impressions usually are not, and without them you cannot tell disinterest from never having been shown the thing.

Log the negative and the neutral as well as the positive. A posting shown ten times and never clicked is information. A search that returned nothing is information. Most teams discover these gaps a year in, when backfilling is no longer possible.

Position matters as much as the impression itself. A role at the top of a feed and the same role at the bottom get very different attention, and comparing their click rates without accounting for where they appeared measures the layout rather than the candidate’s preference.

• Impressions, with position — not only clicks
• Searches that returned nothing or were immediately refined
• Saves and dismissals, which are stronger signals than views
• Application outcomes, however sparse and delayed
• Which model or ranking version produced each impression

Recomputing everyone’s recommendations on every request is expensive and mostly wasted, since the underlying profile changes slowly. Precompute a candidate set per user on a schedule and serve it instantly.

Then layer real-time signals on top. What someone searched for two minutes ago should influence what they see now, and that is a cheap re-ranking of an already-computed set rather than a fresh retrieval.

Batching needs an invalidation path or it becomes the stale thing it was meant to avoid. A changed target role, a new constraint or a submitted application should all mark the set for regeneration rather than waiting for tomorrow’s run.

An on-site feed can repeat itself harmlessly; a daily email cannot. Sending the same five roles three mornings running is how a job alert becomes an unsubscribe.

Track what each user has been sent and exclude it for a window. Cap the number of roles from one employer. And be willing to send fewer, or nothing — a thin honest digest survives longer than a padded one.

Frequency should follow supply rather than a schedule. In a field where six suitable roles appear a week, a daily email is four empty mornings and two thin ones, and a weekly digest of six is a better product than seven attempts to find something to say.

A new user has no behaviour, and this is the moment that decides whether they come back. Falling through to "most popular jobs" is the common answer and a poor one, because popular roles are popular with everyone and relevant to nobody in particular.

Use what they gave you at signup: their CV, their stated target role, their location. Content-based matching works from the first second, which is exactly why embeddings are the sensible backbone for a job recommender rather than collaborative signals.

The first session is also the cheapest time to collect preference data, because the user is engaged and expects to be asked. Three dismissals with a reason attached, gathered while they are looking at concrete roles, are worth more than a week of passive behaviour.

Click-through rate is the easiest metric and the one most likely to mislead. A feed optimised for clicks converges on roles at well-known companies with attractive titles, which produces engagement and not employment.

The metric that matters is applications that reached a human and produced a response, per user, per week. It is sparse and delayed, which is precisely why teams substitute clicks — and why a recommender tuned on clicks drifts away from the thing it exists for.

Hold out a control. A small proportion of users served a simpler ranking is the only way to know whether the last six months of improvements did anything, and it costs almost nothing to run compared with the cost of being wrong about it for a year.

• Clicks measure curiosity; applications measure intent; responses measure fit
• A permanent holdout group, however small
• Feed diversity tracked over time, not just relevance
• Coverage — what proportion of postings ever get recommended to anyone

A recommendation is only good if the application can go somewhere. Recommending roles from employers who never respond produces applications, satisfying your metrics, and no outcomes for anyone.

Fold employer behaviour into ranking: response rate, time to first response, how long postings stay open. Candidates cannot see this and it materially affects whether their effort was worth spending.

Postings that have been reposted repeatedly deserve particular scepticism. A role advertised every month for six months is frequently not a role, and a recommender that keeps surfacing it is spending its users’ effort on something the employer is not actually filling.