Building an AI job application agent

The tempting order is to start with the exciting part — the agent loop — and add plumbing later. That produces an impressive demo over a handful of hand-picked jobs and collapses when pointed at reality.

The order that works starts with the least interesting component, because it is the one most likely to make the whole thing unviable. If you cannot reliably get postings in, nothing downstream matters.

Each stage should be usable on its own before the next begins. Ingestion plus deduplication is already a better job feed than most people have; add scoring and it is a useful product without a single application being submitted, which is both a safer path and a faster one to something real.

• Ingestion first — can you get postings at all, reliably, every day?
• Then storage and deduplication — the same role appears many times
• Then extraction and scoring — is it worth applying to?
• Then tailoring — adapt your material to this posting
• Then submission — the part with permanent consequences
• Then tracking and feedback — the part that makes it improve

Every source has a different answer. Some job boards have APIs; many have feeds; most have neither and prohibit scraping in their terms. Company careers pages are often the most valuable source precisely because they are the least aggregated, and the most varied to parse.

Whatever you build, instrument it. The characteristic failure is silent: a source changes format, returns an empty list rather than an error, and your pipeline reports success while ingesting nothing. Alert on a source returning fewer results than usual, not only on a source erroring.

Settle what you are permitted to fetch before you build on it. A source added without checking its terms becomes a coverage promise to users that a legal review later forces you to withdraw, and by then people depend on it.

The same role appears across sources with different titles, different formatting and sometimes a different company name — an agency posting on the employer’s behalf. Scoring before deduplicating wastes tokens and, worse, can produce several applications to one job.

Exact matching on title and company catches almost nothing. Embedding the description and clustering by similarity, then confirming with a cheap model call on the ambiguous pairs, is the approach that actually works at reasonable cost.

Pick a canonical record per cluster and apply through that one. The employer’s own page is usually the best choice: it is most likely to be current, it accepts the application directly, and it avoids an agency intermediary the candidate never chose.

Extract the posting into a comparable structure, compare it to the candidate’s profile, and produce a score. The design decision that matters most is requiring a written reason alongside the number.

A bare score cannot be debugged or corrected. A score with "matches on Python and data pipelines, but asks for five years of Kubernetes you do not have" can be checked, argued with, and used as training signal for the preference model later. It also makes the output useful to a human deciding what to do.

Keep eligibility out of the score entirely. Right to work, location and a salary floor are gates that run before scoring, because the moment they become weights a confident skills match will eventually outvote one of them.

A daily agent without memory re-surfaces everything it saw yesterday, re-scores what it already scored, and eventually applies twice to the same role. The state is not an optimisation; it is what distinguishes the product from a scheduled script.

Persist a seen set, an application record keyed by candidate and normalised role, the exact documents sent, and the reasons for every rejection. That last one is quietly the most valuable, because it lets a candidate notice their own settings are wrong.

Make every outward action idempotent and check the record before acting rather than after. Retries are inevitable, and a duplicate application is visible to the employer in a way an internal error never is.

Cap things that accumulate. Applications per day, applications per employer, spend per week — all in code, all surviving a restart, because a bug that runs for six hours should produce a contained mess rather than an unbounded one.

• A seen set, so a daily run is not a repeat
• Application records keyed by candidate and normalised role
• The exact documents sent, stored immutably
• Rejection reasons, not just rejections
• Caps that survive restarts

Everything up to a submitted application is reversible. The submission itself is not: it is a permanent record at a company the candidate wanted to work for, with no undo and no second first impression.

So the architecture should be aggressive about autonomy before that line and absolute about approval at it. Search, dedupe, score, tailor, pre-fill the form — all unattended. Submit — a person confirms, seeing exactly what will be sent.

Design the confirmation so it is actually read. A diff against the candidate’s base material with unverifiable claims flagged gets attention; a dialog asking "submit this application?" gets clicked, and a gate nobody reads transfers accountability without providing oversight.

The component teams skip is feedback. Without it the agent applies to the hundredth job exactly as well as it applied to the first, because nothing it learns from outcomes reaches its decisions.

Record what was sent, what came back, and how long it took. Then feed that back into scoring so roles resembling the ones that got replies rank higher. This is the difference between an automation and a system that gets better, and it costs less to build than the ingestion layer did.

Be honest about what one person’s outcomes can support. Tens of confounded results cannot explain why an application failed, so the reliable learning is the concrete kind — corrected preferences, learned form structures, employer response behaviour aggregated across users — rather than inference from a thin sample.