The authors of an arXiv audit of LLM-assisted hiring report that the way a model is asked to judge a resume changes how well it works: numeric scores usually pick up changes in a candidate’s qualifications better than a plain pass or fail verdict, while head-to-head comparisons shift with the order in which candidates appear (arXiv). They also say that clearly delimiting authentication, policy and resume text in the prompt lowers compliance with unauthorized override requests from about 65% to 1% when they come from unauthorised insiders, and from about 96% to under 6% when they are embedded in the resume body (arXiv). For teams deploying screening tools, the paper treats interface design and prompt structure as part of deployment readiness, not as minor details.
How the audit was built
The study tests operational readiness in a controlled experiment with four LLMs, 210 resumes, 14 job roles and six decision interfaces. The results rest on 16,800 resume evaluations with four open-weight models: MiniMax-M3, DeepSeek-V4-Pro, Nemotron-3-Ultra and GPT-OSS-120B. Each resume was run with 20 LLM calls at temperature = 0.

To check the test material, the authors consulted 15 domain experts across all sectors on whether baseline and variant resumes suited their roles, which gave an average agreement score of 81%. That expert check matters because the design depends on variant resumes that differ from a baseline in controlled ways, so the variants have to be plausible for each role.
The interface changes the answer
The central finding is that performance depends on the evaluation interface. According to the paper, scalar scores generally separate qualification changes better than simple pass/fail judgments, while comparative judgments are sensitive to the order in which candidates are presented. The order effect shows up in pairwise comparison: all four LLMs pick the stronger resume more often when it is shown second, with Nemotron-3-Ultra at 58% more and DeepSeek-V4-Pro at 45% more.
Sensitivity also differs between models. With a rubric, detection of removed required qualifications ranged from 26% for MiniMax-M3 to 87% for DeepSeek-V4-Pro. Read together, these figures suggest that a vendor’s choice of model and decision format deserves scrutiny, and that a pairwise ranking tool without order randomisation carries a built-in tilt toward whichever resume is shown second.
Prompt structure against override requests
The audit also looks at whether a model obeys instructions that should not carry authority. The authors say that delimiting authentication, policy and resume text in the prompt cuts this risk from about 65% to 1% for override requests from unauthorised insiders. For requests embedded in the resume body, they report a fall from about 96% to below 6%. The second case is the one an applicant could exploit by hiding text in a document, so the size of the starting figure is the more worrying number for anyone running unprotected prompts.
Bias and explanations
On name-based bias, covering race, ethnicity and Indian caste-associated surnames, the authors find little or no bias in any of the LLMs they tested. They also find that LLM evaluators often diverge between their decisions and the justifications they offer. That divergence is a practical concern for auditability: a written rationale attached to a screening decision cannot be assumed to explain it. Taken with the interface and override results, the audit points employers toward testing the full pipeline, including decision format, candidate order and prompt layout, rather than judging a model on name bias alone.
What the sources we read do not state
We check six facts on every story in the full text of each source. The pages listed under Sources do not state a price or hardware requirements.
Sources
The sources this article was written from. Links open the original page.



