Can AI screen candidates
fairly and well?

“AI in hiring” is not one decision.

A system might draft a job description, search for applicants, parse résumés, rank candidates, transcribe an interview, score spoken answers, or recommend who advances. A scheduling assistant has a different effect from an automated rejection. A summary for a recruiter is different from a score that changes someone’s place in the process.

Start by mapping where the system acts, which information it uses, and what happens to a person after the output. That makes the claim testable: what job-related skill is being measured, and does the result improve the decision at that step?

Reliability is one part of validity.

A scoring system can give similar ratings every time and still measure something irrelevant. Useful evidence asks whether the task is defined clearly, whether the questions let candidates demonstrate that capability, and whether scores relate to outcomes the organization actually needs.

A 2026 study in the Journal of Applied Psychology examined large language models as interview raters using two datasets: interview responses about personality traits from 954 people, and job-related answers from 144 people scored with behaviorally anchored rating scales. The researchers varied evaluation choices such as model, prompt, settings, and number of raters, and examined reliability, several kinds of validity evidence, group differences, and measurement bias.

Under some carefully specified conditions, ensembles of newer models with detailed prompts showed potential for psychometric performance comparable to or better than the study’s human and supervised-model comparisons. That is evidence that a defined scoring task can be studied rigorously. It does not establish that a general chatbot can rank applicants across occupations, predict job performance in every workplace, or produce fair hiring outcomes by itself.

Small changes in the setup can change the finding.

A 2025 study tested 11 language models on résumé and interview materials from applicants to K–12 teaching jobs in one U.S. school district. The researchers also made controlled versions of applications with different names and pronouns to test whether demographic signals changed recommendations while the underlying application remained the same.

In those counterfactual tests, many of the models gave slightly higher scores to synthetic female and non-White applicants than to comparison applicants. The estimated differences were generally a few percentage points at the thresholds tested. In a separate selection-rate analysis, the direction and size of group differences changed with the cutoff; some estimates were imprecise, and the authors explicitly warn that this particular pattern may not generalize to other systems or settings.

The useful conclusion is methodological: look at the decision rule, not just the average score; test the exact job and candidate pool; and investigate signals that could stand in for protected characteristics. This study does not establish the outcome of any employer’s deployed system, and it is not a universal ranking of current models.

Bias can enter before or after the model.

A 2025 qualitative study of Australian employers identified several ways AI hiring systems can create discrimination risks in practice: historical data and proxies, how a tool is configured and used, new barriers in the selection process, failure to provide reasonable adjustments, and intentional misuse. Because it studied employer practice qualitatively, it maps possible pathways; it does not estimate how often each one occurs.

That wider view matters. A well-tested model can still be used with an unsuitable job description, an inaccessible interview format, a threshold chosen without evidence, or an automated cutoff nobody can question. A human reviewer helps only when they can see the evidence, have time to check it, and can change the result.

Ask for evidence at every decision point.

1. Define the job task: What capability should this stage measure, and why is that information relevant to the role?
2. Inspect the input: Does the tool use qualifications and job-related answers, or proxies such as accent, camera behavior, school name, or employment gaps?
3. Validate the measure: Are scores reliable and meaningfully related to the defined capability and the task’s real outcome?
4. Test the threshold: Who advances at each cutoff? Compare false rejections and missed qualifications, not only an aggregate accuracy score.
5. Check groups and access: Examine relevant subgroup results and uncertainty. Test accommodations and alternative ways to demonstrate the same skill.
6. Preserve human review: Can a qualified person inspect the original evidence, correct a bad score, and reverse an automated decision?
7. Protect candidate data: Record what is captured, who can access it, how long it is kept, and whether it is reused for another purpose.
8. Repeat after change: Tie evidence to the model, prompt, job, and workflow tested; rerun checks when any of them changes.

Fair hiring is not a single model property. It is a measured quality of the selection process, including how candidates can participate, how evidence is interpreted, and how people can challenge a consequential mistake.

Follow the evidence into accountability.

For broader ways to test uneven outcomes, read how to measure AI bias ↗ and what explainable AI can tell you ↗. For organizational ownership and European rules, see AI governance ↗ and the EU AI Act guide ↗.