Fairness: the AI follows the hiring team's criteria

We built a test that gives the AI the same CV again and again, changing only the name, the gender or an age signal. If the AI were biased, the score or the place in the ranking would move. Last run: 5 November 2026.

This is our own internal audit. It is not an independent or legal audit (see the scope below).

We scored the exact same CV text again and again to measure that normal variation: a candidate moved in or out of a job's top 3 in 0.3% of comparisons. Changing only the gender: 0.3%. Changing only the age: 0.3%. On average, the groups were at most 0.7 points apart on a 0 to 100 score.

Lower down the list, places moved in about one comparison in five: 17% for the same text, 19% when only the gender changed, 21% when only the age changed.

The age gap has a direction. The version born 15 years earlier scored a little lower in 411 of 1,100 pairs and higher in 160, and for early-career CVs the difference was larger, up to 4.5 points on average for one junior CV. It did not change the shortlists more than normal variation does.

Scores are 0 to 100.

In 2,200 profile rankings (200 Scout lists of 11), changing only the gender or age did not change who reaches the top 3 any more than Scout's own normal variation does.

We ran the same test on Scout's live ranking: the same built profiles, the job's criteria, and one thing changed at a time. Scout never sees the name, so we changed the pronoun and the year of birth. When Scout ranked the exact same list again, a candidate moved in or out of the top 3 in 5.8% of comparisons. Changing only the gender: 3.5%. Changing only the age: 5.8%. On average, the groups were 0.26 points (gender) and 0.17 points (age) apart on a 0 to 100 score. Lower down the list, places moved in about one comparison in three: 37% for the same text, 38% when only the gender changed, 41% when only the age changed.

1. 55 test CVs we built for this audit, modelled on real roles in dentistry, sales, software engineering, nursing and customer support: strong, middle and weak fits for each job. Built profiles mean no real candidate's data is used, and only one thing changes at a time.
2. Each CV gets names from 10 origins, female and male, and two years of birth: 2,200 tests in all.
3. Every version is scored by the same AI match recruiters see in MetaDay Hiring, with the same instructions.
4. What the AI sees. Before scoring, the name is removed from the CV and the profile, and no profile photo is sent. The pronoun (she or he) and the year of birth stay in the text, on purpose: they are the signals the gender and age tests check. So two name versions with the same pronoun and year of birth reach the AI as the exact same text. For the name test, this proves the name never reaches the AI (we check every input), and any score difference between name groups is the AI's normal variation.
5. We compare each CV's scores across its versions, and the order of the candidates inside each job.
6. It passes when no group (gender, name origin, age) is on average more than 3 points apart, and changing only the gender or only the age moves a candidate in or out of a job's top 3 no more often than scoring the same text again does (with a margin of 1 percentage point). We set this rule before the run.
7. We run it again after any change to how the AI applies the criteria, and plan a monthly run. Each result is kept with its date and published here.

Earlier runs

1. 4 November 2026

   A first, smaller run, passed under our first rule (no candidate changes place at all).

2. 5 November 2026

   A larger run stopped part way when our budget for the scoring ran out, and is not counted. It taught us two things. Candidates changed place even when only the name changed, and the AI never sees the name, so "no movement at all" was measuring the AI's normal variation, not bias. And our first age test changed the graduation year but not the job dates, which made a CV look like it had a career gap. Before running again we changed both: the age signal is now only a year of birth, and we compare shortlist changes with the AI's normal variation. We set the new rule before the 2,200 test run.

3. 5 November 2026Latest run

   Passed · 5 November 2026

• Names and profile photos stay out of scoring. When the AI applies the criteria to candidates in Hiring and in Scout, the person's name is removed first and no profile photo is sent: the scoring AI reads text only. It still reads everything the person brings: the CV, the public professional profile and public work such as GitHub, a portfolio or a personal website, for experience, skills, education and location fit for the job. Recruiters still see the name. A photo inside an uploaded CV file stays in that file unless it is removed: when a PDF or image CV is turned into text, that reading step can see it, and the score is based on the text.

• Conversations are judged on what was said. When the AI interviews a candidate, each answer is judged only on what the person said about their own experience and skills, with their own words quoted as evidence. Never on tone, accent, fluency, confidence, personality, age, gender, origin or any other personal trait.

• The AI applies the hiring team's criteria. The hiring team sets what counts for the job and how much. The AI applies those criteria and shows why each candidate got their score.

• A written fairness rule. The AI that applies the criteria is told to judge only skills, experience, education and the hiring team's criteria, and never to consider a person's name, gender, age, origin or photo.

• A repeat bias test. The test above (same CV, only the name, gender or age signal changed) checks that the ranking holds, after any change to how the AI applies the criteria. Our code tests also check that no name reaches the AI.

• Changing the ranking needs a written reason. When someone changes the ranking criteria or their weights, we ask why. Every change needs a reason written in their own words, and it is not saved until they write one. Picking from a short list can help them start, but it is never enough on its own. The only exception is the very first setup of a job or search, before anything was saved. We keep the reason with the change: who made it, when, what changed and why.

• A bias word check on the ranking. When someone writes or changes the ranking criteria, or the reason for a change, MetaDay checks the words as they type. Words that may read as age, gender, family, origin, religion, health or looks bias are flagged, with a short hint to rank on skills and evidence instead. It works in English, Dutch, German and Spanish, for every company. It warns and never blocks: the person decides, and sees why.

• People set the criteria and make the hiring decision. The AI applies the criteria and explains. It never hires anyone and never turns anyone down on its own. A company can let candidates move to the next step automatically under the rules it set. Even a missed must-have answer is flagged for a person to decide.