How to reduce bias in AI: why representative teams build better models

What is AI bias and why does it happen?

AI bias occurs when a model produces outputs that systematically favor or disadvantage certain groups. It shows up in hiring algorithms that filter out qualified candidates, in facial recognition systems that misidentify people of color at significantly higher rates, and in medical AI that underperforms on patient populations underrepresented in the training data.

The causes trace back through the pipeline. Biased training data teaches the model to replicate existing inequalities. Biased labeling encodes the assumptions of whoever is doing the annotation. Biased evaluation criteria miss failure modes that annotators never thought to look for. At each stage, the model learns from the choices humans made and reflects the limits of their perspective.

The Gender Shades project, conducted by Joy Buolamwini at MIT Media Lab, made this concrete. Facial recognition systems from major commercial providers showed error rates as low as 0.8% for light-skinned males, rising to 34.7% for dark-skinned females. The gap wasn't a software glitch. It was a data problem rooted in who built the systems and whose faces were used to train them.

Further reading: Buolamwini is one of several women leading the field of AI ethics research, and their work is essential reading for anyone building or evaluating AI systems. From Cathy O'Neil's Weapons of Math Destruction to Ruha Benjamin's Race After Technology, Kate Crawford's Atlas of AI, and Karen Hao's Empire of AI, these researchers have been documenting algorithmic harm long before it became a compliance requirement. PowerToFly's guide to AI ethics covers their work and explains why ethical literacy is becoming a practical career skill.

More recently, a 2025 meta-analysis published in the International Journal of Selection and Assessment examined 41 studies and found that candidates with non-standard accents are consistently disadvantaged in employment interview evaluations. That the bias has less to do with how easy someone is to understand than with the stereotypes evaluators bring to the table. AI-powered hiring tools inherit and amplify exactly those evaluator patterns, performing well for the populations they were implicitly designed for, and poorly for everyone else.

The homogeneous team problem

Approximately 78% of AI researchers are male, according to Stanford HAI's AI Index report. In fact, no country in the world comes close to gender parity among AI researchers and developers. This is a clear indication of who is making the decisions that shape how AI systems see the world.

When a team shares the same demographic background, educational path, and cultural reference points, they share the same gaps. What gets treated as "normal" in training data reflects the team's frame of reference. What gets flagged as an edge case depends on whose experience counts as the center. What gets missed entirely is the failure mode affecting a group no one on the team belongs to. Those blind spots often stay missed until they become a headline or a lawsuit.

This is also a leadership problem. Leaders who don't actively prioritize team composition that is representative of the world when making hiring and staffing decisions are making a choice about what their models will and won't see. Building for innovation in AI requires the range of perspectives that innovation actually depends on. PowerToFly's practical guide to AI ethics for leaders covers how to build that into your AI program from the ground up.

Good intentions don't catch blind spots. Talented, well-meaning teams produce biased models when they lack the range of perspective needed to catch what they can't see. The solution isn't a bias audit after the fact. You have to start with the root of the problem. Building teams diverse enough to surface blind spots during development is the only way to solve this issue.

What the EU AI Act means for biased models

Bias reduction has moved from a values conversation to a legal requirement. Under the EU AI Act, high-risk AI systems (those used in healthcare, hiring, credit assessment, and law enforcement) must be trained and tested with sufficiently representative datasets, and must be traceable and auditable throughout their lifecycle. Full enforcement obligations take effect August 2, 2026.

That auditability requirement has a direct implication for how companies source their training data and annotation work. Documented bias assessments and ongoing monitoring are now mandatory for high-risk systems. Organizations must be able to show regulators not just what data went into the model, but who produced it, with what credentials, and under what oversight.

Anonymous crowd annotation platforms can provide volume. They cannot provide the documented, defensible data provenance that EU AI Act compliance requires. When a regulator or board asks who shaped the model, "unknown workers" is not an answer that holds up.

For US-based companies, parallel pressures are building. New York City's Local Law 144 requires annual independent bias audits for AI used in employment decisions. California's automated decision system regulations took effect in October 2025. The compliance window is narrowing on multiple fronts.

What representative domain experts bring to model training

Building more representative AI teams isn't a DEI initiative bolted onto an AI program. It's a quality decision with measurable consequences for model performance.

Representation in annotation and evaluation

Who labels the data shapes what the model learns. An annotation team that reflects the diversity of the population a model will serve is more likely to catch the failure modes that a homogeneous team misses, not because they're trying harder, but because they bring different lived reference points to the work.

A clinician annotating medical AI who has experienced healthcare disparities firsthand brings something different to the evaluation than a clinician who hasn't. A legal professional who has seen how automated systems affect underrepresented communities evaluates a legal AI tool differently than one who hasn't. That difference shows up in training signal quality and ultimately in model outputs.

Domain depth plus lived experience

The most valuable AI training and evaluation work happens at the intersection of technical understanding and domain expertise. A nurse who has worked in emergency medicine for 15 years and understands how AI models process clinical data brings a combination that no amount of additional compute can replicate. The same applies to a financial analyst with direct experience of how credit systems affect different communities, or a paralegal who has seen firsthand where contract analysis tools fail.

PowerToFly's community of 80K+ AI-skilled professionals is 80% women and 70% BIPOC, spanning clinicians, lawyers, engineers, analysts, and GTM specialists across 190 countries. What that means in practice: training data produced by people who actually represent the populations these models serve.

Empathy as a design capability

Diverse teams don't just bring different data points. They bring a different capacity to anticipate how a system will feel to the people who use it, particularly people whose needs have historically been overlooked.

People who have had to navigate systems that weren't designed for them develop a sharper instinct for where those systems fail. That instinct is a design asset. It's what allows a team to ask not just "does this model produce the right output?" but "does it work for someone who doesn't look like us, speak like us, or move through the world the way we do?" That question catches failure modes before they ship.

This is why diverse teams tend to produce more creative, more resilient, and more human-centered AI solutions. Considering multiple perspectives from the start leads to better questions, broader test cases, and outputs that hold up across the range of people a model is actually meant to serve.

Multilingual and cross-cultural coverage

Models trained primarily on English-language, Western-centric data perform worse in global contexts. Cultural assumptions embedded in training data by a geographically concentrated annotation team don't disappear when the model is deployed in different markets. They surface as errors and compliance risk.

Building annotation and evaluation cohorts with genuine multilingual, cross-cultural coverage isn't optional for companies deploying AI globally. It's a baseline requirement for models that actually perform across the populations they're meant to serve.

Practical steps to reduce bias in AI through team building

Bias reduction as a people strategy requires deliberate action at each stage of model development.

Audit who is currently building and evaluating your models. Before adding a bias mitigation tool or running an audit, understand who is making the decisions that shape your training data. If your annotation and evaluation teams are demographically narrow, your model reflects that narrowness regardless of what technical guardrails you put in place.

Source annotators and evaluators from domain-qualified, diverse talent communities. Generic crowd annotation platforms optimize for volume and speed. For bias-sensitive AI applications in healthcare, legal, financial services, and hiring, domain expertise and demographic diversity are both selection criteria, not optional add-ons.

Build representative evaluation cohorts at every release, not just at initial training. Models drift. New data introduces new patterns. A bias audit at model launch doesn't protect you from bias that emerges as the model encounters data it wasn't trained on. Diverse evaluators at each release cycle catch regression before it becomes a problem.

Document your data provenance. Know who produced the training signal, with what credentials, under what oversight conditions. This documentation is both a compliance requirement under EU AI Act for high-risk systems, and a quality standard that holds up to scrutiny from regulators, enterprise customers, and boards.

Treat bias reduction as a continuous process. The companies that get this right don't run a bias check and move on. They build diverse, domain-qualified evaluation into the model development lifecycle: at training time, at evaluation time, and at every release.

FAQ

What is AI bias?

AI bias is the tendency of an AI model to produce outputs that systematically favor or disadvantage certain groups. It can result from biased training data, biased labeling, biased evaluation criteria, or a combination of all three, typically reflecting the blind spots of the teams that built the model.

What causes bias in AI models?

Bias enters AI models at multiple points: in the selection and composition of training data, in the decisions annotators make when labeling that data, and in the evaluation criteria used to assess model outputs. Each stage reflects the assumptions and perspective of the humans involved, which is why the diversity of those humans matters so much.

How does team diversity reduce AI bias?

Diverse teams surface failure modes that homogeneous teams miss. When annotators, evaluators, and model developers bring a wider range of backgrounds, lived experiences, and domain expertise to the work, they catch the edge cases and blind spots that a narrower team would overlook. Diversity doesn't replace technical bias mitigation. It makes it more effective.

What does the EU AI Act require regarding AI bias?

The EU AI Act requires that high-risk AI systems, including those used in healthcare, hiring, and credit assessment, must be trained on sufficiently representative datasets and be auditable throughout their lifecycle. Full enforcement obligations take effect August 2, 2026. Organizations must document who produced their training data, under what conditions, and how bias risk was assessed and monitored.How do you build a more representative AI team?Start by auditing who is currently doing your annotation and evaluation work. Then source from talent communities that are organized by domain expertise and demographic diversity, not just availability and cost. Build diverse cohorts into every stage of model development, not just at initial training, and document the composition and credentials of the people producing your training signal.

Building diverse, domain-qualified AI teams is how you reduce bias from the ground up. PowerToFly connects companies with verified domain experts across healthcare, legal, financial services, and more: 80% women, 70% BIPOC, spanning 190 countries. Learn how PowerToFly builds representative AI teams.