Vector Institute's UnBias-Plus: An Open-Source Tool to Catch AI Bias

Toronto's Vector Institute released an open-source tool Tuesday that scans text and AI training data for biased language. The tool, called UnBias-Plus, flags language tied to race, gender, age, and political framing, then explains why it was flagged and suggests neutral alternatives.

“The people most harmed by biased language are often the last to know it's there,” Shaina Raza, a Vector Institute applied machine learning scientist, said in a statement. “A patient doesn't see the assumptions buried in their clinical notes. A job candidate doesn't know why a door keeps closing.”

Large-language models train on human-generated data, which means they can replicate social biases. Algorithmic hiring tools have been shown to recreate systemic bias against Black and Asian applicants in the US. A London School of Economics and Political Science study found that AI tools used by UK councils downplayed the severity of women's health issues compared to men's.

Vector's release targets those structural problems. The tool is free and designed to help Canadian organizations align with the country's national AI strategy, which identified bias as a challenge. Canada's new online harms legislation does not propose explicit fixes for bias in AI models. It imposes a “Duty to Act Responsibly” on social media and AI chatbot services, which includes mitigating the risk of exposing users to harmful content.