Five Takeaways from an AI Fairness Conference

A Growing Research Movement Toward Algorithmic Fairness, Accountability and Transparency

What you’ll learn in this article:

• The Fairness, Accountability and Transparency in machine learning community is growing.
• Predictive policing, risk assessments, criminal justice and race were prominent topics at this year’s event.
• Researchers, business and media need to distinguish between the terms “bias” and “un/fairness.”
• Fairness research by computer and data scientists is often conducted in theory, while the impact of AI on people’s lives is very real.
• Governments should consider procuring AI technologies through appropriate processes, not traditional procurement processes.
• Even at a largely academic conference like ACM FAT, industry is present.

We tend to hear a lot about why we should fear ill-conceived AI tools that perpetuate unfairness. But a small and growing group of researchers are diligently working to ensure that AI is more fair, accountable and transparent. RedTail observed them and their work at the annual Association of Computing Machinery Fairness, Accountability and Transparency conference held this week in a Superbowl-frenzied Atlanta.

Longtime FAT participants marveled at a sea of attendees – 560 according to event organizers. This was the largest FAT event so far, bigger than last year’s attendance of 525 and far larger than FAT 2016 in New York which drew around 200. Following a 2018 during which the mainstream consciousness awakened to the presence of artificial intelligence thrust into society with few ethical guardrails and virtually no regulatory constraints, perhaps it’s no wonder that interest in this event dedicated to Fairness, Accountability and Transparency research has blossomed.

FAT is not a conference where business execs from giant corporations wax poetic about the need for more ethical technologies.

But FAT, as it’s known to insiders, is not for the mainstream. And it is not a conference where business execs from giant corporations wax poetic about the need for more ethical technologies. FAT is for the computer and data scientists, the machine learning academics, and in smaller doses, the ethics and philosophy professors and law and policy wonks. People who think and care about how technologies affect culture and society, and in particular aim to stop or prevent adverse impact of algorithmic decisions on underprivileged groups were there. And that is why the crowd is relatively diverse, with lots of women and more black people than you’d see at most tech or law or advocacy events. It is global, too – Boise mingles beside Nairobi beside Brussels and Boston.

RedTail was created during the AI Ethics wake-up call that was 2018, in the hopes of illuminating and understanding this sometimes-insular ecosystem. The last few days in Atlanta, amid chilling temperatures, a city shut down by a cautious newly-elected governor (and SAINTS GOT ROBBED billboards purchased to stick it to LA Rams fans descending on the city for this weekend’s big game) this publication’s sole reporter soaked up a bit of what the FAT community is all about.

Here are some initial takeaways.

1. Criminal Justice and Race Loom Large

This being an event driven by academia, most sessions were based on papers representing research and studies conducted by those very speakers. As it has been in previous years, the use of algorithmic decision-making tools in law enforcement and predictive policing was a prominent theme, driving conversations about race and criminal justice. Attendees sat in on a race theory tutorial devised for the machine learning crowd. They learned about an approach to deriving machine learning data that removes stigmatizing race classifications borne of inequality. They heard firsthand from social justice advocate Reuben Jones about the damaging impact of AI-fueled law enforcement and work-shopped a communal approach to evaluating tools for pretrial risk assessment algorithms.

Attendees work-shopped a communal approach to evaluating tools for pretrial risk assessment algorithms.

2. Refining the Terminology of  “Bias” and “Un/fairness”

In the past year, we’ve seen countless articles decrying “biased” AI systems. Thing is, people who research and produce models and algorithms have a lexicon all their own. So when they employ the term “bias” it often has a completely different meaning than most of us realize.

Put very simply, in the technical sense, the term bias refers to statistical bias, a large gap in a model that over- or under-represents something. Think of an election poll in which mobile phone users are not represented. In such a case, the data set is biased because by ignoring mobile phones, it misses younger people who do not have landline phones.

Of course, when media, policymakers or business executives refer to AI bias, the word typically connotes prejudice, discrimination or unfairness. In general, this has led to a conflation in usage of the terms bias and un/fairness. “In corporate spaces these terms are highly interchangeable,” said Jacob Metcalf, an ethics consultant and researcher who advocated for the FAT community to distinguish between the terms more deliberately.

“Bias is a feature of statistical models. Fairness is a feature of human value judgments.”
– Jacob Metcalf, ethics consultant and researcher

“Bias is a feature of statistical models. Fairness is a feature of human value judgments,” he said during a session on the subject. In his research on the importance of the distinction, he noted, “With that distinction in place, it is easier to articulate how engineering for fairness is an organizational or social function not reducible to a mathematical description.” The FAT community itself has a ways to go on this, though. The common parlance heard on stage throughout the conference suggests that some still employ the term “fairness” when referencing algorithms that reduce statistical bias.

3. Fairness, in Theory

But there is another noticeable disconnect between this academic sphere and others in the AI Ethics community. It’s the gap between theory and practical application.

Outside of computer science circles, discussion around the need for more ethical AI is prompted by concerns of the impact of technologies that will touch our lives. We read and ponder reports of decision-making systems already active and fed by real-world data, enabling semi-autonomous vehicles to turn, or spitting out search engine results.

But on the inside, while decision-making mechanisms and research are presented and discussed among academics at conferences like FAT, the framework is overwhelmingly theoretical in nature. Researchers demonstrated models and algorithms that were proven to be more fair, more explainable, or even mitigate the need to use problematic race classifications in data. However, most of the research presented happened in the vacuum of closed environments. Systems were fed with training data, rather than active in the wild. They operated well — in theory.

“We can talk about the algorithms all day but these algorithms are just being used by people ultimately.”
– Ben Green, applied math PhD at Harvard

“We can talk about the algorithms all day but these algorithms are just being used by people ultimately,” said Ben Green an applied math PhD at Harvard during a chat with RedTail. Green aims to bring the theoretical work of the FAT community into the political and policy-making sphere in practical ways. Last year, he argued, “Data scientists must recognize themselves as political actors engaged in normative constructions of society and, as befits political work, evaluate their work according to its downstream material impacts on people’s lives.”

4. Better Vetting for AI Vendors

Yet, there was a smattering of more practical discussions and empirical research presented. Among the more tangible discussions of the event was one led by keynote speaker Deirdre Mulligan, an associate professor in the School of Information at UC Berkeley, who highlighted an important-yet-overlooked trend in the use of automated systems by governments.

Most federal, state or municipal government agencies in the U.S. that are evaluating or have already purchased algorithmic decision-making systems for security, law enforcement or less-controversial uses, have done so through a traditional procurement process.

But should governments buy AI systems the same way they buy office printers? Agencies are acquiring systems through the procurement process “almost as though they are off the shelf products…without actually realizing that what is embedded in these systems includes numerous policy decisions,” said Mulligan.

“What is embedded in these systems includes numerous policy decisions.”
– Deirdre Mulligan, School of Information at UC Berkeley

Assuming it makes sense to use AI for a certain purpose at all, deciding which system to choose ought to be based on more than cost-efficiency and a transparent bidding process, she said. She suggested an array of preexisting methods agencies might apply in order to better vet the systems and ensure transparency and public engagement in choosing them.

What might work? Input from inside or outside experts and algorithmic impact assessments, to name a couple. Mulligan also stressed the need for community participation, harking back to what attendees heard during the earlier session with Reuben Jones, but this time through a more administrative lens.

5. Refreshingly Free from the Sponsor Expo

Google, Microsoft, Spotify, Twitter and Google’s Deepmind Ethics & Society were among the sponsors of this year’s FAT. Unlike many other tech events, this is one that, as noted in its sponsorship policy, recognizes that “outside contributions raise serious concerns about the independence of the conference and the legitimacy that the conference may confer on sponsors and supporters.”

It’s worth noting that speakers and researchers from some of these sponsor firms were on stage. Microsoft senior researcher Jennifer Wortman Vaughan and Spotify researcher Henriette Cramer shared ways in which they work internally at their firms to incorporate fairness into industry practice. Google engineer Ben Hutchinson gave a fascinating overview of fairness in research over the past 60 years.

While this is one event that is largely unencumbered by industry influence, the threat of industry creep in AI Ethics was a topic of conversation. Whether industry researchers could play a larger role in future FAT events remains to be seen. Yet there’s no question more and more researchers find it difficult to get funding for their work outside of corporate coffers, and that is not likely to change.