DEF CON 26 AI VILLAGE - Aylin Caliskan - The Great Power of AI Algorithmic Mirrors of Society

Following the progress in computing and machine learning algorithms as well as the emergence of big data, artificial intelligence (AI) has become a reality impacting every fabric of our algorithmic society. Despite the explosive growth of machine learning, the common misconception that machines operate on zeros and ones, therefore they should be objective, still holds. But then, why does Google Translate convert these Turkish sentences with gender-neutral pronouns, “O bir doktor. O bir hemşire”, to these English sentences, “He is a doctor. She is a nurse”? As data-driven machine learning brings forth a plethora of challenges, I analyze what could go wrong when algorithms make decisions on behalf of individuals and society if they acquire statistical knowledge of language from historical human data.

In this talk, I show how we can repurpose machine learning as a scientific tool to discover facts about artificial and natural intelligence, and assess social constructs. I prove that machines trained on societal linguistic data inevitably inherit the biases of society. To do so, I derive a method that investigates the construct of language models trained on billions of sentences collected from the World Wide Web. I conclude the talk with future directions and open research questions in the field of ethics of machine learning.

[brianhoward]

This was one of the best talks I've seen at BH/DC, not just for this year's #AIVILLAGE, but in general, and there were LOT of great talks this year...

Man, I with some of my friends from Vegas, with enough reputation and job security, would rank the best talks.

Even better, going along with (Dr?) Aylin's thesis or a tangential semanteme (sic), Id love the community to write papers, if not white papers, extrapolating on talks like this.


dope, this talk. I'm a #pedantictwat / #orospoo but I've never been on the Wall of Sheep, and go every year....someone must feel me on this.

[metaforest]

From a previous talk on deliberately confusing a classifier: and tying it into this talk. If you build a classifier using another classifier as the oracle you get a > 90% chance of replicating the blind spots of the oracle. It doesn't even take a large number of samples from the Oracle. As few as a 500 samples can give enough information to develop an attack against the Oracle classifier. When the Oracle is a human, or a group of humans it doesn't take many samples to begin determining bias and forming a signature. These models can then be used as the thin wedge to develop exploitation of social biases in language as well as visual representations.

[Anna_Swamy_Nageshwar]

Machine learning: garbage in garbage out