Algorithmic Bias and Fairness: Crash Course AI #18

Today, we're going to talk about five common types of algorithmic bias we should pay attention to: data that reflects existing biases, unbalanced classes in training data, data that doesn't capture the right value, data that is amplified by feedback loops, and malicious data. Now bias itself isn't necessarily a terrible thing, our brains often use it to take shortcuts by finding patterns, but bias can become a problem if we don't acknowledge exceptions to patterns or if we allow it to discriminate.

Crash Course is produced in association with PBS Digital Studios:

Thanks to the following patrons for their generous monthly contributions that help keep Crash Course free for everyone forever:

Eric Prestemon, Sam Buck, Mark Brouwer, Efrain R. Pedroza, Matthew Curls, Indika Siriwardena, Avi Yashchin, Timothy J Kwist, Brian Thomas Gossett, Haixiang N/A Liu, Jonathan Zbikowski, Siobhan Sabino, Jennifer Killen, Nathan Catchings, Brandon Westmoreland, dorsey, Kenneth F Penttinen, Trevin Beattie, Erika & Alexa Saur, Justin Zingsheim, Jessica Wode, Tom Trval, Jason Saslow, Nathan Taylor, Khaled El Shalakany, SR Foxley, Yasenia Cruz, Eric Koslow, Caleb Weeks, Tim Curwick, DAVID NOE, Shawn Arnold, William McGraw, Andrei Krishkevich, Rachel Bright, Jirat, Ian Dundore
--

Want to find Crash Course elsewhere on the internet?

#CrashCourse #ArtificialIntelligence #MachineLearning

[nantukoprime]

Did a short stint working on an algorithm that looked for potential pickpockets, trained on video of actual incidents that led to arrest.

Was moved to another project after I kept bringing up the fact that the algorithm was biased as the data set was generally representative of a subset of pickpockets, the ones who get caught. My request for video of successful pickpockets that were not arrested to train the algorithm was not viewed favorably.

[Acoolakim]

During the debate that followed ProPublia's accusations of the COMPAS-algorithm being discriminatory against black people, Kleinberg, Mullainathan and Raghavan showed that there are inherent trade-offs between different notions of fairness.

In the case of COMPAS, for example, the algorithm was "well-calobrated among groups", which means that, independent of skin colour, a group of people classified as, say, 70% to recidive, actually had 70% of people that would recidive.

However, ProPublia objected, that the algorithm produced more false positive predictions for blacks (meaning that blacks were labeled more often wrongly as high risk) and more false negative predictions for whites (meaning that whites were more often labeled wrongly as low risk).

In their paper, the authors showed that these notions of fairness, namely "well balanced among groups", "balance for the negative class" and "balance for the positive class" are mathematically incompatible and exclude each other. One can't have the one and the other at the same time.

So yes, AI-systems will be biased, as insisted upon in the video. But it raises questions about what kind of fairness we want to be implemented and what we're willing to give up.

[saulgalloway]

6 agreements and disagreements.

1. Nurses are 90% female. Programmers are 80% male. Of course you're going to have far more images on average of the dominate sex in those fields. But, sure. Get it to say THEY.

2. The only value understanding gender has is significant behavioral predictions. Algorithm doesn't care about your social Yugioh game to feel special. It's tackling reality.

3. Lack of data on the racial bit. For sure we need greater data samples there.

4. We're gonna' ignore uncomfortable crime stats? Ok.

5. Yes. The kids who are shown to do well often are at a much lesser risk of becoming shitty. Reality sure is complicated.

6. Yes. You can't discriminate when it comes to loans and jobs. Even if there's a significant racial, sex, whatever difference. Things can't change for the better if you force them out and skillful/valuable individuals that aren't part of the problem within' these groups would suffer.

[kajmeijer]

there's great talent in simplifying complicated things and you've got it man 🙌🏼

[Karatop]

If you gave the algorithm more data for protected classes, wouldn't that just bias it towards them? It seems that any learning data would necessarily contain some kind of pre-selected bias to even make a choice.

[ListerTunes]

I'm reminded of the resume-screening AI that taught itself that the best candidates were named Trevor and played high school lacrosse. Biases in culture introduce biases into data, which just replicates the bias.

[squareenix]

"sexual orientation is strongly correlated with certain characteristics of a social media profile photo"
which characteristics? how do i algorithmically optimize the gayness of my profile??

[dmurphy]

Many people are missing the point to the Google analogy. AI hiring systems will learn associated characteristics of a nurse or programmer or what have you from similar datasets. That's not so much the problem- it's what happens next. It discriminates against people who don't meet the average characteristics. The AI system may throw out a resume for a nursing position that has the words "Boy Scout troop leader" because that's not something associated with the average nurse. It may throw out qualified programmer resumes from people who attended HBCUs, because most programmers haven't. If you don't quite get this, please look up the scrapped Amazon AI hiring program. It downgraded resumes from applicants who attended women's colleges.

[KarlRamstedt]

3:28 omg, that's the jealous girlfriend from the stock-photo meme.

[Argacyan]

I'll imagine at least a couple of people will be upset to hear that things like what data is put into their algorithm will bias the outcome which is a very easy concept to grasp, like the chemicals in a reaction will narrow down what products you possibly get or how different fuels to a fire can impact the heat generated by the fire - like I could only imagine disagreement there if getting biased results while claiming there was none would be the intention to begin with...

[HetareKing]

The point of the Google image search example isn't to accuse Google of some grave injustice, it's just an easy to understand example of how just because a computer is generating it doesn't mean its output isn't biased. The society it's getting its data from is biased in favour of female nurses, so it will return mostly pictures of female nurses even when the user is just looking for "nurse" without specifying gender. Once you understand that, it's easy to understand how that can become a problem when the situation is more complicated, the stakes are higher, which is the whole point of the episode.


Let's say there's 10 male nurses in the world and 90 female nurses. Out of those 100 nurses, one man and two women have committed the same misdemeanour on the job. Given that, would it be fair to make decisions on who to to employ as nurse based on the idea that 10% of men have committed this misdemeanour but only ~2% of women have? An AI trained with this data might. Worse yet, you don't even know it's doing this because its decision-making process is more or less a black box.

[wolflink]

Prioritizing resources to areas where statistically in the past there is more likely to be issues makes 100% perfect sense.

[TaylorChildAkaWeapon]

Such a great video, thanks for the crash course!

[Randomfunlife]

Been struggling to focus lately, and I couldn't stop staring at the insanely huge size of his beanie. I must say though, once I was able to focus, the video is very well put together video. Thank you!

[Kalaphant]

0:50 I was not expecting to get a lesson about discrimination when I clicked on a CCAI video XD

[artiphology]

"Do you pledge the axiom?" "Only in my reality class"
(got distracted reading the babel)

[nonsensespeaks]

Non binary?
- me a programmer confused

[Hamstray]

"Algorithms for performing calculation,  data processing,  automated reasoning, and other tasks.", AI (neural networks) are not unambiguous and don't qualify as algorithms. In neural networks biases may emerge spontaneously regardless of the training data.

[redshipley]

I dont get people are upset when AI cant recognize their face. I'd be thrilled to not be recognized. They wouldnt be able to use recognition software on me.

[Kalaphant]

I actually read about a way that a fill in the blank tool was able to not assume gender stereotypes, even tho the training data has a lot