another “great” discord update...

Discord did it again, but is this time different?

Yes. Surprisingly yes.

Discord added two things. First they added a massban feature that allows your server staff to ban a whole bunch of people. Now, hear me out. Yes this can be used to nuke your Discord server. But, I said can, not will. Second, Discord has been handing out random bans to people because they sent an image. Now this aint no PhotoDNA doing it's job and banning creeps or gullible people. This is AI, artificial intelligence using it's lack of intelligence to flag images as bad. And when discord flags your image, you might get banned.

SOCIALS
-----------------------------------------------------------------------------
Discord Server

Twitter

TIMESTAMPS
-----------------------------------------------------------------------------
00:00 - Discord's New Image AI Moderation
03:21 - Reality of AI Systems
05:40 - Massban = EZ raids?
06:47 - Discord used brain expansion?

[okamizer]

When Discord says that they review images with a "human", I believe them just as much as when YouTube says it for videos they flag.

[JovialSkunk]

Honestly Discord trying to stop pedos by banning innocents, while keeping pedos on the app is such a Discord moment

[Joandersonso]

I like to remember how Discord solved the pedo issue by banning people mentioning their age or numbers like "14 years" keeping only the pedos on the app

[GANONdork]

Something about handing perma-ban permissions to an AI blackbox that only has an 80% accuracy rate at best strikes me as being a bad idea.

[phalanx]

Funny how safety is always the excuse to scan someone’s pictures or invade someone’s privacy

[qrzone]

I really think that some of these extremely powerful features like mass banning should have a "three keys launches the nuke" approach where you can configure it so that requesting a mass ban requires the approval of x people with equal or greater permissions

[KagamineR]

I'm part of a community of pixel art/game devs where it is a common thing that things as innocent as plain wall, grass, and wood grain textures get blocked for being "NSFW"... To the point it's a common joke that we draw "eggplants" On them to get around the broken and useless NSFW filter. Nice to know that same worthless wonky algorithm can soon get you randomly banned... When they inevitably get bored of doing it 'manually'.

[TheWizardsOfOz]

Can't wait to be banned by new technology by Discord for absolutely no reason.

[sylvie]

I hope that this can generate enough backlash to convince Discord to reverse their course in this regard. CSAM is a problem, but the risks of false positives threatens to overwhelm and trivialize these regimes and their intended function, which is to crack down on material that exploits and harms children, and sadly the only way this can be done is to limit it to verified abuse content.

[AquaQuokka]

This is what happens when you don't keep AI recognition in check. Too many people are using AI for the sake of using AI, regardless of effectiveness.

EDIT: Why is nobody questioning how the models know what CSAM looks like?

EDIT: I just asked a question, why are y'all saying I'm weird??

[skyelastname]

This is slightly depressing. I can no longer send selfies to my friends because I, a 23 year old, look like I'm 15. I do not want to risk my account.

[NogardCodesmith]

"to protect the children" is right up there with "for the greater good" when it comes to reasoning for censorship and authoritarianism.
It provides an excuse to enact any kind of horrible changes, and then label anyone who objects as "evil".

[kurisu]

This seems like something that should only go into effect if something gets reported instead of it being automated.

[NsPlayz-games]

Love how no-no material deticated servers are still running like clockwork but random people get banned 💀

[urzenAE]

I'm honestly scared to do something on Disord now, their updates are so unexpected and so weird. Like if I get banned just for talking with my homies and sending normal pictures or meme pictures, I'd actually quit Discord 💀

[itarqos]

Discord has a way to add a little touch of ingredients to surprise its members

[cloudthelemon]

Yet the reports about actual grooming go unchecked while they shovel out this bullshit.

[jeshirekitenkatt]

i swear to christ content moderation like this is going to basically ban petite, small-chested women from being seen in public at this rate. it's been an issue for a while and there are some people who even debate if women who look young should be allowed to have relationships on the grounds that people think anyone who dates them is a creep. like it's reasonable and good to want to protect kids but it's also seriously messed up how this ends up impacting a decent chunk of fully grown women. but i don't even know what you'd do about that, save build the concept of false positives into the system itself. tbqfh in some cases i feel like people should be given compensation for a false positive depending on how badly that screws their life, either as someone who's photo was flagged or someone who had that photo.

[mekolayn]

I remember how back in the days my friends were unable to send photos of their palms and fingers. Good thing that Discord didn't think that it was "cheese pizza"

[ItsCreper]

It's terrible that we have to deal with people who are going to send garbage like this, and innocent users have to suffer the consequences of terrible AI and non-AI moderation.