Using Image Recognition to DESTROY Fruit Ninja

This project has been a long time in the making. I set out last year to automate Fruit Ninja with image recognition, but I ran into a couple of road blocks along the way. But luckily, this year I decided to try it again, and it worked out fantastically to say the least!

This video was sponsored by Brilliant

LINKS
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

MUSIC
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Shoutout to @qumumusic for making wonderful remixes of some of my favorite video game songs! Check out their channel if you haven't already!

█▀ █ █ █▄▄ █▀ █▀▀ █▀█ █ █▄▄ █▀▀
▄█ █▄█ █▄█ ▄█ █▄▄ █▀▄ █ █▄█ ██▄

[gaco]

if object = bomb
is bad
else
is good

[lukeseaman]

The somewhat unoptimized nature of the program gives it a lot of personality and comedic value
10/10

[johnsimpsen]

It would be cool if the program waited a while before the fruits were on screen and then calculate how many there are. If there are more than one, it tries to slice them in one go instead of a bunch of slices.

[ElementEvilTeam]

would've been useful 14 years ago

[shadow_blader]

You are writing functions without space between them 😭

[rockinggamerdude]

The problem I see is it doesn’t wait for all the fruit to make sure to get a combo for a points bonus

[McTuber]

The rhythm of the slicing syncs surprisingly well with the music after 8:25

[sbd]

I have made a program in python to automatically complete tasks in among us and wrote code for almost all tasks for the first map. I faced the same problem in some tasks ( like clean vent, clear asteroids etc) where the image recognition would not work properly due to random rotations of the sprites on screen. Your solution to the problem might be perfect in my program and I am gonna try that soon. It might be even better suited since there is even less chance of false positives ( which was caused by the splattering of the fruits on the wall in fruit ninja ).

[SuadoCowboy]

why don't you just instead check if it's a bomb or not? as far as i know the bomb is the most different from them and by only checking the bomb you could optimize this code alot and then you only pass the mouse on things that are on movement except the recognized(s) bomb(s) maybe you could even try using grayscale images or something idk

[ryans]

Being somebody who works in the computer vision field, I feel like it would've been simpler for you to convert the image to HSV, take the value (lightness) channel, and then binarize the image by checking if it's less than a certain threshold. From there you can see how many connected pixels there are, and if there are more than say 1000 black pixels, its a bomb.

[zobiah]

This feels like a strong candidate for reinforcement learning imo. Just give it the average color values you have already collected, a reward function based off the in-game score system (obviously make bombs a high negative), and watch it go.

[cvabds]

This could be better optimized if you didn't based it on color recognition but actually on moving pixels on the grid you created. Based on that principle, you should need to only recognize the black color of the bomb, everything shouldn't be biased on only color. I saw it bug on the background splashed fruit some times

[ODISeth]

Oh wow you've been at this one for quite some time, excited to see how it turned out!

[-ShoeCat]

i haven’t heard about this game in a long time, when i saw this, it reminded me about how i got around 1390 while playing this in a daycare

[Mehbem]

Bro being a student working on segmenting quantum dots with dog noise this is so relatable 😭

[darsh]

This is brilliant thanks for sharing your thought process and code. absolutely loved this

[vitatreat]

I feel honoured to get this video on my feed. This video made me excited for sure 👍

[jawairengkunci]

i hope the developer soon patch the game with an anti cheat update. this isn't healthy for the fruit ninja competitive community

[linkmastr]

2:50 nice explanation of kernels in an image recognition model

[JamesTDG]

I feel like this system could be expanded more if used right. Like for example, if the system detects high red pixel counts in a region, have it take a screenshot and use image recognition to see if it can detect a red arc at around 75% completion, use an or function with it so it can also check the image for an X. If either result comes true, it will determine the region to be dangerous for the next x-amount of frames. Should help improve the system by a bit, and even can use fewer resources if it is able to know the danger region before doing the bomb check.