Using Image Recognition to Find Mario Boos

In this video I automate a minigame called Hide and Boo Seek from New Super Mario Bros using image recognition in Python. It was a fun experience, and if you liked this video then you should check out some of my other image recognition videos on my channel!

LINKS
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

MUSIC
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

█▀ █ █ █▄▄ █▀ █▀▀ █▀█ █ █▄▄ █▀▀
▄█ █▄█ █▄█ ▄█ █▄▄ █▀▄ █ █▄█ ██▄

[pfqniet]

Neat! Couple of things I'd do differently: instead of going to the trouble of calculating speed and angle, just track dx and dy since they only go in straight lines. When they bounce, negate the corresponding delta. Much faster than trig! It might also be neat to have the size of the zigzag be configurable, and find out how small of a revealed area you can pull off :)

[kazii_the_avali]

one of the problems i am suprised that didnt come up was the fact that the boos also bounce off of the light when the light is off.

[kennycarter]

i love seeing minigames maxed out, its so entertaining. im sure someday there will be a 2nd part to this closer to halloween day. where the AI maxes out the mini game

[mucuslime]

thanks for putting music credit in the description!! i love qumu and i wanna see them get the credit they deserve :3

[nixisawesome]

I was so excited to see this becoming a video! Omg I hope more SM64DS/NSMBDS minigames become image recognised. Awesome video

[gensteps]

This is fascinating, amazing how you were able to code this. Really cool video.

[Amonimus]

Screen swipes are quite wide, I was expecting a small square/triangular swipe around each boo. With how fast it swipes the cursor, may as well to reveal the darkness completely for a guaranteed win, no path prediction needed.

[theAstarrr]

Ay very cool.

Maybe when they interact, you could add code that draws a slightly bigger box (based on how many) to temporarily cover multiple, in case the light turns off at that time

[bilalmahmoud]

Wouldn't it also have been possible to instead take the first e.g. 2 frames, interpolate the speed and angle, then just extrapolate the end position? I feel like that'd be quite the cool solution as well.

I also don't think full image recognition is needed here, because the background is always static, it should be possible to simply XOR-ing the background with the current image. OpenCV has a function for this "absdiff" that might be worth checking out.

[CutmanDrawz]

Isn't AI replacing Luigi basically the plot of Luigi's Mansion (Goouigi)

[varietychan]

you forgot the other "B" in the title in Boos

[freezgolem]

I love these videos, made my day better seeing it, genuinely

[TheRealLoopDev]

nice alternative to clearing all the screen

[Furkanh]

Idk how to code, but i can see how people can take inspiration from things like this

[kplays_]

You could drop the trigonometry completely to simplify this - and simply track the x and y deltas and flip as the ghost is predicted to bounce

[HorseyTime]

I love your videos but any shot you can make them smaller scope so they can come out more often? Fr love your shit just want you to make more

[snoozyboio]

I wonder if there's a most optimal amount of boos on the screen, such so that it takes the longest to find them all but small enough of a number you don't accidentally find the others by swiping across your screen

[chrizzzly_hh]

Next level: minimize the squiggly draw pattern for boos close enough to each other :)

[ado-]

Maybe you could try prioritizing the Boo model with the least corroberation, since that was the Boo with the most overlap: i.e: there were other Boos around there that could be captured earlier. Then, you go for the ones with the most corroberation, since the extra ones you would have hopefully captured early would have low corroberation.

[Manabender]

With how fast that zigzag is, wouldn't you be able to trivialize the whole thing by just zigzagging over the entire screen?