I tried to make a Valorant AI using computer vision

I went down a rabbit-hole of trying to make a Python program that can play Valorant using computer vision and some radio shenanigans.

This video intentionally doesn't go into too much technical detail - not sure if that's something people want or not. I tried to present enough so that you can at understand what this bot can and can't do, and also understand some of the problems it's having. And if you don't play Valorant, hopefully the premise is understandable - shoot the bad guys.

If you're worried about this being a hack, you can rest easy. It's not like a wall hack where it looks at Valorant's process memory to get information that's supposed to be secret. The bot's not at the level of advanced Valorant strategy right now, but I have lots of ideas for future development.

Software used include:
* labelImg - used for labeling the data set
* PyTorch - similar to TensorFlow
* NumPy - amazing library for working with matrices
* OpenCV - great library for doing some image processing (in conjunction with NumPy)
* Google Colab and Jupyter Lab - great for exploratory programming, especially when working with images
* PySide2 - y u conflict with torchvision dependencies??

Some people doubted that the OpenAI shell video I made was real despite the mediocre results shown, so I hope that by showing even worse results in this video more people will believe it's real.

Images:

Music:

Corbyn Kites - Shadowing

Licensed under Creative Commons: By Attribution 4.0 License

NoMBe - Take Me Down to the Fashion Show

Kwon - Pluckandplay

[iceaf]

It's amazing how you can easily replicate my teammates in comp

[khalidjamonday]

This video made me understand why my friends call me a bot.

[stellasoleclark]

this randomly came in my algorithm 3 years later

[DeSinc]

One thing I thought you could do is make it so a label has to show up for 150ms (roughly a standard pro reaction time) - that way it doesn't shoot at every 1 frame ghost it thinks it sees, but only at persistent threats - and also it makes it seem more realistic and human-like by having somewhat realistic reaction times. You could also have it move the aim slowly over time in trial movements until it's over the top of the marked target and only shoot once it lines up, which would not only improve reliability of the aim but make it seem even more human-like and I just realised this video was from may 2021 and you're likely not even working on this any more oh well

[realcytv]

Calling Valorant a csgo gamemode is the funniest and most fitting description of the game I've ever heard

[Darksnipa]

Just got this recommended. Really good work!

I am impressed by the performance you can achieve with transfer learning on your "small" annotated Valorant dataset. You still remember how high the performance for different objects was on your test set (accuracy or MAP if you have computed that)?

It also really hurt me not seeing your model TURN upon hearing someone behind :D Would really love to see you including audio next, and then seeing some nice 180 flicks in version 2.0.

[darkshadowsx]

the AI is like a noob and a pro fighting over the controls.
you're getting close.

[urmom-chbb]

This takes "My teammate is a bot" to a whole new level

[hihi-hehe]

When i was doing my project with computer vision, I gained almost x10 performance increase just by downscaling input image by some ratio. Of course, it lowers the accuracy of results, but, sometimes full resolution is MUCH bigger than enough and downscaling isn't going to affect the results at all.
So, by downscaling input images you can boost performance for free by finding the optimal level of downscaling.

[Gone-Rogue]

Anyone else just get recommended this video 1 year later?

Great video btw

[Menezeris]

Why do I honestly think this bot could at least get bronze... Iron is a weird place

[suguru]

Not me getting false banned for "3rd party program" then this guys making an ai for valorant 💀💀

[flixgribv]

Pretty interesting video, ans it's really well made as well. And it has subtitles! Thanks!

[nathangavenski]

So, I know this video is kind of old, but I just discovered your channel and I'm watching all your videos 😅. I'm a PhD in machine learning, and I saw on the quick code that appears on the video that you are using large images. I don't think that this is necessary. You could downscale the images, use on your model, and recalculate afterwards where the BB is on the real feed. A second thing I would suggest os already use a pretrained huggingface object detection model just to see if it detects the caracteres as a person and use simple code to see the color of the border. This solution should help with the low data amount. You could even create data this way :) I don't have a solution for the spikes and mollies tough. Either way awesome video!

[thewisemonke]

This is a prime example of a YouTuber who needs a shit ton more attention. Well done!

[BLettuce]

This is gold. Things I think you can do (although it's been a year so who knows what happened) is obviously have it be aware to object presence, as you have said in the video, but also respond to sounds, voice commands (via wheel ore voice chat), awareness to economy, and most importantly, have it teabag other players.

[johnandrewdivina]

The only flaw I see is that it doesn't know to trash talk

[SelfSimilarJosh]

As someone getting a PhD in Machine Learning, you're doing the work of someone getting a PhD in Machine Learning.

[nauchism]

Bro said he didnt share the code but somehow I see this in all of my ranked games

[EpochIsEpic]

This project really is incredible - the way the video was captured, the way inputs were sent to the game, the problem solving of getting a used dongle when the exploit was patched, all of it was wild!