filmov
tv
Fast Window Capture - OpenCV Object Detection in Games #4
Показать описание
Learn how to capture window data in real-time as a video stream for processing with OpenCV. We try several different methods searching for the fastest way possible. In this tutorial I also discuss the importance of good Google search skills as a programmer, and we revisit some basic object-oriented programming concepts.
Research discussed:
0:47 Main capture loop
3:16 Using PyAutoGUI screenshot()
6:24 Measure FPS
8:27 Using Pillow ImageGrab
9:31 Using Pywin32 for screenshots
13:48 Converting win32ui.CreateBitmap() for OpenCV
17:23 Confining screenshots to a specific window
18:42 Creating a WindowCapture class
25:05 Trimming the window capture
28:15 Image to screen position conversion
29:35 Wrap up
Up to this point, we've been using OpenCV to detect objects in static images. Now we're ready to apply those same techniques to video games in real time.
Remember that video is just a series of still images shown in rapid succession. In this tutorial our goal is to capture screenshots as fast as possible and display them in an OpenCV window, so that we get a real time video stream of the game we're interested in. This will set us up to begin processing image data with OpenCV in real-time.
OpenCV has a tutorial on "Getting Started with Videos" that will serve as the basis for our code. Our starting point differs from the official tutorial only in that we are preparing to work with screenshot data instead of frames from a camera.
And this would work, but there are several benefits to calling the Windows API directly instead. Firstly, we approach the theoretical limit for how fast we can take these screenshots by dealing right with the operating system itself. Secondly, the Windows API has methods that will allow us to grab the screen data for only the window we're interested in, even when it's minimized or off screen.
To do this, we must first pip install pywin32 to get access to the Win32 API in Python.
The next step is to modify this function so that instead of saving an image file, it instead returns the image data, formatted to work with OpenCV.
Now we can call this function from our original infinite loop and get a real-time stream of our desktop.
To improve upon this, we can use win32gui.FindWindow(None, window_name) to capture just the window we're interested in. Replace the window_name with a string that contains the name found in the title bar of the window you want to capture. Doing so will allow you to capture the frames from that window even when it's hidden behind other windows.
If you're having trouble figuring out the name of the window you want, you can use this code to list the names of all your existing windows:
We can improve our code further by trimming off the excess around the window we're interested in. When you run the above code, you will notice black space to the right and below the window image, as well as the window borders and title bar. Removing these will not only clean things up, it will also improve our frame rate. We can also get improvements by not calling win32gui.FindWindow() on every call to get_screenshot(), so let's turn this into a class.
Finally, we'll need a way to convert positions we detect in our screenshots back to pixel positions on our actual monitor. In the WindowCapture class constructor, I've already included code to calculate the window offset using the window position data from win32gui.GetWindowRect(). Let's add a method to our class that uses this offset to return that converted screen position.
Research discussed:
0:47 Main capture loop
3:16 Using PyAutoGUI screenshot()
6:24 Measure FPS
8:27 Using Pillow ImageGrab
9:31 Using Pywin32 for screenshots
13:48 Converting win32ui.CreateBitmap() for OpenCV
17:23 Confining screenshots to a specific window
18:42 Creating a WindowCapture class
25:05 Trimming the window capture
28:15 Image to screen position conversion
29:35 Wrap up
Up to this point, we've been using OpenCV to detect objects in static images. Now we're ready to apply those same techniques to video games in real time.
Remember that video is just a series of still images shown in rapid succession. In this tutorial our goal is to capture screenshots as fast as possible and display them in an OpenCV window, so that we get a real time video stream of the game we're interested in. This will set us up to begin processing image data with OpenCV in real-time.
OpenCV has a tutorial on "Getting Started with Videos" that will serve as the basis for our code. Our starting point differs from the official tutorial only in that we are preparing to work with screenshot data instead of frames from a camera.
And this would work, but there are several benefits to calling the Windows API directly instead. Firstly, we approach the theoretical limit for how fast we can take these screenshots by dealing right with the operating system itself. Secondly, the Windows API has methods that will allow us to grab the screen data for only the window we're interested in, even when it's minimized or off screen.
To do this, we must first pip install pywin32 to get access to the Win32 API in Python.
The next step is to modify this function so that instead of saving an image file, it instead returns the image data, formatted to work with OpenCV.
Now we can call this function from our original infinite loop and get a real-time stream of our desktop.
To improve upon this, we can use win32gui.FindWindow(None, window_name) to capture just the window we're interested in. Replace the window_name with a string that contains the name found in the title bar of the window you want to capture. Doing so will allow you to capture the frames from that window even when it's hidden behind other windows.
If you're having trouble figuring out the name of the window you want, you can use this code to list the names of all your existing windows:
We can improve our code further by trimming off the excess around the window we're interested in. When you run the above code, you will notice black space to the right and below the window image, as well as the window borders and title bar. Removing these will not only clean things up, it will also improve our frame rate. We can also get improvements by not calling win32gui.FindWindow() on every call to get_screenshot(), so let's turn this into a class.
Finally, we'll need a way to convert positions we detect in our screenshots back to pixel positions on our actual monitor. In the WindowCapture class constructor, I've already included code to calculate the window offset using the window position data from win32gui.GetWindowRect(). Let's add a method to our class that uses this offset to return that converted screen position.
0:30:48
Fast Window Capture - OpenCV Object Detection in Games #4
0:02:32
60 FPS Screen Capture - Python
0:07:09
FAST Screenshots in Python for Computer Vision: mss vs. PIL vs. pyautogui
0:10:57
How to Capture Screen with Python (Very Fast!!!)
0:10:08
Capture Window with OpenCV in C++
0:20:25
Dominating an Online Game with Object Detection Using OpenCV - Template Matching.
0:11:36
Capturing Aseprite preview (or any other windows) with OpenCV | Devlog/Tutorial |
0:00:14
Webcam Image Capture, 60 sec Interval,noCamera Window, Python OpenCV,Time #time #60seconds #capture
0:11:16
Build a Screen Recorder Using Python | Screen Capture | OpenCV
0:00:46
NEVER buy from the Dark Web.. #shorts
0:12:42
PyAutoGUI - Locate anything on your screen | Simple Pyautogui project
0:01:37
My first Python Opencv Realtime Screen capture
0:11:06
How to record desktop screen using Python, OpenCV & PyAutoGUI
0:01:00
Take Screenshots Fast in Python on Windows, Linux and MacOS
0:15:01
How to make advanced image recognition bots using python
0:19:05
Build a Screen Recorder Using Python and OpenCV| Screen Capture | Python Projects
0:05:33
OpenCV Tutorial in 5 minutes - All Modules Overview
0:08:44
Capture Video using Opencv Python | OPENCV Webcam | Save and Play Video File | #opencv #python
0:06:09
Blender & OpenCV - Windows Guide - Facial Motion Capture
0:19:23
I tried to make a Valorant AI using computer vision
0:00:23
3D hologram fan portrait solution. Who wanna date this holographic sexy lady #3dhologramfan
0:07:58
Capture Real time image in opencv || PythonEpoint Tutorial
0:10:36
Measure Speed With Raspberry Pi + OpenCV + Python
0:08:36
Python OpenCV Tutorial To Capture Images From Webcam Full Project For Beginners
Комментарии