Master web scraping with AutoHotkey: The ULTIMATE guide for dominating in 2023

Показать описание

Web Scraping with AutoHotkey is a complicated topic. In this AHK Hero webinar we walk through 9 distinct approaches to Web Scraping with AutoHotkey. We discuss pros and cons of each approach and why you would choose one over the other. Learn how to Web Scrape today!

Here are a few links mentioned in the video:

TLDR: Selenium is a reliable library for web developers to automate tasks on multiple browsers, while HTTP requests and multi-threading capabilities can help with web scraping.

1. 00:00 🤔 Image search and hotkeys are unreliable for web scraping, but Selenium is reliable and good for web developers who need to test automation on multiple browsers.
1.1 We are discussing different approaches to web scraping with AutoHotkey and their pros and cons.
1.2 We rated nine different options on a 1-10 scale to help determine which one to use based on application and importance.
1.3 Image search and hotkeys are reliable ways to automate tasks on one computer, but can be unreliable when used on different computers.
1.4 Image search is not reliable and a fuzzy match with pixels is a better approach for web scraping.
1.5 Sending keys, tabs, and clicks can be unreliable as programs may not allow type navigation, tab stops may be disabled, and the order of controls may not be what is expected.
1.6 Selenium is reliable and good for web developers who need to test automation on multiple browsers, but it requires downloading and installing several files and is not a portable solution.

2. 11:15 🤖 UIA Library provides a stable library for automating Chrome or Firefox, but Chrome HK is difficult to use due to ActiveX and pipe connections.
2.1 UIA Library is easy to work with and provides a stable library for automating Chrome or Firefox.
2.2 Understanding UI patterns and how to program and use a class is essential for working with UI.
2.3 Web browsing is complicated due to different page development, but a programmatic approach that imitates human clicks can help improve the process.
2.4 When accessing cookies programmatically, browsers will only send a subset of them, but with Selenium, all cookies are available.
2.5 UIA is great for simple tasks like filling in forms and setting passwords, but it does not have access to the DOM like Chrome does.
2.6 Chrome AHK was created in January 2018 and has a decent score, but it uses ActiveX and pipe connections which makes it difficult to use.

3. 18:40 🤔 Coding experience and Chrome extensions can help automate processes and control access, but require JavaScript knowledge and the ability to install programs.
3.1 The library often enters an infinite loop due to a lack of response from messages sent.
3.2 He was unable to use modern websockets to connect to the Chrome browser with a laptop based on Titan speed due to code issues.
3.3 I have been coding for a long time and have seen many things.
3.4 Chrome's webview event triggered the highest score of 10 due to its simplicity.
3.5 We need to watch events and have a laptop connection to control access and perform perfectly.
3.6 Chrome extensions can be used to automate processes, but require JavaScript knowledge and the ability to install programs.

4. 27:21 🤖 Automate major browsers with Graphadium for efficient web automation, but be aware of security risks when using web driver modes.
4.1 Using Auto Control and Hot Peas, you can communicate between programs using the clipboard as a Global variable.
4.2 We used Chrome and Autohotkey to do API calls, retrieve JSON objects, and store them in a database, but some of the cookies were encrypted and we couldn't decrypt them.
4.3 Rufadium is a reliable tool that provides direct access to the DOM and is easy to code with.
4.4 Selenium Webdriver is an intuitive library for automating browsers, but manual login is required to get cookies.
4.5 Graphadium allows us to automate major browsers with the same command, making it a reliable and efficient tool for web automation.

5. 39:12 🤖 Disabling the web driver view option and using API calls are the most reliable ways to do web scraping without a browser.

6. 46:49 💻 Automating web tasks with HTTP requests can be tricky, but can be made easier with the help of developer tools or Fiddler.

7. 59:24 🤔 When accessing data from a website, use the simplest approach and be aware of potential issues with binaries and profiles.

8. 01:08:09 🤩 Join the AHK Hero Member program to access live office hours and help each other learn Python's multi-threading capabilities for web scraping!
8.1 Join the Hero Member program to access live office hours and help each other learn.