Web Scraping JSON Data

Tutorials on basic Web Scraping.
- - URL From VIdeo - -
- - Notes - -

Support:

[DV-mlfm]

This is a great tutorial on json. I love your teaching style of learning by doing.

[kaslmineer]

great video man, i like everything about it

[johnanth]

Hi Kris, thanks for the great tutorial on JSON. I am interested more in the first part and want to know the 'proper' way to do it when a service offers an API.

I have a personal wiki and I'm looking to track films I've watched, games I've played and that kind of thing, as I'm not satisfied with using web services for cataloguing anymore. I miss however the rich data.

There are database for this, for example in games there is IGDB and the much less complete RAWG. These offer REST APIs but I would love a video showing end-to-end how to query them and get usable data, just for the fields and information I want.

I can get a CSV export of every game I have played from my wiki for example, and I want to query that against the APIs to get things like cover art (these are image files so some complexity there probably), publication date, platforms, development studio, et al.

I think you are meant to put in a delay as well when using APIs. At work, the team uses Postman but I'm not going to pay for that, and it strikes me as a professional-grade tool that's overly complex to learn for my basic purposes. Moreover, I'm concerned it abstracts too many things out while the simple command-line tools you favour do not.

I'd be interested in seeing a video on getting a filtered selection of data from an API if you'd be interested in making it.

[ricochet]

Really useful ! thanks Kris "with a K"

[xgamer]

Very good, all the tutorials on jq on the web are horribles.
Yours is good

[ChristopherBruns-oo]

1:20 there is an ^I amd ^J hot key depending on browser, and make/model/device.

Is this where the escape characters get important, since the system reads "password" from use of variable?

This would make all passwords (almost, linux wouldn't) are buffer overflows?

Because escaped bytes still exist.

What about scaling. Like how to make the same pong game client side offline then again from backend single player mode.

Or at least how to hold a session when using a redirect and credentials. || build jq from help and python maybe?

Rebuilding jq would be super impressive actually. Doit...

Cant you encode a wget command and do some php to get it out from base64 or something. So the spaces get preserved and from GET?

7:10 Encode the delimiters while decoding == True

&& also why is the difference in and interactive download and not interactive download. ie wget vs curl. eg `curl -L curl`, !wget -r