Scraping comments and posts from reddit in Python from scratch

We'll use a little-known tool called subreddits-comments-dl to easily pull comments from any subreddit in any time range that we wish in Python in only a few lines of code. This can then be exported in csv format and processed for use in sentiment trading bots, or scientific research on how social networks operate.

Using the tool allows us to much more easily download the comments in csv format than dealing with the API directly, more tutorials to follow on that matter.

Link to github repo used in video:

^ You can find the full command to start the script from when my head was in the way there.

[NS-vzjz]

You're doing stuff that noone else is doing in the space. Good job

[DrChrisCopeland]

Thank you for this, it is exactly what I was looking for!

[duygubayram]

I've been looking for a depth first approach to comments!

[leanhhoang]

How do i get it to scrape comments from a single reddit post?

[naimcivan]

I'm glad as if I found treasure :)

[puffdamgcdrgn]

Hey man, Is there a way to filter posts by number of comments, or ignore comments by bots? Or does this have to be done in the .csv or dataframe after everything is scraped?

[amanihamdan]

Thank You for sharing, that is really helpful. I have a question please, the total number of submissions to be retrieved should be (batch size*laps), I used a batch size of 1000 & laps of 3, but the total number of retrieved submissions were 1200 only, any idea about the reason?

[MainaShallangwa]

can i scrape the post related to an event based on the location of the user? Thank you

[philipung]

Is there a limit on how many comments that can be extracted?