Python Programming Tutorial - 35 - Word Frequency Counter (1/3)

[davidr]

Richard Feynman was one of the greatest American physicists of the 20th century. Actually, he teaches (hes books) physics a lot like you teach python, very simply and understandably. Keep it up!

[jasonxu]

Thanks for your course, Bucky, I am not IT-related staff, only as a hobby to learn python, as learner from scratch, your course impressed me, it seems not so difficult to start from 0, I learn the basic python grammar, and the logical for in the code, which help me to understand some open source code, to realize some automated office in my daily work.

[niname]

Instead of the inner "for loop" for adding the words in each post to word_list, can be used. list.extend() basically takes a list and adds each element seperately, not adding them as "sublist" as append would do.

[TheVideoVolcano]

content = post_text.text
SEEMS TO FIX THAT NONETYPE FOR THE .LOWER()

[peng]

ah i tried to write my own word counter before came across this video... this is gold for me to compare the codes and improve now...

[DavidAnatolie]

Wonderful series! I avoided the inner for loop altogether by using the list method extend instead of append.

for source_text in soup.findAll('a', {'class': "title text-semibold"}):
text = source_text.string.strip()
text_list = text.lower().split()
words.extend(text_list)

[chaitanyasahu]

You were RICHARD FEYNMAN in your past life, when programming wasn't cool, so you enjoyed physics instead! And when you were reborn as Bucky Snitzleberg, you know what happened. Thanks for the tutorials tho, in return, I pray you attain Nirvana!

[samueldovbenyuk]

if anyone is getting the error that tell you to change this:

BeautifulSoup([your markup])

to this:

BeautifulSoup([your markup], "html.parser")

just change the part that says:

soup = BeautifulSoup(source_code)

to this:

soup = BeautifulSoup(source_code, "html.parser")

(It fixed my problem)

[thanosxania]

i have the exact same code, but i get AttributeError: 'NoneType' object has no attribute 'lower' 

[rj]

For those who lose track of the code always, make a todo.txt file first. And refer it parallelly while coding...

[pawebrysch]

wyraz.lower() - zmienia litery na małe
lista_wyrazow = zdanie.split() - dzieli zdanie na wyrazy

[vaishnavm]

anyonr who gets an error like below
To get rid of this warning, change this:

BeautifulSoup([your markup])

to this:

BeautifulSoup([your markup], "html5lib")

markup_type=markup_type))

or
UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 23 of the file Search Engine.py. To get rid of this warning, change code that looks like this:

BeautifulSoup(YOUR_MARKUP})

to this:

BeautifulSoup(YOUR_MARKUP, "html.parser")

markup_type=markup_type))


try this:

soup = BeautifulSoup(text, "html.parser")
use html parser close to the beautiful soup object
This should solve the problem.
Thanks for reading

[MrPaceTv]

# word frequency counter if anybody can't find website or for debugging your code *2018 Python 3*

import requests
from bs4 import BeautifulSoup
import operator


def start(url):
word_list = []
source_code = requests.get(url).text
soup = BeautifulSoup(source_code, features="html.parser")

for post_text in soup.findAll('span', {'class':'text'}):
content = post_text.string
words = content.lower().split()

for each_word in words:
word_list.append(each_word)
print(each_word)


# Edit : re-write the last line if you getting syntax error due to copying from youtube

[xpsprasain]

you are scraping text from anchor tag which has a unique class, what if the anchor tag don't have the class ? How can I scrap the text from anchor tag if that anchor tag doesn't have any unique class like this "<a . I want that Coco text, but i'm not getting this. Please help !!

[Dipakanands]

This is the error is got while trying to run the same code.

Traceback (most recent call last):
line 153, in <module>
import requests
ModuleNotFoundError: No module named 'requests'

[frozenyeti]

man I got it! Very useful tutorial bucky!

[GelsYT]

hey bucky It saids that it can find the requests module, how to fix? please? I think I need to download it but can I download It from pycharm? how

[codename-water]

UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("lxml"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 8 of the file <FILE PATH>. To get rid of this warning, pass the additional argument 'features="lxml"' to the BeautifulSoup constructor.

[pacoworld]

Google Chrome doesn't have this option "Inspect element" anymore, I mean is available when I click right click, but it doesn't do anything.

[kiranitaliya]

hey bucky.
I want to get string from specified class name like in this line (<div class="_3wU53n">Mitashi 60.96cm (24 inch) HD Ready LED TV</div>) in this i want to take only string whose class is(class="_3wU53n") so what is the soup line code for that
please tell me