Build A Custom Search Engine In Python With Filtering

In this project, we'll build a custom search engine that uses filtering to rank results. The engine will get results from the Google Custom Search API, store them, then rank them based on filters we define.

We'll filter based on the number of trackers on the page, and the length of the content. The framework will be extensible, so you can add your own filters, including ones that use machine learning.

We'll also use Flask to render a basic search page and results list so you can use the engine.

Chapters

00:00 Introduction
02:38 Getting a Custom Search Engine API key
05:46 Initializing our project with PyCharm
09:57 Storing our results with sqlite3
17:36 Querying the search API
30:16 Creating a flask web application
41:48 Filtering results by page content
48:20 Filtering by trackers and ads
55:30 Adding in relevance scores
1:03:21 Next steps with this project

---------------------------------
Join 1M+ Dataquest learners today!
Master data skills and change your life.

[MangoBee]

Amazing content. Very intuitive and straight forward, so great that even non-programmer like me can enjoy it to the end without any extra google search. Good works!!

[dataprofessor_]

Amazing tutorial, still not finished but already super excited for all the upcoming ones. Please keep doing amazing work.

[MV-sybz]

I used this project to structure my script to query google and use the resultant link to parse the specific website in order to get product information for the website we are trying to build for small business.

[Quantumvp]

I have fallen in love the is was what I was asking for without asking 😍

[unstablecoding]

I didn't finish it but that was cause I already have my own search engine but just needed to know how to get the api to work. Thanks!

[newchaoz]

Thanks so much for the great tutorial! Do you also have a tutorial to deploy this falsk app on for example GCP?

[newchaoz]

I had to store the data in json because the pandas DF has errors like: KeyError: "['link', 'snippet', 'title'] not in index"

[tejaswikarthik]

I liked this very much and i followed so
Can anyone say where we can deploy this project

[utkarshkumar]

Getting a syntax error (Invalid syntax) whilst running "flask --debug run --port 5001". Any idea why it could be happening ?

[TheChrisSoria]

my search results are not so good! any idea how to improve them?

[gregorpanteleyevicmelehov]

How can we show the codes we have drawn by designing them in the layout?

[AaronStierCohen]

Hey there. I do not want to search the entire web. I want to search specific news sites and possibly social media pages. Do you know where I can learn how/why to index different websites, domains, etc. so I'm searching only those sources at whatever depth? Thanks for the videos!

[ashusharmacseaspirantsstud]

Hi In above depolyment I am getting error as Keyerrors "items" how to resolve please guide into it