ElixirConf 2022 - Constantin Cheptea - The Elixir Of Web Scraping

This talk is about how we implemented a simple crawler for an MVP for one of our clients. I will talk about how we used a Stream to implement the crawler, the Eager vs Lazy topic, applying concurrency to improve performance, understanding the difference between concurrency and parallelism, and how it helped us optimize resources. I will also allocate some time to cover ethical web crawling.