Introduction to Web Scraping

Welcome to the wide world of web scraping! Web scraping is used by many fields to collect data not easily available in other formats. You could be a journalist, working on a new story, or a data scientist extracting a new dataset. Web scraping is a useful tool even for just a casual programmer, if you need to check your latest homework assignments on your university page and have them emailed to you. Whatever your motivation, we hope you are ready to learn!

In this chapter, we will cover the following topics:

• Introducing the field of web scraping
• Explaining the legal challenges
• Explaining Python 3 setup
• Performing background research on our target website
• Progressively building our own advanced web crawler
• Using non-standard libraries to help scrape the Web