Summary

This chapter focused on how to fetch and process data directly from the Web, including some problems with downloading files, processing XML and JSON formats, parsing HTML tables, applying XPath selectors to extract data from HTML pages, and interacting with RESTful APIs.

Although some examples in this chapter might appear to have been an idle struggle with the Socrata API, it turned out that the RSocrata package provides production-ready access to all those data. However, please bear in mind that you will face some situations without ready-made R packages; thus, as a data hacker, you will have to get your hands dirty with all the JSON, HTML and XML sources.

In the next chapter, we will discover how to filter and aggregate the already acquired and loaded data with the top, most-used methods for reshaping and restructuring data.