Author: Laura Ellis

Laura Ellis is a data geek who aims to make data science and analytics accessible to everyone. She started her analytic journey with databases and has been hooked ever since. As a natural progression, she moved from databases to business intelligence and dashboarding. Being able to more easily visualize the data allowed her to consider broader questions like "how different are these trends really?" and "how can we use this data to drive better performance?" This led her into the path of data science and predictive modelling. She has worked in the data field holding a variety of positions for 13 years. She holds a Bachelor of Engineering Science, Software Engineering from the University of Western Ontario and a Master of Science in Predictive Analytics from Northwestern University. Check out her blog: www.littlemissdata.com

How to Screen Scrape Using R

One comment

STEP 1: BACKGROUND

Screen scraping is an effective technique that can be used to gather data off of web pages. Typically the data is gathered for further analysis or aggregation.

In this tutorial, we are going to screen scrape Googles “Best of 2017” App lists. We are using screen scraping as a technique to automate copying data off of websites. For data wranglers, there are a number of libraries and packages that have been developed to make screen scraping relatively straightforward. In Python, the package Beautiful Soup has a large following. In R, the package rvest has been getting a lot of traction.

In this tutorial, we will use the rvest package to scrape data from the Google Best Apps of 2017 website and store it in a data frame. We will then use a few of R packages to analyze the dataset further.

Note: The full R code can be downloaded here.