R web scraping download files

This takes some trial and error, but eventually I was able to figure out the the correct combinations to get the links to the pages. Something tells me if I check the base::length of Links with the base::nrow s in ExOffndrs …there will be twice as many links as rows in executed offenders.

Good—this is what I want. That means each row in ExOffndrs has two links associated with their name. The stringr package can help me wrangle this long vector into the ExOffndrs tibble.

Finally, add the first portion of the url used above. Now I want the offender information links so I omit the links with last in the pattern. Want to share your content on R-bloggers? Gratuitous picture: a simple summer lunch Photo: Luis. Policy programming R rblogs. Never miss an update! It also guessed that the second column should be of the type double, i.

This will not be necessary in most situations, but could increase computational speed with very large datasets, as integers can be saved more efficiently. More on the definition of column types can be found in the help file. Among these files are two files with statistics on the number of people eligible to vote and on the number that actually voted, both by binary gender, year of birth, and additional indicators. Your first impulse would be to download the files manually and then parse them.

This will work just fine, but we could also download the files directly from our code. Due to the small number of files in this example, this may not even be the most efficient way, but when you start handling larger amounts of files or want to regularly update files, downloading them from within your code is a safer and more efficient option. To understand the construction of the css selector, read chapter 5. Note that this is only one of many possible selectors that can achieve the selection of both links of interest.

We succeeded in extracting the links, but we can also see that these links are not complete since they are missing their base URL. This works on the website, as the link is relative. To access the files directly though, we need an absolute link — the full path. As before sub section 7.

For this we need some basic knowledge in string manipulation and regular expressions, which we will gain in chapter So for now, we will just use the links as they come. Now that we have complete absolute links, we can download the files to our hard drive using the base R function download.

We specify a URL as its first argument and also specify a path and file name for the destfile argument. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from Problem: How do I get all the complaints, affidavits, and indictments all at once!

The rvest package has easy functions for scraping the web. How many defendants do we have? How many documents do we have? Let's get the contents of each row into a list but drop the header row. The dir functions are from the fs package.