Showing posts with label text analysis. Show all posts
Showing posts with label text analysis. Show all posts

13 October 2011

Simple Text Web Crawler

I put together a simple web crawler for R. It's useful if you are doing any text analysis and need to make .txt files from webpages. If you have a data frame of URLs it will cycle through them and grab all the websites. It strips out the HTML code. Then it saves each webpage as an individual text file.

Thanks to Rex Douglass, also.

 Enjoy (and please feel free to improve)