A quick follow up on the last post. I forgot to write about plotting ROC curves in R based on the different models. In the last post I created 5 progressively more complicated decision trees which didn’t really add any benefit when looking at the accuracy of the model. But accuracy is just one metric, what do the ROC curves look like and what are the areas under the curves? Read More
Now that the land registry data has been imported and had some initial exploratory work done to it lets have a go at making a price prediction model. I’ll use a small subset of the data and initially only try to predict whether or not the house is worth more or less than £500k, rather than the more complicated process of predicting the price. The code used in this post is largely based upon the DataCamp course “Introduction to Machine Learning”. Code for this project is on my GitHub page here. This post focuses on decision trees using the package rpart. Read More
My computer has been struggling with some of the code I’ve been trying to run, it is pretty old and doesn’t have enough memory for large datasets in R. So rather than buy a better laptop I’ve set up an Amazon Web Service account and using this guide set up a computer so I don’t have to use mine. I’m only using the free one for now but if I want to have a go at processing something larger this will allow me to pay a small fee to use a more powerful machine for a short period of time.
After my last post on the ONS data structure this post is the first of a few on using that structure and some other public data, mostly UK government data, and mapping it using R. This first post is about getting shapefiles from various locations, loading them into R and plotting them.
I have been looking through the ONS geographic data on their Geo Portal and there are acronyms and variables everywhere so I thought it best to understand what they all mean. Whenever I refer to the output areas and super output areas I’m referring to the ones as at the 2011 census in England and Wales.
Sometimes I have a spreadsheet containing lots of spreadsheets of similar formats all using the same colour scheme. And if after a while I decide I don’t like the colours any more then it can be quite annoying to change all of the colours. So I decided to write this short little macro to change the colours.
In SAS it is easy to loop a macro between two numbers
%DO I=1 %TO 10; ... %END;
But if you have a list of non-sequential numbers or text you want to run your macro over, e.g. a list of towns, it can be a bit trickier. This SUGI paper gives a macro which lets you do just that. A few years ago I looked at that macro and didn't really understand how it worked and not wanting to use code I didn't understand I wrote my own version.