Looping over a string in SAS

In SAS it is easy to loop a macro between two numbers

%DO I=1 %TO 10;
...
%END;

But if you have a list of non-sequential  numbers or text you want to run your macro over, e.g. a list of towns, it can be a bit trickier. This SUGI paper gives a macro which lets you do just that. A few years ago I looked at that macro and didn't really understand how it worked and not wanting to use code I didn't understand I wrote my own version.

Read More

Further explorations into choropleth maps in Excel

I’ve been trying to improve my Excel choropleth map spreadsheet from my first post.

The first thing I tried was to try to update it using some maps created on ClearlyandSimply.com. So I’ve created two new versions, one for Europe and one of the World. Both using the maps from ClearlyandSimply.com, with a few small alterations to the code so that on hovering over the map it tells you the country not the abbreviation. I did try to make my own UK maps of Constituency boundaries and Counties but using this technique but no matter how hard I tried I couldn’t get it to work. If you do get it to work, please let me know. Note that in both the Europe and World map the data may not be accurate and is only there for illustrative purposes.

Read More

Pivot Chart Tracker

Sales mix by sale type
One of the dashboard charts

I was setting up some trackers at work the other day using some OLAP cubes in Excel across a number of different variables (about 20) to track monthly sales which I could refresh each month. Once I’d set the sales tracker up I realised that I wanted to look at average price across the same variables so I made a copy of the spreadsheet and went through each of the pivot tables changing  them to track average price. When I then wanted to look at sales mix (%sales that month) I thought there must be a better way so decided to write some VBA to do all of this for me.
Read More

Summarising data in R

Summarising data

One of the most frequent tasks I do is summarising data using either proc sql or proc means with code like this:

proc means data=inputdata nway missing noprint;
    class var1 var2;
    var var3 var4;
    output out=outputdata (drop = _type_ _freq_) sum=;
run;

Given that I use it in SAS a lot I’m going to assume that I’ll use it in R a lot so it seems like the next sensible thing to learn.
Read More

UK Train Data

One thing I keep on wanting to get around to looking at is what publicly available data is available on the trains in the UK and the answer to a few questions?

  • What proportions of the trains in the UK are run by foreign governments?
    • At a rough guess it is in the region of 20% based on not very much
    • How would this be measured? Passenger miles?
    • Why can the French, Dutch and German governments run UK rail franchises but not the UK government? It makes no sense!
  • What subsidies are given to the different franchises?
  • Are the original tenders released publicly?
    • Do the train companies live up to these documents?

Read More

First attempt at simple analysis in R - Part 1

It’s time to start some analysis, albeit very basis analysis. I want to look at the interaction between the ONS Rural score and the average Broadband speed. This will be done using the postcode file created in my previous post. I’m assuming that the more rural a place is the slower its broadband will be. Is this actually the case?

The aim of this exercise is to learn so R skills not do some rigorous analysis. This means that some rather broad and potentially foolish assumptions will be made with the data to make some things easier to code given my novice R skills.
Read More

RSS Challenge 2015

I thought I’d take a look at the RSS (Royal Statistical Society ) “Statistical Analytics Challenge” after being sent it at work today. It involves analysing eye movement on 60 pictures.

Whilst I’m not going to enter the competition I am going to have a go and see how far I get. My plan goes something like this:

  • Read the image into R
  • Split it into a grid
    • Initially a large grid and then progressively smaller ones
  • Calculate some properties of each of the grid cells
  • Check how each of these properties correlate with where the eye movement points are
  • Check the properties of the surrounding grid cells relative the current cell
  • See how these new properties interact with the eye movements.
  • Do each of the above for a number of pictures to come up with a model and then test this on one of the other pictures.

Read More