Monday, May 18, 2015

UK constituency and administrative area KML

A strange thing happened last week, visits to my site were affected by real world events. That’s the first time that has happened. The cause was the UK election and the pages affected by it were the pages devoted to UK electoral constituencies and in particular the constituency of South Thanet. After having a look at those pages, I came to the conclusion that any visitors may have been quite disappointed with the data available on those pages. So I’ve spent a bit of time improving them. They now include the area polygons of each constituency and also provide an estimate of the population and number of households in each constituency.

Whilst i was at it, I also added area polygons for administrative districts and wards. Enjoy!

Thursday, May 14, 2015

No trouble with counties

I wrote recently about the trouble I’ve had figuring out what to do about counties in the postcode data on my site. Thanks to a very helpful comment on that post, I finally figured what I hope is the correct course of action. Previously I only showed county information for postcodes that were located in a county council. The suggestion was to map postcodes to ceremonial counties, which I wasn’t really aware of before. But a quick look at Wikipedia suggested they was a pretty simple mapping between them and administrative areas, with the exception of a little complexity in Stockton-on-Tees. So that’s what I’ve done. You can now download postcode data for each English county here. Hopefully this meets all your county needs!  

Wednesday, May 06, 2015

Distance to sea, house price improvements

I made a few improvements to the website over the Bank Holiday weekend. Ever had a burning desire to know the distance from a postcode to the sea? Well now you can find out.

I’ve also updated the way individual house prices are listed so that all sales for a particular property are grouped together (example)

Saturday, May 02, 2015

UK house price data March 2015

I’ve uploaded the latest Land Registry house price data for March 2015 to my site. Prices continue to chug along at 4-5%, as they have been doing for the last 12 months. The CSV download data now includes a yearly summary and I’ve included charts for this annual data at the postcode district level (monthly data is too volatile to show anything meaningful in such small areas)

The trouble with counties

I include some county information with my UK postcode download data, but I get quite a few questions regarding it. The most common question is why don’t all the postcodes have an associated county. The answer is that county information is only shown for postcodes that are located in an administrative county council. So LA1 postcodes are listed as part of Lancashire but BB1 postcodes aren’t, even though most people would consider Blackburn to be in Lancashire. This page gives you an idea of how this works. Select ‘Counties’ in the dropdown and see all the gaps.

The second question that generally follows is whether I could include county information for each postcode. This is where things get tricky. Have a read of this Wikipedia page on the subject of counties. In short, there are the administrative county councils in use today, there are historical counties whose boundaries have changed many times and there are postal counties that used to be supplied by the Royal Mail.

So if I wanted to add county information for every postcode, my first decision would be which of these to use. The Royal Mail seem pretty keen to get rid of postal counties and the information is not provided with the freely available postcode data, so that’s not an option.

So another option would be to use historical counties. Leaving aside the fact these boundaries changed many times, I’m not sure using them would satisfy most users of my download data anyway. Taking an example from the referenced Wikipedia page, how many people would consider Brixton to be in Surrey?

And the final option would be to fill the county data from the local authority. So Brixton would now be in London, but BB1 would now be in ‘Blackburn with Darwen’. Now in this case it’s obvious that any postcodes in the ‘Blackburn with Darwen’ area should show Lancashire as the county so I could possibly map all these authorities to sensible counties, with some work. But even then, would this provide people with what they want? Kingston would be in London, although some people would expect it to be in Surrey, since Surrey county council is based there (and use Surrey in their address).

So in conclusion, counties are blooming tricky and I suspect no one size would fit all, hence the incomplete set of data on my site. If anyone has an opinion on whether there is a good approach to this, let me know.

Monday, March 30, 2015

Cycling halfway round the world

In July 2012 I started to use Endomondo to track my bike rides. Since then some things have changed in my cycling, I’ve moved from a mountain bike to a road bike, I’ve tried cycling 100 miles in one day (only to be thwarted by the weather)and I’ve started using Strava, but I’ve continued to use Endomondo. And today I reached the milestone of getting halfway round the world.

image

Last year I covered almost 6,000 miles. If I can continue at that level, then in a just over a couple of years I will have completed my virtual trip round the world. If you want to encourage me, then you can sponsor me in this year’s Ride London 100

Saturday, March 28, 2015

Loading Google Maps asynchronously

Google’s PageSpeed keeps telling me I should load scripts asynchronously to improve the performance of my website. Now in an ideal world I’d use something like RequireJS to implement this for all my scripts, but frankly that seems like a bit of a big task and more than likely I’d stuff it up and break large chunks of my web site. So I thought I’d start small and just load up Google Maps asynchronously. Google provide an example of how to do this, but I wanted to encapsulate that in a simple reusable function with a callback function parameter to run after the library had loaded. This is what I came up with in TypeScript.

function loadGoogleMaps(libraries: string, callback: () => void) {

  window["initGM"] = () => {
    callback();
  };

  var script = document.createElement("script");
  script.type = "text/javascript";
  var url: string = "http://maps.google.com/maps/api/js?callback=initGM";
  if (libraries != null && libraries !== "") {
    url += "&libraries=" + libraries;
  }
  script.src = url;
  document.body.appendChild(script);
}

UK house price data February 2015

I’ve uploaded the latest Land Registry data to my site. Prices continue on their seemingly never ending upward march.

Thursday, March 05, 2015

A cross browser XML parser

A post from three years ago detailing how to implement selectSingleNode for XML documents in a cross-browser friendly manner is still getting a good number of hits. Which I guess shows that developers still need to manipulate XML in browsers, even with the increasing popularity of JSON. When we started to rewrite our desktop app on the web, JSON seemed like the obvious choice, but we use XPath in a big way and we wanted our web app to be compatible with our desktop app, so we stuck with XML, which meant having to deal with the different ways XML is supported in different browsers. So we now have a reasonably well featured cross browser XML parser, the source of which you’ll find below.

A couple of things to note, this probably works in IE8 and below but I’ve never tested it since our app needs at least IE9. Also, the code is TypeScript rather than Javascript, since TypeScript is slightly less insane…

// stop TypeScript complaining about stuff we don't have definitions for
interface Window {
  DOMParser;
}
declare var XPathResult;

class XmlWrapper {
  private xmlDoc: any;
  constructor(xml: string) {
    try {
      // try Internet Explorer first. Although later versions have DOMParser, they don't implement evaluate
      this.xmlDoc = new ActiveXObject("Microsoft.XMLDOM");
      this.xmlDoc.async = false;
      this.xmlDoc.loadXML(xml);
      this.xmlDoc.setProperty("SelectionLanguage", "XPath");
    } catch (ex) {
      if (window.DOMParser) {
        var parser = new DOMParser();
        this.xmlDoc = parser.parseFromString(xml, "text/xml");
      } else {
        throw new Error("Can't find an XML parser!");
      }
    }
  }

  public selectSingleElement(xmlNode, elementPath: string): Element {
    return <Element>this.selectSingleNode(xmlNode, elementPath);
  }

  public selectSingleNode(xmlNode, elementPath: string): Node {
    if (xmlNode == null) {
      xmlNode = this.xmlDoc;
    }

    if (this.xmlDoc.evaluate) {
      var doc = xmlNode.ownerDocument;
      if (doc == null) {
        doc = xmlNode;
      }
      var nodes = doc.evaluate(elementPath, xmlNode, null, XPathResult.ANY_TYPE, null);
      var results = nodes.iterateNext();
      return results;
    } else {
      return xmlNode.selectSingleNode(elementPath);
    }
  }

  public selectElements(xmlNode, elementPath: string): Element[] {
    return <Element[]>this.selectNodes(xmlNode, elementPath);
  }

  public selectNodes(xmlNode, elementPath: string): Node[] {
    if (xmlNode == null) {
      xmlNode = this.xmlDoc;
    }

    if (this.xmlDoc.evaluate) {
      var doc = xmlNode.ownerDocument;
      if (doc == null) {
        doc = xmlNode;
      }

      var resultsArray = [];
      var results = doc.evaluate(elementPath, xmlNode, null, XPathResult.ANY_TYPE, null);
      var thisElement = results.iterateNext();
      while (thisElement) {
        resultsArray.push(thisElement);
        thisElement = results.iterateNext();
      }

      return resultsArray;
    } else {
      return xmlNode.selectNodes(elementPath);
    }
  }

  get documentElement(): Element {
    return <Element>(this.xmlDoc.documentElement);
  }

  public xml(): string {
    // for IE
    if (this.xmlDoc.documentElement.xml) {
      return this.xmlDoc.documentElement.xml;
    }

    // Chrome, FireFox
    if (this.xmlDoc.documentElement.outerHTML) {
      return this.xmlDoc.documentElement.outerHTML;
    }

    // Safari
    return (new XMLSerializer()).serializeToString(this.xmlDoc.documentElement);
  }

  public static getNodeText(node: Node): string {
    if (node == null) {
      return "";
    }

    var value: string = node.text;
    if (node.textContent) {
      value = node.textContent;
    }

    return value;
  }

  public static setNodeText(node: Node, value: string): void {
    if (node == null) {
      return;
    }

    if (node.text !== undefined) {
      node.text = value;
    }

    if (node.textContent !== undefined) {
      node.textContent = value;
    }
  }
}