Tuesday, July 29, 2014

Land Registry June 2014 data

I’ve uploaded the Land Registry property sales data for June 2014 to doogal.co.uk. Enjoy!

Friday, July 25, 2014

Setting HttpResponse.StatusDescription silently failing

We recently received a complaint from one of our customers. We provide some fairly simple reporting functionality, that allows more technical users to write their own SQL queries. The customer was building a report and when there was a problem with his SQL, sometimes he’d get a detailed error message, but other times he’d get nothing.

When we try to execute a query and an exception is thrown we catch the exception and write the error message to HttpResponse.StatusDescription so it can be displayed in the browser. This can fail if the message is longer than 512 characters long, but this is fairly obvious since setting the StatusDescription property will throw an exception. That clearly wasn’t the problem here.

I finally managed to reproduce the problem but was still confused about what was causing the issue. Then I spotted the difference between the working case and the non-working case. The non-working case contained new line characters. Thinking about it, this was fairly obviously going to be a problem. The HTTP response would not be valid, since the status description header would be split over two lines. But it would certainly be preferable if .NET threw an exception in this case, rather than just silently failing to set the status description.

So our code now looks something like this

        context.Response.StatusCode = 500;
        context.Response.TrySkipIisCustomErrors = true;
        string description = ex.Message.Replace('\n', ' ').Replace('\r', ' ');
        if (description.Length > 512)
          description = description.Substring(0, 512);

        context.Response.StatusDescription = description;
        context.Response.ContentType = "text/plain";
        context.Response.Write("An error occurred - " + ex.Message);

Sunday, June 29, 2014

Land Registry May 2014 data

I’ve uploaded the Land Registry data for May 2014 to doogal.co.uk. For all the talk of a house price bubble, the Land Registry data (arguably the most accurate of all the house price indices) doesn’t seem to show much movement at all over the last few months. Of course it’s a different story in London. There was a story about the housing insanity in Hackney a few months ago, and looking at the sales in East London does show prices shooting up over the last year.

Tuesday, May 27, 2014

ONS postcode data for May 2014 uploaded to doogal.co.uk

I’ve just uploaded the latest postcode data from the ONS to my site. There are over 2.5 million postcodes in there, alive and dead. My data checks suggest everything is in order, but let me know if you find a problem.

Monday, May 19, 2014

Downloading Javascript generated data

I have a number of web pages that generate some data in text areas using Javascript. The only way users could download this data was to copy and paste the contents of these text areas, but I wanted to add a download button to simplify the process. The problem is that this simply isn’t possible in Javascript. The only client-side solutions I’ve seen either require Flash or are not supported in all browsers.

So I came up with a slightly hacky and inefficient solution. The basic idea is to post the data to the server and get the server to return it to the client as a download. The HTML looks like this

      <form action="download.php" method="post">
        <div>
          <input type="hidden" name="fileName" value="locations.csv" />
          <input type="submit" value="Download" />
        </div>
        <textarea id="geocodedPostcodes" style="width:100%;" rows="20" name="data"></textarea>
      </form>

All that is needed is a hidden field that tells the server-side script what the download file name should be and a text area with a name of “data”.

The server-side script is pretty simple, it looks like this

<?php
  header('Content-Disposition: attachment; filename="' . $_POST["fileName"] . '"');

  // add data
  print($_POST["data"]);
?>

All it does is get the requested file name and echo back the data.

It’s seems a bit crazy (and a waste of bandwidth) that this seems to be the only way to achieve a seemingly simple task, but that looks to be the case. I’d be happy to be proved wrong.

Sunday, May 11, 2014

Help me go on a bike ride

Last year I saw the various Ride London rides on the telly and rolling through Kingston and fancied doing it myself. Riding round London and Surrey on traffic-free roads is very appealing, compared to the usual stop-start, take your life in your hands experience of cycling round these parts. So the first chance I had, I applied in the ballot for the Ride London-Surrey ballot. And in January I heard I’d missed out on getting a place.

There was one more option. Sign up with a charity and raise some money and get a guaranteed place. So I decided to try and help out Cancer Research. Why Cancer Research? Primarily because cancer affects so many people at all stages of life but also, on a personal level, one of my partner’s best friends lost her life to cancer a couple of years ago, before she reached the age of 40.

I’ve set up a page for donations, added a link from my website and been amazed by the number of people who don’t know me who’ve already donated. If you’ve found this blog or my website useful, or are just feeling generous, then please consider donating some money. I will certainly appreciate it, as will Cancer Research.