Land Registry data for May 2017

I’ve uploaded the latest Land Registry data for England and Wales to my website. My earlier predictions of house price growth going negative appear to be incorrect. Prices continue to go up but at an ever decreasing pace.

Recovering data from a crashed Garmin 810

I love my Garmin and it would be a perfect piece of tech if it weren’t for one problem. Almost every time I use it to follow a course, it will die at some point during the ride. I can turn it back on and after several minutes of confusion it will figure out where it is and I can resume recording. The suggestions on the Garmin forum mostly seem to be to downgrade to an older version of the firmware, but I’m too lazy to do that and keep hoping Garmin will fix it at some point.

But they haven’t yet, so I just live with it. So I thought I’d document what I do to recover the data from before the crash. When I plug the Garmin in, the data from after the crash will sync to Strava but it will also complain that something failed to sync. So the first thing to do is to manually upload the pre-crash data to Strava. This will be a .fit file in the Garmin\Activities folder of the device. Strava seems to be able to cope with the file, even though the Garmin sync can’t.

You may be happy enough just doing that, but I like to have an accurate record of my rides so I want to merge the two together. So I now

• Export the two activities from Strava as GPX files.
• Open them both in this tool.
• Select both files, right click and merge them
• In the newly created file, select both tracks, right click and select Restructuring/Join
• Save the new file
• Delete the two original activities from Strava
• Upload the newly created file to Strava and feel satisfied with all the PRs I’ve hopefully achieved…

Property data for England and Wales April 2017

I’ve uploaded the April 2017 Land Registry data to my website. The annual change is still positive and although it’s heading towards zero, the progress is slowing.

GB postcode data May 2017

I’ve updated my postcode pages with the latest ONS postcode data. Any problems, let me know

Land Registry house price data for March 2017

Another month passes by and we have a new batch of house price data from the Land Registry. The annual change remains positive but continues to creep downwards. In some areas, the annual change has gone negative, you can see more detailed figures here.

Land Registry house price data February 2017

The latest Land Registry sales data for England and Wales has been released and my server is currently churning through the data. The top level annual change remains positive but continues to trend downwards.

Postcode data for February 2017

The ONS has released its latest postcode data, which is now on my site. I’ve run my usual sanity checks and it all seems to be in order, but let me know if you spot any anomalies.

Land Registry data for January 2017

Another month has flown by and it’s time for a house price update from the Land Registry. Sale volumes remain stable and prices continue to rise at an ever slower rate. It looks like the annual change will turn negative in 5 months if things continue in the same way.

Land Registry house price data December 2016

I’ve uploaded the Land Registry house price data for December 2016 to my website. The annual change is still positive but continues to head downwards. It now looks like it may turn negative in 6 months time.

Property sale data November 2016

I’ve uploaded the latest house price data from the Land Registry for England and Wales to my website. I’ve also made some changes to the way annual changes are calculated. This means the annual change figure is much less volatile, but it does mean changes in the directions of travel are slower to appear. The current annual change is 3.5% but reducing each month. If the current trend continues, the annual change will go negative in about 9 months time.

UK stations updated with 2016 usage data

I’ve uploaded the UK station usage data for 2016 to my site. I’ve also made a few minor tweaks to the layout which hopefully make the data more useful

UK postcodes for November 2016

I’ve uploaded the latest postcode data from the ONS to my website. I’ve run my usual checks and it seems to be OK, but as ever let me know if something seems amiss

Land Registry house price data for October 2016

I’ve uploaded the October 2016 Land Registry house price data to my website. Prices are just about managing to maintain a positive annual increase. Although houses are continuing their upward march, there seems to have been a mini collapse in the price of flats, I’m guessing due to the Buy To Let tax changes.

Land Registry data for September 2016

I’ve uploaded the latest house price data to my website. Prices keep creeping upwards, although there are lots of regional variations.

England and Wales house price data August 2016

As I type this, my server is churning through the latest Land Registry data for August 2016. The top level data has been imported and it suggests prices continue to glide ever upwards. The only difference seems to be the annual change is getting smaller, whereas it’s been around 5% for the last few years, it’s now around 2%.

If you look back at the data for earlier this year, you can now clearly see the spike in sales in March just before the buy to let Stamp Duty hike. Sales now seem to have returned to the levels seen after the financial crisis.

England and Wales house price data for July 2016

I’ve uploaded the latest house price stats from the Land Registry to my website. As usual, there is very little to report, prices continue to rise gently. Maybe this apparent calm is what leads up to the Minsky Moment?

UK Postcode data August 2016

I’ve uploaded the latest data from the ONS to my website. I have run my usual sanity checks, let me know if you spot anything amiss

Google Maps Distance Matrix may not be what you’re after

For many years I’ve been using the Google Maps APIs on my website. It’s been fun to use and until recently the licensing has been very unrestrictive. If an API returned a response saying you’d gone over the query limit, just wait for a second or so, try again and generally it would work. So with some use of setTimeout, it was possible to build reasonably scalable apps that cost nothing.

It’s looking like those days are coming to an end. The various APIs are starting to introduce hard limits on their usage. Once you’re over the limit, that’s it until the counter resets at the start of the next day. I first hit this with my use of the Directions API in my Driving Distances page. I can’t say I’m too happy with the way it was introduced, the Google API Console had given no previous indication of my usage of the API but on the same day as they started displaying the usage report, the hard limit was also introduced. Since my site was way over the limit, that page fell over almost immediately.

So I had a Baldrick cunning plan. I’d swap out the Directions API for the Distance Matrix API. This is exactly the kind of application this API was designed for. Unfortunately I failed to read the usage limits correctly and after I uploaded the new code, the page fell over in a heap again after a few hours. It turns out the usage limits apply to the elements passed to the Distance Matrix API, not the number of requests. So a 10 by 10 matrix counts as 100 towards the free 2,500 limit, not 1 as I had assumed. Given that this API provides less information and has fewer options than the Directions API but has the exact same usage limits, this is rather disappointing!

I am trying to figure out the best way forward now. I could start to pay for extra requests, but since a 100 by 100 matrix would cost $5, the costs could mount up quickly. I can put a maximum daily cost on the account so I don’t have to pay enormous amounts if someone overuses the page, but this could lead to the page becoming unavailable again.

I suspect the outcome will be me removing the page, or at least no longer linking to it from the rest of the site. I make a tidy sum from Google AdSense advertising on the site, but I fear this may just be the start, as more and more APIs start to introduce a hard limit and I don’t particularly want to pay that money back to Google every month to pay for their APIs. It was fun whilst it lasted I guess!

textareas slow in Chrome

So there’s a page on my site, https://www.doogal.co.uk/BatchGeocoding.php, that someone complained about. Specifically, they complained that if they tried to geocode 3,000 postcodes, it was terribly slow. I tried it myself and experienced the same problem. When geocoding postcodes, the page uses my own internal database, so it should suffer none of the throttling issues of Google Maps. No worries I thought, I can reproduce the problem, which is generally the biggest hurdle, fixing it should be straightforward.

So I fired up Chrome’s profiler and found… absolutely nothing… None of the delays were in my code. So I tried Microsoft Edge and it was super quick. I pretty much gave up at that point and suggested the user tried MS Edge.

Three weeks later I had a look with fresh eyes. And something popped up from the recesses of my mind, spellcheck=”false”. I vaguely remembered setting that attribute on a textarea in the past had improved performance and once again, this fixed the issue. A single geocode was previously taking a second, now all 3,000 took a couple of minutes. This may be a bug in Chrome or maybe spellchecking is a very CPU intensive process. Either way, turning it off makes everything better.

As always this is just a reminder for me and maybe it will be useful to someone passing through.

Property sales data for May 2016

I’ve uploaded the latest Land Registry data to my site. Every month I say the same thing and I will do so again. Prices continue to rise at a steady pace…