Google’s weird support for KML

KML is a standard XML format used to describe geographic data. It’s been owned by Google ever since they bought the company who originally developed Google Earth. So you’d think they’d have great support for KML in their products, right? I thought so as well until the other day when I was asked about a problem with some of the KML from my site. So here’s a summary of how different Google products cope with KML.

Google Earth – As you’d expect, Google Earth has full support for KML

Google Maps Javascript API – Using the KmlLayer class, it’s possible to load up KML files from anywhere on the web. There are some limitations on file sizes and not all of the features of the KML file format are supported, but for simple points and polygons it works perfectly well.

Of course, not everyone is a programmer, so using the Javascript API is not really an option for a lot of people. Unfortunately this is where things get messier…

Custom Maps (Maps Engine) – If you try to create a custom map in the new Google Maps, importing from KML is not even an option (the user interface says ‘Google Engine Lite’, so maybe if you pay some money to Google you’ll be able to import KML), which means you have to switch back to…

Custom Maps (Classic) – This claims to support importing KML. My experiments suggests it copes with KML files containing points (up to a fairly small maximum size), but I couldn’t get it working with files containing polygons.

Fusion Tables – Hidden in a corner of the web, is Google Fusion Tables. Basically it’s a spreadsheet with maps and has always seemed like a cool piece of technology but it hasn’t been pushed by Google much. But it seems Fusion’s support for KML is much better than either of the Custom Maps options. Points and polygons worked for me.

What I find weird is that all the web based technologies above are produced by a single company and all do pretty much the same thing, take a KML file and display it on a map, and yet they all clearly use completely different code. Maybe it’s time for some consolidation there?

Bing Maps – If none of the Google options work for you, maybe Bing Maps will do the trick. Points and polygons seem to work, with a couple of restrictions. There’s a fairly low limit on the maximum file size and the Import button only seems available in Internet Explorer.

No doubt there are other options, so let me know if you find any better alternatives.

Land Registry sales data back to 1995 and a new server

After much fighting with my old server trying to import the 18 million property sales from 1995 to today, I decided it was time to upgrade the server. I’ve now got 4 times as much memory, twice as many CPU cores and I’ve successfully imported all the Land Registry data. I’ve flicked the switch to point doogal.co.uk to the new server and so far everything is looking good. Let me know if you see any problems or have any performance problems. Also let me know if you’d like the property sales data sliced and diced in any other ways.

Handler "PHP55_via_FastCGI" has a bad module "FastCgiModule" in its module list

Trying to get a PHP app working on Windows Server 2012, I got the following error message

“HTTP Error 500.21 - Internal Server Error

Handler "PHP55_via_FastCGI" has a bad module "FastCgiModule" in its module list”

I’d installed PHP via the IIS Web Installer but had failed to do one more thing, enable CGI in ‘Add Roles and Features’ in Server Manager, under Web server (IIS)/Web Server/Application Development. After enabling that, everything sprang to life

The Land Registry destroyed my database

I started to import all the Land Registry data back to 1995 into doogal.co.uk, which was taking a looooong time. So I tried a little optimisation which involved clearing out some of the data I had already imported. Unfortunately that corrupted my database, which meant I had to drop my property sales table and start again from scratch.

So the property sales data on the site has less information than it did, but hopefully it should all be back online in the next few days. 

Kingston Council optimise their revenue capturing techniques

Not content with having one of the highest council taxes in the UK…*

Local authority	2013-14 Average Band D (excluding Parish Precepts)
Gateshead	1,443.20
Rutland UA	1,430.51
Hartlepool UA	1,418.70
Walsall	1,410.26
Nottingham UA	1,404.42
Stockport	1,397.05
Oldham	1,392.95
Kingston-upon-Thames	1,379.65

Not content with charging us £80 a year to park on a street somewhere near our house…

Kingston Council have made some more changes. Now not only can we not park on our own street most of the day, we can’t load there either. Practically I’m not sure how this is meant to work. How are we ever meant to get something delivered? How are older residents meant to get picked up from their house? But not happy with this change, the council have also introduced a parking enforcement smart car that roams the streets trying to catch any parking offenders. This was a big money spinner for Sutton council and I guess it will be the same for Kingston. Last Saturday I was caught unloading my car by this roaming enforcer. Unlike traffic wardens, it’s impossible to have any kind of rational discussion with them since they drive off as soon as they’ve got their photographic evidence.

So what to do? Well Kingston Council maybe behaving like the Goldman Sachs vampire squid, but fortunately there are local elections coming up soon. Frankly anyone who stands in that election who will stand up for residents’ interests will get my vote. Which pretty much excludes any of the three main parties…

And maybe another option is just to not play the game anymore. Get rid of the car, try to reduce my income to a level where I no longer have to pay the full council tax. Or just move somewhere with a more sensible level of council taxation.

Update – I appealed against the ticket since there were no markings on the pavement to show loading and unloading was restricted, and my appeal was accepted. I’m happier, but still not satisfied with the council’s tactics.

*Just look at the original document I pulled this data from to see the dramatic difference in council tax levels, can anyone explain these vast differences?     

Land Registry Oct 2013 data uploaded to doogal.co.uk

I’ve uploaded the latest Land Registry data to my website covering sales in October 2013.

In more interesting news, with a few days to spare the Land Registry have met their pledge to upload all sales data from 1995 to today. That will take a little longer to upload…

November 2013 ONS postcode data uploaded to doogal.co.uk

I’ve uploaded the latest ONS postcode data to my website. My data integrity check suggest it’s all fine but the size of the CSV download has shrunk a bit. I’m not entirely sure why that is, so let me know if you spot any anomalies.

Joining the dots

Today's BBC business pages contain these two headlines
Black Friday flurry for UK retailers
UK household debt hits record high
Black Friday seems to be another pointless tradition we've just picked up from the US, to give us all another chance to buy more shit we don't need and, it would appear, we can't even afford. I will not be participating.

Making cycling safer in London

Cycling is in the news due to several deaths in London. As someone who has cycled quite a lot in and around London (although very little in Central London) I have a few suggestions.

1. Don’t panic. Yes, cyclists have died and that is a tragedy for the families involved. But the number of deaths has remained fairly constant, even with an increasing number of cyclists on the road (see this document for details). And more pedestrians die than cyclists (where is the uproar about that?)

2. Get involved. Cycling may be no more unsafe than in the past but we still compare badly to other countries, even if our own government ministers try to bend this obvious truth (of course Norman Baker is an utter berk, just read his poorly researched conspiracy theory about the death of David Kelly). And pressure from the public may make a difference. Space for Cycling are targeting the upcoming local elections, which seems like a good idea.

Another obvious target for pressure is Boris Johnson. Boris seems to want to be a friend to motorists and to cyclists, which is a noble plan but is doomed to failure. Cars and bikes do not mix, so anything that is going to improve life for cyclists will almost certainly make life worse for cars. So Boris, which side are you on?

3. Help yourself. Here are some things I do when I’m riding my bike

• Find a route that you are comfortable with. One of the weird things about the cycling super highways is they encourage cyclists on to major arterial roads. Generally I try to avoid main roads as much as possible, since they are a pretty unpleasant experience. If you’re commuting to the same location day after day and you don’t feel safe on the route you’re using, then find an alternative. This is is where a smart phone comes in really useful. You can nip down some unfamiliar roads to see if the route is better and if you get lost, you can find your way using the phone’s maps. If the route is better, your phone should be recording where you went.
• Wear a helmet. I’m still surprised when I see a cyclist without a helmet, or even more bizarrely a child with a helmet sharing a bike with a parent without a helmet. It might not help if you get dragged under a lorry, but it will certainly help for less serious accidents.
• Don’t jump red lights. I have to admit I don’t always stick to this, but there are many junctions where you would have to be insane to ignore the lights.
• Assume every driver is an idiot. Most drivers are sensible but there are still plenty who will do daft things, like turning at a junction without indicating, or pulling out right in front of you.
• Don’t undertake lorries. Undertaking in general is not a great idea, but undertaking lorries is an even worse idea since they likely won’t be able to see you at all.

Getting the icon for the user’s default browser in C#

In my last post I showed how to get hold of the associated icon for a file. Now, say you want to get hold of the icon for the user’s default browser, you may think you can just grab the icon for HTML files and that’ll do the job. Whilst that will work for some users, it won’t work for all of them. The application used to open HTML files does not need to be the same application that is used to open websites. For example changing your default browser to Chrome won’t change the default application for HTML files to Chrome. So here’s an extension to that previous code to get the icon for the default browser. It won’t work for XP since the way default applications are handled has changed but it should work with all later OSes.

    private static string GetDefaultBrowserPath()
    {
      const string userChoice = @"Software\Microsoft\Windows\Shell\Associations\UrlAssociations\http\UserChoice";
      using (RegistryKey userChoiceKey = Registry.CurrentUser.OpenSubKey(userChoice))
      {
        if (userChoiceKey != null)
        {
          object progIdValue = userChoiceKey.GetValue("Progid");
          if (progIdValue != null)
          {
            string progId = progIdValue.ToString();
            const string exeSuffix = ".exe";
            string progIdPath = progId + @"\shell\open\command";
            using (RegistryKey pathKey = Registry.ClassesRoot.OpenSubKey(progIdPath))
            {
              if (pathKey != null)
              {
                string path = pathKey.GetValue(null).ToString().ToLower().Replace("\"", "");
                if (!path.EndsWith(exeSuffix))
                {
                  path = path.Substring(0, path.LastIndexOf(exeSuffix, StringComparison.Ordinal) + exeSuffix.Length);
                }
                return path;
              }
            }
          }
        }
      }

      return null;
    }

    public static Icon GetDefaultBrowserLargeIcon()
    {
      string browserPath = GetDefaultBrowserPath();

      if (!string.IsNullOrEmpty(browserPath))
        return GetLargeIcon(browserPath);

      // last chance (probably XP), just grab the icon for HTML files
      return GetLargeIcon("test.html");
    }

    public static Icon GetDefaultBrowserSmallIcon()
    {
      string browserPath = GetDefaultBrowserPath();

      if (!string.IsNullOrEmpty(browserPath))
        return GetSmallIcon(browserPath);

      // last chance (probably XP), just grab the icon for HTML files
      return GetSmallIcon("test.html");
    }

Getting the associated icon for a file in C#

Sometimes it’s useful to display an icon for a file in an application and probably the best icon to display is whatever the operating system uses. The Windows API provides the SHGetFileInfo function for this purpose, so here’s a little wrapper around it. All the methods take a file name parameter, but the file doesn’t need to actually exist.

using System;
using System.Collections.Generic;
using System.Drawing;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;

namespace TestIcons
{
  public static class ImageUtilities
  {
    [StructLayout(LayoutKind.Sequential)]
    struct SHFILEINFO
    {
      public IntPtr hIcon;
      public IntPtr iIcon;
      public uint dwAttributes;
      [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 260)]
      public string szDisplayName;
      [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 80)]
      public string szTypeName;
    };

    static class Win32
    {
      internal const uint SHGFI_ICON = 0x100;
      internal const uint SHGFI_LARGEICON = 0x0; // 'Large icon
      internal const uint SHGFI_SMALLICON = 0x1; // 'Small icon
      internal const uint SHGFI_USEFILEATTRIBUTES = 0x10;
      internal const uint SHGFI_LINKOVERLAY = 0x8000;

      [DllImport("shell32.dll")]
      public static extern IntPtr SHGetFileInfo(string pszPath, uint dwFileAttributes, ref SHFILEINFO psfi, uint cbSizeFileInfo, uint uFlags);

      [DllImport("User32.dll")]
      public static extern int DestroyIcon(IntPtr hIcon);
    }

    public static Icon GetSmallIcon(string fileName)
    {
      return GetIcon(fileName, Win32.SHGFI_SMALLICON);
    }

    public static Icon GetSmallOverlayIcon(string fileName)
    {
      return GetIcon(fileName, Win32.SHGFI_SMALLICON | Win32.SHGFI_LINKOVERLAY);
    }

    public static Icon GetLargeIcon(string fileName)
    {
      return GetIcon(fileName, Win32.SHGFI_LARGEICON);
    }

    private static Icon GetIcon(string fileName, uint flags)
    {
      SHFILEINFO shinfo = new SHFILEINFO();
      IntPtr hImgSmall = Win32.SHGetFileInfo(fileName, 0, ref shinfo, (uint)Marshal.SizeOf(shinfo),
        Win32.SHGFI_ICON | Win32.SHGFI_USEFILEATTRIBUTES | flags);

      if (hImgSmall == IntPtr.Zero)
        return null;

      Icon icon = (Icon)Icon.FromHandle(shinfo.hIcon).Clone();
      Win32.DestroyIcon(shinfo.hIcon);
      return icon;
    }
  }
}

None of our politicians have a clue

So we have a Labour party who are talking about a cost of living crisis. Good on them. Until you hear their main policy, which seems to be fixing the cost of energy. I’m pretty sure the talk of the lights going out are completely alarmist, but I doubt trying to fix prices is going to work, the fact is fossil fuel costs go up and down and no politician can stop that happening.

But Ed is missing the point a little anyway. For most people, energy is a big cost but it’s generally outweighed by something else, housing. We hear a lot about energy price rises, but not so much about rises in rent. And if house prices go up, that’s a good thing apparently? As a home owner, I don’t get it. If I want to move up the ladder, higher prices mean I need a bigger mortgage to move anywhere. First time buyers presumably don’t want higher prices. As a parent, I don’t want housing to be completely unaffordable for my child when she gets to an age when she wants her own place. Yes, there are people who gain from house price rises, but I’m pretty sure they are outnumbered by people who would actually gain from falling prices.

But enough of the Labour party, what have the Conservatives got to offer? Oh yes, Help To Buy, the scheme that seems most people are a little concerned about. Apparently it may cause a new housing bubble. What the ….? We haven’t escaped the last bubble yet. We are still number one in the housing bubble stakes. What George seems to have missed is that the problem with our housing market is not a demand problem, there’s plenty of demand. The problem is not enough houses are being built and the ones that are being built cost too much!

So what’s the solution? First, the British need to get over their obsession with house prices and realise houses are a place to live, not pay for their retirement. Second, we need to radically change our planning laws, so we can build a lot more houses. Third, the government needs to start building some council houses.

The sad thing is none of this looks likely to happen. No political party is going to say “We will build houses in your back yard and let other people do the same and the price of your property will then fall, but in the long run it will be good for the country”

Our dog really hates cats

Eyes and nose removed, most internal organs gone. Fortunately it wasn’t a real cat, just a toy.

Postcode populations

doogal.co.uk has some exciting new postcode data (depending on your definition of exciting…). I’m in the process* of importing population data for each active postcode in England and Wales. This includes the estimated population and number of households and comes from the ONS website based on the Census of 2011, which I’ve just realised only covers England and Wales. Scottish data is a separate download and will be my next task. Northern Ireland doesn’t seem to provide population data per postcode.

The data appears on the information page for each postcode and will become part of the CSV downloads once the import is complete. I could aggregate this data up to postcode district and sector level, so let me know if that would be useful to you.

* A database of 2.5 million postcodes can take some time to update!

More UK parliamentary constituency postcode data on doogal.co.uk

I’ve added a UK parliamentary constituency page to doogal.co.uk for England, Wales and Scotland (Northern Ireland continues to not provide its data under an open license). You can view maps and grab KML and CSV downloads of the postcodes for each constituency.

If you have requests for any different slicing and dicing of this data, let me know.

National Park postcodes

Postcode data can be diced and sliced in lots of different ways. So I’ve sliced the UK postcodes so you can grab data for each National Park. I’ve no idea if this will be useful to anyone, but sometimes the weirdest things turn out to be hugely popular, so here it is

http://www.doogal.co.uk/NationalParks.php

Data is available in CSV and KML formats.

“unknown CMAP subtable format” error when generating PDFs

Whenever I get an unusual exception being thrown by my application that I don’t know how to fix, I Google the error message which generally leads to some kind of explanation of what’s happening. But Google seems to have a hole in its knowledge regarding this “unknown CMAP subtable format” error I was getting. So I’ll try and fill that hole.

Our software uses XSLT to generate XSL:FO files that are then converted to PDF files using AltSoft’s XML2PDF engine. The error was occurring deep inside their assembly and the only clue I could get from looking at the call stack was that the problem was font related. My immediate suspicion was that the error was due to the user using a strange font in their input, but the fonts in use were fairly standard (Arial, Calibri).

I eventually figured out the problem was due to some of the particular characters being used in the input text. It seems the user had pasted some text from Word that contained some characters that weren’t available in the fonts used in the output. Cleaning up the user’s input text stopped the exception being thrown.

Postcode information sites

The release of free postcode data from the Ordnance Survey with very relaxed licensing has led to a raft of sites showing off this data in different ways. I’ve compiled a list of the ones I’ve spotted and will add to this list if I find anymore.

http://www.doogal.co.uk/UKPostcodes.php – Obviously first up is my own site. I’m unable to write an objective review, so won’t.

http://www.free-postcode-maps.co.uk/ – This site doesn’t provide all the postcode data (understandably since it looks like it’s been set up by a company who sell this data in various flavours) but does provide some nice map overlays showing postcode districts and sectors. So nice in fact, I may borrow some of their ideas for my own site.

http://www.britishpostcodes.info/ – I was a bit upset when a search for my own postcode brought up this site above my own site, but it does provide a lot of useful information for each individual postcode. I haven’t figured out where all the data has come from, but once I do, I’ll probably get some of it on my site.

http://postcodeof.co.uk/ – Again a search for a postcode brought up this site above my own, very disappointing. But the site was intriguing, if only for the broken English in the content. A quick domain check showed it was registered by someone in Serbia. I never realised Serbians had a passion for UK postcodes! The other interesting thing is the use of sub-domains for each individual postcode, presumably for SEO reasons. There’s not much data for each postcode, but things may improve.

http://mapit.mysociety.org/ – If you have a postcode and you want to find out what administrative areas it is part of, this site if useful. You can find the electoral constituency, local authority wards, census areas etc and also grab the boundaries of those areas. It also has an API, so you can automate your data grab!

Land Registry to release all data from 1995 onwards

The Land Registry has been releasing its data under the Open Government Licence since the start of 2012. This was great but only covering the last year and a bit meant it wasn’t that useful. But when I popped over to the Land Registry site the other day I noticed they’d got some historical data and are planning to release all of their data from 1995 onwards.

I’ve sucked this data into my site, but I haven’t done anything particularly exciting with it yet. It’s interesting (to me at least) that the average price that I’ve calculated using a straightforward mean (about £230,000) is so different to the average price that the Land Registry comes up with (about £160,000). They do some fancy maths on their numbers but it still seems odd that they are so different.

But me and the Land Registry agree on one thing, the trend. House prices have basically gone nowhere for the last 3 or 4 years.

Postcode electoral constituency data on doogal.co.uk

I’ve just recently imported the latest ONS UK postcode data and added the Westminster constituency data at the same time (except for the Northern Irish postcodes, which still aren’t covered by the OpenData initiative). I’ve only surfaced this on the map pages for individual postcodes, but I’ll show it in other places as time allows. Let me know if you need this chopped up in any particular way.

Website ideas

I often have ideas for websites, but due to a lack of time I never actually get round to doing anything about them. They may well be terrible ideas, or may already exist in some form already, and almost all of them are guaranteed to not make any money, but I present them here in case anyone wants to pick up the baton and run with them.

Random acts of kindness

Say you have some spare cash and you want to give it to a good cause, but you don’t have a good idea about who to give it to. Wouldn’t it be great if there was a website where you could post a message saying ‘I have some money, who wants it?’ and from the other side people could post messages asking for money, help or whatever? When you find someone you want to help, you can contact each other and arrange to hand over the cash (although there needs to be some kind of protection here against receiving shedloads of begging emails).

Domain reminder

So you’re interested in buying that doogal.com domain (oh, that’s just me is it?) but somebody has bought it already and is sitting on it, only being prepared to sell it for a ludicrous 2800 euros? If you’re willing to wait, wouldn’t it be great if there was a website that would send you reminders when the domain name was about to expire? And whenever the status of the domain changed in any other way?

Foreign exchange market

The difference between the official exchange rates and the price you’ll pay to exchange money yourself can be huge. But whenever I convert 100 pounds to euros, there’ll be thousands of other people who want to take their euros and convert them into pounds, so how about a website to match the buyers and sellers? I’ve not really thought through how the exchange takes place, but that’s just a minor detail…

Re-open nominations

This is more than just a website, this is a political movement. Back in the day when I was at university, any election would include RON, re-open nominations. So if you weren’t happy with any of the candidates, you could vote for RON and if enough people voted for him then the election would be rerun. Currently I’m pretty disappointed with all three of our major parties, since there seems to be very little difference between them in terms of policy and they all seem to be led by people who have no experience of the real world. But there is no way to register the fact that I still believe in the democratic process but don’t want to vote for any of the options available to me. So for my take on this idea in real elections, the RON Party would stand in every election, promising to immediately stand down if elected. Along with that, it would lobby the government to include RON as an option in all elections.

Fat cyclists

I enjoy cycling, but have no desire to wear lycra. There are lots of clubs for cyclists, but they are generally full of extremely fit people who cycle 50 miles a day. So how about a web community for people who like to cycle but aren’t necessarily as fit as they could be and don’t take it too seriously so they can get together and have a slow ride together?

Poor man’s XSLT profiling in detail

A couple of years ago I wrote a post giving a vague idea of how to profile XSL transforms without shelling out for Visual Studio Team System. I went back to it recently and realised it didn’t really provide enough information on how to actually perform the profiling. So here’s another go at it.

The first thing we need to do is compile the XSLT into a .NET assembly. For this we need to use the XSLTC.EXE tool that ships with the Windows SDK. You can find this somewhere like this C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools. The command-line required will be something like this xsltc.exe /settings:dtd "/out:c:\temp\style.dll" /class:MyXslt "C:\temp\TEST.xsl"

So now we have an assembly for our XSLT file. Next we need a little wrapper program to load it up and call the transform. I used a simple command line app, with code as follows

    static void Main(string[] args)
    {
      XslCompiledTransform transform = new XslCompiledTransform();
      transform.Load(typeof(MyXslt));
      XmlTextReader reader = new XmlTextReader(@"C:\temp\input.xml");
      XmlTextWriter writer = new XmlTextWriter(@"C:\temp\out.html", Encoding.UTF8);
      transform.Transform(reader, writer);
    }

Now we are ready to profile the XSLT. In the past I’ve always recommended AQTime, but my old version of it doesn’t seem to work on Windows 8. I didn’t fancy paying for an upgrade, so looked around for alternatives and found the Eqatec profiler. The free version seems to work pretty well, certainly good enough for my limited needs.

So after running my little test app through the profiler, I had a much better idea of where things were slow in my XSLT. As is often the case, it was due to some XPath using “//element” to find elements. Often you can get away with it, but if the XML document is large or that piece of code is getting hit a lot then it can bite you. 

There is another slight complication with this approach to XSLT profiling. Some of the compiled templates may have a name of “compiler:generated”, meaning it’s pretty tricky to figure out how they relate to the original XSLT. From my experience, it seems that these templates are generated by the compiler itself when it decides to split larger templates into smaller compiled templates. I found ILSpy was pretty useful here to match the compiled code back to the original XSLT.

Traversing JavaScript objects in .NET

I’ve been playing around with hosting a Google Map in a .NET WinForms control, for no good reason other than to see if it can be done. The key part of this integration is calling JavaScript on a web page from .NET. This appears reasonably straightforward, the WebBrowser control has a Document.InvokeScript method that does the trick. If your JavaScript function returns a value, you can pick this up from the return value of InvokeScript.

This works great for simple types, but what if your JavaScript function returns a more complex type? This is where things get a bit more tricky. The returned object has a type of System.__ComObject and initially it looks like this has no useful methods on it. This is the object that is used when calling any COM object, but generally you’ll be able to import a type library to create a .NET friendly wrapper around the raw object. This obviously isn’t the case here.

So my first thought was to find the real underlying type of the __ComObject. This piece of code helped out here. It turns out the actual underlying type was a JScriptTypeInfo but there’s very little information out there about what this type does or how to use it.

But it turns out there’s a much simpler way to access the returned object, cast the returned object as IReflect and use its methods to get properties etc. So the code for my .NET property looks like this

    [Category("Map")]
    public LatLng Centre
    {
      get 
      {
        object centre = webBrowser.Document.InvokeScript("getCentre");
        if (centre == null)
          return new LatLng(0, 0);

        IReflect reflect = centre as IReflect;
        double lat = (double)reflect.InvokeMember("lat", BindingFlags.InvokeMethod, null, 
          centre, null, null, null, null);
        double lng = (double)reflect.InvokeMember("lng", BindingFlags.InvokeMethod, null, 
          centre, null, null, null, null);

        return new LatLng(lat, lng);
      }
      set
      {
        webBrowser.Document.InvokeScript("setCentre", 
          new object[] { value.Latitude, value.Longitude });
      }
    }

Another approach to this would be to have two JavaScript functions, getCentreLat and getCentreLng, which just return simple types, but that could get cumbersome with really complex types.

I’m not sure if it’s possible to pass complex objects into JavaScript but I haven’t needed that yet. Again, for really complex types, passing in each part of them could get messy.

A dog’s life

It’s a hard life for a dog. It all starts when the sun comes up. “I must inform my master that the sun has come up. Woof woof…”.

Some time later “… woof, woof. Ah good, my master is up to see that the sun has come up, I can now go back to sleep”. Thank you dog!

Around the world in 7 years

According to Endomondo, I’ve cycled 2668 miles since I signed up with them, which is just over a tenth of the distance around the world. According to my circle drawing page, this would take me from my house to the North Pole. I’m quite pleased with that.

The distance round the earth is about 24,900 miles, depending on how you measure it. I try to cycle 300 miles a month, so the time it will take me to cycle the whole way round the earth is 24,900/300 which is 83 months, or 6.9 years. So in about 6.1 years I’ll have completed my virtual trip. I’m hoping Endomondo is still around to record my achievement!

404s and Google Webmaster Tools

Trying to fix all the things Google Webmaster Tools complains about would drive you insane. Case in point, it will warn you about soft 404s, which are URLs that don’t exist (or contain no useful content) but don’t return a 404 HTTP error code.

I had a few of these on my site where a rebuild of the postcode database meant a few old postcodes had gone missing (here’s one). Although they weren’t any links to them from anywhere, Google never forgets about pages it’s crawled. I guess I could ask Google to remove all these URLs from its index, but I have a life.

So I started returning 404 errors from these pages, hoping it would make Webmaster Tools happy and make the web a better place. Then a few days later I got a warning message from Google telling me there had been an increase in not found errors on my site. Duh, well yeh, because that’s what you told me to do.

So all I’ve managed to do is move my errors from the soft 404 bucket to the not found bucket. Ho hum. It seems like keeping Webmaster Tools happy is actually impossible.  

The quickest way to charge a Nokia Lumia 800

I can’t say I’ve ever considered the differences between charging my Nokia Lumia 800 via USB and via the AC adaptor. If I had thought about, I would probably have assumed they charge at just the same rate. But this morning when I was thinking about going on a bike ride, I realised my phone was out of juice, so wanted to charge it ASAP. A little research showed using the AC adaptor would be much quicker than plugging into USB.

But in case you don’t believe me (And to be honest not believing random people on the internet is probably a good thing), you can check it out for yourself. From the phone dialler, type ##643# and the diagnostics tool will be installed. Run this app and look at the battery status section whilst charging. I found the AC adaptor was providing about twice as much power as the USB connection.

Post Office Recorded Signed For–Mmm, not so much

I needed to send my driving licence off to a company to prove I am who I say I am. Something to do with money laundering, although I’m not entirely convinced sending them my driving licence proves I’m not a criminal. Maybe I’m just a money launderer who can drive?

Anyway, wanting some peace of mind, I decided to use the Post Office’s ‘Recorded Signed For’ service which has “guaranteed signature on delivery” according to their website. That was on 17th December. Over 2 weeks later, when I try and track my package, the Post Office website tells me

Item BY260667256GB was posted at 46 Hawks Road KT1 3EG on 17/12/12 and is being progressed through our network for delivery.

The thing is I know this isn’t true, since the driving licence arrived back at my house a few days ago. So much for a guaranteed signature on delivery. I wonder if the status of my item will get updated to

Item BY260667256GB was posted at 46 Hawks Road KT1 3EG on 17/12/12 and we haven’t got an effing clue where it is.

Endomondo discovers how to get my money

When I logged into Endomondo the other day I discovered that in the process of redesigning their website, they had also decided to hide some of the useful stats from us users who weren’t paying them any money.

I could have started shouting and screaming at this point, since they’d had the temerity to take away my free toys. But frankly $20 a year for a very useful service is a price worth paying if it helps them to keep the lights on.

Of course this approach only works for sites that offer a valuable service, it probably wouldn’t work for quite a few sites (I’m looking at you Facebook!)

Windows 8 – How bad are these apps?

For most of my time I’ve been using Windows 8 much as I used previous versions of Windows and stayed well away from the new interface and apps. But I’ve recently started playing with some of the apps and frankly it’s all quite disappointing.

For a start, every app seems to take an age to fire up. This makes no sense, I have a reasonably specced laptop and yet apps take longer to load up than on my phone, even though the apps do the same thing! I really hope this is a problem with the apps, rather than the underlying platform, since at least the apps can get updated quite easily.

The other problem is some of these apps are pretty buggy. We’ve been playing solitaire on our Windows PCs for over twenty years and I can’t remember it ever crashing before, but it’s just happened about 5 times in the space of a few minutes. Ah well, maybe one of those updates advertised in the store is for solitaire? Hard to say since the store keeps telling me I’m not connected to the internet, even though I definitely am. And when I do manage to access it, I can’t actually get to the updates.

I’m not sure the good folk at Apple will be quaking in their boots at the moment.

UK postcode data is free, mostly

Today someone posted a comment to my site saying this

at [redacted] you can find a full uk postcode database with all postcodes and long / lat values. it is the only source i found where northern ireland is included. there is a download fee, future updates are free though. it contains approx 1.9 million postcodes and is complete and correct. despite the costs it seems to be recommendable for commercial projects given the completeness of the data and the update-service.

This immediately smelt a bit funny, so I headed off to their website. First I noticed the company was registered in the British Virgin Islands, which looked slightly odd. Next I checked the IP address of the commenter. Apparently he/she was located in the Philippines, which also seemed a bit strange for someone talking about UK postcodes.

So I’d be inclined to think this may not be the best company to buy UK postcode data from. But more importantly, this level of postcode data is available for nothing, with a few caveats (Northern Ireland, Channel Islands and Isle of Man are missing). Just grab it direct from the Ordnance Survey or from my website or a host of other sites packaging up the data in various ways.

Not that there is anything necessarily wrong with paying for postcode data, I provide the data for nothing and don’t promise to provide any kind of level of support, though I try to answer any questions people have. So if you want some kind of guaranteed support, paying might make sense.

One thing the free postcode data is missing is address level data. Whereas there are 1.8 million postcodes, there are 28 million addresses in the UK. To get hold of this data, you will need to pay money. And if you do want to pay for postcode data, this is almost certainly the dataset to buy, since it probably won’t cost much more than the smaller dataset and is much more versatile. And if you do decide to buy this dataset, I suggest buying it from one of the Royal Mail’s approved resellers, rather than some random dodgy site.

Update – I’ve only just been made aware of the fact that even the PAF dataset can be free if you are a small charitable organisation or a micro business. I’m not sure of the exact requirements you need to meet to get it for free, but it is certainly worth looking into.

Windows 8 start screen and search is broken

Before I start, I have to point out this is not a me too complaint about not liking the new Windows 8 user interface. I have no problems with it. Or at least I didn’t have until it started to look like this, with no icons. If I press the little [–] button at bottom right then small versions of my icons appear and I can then see the full size versions of them if I click somewhere on the screen. But if I want to search for an app, although Windows says there are apps there, nothing appears in the main window. Search results for settings and files do appear however.

 

I have no idea how it got into this state, or why, or how to get my icons and search results back. Since this is, I think, the only way to get to applications, this is kind of frustrating. It seems I now have to search round in Explorer to find the relevant EXE…

Update -  Turns out the fix for this is the old favourite, a reboot…

Windows 8 - The Module DLL C:\Windows\system32\inetsrv\rewrite.dll failed to load

After installing Windows 8, one of my AppPools in IIS kept stopping with the error ‘The Module DLL C:\Windows\system32\inetsrv\rewrite.dll failed to load’ appearing in my Event Log. Since I’m not really using the IIS URL Rewrite module, I tried removing it from IIS, but this didn’t fix the problem. But uninstalling it via Control Panel did fix the issue. Not sure what the underlying problem was and obviously this isn’t a great solution if you are using the URL Rewrite module, but it worked for me!

Windows 8 – what’s all the fuss about?

I’d read a lot about what a bad OS Windows 8 is for desktop systems, since it’s all about adding features for use on tablets. So being the masochist I am, I thought I’d install it on my desktop machine, a none too modern Dell box with a couple of monitors. Several hours later, it was done and I hit my first problem. Windows didn’t like my mouse. This made navigating round the new user interface a little tricky. But after swapping out the mouse for another one, things got better.

So the big complaint about Windows 8 is that the Start button is no more. But move the mouse to the bottom left of the screen and a little Start popup appears. Click on that and most of your apps are listed. For the ones that aren’t, just start typing and a list of matching apps appear. And here’s the important thing about that search facility, it’s much faster than the same search functionality in Windows 7, which always seemed to hang for seconds before doing anything. The learning curve for this was about 5 minutes, although I have to admit I do occasionally mistakenly click on the task bar rather than the Start popup.

So the biggest issue seems to be a non issue, for me at least. The things that I like so far are much faster boot times, a task bar that stretches across multiple monitors and a much improved Task Manager. Nothing revolutionary but welcome none the less.

I then installed it on my laptop. Much the same experience, although Minecraft no longer runs due to my graphics card driver not supporting OpenGL. I am not popular with my daughter.

There are shedloads of new things in that Start screen, but as a desktop user I was more concerned about getting to all the stuff I currently use and that all works fine. So what’s all the fuss about?

Executing multiple SQL statements against multiple databases

In my day job, each of the customers running in our hosted environment have their own SQL Server database. As we develop the software that runs on top of these databases, we often need to update the schema of each database. I knocked together a little console application in C# to do the job and here are the interesting parts of it. First, we need to get a list of the available databases, which can be achieved quite easily.

      List<string> DBs = new List<string>();
      // get list of available databases
      SqlCommand command = conn.CreateCommand();
      command.CommandText = "select * from master.sys.databases where DataBase_ID > 4";
      using (SqlDataReader reader = command.ExecuteReader())
      {
        while (reader.Read())
        {
          DBs.Add(reader.GetString(0));
        }
      }
      DBs.Sort();

Next, we need to execute the SQL updates. This isn’t quite as easy as you’d think. If you want to execute multiple SQL statements with GO statements between them, SqlCommand.ExecuteNonQuery won’t handle them. Fortunately SQL Server comes with some assemblies that solve the problem. So you need to add references to Microsoft.SqlServer.ConnectionInfo.dll, Microsoft.SqlServer.Management.Sdk.Sfc.dll and Microsoft.SqlServer.Smo.dll. Once they have been referenced, the following code should be able to handle any SQL you throw at it.

      foreach (string database in DBs)
      {
        Microsoft.SqlServer.Management.Smo.Server server = 
          new Microsoft.SqlServer.Management.Smo.Server(new ServerConnection(conn));
        server.ConnectionContext.ExecuteNonQuery("USE " + database + "\nGO\n" + sql);
      }

Lets build a socialist network

It was five years ago. We’d driven down to Spain to see my dad and as we sat in his living room I was amazed to see my teenage nephew and his mate typing ferociously on their laptops as they chatted away on the social network du jour (I’m guessing Facebook but I might be wrong). I remember thinking at the time what would happen if we could harness all that energy into doing something productive.

What I hadn’t realised is that Facebook had already figured out how to harness that energy. Most websites (ignoring the ones that sell stuff) draw viewers in by producing interesting content and make their money through advertising on the site. Facebook works in the same way, but it doesn’t produce its own content, it gets its users to do that. In fact it’s a brilliant system since the content that is produced helps to decide what adverts should get shown, but I can’t help feeling there’s a better way. If I write some content, I want to have the chance to earn some income from it. I know from my experiences here that the chances of me making a reasonable amount of cash are pretty low, but I still think the system where the advertising network takes a cut and the content producer takes a cut is a fairer approach.

I think there are two approaches possible. The first is to build a competitor to Facebook, where the users take a cut of profits. I think this could have a few problems, how do you decide who gets what?, how do you stop fake users signing up just for a cut?, how do you stop click through fraud to increase earnings?

So here’s another option, we develop open protocols that allow different sites to hook up together and provide similar functionality to Facebook (and Twitter and Google+ and LinkedIn). There’s not a lot to it as far as I can see, the basic actions are add a friend, follow someone (that’s pretty much RSS), like something, send a message to someone and publish a message (that’s essentially a blog post). None of it is rocket science. And then we all have a choice. A geek like me could mash all my social networking stuff together on my own site and have the chance to earn some money. Ordinary users could continue to use the services they want, new services would crop up that aggregated all your social networks together. Everyone wins…

Richmond Park and Thames route

Just testing out embedding a route from Endomondo

Bike It example Windows Phone app source code

I started to write my own biking app for Windows Phone then found an app that was much better than I could hope to achieve. So here is the source code for my app, which might be useful if you want to do something similar, or just want to see an example Windows Phone app. Be warned, the code is not particularly pretty.

Endomondo and the app that will never be

I’ve written before about the two bike apps I’ve found for my Windows Phone, MyBikeMap and MapMyRide. I was so convinced that they were the only two options available that I started to write my own bike app. I got quite far with it, then I noticed a new app, Cyclocomp. I downloaded it and was quite impressed but decided it had a fatal flaw, it lets the phone go to sleep, the same problem that MapMyRide had. Wondering if I’m doing something wrong or am using these apps in a way nobody else is I had a look around for a review of it. That led me on to another app Endomondo. So I thought I’d give it a go and was immediately impressed.

The user interface is simple and elegant, the displayed stats can be configured to whatever you prefer, the map takes up the whole screen and you can even receive pep talks (and you can turn them off!). And even better, the data gets uploaded to the Endomondo website where you can view the map and see lots of geeky stats about your rides. And you can even enter challenges with other users, challenges that I am bound to lose since there’s clearly some very fit, active people on there. My only complaint is the map occasionally seems to get confused and stops showing the trail of where you’ve come from. But for free, it’s pretty much perfect.

And it rather puts my feeble effort to shame, so I guess my bike app won’t be seeing the light of day…

I don’t rate Rated People (or maybe I do)

You’ve probably seen the adverts where Phil Spencer, the man whose remarkable negotiating skills can sometimes get £5,000 knocked off a £500,000 house purchase, tells us about this remarkable website which can connect us punters with local tradesmen. To me this seems like a good idea for a website and I have some work that needs doing, so thought I’d give it a spin.

So I filled in my details, entered the job details and waited for a response. And then, nothing. OK, not entirely true, I received a couple of text messages and emails. A couple of days passed and I received an email telling me that I hadn’t had a response and maybe I should expand on my job description to get a response. So off I went and updated the description and hit submit and then I was presented with a 404 error (for the non technical, this means the website is broken). So I tried again and got the same result (definition of insanity - doing the same thing over and over again and expecting different results). And I gave up at that point.

OK, slightly annoying, but no worries, I’ll drop them an email and let them know and I’ll update the description later when it’s fixed. So I go to their contact page. I can ring them, but not at this time or I can send them a tweet or write on their Facebook page or add a comment to their blog. No feedback form, no email address. I didn’t particularly want to berate them on a public forum, maybe the problem is something specific to me, so none of those options appealed. Yes, yes, I see the irony, I am now berating them quite publically (hello my many reader), but I’m annoyed now.

So, so far, Rated People is something of a failure for me. I think if you’re going to be spending a bunch of money on TV advertising, quite a bit of effort should also be spent on the product…

Update – A funny thing happened today, I got a call from Rated People, specifically due to this blog post. I’ve ranted about a few companies on here and never got any response from the company involved, so it was a pleasant surprise to get some feedback. I should also say that I did eventually manage to update my job description and also got a response from an interested builder, who is repairing my pointing as I type. So it would appear my initial review of Rated People may have been overly negative. And what’s more, Phil Spencer is no longer doing the voice over on their ads…

Smart phone + app + bike mounting > bike computer

I recently took possession of a new Nokia Lumia 800, thanks to work, and immediately thought it would make a great replacement for my Veloset GPS 600 bike computer. Hearing Windows Phone didn’t have many apps, I thought I’d have to write my own biking app, but it turns out there are already a couple available, both free.

The first is MapMyRide. This initially seemed pretty good, but it has one fatal flaw, it lets the phone go to sleep. If you want to look at the map as you ride it is pretty tricky. You really don’t want to be fiddling with your phone when you’re riding.

Next up is MyBikeMap, which has one big advantage in that it doesn’t let the phone sleep, so I can look at my map all the time during my ride. It’s also well designed, with a simple user interface and showing just the pertinent information on the screen displayed when riding. It’s not perfect, my main complaint is the lack of support for miles, but it’s hard to complain too much when it’s free and it does most of what I want.

The final piece of the puzzle is something to attach the phone to my bike. I went for this mounting, primarily because it came up first on a Google search. The phone fits it perfectly and I was pleased to realise I could leave the zip open slightly to still have access to the buttons and I was able to use the touch screen through the plastic cover. But it does have a somewhat major problem as I found out today. Although the coupling between cover and bracket on the bike is perfectly adequate when riding on a road, if it gets a jolt then the cover (and the phone) can go flying. And you might not even notice until you look down some time later.

The design of the mounting is kind of weird, the cover is attached to the bracket not once but twice. One of these couplings is pretty secure, but the other isn’t. So the solution I’ve come up with is to glue the weaker coupling and I’m hoping this solves the problem. But you may want to consider this issue before purchasing this mounting.

But other than the teething problems, I’m pretty happy with this set up. It’s great to have a map in front of me as I ride, since it gives me much more opportunity to try heading off down a road or track that I don’t know without worrying about getting completely lost. And it makes me think the market for high end bike computers may not last much longer. Why spend £200 on one of them when a smart phone can do the same job? 

KmlLayer error handling in Google Maps

Two years ago I noted that the KmLayer in Google Maps didn’t provide an event to inform me if there was a problem when loading the KML. Things have moved on and an event has been added to the API at some point. Usage is as follows.

  var kmlLayer = new google.maps.KmlLayer(url, { map: map });
  google.maps.event.addListener(kmlLayer, 'status_changed', function() {
    if (kmlLayer.getStatus() == 'OK')
      $('#status').html('');
    else
      $('#status').html('KML loading problem - ' + kmlLayer.getStatus());
  });

How to increase your AdSense revenue

My proper website has been around for about 12 years. For most of that time it had very few visitors and the money I made from advertising was minimal. Then a couple of years ago advertising revenue started going up. It’s now plateaued but I earn a nice wedge of cash ever month, not enough to give up the day job but enough to add a decent amount to my income. So what’s my secret and how can you do the same thing?

Firstly, you can optimise the placement, colours and number of AdSense units. Now this may well improve your revenue but the problem is it is almost impossible to figure out if revenue increased due to changes to your ads or just due to random fluctuations. Take a look at my daily income over the past month.

There’s a lot of volatility in those figures. Admittedly you can slice the data in different ways which can produce better results but even with no changes to my ad setup, the daily numbers are all pretty volatile. And if you aren’t making a lot of cash from your ads, your figures are likely even more volatile. So I figure if you want to optimise you ad setup, then you need to be looking at weekly or monthly numbers, so the feedback loop on changes is going to be pretty slow. To be frank the limit of my optimisation of ads is to increase the number of ad units to three (the maximum allowed and also probably the maximum number that wouldn’t be too annoying to a user of you site).

So what did cause the increase in my income? Simple really, an increasing number of visitors. My graph of monthly visitors and my graph of monthly AdSense revenue are almost identical. So unfortunately the answer to the question of how to increase AdSense revenue is another question, how to increase the number of visitors to your site.

And the answer to that question is actually pretty straightforward I reckon. The first part is fully within your control - Produce a lot of content of a reasonably quality. i.e Content that search engines will consider to be of value and hence index. The second part isn’t completely within your control, gain some inbound links from reputable sites. I actually think this will naturally follow from producing content that people appreciate.

In some ways I can’t help thinking this is the business model that Facebook has adopted, get a shedload of content (although in their case its helpfully created by their own users) and shove ads on it.

Fun with flags

The Big Bang Theory hasn’t been quite the same since the geeks started to pair off with girlfriends (since half the humour was about them failing spectacularly in their endeavours with the opposite sex), but Sheldon’s recent video podcast ‘Fun with flags’ was rather splendid.

Creepy ads

I’m used to visiting a website and their adverts then following me round the web afterwards. It’s happened with Dell and some property website. They even seem to make an effort to show me relevant stuff (laptops I was looking at or properties in the area I looked at). Then after a few days they disappear. I find it a bit creepy and it’s a bit of an eye opener to realise what the ad companies know about me (or about the cookie that’s sat on my machine).

But there’s an advert that’s been following me around for months now, for LeanKit Enterprise Kanban. I’m pretty sure this ad can’t be getting shown to everybody on the web, since this is a pretty niche product, so I can only assume it’s appearing because I visited their site ages ago. This has gone beyond creepy, this is just plain weird. Is showing me the same ad for months on end actually effective? Are the advertisers paying a premium for it?

Cross browser selectSingleNode for XML in Javascript

Internet Explorer has a non-standard method available in XML documents returned from AJAX calls called selectSingleNode. Pass in some XPath and it returns the first node that matches the XPath. Other browsers have ways of doing the same thing, but they are more long winded. So in short, I like the IE implementation and since I don’t fiddle with XML in JavaScript that often I often forget that selectSingleNode is not supported on all browsers (and if you’re wondering why I use IE as my primary browser, it’s because it’s still, just about, the most popular browser out there).

So here’s a cross browser version of selectSingleNode (not my own work, copy and pasted from somewhere I can’t remember on the web)

    function SelectSingleNode(xmlDoc, elementPath) {
      if (xmlDoc.evaluate) {
        var nodes = xmlDoc.evaluate(elementPath, xmlDoc, null, XPathResult.ANY_TYPE, null);
        var results = nodes.iterateNext();
        return results;
      }
      else
        return xmlDoc.selectSingleNode(elementPath); 
    }

Update – This no longer works in IE10, since selectSingleNode has been removed from the XML document returned from AJAX calls. This can be worked around by setting the response type of the XmlHttpRequest, like so

xhr.responseType =  'msxml-document';

More info

Update – There’s a more fully featured version of this now available

Business Optix blog now live

The blog of my employer Business Optix has gone live. We’ll be posting there as we roll out new features to our web and desktop products, along with tips and tricks to use the current software and any other company news. Keep up to date by signing up to the RSS feed.

Veloset GPS 600 bike computer review

My first foray into the world of bike computers was a Sigma 1609 device. I quite liked it, since it was simple to use with a clear display, big chunky buttons and a very low price. The only minor downside was a certain amount of faffing around required to get it configured initially since it needed to know how big my wheels were.

But one day on my ride to work, a driver decided it would be a good idea to pull out at a junction without looking to see if anybody was coming and knocked me to the ground in the process. As I dusted myself off at the side of the road I heard a crunching sound and realised me and my bike had successfully got to the road side but my Sigma hadn’t.

So it was time to get another bike computer, since they are quite addictive. Being a map geek, I thought it was time I bought one with GPS, so I could map my routes on my PC. The Veloset GPS 600 caught my eye, since it was probably the cheapest GPS enabled bike computer available.

And it when it arrived first impressions were good, coming in a nice attractive box. But firing it up was disappointing. For some reason it uses a butt ugly font to display text and since it’s only a small device it’s not only ugly but difficult to read as well.

And things got worse when I fired up the accompanying software, which is possibly the ugliest software I’ve seen in my life. For people used to fondling iPads and the like, this will probably be a grave disappointment. But it does what it needs to do, importing data from the computer and displaying it in various graphs (speed, altitude) and on a map. It does consistently show my average speed as being higher than my maximum speed, which my basic grasp of maths suggests is incorrect, but other than that it seems to function correctly.

Which can also be said for the bike computer itself. It’s not the prettiest device ever but it does what it needs to do, displaying current speed, average speed (this time accurately!), distance covered, altitude, journey time etc. I’m not particularly keen on the touchpad buttons, since it’s easy to press them by accident and it’s impossible to press them at all when wearing gloves. Also on occasion the GPS seems to take a while to switch itself on, so the starts of journeys are sometimes lost.

So in conclusion, it’s the cheapest GPS enabled bike computer available and it shows in the presentation. It’s not a device you’ll fall in love with, but it does do everything it needs to do.

Finding a UserPrincipal for an email address

I wanted to let users log in to our application using their Windows user name or their email address. In order to achieve this aim, I needed to get hold of a UserPrincipal from the user’s email address so I could then test the password they entered was correct. It took quite a lot of searching to find the relevant code, so I thought I’d post the pertinent part here.

      PrincipalContext context = new PrincipalContext(
        ContextType.Domain, Environment.UserDomainName);
      
      UserPrincipal user = new UserPrincipal(context);
      user.EmailAddress = "test@test.com";

      // create a principal searcher for running a search operation
      PrincipalSearcher pS = new PrincipalSearcher(user);

      // run the query
      PrincipalSearchResult<Principal> results = pS.FindAll();

      foreach (Principal result in results)
      {
        // do something useful...
      }

The scale of the universe

If like me you find it hard to grasp how the tiny the tiniest things are and how massively huge the biggest things are, then have a look at this

http://htwins.net/scale2/

I’m not sure our minds can ever fully comprehend these different distances and sizes, but maybe it’ll help…

Let’s outsource the bankers

The main argument for bankers earning vast amounts of money seems to be that in a global market, they’ll just bugger off somewhere else if we can’t match the money available elsewhere. But the odd thing is, for the rest of us, globalization has led to stagnating wages as jobs have been outsourced to countries with cheaper labour. Odd that the free market doesn’t work in the same way for the rich as it does for everyone else.

So if we assume that maybe this argument is a bit of a fib, and assuming we can find decent replacements in some far off land who are willing to work for a much smaller pay packet, can’t we just outsource all our bankers along with some of our well paid CEOs? That’s some offshoring I wouldn’t mind seeing.

Developer interview questions

A while back I had to interview some people for a developer role at work so came up with a few questions, combining a few from the web with some of my own. This is essentially a note to myself for next time I’m interviewing.

jQuery
What's a jQuery selector? How would you select an item by its ID? By its class?
Give some examples of JQuery UI effects and widgets and what they could be used for

.NET
What's an interface? Compare and contrast with an abstract class
How does memory management differ between .NET and a non-managed language? How can we make .NET behave more like a non-managed language?
What are generics? Why use List<> instead of ArrayList?
What's a virtual function? How does it relate to OOP?

Web
What is a RESTful web service? Why are they preferred to SOAP web services?
What are the common data formats returned by an AJAX web service call? Is one better than the other? What about if you wanted to call it from a fat client?
Name and describe several HTTP status codes
Name the various HTTP verbs and when they are used

General
Your application has a performance problem, how would you investigate the issue?
An error occurs in your web application but only in production and only happens occasionally, how do you go about tracking down the problem?
What are some of the issues around multi-threaded applications?
Discuss some ways of ensuring code quality remains high in a project

Do it yourself inbound link alerts

Embedded Analytics provide a nice service that will email you whenever somebody clicks on a new link to your site. I’ve been signed up for a while and it’s interesting to see who’s linked to my site. But I received an email last week informing me that my site had so many inbound links that I would have to start paying for the service. To be fair the amount they were going to charge me wasn’t a lot, but I couldn’t really justify spending money on something that is essentially just a way to waste a bit of time for me. And I also figured I could probably do the same thing myself through the Google Analytics API, since this is what Embedded Analytics uses.

I’m assuming that Embedded Analytics uses the source for visitors to your site to spot new links. There is a downside to this since it won’t spot links that have been added but have not been clicked on, but generally these won’t be that interesting, since they presumably are links on low traffic sites.

So to implement this requires a few steps. Pull out the data from Google Analytics and store this data somewhere (DB, XML file, whatever). Then next time we pull the data out of Google, check for new URLs in the returned data and send a notification of these new URLs. Embedded Analytics also goes a step further and validates that the links are valid and that the pages containing them are available from the web. I was only really interested in the first part of this solution so have written a piece of code to pull out the URLs using the Google Data API for .NET. The rest of the work is left as an exercise for the reader!

using System;
using Google.GData.Analytics;

namespace GoogleAnalytics
{
  class Program
  {
    static void Main(string[] args)
    {
      AccountQuery feedQuery = new AccountQuery();
      AnalyticsService service = new AnalyticsService("DoogalAnalytics");
      service.setUserCredentials("email", "password");

      DataQuery pageViewQuery = new DataQuery("https://www.google.com/analytics/feeds/data");
      pageViewQuery.Ids = "ga:202885";
      pageViewQuery.Metrics = "ga:visits";
      pageViewQuery.Dimensions = "ga:source,ga:referralPath";
      pageViewQuery.Sort = "ga:source,ga:referralPath";
      pageViewQuery.GAStartDate = DateTime.Now.AddMonths(-1).ToString("yyyy-MM-dd");
      pageViewQuery.GAEndDate = DateTime.Now.ToString("yyyy-MM-dd");

      DataFeed feed = service.Query(pageViewQuery);
      for (int i = 0; i < feed.Entries.Count; i++)
      {
        DataEntry pvEntry = (DataEntry)feed.Entries[i];
        string host = pvEntry.Dimensions[0].Value;
        string path = pvEntry.Dimensions[1].Value;
        Console.WriteLine("http://" + host + path);
      }

      Console.ReadLine();
    }
  }
}

Another take on peer to peer lending

It seems a little strange to me that at a time when banks are meant to be desperately trying to increase their balance sheets, they are offering such meagre interest rates. I dunnow but maybe offer some decent rates and people will shove their money in your bank? But it seems they have decided it’s better to offer crap rates and hope we are too stupid to realise we can get better returns elsewhere.

So for a few years I’ve been stashing cash in Zopa and getting a return that manages to beat inflation by lending money directly to people, with the caveat that there is a higher level of risk than having money in the bank.

But I’m obviously not the only person to choose this option and rates have been falling of late. And although I’m happy to finance personal loans, i did have a hankering to lend money to businesses, especially since this seems to be something else banks are failing to do properly*

Which is where Funding Circle comes in. This is peer to peer lending but the other party is a business. Interest rates are currently some what better than Zopa. Time will tell how bad the default rates are but I’m going to drip feed some money in there and see how it pans out.

*I always thought banking was pretty straightforward, take in money from deposits, then lend it out to other people and skim a bit of profit from the transaction. Simple and a bit boring. Maybe they should have stuck to this slightly dull job, rather than inventing, buying and selling insane derivatives that nobody understands and nobody can quantify the associated risk. Get back to doing what you’re meant to do and people might get off your back a bit…