Doogal Bell's bloggy thing

The scale of the universe

2012-02-14T21:31:00.001Z

If like me you find it hard to grasp how the tiny the tiniest things are and how massively huge the biggest things are, then have a look at this

http://htwins.net/scale2/

I’m not sure our minds can ever fully comprehend these different distances and sizes, but maybe it’ll help…

Let’s outsource the bankers

2012-01-30T22:20:00.001Z

The main argument for bankers earning vast amounts of money seems to be that in a global market, they’ll just bugger off somewhere else if we can’t match the money available elsewhere. But the odd thing is, for the rest of us, globalization has led to stagnating wages as jobs have been outsourced to countries with cheaper labour. Odd that the free market doesn’t work in the same way for the rich as it does for everyone else.

So if we assume that maybe this argument is a bit of a fib, and assuming we can find decent replacements in some far off land who are willing to work for a much smaller pay packet, can’t we just outsource all our bankers along with some of our well paid CEOs? That’s some offshoring I wouldn’t mind seeing.

Developer interview questions

2012-01-28T21:20:00.001Z

A while back I had to interview some people for a developer role at work so came up with a few questions, combining a few from the web with some of my own. This is essentially a note to myself for next time I’m interviewing.

jQuery
What's a jQuery selector? How would you select an item by its ID? By its class?
Give some examples of JQuery UI effects and widgets and what they could be used for

.NET
What's an interface? Compare and contrast with an abstract class
How does memory management differ between .NET and a non-managed language? How can we make .NET behave more like a non-managed language?
What are generics? Why use List<> instead of ArrayList?
What's a virtual function? How does it relate to OOP?

Web
What is a RESTful web service? Why are they preferred to SOAP web services?
What are the common data formats returned by an AJAX web service call? Is one better than the other? What about if you wanted to call it from a fat client?
Name and describe several HTTP status codes
Name the various HTTP verbs and when they are used

General
Your application has a performance problem, how would you investigate the issue?
An error occurs in your web application but only in production and only happens occasionally, how do you go about tracking down the problem?
What are some of the issues around multi-threaded applications?
Discuss some ways of ensuring code quality remains high in a project

Do it yourself inbound link alerts

2012-01-21T21:33:00.001Z

Embedded Analytics provide a nice service that will email you whenever somebody clicks on a new link to your site. I’ve been signed up for a while and it’s interesting to see who’s linked to my site. But I received an email last week informing me that my site had so many inbound links that I would have to start paying for the service. To be fair the amount they were going to charge me wasn’t a lot, but I couldn’t really justify spending money on something that is essentially just a way to waste a bit of time for me. And I also figured I could probably do the same thing myself through the Google Analytics API, since this is what Embedded Analytics uses.

I’m assuming that Embedded Analytics uses the source for visitors to your site to spot new links. There is a downside to this since it won’t spot links that have been added but have not been clicked on, but generally these won’t be that interesting, since they presumably are links on low traffic sites.

So to implement this requires a few steps. Pull out the data from Google Analytics and store this data somewhere (DB, XML file, whatever). Then next time we pull the data out of Google, check for new URLs in the returned data and send a notification of these new URLs. Embedded Analytics also goes a step further and validates that the links are valid and that the pages containing them are available from the web. I was only really interested in the first part of this solution so have written a piece of code to pull out the URLs using the Google Data API for .NET. The rest of the work is left as an exercise for the reader!

using System;
using Google.GData.Analytics;

namespace GoogleAnalytics
{
  class Program
  {
    static void Main(string[] args)
    {
      AccountQuery feedQuery = new AccountQuery();
      AnalyticsService service = new AnalyticsService("DoogalAnalytics");
      service.setUserCredentials("email", "password");

      DataQuery pageViewQuery = new DataQuery("https://www.google.com/analytics/feeds/data");
      pageViewQuery.Ids = "ga:202885";
      pageViewQuery.Metrics = "ga:visits";
      pageViewQuery.Dimensions = "ga:source,ga:referralPath";
      pageViewQuery.Sort = "ga:source,ga:referralPath";
      pageViewQuery.GAStartDate = DateTime.Now.AddMonths(-1).ToString("yyyy-MM-dd");
      pageViewQuery.GAEndDate = DateTime.Now.ToString("yyyy-MM-dd");

      DataFeed feed = service.Query(pageViewQuery);
      for (int i = 0; i < feed.Entries.Count; i++)
      {
        DataEntry pvEntry = (DataEntry)feed.Entries[i];
        string host = pvEntry.Dimensions[0].Value;
        string path = pvEntry.Dimensions[1].Value;
        Console.WriteLine("http://" + host + path);
      }

      Console.ReadLine();
    }
  }
}

Another take on peer to peer lending

2012-01-06T21:44:00.001Z

It seems a little strange to me that at a time when banks are meant to be desperately trying to increase their balance sheets, they are offering such meagre interest rates. I dunnow but maybe offer some decent rates and people will shove their money in your bank? But it seems they have decided it’s better to offer crap rates and hope we are too stupid to realise we can get better returns elsewhere.

So for a few years I’ve been stashing cash in Zopa and getting a return that manages to beat inflation by lending money directly to people, with the caveat that there is a higher level of risk than having money in the bank.

But I’m obviously not the only person to choose this option and rates have been falling of late. And although I’m happy to finance personal loans, i did have a hankering to lend money to businesses, especially since this seems to be something else banks are failing to do properly*

Which is where Funding Circle comes in. This is peer to peer lending but the other party is a business. Interest rates are currently some what better than Zopa. Time will tell how bad the default rates are but I’m going to drip feed some money in there and see how it pans out.

*I always thought banking was pretty straightforward, take in money from deposits, then lend it out to other people and skim a bit of profit from the transaction. Simple and a bit boring. Maybe they should have stuck to this slightly dull job, rather than inventing, buying and selling insane derivatives that nobody understands and nobody can quantify the associated risk. Get back to doing what you’re meant to do and people might get off your back a bit…

Why all the free ads for Facebook?

2011-12-30T11:16:00.001Z

A while back I picked up a copy of the Evening Standard and was surprised to see Facebook and Twitter logos at the top of every page, suggested readers follow ES on these two websites. And this is just an extreme example of what I’m seeing more and more. Ads on the telly and in newspapers no longer show the URL of the company’s website, but the URL of their Facebook page instead.

But the thing that confuses me is why big companies would choose Facebook as their main point of contact with their customers? Sure, social networks are the big thing at the moment and getting users to follow you or like your product might have some benefit, but there seem to be a number of downsides.

First, who owns all the data being collected about your customers? My guess is Facebook. Can you extract that data if Facebook decide they don’t want you anymore or you decide to move? I guess it’s probably possible via the Facebook API but it seems somewhat risky. And even if you do extract it, Facebook will no doubt keep hold of it as well.

Then there’s the question of ads. The company pages I’ve seen on Facebook seem to have the same ads as any other page. I’ve found no indication that companies get any of the income from those ads, so why drive traffic to Facebook so they can make money from your brand? And what if Facebook decide to show ads for one of your competitors?

Frankly it all seems a bit odd. Big companies have big IT departments and generally have their own websites, fully under their control. It is pretty simple to add some Facebook widgets to your own site and get integration that way which seems a more sane approach if you want to get hooked into Facebook.

For a one man band kind of company, I can see the sense in putting you web presence on Facebook, it’s a lot simpler and cheaper than building your own website, but for multi-nationals my prediction for 2012 is that this is something they do much less of.

I am Sheldon…

2011-12-26T19:46:00.001Z

…with a somewhat larger waste line, somewhat smaller IQ and hopefully with less OCD tendencies.

Goodbye Google Friend Connect

2011-12-02T20:49:00.001Z

I’ve been disenchanted with Google Friend Connect for a while. I only used it for commenting on my website but it has a number of weaknesses

A crappy user experience
No notifications of new comments, which is a pain if you only receive a handful of comments
Ignoring query strings in URLs, so comments don’t stick to the right page
Weird date formatting and no control over how they appear

For a long while I assumed Google would actually update the widgets but I’m not sure anything ever got changed after the initial release. It was released and then, nothing. Even a visit to the site shows a copyright notice from 2009, which suggests they haven’t done much with it for some time.

So I had been meaning to convert to some other system but just hadn’t got round to it. Then I noticed Google have decided to can it (which of course hasn’t been mentioned anywhere on the Friend Connect site itself), so I decided it was time to finally do something. Google’s suggested solution is to hook into Google+, but that seems like a pretty useless way to add commenting to a website. So my suggested solution is to sign up to Disqus, it takes about 10 minutes to plug it in to your website and looks pretty good straight out of the box.

Google Maps in a desktop app

2011-10-11T21:51:00.001+01:00

I’ve seen one or two examples of using Google Maps in a WinForms desktop app, but the ones I’ve seen seem to involve loading image tiles from the Google server directly. There’s nothing wrong with that approach but I thought it would be a lot simpler to simply host a local web page using the Google Maps API in a WebBrowser control in an application. Here is a very simple example of this idea.

If you want to extend this example, it’s possible to call scripts in the page via the WebBrowser.Document.InvokeScript method and the application can respond to events in the page via the WebBrowser.ObjectForScripting property.

As an aside, similar ideas can be applied to hosting other Javascript web components, such as an HTML editor like CKEditor.

Google Maps and Friend Connect weirdness in IE8

2011-07-14T22:08:00.001+01:00

I received a bug report about my website. The maps that appear on my UK postcodes pages weren’t working in IE8. I hadn’t noticed since they were working fine in IE9 but when I switched to compatibility view in IE9 I started to see the same problem. This helped because I was then able to debug my JavaScript (much as I love IETester, I’d love it even more if I had access to the developer tools for each version of IE). And debugging the script revealed google.maps was null.

At this point I assumed this was a problem with my Google Maps script, so tried loading Google Maps asynchronously then tried specifying an older version of the API. Neither helped. I tried inserting the script in my HTML head but with no luck.

Finally I had a look at the google object and saw the only thing defined in there was friendconnect. Now my suspicions moved away from Google Maps to Google Friend Connect. my Friend Connect stuff appears at the bottom of each page and the script reference for it was also down the bottom of the page. So I thought I’d tried moving the Friend Connect script reference to the html header. And voila, the maps started working again.

So the conclusion? I’m not really sure, although I suspect Friend Connect is removing the google.maps object, although it seems odd that it only happens on IE8.

Retrieving the most popular pages using Google Analytics API

2011-06-04T22:43:00.001+01:00

For a long time I’ve shown the most popular pages on the home page of my website. I did this by logging every page that was viewed to the MySql database on the back end. This kind of worked but had a few problems. First, it wasn’t very clever since it couldn’t tell the difference between a real visitor and a search engine bot. Second, since I’ve started to get quite a few visitors (no, really), it was writing a large amount of data to the database.

So I thought there must be a better solution. Figuring that all the information I needed was already being collected by Google Analytics, I thought I could grab this data and dump it into a much smaller with just the page URL and the number of visits (rather than adding a row for every visit). So I coded up a solution using the .NET wrapper around the Google Analytics API. And this is what it looks like (with the database access code removed for clarity). You’ll need to provide your own email address, password and Google Analytics account table ID to get this to work, for obvious reasons.

using System;
using Google.GData.Analytics;

namespace GoogleAnalytics
{
  class Program
  {
    static void Main(string[] args)
    {
      AccountQuery feedQuery = new AccountQuery();
      AnalyticsService service = new AnalyticsService("DoogalAnalytics");
      service.setUserCredentials("email address", "password");

      DataQuery pageViewQuery = new DataQuery("https://www.google.com/analytics/feeds/data");
      pageViewQuery.Ids = "Google Analytics account table ID";
      pageViewQuery.Metrics = "ga:visits";
      pageViewQuery.Dimensions = "ga:pagePath";
      pageViewQuery.Sort = "-ga:visits";
      pageViewQuery.GAStartDate = DateTime.Now.AddMonths(-1).ToString("yyyy-MM-dd");
      pageViewQuery.GAEndDate = DateTime.Now.ToString("yyyy-MM-dd");

      DataFeed feed = service.Query(pageViewQuery);
      for (int i = 0; i < 20; i++)
      {
        DataEntry pvEntry = (DataEntry)feed.Entries[i];
        string page = pvEntry.Dimensions[0].Value.Substring(1);
        string visits = pvEntry.Metrics[0].Value;

        Console.WriteLine(page + ": " + visits);
      }

      Console.ReadLine();
    }
  }
}

Poor man’s XSLT profiling for .NET

2011-05-30T21:08:00.001+01:00

If you’ve ever looked round for a profiler for XSL transformations then chances are you’ve found the Microsoft add-on for Visual Studio, which looks like it’s just the ticket, if you happen to have Visual Studio Team System. But if you don’t happen to own that version, then it might look like you have to upgrade your VS license or buy some other XSLT profiler.

But if you happen to own a .NET profiler (I highly recommend AQTime) then there may be another solution. Visual Studio comes with the XSLTC tool that can be used to generate an assembly from an XSL transformation. Once we’ve got an assembly, then we can build a small wrapper application that loads up the assembly, passes it to an instance of the XslCompiledTransform class and calls the transform. And once we’ve got that, we can use a standard .NET profiler to find bottlenecks.

And as I understand it, the XSLT profiler add-on for Visual Studio works in just this way so profiling using this technique should be just as effective as the Microsoft version.

Spotify not too good to be true anymore

2011-04-14T22:05:00.001+01:00

Apparently I’ve been using Spotfiy for over two years. Funny, it seems longer than that. It was the perfect music service for me, unlimited music of my choosing on my PC, which is where I listen to music most of the time, with the only minor downside being some adverts that play occasionally between tracks. But it looks like it won’t be quite so perfect anymore. Free users can only listen to a track a total of five times and total listening time will be limited to 10 hours a month.

As a frequent user of Spotify I can see why they are doing this. First, it’s obvious that advertising revenue is not what they were hoping for, most of the ads are still for Spotify itself. Second, using it has had a perhaps not unexpected effect on my music buying behaviour. First example, U2 put their last album up on Spotify before its official release. I had a listen and realised it was rubbish, so as a marketing exercise I doubt it was a huge success. Second example/s, quite a few new releases are put onto Spotify Premium upon release. On a couple of occasions I’ve then purchased the album before it’s become available on the free version (Elbow, Arcade Fire if you’re wondering). I’m guessing this isn’t what Spotify wanted me to do, they were presumably hoping I’d pony up the Premium version. And then when those albums did become available on the free version (generally only a few weeks after release) I was hit with a mild feeling of regret for spending money that I didn’t really need to and deciding to think twice before making another purchase. Again, probably not what the music industry sponsors of Spotify were hoping for.

So now I’ve got a choice, sign up for a tenner a month and continue on as I am at the moment or spend that tenner on a CD every month. I guess the music industry don’t care too much which way that ten quid gets to them, so it’s purely a personal dilemma. But the people who can’t or won’t spend a tenner (teenagers, students mostly I guess) will probably rediscover the skill of searching for pirated albums on Google. The music industry is still caught between a rock and a hard place.

Updating the Code-Point postcode datataset in MySql

2011-03-12T10:49:00.001Z

Some time ago I imported the Ordnance Survey Code-Point postcode dataset into MySql. It looks like there’s a new version of that dataset available which includes new postcodes so I wanted to update my database. I guess I could just empty the table and re-import the data, but since it takes some time import and the data is live on the web, this wasn’t the ideal solution. Fortunately, MySql has a useful IGNORE keyword which will ignore failed inserts so any old postcodes will be ignored (since the postcode is used as the primary key on the table) whilst new ones are inserted. Of course, this assumes that the latitude and longitude of old postcodes doesn’t change, which I’m hoping is a reasonable assumption. So my new code looks like this.

using System;
using System.IO;
using DotNetCoords;
using LumenWorks.Framework.IO.Csv;
using MySql.Data.MySqlClient;

namespace ImportCodepoint
{
  class Program
  {
    static void Main(string[] args)
    {
      string[] files = Directory.GetFiles(@"C:\Users\Doogal\Downloads\codepo_gb\Code-Point Open\Data");
      foreach (string file in files)
      {
        ReadFile(file);
      }

    }

    private static void ReadFile(string file)
    {
      using (StreamReader reader = new StreamReader(file))
      {
        CsvReader csvReader = new CsvReader(reader, false);
        using (MySqlConnection conn = new MySqlConnection(
          "server=server;uid=username;pwd=password;database=database;"))
        {
          conn.Open();
          foreach (string[] data in csvReader)
          {
            string postcode = data[0];
            // some postcodes have spaces, some don't
            if (postcode.IndexOf(' ') < 0)
              postcode = data[0].Substring(0, data[0].Length - 3) + " " + data[0].Substring(data[0].Length - 3);
            // some have two spaces...
            postcode = postcode.Replace("  ", " ");
            
            double easting = double.Parse(data[10]);
            double northing = double.Parse(data[11]);

            // there are some postcodes with no location
            if ((easting != 0) && (northing != 0))
            {
              // convert easting/northing to lat/long
              OSRef osRef = new OSRef(easting, northing);
              LatLng latLng = osRef.ToLatLng();
              latLng.ToWGS84();

              using (MySqlCommand command = conn.CreateCommand())
              {
                Console.WriteLine(postcode);
                command.CommandTimeout = 60;
                command.CommandText = string.Format(
                  "INSERT IGNORE INTO Postcodes (Postcode, Latitude, Longitude) " +
                  "VALUES ('{0}', {1}, {2})",
                  postcode, latLng.Latitude, latLng.Longitude);
                int count = command.ExecuteNonQuery();
                if (count > 0)
                  Console.WriteLine("Added");
              }
            }
          }
        }
      }
    }
  }
}

GZipping all content served up by ASP.NET

2011-02-22T08:13:00.001Z

Update – I now realise this post is kind of pointless, there is a module for compression of dynamic content, called unsurprisingly DynamicCompressionModule… But the approach described may be useful for someone somewhere…

I couldn’t find anything that will GZip all the content returned by ASP.NET. There’s a module for compression of static files but nothing for dynamic content. There may be a good reason for this, perhaps the overhead of GZipping content on the fly can kill your server, but since my current project has no static content I thought it would be useful to give it a go. The solution is pretty simple, register the following module in web.config and you’re good to go.

using System;
using System.IO.Compression;
using System.Web;

namespace MyNamespace
{
  public class GzipModule : IHttpModule
  {
    public void Dispose()
    {
      
    }

    public void Init(HttpApplication context)
    {
      context.BeginRequest += new EventHandler(context_BeginRequest);
    }

    void context_BeginRequest(object sender, EventArgs e)
    {
      HttpApplication app = (HttpApplication)sender;
      if ((app.Request.Headers["Accept-Encoding"] != null) &&
            (app.Request.Headers["Accept-Encoding"].Contains("gzip")))
      {
        app.Response.Filter = new GZipStream(app.Response.Filter, CompressionMode.Compress);
        app.Response.AppendHeader("Content-encoding", "gzip");
        app.Response.Cache.VaryByHeaders["Accept-encoding"] = true;
      }
    }
  }
}

Registration is as follows

    <modules>
            <add name="GzipModule" type="MyNamespace.GzipModule" />
    </modules>

Fixing 404 errors when using ASP.NET 4 routing

2011-02-05T21:34:00.001Z

It took me a while to figure this out. Routing is meant to be baked into ASP.NET 4 but when i tried to set it up, all I got was 404 errors. I did a lot of Googling but couldn’t find anything. It turned out all I was missing was this in web.config

<system.webServer>
    <modules runAllManagedModulesForAllRequests="true"></modules>

Northern Ireland postcode data

2011-02-03T20:43:00.001Z

The OS Code-Point Open dataset is great, except for a few omissions. It doesn’t include data for Northern Ireland, the Isle of Man or the Channel Islands. It turns out that the Northern Irish postcode data can be found here. Unfortunately that data is in ESRI and MapInfo formats, which I’m not sure how to read. Fortunately Jamie Thompson has converted it to CSV, which is a little easier to deal with.

From that CSV file, it’s quite simple to import the data into SQL Server or MySql using code slightly modified from my Code-Point examples (SQL Server here and MySql here). The only thing to note is that the CSV file uses Irish grid references rather than OS grid references.

Now to figure out where to get hold of the Isle of Man and Channel Islands data…

Don’t believe everything that Reflector tells you

2011-01-31T22:12:00.001Z

Every .NET developer loves Reflector, since it gives us a chance to see inside assemblies that we don’t have the source for. And I’ve even read bloggers showing off the code that has been reverse engineered by it as evidence of poor coding practices at some organisation or another (“look, these guys use gotos!”). But though Reflector is a brilliant tool, its reverse engineering skills are not perfect. See this fairly innocent looking switch statement from some code I’m working on

        switch(type)
        {
          case "gateway":
            SetValue(component, "@type", "decision", true);
            string xml = component.InnerXml;
            xml = xml.Replace("gateway", "decision");
            component.InnerXml = xml;
            break;

          case "deliverable":
          case "dataObject":
            SetValue(component, "@type", "document", true);
            break;

          case "annotation":
            SetValue(component, "@type", "note", true);
            break;
        }

And this is what Reflector shows from the compiled assembly

        if (CS$4$0001 != null)
        {
            if (!(CS$4$0001 == "gateway"))
            {
                if ((CS$4$0001 == "deliverable") || (CS$4$0001 == "dataObject"))
                {
                    goto Label_00B0;
                }
                if (CS$4$0001 == "annotation")
                {
                    goto Label_00C5;
                }
            }
            else
            {
                this.SetValue(component, "@type", "decision", true);
                string xml = component.InnerXml.Replace("gateway", "decision");
                component.InnerXml = xml;
            }
        }
        goto Label_00DA;
    Label_00B0:
        this.SetValue(component, "@type", "document", true);
        goto Label_00DA;
    Label_00C5:
        this.SetValue(component, "@type", "note", true);
    Label_00DA:;

Which I think proves my point…

Better debugging of .NET services

2011-01-13T21:59:00.001Z

For a long while I’ve been debugging a .NET service using the recommended approach. Whilst this works, it’s kind of painful. the steps are something like this

Build the service
Realise the service was already running. Stop it from the Services Control Panel applet.
Build the service again
Start the service from the Services Control Panel applet.
Attach to the process from Visual Studio
Realise the service has already executed the piece of code I wanted to debug.
Goto step 1.

There had to be a better way. And a bit of Googling brought up this approach. But I didn’t really understand how it worked. I guess I’d assumed the error shown in Visual Studio when you try to debug a service was actually coming from Visual Studio, but I now realise the error is coming from .NET. So by having a different piece of code run when in debug mode, the service is treated like any old application.

My solution is slightly different so I can also test the service starting and stopping. My service implementation has StartService and StopService public methods, which are called from OnStart and OnStop. And my Main method looks like this.

    static void Main()
    {
      #if (!DEBUG)
      ServiceBase[] ServicesToRun;
      ServicesToRun = new ServiceBase[] 
            { 
                new MyService() 
            };
      ServiceBase.Run(ServicesToRun);
      #else
      WorkflowService service = new MyService();
      service.StartService();
      service.DoStuff();
      service.StopService();
      #endif               
    }

Onshoring, the new offshoring

2010-12-10T22:28:00.001Z

Us software developers have had to live with offshoring for years. And I can understand why companies do it. If software isn’t your raison d'être then why not outsource it, and if you’re going to outsource it, why not send it offshore where you can get probably the work done cheaper?

But there are problems with offshoring. There are the time differences and cultural differences that may make it not go as smoothly as hoped. But I’m not one to think there’s any reason why software developed in another country should be any worse than software produced at home. I’ve dealt with many developers, at home and overseas and there are good and bad developers everywhere.

But now things have changed again. The UK government introduced a scheme, the intra company transfer visa, which is intended to bring in people to the UK to fill skills gaps that can’t be filled by the UK population. But big business have found some sweet loopholes in this scheme that means they can pay pretty much minimum wage to their developers whilst claiming back their expenses and not paying NI for them. I can understand why this is attractive to companies, it saves them money after all, and why should they care about the greater good? I doubt Henry Ford would survive very long in the modern world. But I can’t see it’s particularly good for the country as a whole, with lower tax revenue and higher unemployment being the obvious outcome. So what the hell was the government thinking of? I’m quite happy to compete with anybody on a level playing field, but when the playing field has been slanted like this, I’m pretty baffled about what’s going on.

Yesterday we had students smashing up the streets because they think it’s unfair that they are going to be in massive debt after they complete their degrees. In the past, graduates would be the people going into these relatively low paid jobs. How are they going to react when they get out of university to find there are no jobs available for them, due to this wonderful scheme?

Adding a simple website hit counter using the Google Analytics API

2010-11-22T21:29:00.001Z

Last time, I built a simple app to get the number of visitors to a site using the Google Analytics API. So the obvious next step is to get that data displayed on a web page. The simplest way to do this would be to add some server-side code to an ASP.NET page. But if you’ve run the code in the previous blog entry, you’ll notice it’s pretty slow, generally taking over a second to get the data back from Google. Adding a second to every page load isn’t such a good idea, so this is my slightly more complicated solution, which loads up the hit count via AJAX.

First we need a generic handler, looking something like this

<%@ WebHandler Language="C#" Class="visitorCounter" %>

using System;
using System.Web;
using Google.GData.Analytics;

public class visitorCounter : IHttpHandler 
{
  public void ProcessRequest (HttpContext context) 
  {
    AccountQuery feedQuery = new AccountQuery();
    AnalyticsService service = new AnalyticsService("DoogalAnalytics");
    service.setUserCredentials("email", "password");
    DataQuery pageViewQuery = new DataQuery("https://www.google.com/analytics/feeds/data");
    pageViewQuery.Ids = "ga:103173";
    pageViewQuery.Metrics = "ga:visits";
    pageViewQuery.GAStartDate = DateTime.Now.AddMonths(-1).ToString("yyyy-MM-dd");
    pageViewQuery.GAEndDate = DateTime.Now.ToString("yyyy-MM-dd");
    DataEntry pvEntry = service.Query(pageViewQuery).Entries[0] as DataEntry;

    context.Response.ContentType = "text/plain";
    context.Response.Write(pvEntry.Metrics[0].Value);
  }
 
  public bool IsReusable {
    get {
       return false;
    }
  }
}

Then we just need a bit of jQuery magic to show it on the page.

    <p>
      Visitors this month: <span id="visitorCount"></span>
      <script type="text/javascript">
        $(document).ready(function () {
          $.get('visitorCounter.ashx', function (data) {
            $('#visitorCount').html(data);
          });
        });

      </script>
    </p>


With a little work this could probably be wrapped up as an ASP.NET control, but I’ll leave that as an exercise for the reader.

Simple Google Analytics .NET API usage

2010-11-15T21:47:00.001Z

So if you want to create your own version of Embedded analytics, the first thing to do is figure out how to use the Google Analytics API. If you’re wanting to use the .NET API, then this little snippet might help you get started. It loops through all the registered sites and shows the number of visits to each site over the last month.

  class Program
  {
    static void Main(string[] args)
    {
      AccountQuery feedQuery = new AccountQuery();
      AnalyticsService service = new AnalyticsService("DoogalAnalytics");
      service.setUserCredentials("email", "password");
      foreach (AccountEntry entry in service.Query(feedQuery).Entries)
      {
        Console.WriteLine(entry.Title.Text);

        DataQuery pageViewQuery = new DataQuery("https://www.google.com/analytics/feeds/data");
        pageViewQuery.Ids = entry.ProfileId.Value;
        pageViewQuery.Metrics = "ga:visits";
        pageViewQuery.GAStartDate = DateTime.Now.AddMonths(-1).ToString("yyyy-MM-dd");
        pageViewQuery.GAEndDate = DateTime.Now.ToString("yyyy-MM-dd");
        foreach (DataEntry pvEntry in service.Query(pageViewQuery).Entries)
        {
          Console.WriteLine(pvEntry.Metrics[0].Value);
        }

      }
      Console.ReadLine();
    }
  }

Embedded Analytics

2010-11-14T21:21:00.001Z

I’ve been working on a website for somebody and they wanted to show some information about the number of visitors to the site on the site itself. Since we already had Google Analytics installed, the obvious solution was something that hooked into that. I knew there was a Google Analytics API which I could hook into, but being lazy I hoped somebody else had already implemented something for me.

Luckily for me, there is a company called Embedded Analytics that do provide this kind of service. There are various free tools available although it does cost money if you want to use it for multiple sites. But the free service was good enough for my current needs.

Setup was simple, although you do obviously need to let them have access to your Google Analytics data. If you don’t feel comfortable with that then this isn’t the service for you. If you are OK with that, then you can quickly knock together a chart showing the number of visitors or page views in the recent past. Another nice little tool is the ability to get email alerts when somebody has linked to your site.

And now I think I might want to add this to other sites, but it’s going to start costing money to use on more than one site. When my laziness is confronted by my tight fistedness, my tight fistedness always wins out. So now I have to go back to that API and figure out how to do all this and more myself.

Creating a valid file name in C#

2010-11-11T21:43:00.001Z

This was new to me so I thought I’d share. If you want to generate a file name based on some text, but that text may contain characters that aren’t allowed in file names, what to do? In the past, I’ve fumbled around, replacing invalid characters as bugs have presented themselves. But there is a much more robust way, once I noticed the Path.GetInvalidFileNameChars method. That led to this very simple method.

    private string ReplaceInvalidFileNameChars(string fileName)
    {
      foreach (char invalidChar in Path.GetInvalidFileNameChars())
      {
        fileName = fileName.Replace(invalidChar, '_');
      }

      return fileName;
    }

Styled maps–Showing urban areas

2010-11-07T20:45:00.001Z

Styled Google Maps but I haven’t found a use for them up until now. But I had a desire to view urban areas in the UK and realised I could do that with styled maps, so here it is.

Loading map...

Uploading files and folders to a specific Google Docs folder using the .NET API

2010-10-12T20:17:00.001+01:00

The .NET API for Google Docs is a strange beast. First, it has a DocumentsRequest class and a DocumentsService class. Sometimes you’ll need to use one class, sometimes the other and it’s not clear what the distinction is between the two. And I’d assumed the .NET API supported everything you could do with the raw HTTP API and hence I thought uploading a file or folder to a specific folder wasn’t possible. This led to further confusion when I tried to move a folder and it did move but left the original folder behind and I couldn’t figure out how to delete that folder. And as with most things from Google, the documentation is somewhat lacking.

So I eventually figured out I’d have to do a little work myself and came up with a couple of extension methods, one to create a folder and one to upload a file, but both accepting a parent folder. When I say I needed to do a little work, what I really mean is I used Reflector to see how the current methods worked and employed some copy and paste…

  public static class GoogleDocsExtensions
  {
    public static Document CreateFolder(this DocumentsRequest docsRequest, string folderName, Document parentFolder)
    {
      Document doc = new Document();
      doc.Type = Google.Documents.Document.DocumentType.Folder;
      doc.Title = folderName;
      if (parentFolder == null)
        return docsRequest.Insert<Document>(new Uri(DocumentsListQuery.documentsBaseUri), doc);
      else
      {
        return docsRequest.Insert<Document>(new Uri(parentFolder.AtomEntry.Content.AbsoluteUri), doc);
      }
    }

    public static void UploadFile(this DocumentsService docsService, string file, Document folder)
    {
      FileInfo info = new FileInfo(file);
      using (FileStream input = info.Open(FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
      {
        Uri uri = new Uri(folder.AtomEntry.Content.AbsoluteUri);

        string str = info.Extension.ToUpper().Substring(1);
        string contentType = (string)DocumentsService.DocumentTypes[str];
        docsService.Insert(uri, input, contentType, info.Name);
      }
    }
  }

Installing PHP 5.2 on Windows 7 IIS 7.5

2010-10-10T12:18:00.001+01:00

I had a bit of fun upgrading my Windows 7 machine to PHP 5.2 the other day. There’s an installer available that goes most of the way but didn’t quite work for me (there’s also an installer available on www.php.net, I’m not sure how they differ).

The first problem was that my installation of IIS didn’t have the FastCGI module installed. This was pretty easy to rectify, in the IIS administrator go to Modules and then click the ‘Configure Native Modules’ link and then click the ‘Register’ button. The FastCGI module will live somewhere like ‘C:\Windows\System32\inetsrv\iisfcgi.dll’. Then I manually added a handler for PHP files, although if you add the module with the correct ‘FastCgiModule’ name, a re-install should correctly add the handler.

After this was set up, I was able to load up PHP pages but they weren’t being rendered properly, because it looked like the PHP code wasn’t being evaluated and was just getting outputted as plain text. This confused me for a while and I thought the installer hadn’t done its job. But after firing up Fiddler, I saw this header in the response from the server – X-Powered-By: PHP/5.2.14, which suggested the file was getting handed to PHP which was doing something with it. So I guessed there must be a problem somewhere in the PHP config file, php.ini. After searching around in there, I discovered this setting - short_open_tag = Off. This means any PHP files that use <? as their opening tag instead of <?php will not get evaluated, and turning that option on fixed my problems. One other thing to note is that an IIS reset seems to be required for any changes to that config file, since when using FastCGI the PHP executable stays resident in memory rather than restarting for every request.

WinForms ProgressBar with text

2010-10-05T08:37:00.001+01:00

Somebody asked how to use my transparent label on top of a ProgressBar control, since it disappeared when the progress bar changed its state. I had a look at it, but the transparent label wasn’t getting any notification of changes, even when I hooked into the WndProc method. So I wondered if it would be possible to sub-class the ProgressBar control and write some text on top of it that way. And that seems to work OK, as shown below.

Update – I’ve fixed the flickering

Update 2 – It now picks up the text colour from the ForeColor property

Update 3 – Should now work on Windows XP

using System.ComponentModel;
using System.Drawing;
using System.Windows.Forms;
using System;

namespace WinFormControls
{
 
  public class TextProgressBar : ProgressBar
  {
    protected override CreateParams CreateParams
    {
      get
      {
        CreateParams result = base.CreateParams;
        if (Environment.OSVersion.Platform == PlatformID.Win32NT
            && Environment.OSVersion.Version.Major >= 6)
        {
          result.ExStyle |= 0x02000000; // WS_EX_COMPOSITED 
        }

        return result;
      }
    }

    protected override void WndProc(ref Message m)
    {
      base.WndProc(ref m);
      if (m.Msg == 0x000F)
      {
        using (Graphics graphics = CreateGraphics())
        using (SolidBrush brush = new SolidBrush(ForeColor))
        {
          SizeF textSize = graphics.MeasureString(Text, SystemFonts.DefaultFont);
          graphics.DrawString(Text, SystemFonts.DefaultFont, brush, (Width - textSize.Width) / 2, (Height - textSize.Height) / 2);
        }
      }
    }

    [EditorBrowsable(EditorBrowsableState.Always)]
    [Browsable(true)]
    public override string Text
    {
      get
      {
        return base.Text;
      }
      set
      {
        base.Text = value;
        Refresh();
      }
    }
  }
}

The problems with financial theories

2010-10-03T17:12:00.001+01:00

We’re all looking for people to blame for the current mess we are in and apparently there are 38 causes of the current financial crisis. I’m not sure where to get hold of the original report so I’m not sure of what’s on the list but it made me think again about what I learned during my brief time working in the world of high finance a few years ago. I was there for a couple of years, so I'm clearly not hugely knowledgeable but the assumptions made when concocting the financial models used by many investors in the city always seemed a little fishy to me. Here are some of the main assumptions made and the problems I think are associated with them. Almost every financial theory will use one or more of these assumptions.

The market is efficient – All information about an asset is known and priced into the cost of that asset. Peston says this is one of the 38 causes. There are several issues with this theory, my favourite being this paradox – if the market is efficient then there is very little point in trying to play the market, since the market will always be priced accurately so there will be no opportunity to beat it by buying low, selling high. But if there are no players in the market, then it won’t be efficient. Which suggests it can never be perfectly efficient.

Another problem with the efficient market theory can be seen with something like the dot com bubble, where prices bore no relation to their underlying value, how is that sensible? The fact is share prices often bear no relation to the true worth of the company, because people think their value will continue to grow so they will be able to sell them on to a greater fool.

Investors are rational - Given two different assets with the same risk, investors will choose the one with the highest return. Clearly this isn't the case, otherwise bubbles would never happen. Another problem is that even a rational investor may not know the true risk/reward relationship. Shares may fall in value, making their dividend yield look very attractive, but there may be a good reason for this, like the economy is hitting trouble. Finally, some investors are just taking a punt, with no rational thought behind their trading.

Finally, if we decide that not all investors are rational, then we have another reason why the market will not be efficient.

There are no transaction charges – This may be nearly true for large institutional investors but for the man on the street this is very much not true, meaning any kind of investment strategy that involves a lot of trading will incur huge transaction costs, which will eat away at any profits made.

The market is liquid - This assumption basically says if you want to buy or sell, then you can. Whilst this holds true for most shares most of the time, it falls over at the most vital point, when markets are crashing, because everybody is trying to sell and there is nobody willing to buy. It is also not true for quite a few asset classes, as anyone who's tried to sell a house will know.

Normal distribution – A lot of financial theories use the normal distribution for the distribution of volatility/risk. Unfortunately this doesn’t seem to be the case. The black swans seem to occur a lot more frequently than a normal distribution would suggest. For the banks out there, this doesn’t seem to be such a problem, since they take the profits during the good times and get bailed out when those once in a lifetime occurrences strike a little too often.

And it’s the combination of all these flawed assumptions that caused at least some of the problems. They were used to build mathematical models that seemingly abolished risk or certainly incorrectly measured it.

IE9 hardware acceleration not working

2010-09-29T20:50:00.001+01:00

I downloaded the IE9 beta a week or so ago. Though it looked pretty good, the performance was less than stellar. When I looked at the options dialog, I saw that ‘Use software rendering instead of GPU rendering’ was checked and also disabled, which made unchecking it quite difficult. Apparently it will be disabled if Internet Explorer decides your graphics card isn’t good enough. I’m not exactly sure what the minimum requirements are, although I have heard rumours that the card must support DirectX 11.

Anyway it seems my old Radeon X600 wasn’t up to the job so I purchased a Radeon HD5450. I have no idea if it’s a good graphics card, but it’s pretty cheap and it does support DirectX 11. After installing that, IE9 was still pretty slow. Until I remembered to uncheck that checkbox…

Structured storage viewer

2010-09-09T21:08:00.001+01:00

Before the advent of .NET, Windows development involved a lot of fiddling around with COM interfaces and trying to figure out HRESULT error codes. One prime example of this is the interfaces used to access structure storage. Structured storage is a strange beast itself, providing a way to stitch together composite files, sort of like a seriously complicated version of ZIP files. Of course this was a solution in search of a problem, it turns out ZIP files are pretty much adequate for most situations, which is probably why Office now uses them.

But these technologies have a habit of sticking around (I guess Windows will continue supporting structured storage for the rest of time or until Windows no longer exists, whichever comes sooner). So I was faced with writing some code that had to interact with structured storage. .NET wisely doesn’t have any support for it, so I was faced with dealing with the really horrible COM interfaces. And I was even thinking I’d have to write a little application to let me see inside these composite files, so I’d have a clue what was going on. But fortunately I discovered somebody had already done the job for me, and I finally get to the point of this post. If you need a structured storage viewer, download it from here. http://www.mitec.cz/ssv.html

Converting a Cursor to a Bitmap in .NET

2010-08-22T17:57:00.001+01:00

I’ve been playing around with a little tool I’ve been developing and needed to display a Cursor in a PictureBox. There doesn’t seem to be built-in way to convert a Cursor to a Bitmap, which kind of makes sense since it might be an animated cursor so it wouldn’t be sensible to convert it to a Bitmap. I couldn’t find any examples on the web, so thought I’d post this code that does the trick.

          System.Windows.Forms.Cursor cursor = new System.Windows.Forms.Cursor(stream);
          Bitmap bitmap = new Bitmap(cursor.Size.Width, cursor.Size.Height);
          using (Graphics gBmp = Graphics.FromImage(bitmap))
          {
            cursor.Draw(gBmp, new Rectangle(0, 0, cursor.Size.Width, cursor.Size.Height));
            pictureBox.Image = bitmap;
          }

Storing and restoring the clipboard contents

2010-08-12T20:13:00.001+01:00

My hacky method to get HTML into Word programmatically had a slight flaw in that it splatted anything the user had placed on the clipboard. I consider it to be bad manners to mess with the clipboard because it’s the user’s clipboard, not my application’s. Many years ago I was forced to use Lotus Notes, which used the clipboard whenever I replied to an email, so I can’t really write code that does something that has annoyed me in the past.

So I had to figure out how to fix it. The below did the trick, although I haven’t tested it with all clipboard formats so it may not work with all of them. And if you’re wondering why this is in VB.NET and the previous example was in C#, I was forced to convert all the code to VB.NET, much to my chagrin.

      ' store current contents of clipboard
      Dim currentData As IDataObject = Clipboard.GetDataObject()
      Dim formats As String() = currentData.GetFormats()
      Dim currentClipboard As Dictionary(Of String, Object) = New Dictionary(Of String, Object)
      For Each Format As String In formats
        currentClipboard.Add(Format, currentData.GetData(Format))
      Next

      ' do stuff...

      ' put back old clipboard contents
      Dim newData As DataObject = New DataObject()
      For Each savedFormat As KeyValuePair(Of String, Object) In currentClipboard
        newData.SetData(savedFormat.Key, savedFormat.Value)
      Next
      Clipboard.SetDataObject(newData)

FxCop 10

2010-08-11T20:39:00.001+01:00

When I fired up FxCop 1.36 today, the update check, for probably the first time ever, told me there was a new version available. So I went off to get the update. The odd thing was that I wasn’t pointed towards an installer but a text file. That pointed out that FxCop was now part of the Windows SDK, so installing it is now something of a bigger job than it used to be. In fact, installing the SDK doesn’t install FxCop, it just copies the installer to your machine which then has to be run separately.

But the fact that the version number had increased so much got me excited, thinking there must be lots of improvements… After loading it up and running it against a project, I was a little disappointed. I didn’t spot any new rules and the only difference I noticed was that my custom rules assembly no longer worked.

Updating my rules assembly was pretty simple. First remove the references to the 1.36 assemblies (FxcopSdk.dll and Microsoft.Cci.dll), then update the project to target the .NET Framework 4 (of course this means it has to be built in VS 2010), add references to the version 10 assemblies and rebuild. And it all worked first time. My rules are pretty simple so I don’t know if it will be as easy to upgrade more complex rules assemblies but it all looks pretty good for backwards compatibility.

That said, there doesn’t seem to be any compelling reason to upgrade, unless I’ve missed something.

Things are a bit quiet round here

2010-07-31T22:26:00.001+01:00

If you are one of the many subscribers to my blog (where many is a value is slightly larger than zero) you may have noticed things have been a bit quiet round here. There is a reason for this. My proper website has suddenly become relatively popular for some reason, so I’ve made the decision to spend my spare time updating that more regularly, since it is actually producing income for me, which in the current climate is definitely a good thing. I’ll still post here, but it may not be so frequent.

Quick hack to convert HTML to Word documents

2010-07-16T22:16:00.001+01:00

There are lots of products that will convert HTML files to Word format or RTF, but they mostly cost money. And there are so many of them, it’s hard to know which are good without testing them all. Word of course can open HTML files directly but one issue I found with this is if you then save the file as a DOC file any images are not embedded in the file (although they are in DOCX files).

But being the cheapskate I am, I thought there must be a way to implement this myself, so came up with this quick hack, which loads the HTML into IE, copies it to the clipboard, pastes it into Word and then saves it as a Word document. It does require that Word is installed on the user’s machine and it does splat any contents of the clipboard but other than that it seems reasonably robust. My only other thought is how horrible using the Word COM API is from .NET, bring on optional parameters!

    private bool loadComplete;
    public void ConvertToWord(string htmlFile, string filename)
    {
      // open in IE
      using (WebBrowser browser = new WebBrowser())
      {
        loadComplete = false;
        browser.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(
          browser_DocumentCompleted);
        browser.Navigate(htmlFile);
        while (!loadComplete)
        {
          Thread.Sleep(50);
          System.Windows.Forms.Application.DoEvents();
        }

        // copy to clipboard
        browser.Document.ExecCommand("SelectAll", true, null);
        browser.Document.ExecCommand("Copy", true, null);
        
        // open Word, paste and save...
        object dummy = Type.Missing;
        ApplicationClass wordApp = new ApplicationClass();
        try
        {
          Document newDoc = wordApp.Documents.Add(ref dummy, ref dummy, ref dummy, ref dummy);

          wordApp.Selection.Paste();

          object fileName = filename;
          newDoc.SaveAs(ref fileName, ref dummy, ref dummy, ref dummy, ref dummy, ref dummy, 
            ref dummy, ref dummy, ref dummy, ref dummy, ref dummy, ref dummy, ref dummy, 
            ref dummy, ref dummy, ref dummy);
          ((_Document)newDoc).Close(ref dummy, ref dummy, ref dummy);
        }
        finally
        {
          wordApp.Quit(ref dummy, ref dummy, ref dummy);
        }
      }
    }

    void browser_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
    {
      loadComplete = true;
    }

Visio API ‘Cell is guarded’ exception

2010-06-29T21:39:00.001+01:00

If you’re using the Visio COM API to generate or modify Visio diagrams, you may have come across the exception ‘Cell is guarded’ when trying to set a cell’s FormulaU property (and probably when setting the Formula property). For me it only happened on a machine with Visio 2007 installed, Visio 2003 was fine. The solution was pretty simple, use the FormulaForceU property instead (although potentially this may lead to unexpected behaviour apparently, worked fine for me though).

Importing the Code-Point dataset into MySql

2010-06-24T19:33:00.001+01:00

I recently realised that capturing the geocoding results from Google wasn’t such a good idea. It seems that postcodes that are only partially correct will still be geocoded, but will be returned with a default latitude and longitude which are kind of in the right area for that postcode area but obviously aren’t right, since the postcode doesn’t exist.

So I decided it was time to grasp the nettle and use the Code-Point dataset for my UK postcodes page, so the data should be much more accurate. The only issue with the Code-Point dataset is that it doesn’t contain postcodes in Northern Ireland, Guernsey, Jersey and the Isle of Man for some reason.

So the first thing to do was to download the .NET connector for MySql. Then I modified my SQL Server import code to use MySql. As mentioned in that post, the code has dependencies on this CSV parser and my .NET coordinates library. I made some modifications to add spaces to postcodes as necessary and also to not bother inserting postcodes with no geographical data (I’m not sure if this is missing data or if these are simply non-geographical postcodes).

The code is shown below. Although it works, this is a simple implementation and is not particularly fast. It will probably take a few hours running against a local MySql installation but running against a remote installation will probably take longer. I only plan to run it once, so I’m not particularly concerned with performance, but if performance is an issue, you’ll probably want to take a look at the MySqlBulkLoader class or peruse the MySql documentation on improving the performance of inserts.

using System;
using System.IO;
using DotNetCoords;
using LumenWorks.Framework.IO.Csv;
using MySql.Data.MySqlClient;

namespace ImportCodepoint
{
  class Program
  {
    static void Main(string[] args)
    {
      string[] files = Directory.GetFiles(@"C:\Users\Doogal\Downloads\codepo_gb\Code-Point Open\data\CSV");
      foreach (string file in files)
      {
        ReadFile(file);
      }

    }

    private static void ReadFile(string file)
    {
      using (StreamReader reader = new StreamReader(file))
      {
        CsvReader csvReader = new CsvReader(reader, false);
        using (MySqlConnection conn = new MySqlConnection(
          "server=127.0.0.1;uid=root;pwd=password;database=test;"))
        {
          conn.Open();
          foreach (string[] data in csvReader)
          {
            string postcode = data[0];
            // some postcodes have spaces, some don't
            if (postcode.IndexOf(' ') < 0)
              postcode = data[0].Substring(0, data[0].Length - 3) + " " + data[0].Substring(data[0].Length - 3);
            double easting = double.Parse(data[10]);
            double northing = double.Parse(data[11]);

            // there are some postcodes with no location
            if ((easting != 0) && (northing != 0))
            {
              // convert easting/northing to lat/long
              OSRef osRef = new OSRef(easting, northing);
              LatLng latLng = osRef.ToLatLng();
              latLng.ToWGS84();

              using (MySqlCommand command = conn.CreateCommand())
              {
                Console.WriteLine(postcode);
                command.CommandTimeout = 60;
                command.CommandText = string.Format("INSERT INTO Postcodes (Postcode, Latitude, Longitude) " +
                  "VALUES ('{0}', {1}, {2})",
                  postcode, latLng.Latitude, latLng.Longitude);
                command.ExecuteNonQuery();
              }
            }
          }
        }
      }
    }
  }
}

MySql Workbench

2010-06-22T21:22:00.001+01:00

I’ve used phpMyAdmin for a long time to administer MySql databases on the web. I’ve been pretty happy with it, although it can occasionally behave a bit strangely if it times out. But I was at the MySql website the other day to download MySql for my laptop when I spotted MySql Workbench, so I thought I’d give it a go. And it’s a very nice piece of free software. Call me an old timer but I still prefer a desktop application and it delivers. It looks good and offers all the features I need. I’ve been spoiled by SQL Server Management Studio (and not spoiled by the Oracle admin tools…) but MySql Workbench seems comparable in terms of functionality and ease of use. The pre-release 5.2 seems quite a lot better than the official 5.1 release, so go for that if you feel brave.

FxCop custom rules

2010-06-18T21:30:00.001+01:00

There are quite a few examples of custom rules for FxCop (this was the best I found) but I was unable to find an example Visual Studio project to download. So after I got my simple rules assembly up and running, I thought I’d upload it to the web, you can grab it from here. It just contains one rule, taken from the site mentioned before, which checks for the usage of the ArrayList class.

My only advice is if you are having trouble getting your rules loaded up into FxCop then debug your assembly through Visual Studio and make sure you have set up Visual Studio to break when CLR exceptions are thrown. If any exceptions are thrown when trying to load your assembly, FxCop will catch the exception and not display the rules.

Getting the height of a background image

2010-06-17T21:09:00.001+01:00

Getting hold of the physical height of an image would initially appear to be a pretty straightforward thing to do. This may well be the case with an image held in an <img> tag (at least if the height of the <img> tag hasn’t been set) since it will resize to fit the image so you just need to get the height of the <img> element. But with a background image it’s not quite so simple. I wanted to get the height of an image used as a background image so I could resize the header of some HTML output when users supplied their own custom background images.

Even so, this should be pretty easy I thought. Load up the image using the Image JavaScript object and get the height from there. There were a couple of issues with this. I decided to pick up the image URL from the background-image CSS, rather than hard coding it. But this was complicated by the fact that different browsers store the URL in different ways, some with quotes, some without.

The other crazy thing I encountered was Internet Explorer seemingly loading up an old version of the image and returning the dimensions of that even though a new different sized image had replaced it. Emptying my cache and fiddling with the options dialog made no difference, so I was forced to add a query string to stop this insane caching.

Another thing which seems to have tripped up other people when I was trawling the web is the need to set up the onload handler before setting the src property.

Here’s the code, hope it’s useful.

      // get the background image
      var backgroundImg = new Image();
      backgroundImg.onload = function() {
        headerHeight = backgroundImg.height;
        // do other stuff
      };
      var imgSrc = $('.header').css('background-image');
      // IE and FireFox have quotes, Safari and Chrome don't
      imgSrc = imgSrc.replace(/["']/g, '');
      imgSrc = imgSrc.substring(4);
      imgSrc = imgSrc.substring(0, imgSrc.length - 1);
      // IE seems to be too aggressive in caching
      backgroundImg.src = imgSrc + '?' + Math.random();

Importing the Code-Point dataset into SQL Server

2010-06-13T21:53:00.001+01:00

Many moons ago, I imported my own postcode dataset into SQL Server but ever since the Ordnance Survey have made their Code-Point dataset available for free, I’ve been meaning to import that, since it’s likely their data will be more correct than my own.

It comes in a series of CSV files and contains the postcode, easting, northing and various other bits of data. Unfortunately the documentation is somewhat lacking so it’s not entirely clear what all the other data is. But those three bits of data are all I was after.

So I wrote a little console application to import the data, that looks like this

using System;
using System.Data.SqlClient;
using System.IO;
using DotNetCoords;
using LumenWorks.Framework.IO.Csv;

namespace ImportCodepoint
{
  class Program
  {
    static void Main(string[] args)
    {
      string[] files = Directory.GetFiles(@"C:\Users\Doogal\Downloads\codepo_gb\Code-Point Open\data\CSV");
      foreach (string file in files)
      {
        Console.WriteLine("Importing file " + Path.GetFileName(file));
        ReadFile(file);
      }

    }

    private static void ReadFile(string file)
    {
      using (StreamReader reader = new StreamReader(file))
      {
        CsvReader csvReader = new CsvReader(reader, false);
        using (SqlConnection conn = new SqlConnection(
          "Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=Postcodes;Data Source=DOOGAL-LAPTOP\\SQLEXPRESS"))
        {
          conn.Open();
          foreach (string[] data in csvReader)
          {
            string postcode = data[0];
            double easting = double.Parse(data[10]);
            double northing = double.Parse(data[11]);

            // convert easting/northing to lat/long
            OSRef osRef = new OSRef(easting, northing);
            LatLng latLng = osRef.ToLatLng();
            latLng.ToWGS84();

            using (SqlCommand command = conn.CreateCommand())
            {
              command.CommandText = string.Format("INSERT INTO Postcodes (Postcode, Coordinates) " +
                "VALUES ('{0}', geography::STPointFromText('POINT({2} {1})', 4326))",
                data[0], latLng.Latitude, latLng.Longitude);
              command.ExecuteNonQuery();
            }
          }
        }
      }
    }
  }
}

A couple of points, I’ve used this CSV parser, rather than re-inventing the wheel. It would be pretty easy to do this parsing myself, since only a few fields are quoted and no fields contain any special characters, but I’ve been pretty happy with this CSV parser in the past so saw no point re-inventing the wheel.

I’ve also used my .Net coordinates library to convert GB OS references to latitude/longitude, but obviously you can store the postcodes using eastings and northings if you prefer.

The final thing to do is to check that the imported data looks sensible. The easiest way to do this is to query the table in SQL Server Management Studio and look at the spatial results. Unfortunately this is limited to 5000 results, so I used this little bit of SQL (based on this blog post) to select a subset of the data to make sure the general shape looked correct. Obviously this doesn’t check all the data but did build some confidence that what I’d done was correct.

SELECT 
   EXPR2.*
FROM 
   (
   SELECT 
      Postcode, Coordinates,
      RowNumber,
      (EXPR1.RowNumber % 400) AS ROW_MOD
   FROM
      (
      SELECT 
         Postcode, Coordinates, 
         ROW_NUMBER() OVER ( ORDER BY Postcode ) AS 'RowNumber'
      FROM 
         Postcodes 
      ) EXPR1
   ) EXPR2
WHERE 
   EXPR2.ROW_MOD = 0

Image file locking in the .NET Bitmap constructor

2010-06-11T17:13:00.001+01:00

Consider the following code

      Dim bitmap As New Drawing.Bitmap(fileName)
      Return bitmap

It certainly looks fairly innocuous but it had a problem, or rather it caused a problem further down the track. We wanted to replace some of the image files that we had previously loaded into the application. But trying to do that threw an exception telling us “The process cannot access the file '…’ because it is being used by another process.”. Which wasn’t entirely accurate, actually we couldn’t access the file because it was in use by this process. But nitpicking aside, the reason for this was the Bitmap constructor above which loads the file but doesn’t unlock it, only a call to Bitmap.Dispose() unlocks it again.

So my first attempt to fix this looked something like this

      Dim fs As New FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
      Dim bitmap As New Drawing.Bitmap(fs)
      fs.Close()
      Return bitmap

Since this opened the image via a FileStream and closed it afterwards, this should solve the problem. And it did on my Windows 7 machine, but it caused even bigger problems on a Windows XP machine. We started getting an out of memory exception for no apparent reason.

I eventually came to the conclusion that this was down to the Bitmap constructor that took a FileStream instance. When it’s passed a file name, it can make a pretty good guess at what kind of file it’s loading based on the extension, but when it is just given a bunch of bytes, it has to make a guess as to what type of image it’s looking at and I presume the heuristics in XP weren’t as good as they are in Windows 7. I’m guessing here, since all this happens in the guts of the GDI+ API.

So my final attempt at fixing this looks like this

      Dim fileBitmap As New Drawing.Bitmap(fileName)
      Dim bitmap As New Drawing.Bitmap(fileBitmap)
      fileBitmap.Dispose()
      Return bitmap

Which so far hasn’t caused any more issues…

Debugging KML loading problems in Google Maps

2010-06-05T11:39:00.001+01:00

I was attempting to load some KML into Google Maps via the KmlLayer class and not having much luck. Nothing was appearing. Unfortunately the KmlLayer class doesn’t provide an event to indicate an error has occurred. So what is available to debug these problems and what are some of the issues that can cause load errors?

Fiddler may help, although it didn’t give any indication to me about what was going on. But I have seen some HTTP responses with 400 bad request status codes in the past. Unfortunately they didn’t say why the request was bad.
Check that the KML is valid using Feed Validator. I’ve found this doesn’t work too well with large KML files in IE, so use some other browser instead.
When Google Maps does server side rendering of KML it appears to do some caching of the data, so if you change your KML, changes may not appear immediately.
There are limits to the size of KML file supported by Google Maps, have a look at this page to see if this is your problem. I think this is what caused my issues.
Try loading the KML in Google Earth to see if it’s an issue with the KML or Google Maps’ handling of it.

Moving from Google Maps 2 to Google Maps 3

2010-06-01T22:10:00.001+01:00

After playing with version 3 of the Google Maps API and trying to convert my ScratchPad application, I’m pretty impressed. It’s a much more consistent API than version 2 and provides quite a few new features. But one thing is for sure, converting an application from version 2 to version 3 will be a big chunk of work. Pretty much every line will need rewriting.

So I thought I’d provide a little crib sheet to help people who are trying to upgrade. I don’t claim that this list is in any way complete. I’ll update it as I find more things out, but even so I only intend it to be a pointer to the right class/method to use. In most cases the way methods or classes are used has also changed. If you have any other information to add, please add a comment.

Google Maps version 2	Google Maps version 3
GMap2 class	google.maps.Map class
GMap2.disableDoubleClickZoom()	MapOptions.disableDoubleClickZoom:true passed to Map constructor or Map.setOptions()
GMap2.getDragObject().setDraggableCursor()	MapOptions.draggableCursor passed to Map constructor or Map.setOptions()
GMap2.addOverlay()	Call setMap() on the layer to add
GMap2.clearOverlays()	???
GMap2.removeOverlay()	Call setMap(null) on the layer to remove
GLatLng class	google.maps.LatLng class
GMarker class	google.maps.Marker class
GMarker.bindInfoWindowHtml()	google.maps.InfoWindow class, but you need to manually open the info window using google.maps.event.addListener(marker, 'click', function () { infoWindow.open(map, marker); });
GEvent.bind(), GEvent.addListener	google.maps.event.addListener(), although it doesn’t seem possible to bind the event to particular object instance (where the keyword this can be used in the event handler)
GEvent.removeListener()	google.maps.event.removeListener()
GNavLabelControl class	???
GAdsManager class	???
GGeoXml class	google.maps.KmlLayer class
GGeoXml.load event	???
GGeoXml.getDefaultCenter()	KmlLayer.getDefaultViewport().getCenter()
GStreetviewOverlay class	This is now integrated into the map, pass streetViewControl: true into MapOptions to make it appear
GClientGeocoder class	google.maps.Geocoder class
GClientGeocoder.getLatLng()	Geocoder.geocode()
GPolyline class	google.maps.Polyline class
GPolyline.getLength()	There is no direct replacement, but this forum post discusses how to implement something similar yourself.
GPolyline.getVertexCount()	Polyline.getPath().length()
GDirections class	google.maps.DirectionsService class to query for directions, google.maps.DirectionsRenderer class to render them on a map
GDirections.load()	DirectionService.route()
GDirections.loadFromWaypoints()	DirectionsRequest.waypoints property
GDirections.clear()	No equivalent
GTrafficOverlay class	google.maps.TrafficLayer class
GLayer class	??? – This is in the API issues database, star it if you want it implemented
GTileLayerOverlay class	???

Improving the performance of REGEXP queries in MySql

2010-05-31T12:40:00.001+01:00

Say we’re trying to query a table of UK postcode and just want to return the postcodes in a particular area(the Birmingham area B for instance). A naive implementation of this may be something like this

SELECT * FROM Postcodes WHERE Postcode LIKE 'B%'

This works for some postcodes but doesn’t work for this particular example because it also returns any postcodes in the BA, BB, BT etc areas. So it looks like a regular expression is required, to ensure the area code is followed by a number, so we try the following

SELECT * FROM Postcodes WHERE Postcode REGEXP'^B[0-9]'

This works and returns what we expect but it is much slower than the LIKE query. So is there any way to speed it up? Actually, it’s pretty straightforward, just combine the LIKE and the REGEXP queries, like so

SELECT * FROM Postcodes WHERE Postcode LIKE 'B%' AND Postcode REGEXP'^B[0-9]'

This give MySql the chance to first filter the data based on the LIKE clause then only use the regular expression on the filtered data. It’s not quite as fast as the original LIKE implementation but it’s much quicker than REGEXP on its own and unlike the original LIKE implementation it actually works.

The Fibonacci Sequence

2010-05-30T18:58:00.001+01:00

For Christmas 1987, my brother gave me ‘yet another interesting book’ (his words), “The Penguin Dictionary of Curious and Interesting Numbers”. Clearly it must be pretty interesting, since I was surprised to see it is still available. Being something of a maths geek, I did enjoy reading it, but it has been languishing in my loft for a few years now.

Jump forward a couple of decades and I was reading “The Rabbit Problem” as a bedtime story to my daughter and was wondering why it was set in Fibonacci’s Field. Clearly this was related to the Fibonacci Sequence (and reading the back cover gave the game away) but I was unsure how the Fibonacci Sequence was related to rabbits. So time to climb up into the loft to find out. Turns out the Fibonacci Sequence was the solution to a problem about rabbits breeding. And it also appears a lot in nature, specifically in the number of petals etc in plants.

As if that wasn’t enough, the ratios of successive terms of the Fibonacci Sequence tend towards the Golden Ratio, another interesting number that you can find out about in the dictionary. And here’s a little Javascript example to show that convergence.

Free XML sitemap generator with unlimited pages

2010-05-30T11:19:00.001+01:00

I’ve been happily using this free XML sitemap generator for a while, but as my sites have got bigger I keep hitting the 500 page limit. Since I’m too tight to actually pay for this kind of service I’ve been searching around for a while for another free alternative. I was even thinking of writing my own, but before I was forced down that road, I found this alternative generator. I’m not too keen on Java applets generally, but this one seems to work very well. If it fails to download any pages, you can go back and retry downloading the failed pages. It also gives details of how long pages take to download, so it can also be a useful performance checker of a website.

Even so, I’m a little ambivalent towards sitemaps, especially if they are generated using one of these tools that just crawl your website to find all the pages. What information does this provide to search engines that they can’t find for themselves by crawling the site themselves? And If I do want to provide any more useful information, I have to edit the thing by hand, which I really can’t be bothered doing.

Google Maps elevation API

2010-05-28T21:21:00.001+01:00

Version 3 of the Google Maps API adds a new dimension to maps, the elevation of locations. I did plan to knock together an example of how this works, but the example provided by Google is pretty good itself.

I think this new API could be very useful. The obvious use would be when planning a walk or a bike ride and wanting to get an idea for the terrain, but I’m sure there are many more uses.

Google Maps styled maps

2010-05-28T20:17:00.001+01:00

Version 3 of the Google Maps API adds support for styled maps, which means it is now possible to hide features or display them differently on the base map. This could lead to some interesting customisations, and also leads to some questions about how exactly Google have implemented this? Are they creating map tiles on the fly? Or have they stored every possible permutation of styling? Surely the former, but how come it doesn’t seem to make any difference to the performance of rendering?

Anyway I had a play with the styled map wizard (which doesn’t work in IE for some reason) and thought I’d experiment with a map displaying just rail lines. And I kind of got this working, but there was a fundamental problem. Rail lines are only displayed when the map is zoomed in below a certain threshold so the map wasn’t any use to get a broad overview of rail lines in the UK for instance. And the styled maps API doesn’t seem to provide any control over this thresholding.

So it looks good as an initial implementation, but I think more control is required to be a completely useful API.

Google Maps geolocation API

2010-05-25T20:58:00.001+01:00

Google have had an API to get the user’s location for a while (via the Google loader), but I didn’t think it was very good. But now there are two new methods of figuring out the user’s location, one using the new W3C standard and the other using Google Gears. The good news is they seem to do a very good job of locating the user (scarily so in fact), the bad news is that neither are supported by Internet Explorer or Safari.

Google Chrome uses Google Gears to do the job, FireFox uses the W3C method and frankly I’m not too bothered what Opera supports, since so few people use it.

One annoyance is the prompting you get in the browser, although this is fairly understandable since this information could potentially be used for evil. And it won’t be an especially useful feature until it’s adopted by more browsers. But a combination of all three methods may produce a reasonable compromise.

Google Maps geocoding still sucks

2010-05-21T20:36:00.001+01:00

I spent a little time today starting to convert some of my pages to the Google Maps API version 3. At the same time I thought I’d convert my geocoding to use the latest version since I was sure it would have improved since it was added to the API and failed to work too well three years ago. But no, it still sucks big time for postcode geocoding, as this test page demonstrates. But what is strange is that the GlocalSearch class still does a fine job of geocoding. Same company, different APIs and different results. This can’t be down to the Royal Mail throwing their toys out of the pram since the Ordnance Survey now provides geocoded postcode data for free so I assume it’s down to Google stuffing up.

Converting to Google Maps version 3

2010-05-21T14:35:00.001+01:00

Google Maps API version 3 has just moved from the Google Labs to live so I thought I’d try converting a simple version 2 map to version 3. This is what the version 2 code looks like

  <script src="http://maps.google.com/maps?file=api&amp;v=2&amp;key=ABQIAAAAjtZCgAx5i04BiZDO6HlxhRQUdBDpWCOMRMbgTcqadX0jQ8HOERSxXxhk24TIBUpivovAKLrnpSio9w" type="text/javascript"></script>
  <script type="text/javascript">
    window.onload = function () {
      if (GBrowserIsCompatible()) {
        var map = new GMap2(document.getElementById("map"));
        map.setCenter(new GLatLng(51.49506473014368, -0.130462646484375), 10);
        map.addControl(new GSmallMapControl());
        map.addControl(new GMapTypeControl());
        map.enableDoubleClickZoom();
        map.enableScrollWheelZoom();
        var layer = new GGeoXml("http://www.doogal.co.uk/LondonStationsKML.php");
        map.addOverlay(layer);
      }
    }
  </script>

And this is what the version 3 code looks like

  <script src="http://maps.google.com/maps/api/js?sensor=false" type="text/javascript"></script>
  <script type="text/javascript">
    window.onload = function () {
      var latlng = new google.maps.LatLng(51.49506473014368, -0.130462646484375);
      var options = {
        zoom: 10,
        center: latlng,
        mapTypeId: google.maps.MapTypeId.ROADMAP
      };

      var map = new google.maps.Map(document.getElementById("map"), options);

      var georssLayer = new google.maps.KmlLayer('http://www.doogal.co.uk/LondonStationsKML.php');
      georssLayer.setMap(map);
    }
  </script>

The first thing I noticed is that the API key is no longer required which is very welcome, since it makes everything simpler (no more hassle when moving scripts between domains or trying to debug somewhere other than the domain or localhost). I guess the namespaces are a good thing and the class names seem more memorable. At this point I’m not sure if the API is any better than the old one, I’ll have to do some more digging before I can make a decision. But it does look like there’ll be quite a bit of work to convert old sites to the new API. Not that we’re forced to upgrade of course, at least for the moment.

JavaScript not executing in Internet Explorer 8

2010-04-23T10:48:00.001+01:00

I’ve been caught out by this on numerous occasions and still haven’t learnt my lesson. So in an attempt to remember in the future and perhaps help somebody else out, here’s the situation. Take the following script declaration

<script type="text/javascript" src="Scripts/jquery-1.4.1.min.js" />

This may look valid and probably should be, but Internet Explorer thinks the script tag hasn’t been closed, which means any script blocks after it will be ignored. The declaration needs to be

<script type="text/javascript" src="Scripts/jquery-1.4.1.min.js"></script>

Building a random postcode generator

2010-04-11T11:27:00.001+01:00

Some time a go I noticed that the keywords ‘random postcode generator’ were landing quite a few people at my website, even though I didn’t have a random postcode generator. Which got me thinking there must be a need for such a tool, if people were ending up at a site that didn’t have one desperately looking for it. So I built one and it’s now the most visited page on my site.

So that’s my little success story. But then I started wondering why anybody would want a random postcode anyway. I guess some people are just after a fake postcode they can plug into a website registration form, for whatever reason, and that page will do the trick nicely. But I imagine there are other people who need to generate random postcodes programmatically for some reason, so I’m going to explain how my postcode generator works. It may not be the best approach, but it seems to work.

I guess one approach would be to have a complete list of UK postcodes and just select one at random. I didn’t have a complete list of postcodes and you probably won’t either, unless you’ve bought the PAF database from the Royal Mail. So I went for a more long winded approach that requires less initial data to get working. First I got together a table of postcode districts (original list and my more programmer friendly version), which is the first half of a postcode. Potentially you could try to create this part of the postcode randomly but you’d end up creating invalid postcodes a lot of the time. Also, not all postcode districts are of the same format.

So with this table in place, I get a random postcode district and then append a random second half of the postcode (which is always of the form digit-letter-letter). I did this in PHP using the following (this contains some code which is specific to my website but you should be able to modify for your own needs)

$result = DbQuery("SELECT Postcode FROM PostcodeDistricts ORDER BY rand( ) LIMIT 1");
$line = mysql_fetch_array($result, MYSQL_ASSOC);
$num = rand(0,9);
$firstChar = rand(1, 26);
$secondChar = rand(1, 26);
print($line["Postcode"] . " " . $num . chr(64+$firstChar) . chr(64+$secondChar));
mysql_free_result($result);

The only problem with this is that it will sometimes generate invalid postcodes, so the next step is to check the postcode is valid. I did this using Google’s Local Search API from JavaScript, as so.

  <script type="text/javascript">
    /* <![CDATA[ */
    function CreateRandomPostcode()
    {
      document.getElementById("postcode").innerHTML = "Thinking";
      TryCreateRandomPostcode();
    }
 
    function TryCreateRandomPostcode()
    {
      document.getElementById("postcode").innerHTML += ".";
      var xmlhttp;
      if (window.XMLHttpRequest) {
        xmlhttp = new XMLHttpRequest();     // Firefox, Safari, ...
      }
      else if (window.ActiveXObject)   // ActiveX version
      {
        xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");  // Internet Explorer
      }
 
      xmlhttp.onreadystatechange = function() {
        if(xmlhttp.readyState == 4 && xmlhttp.status == 200) {
          Geocode(xmlhttp.responseText);
        }
      }
      xmlhttp.open("GET", "CreateRandomPostcode.php", true);
      xmlhttp.send();
    }
 
    function Geocode(postcode)
    {
      var localSearch = new GlocalSearch();
      localSearch.setSearchCompleteCallback(null,
        function()
        {
          if (localSearch.results[0])
          {
            var results = localSearch.results[0];
 
            if ((results.lat > 49) && (results.lat < 61) && (results.lng > -12) && (results.lng < 3)) {
              document.getElementById("postcode").innerHTML = "Your random postcode is " + postcode;
              document.getElementById("lat").value = results.lat;
              document.getElementById("long").value = results.lng;
            }
            else {
              TryCreateRandomPostcode();
            }
          }
          else
          {
            TryCreateRandomPostcode();
          }
        });
 
      localSearch.execute(postcode + ", UK");
    }
 
    /* ]]> */
  </script>

Basically it tries to geocode the postcode and if it fails (or it lies outside the UK), assumes the postcode is not valid and tries a different postcode until a valid one is produced. This could all be done server-side by hacking the Local Search API, but this met my simple needs.

Temporary file class for C#

2010-04-02T11:40:00.001+01:00

Quite frequently I need to create a temporary file, do some processing on it, then delete it. In order to ensure the file gets deleted, I put the file deletion code in a try…finally, which got me thinking about writing a simple class that implements IDisposable to handle this scenario, allowing me to use using. It’s very simple, but here it is anyway.

  public class TemporaryFile : IDisposable
  {
    public TemporaryFile(string fileName)
    {
      this.fileName = fileName;
    }

    ~TemporaryFile()
    {
      DeleteFile();
    }

    public void Dispose()
    {
      DeleteFile();
      GC.SuppressFinalize(this);
    }

    private string fileName;
    public string FileName
    {
      get { return fileName;  }
    }

    private void DeleteFile()
    {
      if (File.Exists(fileName))
        File.Delete(fileName);
    }
  }

jQuery zoom addin with image map support

2010-03-27T11:23:00.001Z

In the past I fiddled around with the moozoom plugin to add support for image maps and on a new project I needed the same thing. But this new project uses a lot of jQuery stuff and jQuery and mootools don’t play together particularly well. Although there are a lot of image zooming plugins available for jQuery, none of them did exactly what I needed. I just wanted image zooming, panning and support for image maps. Anyway, this is what I came up with. This is my first adventure in the world of jQuery plugins, so it may not be a perfect implementation but it works for me…

Usage is pretty simple

$('.selector').zoomable();

where .selector is the selector for your image. View an example here. You can now also programmatically zoom in and out using $('.selector').zoomable('zoomIn'); and $('.selector').zoomable('zoomOut'); which you can hook up to buttons for users without a mouse wheel

Here’s the code (requires jQuery and jQuery UI)

(function ($) {
  $.fn.zoomable = function (method) {
 
    return this.each(function (index, value) {
      // restore data, if there is any for this element
      var zoomData;
      if ($(this).data('zoomData') == null) {
        zoomData = {
          busy: false,
          x_fact: 1.2,
          currentZoom: 1,
          originalMap: null,
          currentX: 0,
          currentY: 0
        };
        $(this).data('zoomData', zoomData);
      }
      else
        zoomData = $(this).data('zoomData');
      
      var init = function() {
        if (value.useMap != "") {
          var tempOriginalMap = document.getElementById(value.useMap.substring(1));
          zoomData.originalMap = tempOriginalMap.cloneNode(true);
          // for IE6, we need to manually copy the areas' coords
          for (var i = 0; i < zoomData.originalMap.areas.length; i++)
            zoomData.originalMap.areas[i].coords = tempOriginalMap.areas[i].coords;
        }

        $(value).css('position', 'relative').css('left', '0').css('top', 0).css('margin', '0');

        $(value).draggable();

        // jquery mousewheel not working in FireFox for some reason
        if ($.browser.mozilla) {
          value.addEventListener('DOMMouseScroll', function (e) {
            e.preventDefault();
            zoomMouse(-e.detail);
          }, false);
          if (value.useMap != "") {
            $(value.useMap)[0].addEventListener('DOMMouseScroll', function (e) {
              e.preventDefault();
              zoomMouse(-e.detail);
            }, false);
          }
        }
        else {
          $(value).bind('mousewheel', function (e) {
            e.preventDefault();
            zoomMouse(e.wheelDelta);
          });
          if (value.useMap != "") {
            $(value.useMap).bind('mousewheel', function (e) {
              e.preventDefault();
              zoomMouse(e.wheelDelta);
            });
          }
        }

        $(value).bind('mousemove', function (e) {
          zoomData.currentX = e.pageX;
          zoomData.currentY = e.pageY;
        });
      };

      var left = function() {
        return parseInt($(value).css('left'));
      };
      
      var top = function() {
        return parseInt($(value).css('top'));
      }
      
      var zoomIn = function() {
        // zoom as if mouse is in centre of image
        var parent = $(value).parent()[0];
        zoom(zoomData.x_fact, left()+parent.offsetLeft+(value.width/2), top()+parent.offsetTop+(value.height/2));
      };
      
      var zoomOut = function() {
        // zoom as if mouse is in centre of image
        var yi = parseInt($(value).css('top'));
        var parent = $(value).parent()[0];
        zoom(1 / zoomData.x_fact, left()+parent.offsetLeft+(value.width/2), top()+parent.offsetTop+(value.height/2));
      };
      
      var zoomMouse = function (delta) {

        // zoom out ---------------
        if (delta < 0) {
          zoom(1 / zoomData.x_fact, zoomData.currentX, zoomData.currentY);
        }

        // zoom in -----------
        else if (delta > 0) {
          zoom(zoomData.x_fact, zoomData.currentX, zoomData.currentY);
        }
      };

      var zoomMap = function () {
        // resize image map
        var map = document.getElementById(value.useMap.substring(1));
        if (map != null) {
          for (var i = 0; i < map.areas.length; i++) {
            var area = map.areas[i];
            var originalArea = zoomData.originalMap.areas[i];
            var coords = originalArea.coords.split(',');
            for (var j = 0; j < coords.length; j++) {
              coords[j] = Math.round(coords[j] * zoomData.currentZoom);
            }
            var coordsString = "";
            for (var k = 0; k < coords.length; k++) {
              if (k > 0)
                coordsString += ",";
              coordsString += coords[k];
            }
            area.coords = coordsString;
          }
        }
      };

      var zoom = function (fact, mouseX, mouseY) {
        if (!zoomData.busy) {
          zoomData.busy = true;

          var xi = left();
          var yi = top();

          var new_h = (value.height * fact);
          var new_w = (value.width * fact);
          zoomData.currentZoom = zoomData.currentZoom * fact;

          // calculate new X and y based on mouse position
          var parent = $(value).parent()[0];
          mouseX = mouseX - parent.offsetLeft
          var newImageX = (mouseX - xi) * fact;
          xi = mouseX - newImageX;

          mouseY = mouseY - parent.offsetTop
          var newImageY = (mouseY - yi) * fact;
          yi = mouseY - newImageY;

          $(value).animate({
            left: xi,
            top: yi,
            height: new_h,
            width: new_w
          }, 100, function () {
            zoomData.busy = false;
          });

          zoomMap();
        }
      };
      
      if (method == "zoomIn")
        zoomIn();
      else if (method == "zoomOut")
        zoomOut();
      else
        init();
    });
  };
})(jQuery);

jqGrid hints and tips

2010-03-23T21:32:00.001Z

I’ve been spending a lot of time with jqGrid recently. It’s a marvellous piece of work but the documentation doesn’t always match up to the quality of the code. I’m not complaining, I realise it’s free, it’s flipping great and as a developer myself I know documentation is always the last thing I want to tackle. But here are a few things I’ve discovered along the way where it wasn’t immediately apparent what was causing the problem.

Multiple rows being selected – This can be caused by a couple of things. The first thing to check is that the row IDs being used are sensible. My problem was caused by having some rows with the same ID. Another time the problem cropped up when I used email addresses as my row IDs. It seems the grid has problems with certain characters being used in IDs. The @ symbol did it for me but there are others, like spaces, that can cause problems.

Arbitrary XML – The docs say that jqGrid can accept any arbitrary XML document, so long as you provide a mapping to describe how to map it to the standard XML format. This is almost true, but falls over if your XML document contains attributes that you want to pull into the grid. Attributes aren’t supported at all. I managed to get round this in a couple of steps. First I had to hack jqGrid so the loadComplete event was triggered before the grid was actually populated, rather than after. Then I could fiddle with my returned XML before jqGrid tried to read it. Then I wrote some code to convert attributes to elements with he same name (this could probably be improved with some jQuery magic but it did the job for me) and called it for the relevant attributes.

function setNodeText(newElem, text) {
  if (text != null) {
    try {
      newElem.textContent = text;
    }
    catch (e) {
      newElem.text = text;
    }
  }
}

function convertAttributeToElement(xml, node, attributeName) {
  var name = node.getAttribute(attributeName);
  addChildNode(xml, node, attributeName, name);
}

function addChildNode(xml, parent, nodeName, value) {
  var newElem = xml.createElement(nodeName);
  setNodeText(newElem, value);
  parent.appendChild(newElem);
}

Word wrap – I wish this was on by default, since it is now the first thing I do when I’m setting up a grid for the first time. Just modify the ui.jqgrid.css file so the definition for .ui-jqgrid tr.jqgrow td includes the following white-space: normal;

Forcing a .NET application to run as 32-bit under 64-bit Windows

2010-03-14T21:39:00.001Z

I was having problems with running a .NET application under 64-bit Windows 7. This was because by default, .NET applications are built to target any platform. This app clearly hadn’t been tested on 64-bit Windows so blew up when it tried to instantiate a 32-bit COM object. The following link explains some of the options on how to fix this problem.

http://www.lostechies.com/blogs/gabrielschenker/archive/2009/10/21/force-net-application-to-run-in-32bit-process-on-64bit-os.aspx

Obviously setting the platform target wasn’t an option for me since I didn’t have access to the source code. Using corflags.exe also wasn’t possible because the application was strong named so setting that flag would break the strong naming. So I was forced to create a wrapper application that was compiled for 32-bits. This initially also didn’t work but the only thing I needed to do was add the STAThread attribute to my wrapper’s Main method since the application I was calling also had this attribute on its Main method. Then it all worked, hurrah!

Now I just need to figure out how to do the same thing for a .NET service… And for all you .NET developers out there, if you don’t have the time or resources to test on 64-bit Windows, just flip that platform target switch to x86. Even the cheapest new PCs come with 64-bit versions of Windows (my laptop cost £400) and flipping that switch pretty much guarantees your app will work on 64-bit platforms.

XML encoding in PHP

2010-03-12T10:06:00.001Z

I’ve needed it in JavaScript in the past and now I need it in PHP. Escape all those pesky special XML characters like so

  function XmlEncode($content)
  {
    $trans = array("&" => "&amp;", "<" => "&lt;", ">" => "&gt;", "'" => "&apos;",
      "\"" => "&quot;", "’" => "&apos;");
  	return strtr($content, $trans);
  }

Collatz conjecture implementation

2010-03-05T21:27:00.006Z

I only learnt about the Collatz conjecture today so thought I'd implement it in JavaScript. Type in a starting number and watch as the process converges to 1, probably

Hollands Pies reviewed

2010-02-15T21:17:00.001Z

So I’m slowly working my way through the pies I purchased the other day and he’s what I think of them so far.

Meat pie – this is a little pie, about the size of a standard pork pie but heated up. It contains pork and beef and tastes pretty yummy. My only complaint is the scalding hot liquid contained within which burnt my mouth first time, although this was easily rectified second time around by slicing the pie in half.

Cheese and onion – many years ago when I was a vegetarian, this was one of the few pies I was able to partake of. That and the butter pie, which seems to have disappeared from the face of the earth, probably due to its unfortunate name. It did contain more than just butter, which is why I didn’t collapse from heart failure. But back to the cheese and onion pie. Unfortunately this was disappointing, there was very little evidence of onion, just some bright yellow gloop that may well have originated at the Sellafield reprocessing plant (I joke of course, before anyone tries to sue me).

Potato and meat – before some government organisation got involved, this was known as a meat and potato pie. Now it’s just a perfectly decent pie with a silly name.

Steak and kidney pudding – this is where things get interesting. This is not like any other pie you’ll ever experience. With most pies the pastry is essentially just some hard packaging to keep the contents together, with little merit of its own. But with the steak and kidney pudding, the suet case is an essential part of the whole experience. Not only are the contents tasty, but so is the pastry. Even the cooking instructions are unique. No oven for this pie, but a pan of boiling water. There is one downside to this. Whereas pies are generally a pretty portable food stuff, the steak and kidney pudding needs a plate and fork.

So in conclusion, I would recommend all of these pies from Hollands, except the cheese and onion. But for ultimate pleasure, order ten steak and kidney puddings…

FreeFlow now open source

2010-02-13T16:01:00.001Z

I don’t really have time to maintain it anymore, so I’ve open sourced FreeFlow, my little toolkit and administration tool for Metastorm BPM. Go grab it from CodePlex and make it better!

Using PUT and DELETE in WCF web services

2010-02-10T10:17:00.001Z

I’ve been doing a little work on a web page that needs to grab some data from a web service and then update that data via PUT and DELETE requests. I haven’t got access to the web service currently so decided to write my own dummy implementation using WCF.

The first problem I encountered was that by default PUT and DELETE calls won’t be allowed at all by IIS. This can be resolved by fiddling with the Handler Mappings options in IIS7 to allow the required verbs, as shown on the left.

My next problem was that I wanted to use the same URL for both PUT and DELETE requests. The WebInvoke attribute that is applied to your WCF method doesn’t support multiple HTTP verbs, so I was a little stumped at this point. The solution is actually pretty straightforward, have multiple methods pointing to the same URL, like so

[WebInvoke(Method = "DELETE", UriTemplate = "EditFolder")]
public void DeleteFolder(Stream input)

This is actually probably a good restriction by WCF. If you want different verb types on the same URL, the chances are that you have two different operations anyway.

Unminifying Javascript files

2010-02-03T21:37:00.001Z

Am I a web developer? I dunnow, I’ve spent a lot of time building web sites but I don’t really feel like a web developer. The web just plain scares me. Debug any web application and you’re faced with shedloads of anonymous functions, eval code and general weirdness. Look at the poster boys of Web 2.0, JQuery and its children JQuery UI and JQGrid. Yes they are fecking marvellous bits of technology but try stepping through the code and tell me you have the faintest idea what on earth is going on. It feels like the web is held together with a pile of JavaScript sellotape.

But in some kind of sado-masochistic twist, the web is now even more convoluted because the JavaScript is even less readable because it’s all been minified in an attempt to save bandwidth. So now if I find myself in a debugger trying to figure out what some JavaScript is doing, I see one long line of code with useless function names and I go off to bang my head against a wall because it’s less painful. But I have found one tool that improves the situation, JS Beautifier, which will attempt to make your minified code more readable and understandable by a human. Beyond the initial rant, this is primarily a reminder to myself.

David Cameron defaced

2010-01-31T22:03:00.001Z

When the smug face of David Cameron started popping up on billboards all over the place, I had a very strong urge to deface it. It would appear I wasn’t the only one, although these are generally more amusing than what I had in mind.

http://www.mydavidcameron.com/

Metastorm release BPM 9 and don’t tell anyone

2010-01-30T21:35:00.001Z

Here’s an odd thing. When most software companies release a new version of their software, they shout about it until they are hoarse. But Metastorm have released version 9 of their BPM software and as far as I can see they haven’t even produced a press release to announce it to the world. Searching on Google News brings back no results and I can’t see any mention of it on the Metastorm website.

Initially I’d assumed this was because version 9 was released in the quiet period before Christmas and the PR onslaught would start in the new year, but here we are well into 2010 and there is still silence.

Now generally if it’s a choice between cock-up and conspiracy, I’ll plump for cock-up every time but I really can’t believe that any software company can forget to produce a press release to announce their new baby to the world so I have to go for the conspiracy option but what is the conspiracy? Answers on a postcode or in the comments…

Buying steak and kidney puddings on the web

2010-01-28T20:57:00.001Z

I was reminded of steak and kidney puddings some time ago when I read Stuart Maconie’s “Pies and Prejudice” and suddenly had a strong desire to once again experience the long forgotten taste of them. But my searches on the web to find somewhere to purchase them in this culinary wasteland called London led me nowhere. Then the other day I received an email from Holland's Pies and ventured back to their website. I then discovered they have recently opened their online shop with a selection of their pies available. So £18 and a couple of days later I am now in possession of ten pies (including several steak and kidney pudds)and am looking forward to sampling one for my lunch tomorrow…

Cost benefit analysis of energy efficient bulbs

2010-01-25T21:25:00.001Z

We’ve had six GU10 bulbs in our kitchen for a while now and I’ve never really been very happy with them. They swallow a lot of power, don’t seem to last very long and are pretty expensive to replace. I got hold of an LED replacement some time ago and was less than impressed. It didn’t produce enough light, was really really expensive and then died after a couple of months. So when I popped into Maplin at the weekend, I thought I’d try out their low-energy GU10 replacement. Admittedly these are even more expensive than the full powered version, but they claim to last for 8000 hours and use a lot less power. I’m happy with the amount of light they throw out, although they suffer from the usual problem of taking a while to warm up. Anyway, I thought I’d do a quick calculation of how long it would take me to earn back the money I shelled out for them.

Cost per KWH – 0.15

Old 6 bulbs @ 50W = 300W

New 6 bulbs @ 11W = 66W

KWHs saved = 0.3-0.066 = 0.234 KWHs

Cost of bulbs = £7.49*6 = £45

Number of KWHs required to cover cost = 45/0.15 = 300KWHs

Number of hours required to cover cost = 300/0.234 = 1282 hours

Assuming 5 hours use a day = 1282/5 days = 256 days = 8.5 months

So based on this very rough calculation and assuming I’ve calculated it correctly, this would appear to be a good deal. Fingers crossed they actually last for as long as claimed, which I’m reasonably confident about based on my experiences with other energy efficient bulbs.

Getting all the points in a SqlGeometry

2010-01-04T21:28:00.001Z

It’s quite simple to get hold of the points in a SQL Server geometry polygon using the STNumPoints and STPointN SQL functions, but this requires quite a few queries (or some kind of stored procedure). I tend not to like running lots of queries, even if that does smell like premature optimisation, and for my little project I don’t want to be adding stored procedures but then I realised the SqlGeometry type is implemented as a .NET type, so all its properties and methods are available from .NET code, so here’s a little extension method to get all the points for a polygon

public static class GeometryHelper
{
  public static SqlGeometry[] Points(this SqlGeometry geometry)
  {
    List<SqlGeometry> points = new List<SqlGeometry>();
    for (int i = 1; i <= geometry.STNumPoints(); i++)
    {
      points.Add(geometry.STPointN(i));
    }
    return points.ToArray();
  }
}

Converting a SqlDouble to a .NET double

2010-01-03T22:00:00.001Z

Note to self - this is simple, just typecast like so

(double)points[i].STX

Improving performance with Gzip compression in PHP

2010-01-03T13:14:00.001Z

I was perusing through some of the data provided by Google Webmaster Tools when I came upon the Site Performance section. this told me that doogal.co.uk took longer to load than 96% of sites on the web, which was a little embarrassing. I knew what was causing at least some of this slowdown, the incomplete list of UK postcodes which has been getting gradually slower as the number of postcodes has got longer and longer. I’d already started to address that by paging the data, but the CSV data couldn’t really be paged, without reducing the value of it. So I had a look at the suggestion provided by Google, which was to use Gzip compression on the page. I’ve not really thought much about GZip compression before, assuming if it was so useful it would be on by default but I thought I’d give it a go anyway. So I fired up Fiddler and tried downloading my big CSV page and it took approximately 9.5 seconds to fully download. I arrived at this figure as an average after several reloads.

Next I added this line to my PHP source file, ob_start("ob_gzhandler");, and measured the difference. It now took 3.7 seconds to download, wow! A lot of gain for very little effort.

At this point, I wondered if I could add Gzip compression to all my pages. It turns out this is straightforward, just add the following to the .htaccess file

php_flag zlib.output_compression on

After adding this, the CSV page now arrived in 2.7 seconds. I’m not sure why this is even faster, but I’m not complaining. Now the rest of the site feels much snappier as well. So what am I missing? Is there a reason GZip compression isn’t on by default? Am I going to get bitten by this at some point in the future?

Getting boundary data out of OS OpenSpace

2009-12-22T21:30:00.001Z

I’ve been able to pull out a single county’s boundary data from OS OpenSpace using its JavaScript API, but I wanted to get the data for all counties. Now I could have used my little example webpage to do this manually for each county, but that would be extremely tedious. So based on the fact that all AJAX APIs can be considered as simple HTTP requests, I wrote a little piece of C# code that made the required HTTP request and dumped the data into my SQL Server table. This is very rough and ready but it met my needs. Some time soon I will probably put together a class to handle all the OS Open Space interaction a bit more nicely and with more functionality.

      HttpWebRequest req = (HttpWebRequest)WebRequest.Create(
        "http://openspace.ordnancesurvey.co.uk/osmapapi/boundary?geometry=true&"+
        "key=74B560DED619715AE0405F0AF060615A&f=xml&" +
        "url=http%3A%2F%2Flocalhost%2FOpenSpaceBoundary%2FDefault.aspx&area_code=CTY&" +
        "bbox=0%2C0%2C1400000%2C1400000&resolution=1000&dojo.preventCache=1261317566652&" +
        "callback=dojo.io.script.jsonp_dojoIoScript4._jsonpCallback");
      using (WebResponse resp = req.GetResponse())
      using (Stream respStream = resp.GetResponseStream())
      using (StreamReader reader = new StreamReader(respStream))
      {
        string response = reader.ReadToEnd();
        XmlDocument xmlDoc = new XmlDocument();
        xmlDoc.LoadXml(response);

        using (SqlConnection conn = new SqlConnection("Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=OGC;Data Source=NEWDOOGAL"))
        {
          conn.Open();
          using (SqlCommand command = conn.CreateCommand())
          {
            command.CommandText = "DELETE FROM CountyBoundaries";
            command.ExecuteNonQuery();
          }

          JavaScriptSerializer serializer = new JavaScriptSerializer();
          XmlNodeList nodes = xmlDoc.SelectNodes("BoundaryResultVO/items/item/geojson");
          foreach (XmlElement element in nodes)
          {
            Dictionary<string, object> data = 
              (Dictionary<string, object>)serializer.DeserializeObject(element.InnerText);
            Dictionary<string, object> properties = (Dictionary<string, object>)data["properties"];
            string countyName = (string)properties["NAME"];

            // get polygon data
            Dictionary<string, object> geometry = (Dictionary<string, object>)data["geometry"];
            object[] coordinates = (object[])geometry["coordinates"];
            
            for (int i = 0; i < coordinates.Length; i++)
            {
              StringBuilder wktBuilder = new StringBuilder();
              object[] polygonParts = (object[])coordinates[i];
              wktBuilder.Append("POLYGON(");
              for (int j = 0; j < polygonParts.Length; j++)
              {
                if (j>0)
                  wktBuilder.Append(",");
                wktBuilder.Append("(");
                object[] points = (object[])polygonParts[j];
                for (int k = 0; k < points.Length; k++)
                {
                  object[] point = (object[])points[k];
                  if (k > 0)
                    wktBuilder.Append(",");
                  wktBuilder.Append(point[0]);
                  wktBuilder.Append(" ");
                  wktBuilder.Append(point[1]);
                }
                wktBuilder.Append(")");
              }
              wktBuilder.Append(")");

              // put into database
              using (SqlCommand command = conn.CreateCommand())
              {
                command.CommandText = string.Format(
                  "INSERT INTO CountyBoundaries (County, Boundary) VALUES ('{0}', " +
                  "geometry::STPolyFromText('{1}', 4277))", countyName, wktBuilder.ToString());
                command.ExecuteNonQuery();
              }
            }
          }
        }
      }

Getting all available geographic reference systems out of SQL Server 2008

2009-12-10T06:22:00.001Z

As a reminder to myself, the query required to get a list of all available geographic reference systems out of SQL Server 2008 is

SELECT * FROM sys.spatial_reference_systems

uDig – free GIS

2009-12-09T21:52:00.001Z

To be frank I don’t much like anything Java based but uDig is the exception that probably proves the rule. It’s a free GIS based on Eclipse and it is pretty user friendly. Unfortunately it doesn’t handle data stored in SQL Server 2008, which marks it down a bit in my eyes, but it does support most other data sources.

Using OGC functions in SQL Server 2008 part 5 – Google Maps and polygons

2009-11-28T22:00:00.001Z

In this series of posts I’ve so far managed to pull some spatial data into SQL Server, query it in a number of ways and finally been able to display some of that data in Google Maps. The next thing I wanted to do was to display the county boundary for Lancashire, since it was the postcodes for that great county I was displaying. I’ve already got the data for the county boundary from OS OpenSpace but the problem is this data is in UK OS coordinates rather than latitude and longitude as used by Google Maps. There are two ways to deal with this. Convert the data to latitude/longitude before displaying it or implement the GProjection interface to handle the conversion in the web page. The latter is something I want to look at at some point but for this I decided to do the conversion server-side.

So the first thing I did was to write a simple generic handler to generate an XML document containing the points for the county boundary. This code uses the .NET Coordinates library I have converted from the Java original. This is what the handler looks like

  public void ProcessRequest (HttpContext context) 
  {
    context.Response.ContentType = "text/xml";
    StringWriter stringWriter = new StringWriter();
    XmlTextWriter writer = new XmlTextWriter(stringWriter);
    writer.WriteStartElement("points");
    using (SqlConnection conn = new SqlConnection(ConfigurationManager.ConnectionStrings["postcodesDatabase"].ConnectionString))
    {
      conn.Open();
      using (SqlCommand comm = conn.CreateCommand())
      {
        comm.CommandText = "SELECT Boundary.STNumPoints() FROM CountyBoundaries WHERE County='Lancashire'";
        int numPoints = (int)comm.ExecuteScalar();
        for (int i = 1; i <= numPoints; i++)
        {
          writer.WriteStartElement("point");
          // get X
          comm.CommandText = "SELECT Boundary.STPointN(" + i + ").STX FROM CountyBoundaries WHERE County='Lancashire'";
          double x = (double)comm.ExecuteScalar();

          // get Y
          comm.CommandText = "SELECT Boundary.STPointN(" + i + ").STY FROM CountyBoundaries WHERE County='Lancashire'";
          double y = (double)comm.ExecuteScalar();

          // convert to lat/long
          OSRef osRef = new OSRef(x, y);
          LatLng latLong = osRef.ToLatLng();

          writer.WriteAttributeString("x", latLong.Longitude.ToString());
          writer.WriteAttributeString("y", latLong.Latitude.ToString());
          
          writer.WriteEndElement();
        } 
      }
    }
    writer.WriteEndElement(); // points
    context.Response.Write(stringWriter.ToString());
  }

Next up was the JavaScript required in the web page to display the data. This is pretty simple

        // load boundary
        var boundReq = new XMLHttpRequest();
        boundReq.open("GET", "GetBoundary.ashx", true);
        boundReq.onreadystatechange = function() {
          if (boundReq.readyState == 4) {
            if (boundReq.status == 200) {
              var nodes = boundReq.responseXML.selectNodes("points/point");
              var latLongs = new Array();
              for (var i = 0; i < nodes.length; i++) {
                var lng = nodes[i].getAttribute("x");
                var lat = nodes[i].getAttribute("y");
                var latLong = new GLatLng(lat, lng);
                latLongs.push(latLong);
              }
              var polygon = new GPolygon(latLongs, "#FF0000", 3);
              map.addOverlay(polygon);
            }
          }
        };
        boundReq.send();

If you look at the image above you may think the boundary displayed is not correct since it doesn’t include some places that are quite obviously in Lancashire (Blackburn and Darwen for instance) and this confused me for a while. But the boundary displayed is that of Lancashire County council, rather than the full county, which doesn’t include some areas which are in unitary authorities but still within the county.

Using OGC functions in SQL Server 2008 part 4 – Google Maps

2009-11-25T21:29:00.001Z

So I’d now got to the stage where I’d pulled in some postcode data into SQL Server, got the county boundaries for one county (the mighty Lancashire) and figured out how to do a spatial query to tell me which postcodes were in that county. Now I wanted to display those postcodes on a map. So time to turn to Google Maps and Visual Studio.

First I created a simple generic handler in C# that returned an XML document containing the geometries of the postcodes. This could return JSON if that floats your boat but here is what my version looks like.

    public void ProcessRequest (HttpContext context) 
    {
      context.Response.ContentType = "text/xml";
      StringWriter stringWriter = new StringWriter();
      XmlTextWriter writer = new XmlTextWriter(stringWriter);
      writer.WriteStartElement("points");
      using (SqlConnection conn = new SqlConnection(ConfigurationManager.ConnectionStrings["postcodesDatabase"].ConnectionString))
      {
        conn.Open();
        using (SqlCommand comm = conn.CreateCommand())
        {
          comm.CommandText = "SELECT Location.STX AS X, Location.STY AS Y, City, Region, Postcode FROM Postcodes WHERE " +
            "(SELECT Boundary FROM CountyBoundaries).STContains(OsLocation)=1";
          using (SqlDataReader reader = comm.ExecuteReader())
          {
            while (reader.Read())
            {
              writer.WriteStartElement("point");
              writer.WriteAttributeString("description", reader["Postcode"].ToString() + ", " + reader["City"].ToString() + ", " + reader["Region"].ToString());
              writer.WriteAttributeString("x", reader["X"].ToString());
              writer.WriteAttributeString("y", reader["Y"].ToString());
              writer.WriteEndElement();
            }
          }
        }
      }
      writer.WriteEndElement(); // points
      context.Response.Write(stringWriter.ToString());
    }

Next I created a HTML page with a bit of JavaScript to load up the postcode data, which looked like this.

      function loadMap() {
      var map;
      if (GBrowserIsCompatible()) {
        map = new GMap2(document.getElementById("map"));
        map.setCenter(new GLatLng(53.756830663572174, -2.73834228515625), 9);
        map.addControl(new GSmallMapControl());
        map.addControl(new GMapTypeControl());
        map.enableDoubleClickZoom();
        map.enableScrollWheelZoom();

        // load data
        var req = new XMLHttpRequest();
        req.open("GET", "GetPoints.ashx", true);
        req.onreadystatechange = function() {
          if (req.readyState == 4) {
            if (req.status == 200) {
              var nodes = req.responseXML.selectNodes("points/point");
              for (var i = 0; i < nodes.length; i++) {
                var lng = nodes[i].getAttribute("x");
                var lat = nodes[i].getAttribute("y");
                var marker = new GMarker(new GLatLng(lat, lng), { title: nodes[i].getAttribute("description") });
                map.addOverlay(marker);
              }
            }
          }
        };
        req.send();
      }
    }

Again nothing groundbreaking but the next stage could me more challenging. That is pulling out the data for the county boundary and plotting that on the map. And what is this all for? All part of my plans for world domination.

Using OGC functions in SQL Server 2008 part 3

2009-11-15T21:22:00.001Z

It’s been a while since I last tried to do anything with the spatial features in SQL Server 2008. My next plan was to query the database to see which postcodes were in a particular county. To do this I got hold of the boundary data for some counties using the OpenSpace API. I just added one county, the one and only Lancashire, with the following code.

INSERT INTO CountyBoundaries (County, Boundary) VALUES ('Lancashire', 
geometry::STPolyFromText('POLYGON((335390.6 402509.3,335318.8 405919.1,337285.8 405717.9,341638.1 403012.7,341198.7 401158.3,

345355.3 399036.6,348231.1 404146.9,349111.4 402150.3,351662.6 402905.5,353409.1 407181.8,352474 409073.9,354484.8 412190.4,

358109.6 412599.6,358670 410970.3,360670.3 412859.2,362419.8 411151.7,366280.6 414615.7,364921.9 418843.3,366064.7 421813,

363060.8 427981.2,363748.5 429031.4,369403 431708.6,372966.5 423266.5,375369.8 421263.6,375568.6 419018,379162.2 417655.1,

380342.3 418953.7,382117.1 413139.9,383850.1 418586.1,385712 419187.4,387839.3 415759.4,389429.8 416106,390432.6 420649.6,

388662.6 425190.8,391447.8 428359.4,392703.4 434383.5,396065.9 436596.5,397063 439322.3,394185.1 441332.3,392652.8 446613.3,

388130.5 448821.7,387988.3 450699.5,387239.2 452627.7,385286.7 451690.4,384991.7 453964.6,380792.9 453190.5,379173.9 455357.1,

376997.3 455460,377844.2 457098.1,375605.7 461521.8,372209.8 460317.7,369352.3 461239,369714.2 464470.9,363191.2 470277.6,

363491 473169.9,365227.3 473672.8,370052.2 481319.2,370144.8 482748.7,362512.5 477911.7,358350.8 478665.3,355703.4 474137.7,

347972.3 478292.1,345446.8 475698.7,342749 476087.5,342784.7 474615.5,340414.8 475469.9,342882.8 469480.8,341304.5 465488.7,

344238.6 465331.3,340055 463217.2,340345.5 460141.4,335601.4 455045.4,336057.5 453597.1,332947.7 450366,329952.5 450443.2,

330737.1 442705.2,333041.8 441857.2,333572.8 437130.7,335319.6 433501.9,333963.4 433247,334425.2 431366.8,329729.3 431451.9,

329199.8 427517.3,328505.3 422791.4,330070.5 422095.3,336522 423531.2,337967.3 418422.4,332016.4 412241.1,332374.5 408594.8,

330736.4 405654.3,335390.6 402509.3))', 4277))

This is where things got a little tricky. The county boundaries used the OS coordinate system, whereas my postcodes were stored in latitude/longitude format. I was hopeful that SQL Server would do some magic for me and handle the different coordinate systems, but it would appear this isn’t the case.

So it looked like the only option was to convert my postcode points into the OS coordinate system. I’ve used phpCoord to do this conversion in the past, but this time I was looking for a .NET solution. So I took the Java implementation and converted it C#. That took a while (and I’ll upload the code at some point when I’ve OKed it with the original author) but after modifying my code to insert postcodes into the database I was able to run the following query and successfully pull out postcodes that were in the Lancashire area.

SELECT * FROM Postcodes WHERE (SELECT Boundary FROM CountyBoundaries).STContains(OsLocation)=1

Legal action against free postcode lookup

2009-10-17T21:39:00.001+01:00

I only discovered the Ernest Maples website today. Ernest Maples was the man who introduced the UK postcode system. The website was providing a useful service that took a UK postcode and returned the latitude and longitude of that postcode, which seems kind of familiar… But they’ve recently received a threatening legal letter from the Royal Mail’s lawyers telling them to remove this service from their site.

This raises so many questions. When will I get my threatening letter? When will Google get their threatening letter, since they provide this exact same feature but through a JavaScript API rather than via a URL? Why is the publically owned Royal Mail charging for this data anyway, since it was taxpayer money that paid for its creation in the first place?

Google Maps tools

2009-10-12T21:26:00.001+01:00

This is an interesting tool to see what is possible with the Google Maps API. Something I feel missing is real-time update of the shapes being drawn as the user moves the mouse around, something our mapping tool will provide when we ship the next version very soon.

Micro Men

2009-10-11T22:25:00.001+01:00

BBC3 and BBC4 often have some interesting programmes that are easily missed. I remembered to record Micro Men, a dramatisation of the work of Clive Sinclair and Chris Curry who produced the ZX Spectrum and BBC Micro respectively. It was good fun, although Sinclair didn’t come across as a particularly likeable character, which I found disappointing since he was always something of a hero for me as a child. Acorn lives on with its ARM processor powering most of the world’s mobile phones and Sinclair continues to invent things, although with somewhat less success than the ZX Spectrum. Anyway, Micro Men is available to watch on BBC iPlayer for a while.

Zopa – 18 months on

2009-10-11T20:50:00.001+01:00

About 18 months ago, I started to siphon some of my savings into Zopa in an attempt to get a slightly better return on my money. So I thought now would be a good time to look back on how well it has gone. First up, it certainly provides better returns than having your money in a bank account, with interest rates as low as they are. On the downside, this increased return does mean increased risk, the old risk premium again. Those loans can sometimes turn bad and when they do, all the outstanding debt is written off. Well I guess some of it may be returned, but I’ve not had any bad debt returned yet.

But the main downside is that the money is tied up for a long time, and by my calculations the return from part paying off my mortgage is still better than the returns on Zopa. And although doing that means that the money is gone forever, in the future I’ll probably be more likely to do that than put more money into Zopa.

Checking a latitude/longitude is in the UK

2009-10-01T21:29:00.001+01:00

I found quite a lot of duff data appearing in my incomplete database of UK postcodes, so thought I’d better filter out some of the rubbish. So the obvious thing to do was figure out which latitude and longitudes weren’t in the UK and stop them being put into in the database in the first place and also delete the ones already in there. There are definitely better ways of achieving this but my simple solution seems to have done the trick. First check the latitude is greater than 49 and less than 61, then check the longitude is greater than –12 and less than 3. It’s not perfect but I think it’s probably good enough for many applications.

OS OpenSpace API

2009-09-29T21:21:00.001+01:00

I’m pretty new to the world of GIS but one thing seems blindingly obvious to me, maps are becoming commoditised. With Google Maps and Virtual Earth already on the block, who is ever going to pay for maps anymore? New to the fray is the OS OpenSpace API, from the Ordnance Survey. It looks pretty decent, it includes some things not available in the other map providers, such as administrative boundaries, but misses other things like aerial imagery. When I get chance I will play with the API to see how good it is and get some ideas for our own JavaScript API.

Using OGC functions in SQL Server 2008 part 2

2009-09-27T18:00:00.001+01:00

In part 1 I created a table that contained some spatial UK postcode data. Now the next task is to work out how to query this data. A typical question we may want to answer is ‘what are the nearest postcodes to a particular postcode?’. OK, what we probably want to answer is ‘what is the nearest station/cash machine/post office etc to a particular postcode?’ but the only spatial data I’ve got is for postcodes, so we’ll use that. And the query to answer this question is pretty straightforward.

SELECT TOP 100 P.Postcode, P.Location.STDistance(PL.Location) AS Distance 
FROM Postcodes P, Postcodes PL
WHERE PL.Postcode='KT1 3EG'
ORDER BY Distance

The MSDN documentation for the STDistance function is a bit sparse but from what I can find from other sources, it will return the distance between two locations in metres.

Obviously this has all been possible in the past by writing your own code to do the calculation, but executing it directly against the database makes life a lot simpler. I have no idea if it will be quicker but simpler generally wins out for me.

XML encoding in JavaScript

2009-09-26T21:23:00.001+01:00

As far as I know there is no cross browser way to generate valid XML documents, so it seems like string concatenation is the way to go. So here’s a little function to do the required encoding of characters that need to be specially encoded for XML. As ever, I’ve not done any kind of thorough testing on this…

    function XmlEncode(text) {
      text = text.replace(/&/g, '&amp;');
      text = text.replace(/\"/g, '&quot;');
      text = text.replace(/\'/g, '&apos;');
      text = text.replace(/</g, '&lt;');
      text = text.replace(/>/g, '&gt;');
      return text;
    }

Using OGC functions in SQL Server 2008 part 1

2009-09-25T21:41:00.001+01:00

One great thing about having a new job is being faced with new technologies. So I’ve just come across the new geography and geometry data types in SQL Server 2008. These allow you to store geographical data in a database. In some ways this is nothing new, you could always store latitude and longitude or eastings and northings if you only cared about the UK area. But what is new is that the database actually understands what these values are, meaning it’s possible to write queries that are based on geographical location, which to my mind is absolutely frigging awesome.

So I thought I’d play around with these new features in SQL Server. Oracle also now has spatial features, but frankly I try and avoid using Oracle wherever possible. First up I created a new table with a geography column, as shown above. Next I had to populate it. Fortunately I knew where I could find some data. With a little help from C# and Visual Studio, it was pretty easy to get this into the database.

  class Program
  {
    static void Main(string[] args)
    {
      string[] lines = File.ReadAllLines(@"C:\Users\Doogal\Desktop\Postcodes.txt");
      using (SqlConnection conn = new SqlConnection("Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=OGC;Data Source=NEWDOOGAL"))
      {
        conn.Open();
        foreach (string line in lines)
        {
          string[] data = line.Split(',');
          using (SqlCommand command = conn.CreateCommand())
          {
            command.CommandText = string.Format("INSERT INTO Postcodes (Postcode, Location, City, Region) " +
              "VALUES ('{0}', geography::STPointFromText('POINT({2} {1})', 4326), '{3}', '{4}')",
              SqlEscape(data[0]), data[1], data[2], SqlEscape(data[3]), SqlEscape(data[4]));
            command.ExecuteNonQuery();
          }
        }
      }
    }

    private static string SqlEscape(string sql)
    {
      if (sql == null)
        return null;
      return sql.Replace("'", "''");
    }
  }

So after getting the data in the database, the next step was to query it and see what we got. And after running a query against a spatial table in Management Studio, the first thing you notice is the new ‘Spatial results’ tab. Clicking on that showed the diagram shown on the left. And that looked remarkably like a map of the UK.

I guess that in itself is not hugely impressive, but it certainly pleased me. Next I have to move onto real queries, such as ‘give me all the postcodes within 1km of this location’, which is where spatial databases potentially come into there own.

When will IE6 die?

2009-09-07T20:44:00.001+01:00

Just like everybody else who has to do any kind of web development, I’m quite keen to see the back of IE6 so I thought I’d make an estimate of when IE6 would die. I took the Google Analytics data from the Random Pub Finder and pulled out the monthly percentage of IE users who were using IE6, which produced the graph to the left. It looked pretty linear, except for the initial drop off after IE7 was released, so I added a linear trend line. And voila, IE6 will die at the beginning of 2011. Well, one can live in hope…

RBS. Make it happen

2009-09-02T21:41:00.001+01:00

I’m constantly amazed at the number of adverts I see around for RBS. I guess all the sponsorship deals they signed up for before they blew up have to be honoured so they still appear all over the place. I’ve passed through Glasgow airport a couple of times recently and it is plastered in ads for RBS saying ‘RBS. Make it happen’. I’m not sure if this a command to readers of the ad or they are just telling the reader that RBS make ‘it’ happen. I’m assuming it’s the latter. So what is it they make happen? Global financial meltdown? A massive public deficit due to bailing them out? Either way, I think perhaps they should update their ads to say something like ‘Look we are really really fecking sorry’, it might be more effective.

Installing and debugging a .NET service built in Visual Studio 2008

2009-08-28T21:51:00.001+01:00

There seem to be several tutorials on the web on how to create a Windows service using .NET in Visual Studio 2005 and before but I was unable to find one for Visual Studio 2008, so thought I’d write my own since I got tripped up a couple of times when I tried to install it.

The first step is easy, create a new project using New Project/Windows Service. The next obvious thing to try and do is to build it and run it. You will then be presented with the following error -

Cannot start service from the command line or a debugger. A Windows Service must first be installed (using installutil.exe) and then started with the ServerExplorer, Windows Services Administrative tool or the NET START command.

So then you fire up installutil from the command line passing the EXE’s full path as the only parameter. If you make my schoolboy error, you’ll be faced with the following error message -

Exception occurred while initializing the installation: System.BadImageFormatException: The format of the file 'WindowsService1.exe' is invalid.

This is because I ran installutil from the standard command prompt, rather than the .NET command prompt and it picked up installutil from the .NET Framework 1.1. So when I figured that out I ran the correct version of installutil and got the following error message -

No public installers with the RunInstallerAttribute.Yes attribute could be found in the C:\Source\dotNET2\WindowsService1\WindowsService1\bin\Debug\WindowsService1.exe assembly.

So the next thing to do is to add an installer to the service. This is achieved by right clicking on the service’s design surface and selecting ‘Add Installer’. after doing this and running installutil once again, the service should appear in the Services applet.

Finally, to debug it you’ll need to start it from the Services applet and then use the ‘Debug/Attach to Process…’ menu item in Visual Studio, making sure the ‘Show processes in all sessions’ checkbox is checked if the service is running under a different account to your own.

Regards WTF?

2009-08-22T23:18:00.001+01:00

When I was a young lad and was taught how to write a letter (a skill that has turned out to be not especially useful) I was told to end the letter with “yours sincerely”, which kind of baffled me at the time and still does. What does it mean, why would I want to be owned by the reader of the letter? why would I not be sincere?

And now, when my most of communication is via email, I’m receiving more and more of them ending with “regards” or “kind regards”. And now I’m asking myself the same question - what does that mean? It’s even happening with people I’ve known for years who have never put any strange incantations at the end of their emails in the past and have now started this odd practice. I guess emails are the new letters and we have to have some way of signing off from them. I’d always thought of emails as a fairly informal medium, but perhaps not anymore.

Regards Doogal

Virtual PC on Windows 7

2009-08-15T22:38:00.001+01:00

I decided to jump in and upgrade to Windows 7, because I keep hearing it’s what Vista should have been. And it’s OK. I think I’m getting too old to get excited about OS upgrades, these days I just get annoyed about what’s been broken.

But today I had need to fire up a virtual machine so launched Virtual PC 2007. It told me it was unable to get the network connection up and running and offered to fix it, but then couldn’t find the location of my setup files. Not to worry I thought, probably the best plan is to just reinstall the whole thing. So I uninstalled then attempted to re-install but was told Virtual PC 2007 doesn’t work on Windows 7 so it refused to install it (even though it was clearly working at least a little bit for me). But then I noticed there was a new version of Virtual PC available that would work on Windows 7. So I downloaded that to try and fix my problem. Unfortunately this new version of Virtual PC requires a CPU that has virtualisation built-in. Sadly my CPU is lacking that feature so I couldn’t install this new version.

So if you have Windows 7 and a CPU that doesn’t have the right features then you are basically stuffed. I have managed to get round the problem by rolling back my Windows installation a couple of days using System Restore. Virtual PC 2007 seems to work fine except for the lack of network support, but if you don’t already have Virtual PC installed then you will be somewhat stuck.

Using the StateDropDownEditor in state machine workflows

2009-08-15T13:39:00.001+01:00

Many moons ago I wrote about creating custom state machine activities. I’ve finally got back to doing some more work on this and thought it would be nice to add a dropdown to choose the target state for my custom activity. This looks pretty easy to implement, just decorate the property with the following attribute

[Editor(typeof(StateDropDownEditor), typeof(UITypeEditor)), DefaultValue((string) null)]

Unfortunately, StateDropDownEditor is an internal class so this won’t compile. The way round this is to fire up Reflector and copy the code for the StateDropDownEditor class. Not sure what the copyright issues are for this, but it starts to solve the problem. Unfortunately the StateDropDownEditor class references a StateMachineHelpers class which is also internal. We can again use Reflector to get hold of the implementation of this class, but things can start to get messy at this point. We end up having to pull in pretty much the whole state machine infrastructure code but the solution is pretty simple, just remove the GetCurrentState method and then no more code needs to be ‘borrowed’. I now have a nice dropdown for selecting the target state.

Poor man's profiling

2009-08-09T21:38:00.003+01:00

If you don't have access to a profiler or you can't be bothered to fire up your profiler to see where you've got performance problems, there's a pretty simple way to find performance bottlenecks in your code. Just keep breaking into your code and if a particular piece of code is causing problems, then chances are you will keep being taken to that piece of code.

More on string.Concat vs the + operator in C#

2009-07-09T10:06:00.001+01:00

A post of mine from 3 years ago about the performance differences between using string.Concat and the string class’s + operator got its first comment yesterday so I thought I’d flesh out what I said there to clarify what happens. First, here’s a little test program to show different ways to concatenate strings.

  class Program
  {
    static void Main(string[] args)
    {
      Console.WriteLine("a" + "b" + "c" + "d");
      Console.WriteLine(string.Concat("a", "b", "c", "d"));

      string a = "a";
      string b = "b";
      string c = "c";
      string d = "d";
      Console.WriteLine(string.Concat(a, b, c, d));
      Console.WriteLine(a + b + c + d);
    }
  }

So now lets look at the IL generated from that, using our old friend Reflector.

.method private hidebysig static void Main(string[] args) cil managed
{
    .entrypoint
    .maxstack 4
    .locals init (
        [0] string a,
        [1] string b,
        [2] string c,
        [3] string d)
    L_0000: nop 
    L_0001: ldstr "abcd"
    L_0006: call void [mscorlib]System.Console::WriteLine(string)
    L_000b: nop 
    L_000c: ldstr "a"
    L_0011: ldstr "b"
    L_0016: ldstr "c"
    L_001b: ldstr "d"
    L_0020: call string [mscorlib]System.String::Concat(string, string, string, string)
    L_0025: call void [mscorlib]System.Console::WriteLine(string)
    L_002a: nop 
    L_002b: ldstr "a"
    L_0030: stloc.0 
    L_0031: ldstr "b"
    L_0036: stloc.1 
    L_0037: ldstr "c"
    L_003c: stloc.2 
    L_003d: ldstr "d"
    L_0042: stloc.3 
    L_0043: ldloc.0 
    L_0044: ldloc.1 
    L_0045: ldloc.2 
    L_0046: ldloc.3 
    L_0047: call string [mscorlib]System.String::Concat(string, string, string, string)
    L_004c: call void [mscorlib]System.Console::WriteLine(string)
    L_0051: nop 
    L_0052: ldloc.0 
    L_0053: ldloc.1 
    L_0054: ldloc.2 
    L_0055: ldloc.3 
    L_0056: call string [mscorlib]System.String::Concat(string, string, string, string)
    L_005b: call void [mscorlib]System.Console::WriteLine(string)
    L_0060: nop 
    L_0061: ret 
}

OK, so looking at the first method where we concatenate string literals using the + operator and we can see the compiler helps us out by concatenating the strings at compile time, which is going to be as optimal as possible. The second example shows that the compiler doesn’t do this magic when we use string.Concat so string.Concat is actually slower in this scenario.

Now if we look at the next examples where we concatenate string variables, the generated IL is exactly the same! So the performance characteristics are likely to be somewhat similar to say the least. Things get more interesting when you get beyond 4 strings since there is no version of string.Concat that takes more than 4 parameters, so they have to be pushed into an array but the result is the same, the + operator generates the exact same code as string.Concat.

So I can’t see a scenario where you’d want to use string.Concat (unless you’re particularly fond of it) and if string concatenation performance is an issue, you probably should be using the StringBuilder class.

Encryption in Metastorm BPM

2009-07-08T16:42:00.001+01:00

A question came up on the Metastorm forums about encrypting sensitive data contained in custom variables so I thought I’d see what I could come up with. I took this C# code and translated it to JScript.NET, which looks something like this

import System;
import System.IO;
import System.Security.Cryptography;
import System.Text;
import eWork.Engine.ScriptObject;

package Encrypt.Encrypt
{
    public class Encryption
    {
        private static const password : String = "password";
        public static function Encrypt( ework: SyncProcessData, args: Object[] ) : Object
        {
            // args[0] - string to encrypt
            // returns the encrypted string

            var encrypt : Encryption = new Encryption(password);
            return encrypt.Encrypt(args[0]);
        }

        public static function Decrypt( ework: SyncProcessData, args: Object[] ) : Object
        {
            // args[0] - string to decrypt
            // returns the decrypted string
            if (args[0] == "")
                return "";

            var encrypt : Encryption = new Encryption(password);
            return encrypt.Decrypt(args[0]);
        }

        function Encryption(password : String)
        {
            GenerateKey(password);
        }

private var Key : byte[];
private var Vector : byte[];

        private function GenerateKey(password : String)
        {
            var sha : SHA384Managed = new SHA384Managed();
            var b : byte[] = sha.ComputeHash(new ASCIIEncoding().GetBytes(password));

Key = new byte[32];
Vector = new byte[16];

            System.Array.Copy(b, 0, Key, 0, 32);
            System.Array.Copy(b, 32, Vector, 0, 16);
        }

        public function Encrypt(plainText : String) : String
        {
            var data : byte[] = new ASCIIEncoding().GetBytes(plainText);

var crypto : RijndaelManaged = new RijndaelManaged();
var encryptor : ICryptoTransform = crypto.CreateEncryptor(Key, Vector);

var memoryStream : MemoryStream = new MemoryStream();
var crptoStream : CryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write);

crptoStream.Write(data, 0, data.Length);
crptoStream.FlushFinalBlock();

crptoStream.Close();
memoryStream.Close();

return Convert.ToBase64String(memoryStream.ToArray());
}

        public function Decrypt(encryptedText : String) : String
        {
            var cipher : byte[] = Convert.FromBase64String(encryptedText);

var crypto : RijndaelManaged = new RijndaelManaged();
var encryptor : ICryptoTransform = crypto.CreateDecryptor(Key, Vector);

var memoryStream : MemoryStream = new MemoryStream(cipher);
var crptoStream : CryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Read);

var data : byte[] = new byte[cipher.Length];
var dataLength : int = crptoStream.Read(data, 0, data.Length);

memoryStream.Close();
crptoStream.Close();

            return (new ASCIIEncoding()).GetString(data, 0, dataLength);
        }
    }
}

Then all that is required is to decrypt the string when the form is loaded and encrypt it when the form is saved, like so

%sensitive:=%ScriptEval(JScript.NET,,%Procedure.Name,%MapName,"Encrypt.Encrypt.Encryption.Decrypt",%sensitive )

%sensitive:=%ScriptEval(JScript.NET,,%Procedure.Name,%MapName,"Encrypt.Encrypt.Encryption.Encrypt",%sensitive)

Now the user should see the unencrypted text and the encrypted version will be stored in the database. You will also need to decrypt the data anywhere else you need to use it.

One thing to realise at this point is that the system is still not secure. Although it will stop casual viewers who just run a query against the custom variable table, it won’t stop a more professional hacker. The script text is also stored in the database, so a hacker can have a look at that and find the password used to encrypt/decrypt the data. A more secure implementation would store the password in a location that only the engine account has access to, assuming the engine account is also locked down.

Download the demo procedure

MooZoom with image maps

2009-07-04T15:28:00.001+01:00

MooZoom is a nice piece of JavaScript that adds zoom and pan functionality to images, built on top of MooTools. It wasn’t exactly what I needed. I didn’t want the zoom/pan to be constrained to the original size, I wanted the ability to zoom out beyond the original size and I wanted image maps to be handled correctly. I’m quite pleased with the results and you can download the source here.

To use it, in your image simply set

class="moozoom"

Day 21 – where I get offered a job

2009-07-03T21:36:00.001+01:00

3 weeks in and I get a job offer, pretty good going I think. I have the weekend to think about it. And given that a bird in the hand is worth two in the bush (even if the birds in the bush have really nice plumage) I will probably accept it.

For any other IT job hunters in the current climate, here’s my advice.

Throw your CV onto every job website out there.
Make sure everyone you know is aware you are looking for work, have no shame!
If you have a website or blog, make it obvious you are looking for work. Play the percentages game, every person who sees you are looking for work may be a potential employer.
Don’t demand to get paid as much as you were paid before. Assuming you’re out of work like me, your current income is zero or thereabouts, so your previous salary is pretty much irrelevant.
Accept all interviews. Even if the job isn’t a perfect fit, it’s good to get back into the interviewing groove, which will help when a better role turns up. And who knows, a job that doesn’t appear perfect on paper may turn out to be better than expected.
When you have an interview and you get asked about something you have no knowledge of, go off and investigate it. OK, it’s too late for that particular interview but it might come up again.
Learn about other technologies which may have passed you by in the past. Today the guy I spoke to said he was impressed with my use of the JavaScriptSerializer class, which I’d only started playing with the day before my technical test.

Technorati claim

2009-07-02T09:25:00.001+01:00

Technorati Profile

Resizing an image map when zooming an image

2009-07-01T21:43:00.001+01:00

There are some cool libraries out there for zooming images in a HTML document, but none that I’ve seen handle resizing an image map attached to the image. My research may be incomplete so I might be recreating the wheel here, but this simple solution seems to fix the problem. Now I need to integrate with one of those libraries.

First up my HTML looks like this

      <map id="map" name="map">
        <area coords="15,92,568,247" alt="Blah" href="javascript:alert('hello');" />
        <area coords="18,259,546,432" alt="Blah" href="javascript:alert('hello 2');" />
      </map>
      <input type="button" value="+" onclick="javascript:ZoomIn();"/>
      <input type="button" value="-" onclick="javascript:ZoomOut();"/><br />
      <img src="Highlight cells.png" usemap="#map" id="image" />

And then there is some JavaScript to do the resizing, that looks like this

  <script type="text/javascript">
    function ZoomIn() {
      Zoom(1.1);
    }

    function ZoomOut() {
      Zoom(0.9);
    }

    function Zoom(amount) {
      // resize image
      var image = document.getElementById('image');
      image.height = image.height * amount;

      // resize image map
      var map = document.getElementById('map');
      for (var i = 0; i < map.areas.length; i++) {
        var area = map.areas[i];
        var coords = area.coords.split(',');
        for (var j = 0; j < coords.length; j++) {
          coords[j] = coords[j] * amount;
        }
        area.coords = coords[0] + ',' + coords[1] + ',' + coords[2] + ',' + coords[3];
      }
    }
  </script>