Error logging in ASP.NET using a HTTP module

I wrote many moons ago about logging errors from an ASP.NET website but even then I knew there was a better way of doing it. The problem with that implementation was that it assumed the ASP.NET application was under my control. What happens if you want to add error logging to somebody’s else application for which you don’t have the source? Or what about an application you’ve written yourself that you’re selling to other people but also using internally, and you don’t want to prescribe to your customers how they should implement their logging? What is required is a more flexible approach to logging, something that can be plugged into any ASP.NET application.

Fortunately ASP.NET provides such a mechanism, via HTTP modules. Another reason I’d never looked at implementing logging this way was that I thought it might be tricky to do, but in fact it is so simple it took me all of an hour to produce a working solution.

The first thing to do is create a new class library and add a class that implements the IHttpModule interface. This interface has only two methods, Init and Dispose. Here's how I implemented it.

  public class ErrorLogger : IHttpModule
  {
    public void Init(HttpApplication context)
    {
      context.Error += new EventHandler(context_Error);
    }

    public void Dispose()
    {
    }

    private void context_Error(object sender, EventArgs e)
    {
      Exception ex = HttpContext.Current.Server.GetLastError();
      // log the exception however you like
    }
  }

Pretty simple. The logging code isn’t shown for brevity’s sake but all the class does is hook up an event handler for the Error event and within that event handler do whatever is necessary to log the error. In our case, we email the error and put it in a database table, but you can log it however you wish. (here’s some exception logging code for you)

Now all that’s required is to drop the assembly in the app’s bin directory and tell the ASP.NET application about the HTTP module. This is done via the web.config file with the following in the <system.web> section

  <httpModules>
    <add type="Utilities.ErrorLogger" name="ErrorLoggingModule"/>
  </httpModules>

type is the .NET type and name is whatever you like. And, er, that’s it.

As you’ve probably realised HTTP modules are powerful little things that can be used to do all kinds of stuff in a flexible manner.

Update – for those not reading the comments, there’s a free logging tool called ELMAH that can do all this for you without needing to write any code.

IE8’s compatibility view is not IE7

It would appear that IE8 has been released today which coincides with me finding out something unpleasant about its compatibility view. I’d assumed switching to compatibility view would show the web page using IE7’s rendering engine. But this doesn’t seem to be the case. Go to the Process Mapping website and try out the dropdown menus. When you move outside the boundaries of a menu it should disappear, as you’d expect. Using IE6, IE7, IE8 (native mode), FireFox, Safari and Chrome that’s exactly what happens. But in IE8’s compatibility view the menu stays where it is.

So I can only assume compatibility view is not IE7 in new clothing. Given that anybody can switch to compatibility view for whatever reason and returning to that site will stick with that setting, this means that the introduction of IE8 means rather than having one other browser to test against, we have two instead! Oh great…

Whatever happened to NDoc?

NDoc was one of my favourite tools for .NET. Document your assemblies with the .NET XML documentation tags, point NDoc at them and out comes a help file. It was simple to use but was also powerful enough to customize the output in most ways you’d want.

Go to the linked website and you’ll see not much has happened on the NDoc project since 2005. Unfortunately support for .NET 2 was never completed before the original author decided to give up on the project and it looks like nobody has picked up the baton to develop it further. This is understandable to an extent. After all Microsoft have come out with Sandcastle which essentially does the same thing as NDoc. But Sandcastle is difficult to love. It has no user interface and builds take forever compared to NDoc.

The user interface problem can be solved by using Sandcastle Help File Builder, which does a nice job of looking like NDoc, but is still hampered by the underlying Sandcastle technology. After much frigging around I’m still unable to figure out how to add custom HTML into my documentation (for Google Analytics and the like) on a per project basis. I can do it globally by messing with the templates but this means I have to modify the templates for each build of a particular project, which is not ideal.

Perhaps I’m missing something, but it seems sad that 4 years since development on NDoc stopped, we still don’t have something as easy to use or as fast, even with the might of Microsoft behind it. I’m not surprised the original author decided to stop working on it, apparently he was getting threatened because his support for .NET 2 wasn’t coming along fast enough… And I’m sure the thought of trying to compete with Microsoft didn’t encourage his efforts either. But Microsoft have failed to deliver IMHO so I’m still missing NDoc after all this time.

US states table for SQL Server

I couldn’t find any SQL to generate a table of US states for SQL Server though I found this for MySql, so I modified it slightly and came up with this for SQL Server. Create the table with this

CREATE TABLE States(
    StateCode char(2) NOT NULL,
    StateName varchar(250) NOT NULL,
 CONSTRAINT PK_States PRIMARY KEY CLUSTERED 
 (
    StateCode ASC
 )
)

And populate it with this

insert into States values ('AL', 'Alabama');
insert into States values ('AK', 'Alaska');
insert into States values ('AZ', 'Arizona');
insert into States values ('AR', 'Arkansas');
insert into States values ('CA', 'California');
insert into States values ('CO', 'Colorado');
insert into States values ('CT', 'Connecticut');
insert into States values ('DE', 'Delaware');
insert into States values ('DC', 'District of Columbia');
insert into States values ('FL', 'Florida');
insert into States values ('GA', 'Georgia');
insert into States values ('HI', 'Hawaii');
insert into States values ('ID', 'Idaho');
insert into States values ('IL', 'Illinois');
insert into States values ('IN', 'Indiana');
insert into States values ('IA', 'Iowa');
insert into States values ('KS', 'Kansas');
insert into States values ('KY', 'Kentucky');
insert into States values ('LA', 'Louisiana');
insert into States values ('ME', 'Maine');
insert into States values ('MD', 'Maryland');
insert into States values ('MA', 'Massachusetts');
insert into States values ('MI', 'Michigan');
insert into States values ('MN', 'Minnesota');
insert into States values ('MS', 'Mississippi');
insert into States values ('MO', 'Missouri');
insert into States values ('MT', 'Montana');
insert into States values ('NE', 'Nebraska');
insert into States values ('NV', 'Nevada');
insert into States values ('NH', 'New Hampshire');
insert into States values ('NJ', 'New Jersey');
insert into States values ('NM', 'New Mexico');
insert into States values ('NY', 'New York');
insert into States values ('NC', 'North Carolina');
insert into States values ('ND', 'North Dakota');
insert into States values ('OH', 'Ohio');
insert into States values ('OK', 'Oklahoma');
insert into States values ('OR', 'Oregon');
insert into States values ('PA', 'Pennsylvania');
insert into States values ('RI', 'Rhode Island');
insert into States values ('SC', 'South Carolina');
insert into States values ('SD', 'South Dakota');
insert into States values ('TN', 'Tennessee');
insert into States values ('TX', 'Texas');
insert into States values ('UT', 'Utah');
insert into States values ('VT', 'Vermont');
insert into States values ('VA', 'Virginia');
insert into States values ('WA', 'Washington');
insert into States values ('WV', 'West Virginia');
insert into States values ('WI', 'Wisconsin');
insert into States values ('WY', 'Wyoming');

WPF ASCII grid part 3

It’s been a while since I last posted about my efforts to write a simple WPF app. At the end of my last post I had managed to create a custom control to display my ASCII grid and things were coming along nicely. The next thing I wanted to was pretty straightforward, or so I thought… I wanted to add a property to my control that decides whether the grid shows the standard ASCII values or the extended ASCII range. To do this I thought the simplest thing to do, when the property value changed, was clear out my grid and re-add the cells with the new values. Maybe not the most efficient approach but it seemed like the simplest option since there doesn’t seem a way to access the cell contents of a grid to change the text values.

But the problem with this approach is it doesn’t really work. Changing the property value calls the code but afterwards all that can be seen is the grid lines, not the new contents of the cells. Resizing the window does show the new cell contents so I assumed this was a refresh issue. Looking in to how to refresh a WPF control came up with these possible options

• Call Dispatcher.Invoke with a priority of DispatcherPriority.Render
• Call InvalidateVisual
• Call InvalidateMeasure
• Call InvalidateArrange
• Call Measure
• Call OnChildDesiredSizeChanged

I’ve no idea which of these I should be calling but I tried all of them and none had any effect. So I am at something of a loss. I think the next step is to rework my code so I don’t recreate the TextBlocks I’m using to display the ASCII values and just update the Text property of the existing TextBlocks instead.

FEEDJIT

If you look further down this page, you’ll see a map of the world covered with what looks like the pox. Click on the map and you’ll be whisked off to the Feedjit site which shows the location of recent hits to this site and which pages were visited. I guess it isn’t really providing any more information than I can get from Google Analytics but I think it is displayed in a much more intuitive way. Are you listening Google?

Google AJAX APIs Playground

Mostly as a reminder to myself, Google have added a AJAX APIs Playground page that shows off some of the many features they provide in Google Maps, Local Search etc. There’s lots of things there that I had no idea about and you can edit the code in place to play around with the features. Sweet.

Copying a stream to another stream

It’s obvious that the ability to copy the contents of one stream to another is something that you’re likely to want to do fairly regularly. I vaguely recall that this will be added to some version of the .NET Framework but until that time, here’s a simple implementation that should do the trick. It could easily be converted into an extension method so it plays better in the .NET 3 world.

    /// <summary>
    /// Copies a stream to another stream
    /// </summary>
    /// <param name="copyFrom">The stream to copy from</param>
    /// <param name="copyTo">The stream to copy to</param>
    public static void CopyTo(Stream copyFrom, Stream copyTo)
    {
      const int buffSize = 128;
      byte[] buff = new byte[buffSize];
      int count;
      do
      {
        count = copyFrom.Read(buff, 0, buffSize);
        copyTo.Write(buff, 0, count);
      }
      while (count > 0);
    }

Spotify – the future of music?

I’ve just discovered Spotify (via Tim Anderson) and so far I’m thoroughly impressed. Previous implementations of this kind of thing (last.fm, Pandora) have restricted your choices of what you can listen to, making them feel more like listening to a radio station than listening to your own music collection. But Spotify lets you choose exactly what you want to listen to from its database. It doesn’t have every song by every artist, like most online music sources there’s no Beatles (obviously Jacko doesn’t need the extra cash, which doesn’t quite match with what I’d heard), some of my obscure musical preferences are missing (no Godspeed You Black Emperor) and even the artists they do have aren’t generally complete. But that’s a minor quibble, there’s a still a huge range of music available.

Streaming is super quick and the ads are currently pretty infrequent and unobtrusive. The bandwidth requirements seem pretty minimal too, I could happily work in a Remote Desktop session whilst listening and I didn’t notice any dropouts.

So is this the future of music listening? I certainly hope so. The big question is whether their business model works. Will the ads cover costs? Will the music companies suddenly get cold feet like they did with Pandora in the UK? Will their servers continue to be so snappy when they become super popular? If the ads become too frequent or in your face, listening might become painful. OK, you can get rid of the ads by paying $9.99 a month but if I was to pay for any online music service I’d want to have the physical bits (with no DRM) so I’d be sure I can still listen to them if the company folded.

The only thing I’d really like to see is the integration of the online stuff with my own music library. So if I want to listen to my own ripped music I can from the same player, rather than having to jump out to iTunes. But in the mean time, I’ve got a lot of new (to me) music to listen to.

BPM for business users

From my time on the Metastorm forums one thing is obvious, most people implementing Metastorm solutions are not business people. Mostly they are technical people, with a few not so technical people. And it’s generally the not so technical people who are having the biggest problems getting to grips with the product. Is that a failing of the software? I’d say not, although it could arguably be a failing of the marketing of the software, which like most BPM products suggests a non technical person can use it from start to finish to produce a working system.

This situation reminds me of other areas of software development, like the development of websites, or application development. A while ago, people thought anyone with a copy of FrontPage could develop their own website or anyone with a copy of VB could knock together a working application. But these other areas have matured enough to recognise that actually those pesky developers are still required (or to put it another way, designers can’t code and coders can’t design) and the development environments are now set up in such a way that the designers can work with the coders to produce a working system. For a website, the designer can produce a HTML layout that looks good and then the coder can add the required code so it does something. In the world of Windows development we are moving to WPF and other technologies where the layout is separated from the code, so the designer can produce the pretty screens and the coder can then add code to do stuff without both sides stepping on each others toes too much.

And for me, that’s the approach that BPM will have to take at some point, realising that the business people can’t code and the coders don’t understand the business, so trying to put only one of these people in charge of the BPM system will never fly. I think Windows Workflow is a step in the right direction with its ability to create custom activities. The coder can create all the required custom activities and then the business person can plug these together like Lego as required. Or this could work in the opposite direction, where the business user produces their process and adds dummy activities where they need functionality to be implemented and the coder goes off and creates the required activities.

With that in mind, I created a simple PrototypeActivity that can be added to a workflow and lets the user specify the inputs and outputs that will be need to be implemented. The screenshot shows how it looks in operation (although a proper implementation would probably provide a better user interface). The code is below.

  public enum PropertyDirection
  {
    In,
    Out,
    InOut
  }

  public class ActivityProperty
  {
    private string propertyName = "";
    public string PropertyName
    {
      get { return propertyName; }
      set { propertyName = value; }
    }

    private string description;
    public string Description
    {
      get { return description; }
      set { description = value; }
    }

    private PropertyDirection direction;
    public PropertyDirection Direction
    {
      get { return direction; }
      set { direction = value; }
    }

    public override string ToString()
    {
      return direction.ToString() + " - " + propertyName.ToString();
    }
  }

  public class PrototypeActivity: Activity
  {
    public static DependencyProperty PropertiesProperty =
      DependencyProperty.Register("Properties", typeof(ActivityProperty[]), typeof(PrototypeActivity));

    [DesignerSerializationVisibility(DesignerSerializationVisibility.Visible)]
    [Browsable(true)]
    [Description("The properties the activity requires")]
    public ActivityProperty[] Properties
    {
      get
      {
        return ((ActivityProperty[])(base.GetValue(PrototypeActivity.PropertiesProperty)));
      }
      set
      {
        base.SetValue(PrototypeActivity.PropertiesProperty, value);
      }
    }
  }

Rude Swiss

Continuing with the rude theme, here’s a photo from my not so recent trip to Basel in Switzerland. We may assume it’s the Dutch who have the monopoly on lewd behaviour and the Swiss are more interested in banking, cuckoo clocks and chocolate. But perhaps the racial stereotypes are incorrect, here’s a shop devoted to selling condoms. I’m probably showing off my ignorance, but how can you have a shop that only sells condoms? Are there enough varieties to fill it? I didn’t enter to find out the answer.

Rude snowman

Perhaps it’s just my dirty mind, but the snowman that Jo and Lola made the other day ended up looking less like a man and more like, well something else entirely. The huge erection has now melted away…

Not getting album artwork from iTunes

I had a thought the other day, how does iTunes get its album artwork? I guessed it was some kind of web service so fired up Fiddler to see if anything was getting thrown around. And sure enough iTunes was throwing a HTTP request to some server, that looked something like this (this is somewhat simplified since you can also pass CDDB information which presumably increases the chances of getting a correct hit).

http://ax.itunes.apple.com/WebObjects/MZStoreServices.woa?an=Cult&pn=Love

Put that in your browser and you’ll see an XML document is returned. It’s in Apple’s favoured XML format and it looks like there is a URL in there that can be used to retrieve the album artwork. Unfortunately, plugging that URL into your browser won’t return the image… At this point I kind of lost interest because I didn’t actually have any use for getting hold of album artwork (unless I wanted to automate the process for iTunes but I’m not that anal about getting artwork). But I did have a bit of a look round the web and found these two articles on doing the same thing. And it would appear Apple are pretty keen on people not downloading artwork via their own code. Dunnow why, copyright issues I guess, but it looks like even if it is possible to hack the system, Apple will change the way it works to stop your code working.

If you do want to download artwork, I suspect Amazon Web Services is going to be a better approach.

How to confuse IE with a 302 redirect

It’s very simple, create a PHP pages called test.php, the contents of which are

<?php
  header("Location: test.php");
  exit;
?>

Now point IE8 (probably earlier versions of IE as well I’d guess) at the page and watch as it tries in vain to fetch the page. It eventually times out, after trying to fetch the same page over 1000 times (take a look with Fiddler to see the problem). FireFox and Chrome are much more sensible in this scenario and give up quite soon. As the dumb developer who caused this problem, it took me quite some time to figure out my redirect wasn’t working rather than the website being very ill

WPF ASCII grid part 2

This is the second in a series where I try to learn how to develop WPF apps. The first part is here.

So my next step was to put all the ASCII grid code into a custom control. My first attempt was to inherit from Grid and add the logic to create the cells in the inherited class. The problem with this approach was that the grid’s row and column definitions could be edited in Visual Studio which I didn’t want. So I decided to inherit from Panel and add the grid as a child control. When I did that, running the application showed up nothing at all. I finally realised that was the wrong approach, since the Panel class is designed to allow the end user to add their own child controls. I guess the WPF runtime is looking at the XAML to see which children should be added to the panel and there aren’t any. So finally I inherited from Control and then had to figure out how to add the grid as a child. Control doesn’t have a Children property so it’s not obvious how to add your own child controls. But it turns out it’s pretty easy to do. You just have to override VisualChildrenCount and GetVisualChild as shown below.

  public class AsciiGrid : Control
  {
    static AsciiGrid()
    {
      DefaultStyleKeyProperty.OverrideMetadata(typeof(AsciiGrid), new FrameworkPropertyMetadata(typeof(AsciiGrid)));
    }

    private Grid grid;
    public AsciiGrid() : base()
    {
      grid = new Grid();

      const int width = 16;
      const int height = 16;
      for (int x = 0; x < width; x++)
      {
        grid.ColumnDefinitions.Add(new ColumnDefinition() { Width = new GridLength(1, GridUnitType.Star), });
      }
      for (int y = 0; y < height; y++)
      {
        grid.RowDefinitions.Add(new RowDefinition() { Height = new GridLength(1, GridUnitType.Star), });
      }
      for (int x = 0; x < width; x++)
      {
        for (int y = 0; y < height / 2; y++)
        {
          int asciiValue = ((y * width) + (x + 1));

          // number
          TextBlock rect = new TextBlock();
          rect.Text = asciiValue.ToString();
          rect.SetValue(Grid.RowProperty, y * 2);
          rect.SetValue(Grid.ColumnProperty, x);
          grid.Children.Add(rect);

          // ASCII value
          rect = new TextBlock();
          rect.Text = Convert.ToChar(asciiValue).ToString();
          rect.SetValue(Grid.RowProperty, (y * 2) + 1);
          rect.SetValue(Grid.ColumnProperty, x);
          grid.Children.Add(rect);
        }
      }
    }

    protected override int VisualChildrenCount
    {
      get
      {
        return 1;
      }
    }

    protected override Visual GetVisualChild(int index)
    {
      return grid;
    }
  }

So I now have a standalone control in my application. The application looks exactly as it did before but I’ve now got an exceedingly useful re-usable ASCII grid control. OK, maybe not so useful, but this is a learning experience, so cut me some slack. The only thing to note is the changes required to the XAML to host the control, which looks like this.

<Window x:Class="Ascii.AsciiWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:Ascii="clr-namespace:Ascii"    
    Title="Ascii" Height="370" Width="413" Loaded="Window_Loaded">
    <Ascii:AsciiGrid x:Name="m_Grid">
    </Ascii:AsciiGrid>
</Window>

WPF ASCII grid part 1

I thought it was about time I learned about WPF so thought I’d start with a very simple application, a grid showing the ASCII characters. It’s kind of pointless at the moment since you can find this information on the web pretty easily but I think I can make it somewhat more powerful and in the process learn more about WPF. This is what I’d like to add in the future and will hopefully blog about as I implement them

• Turn the grid into a control
• Show the extended ASCII characters
• Make it look much sexier
• Let the user choose the font
• Let the user change the code page

The code currently looks like this. It’s all pretty straighforward. The first thing to note is the GridUnitType.Star enumerated value which means each column and row will be spaced equally in the window.

The other thing to note is the weird way the text blocks are assigned to the correct cells of the grid. It doesn’t seem a natural way to do it, but I guess once you know that’s how it’s done, it’s pretty straightforward.

    public AsciiWindow()
    {
      InitializeComponent();
      BindGrid();
    }

    private void BindGrid() 
    { 
      const int width = 16;
      const int height = 16;
      for (int x = 0; x < width; x++) 
      { 
        m_Grid.ColumnDefinitions.Add(new ColumnDefinition() 
        { Width = new GridLength(1, GridUnitType.Star), }); 
      }
      for (int y = 0; y < height; y++) 
      { m_Grid.RowDefinitions.Add(new RowDefinition() 
        { Height = new GridLength(1, GridUnitType.Star), }); 
      }
      for (int x = 0; x < width; x++) 
      {
        for (int y = 0; y < height/2; y++) 
        {  
          int asciiValue = ((y*width)+(x+1));

          // number
          TextBlock rect = new TextBlock();
          rect.Text = asciiValue.ToString();
          rect.SetValue(Grid.RowProperty, y*2); 
          rect.SetValue(Grid.ColumnProperty, x); 
          m_Grid.Children.Add(rect); 

          // ASCII value
          rect = new TextBlock();
          rect.Text = Convert.ToChar(asciiValue).ToString();
          rect.SetValue(Grid.RowProperty, (y * 2)+1);
          rect.SetValue(Grid.ColumnProperty, x);
          m_Grid.Children.Add(rect); 
        } 
      } 
    }

Read part 2

Bugs = complexity

Many moons ago I discovered a bug in some software I use on a fairly regular basis (I won’t tell you which software since it isn’t relevant to the story and I don’t want this to appear to be a critique of the company or their development process, though you can probably figure out who it is if you look round the rest of this site). The application allows you to write server scripts in JScript.NET. It also does some validation of these scripts but in certain circumstances this validation incorrectly flags up problems with scripts that aren’t actually problems. It was a fairly innocuous bug so I reported it but didn’t push for it to be fixed.

Fast forward a year or so and a new version of the software comes out with a new feature that validates your scripts when you try to deploy a project to the server. Generally a useful feature except the previous bug is still there. Meaning I now can’t deploy perfectly valid projects due to the interaction of the previous bug with this new feature. So a minor unfixed bug has blown up into a major issue.

Which got me thinking. The kind of bugs that don’t get fixed are these kind of things. They only occur in particular rare scenarios. If they happened all the time, the chances are they would get fixed, if they are serious enough. But they do add complexity to the product and testing. No longer can we assume doing A will cause B to happen. Most of the time it will be true but occasionally doing A will cause C to happen. So any testing of new features will have to take this into account. In fact, testing of any new feature will have to take into account any unfixed bugs that may impact the new functionality.

There are many reasons for trying to fix as many bugs as possible (keeping customers happy, trying to fight the constant entropy inherent in software development), may I add this to the list? 

Backups

My backup strategy over the last few years has been pretty lame. Every month, Outlook reminds me I should do a backup of my source and I burn a CD. This kind of worked for a while but I’ve been getting more and more nervous about it. Am I sure I’m backing up everything I need to? Most of my email is stored on the web in GMail but there’s heaps of other things on my PC that may be vital but I just haven’t realised it yet. Not only that, but I was getting to the stage where a single CD wasn’t enough and the time taken to burn even that single CD was starting to bore me.

So today when I popped into Maplin for a USB cable I thought I’d have a look at the external hard drives. The Toshiba Stor E looked like the best bang for buck, 500GB for £65. So far I’m impressed. It looks pretty robust, with an aluminium case and no pointless bling. Installation was exceedingly straightforward. Plug it in and Vista recognised it immediately. Then simply configure Vista to do an automatic backup every week and that’s it.

Admittedly my first backup is currently happening and it’s not super fast, but I guess that will improve once it’s done the first full backup. Once it’s finished I’ll have to check that it’s actually backed up everything I want it to, since the backup utility is pretty vague about what it’s going to do. And then I’ll have to think about taking my backup offsite, which for me means carrying the drive into the house. If my shed and house burn down, I’ll have bigger worries than having a decent backup. 

Dan le Sac vs Scroobius Pip "Thou Shalt always Kill"

I just love this. It’s funny, has a great beat and what a beard!

One of the problems with working from home

However much it snows this evening, I can’t call the office in the morning claiming the weather is too bad for me to get to work.

A simple plug-in framework for .NET apps

If you want to add plug-in support to your application then you might start looking at Microsoft’s Managed Extensibility Framework or System.Addin. Whilst I have no experience of these frameworks, when I looked at them I decided they were just too complicated for my needs. Also I was targeting .NET 1.1 so couldn’t use either of them. All I wanted to be able to do was to write a class that implements a particular interface (for importing files from one of a number of external file formats), dump it in the application’s directory and for the application to recognise the assembly and add the supported file type to its list of available file types. I won’t bore you with the details of the interface, since it’s obviously specific to the application being developed, but here’s the code for loading the assemblies and creating instances of the available importers.

      // load all importers
      string applicationDirectory = System.IO.Path.GetDirectoryName(
        Assembly.GetExecutingAssembly().Location);
      DirectoryInfo info = new DirectoryInfo(applicationDirectory);
      FileInfo[] files = info.GetFiles();
      List<Type> types = new List<Type>();
      for (int i = 0; i < files.Length; i++)
      {
        // load assemblies and see what is in there
        if (files[i].Extension.ToLower() == ".dll")
        {
          Assembly thisAssembly = Assembly.LoadFrom(files[i].FullName);
          Type[] assemblyTypes = thisAssembly.GetTypes();
          for (int j = 0; j < assemblyTypes.Length; j++)
          {
            if (assemblyTypes[j].GetInterface("IImport") != null)
            {
              ConstructorInfo wfConstructor = assemblyTypes[j].GetConstructor(Type.EmptyTypes);
              object obj = wfConstructor.Invoke(null);
              IImport importer = (IImport)obj;

              importers.Add(importer);
            }
          }
        }
      }

This worked well for me. I’m sure it wouldn’t work in all scenarios and MEF/System.Addin may be more appropriate to more complex scenarios (if you are worried about versioning or running your add-in in a sandbox for instance) but it’s a quick and dirty method for implementing plug-ins.

Turn on script debugging!

The other day I was pointed in the direction of a new website for a company I know. The owner was very pleased with the work a web design agency had done for him. Unfortunately the first thing I saw was a script error, then another then finally the page appeared. Pretty nice it was too, but the whole experience was somewhat spoiled by those script errors. Of course most people have script debugging turned off so they don’t see these errors but if you’re paying someone to develop your website it is something you need to do.

I wouldn’t recommend having it turned on all the time, the fact is the web is almost unusable if you do have script debugging enabled. Some websites cause so many errors that the only solution is to kill your browser and a large number of high profile websites have some kind of script error. Of course, some would say this is only an IE problem and point to the crappy way IE implements the debugger notification in a modal dialog box and that is certainly true. Perhaps it was done that way so that it is so in your face, that if you’re the developer of the site you just have to do something about it. I think the actual consequence is that anybody who has turned it on turns it off at the earliest opportunity.

But I digress, the point is if you’ve paid somebody to develop your website, make sure they’ve actually done a decent job, even if the only people who’ll notice they haven’t are anal geeks like me.

The value violated the integrity constraints for the column

I was trying to import some data from an Excel spreadsheet into SQL Server today and kept getting this error message.

Error 0xc020901c: Data Flow Task: There was an error with input column "PHNUMBR1" (207) on input "Destination Input" (96). The column status returned was: "The value violated the integrity constraints for the column.".
(SQL Server Import and Export Wizard)

I was having trouble working out what SQL Server was trying to tell me. The table didn’t have any constraints that I could see so I was at a loss. The only seemingly related post was this one, unfortunately in Chinese. But it gave me a clue. I saw the words ‘NOT NULL’ and realised that was the problem. The table had some columns that did not allow NULL entries. I removed these restrictions and the import worked.

So the questions for today are - am I just stupid? Should SQL Server provide a better error message? Should it just populate NOT NULL columns with empty strings?

Waiting for ever to be an overnight success

Mister Coding Horror Jeff Atwood talks some sense about overnight success, namely that it doesn’t generally happen. Success is a long road, it can takes years for a blog or software to reach any kind of critical mass. But I think he misses an important point. Namely, that success may never come, all that hard work may be for nothing. Take blogging. How many blogs are out there? Millions probably. How many are successful? A few thousand perhaps, so the chances of hitting it big are pretty remote. What about software? Since the barriers to entry are a little higher, there’s not so much software around but how many successful bits of software are there in each mature category you look at? Two or three generally. How many software products have fallen by the wayside? Generally many more. Websites? The same kind of success rates I’d guess.

So if you enjoy writing your blog, cool. If you enjoy hacking code, great. But don’t assume you’ll be successful, over any time span. Don’t think you will be able to retire to your own private island, however much effort you put into it. Have a plan B. Get a job that pays a reasonable wage (not just piles of stock options), put some money away, pay down your debts. Oh, and don’t call your blog ‘Bell End’.

XslTransform not generating XML declarations

I’d just started having a look at some .NET 1.1 code using the XslTransform class and I was having a bit of trouble. The program that read the generated XML expected an XML declaration at the top of the document. No problem I thought, just add an omit-xml-declaration="no" attribute to the XSLT. I did that and the XML declaration still didn’t turn up. So I checked the XslTransform class to see if that had some property to force the declaration to appear but no. Then I tried fiddling with the other xsl:output attributes in the XSLT, but still nothing.

So what about the XmlTextWriter class then? No, nothing on that to control the XML declaration (although it looks like later versions of the .NET Framework do include support for this). But hang on, why am I using the XmlTextWriter class anyway? The XslTransform class will take a stream instead. And tada! That fixed the problem and also saved a few lines of code. So only an hour of my life wasted. Hopefully this will save you that time.

Easy SQL

Accessing databases on Windows has been a constantly moving target over the years. First there was ODBC, a pretty simple C based API. When COM came along, MS decided that they needed to have an object based data access library so out came DAO, followed pretty swiftly by RDO, OLEDB and ADO. When .NET came out we got ADO.NET and then recently we've had LINQ to SQL and now Entity Framework. LINQ to SQL lasted such a short time that I didn't even have time to look at it properly before it died. And there's probably others I've forgotten about.

Some may not think this is a problem. After all, the older technologies are still available so there's nothing stopping you continuing to use them, but the chances are you'll have to maintain a project at some point that uses another data access technology so you'll need to have a passing knowledge of all of them.

So I have an idea, Microsoft. How about adding a common API to all your data access technologies? Here's my suggested interface.

public interface IEasySQL
{
  void Connect(string connectionString);

  void ExecuteSQL(string sql);

  object[][] SelectSQL(string sql);
}

I think it's fairly obvious how this interface works, so I won't bore you with the details. Now some of you are probably thinking it's missing a few features. Things like transactions, parameterised queries, that kind of thing. Fair point, but for me at least, this would be adequate 90% of the time. If I need the more complicated stuff I can always go to the underlying API, but for most of the time I can just throw a query at the database, because that's all I need to do. Yeh, it's not scalable, but generally my queries only return a few results, maybe a thousand, and this will work fine. So how about it, MS? And if anybody is interested, I can knock together an implementation using ADO.NET.

The Black Swan - Nassim Nicholas Taleb

I don't think the author of this book is the person who coined the saying 'black swan' but that isn't really important. What is important is what he's trying to say, that we tend to ignore the impact of highly improbable events, mostly because we aren't able to predict them. The saying comes from the fact that we thought all swans were white until somebody found a black one and our definition of a swan was turned on its head.

I think this book has become a commercial success due to a little bit of serendipity, it arrived on the shelves of bookshops at about the same time as the financial meltdown kicked off, arguably a black swan in itself (although I'd argue quite a few people saw this coming but not many realised how serious it would become). I think that's what they call ironic.

Anyway onto the book. For me it only really comes into itself in the last few chapters where he gets into the technical details of our failure to predict, which is mostly down to our use of the Gaussian/normal distribution where it isn't applicable. We should be using fractals instead (before reading this I thought fractals were purely used to produce pretty pictures). The earlier chapters could have been shortened more for my liking, there's only so many ways you can say we fail to take notice of the impact of the highly improbable (except after the event when we manage to produce a narrative to explain why it happened and wasn't totally unexpected). I guess I'm more interested in the hard facts, rather than the philosophical musings. But that said, it's still a very good read and certainly makes one think. I guess the important point is that we should try to increase our exposure to potential positive black swans (business opportunities etc) whilst reducing our exposure to potential negative ones.

Recycling is wrong

Our council finally got its recycling strategy sorted out recently. This means that recycling is collected once a week instead of every other week and they now collect cardboard and food waste which they didn't do previously. I'm quite happy with this since it means I don't have to make regular trips to the tip anymore. Of course there have been some teething problems and I'm seriously bored of all the complaints from my neighbours. It's no wonder that Australians call us whinging poms...

But I do keep thinking there is something wrong with the whole concept of recycling. I buy a bottle of wine, drink it, then the bottle gets taken away and recycled and made into another bottle. But, but... How much energy is being used up in producing a bottle that is only used once? When I was a lad, we returned bottles to the shop (fizzy pop rather than wine in those days) and got back 2p for our bother. The bottle then got sent back to the fizzy drinks company who refilled it and sent it out again. In fact, go to India and many other 'less developed' countries today and you'll find the same thing. When you buy a drink, it comes in a scratched old bottle that has obviously been used several hundred times before. Isn't this the model we should be adopting? It seems a much better use of resources. The thing is the supply chain is already in place, all those bottles get delivered to the shops, so the same lorries could be used to return the empties back to their original source. What am I missing?

Content is still king

One of the phrases that was coined during the early years of the web was 'content is king'. The basic idea was that content drives traffic to your site. This is clearly correct for the simple case - if you have no content at all you'll get no visitors and when you add some content you'll hopefully get more than zero visitors.

But that was in the days of not so clever search engines. These days search engines don't just take notice of the content on your site, they look at all kinds of other things so there's much more emphasis on using search engine optimisation to optimise content to get as many hits as possible. I don't think there's anything wrong with using SEO and fiddling around with keywords, layouts, titles, URLs and reciprocal linking can certainly help (although it's quite possible to get it wrong as well and lose traffic) but it seems to miss the point. The absolute simplest way to increase traffic to your site is to add more content.

To illustrate, lets take a look at some real data. I've used archive.org and data from Google Analytics to produce this graph for the Random Pub Finder.

What does this show? I've taken the number of monthly visitors to the site and divided them by the number of pubs reviews on the website. This doesn't give the complete picture of the amount of content on site but I'm not sure how I can work out the number of unique URLs historically. Note, using page views instead of visitors produces a pretty similar graph.

So does it prove anything? I guess if content was the only thing affecting the number of visitors then we'd expect the trend line to be horizontal so there are clearly other factors at work. We've got more inbound links than we used to have (that big spike is when we got linked from another site). I also think age of content helps search engine rankings, although I don't have any proof for this theory. But given that I've done very little SEO work on the site, it does suggest increasing content increases visitors above and beyond a direct linear increase.

As a comparison, looking at the number of visitors to www.doogal.co.uk, where the amount of content has not increased very much, shows visitor numbers haven't increased as dramatically.

To look at it another way, every new page you add to your site will add X new visitors to your site, where X can be anything from zero to a very large number. Adding a new page will probably increase the number of visitors to your site, however small.

So in conclusion, don't waste your time and money on snake oil SEO 'experts', just add some more content to your site if you want to get more visitors.

Windows Search now slows down Internet Explorer as well

One of the things that caused bad initial experiences of Windows Vista was Windows Search. As soon as you'd installed Vista, off it went indexing files on your hard disk. This could take some time and although it runs as a background task it still seems to impact the performance of your PC. Lots of people just turned off the service, although if they'd left it on for a while they'd have discovered that once the initial scan had been finished, it generally had a pretty low impact on system performance.

I left it on but I've never been overly happy with it. Fact is I don't do much searching on my PC and when I do I'm not too bothered if it takes a while. So having a constant tax on my performance (however small) seems a waste. Not only that, but it doesn't seem to index the files that are important to me. So when I do do a search I invariably have to switch to the 'search everything' mode which is really slow meaning the whole indexing palaver is completely wasted. I guess I should have set it up to index everything but never bothered, thinking it would cause the indexer to use up even more processor time.

But along comes IE8 and Microsoft have decided to use Windows Search technology to power the drop down list of suggested sites. Previously I'd used this a lot, type the first few letters of a site that I visit frequently and it would appear in the drop down list and I could select it rather than having to type the full URL. In theory that still works, but Windows Search always takes several seconds to decide what to show in the list, so it's as quick just to type in the full URL. I guess the list of suggested sites is better but the delay means it's completely useless. And typing another letter causes the search to start again from scratch...

But there is a solution to this. Disable the Windows Search service and the drop down works just like it used to do and I'm happy again. I guess this is a classic unintended consequence of powering IE8 with Windows Search, that even more people disable it.

Revenue models for free software

I have no ideological belief that software should be free, but it certainly seems to be an increasingly prevalent model for software.  Since I don't run my own software company the code I produce doesn't have pay the bills. But although I enjoy writing it, in an ideal world I'd still like to make some kind of income from my work. Probably my biggest piece of code that I've given away has been the FreeFlow library and Administrator, that can be used with Metastorm's BPM product. I've tried out a few different revenue generation techniques with varying degrees of success.

Sell the source - This was the initial model, let anyone download the binaries but charge for the source. It worked reasonably well for the FreeFlow library, which is used by developers to talk to the Metastorm engine. I guess developers are keen to get hold of the source for libraries they are using, to modify for their own needs and reduce the risk if the developer disappears. Unfortunately Metastorm introduced their own .NET library and as time has gone on there's been a lot more interest in the standalone FreeFlow Administrator application. With this, I suspect less people care about the code since if I disappear it won't actually affect their own applications, so sales of the source code have dropped off.

Website advertising - There's been advertising on the FreeFlow website for a long while and I've made a bit of spending money from it, but that's all. One positive of writing software for a BPM product is that most of the ads are for competing BPM products which seem to earn a good amount of money. The downside is that Metastorm is just one of many BPM companies and BPM is still a fairly niche area so I don't get a massive number of visitors to the site.

Donations - This is my latest attempt at earning a bit of cash. The website and the FreeFlow Administrator now have 'Donation' buttons. So far I've not earned anything from this at all. I'm not hugely surprised. If I'm using a free piece of software I very rarely donate any money. If it's shareware, I'm unlikely to pay for the full version even if it has nag screens (WinZip anyone?). The exception to this is SmartFTP, which I paid for because it needed re-installing every few months and it's actually a nice bit of software.

Ads in the application - I haven't tried this yet and I'm not even sure how ads can be added to a desktop application but it seems to be the path FeedDemon is going down. I may wait and see how well it goes for that software and my own donations before going down the same route. One of the nice advantages of not needing to earn any money from my software is I can try out different models for as long as I want before trying out something else.

A buy out - There has been interest from people wishing to buy FreeFlow off me but there have been no concrete offers as yet. I'd definitely consider a serious offer. Even though I have some emotional attachment to the code I've written I'm not dumb enough to refuse a reasonable amount of money if somebody offered it. Unfortunately as this is a niche product, I can't really see there being a huge number of suitors... 

In conclusion, I think if I was starting from scratch I'd probably try writing software with a potentially wider audience. Something like TimeSnapper, which is of use to almost anybody. And I'd probably try a similar pricing model to their software, a basic free version and a more feature-rich professional version. Or an ad-sponsored free version and a paid-for non-ads version. Now I just have to figure out what that software would be...

Free controls for .NET

I've come across a couple of free controls for .NET recently which look pretty cool.

DevExpress - I've always loved DevExpress. Their Delphi controls rocked (and probably still do), their .NET controls rock and their pricing has always been pretty reasonable ($2000* for every .NET control they produce for a year). Now you can get some of them for free. The download is a bit large since it includes trial versions of everything they do but some of them look pretty useful. I haven't looked too closely but from a cursory look it seems like the ASP.NET controls are more useful than the WinForms controls, which are mostly just some edit controls.

Chart control - Microsoft bought the Dundas chart controls and now you can use the Microsoft updated versions in your ASP.NET and WinForms applications for free. I assume at some point these will get rolled into the .NET Framework meaning it will be possible to add charting to apps with no extra installation woes to worry about. It will be interesting to see what now happens to Dundas (and ChartFX and any other chart control companies for that matter). I guess they have products for other platforms and more specialised charting controls that will keep them in business. If not, they may live to regret the decision to sell their code...

* Actually $1999.99 but it's quite close to $2000 - does the .99 cent/pence ruse actually still work?

The predefined type 'System.Runtime.CompilerServices.ExtensionAttribute' is defined in multiple assemblies in the global alias

Extension methods are cool and they can be used from .NET 2.0. I'm not the first to suss this out but the trick is to add the following to your code.

namespace System.Runtime.CompilerServices
{
  internal sealed class ExtensionAttribute : Attribute { }
}

This works well but there is a gotcha. Say I have two assemblies and I've added the definition to both assemblies because I want to add extension methods to both, I get the warning message

The predefined type 'System.Runtime.CompilerServices.ExtensionAttribute' is defined in multiple assemblies in the global alias

I try to produce code that doesn't generate any warnings since it's easy to miss real warnings if there are these kind of false positives. One solution is to make the definition public in one of the assemblies but I wasn't keen on that idea because the assembly is a publicly available class library whose public API I didn't want to pollute with this hack.

So the final solution was to use the InternalVisibleTo attribute in one of the assemblies. This means ExtensionAttribute only needs to be defined in one assembly. There's more info here on how to do that with strongly named assemblies since it's not completely straight forward.

One final problem is that using the assembly in a .NET 3.5 application will show the warning again since ExtensionAttribute is already defined in the .NET Framework. As far as I'm aware there isn't a solution for this problem, except producing separate .NET 2 and .NET 3.5 versions of the assembly.

The RPF book has a cover

Thanks to my other half, the Random Pub Finder book now has a proper cover. Unfortunately, in the mean time, Lulu has changed their publishing prices which means the cost for an A5 book is now much more than their standard sizes. Which, in these credit crunch times, may be putting off some purchasers. So now I'm busily repaginating it to fit into a US Trade size, whatever that is. Whatever it is, it's much cheaper to publish. So you still may want to hold off purchasing it for a bit, if you were planning on doing.

Sales so far - 4. Ho hum...

"One or more waypoints are not routable in the specified data source" error in MapPoint

I've searched on the internet for any information about this error message returned by the MapPoint web service but haven't been able to find anything, so this is a rundown on what I've found, although this is fairly obvious stuff.

Say you're trying to route between points A, B, C, D and E. Basically the error is saying that MapPoint was unable to route between at least two of these points. So the first thing to do is figure out which points MapPoint didn't like. this can be achieved by breaking the route down into its constituent parts and try to route A to B, B to C, C to D and D to E. The chances are that at least one of these will fail.

Now it's time to take a closer look at the two points that failed. I think the best way to do this is to visualise the route using Google Maps and/or Live Maps. For me, both sites successfully routed between the two points but Google Maps showed something strange. The suggested route involved driving to a T junction, turning right, driving to the end of the road and making a  U turn and driving back along it. Moving the point to the end of the road with the T junction fixed the MapPoint error. For me, being able to route between the points was more important than being completely accurate. I'm not sure what the solution would be if complete accuracy was required.

There's now a Live Maps web service that seems to have a very similar API to the MapPooint web service. I've not looked at this too closely but I'd be interested to know if it has a better algorithm for routing so these kind of errors are reduced. I'd also be interested to know if people have found different solutions to this error message.

How we got into this mess

I read 'Liar's Poker' a few years ago and it was a fascinating insight into the people who work on Wall Street (or did during the 80s). The author of that book has written an article chronicling the insanity that has led us to the financial turmoil we find ourselves in. Most of it will be familiar to many people, mortgages being sold to people who had no hope of repaying them, this toxic debt being packaged up to look like low risk AAA rated bonds and the fallout when this debt started turning bad.

the article mostly talks about Steve Eisman, who seems to be one of the financiers to see this trouble coming and started shorting the market. Ironically this actually made things worse...

But when Eisman bought a credit-default swap, he enabled Deutsche Bank to create another bond identical in every respect but one to the original. The only difference was that there was no actual homebuyer or borrower.

Which may help to explain why things have got so bad. Not only have a lots of loans gone bad, but lots of fantasy loans that never actually existed have gone bad as well.

Creating images on the fly in ASP.NET

Creating images on the fly in ASP.NET is pretty simple. Here is some sample code for a generic handler that can be used as a starting point for your own implementation.

<%@ WebHandler Language="C#" Class="ImageExample" %>

using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.Web;

public class ImageExample : IHttpHandler
{
  public void ProcessRequest (HttpContext context)
  {
    using (Bitmap bitmap = new Bitmap(300, 200))
    using (Graphics graphics = Graphics.FromImage(bitmap))
    using (SolidBrush brush = new SolidBrush(Color.Red))
    {
      graphics.FillRectangle(brush, 0, 0, 300, 200);
      // TODO - draw your image

      context.Response.ContentType = "image/jpeg";
      bitmap.Save(context.Response.OutputStream, ImageFormat.Jpeg);
    }
  }

  public bool IsReusable
  {
    get
    {
      return false;
    }
  }
}

How to recover the hits to your website

I discussed a month or two ago about how the Random Pub Finder database died and our visitors disappeared. So how to recover from that? It turns out we didn't need to do anything, Google (and presumably the other search engines, but really who cares about them?) re-indexed our pages and presumably ranked them as highly as before because October was our best month yet for visitors, finally over the 10,000 mark.

Custom activities in Windows Workflow 4.0

Another presentation from PDC, http://channel9.msdn.com/pdc2008/TL21/

Looks like authoring custom activities has got a whole lot easier and more powerful. Designer rehosting is much simpler. Looks like I might have to finally learn some WPF though.

Windows Workflow 4 at PDC

See the intro to WF 4.0 at http://channel9.msdn.com/pdc2008/TL17/. Watching at the moment and looks pretty cool. Fully declarative workflows, better designer, store workflow data in a proper queryable manner in the database.

Metastorm BPM 7.6 and Windows Workflow part 6 - state machines

This will almost certainly be my last post about executing Windows Workflows from Metastorm BPM as I think I've covered pretty much everything there is to cover (although leave a comment if there is something I've missed).

One of the comments from one of the previous posts was regarding state machine workflows and whether these could be executed from the Metastorm environment. So I knocked together a simple state machine workflow in Visual Studio that starts, waits in its initial state for 5 minutes then moves to the final state.

As discussed before, there is no support for publishing your own VS authored workflows to the Metastorm database so I modified the little command-line tool I wrote previously to cope with state machine workflows. For brevity I won't post all the source, just the change required, which is as follows.

        for (int i=0; i<types.Length; i++)
        {
          if (types[i].IsSubclassOf(typeof(SequentialWorkflowActivity)))
          {
            workflowType = types[i];
            break;
          }
          else if (types[i].IsSubclassOf(typeof(StateMachineWorkflowActivity)))
          {
            workflowType = types[i];
            break;
          }
        }

So after publishing the workflow, I tried to execute it from Metastorm and it worked as expected. Cool.

The next test was to see what happened when I restarted the engine whilst a workflow was executing, since this caused problems with sequential workflows. I was expecting the same problem, since state machine workflows execute in the same way as sequential workflows. And I was correct in my supposition, the workflow never completed.

So in conclusion, state machines can be used in Metastorm but you'll need to hack the database. But WF in Metastorm won't be of use to you if you need to execute long running workflows. Which is a shame since this is probably the main purpose of WF.

Part 1 - The basics

Part 2 - The database tables

Part 3 - Using Visual Studio

Part 4 - Using your own activities

Part 5 - Long running workflows

Oracle on Windows - the most painful application ever

I'd been putting it off for as long as possible but I finally decided it was time for me to get the FreeFlow Administrator supporting Oracle as well SQL Server. Why did I put it off for so long? Because Oracle on Windows is horrible. I've never used it on Unix but I  assume it must be better than the Windows version, or how have Oracle survived so long? 

SQL Server just works, install it and you've got the database, management tools and drivers installed no problem. Try the same with Oracle and the first problem you've got is that the installer is some weird proprietary Java thing that doesn't behave like a normal Windows installer. Then you have to try and install ODBC and OLE DB drivers. My first installation attempt managed to destroy my database, the second time, after many Google searches, actually worked.

Another nice feature of the Oracle drivers for Windows is they seem to provide new versions every week that fix some bugs and introduce some other bugs, which all adds to the excitement. And how about this, an OLE DB driver is a 200MB download, WTF?

To be fair, some things have improved since my last foray into Oracle development. They now have a free version of their database, Oracle XE, which actually uses a proper MSI installer.  They have a new admin tool, SQL Developer, which does make life easier. Of course it's Java based so the UI is dreadful and there doesn't seem to be a way to create a table space which seems like the first thing you'd want to do but it is a step in the right direction.

So what's the point of this post? None really, except I finally got it all working and I'm feeling quite pleased with myself. Perhaps that's why people use Oracle, because they feel like they've actually achieved something after just installing the damn thing.

Do you really need to use a CAPTCHA?

I am completely lazy so I've put up with the spam coming to me from the Random Pub Finder for a long while now. Eventually I realised the signal to noise ratio was so low that I was actually missing real people submitting pub suggestions so I decided I had to do something. Again due to my laziness my first attempt at fixing the problem was just to block submissions from the common IP addresses used by the bots trying to submit things. This helped but still stuff was getting through.

So I started to think about implementing a CAPTCHA system, but even though there seems to be at least one good PHP implementation out there, laziness again got the better of me. I remembered Coding Horror which has used a sort of CAPTCHA system for several years but the difference is that the image is always the same. And has been as long as I remember. So I must assume that it is reasonably effective, given it's a site that gets a lot of hits, so I thought I'd try out the same system on the Random Pub Finder, since it's dead simple to implement.

I've only just implemented it and it's working 100% so far. If it continues to work, then it suggests all these spammy form submissions are completed automated and there is no human behind them at all. Which then leads to the conclusion that a full-blown CAPTCHA is probably not necessary for most sites.

Recursively copy a folder in C#

Perhaps I need to RTFM but I couldn't find any methods in the .NET Framework to copy a folder and its contents to another location. So this is my attempt. I haven't thoroughly tested it but it should be a reasonable starting point and worked for my single test case.

    private void CopyFolder(string sourceFolder, string outputFolder)
    {
      System.IO.Directory.CreateDirectory(outputFolder);

      string[] files = Directory.GetFiles(sourceFolder);
      foreach (string file in files)
      {
        File.Copy(file, Path.Combine(outputFolder, Path.GetFileName(file)), true);
      }

      string[] folders = Directory.GetDirectories(sourceFolder);
      foreach (string folder in folders)
      {
        string[] splitFolders = folder.Split('\\');
        string folderName = splitFolders[splitFolders.Length - 1];
        CopyFolder(folder, Path.Combine(outputFolder, folderName)); 
      }
    }

Finding table and column references in SQL Server stored procedures

I keep needing to go off and search for this, so as a note to myself, here's the required SQL to find out where tables and columns are referenced in SQL Server stored procedures.

select * from information_schema.routines where routine_definition like '%tablename%'

I believe this only searches the first 4000 characters of the stored procedure, but this has generally been enough for my needs.

The risk premium

It looks like several hundred thousand people in the UK have discovered the meaning of the risk premium today. Whilst it's difficult not to have sympathy for all these people who have potentially lost money, it may help everybody else realise that no place is a completely safe haven for your savings and not all banks are made equal. So how to tell if a bank could be in difficulties? Well, firstly look at the rates they are prepared to pay to get hold of your savings. Banks don't offer higher rates because they are generous people who want to make you happy, it is fairly likely they are after your money because they need it because they are short of capital, which suggests they are more prone to fail. And that is the risk premium, you get a better interest rate because you're more likely to lose your money.

Another thing to look at is CDS spreads. Unfortunately these are difficult to get hold of unless you happen to have access to a Bloomberg terminal. Here's a list from about three weeks ago, a higher number means the market considers the bank to be higher risk. And as you can see, quite a few of the banks near the top of the list have already got into difficulty. It was this list (along with the failure of Glitnir) that made me pull my money out of Icesave (Landsbanski) a week ago. Phew.

Kazkommerts 1040
Glitnir Bank 950
Kaupthing 850
IKB 625
Wachovia Corp 572
VTB Bank 570
Landsbanki 550
Anglo Irish 485
Banca Italease 462
Capital One Bank 424
ICICI 405
HBOS 327
Bank of Ireland 310
Allied Irish Banks 268
Citigroup Inc 264
Nationwide 230
Bank of America 201
Standard Chartered Bank 199
Natixis 192
JPMorgan Chase 191
Wells Fargo 188
UBS 183
Barclays 169
Erste 166.7
RBS 162
Raiffeisen 150.0
Fortis Bank 149
Danske Bank 138
ING 133
Credit Agricole 127
Credit Suisse 124
Lloyds 123
Societe Gen 122
ABN Amro 119
Dresdner 118
Santander 117
Deutsche 114
Commerzbank 111
Standard Life Bank Ltd 108
Unicredit 95.2
HSBC 94
Nordea Bank 91
BNP 73
Svenska Handelsbanken 72.3

Stuart's Stories

I reviewed Stuart Maconie's book about the North some time ago and now Visit England's North West are offering a free book called "Stuart's Stories", which contains some of his writings about the North West. I'm not sure if they are straight from the book or if this is new content, but either way it can't be bad for free can it?