Content is still king

One of the phrases that was coined during the early years of the web was 'content is king'. The basic idea was that content drives traffic to your site. This is clearly correct for the simple case - if you have no content at all you'll get no visitors and when you add some content you'll hopefully get more than zero visitors.

But that was in the days of not so clever search engines. These days search engines don't just take notice of the content on your site, they look at all kinds of other things so there's much more emphasis on using search engine optimisation to optimise content to get as many hits as possible. I don't think there's anything wrong with using SEO and fiddling around with keywords, layouts, titles, URLs and reciprocal linking can certainly help (although it's quite possible to get it wrong as well and lose traffic) but it seems to miss the point. The absolute simplest way to increase traffic to your site is to add more content.

To illustrate, lets take a look at some real data. I've used archive.org and data from Google Analytics to produce this graph for the Random Pub Finder.

What does this show? I've taken the number of monthly visitors to the site and divided them by the number of pubs reviews on the website. This doesn't give the complete picture of the amount of content on site but I'm not sure how I can work out the number of unique URLs historically. Note, using page views instead of visitors produces a pretty similar graph.

So does it prove anything? I guess if content was the only thing affecting the number of visitors then we'd expect the trend line to be horizontal so there are clearly other factors at work. We've got more inbound links than we used to have (that big spike is when we got linked from another site). I also think age of content helps search engine rankings, although I don't have any proof for this theory. But given that I've done very little SEO work on the site, it does suggest increasing content increases visitors above and beyond a direct linear increase.

As a comparison, looking at the number of visitors to www.doogal.co.uk, where the amount of content has not increased very much, shows visitor numbers haven't increased as dramatically.

To look at it another way, every new page you add to your site will add X new visitors to your site, where X can be anything from zero to a very large number. Adding a new page will probably increase the number of visitors to your site, however small.

So in conclusion, don't waste your time and money on snake oil SEO 'experts', just add some more content to your site if you want to get more visitors.

Windows Search now slows down Internet Explorer as well

One of the things that caused bad initial experiences of Windows Vista was Windows Search. As soon as you'd installed Vista, off it went indexing files on your hard disk. This could take some time and although it runs as a background task it still seems to impact the performance of your PC. Lots of people just turned off the service, although if they'd left it on for a while they'd have discovered that once the initial scan had been finished, it generally had a pretty low impact on system performance.

I left it on but I've never been overly happy with it. Fact is I don't do much searching on my PC and when I do I'm not too bothered if it takes a while. So having a constant tax on my performance (however small) seems a waste. Not only that, but it doesn't seem to index the files that are important to me. So when I do do a search I invariably have to switch to the 'search everything' mode which is really slow meaning the whole indexing palaver is completely wasted. I guess I should have set it up to index everything but never bothered, thinking it would cause the indexer to use up even more processor time.

But along comes IE8 and Microsoft have decided to use Windows Search technology to power the drop down list of suggested sites. Previously I'd used this a lot, type the first few letters of a site that I visit frequently and it would appear in the drop down list and I could select it rather than having to type the full URL. In theory that still works, but Windows Search always takes several seconds to decide what to show in the list, so it's as quick just to type in the full URL. I guess the list of suggested sites is better but the delay means it's completely useless. And typing another letter causes the search to start again from scratch...

But there is a solution to this. Disable the Windows Search service and the drop down works just like it used to do and I'm happy again. I guess this is a classic unintended consequence of powering IE8 with Windows Search, that even more people disable it.

Revenue models for free software

I have no ideological belief that software should be free, but it certainly seems to be an increasingly prevalent model for software.  Since I don't run my own software company the code I produce doesn't have pay the bills. But although I enjoy writing it, in an ideal world I'd still like to make some kind of income from my work. Probably my biggest piece of code that I've given away has been the FreeFlow library and Administrator, that can be used with Metastorm's BPM product. I've tried out a few different revenue generation techniques with varying degrees of success.

Sell the source - This was the initial model, let anyone download the binaries but charge for the source. It worked reasonably well for the FreeFlow library, which is used by developers to talk to the Metastorm engine. I guess developers are keen to get hold of the source for libraries they are using, to modify for their own needs and reduce the risk if the developer disappears. Unfortunately Metastorm introduced their own .NET library and as time has gone on there's been a lot more interest in the standalone FreeFlow Administrator application. With this, I suspect less people care about the code since if I disappear it won't actually affect their own applications, so sales of the source code have dropped off.

Website advertising - There's been advertising on the FreeFlow website for a long while and I've made a bit of spending money from it, but that's all. One positive of writing software for a BPM product is that most of the ads are for competing BPM products which seem to earn a good amount of money. The downside is that Metastorm is just one of many BPM companies and BPM is still a fairly niche area so I don't get a massive number of visitors to the site.

Donations - This is my latest attempt at earning a bit of cash. The website and the FreeFlow Administrator now have 'Donation' buttons. So far I've not earned anything from this at all. I'm not hugely surprised. If I'm using a free piece of software I very rarely donate any money. If it's shareware, I'm unlikely to pay for the full version even if it has nag screens (WinZip anyone?). The exception to this is SmartFTP, which I paid for because it needed re-installing every few months and it's actually a nice bit of software.

Ads in the application - I haven't tried this yet and I'm not even sure how ads can be added to a desktop application but it seems to be the path FeedDemon is going down. I may wait and see how well it goes for that software and my own donations before going down the same route. One of the nice advantages of not needing to earn any money from my software is I can try out different models for as long as I want before trying out something else.

A buy out - There has been interest from people wishing to buy FreeFlow off me but there have been no concrete offers as yet. I'd definitely consider a serious offer. Even though I have some emotional attachment to the code I've written I'm not dumb enough to refuse a reasonable amount of money if somebody offered it. Unfortunately as this is a niche product, I can't really see there being a huge number of suitors... 

In conclusion, I think if I was starting from scratch I'd probably try writing software with a potentially wider audience. Something like TimeSnapper, which is of use to almost anybody. And I'd probably try a similar pricing model to their software, a basic free version and a more feature-rich professional version. Or an ad-sponsored free version and a paid-for non-ads version. Now I just have to figure out what that software would be...

Free controls for .NET

I've come across a couple of free controls for .NET recently which look pretty cool.

DevExpress - I've always loved DevExpress. Their Delphi controls rocked (and probably still do), their .NET controls rock and their pricing has always been pretty reasonable ($2000* for every .NET control they produce for a year). Now you can get some of them for free. The download is a bit large since it includes trial versions of everything they do but some of them look pretty useful. I haven't looked too closely but from a cursory look it seems like the ASP.NET controls are more useful than the WinForms controls, which are mostly just some edit controls.

Chart control - Microsoft bought the Dundas chart controls and now you can use the Microsoft updated versions in your ASP.NET and WinForms applications for free. I assume at some point these will get rolled into the .NET Framework meaning it will be possible to add charting to apps with no extra installation woes to worry about. It will be interesting to see what now happens to Dundas (and ChartFX and any other chart control companies for that matter). I guess they have products for other platforms and more specialised charting controls that will keep them in business. If not, they may live to regret the decision to sell their code...

* Actually $1999.99 but it's quite close to $2000 - does the .99 cent/pence ruse actually still work?

The predefined type 'System.Runtime.CompilerServices.ExtensionAttribute' is defined in multiple assemblies in the global alias

Extension methods are cool and they can be used from .NET 2.0. I'm not the first to suss this out but the trick is to add the following to your code.

namespace System.Runtime.CompilerServices
{
  internal sealed class ExtensionAttribute : Attribute { }
}

This works well but there is a gotcha. Say I have two assemblies and I've added the definition to both assemblies because I want to add extension methods to both, I get the warning message

The predefined type 'System.Runtime.CompilerServices.ExtensionAttribute' is defined in multiple assemblies in the global alias

I try to produce code that doesn't generate any warnings since it's easy to miss real warnings if there are these kind of false positives. One solution is to make the definition public in one of the assemblies but I wasn't keen on that idea because the assembly is a publicly available class library whose public API I didn't want to pollute with this hack.

So the final solution was to use the InternalVisibleTo attribute in one of the assemblies. This means ExtensionAttribute only needs to be defined in one assembly. There's more info here on how to do that with strongly named assemblies since it's not completely straight forward.

One final problem is that using the assembly in a .NET 3.5 application will show the warning again since ExtensionAttribute is already defined in the .NET Framework. As far as I'm aware there isn't a solution for this problem, except producing separate .NET 2 and .NET 3.5 versions of the assembly.

The RPF book has a cover

Thanks to my other half, the Random Pub Finder book now has a proper cover. Unfortunately, in the mean time, Lulu has changed their publishing prices which means the cost for an A5 book is now much more than their standard sizes. Which, in these credit crunch times, may be putting off some purchasers. So now I'm busily repaginating it to fit into a US Trade size, whatever that is. Whatever it is, it's much cheaper to publish. So you still may want to hold off purchasing it for a bit, if you were planning on doing.

Sales so far - 4. Ho hum...

"One or more waypoints are not routable in the specified data source" error in MapPoint

I've searched on the internet for any information about this error message returned by the MapPoint web service but haven't been able to find anything, so this is a rundown on what I've found, although this is fairly obvious stuff.

Say you're trying to route between points A, B, C, D and E. Basically the error is saying that MapPoint was unable to route between at least two of these points. So the first thing to do is figure out which points MapPoint didn't like. this can be achieved by breaking the route down into its constituent parts and try to route A to B, B to C, C to D and D to E. The chances are that at least one of these will fail.

Now it's time to take a closer look at the two points that failed. I think the best way to do this is to visualise the route using Google Maps and/or Live Maps. For me, both sites successfully routed between the two points but Google Maps showed something strange. The suggested route involved driving to a T junction, turning right, driving to the end of the road and making a  U turn and driving back along it. Moving the point to the end of the road with the T junction fixed the MapPoint error. For me, being able to route between the points was more important than being completely accurate. I'm not sure what the solution would be if complete accuracy was required.

There's now a Live Maps web service that seems to have a very similar API to the MapPooint web service. I've not looked at this too closely but I'd be interested to know if it has a better algorithm for routing so these kind of errors are reduced. I'd also be interested to know if people have found different solutions to this error message.

How we got into this mess

I read 'Liar's Poker' a few years ago and it was a fascinating insight into the people who work on Wall Street (or did during the 80s). The author of that book has written an article chronicling the insanity that has led us to the financial turmoil we find ourselves in. Most of it will be familiar to many people, mortgages being sold to people who had no hope of repaying them, this toxic debt being packaged up to look like low risk AAA rated bonds and the fallout when this debt started turning bad.

the article mostly talks about Steve Eisman, who seems to be one of the financiers to see this trouble coming and started shorting the market. Ironically this actually made things worse...

But when Eisman bought a credit-default swap, he enabled Deutsche Bank to create another bond identical in every respect but one to the original. The only difference was that there was no actual homebuyer or borrower.

Which may help to explain why things have got so bad. Not only have a lots of loans gone bad, but lots of fantasy loans that never actually existed have gone bad as well.

Creating images on the fly in ASP.NET

Creating images on the fly in ASP.NET is pretty simple. Here is some sample code for a generic handler that can be used as a starting point for your own implementation.

<%@ WebHandler Language="C#" Class="ImageExample" %>

using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.Web;

public class ImageExample : IHttpHandler
{
  public void ProcessRequest (HttpContext context)
  {
    using (Bitmap bitmap = new Bitmap(300, 200))
    using (Graphics graphics = Graphics.FromImage(bitmap))
    using (SolidBrush brush = new SolidBrush(Color.Red))
    {
      graphics.FillRectangle(brush, 0, 0, 300, 200);
      // TODO - draw your image

      context.Response.ContentType = "image/jpeg";
      bitmap.Save(context.Response.OutputStream, ImageFormat.Jpeg);
    }
  }

  public bool IsReusable
  {
    get
    {
      return false;
    }
  }
}

How to recover the hits to your website

I discussed a month or two ago about how the Random Pub Finder database died and our visitors disappeared. So how to recover from that? It turns out we didn't need to do anything, Google (and presumably the other search engines, but really who cares about them?) re-indexed our pages and presumably ranked them as highly as before because October was our best month yet for visitors, finally over the 10,000 mark.

Custom activities in Windows Workflow 4.0

Another presentation from PDC, http://channel9.msdn.com/pdc2008/TL21/

Looks like authoring custom activities has got a whole lot easier and more powerful. Designer rehosting is much simpler. Looks like I might have to finally learn some WPF though.

Windows Workflow 4 at PDC

See the intro to WF 4.0 at http://channel9.msdn.com/pdc2008/TL17/. Watching at the moment and looks pretty cool. Fully declarative workflows, better designer, store workflow data in a proper queryable manner in the database.

Metastorm BPM 7.6 and Windows Workflow part 6 - state machines

This will almost certainly be my last post about executing Windows Workflows from Metastorm BPM as I think I've covered pretty much everything there is to cover (although leave a comment if there is something I've missed).

One of the comments from one of the previous posts was regarding state machine workflows and whether these could be executed from the Metastorm environment. So I knocked together a simple state machine workflow in Visual Studio that starts, waits in its initial state for 5 minutes then moves to the final state.

As discussed before, there is no support for publishing your own VS authored workflows to the Metastorm database so I modified the little command-line tool I wrote previously to cope with state machine workflows. For brevity I won't post all the source, just the change required, which is as follows.

        for (int i=0; i<types.Length; i++)
        {
          if (types[i].IsSubclassOf(typeof(SequentialWorkflowActivity)))
          {
            workflowType = types[i];
            break;
          }
          else if (types[i].IsSubclassOf(typeof(StateMachineWorkflowActivity)))
          {
            workflowType = types[i];
            break;
          }
        }

So after publishing the workflow, I tried to execute it from Metastorm and it worked as expected. Cool.

The next test was to see what happened when I restarted the engine whilst a workflow was executing, since this caused problems with sequential workflows. I was expecting the same problem, since state machine workflows execute in the same way as sequential workflows. And I was correct in my supposition, the workflow never completed.

So in conclusion, state machines can be used in Metastorm but you'll need to hack the database. But WF in Metastorm won't be of use to you if you need to execute long running workflows. Which is a shame since this is probably the main purpose of WF.

Part 1 - The basics

Part 2 - The database tables

Part 3 - Using Visual Studio

Part 4 - Using your own activities

Part 5 - Long running workflows

Oracle on Windows - the most painful application ever

I'd been putting it off for as long as possible but I finally decided it was time for me to get the FreeFlow Administrator supporting Oracle as well SQL Server. Why did I put it off for so long? Because Oracle on Windows is horrible. I've never used it on Unix but I  assume it must be better than the Windows version, or how have Oracle survived so long? 

SQL Server just works, install it and you've got the database, management tools and drivers installed no problem. Try the same with Oracle and the first problem you've got is that the installer is some weird proprietary Java thing that doesn't behave like a normal Windows installer. Then you have to try and install ODBC and OLE DB drivers. My first installation attempt managed to destroy my database, the second time, after many Google searches, actually worked.

Another nice feature of the Oracle drivers for Windows is they seem to provide new versions every week that fix some bugs and introduce some other bugs, which all adds to the excitement. And how about this, an OLE DB driver is a 200MB download, WTF?

To be fair, some things have improved since my last foray into Oracle development. They now have a free version of their database, Oracle XE, which actually uses a proper MSI installer.  They have a new admin tool, SQL Developer, which does make life easier. Of course it's Java based so the UI is dreadful and there doesn't seem to be a way to create a table space which seems like the first thing you'd want to do but it is a step in the right direction.

So what's the point of this post? None really, except I finally got it all working and I'm feeling quite pleased with myself. Perhaps that's why people use Oracle, because they feel like they've actually achieved something after just installing the damn thing.

Do you really need to use a CAPTCHA?

I am completely lazy so I've put up with the spam coming to me from the Random Pub Finder for a long while now. Eventually I realised the signal to noise ratio was so low that I was actually missing real people submitting pub suggestions so I decided I had to do something. Again due to my laziness my first attempt at fixing the problem was just to block submissions from the common IP addresses used by the bots trying to submit things. This helped but still stuff was getting through.

So I started to think about implementing a CAPTCHA system, but even though there seems to be at least one good PHP implementation out there, laziness again got the better of me. I remembered Coding Horror which has used a sort of CAPTCHA system for several years but the difference is that the image is always the same. And has been as long as I remember. So I must assume that it is reasonably effective, given it's a site that gets a lot of hits, so I thought I'd try out the same system on the Random Pub Finder, since it's dead simple to implement.

I've only just implemented it and it's working 100% so far. If it continues to work, then it suggests all these spammy form submissions are completed automated and there is no human behind them at all. Which then leads to the conclusion that a full-blown CAPTCHA is probably not necessary for most sites.

Recursively copy a folder in C#

Perhaps I need to RTFM but I couldn't find any methods in the .NET Framework to copy a folder and its contents to another location. So this is my attempt. I haven't thoroughly tested it but it should be a reasonable starting point and worked for my single test case.

    private void CopyFolder(string sourceFolder, string outputFolder)
    {
      System.IO.Directory.CreateDirectory(outputFolder);

      string[] files = Directory.GetFiles(sourceFolder);
      foreach (string file in files)
      {
        File.Copy(file, Path.Combine(outputFolder, Path.GetFileName(file)), true);
      }

      string[] folders = Directory.GetDirectories(sourceFolder);
      foreach (string folder in folders)
      {
        string[] splitFolders = folder.Split('\\');
        string folderName = splitFolders[splitFolders.Length - 1];
        CopyFolder(folder, Path.Combine(outputFolder, folderName)); 
      }
    }

Finding table and column references in SQL Server stored procedures

I keep needing to go off and search for this, so as a note to myself, here's the required SQL to find out where tables and columns are referenced in SQL Server stored procedures.

select * from information_schema.routines where routine_definition like '%tablename%'

I believe this only searches the first 4000 characters of the stored procedure, but this has generally been enough for my needs.

The risk premium

It looks like several hundred thousand people in the UK have discovered the meaning of the risk premium today. Whilst it's difficult not to have sympathy for all these people who have potentially lost money, it may help everybody else realise that no place is a completely safe haven for your savings and not all banks are made equal. So how to tell if a bank could be in difficulties? Well, firstly look at the rates they are prepared to pay to get hold of your savings. Banks don't offer higher rates because they are generous people who want to make you happy, it is fairly likely they are after your money because they need it because they are short of capital, which suggests they are more prone to fail. And that is the risk premium, you get a better interest rate because you're more likely to lose your money.

Another thing to look at is CDS spreads. Unfortunately these are difficult to get hold of unless you happen to have access to a Bloomberg terminal. Here's a list from about three weeks ago, a higher number means the market considers the bank to be higher risk. And as you can see, quite a few of the banks near the top of the list have already got into difficulty. It was this list (along with the failure of Glitnir) that made me pull my money out of Icesave (Landsbanski) a week ago. Phew.

Kazkommerts 1040
Glitnir Bank 950
Kaupthing 850
IKB 625
Wachovia Corp 572
VTB Bank 570
Landsbanki 550
Anglo Irish 485
Banca Italease 462
Capital One Bank 424
ICICI 405
HBOS 327
Bank of Ireland 310
Allied Irish Banks 268
Citigroup Inc 264
Nationwide 230
Bank of America 201
Standard Chartered Bank 199
Natixis 192
JPMorgan Chase 191
Wells Fargo 188
UBS 183
Barclays 169
Erste 166.7
RBS 162
Raiffeisen 150.0
Fortis Bank 149
Danske Bank 138
ING 133
Credit Agricole 127
Credit Suisse 124
Lloyds 123
Societe Gen 122
ABN Amro 119
Dresdner 118
Santander 117
Deutsche 114
Commerzbank 111
Standard Life Bank Ltd 108
Unicredit 95.2
HSBC 94
Nordea Bank 91
BNP 73
Svenska Handelsbanken 72.3

Stuart's Stories

I reviewed Stuart Maconie's book about the North some time ago and now Visit England's North West are offering a free book called "Stuart's Stories", which contains some of his writings about the North West. I'm not sure if they are straight from the book or if this is new content, but either way it can't be bad for free can it?

Metastorm BPM 7.6 and Windows Workflow part 5 - Long running workflows

OK, I want to go back to basics for this post. What is the point of Windows Workflow? Here's the first paragraph of the preface to "Essential Windows Workflow Foundation"

"Windows Workflow Foundation is a general-purpose programming framework for creating reactive programs that act in response to stimulus from external entities. The basic characteristics of reactive programs is that they pause during their execution, for unknown amounts of time, awaiting input."

I've highlighted what I consider to be the important point here. A WF workflow may lay dormant for minutes, days or weeks. In the normal world of .NET code, implementing this kind of application can be troublesome. For a start, the application may shutdown and restart before the workflow resumes, so the application needs to cope with this possibility. This means the objects must be persisted and restarted with the correct state. A simple solution like making the executing thread sleep for the required amount of time isn't going to work, in terms of robustness or scalability (imagine 1000s of these workflows being executed at the same time).

So WF helps solves this problem. Without this feature, WF is really just a different approach to producing standard .NET code, either through the designer or generating declarative XML documents to explain how the code should execute. It might well be useful in some scenarios but it isn't really a killer reason to use the technology.

So with that in mind I created a very simple workflow to execute from Metastorm BPM, which you can see on the left. All it does is write to a text file, wait for ten minutes, then write to the text file again.

The first thing to ensure is to set the "Microsoft Workflow timeout (secs)" registry setting to something sensible. Unfortunately setting this to zero doesn't mean no timeout, it just uses the default 120 seconds timeout. So this needs setting to something very large. This is a DWORD value with a maximum value of 4294967295, which I calculate is approximately 49710 days, that hopefully is large enough for most workflows.

Executing the workflow worked fine in this scenario. The workflow idled for 10 minutes then resumed and completed.

So the next experiment was to start the workflow, shut down the engine and then restart the engine and see what happened. I waited... and waited... Nothing. I started another workflow just in case the workflow runtime hadn't been kicked off but still nothing. So there's a bit of a problem there, particularly if you want to run long running workflows.

As an aside, if you're running your workflow asynchronously then you won't be able to use most of the activities provided for interaction with Metastorm BPM. This is a shame. In particular the RaiseFlag activity would be exceedingly useful since it could be used to signal to the parent process that the workflow had completed. As it is, you'll need to do this with the ECL activities provided, which are more complicated to configure or write your own code.

Part 1 - The basics

Part 2 - The database tables

Part 3 - Using Visual Studio

Part 4 - Using your own activities

Part 6 - State machines

Autosizing row heights in a WinForms DataGrid

Autosizing column widths in a WinForms DataGrid is pretty easy, if a little hacky. Although it seems that a similar technique could be used to autosize row heights, the private RowAutoResize method doesn't do what I was expecting it to do. I found some useful code here which I have adapted somewhat so that it can handle DataGrids with IList based classes as their data source. It also looks at all cells to work out the required height.

    public void AutosizeRows()
    {
      int numRows = 0;
      if (DataSource is DataTable)
      {
        numRows = ((DataTable)DataSource).Rows.Count;
      }
      else if (DataSource is IList)
      {
        CurrencyManager manager = (CurrencyManager)BindingContext[DataSource];
        numRows = manager.List.Count;
      }

      using (Graphics g = Graphics.FromHwnd(Handle))
      {
        StringFormat sf = new StringFormat(StringFormat.GenericTypographic);

        // Since DataGridRows[] is not exposed directly by the DataGrid  
        // we use reflection to hack internally to it..
        MethodInfo mi = GetType().GetMethod("get_DataGridRows",
          BindingFlags.Instance | BindingFlags.NonPublic);

        Array dgra = (Array)mi.Invoke(this, null);

        // Convert this to an ArrayList, little bit easier to deal with 
        // that way, plus we can strip out the newrow row. 
        List<object> DataGridRows = new List<object>();
        foreach (object dgrr in dgra)
        {
          if (dgrr.ToString().EndsWith("DataGridRelationshipRow"))
            DataGridRows.Add(dgrr);
        }

        int colCount = TableStyles[0].GridColumnStyles.Count;

        // Now loop through all the rows in the grid 
        for (int i = 0; i < numRows; ++i)
        {
          int maxHeight = 0;

          for (int j = 0; j < colCount; j++)
          {
            SizeF size = g.MeasureString(this[i, j].ToString(), Font, 400, sf);
            int h = Convert.ToInt32(size.Height);

            // Little extra cellpadding space 
            h = h + 8;

            maxHeight = Math.Max(h, maxHeight);
          }

          // Now we pick that row out of the DataGridRows[] Array  
          // that we have and set it's Height property to what we 
          // think it should be. 
          PropertyInfo pi = DataGridRows[i].GetType().GetProperty("Height");
          pi.SetValue(DataGridRows[i], maxHeight, null);
        }
      }
    }

Capital boozing - the Random Pub Finder book

It used to be called vanity publishing. An aspiring author, having been rejected by all the standard publishers, would decide the problem wasn't with themselves and their work, but with the publishers who were clearly blind to the author's genius. So the only solution was to go out and get their work published themselves and sell hundreds of thousands of copies that way. Of course what generally happened was that, after paying out thousands to the vanity publisher, the author would be left with 2472 copies of their masterpiece sat in the loft, after giving out the other 28 copies to friends and family.

Things have changed, we can now all be published authors via the wonderful lulu.com. The beauty of this system is that a forest doesn't get destroyed to sate the author's vanity. Books are only produced if somebody is actually interested in them. And because they are published on demand, books can be updated and corrected quickly and easily.

So to take advantage of this system, we've taken the reviews from the Random Pub Finder and put them all together in one convenient book shaped package. I don't know about you but even with all the latest technologies for reading on a screen, I still much prefer a book. No power problems, no waiting to boot up, no random crashes.

Anyway, if you want to buy a copy, hop over to http://www.lulu.com/content/2003276. I'm still waiting for a design for the cover, but other than that it's all done.

Turn it up to 11

I've just noticed the volume on the BBC media player on their website goes up to 11. I assume the developer of this code is an afficionado of Spinal Tap.

I can't say I've noticed the difference in sound levels between this and other web based apps...

Metastorm BPM 7.6 and Windows Workflow part 4 - Using your own activities

My previous posts on Metastorm/WF integration have had a few comments, so I thought I'd follow up on them.

First up, it is possible to reference your own assemblies containing custom activities in the WF Composer, via the Options dialog that can be found on the main menu. For me this means the Composer could potentially be used by non-technical people, although the workflow designer may still be too difficult to hand over to non-techies (this isn't Metastorm's fault BTW, the design surface is all Microsoft's work).

But before I could check out this feature, I had a problem using the Composer. I haven't used it for a few weeks or months and when I tried to do almost anything I got the following error

Could not load file or assembly 'mscorcfg, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified. (mscorlib) 

I suspect this is down to me uninstalling Visual Studio 2005 since I installed the WF Composer which I guess uninstalled the .NET 2.0 SDK which is required by the WF Composer. It's a shame the SDK is required, as it's a 354MB download... If it's just for the one assembly then it's flipping ridiculous.

Anyway, after installing the SDK and reinstalling the Metastorm WF bits, things started working again (the re-install may not be necessary but I uninstalled before I realised what my problem was).

OK, so I thought I'd try to use some of my own activities. My first problem was that any assembly I tried to reference produced the following error

No toolbox items found in assembly!

This was easily solved by adding the following attribute to my activity class definition

[ToolboxItem(true)]

My activities now appear in the toolbox but I'm unable to drag them onto the design surface. I guess I'm missing something in my activity class code but I don't know what. They work in Visual Studio and my own designer so I'm not sure why they fail here. Anyway, although the WF Composer is of passing interest, my main interest is using workflows authored in Visual Studio, so I'm not going to look at it too closely.

Part 1 - The basics

Part 2 - The database tables

Part 3 - Using Visual Studio

Part 5 - Long running workflows

Part 6 - State machines

Complete rewrite - two years on

It's almost two years on since I wrote about one of my previous employers starting a complete rewrite of one of the core pieces of the their application suite. So how have things turned out? To tell you the truth I have no idea, nobody from the company will give me any useful information these days. But one thing that I am sure of, the new software hasn't been released yet. So my back of a fag packet estimate of three and half years development time isn't looking quite as daft as it may have done at the time.

Having said that, it is of course possible that the software is done and the company are just waiting for the right time to release it. I remember being blown away when I discovered that Ultimate Play The Game actually held back the release of the ground-breaking Knight Lore game so they could release the less impressive Sabre Wulf without it looking rubbish.

And if you do know anything about the progress of the software of which I'm talking then post a comment, you can do it anonymously...

Noel Gallagher pushed off stage

I have a theory why the North West has produced more than its fair share of great popular music over the past few years (The Beatles, Joy Division, New Order, The Smiths, Happy Mondays, British Sea Power (although Brighton try to claim them as their own), Elbow, The Fall, The Stone Roses to name but a few). It's the weather. Rain, rain and more rain. People are forced to stay in quite a lot and do something. So a lot of them decide to try their hand at music and some of them succeed. But I would never include Oasis on that list. Although their initial material was catchy, it never really reached the emotional heights that great pop music does. And their output afterwards has been absolutely awful. To be fair, "Wonderwall" could have been a beautiful song had it not been for the vocals of Liam.

Anyway, I'd pretty much forgotten about Oasis, but apparently they are still chugging along with their Beatles parodies. And now we have this wonderful video of Noel being pushed off stage by somebody. Although that's slightly humorous, the bit I love is self-styled tough guy Liam. First he tries to get away from the stage invader, then when the guy has been held down by two or three security guys, he suddenly remembers how hard he really is and walks across to "have a go" at him. I think it really shows how much of a show the whole tough guy image is...

How to kill the hits to your website

I went on holiday a few weeks ago. Unbeknownst to me the Random Pub Finder's database died the day I went away. With no web access I only found out on my return. A few days of emails to our ISP followed (yes we should have had a backup but we didn't; fortunately our ISP did have) and finally the site was back online. A few days after that I went off to Switzerland and what do you know, the database died again. A more paranoid person than myself might think somebody knew I was going away and did it deliberately. I'm not quite that paranoid but am obviously paranoid enough to consider it as a possibility. After the first data loss I'd done a backup, but I didn't have access to it whilst I was out of my office. Anyway, upon my return, the site was restored pretty quickly. So in all the Random Pub finder was offline for about three weeks.

Prior to the database crash, we were averaging about 300-400 visitors a day, now we are barely reaching 100. I guess the broken pages are still in the search engines' caches and who knows if and when we'll get our visitors back...

The wonders of e-ticketing

If you go to Rail Europe and purchase a ticket, chances are you won't actually be sent real tickets. Instead you'll be sent an itinerary with some reference numbers. Then when you get to the station, you go to a machine and pick up the real tickets. So I was trying to get to Basel In Switzerland and chose the train since I had too much baggage to go on the plane and also felt like going on the train was the right thing, environmentally, to do.

So I got to St Pancras and started typing in my reference numbers. My Eurostar tickets popped out but I was unable to get the tickets for the rest of the journey. I thought perhaps this was because I needed to pick them up from the station in Paris so didn't worry too much.

I arrived in Paris, dragged my stuff to Gare d'Est and tried to type in my reference numbers again and got nowhere. The touch screen computer helpfully had an English option with a button so small that it was almost impossible to hit the screen in the correct place to get it working. Nice touch, SNCF. So I started queuing up in the ticket office but time was running out, so I headed for the train. After all, I had an itinerary that included my seat number, so it was obvious that it was my seat. Unfortunately, the ticket inspector didn't see it that way. I didn't have a ticket. Yes, the train was fully booked and nobody was demanding to sit in my seat but I didn't have a ticket. With the man next to me translating, I tried to argue my case, but got nowhere and ended up paying for my ticket again. Ho hum. The ticket inspector did assure me I could get my money back if I went to the ticket office when I got off.

But at least I managed to get to Basel. So the next thing to do was get my ticket refund and get my ticket for the return leg of my journey. Unfortunately the ticket office in Basel informed me I was now in the wrong country to pick up my tickets for the return journey and would have to go to France. Fortunately Basel is only eight minutes from the border so this wasn't too time consuming or expensive. When I got to St Louis, the lady in the ticket office was very nice and her English wasn't too good (although clearly much better than my French) so after getting my return ticket I thought better of trying to explain why she needed to give me some money due to her jobs worth co-worker.

So lesson learned, trains may be more environmentally friendly but train companies don't seem to be in any rush to actually compete with airlines. They cost more, take longer and make life difficult for their customers. To solve one of these problems, here's a suggestion for you, how about sending the tickets along with the itinerary and then I might be tempted to use the train again in Europe?

Babysitting laptops

PCs are so fecking demanding. I've got a few laptops at home ready for some training I'll be doing and every time I switch one of them on I'm assaulted by a bunch of popups telling me I need to update this or that, Windows Update, Java, Macafee, some other crapware installed with Dell. After half an hour of updates, I can finally get round to doing whatever I need to do, which I've forgotten about by now of course.

So how can this be solved? Here's some suggestions

Windows Update - OK, this is probably a necessary evil, although I'm dubious about some of the updates pushed my way. Are they all really necessary?

Macafee - I haven't run a scan for two weeks because the laptop hasn't been switched on at all, I'm not likely to have acquired a virus in that time am I?

Sun Java - Do you really have to keep bugging me to install the Google Toolbar? And what the feck has that got do with Java anyway?

Dell - Stop installing all this crap on your PCs. Offer an option of installing a clean OS, even if it costs a little more. The problem is that all that extra crap you install hits performance and half of it is badly written trash so people start to think Windows is crap and go off and buy a Mac, even though it's actually Dell that's the problem. The irony is that the thing that made PCs successful, open hardware meaning anybody can build them and install whatever they feel like, is the thing that may well kill them off, replaced by the closed Mac...

The Coming Collapse of the Middle Class

This video may be about the US but I'm guessing it also applies to the UK. We may think we are richer than previous generations, but in fact we are spending more on housing, healthcare and have more debt, even though we generally have two parents in work. The problem with having two parents in work is we have no slack in the system. If one parent falls ill or loses their job, we pretty quickly run out of money, since the other parent can't go out to earn extra, they are already in the workforce.

Connecting two PCs with a network cable

Wireless networks are great for home networks, unless you need to copy large amounts of data around. OK, 802.11g is probably pretty fast, but I've got 802.11b here and when I tried to copy 4Gb of data around, things got a bit slow. I had no blank DVDs and my USB drive is only 1Gb so I had to figure out another way to get the data from one machine to the other. So I figured I'd network them the old way, with wires. And here, mostly so I remember how to do it next time, is what's required. This may not all be required but I'm a software man, not hardware, and you know how many software engineers it takes to change a light-bulb right? But given that most of the information I've found on t'internet seems to be "use a cross-over cable", it might be helpful to someone.

First, make sure the two computers are part of the same workgroup. Right click on 'My Computer' and view the computer's properties to see what workgroup the PCs are a member of. If they are on a domain, I guess they need to part of the same domain.

Next, the cable needs to be a cross-over cable. I think I, er, borrowed my cross-over cable from a former employer but they are available from all good computer equipment shops.

Next, make sure TCP/IP is configured to use fixed IP addresses. More often than not TCP/IP is set up to obtain an IP address automatically since they'll get assigned by your ISP or router or some little gremlin who sits in a box on the internet somewhere - look I dunnow how it works. Here your TCP/IP connection is just between two computers, so there is no little gremlin to assign IP addresses. So look at the properties of your network connection and from there look at the properties of the TCP/IP stack. I think you can pretty much choose whatever IP address you like, so long as the two PCs have different IP addresses. I chose 100.100.100.101 and 100.100.100.102 and that worked fine.

Finally, disable your wireless connection on one PC at least. This may well make no difference at all, but it seemed like a good thing to do. If your PC can see another computer via two network connections, how does it choose which connection to use when copying files? Again I don't know, so figured disabling the slow wireless connection would be a good idea.

SonicWall VPN client and wireless networks

I've had this problem for a while. I have a couple of computers connected on my home wireless network which generally can see each other to share files. But sometimes I'm unable to connect to the other machine. I'd always put this down to some weirdness going on in my wireless network and dealt with it by copying files onto a USB drive. But I finally figured it out today. It would appear if I have the SonicWall VPN client running on one of the machines this causes the wireless network to go funny. Shutting SonicWall down fixes the issue.  No idea if this is a bug or if it is required behaviour but now I know why it happens I don't really care.

Cuil not so cuil

It's been done to death already but I can't help agreeing with the pundits who say Cuil will not replace Google as the search engine of choice. I did a search for "Doogal Bell" and most of the returned sites were search listings on other sites that linked to my stuff. OK, I could get to my stuff indirectly this way, but it's hardly what I'm after. Doing a search on Google brought up this site, my home page, the Random Pub Finder and my photos on flickr, which is what I'd expect. It did throw up a few useless directory and search listings sites but further down the list. OK, not very scientific, but it's very rare Google gives me unhelpful results so it's still relevant I think.

The Cuil website says "Rather than rely on superficial popularity metrics, Cuil searches for and ranks pages based on their content and relevance". Isn't this the largely discredited method of searching the web that was used by search engines before Google came along? So how's it going to be better now? It was way too easy to game those search engines by just repeating keywords in the text of your page.

Anyway, it'd be nice if somebody did come up with a decent alternative to Google, it's becoming increasingly obvious that they will soon be a monopoly and we all know what monopolies start getting up to...

Radiohead - Ceremony

One of my favourite bands* covering one of my favourite songs, what's not to like?

And if you like it, download it with KeepVid (this is mostly a reminder for myself, I'm sure you already know about this website since you're probably much more web savvy than me)

* If you've read one of my previous posts, you may think I don't like them anymore, but their last album has mostly restored my faith. As has this cover.

Tracking down memory leaks in managed code

I've been spoilt in the past when I've had memory leaks in my applications. I've worked at places where AQTime has been readily available. But I've recently noticed what appeared to be a memory leak in the FreeFlow Administrator and given that it's a free application I can't justify spending cash on a memory profiler. I had a search on the internet for a free memory profiler and downloaded a trial version of .NET Memory Profiler from Scitech. This was pretty sweet but when the trial ran out I was again unable to justify spending money on buying it.

I then thought there might an API available to capture the objects allocated by the .NET runtime but I was unable to find one. I'm guessing there has to be one, since otherwise how would memory profilers work? Perhaps the API is unmanaged or isn't documented, either way I was unable to find it. But what I did find was useful none the less. Shipping with the .NET runtime is a DLL called SOS.DLL that can be used from Visual Studio to debug memory leaks. I won't go into configuring Visual Studio to use the DLL since this post covers this in detail (the post talks about Visual Studio 2005 but it works just as well in 2008). Instead I'll cover the steps needed to track down a memory leak.

First up, set a breakpoint in your app. When you hit the breakpoint and are in the debugger, these are some useful commands that can be typed into the Immediate window.

.load sos - this loads the SOS.DLL so you can now start to use the SOS commands

!DumpHeap -stat - this will show a list of objects created, grouped by their type and ordered by the total amount of memory used by the objects. You need to analyse this list and look for anything that looks suspicious. I generally ignore the .NET types and concentrate on my own types, since it's most likely that these are causing the problems. Also, it's pretty hard to figure out how many .NET type instances would be too many, there may seem to be lots of string objects around but how many should there be? I was suspicious about a class called FolderControl, so decided to dig deeper into the details for that class.

!DumpHeap -type <type name> - this shows the details of each instance of the specified type (you don't need to specify the fully qualified type name, just the class name will do). The most important detail listed here is the address, since this will be used in the next command.

!GCRoot <address> - this looks for references to an object, which can help track down why an object isn't being garbage collected and hence causing a memory leak. I found the output somewhat confusing but I guess it may be useful.

!help - this lists all the commands available. There are many more than the ones mentioned above.

!help <command name> - get more information about a specific command.

In my case, my dynamically created control was hooking into the Application.Idle event and never unhooking the event handler. Since the Application object remains active for the lifetime of the application, my control was never garbage collected. In my experience, a lot of memory leaks in WinForms applications are caused by event handler issues like this.

After all that, I feel like a proper hardcore geek. Time to learn some assembler...

IE7 vs FireFox : The never ending debate

I was reading a forum today when I spotted yet another debate about FireFox and IE7. Much like the Windows vs Mac vs Linux debate, this is getting incredibly tedious. Both browsers (along with Safari, Opera et al) do a perfectly good job of letting people browse the web. Both have their plus points (e.g. FireFox has better plug-ins, IE tends to work with more sites) so just let it go, will you?

Much like the OS debates, the thing that really gets my goat is the smug superiority of FireFox users. Look, using a minority web browser does not make you a hip happening person, it just means you use a different browser than the majority of people. It doesn't make you a better person just because you're not using software written by the "evil empire".

For the record, I use IE and FireFox and I'm not hugely excited by either...

Zoopla - How much is your house currently worth?

Yet another site for the British obsession with house prices. When I first looked, our house was apparently worth £371,039, now it's worth £375,350, who says house prices are falling? (well me for one...). I'm not sure how valuing it to the nearest pound is possible but hey who am I to argue. Perhaps this is the answer for people having trouble selling their homes, put them on the market for a very exact price, thus showing buyers that you're not messing about and know the value of your house precisely. Well, perhaps.

Any talk of the massive inflation in house prices always makes me think of the quote "House prices are a matter of opinion but debt is real". Even though the value of our house has apparently increased way beyond inflation in the 8 years we have been living here, the debt hasn't reduced dramatically. And due to the way mortgages work, we still have to pay the same amount every month. In fact, come November, we'll probably have to pay a whole lot more. So, why is house price inflation good exactly?

How to escape text in SQL statements

Two things got me thinking about escaping text in SQL statements recently. I had previously thought it was a simple topic that everybody already knew about but given that these two things occurred I can only assume that not everybody knows about why escaping text in SQL is important and how to do it. First people keep ending up on the FreeFlow website because one of the methods in the class library is called SqlEscape, so obviously people want to know how to do it. The implementation is very very simple.

    /// <summary>
    /// Escapes a string so that single quotes are replaced by two quotes for use in SQL expressions.
    /// </summary>
    /// <param name="sql">The string to escape.</param>
    /// <returns>The escaped string</returns>
    public static string SqlEscape(string sql)
    {
      if (sql == null)
        return null;
      return sql.Replace("'", "''");
    }

Second, I got hold of some free PHP code off the web that failed to do any escaping of the text in SQL expressions. I foolishly deployed this to a website without checking the code beforehand. Fortunately I managed to spot the error before any dodgy hacker managed to take advantage of it. In the PHP world, escaping can be done using the mysql_real_escape_string function.

So that covers the how but why is this important? The first, probably least important, reason is that it means queries that include ' in text strings will work. I guess most people come across this problem when they try to enter a name like "O'Connor" into an application and it fails.

The more important issue, particularly with public facing websites, is SQL injection where a hacker can manage to run pretty much any query they like against your database.

There are other ways to solve this problem. Stored procedures and parameterised queries will also do the trick.

The joy of WITH (NOLOCK)

For a long while I thought it odd that reading from a SQL Server database caused the rows that were being read to be locked. Thinking about it for a little longer makes it clear that it's a necessary evil. You don't want some other process to update records as you're reading them, since the results you get back may well be inconsistent. But the thing is this often doesn't matter that much, so long as it's near enough correct. I've only recently come across WITH (NOLOCK) and I think it's wonderful. It's a quick and dirty way to boost the performance of your queries since it means the query can execute without acquiring a lock on the rows you're trying to get hold of.

Opponents of using it will say there are more fundamental problems with your database that could be fixed with some better indices, but sometimes we don't have the time to do proper performance analysis, so a quick fix is welcome. Purists will no doubt look down their noses at using this technique since it's just not the right thing to do. But I don't care, it improves performance for very little cost. Saying that, I do wonder how inconsistent the data may be. For instance, could I read a row that has a half-written text field? I haven't seen anything like this yet but I do wonder if it's possible.

RIP Elmo

Yesterday our pet chinchilla Elmo bit the dust. He had been ill for several weeks. He'd stopped eating and a visit to the vet and some drugs didn't help the little fella.

At least I wasn't to blame for this death unlike our last chinchilla, who I accidentally stood on as he ran down the stairs. A similar fate befell my brother's cat who ended up under the wheels of his car.

I was worried how our daughter would handle his death, but in fact she was very cool about the situation, mostly concerned about when we'd be getting another pet...

IE8 not ready for prime time

I downloaded and installed IE8 beta 1 primarily to have a look at the new XDomainRequest object which allows for cross-domain requests. Unfortunately that old problem security gets in the way. Although the documentation implies it is possible to make requests to any server, it appears the server has to explicitly support the new request protocol. Ho hum.

Anyway, IE8 screws up lots of pages (most notably Google Maps) and although it claims to have an IE7 emulation mode, using this mode seems to make no difference at all, as far as I can see. So time to uninstall, at least until the next release...

IE6 is dead, long live IE7

I'm not one for making predictions. OK, I've made a few in my time but I'm invariably wrong. Predicting that the majority of IE users would be using version 7 within two months was one of my worst. Here we are, over 18 months since it was released and IE7 visitors to the Random Pub Finder have finally started to outnumber IE6 users. So I guess in another 18 months I can start to forget about supporting IE6. Oh, and worry about IE8 instead, sigh... 

How to use Experts Exchange

I don't know about you but a lot of my web searches bring up Experts Exchange articles. When you first look at the page, it looks like you have to register to see the responses to the query. In fact it says "All comments and solutions are available to Premium Service Members only". But look a little further down the page (OK a lot further) and you'll see all the responses without paying a thing.

It may seem surprising that the comments are all visible to anybody who hasn't registered, kind of destroying their whole business model. But I guess they have a problem. I'm guessing most of their traffic comes from search engines so they need to have some decent content to get visitors in. To do that, they need to include all the responses. They could hide the text from the browser whilst making it visible to search engines, but this would almost certainly get them removed from the search engine listings, since this would be considered as black hat SEO. So they have to include the responses visible to everybody and just hope they get enough suckers signed up with the huge blocks of text before you get to the real content.

Metastorm BPM 7.6 and Windows Workflow part 3 - Using Visual Studio

In the first two parts of this series I've given a brief introduction to the integration of Windows Workflow into Metastorm BPM 7.6. For this final part I'll be discussing what interests me, being able to execute workflows authored in Visual Studio from Metastorm BPM.

The first question was, is it even possible? There is no UI for publishing workflows to the Metastorm database, other than those authored in the Metastorm WF Composer. But looking at the database, it looked like the only tables that needed populating were eMSWorkflow and eMSWorkflowDefinition. After some playing around I came up with the following code, a simple command-line tool. This is by no means bulletproof, it will fail if the workflow has already been published (the next version of the FreeFlow Administrator will let you delete workflows) and will likely fail for a host of other reasons. I was using it purely as a proof of concept.

using System;
using System.Data;
using System.Data.Odbc;
using System.Diagnostics;
using System.IO;
using System.Reflection;
using System.Workflow.Activities;

namespace PublishWorkflow
{
  class Program
  {
    static void Main(string[] args)
    {
      if (args.Length < 4)
      {
        Console.WriteLine("PublishWorkflow usage:");
        Console.WriteLine("PublishWorkflow [ODBC DSN] [ODBC username] [ODBC password] [Workflow DLL] ");
      }
      else
      {
        string fileName = args[3];

        // get the assembly
        Assembly assembly = Assembly.LoadFrom(fileName);

        // get the workflow type
        Type workflowType = null;
        Type[] types = assembly.GetTypes();
        for (int i=0; i<types.Length; i++)
        {
          if (types[i].IsSubclassOf(typeof(SequentialWorkflowActivity)))
          {
            workflowType = types[i];
          }
        }

        if (workflowType == null)
          throw new Exception("Can't find a workflow in the assembly");

        string connectionString =
          string.Format("UID={0};PWD={1};DSN={2}", args[1], args[2], args[0]);
        using (OdbcConnection connection = new OdbcConnection(connectionString))
        {
          connection.Open();
         
          Guid guid = Guid.NewGuid();

          // write to eMSWorkflow
          using (OdbcCommand command = connection.CreateCommand())
          {
            command.CommandText = string.Format(
              "INSERT INTO eMSWorkflow (eWorkflowName, eWorkflowGuid) VALUES " +
              "('{0}', '{1}')",
              workflowType.FullName, guid.ToString());
            command.ExecuteNonQuery();
          }

          // write to eMSWorkflowDefinition
          using (OdbcCommand command = connection.CreateCommand())
          {
            string fileVersion = FileVersionInfo.GetVersionInfo(assembly.Location).FileVersion.ToString();

            byte[] buffer;
            using (FileStream stream = new FileStream(fileName, FileMode.OpenOrCreate, FileAccess.Read))
            {
              buffer = new byte[stream.Length];
              stream.Read(buffer, 0, Convert.ToInt32(stream.Length));
              stream.Close();
            }

            command.CommandText = string.Format(
              "INSERT INTO eMSWorkflowDefinition (eWorkflowGuid, eWorkflowName, eFileVersion, eFullyQualifiedTypeName, " +
              "eFullyQualifiedAssemblyName, eWorkflowDefinitionType, eRulesDefinitionName, eLoadedTime, " +
              "eWorkflowDefinition, eWorkflowProject) " +
              "VALUES ('{0}', '{1}', '{2}', '{3}', '{4}', 'A', '', GETDATE(), ?, NULL)",
              guid.ToString(), workflowType.FullName, fileVersion, workflowType.AssemblyQualifiedName,
              assembly.FullName);
            OdbcParameter parameter = new OdbcParameter();
            parameter.Value = buffer;
            parameter.DbType = DbType.Binary;
            parameter.Direction = ParameterDirection.Input;
            command.Parameters.Add(parameter);
            command.ExecuteNonQuery();
          }
        }
      }
    }
  }
}

So I published my simple Visual Studio authored workflow using this code and was able to execute it. It would appear the process context activity provided as part of the Metastorm activities isn't required, although I assume it will be needed if you need to get folder information into your workflow.

The next step was to see if I could execute a workflow that contained custom activities. Obviously the problem here is where will the engine pick up the required assemblies (assuming the custom activities are in their own assembly)? I tried putting the DLL in the engine directory, dotnetbin directory and even the System32 directory (since this is where dllhost lives) but none of them seemed to work. All I got was the following useful error message

Microsoft Workflow evaluation (Instance ID 963fc00a-f2f9-40a8-a0df-7111b0706cd7)
For more information refer to the workflow tracking and event tables.

In fact the tracking and event tables contained no useful information. This error message seems to be the default error message when something goes wrong with a workflow and I've never found any useful information in the database. Debugging the engine through Visual Studio seems to be the only way of getting hold of the real error.

In the end the only way I found to get the engine to pick up my custom activity assembly was to install it in the GAC. Once I did that the workflow executed correctly.

So yes it is possible to execute a VS authored workflow in Metastorm, although some frigging around is required. However I have one concern. One problem I encountered whilst researching this was that workflows would abort after a certain time. Once again, the error above was shown in the Designer Log. Debugging the engine showed the problem was the workflow was timing out. I found a registry setting that controls this timeout, which by default is set to 60 seconds. But how does this timeout work? Is a thread kept alive waiting for the workflow to terminate? If so, this will be a problem for scalability if you wish to execute long running workflows. And that is what Windows Workflow is all about, running some code, sleeping for a week, executing some more code etc. I don't have the answer to this question yet, but will investigate further.

Part 1 - The basics

Part 2 - The database tables

Part 4 - Using your own activities

Part 5 - Long running workflows

Part 6 - State machines

Fixing Flash problems in IE7

I've had this for a while, some of the Flash content on the BBC website wouldn't play (although this is better than the problems I had with their previous Real Player content which always seemed to kill my wireless router). I was told my version of Flash wasn't up to date. Re-installing Flash didn't make any difference and following the suggestions here didn't help either.

So I decided to have a closer inspection. I had a hunch it may have something to do with my user agent string, since Flash content worked in some places (including the BBC iPlayer, go figure). So I checked my user agent using the following HTML page

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" >
<head>
    <title>Untitled Page</title>
</head>
<body onload="javascript:alert(navigator.userAgent)">

</body>
</html>

Weirdly this showed my browser as IE6... Which was odd. Searching a bit further I came across this Microsoft article on user agent strings. BTW, the suggestion to type javascript:alert(navigator.userAgent) into the address bar didn't work for me, my guess is this was a security hole waiting for an exploit so has been disabled. Anyway I had a look in my registry to see what was happening and it turned out I had a registry key under HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\User Agent\Post Platform. Part of it pointed to bsalsa.com. I guess I must have installed some of their stuff some time ago though lord knows why they need to fiddle with user agent strings.

Anyway, after deleting this registry key and a restart of IE my user agent string returned to something more sensible and BBC Flash content suddenly sprang into life. Not only that but I can now login into my Fidelity account as well, although that's not necessarily such a good thing given the current state of the stock market.

Update - Hmm, the registry entry has re-appeared, so one of the apps I use quite regularly is writing to that spot in the registry. Next step is to figure out which app it is...

Update 2 - Looks like PHPEdit was responsible, an app written in Delphi which presumably uses one of the components that come from bsalsa.com. I guess any programs using the component may cause the same problem. Getting the latest build from their website fixed the problem.

Metastorm BPM 7.6 and Windows Workflow part 2 - the database tables

In part 1 I discussed the new Windows Workflow features in Metastorm BPM 7.6. I'll now move onto the database tables used by WF workflows in Metastorm.

First we have two tables that contain process metadata.

eMSWorkflow is pretty simple, it contains one entry for each workflow published to the database. There is a column for the name of the workflow and a GUID column. The name is the name that will be used when executing a workflow from Metastorm. I'm unclear what the GUID column is for. Multiple workflows with the same name aren't possible so it isn't for differentiating between two workflows with the same name.

eMSWorkflowDefinition holds an entry for each workflow version published to the database. Most of this table is pretty self-explanatory. Again there is a GUID column whose purpose isn't clear. There is an interesting column called 'eWorkflowDefinitionType'. I haven't investigated this too far but it suggests that it is possible to publish workflows as assemblies or as uncompiled XOML file.

Next we have tables that hold workflow instance data.

eMSWorkflowTracking contains audit trail information for executing workflows. It looks like Metastorm have implemented their own tracking service which makes sense since it means the workflow audit trail can be connected to the Metastorm folder's audit trail. One problem I have with the current implementation is there appears to be no way to connect a workflow instance to its type, so it will be difficult to figure out how many instances of a particular workflow type are in existence.

eMSWorkflowEvent also includes data about workflow instances. The difference is that this table contains information that is related to the workflow runtime, rather than the workflow itself. So it will contain entries for when an instance has been loaded or idled etc.

Finally there are two other tables. These are the standard tables created for the WF persistence service, CompletedScope and InstanceState. The Microsoft documentation for these tables seems to be pretty much non-existent, so I don't have much idea what is in there. All I can assume is the 'state' column contains the serialized data of the current state of the workflow instance.

So that's the database tables. Part 3 will discuss running workflows generated in Visual Studio (if I get time to actually get it working).

Part 1 - The basics

Part 3 - Using Visual Studio

Part 4 - Using your own activities

Part 5 - Long running workflows

Part 6 - State machines

How record companies can save themselves

I got a lovely book for Christmas, "Factory Records - The Complete Graphic Album", which shows off the artwork for most of the albums and singles that came out on the Factory label along with other artwork used in ads etc. It got me thinking about why people are not buying music from record labels much anymore, instead choosing to download it free over the internet. Of course, one reason for this is the price but I think there's more to it than that. People have always taped music from friends but they still bought it as well.

But look at that Factory artwork again, some of it was superb (and some of it wasn't...). But buying a Factory record wasn't just about music, it generally came in a beautiful package as well. In the days of vinyl, there were little messages in the run-out groove (it wasn't just Factory that did this of course). Being 12" wide meant the artwork could be much more complex. So record companies should go back to vinyl? No, though they could certainly think about it. But the point is that when I bought music in those days I felt a connection with the artist or record company because it felt more than "product", it was sometimes a work of art.

Now if I pop down to a music shop most of the racks seem to be populated with CDs that look like they had absolutely no thought put into their production. Plastic case, crap photo of the artist, no design aesthetic. Why not download it for free? What do I gain by paying a tenner?

But it doesn't have to be this way. I just bought the boxed deluxe edition of U2's "The Joshua Tree". OK, I know, self-important and pretentious, but I do think it was a great album so I'll overlook the earnestness. For just over twice the price of a normal CD, I got 2 CDs (the album and a CD of rarities), a DVD (live concert, documentary and videos), a book and some postcards. All in beautiful packaging. I snapped it up without a second thought. Presumably Island are making money out of this, so why can't other recorded music come like this? Give people a reason for buying your music and they may well be persuaded.

Adding code to a XOML workflow

I was aware that it was possible to add code to a XOML workflow but I hadn't actually seen any examples of it, until I started looking into the Metastorm WF integration. I'm not sure it's something I'd like to do, it's pretty ugly and I like the idea of producing a workflow definition in a purely declarative manner. That way you're forced to put real code in their own activities which I think helps the overall design of your system. However there is one place where it isn't possible to do everything in XML, when you need to add a property to a XOML workflow. So here's an example of that. If you get stuck using this, you're on your own, I don't know much about it and the documentation seems a bit thin on the ground. Here's hoping that MS add support for XOML properties in the next release...

<x:Code><![CDATA[//<?xml version="1.0" encoding="utf-16"?>
//<XCodeItem xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="XCodeProperty">
//  <MemberTypeName>System.Boolean</MemberTypeName>
//  <MemberName>Thingy</MemberName>
//  <EmitDependencyProperty>true</EmitDependencyProperty>
//  <IsMetaProperty>false</IsMetaProperty>
//  <IsAttached>false</IsAttached>
//  <IsReadOnly>false</IsReadOnly>
//</XCodeItem>


public static System.Workflow.ComponentModel.DependencyProperty ThingyProperty = DependencyProperty.Register("Thingy", typeof(System.Boolean), typeof(MyWorkflow));

public bool Thingy
{
get
{
return ((bool)(base.GetValue(MyWorkflow.ThingyProperty)));
}
set
{
base.SetValue(MyWorkflow.ThingyProperty, value);
}
}
]]></x:Code>

Metastorm BPM 7.6 and Windows Workflow part 1 - The basics

The new version of Metastorm BPM, 7.6, has several new features (and I may do a full review at some point), but the one that interests me the most is the integration with Windows Workflow. This has been a fairly long time in development, a screenshot appeared on the web about 18 months ago. The first WF integration appeared in 7.5, which allowed BPM functionality to be called from WF workflows. This was not of great interest to me as it always seemed more useful to be able to call WF workflows from Metastorm. This is what has been added in 7.6.

Installation is easy, although a complete re-install is required, rather than an in-place upgrade. An upgrade will update the 7.5 pieces, but will not install the new 7.6 pieces.

The main thing of interest is the new WF designer (called WF Composer) that is used to develop and publish WF workflows to the Metastorm database. It's quite a nice tool, but my main gripe is that it's not Visual Studio. If it's aimed at developers then integration with Visual Studio would be a more preferable solution. Metastorm don't have the resources to produce an alternative to VS and hence any WF designer they produce will always look poor in comparison. Perhaps developers aren't the target audience, but if not then why does it include a C# editor? This is also much less useable than the Visual Studio code editor.

Another problem is that it isn't possible to add your own activities into the Designer. I'd always imagined WF could be used by a non-technical person, but to achieve this would require developers to go off and write activities specific to their organisation, that could then be plugged together by the non-technical people. So to my mind, the Composer isn't suitable for a non-technical person either.

To be fair, the Composer does add some functionality to make life simpler. When creating a workflow, it adds the Process Context activity, which provides access to the data in the BPM folder. It also simplifies adding properties to the workflow, since you can add a property just by selecting a name and a type (much like adding custom variables in Metastorm BPM). This then adds the relevant code to the generated workflow. It also includes some activities that implement familiar Metastorm BPM functions such as Email, Manager, SelectSQL etc. Not all functions are wrapped as activities but the rest can be accessed via the generic EvaluateFunction activity.

Executing workflows from BPM is pretty straightforward. Add the Workflow Support Library to a procedure and then use the Integration Wizard to execute the workflow synchronously or asynchronously. One thing to consider when executing a long-running workflow is how you're going to track its life time. It will have to be executed asynchronously since a synchronous execution will lock up your BPM folder and will also likely time out. You can get hold of the ID of the WF instance when you execute it and the tracking data is stored in the Metastorm database so it should be possible to get hold of the tracking data to display in a Metastorm grid. If your WF workflow must complete before your BPM workflow can continue, you'll probably need to add a timed loopback action to keep checking until your workflow is complete. Alternatively you could send a message back from the WF workflow to inform the Metastorm folder it has finished. There is unfortunately no user interface to view WF instances which could make debugging more difficult.

That all said, I am still interested in getting WF workflows executing in Metastorm BPM. This could be very useful so we can re-use functionality in different workflow environments and even help us to migrate away from Metastorm BPM when we are looking for a cheaper solution. But given that I would like to use Visual Studio to author my workflows how do we get them into Metastorm? I'm going to investigate this and will report back in part 2.

Part 2 - The database tables

Part 3 - Using Visual Studio

Part 4 - Using your own activities

Part 5 - Long running workflows

Part 6 - State machines