Poor man’s XSLT profiling in detail

A couple of years ago I wrote a post giving a vague idea of how to profile XSL transforms without shelling out for Visual Studio Team System. I went back to it recently and realised it didn’t really provide enough information on how to actually perform the profiling. So here’s another go at it.

The first thing we need to do is compile the XSLT into a .NET assembly. For this we need to use the XSLTC.EXE tool that ships with the Windows SDK. You can find this somewhere like this C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools. The command-line required will be something like this xsltc.exe /settings:dtd "/out:c:\temp\style.dll" /class:MyXslt "C:\temp\TEST.xsl"

So now we have an assembly for our XSLT file. Next we need a little wrapper program to load it up and call the transform. I used a simple command line app, with code as follows

    static void Main(string[] args)
    {
      XslCompiledTransform transform = new XslCompiledTransform();
      transform.Load(typeof(MyXslt));
      XmlTextReader reader = new XmlTextReader(@"C:\temp\input.xml");
      XmlTextWriter writer = new XmlTextWriter(@"C:\temp\out.html", Encoding.UTF8);
      transform.Transform(reader, writer);
    }

Now we are ready to profile the XSLT. In the past I’ve always recommended AQTime, but my old version of it doesn’t seem to work on Windows 8. I didn’t fancy paying for an upgrade, so looked around for alternatives and found the Eqatec profiler. The free version seems to work pretty well, certainly good enough for my limited needs.

So after running my little test app through the profiler, I had a much better idea of where things were slow in my XSLT. As is often the case, it was due to some XPath using “//element” to find elements. Often you can get away with it, but if the XML document is large or that piece of code is getting hit a lot then it can bite you. 

There is another slight complication with this approach to XSLT profiling. Some of the compiled templates may have a name of “compiler:generated”, meaning it’s pretty tricky to figure out how they relate to the original XSLT. From my experience, it seems that these templates are generated by the compiler itself when it decides to split larger templates into smaller compiled templates. I found ILSpy was pretty useful here to match the compiled code back to the original XSLT.

Poor man’s XSLT profiling for .NET

If you’ve ever looked round for a profiler for XSL transformations then chances are you’ve found the Microsoft add-on for Visual Studio, which looks like it’s just the ticket, if you happen to have Visual Studio Team System. But if you don’t happen to own that version, then it might look like you have to upgrade your VS license or buy some other XSLT profiler.

But if you happen to own a .NET profiler (I highly recommend AQTime) then there may be another solution. Visual Studio comes with the XSLTC tool that can be used to generate an assembly from an XSL transformation. Once we’ve got an assembly, then we can build a small wrapper application that loads up the assembly, passes it to an instance of the XslCompiledTransform class and calls the transform. And once we’ve got that, we can use a standard .NET profiler to find bottlenecks.

And as I understand it, the XSLT profiler add-on for Visual Studio works in just this way so profiling using this technique should be just as effective as the Microsoft version.

XSLT to generate an HTML listing of your iTunes library

The data for the music in an iTunes library is stored in an XML file which means that it should be simple to produce an XSLT file to generate an HTML document listing all the albums in your iTunes library. Well, it would be, but the XML format used by iTunes is, how can I put this, idiosyncratic. Other people may use more fragrant language to describe it…

Anyway I found this very useful article describing how to create an XSLT file to do almost what I wanted, but I wasn’t interested in grouping by genre, since the genre data is often not very accurate or helpful. I’m also a little anal about my music collection and wanted it sorted by artist name, rather than album name. This part was slightly tricky because the data may not be complete. The album artist field is often not entered and the artist field may not be the album artist, even when the album is not a compilation. And finally the compilation flag may be set when you don’t expect it to be or conversely not set when you expect it to be. e.g. a greatest hits album may be flagged as a compilation. This is arguably correct, but I wanted these kind of albums to be grouped with the artist. So quite a few changes were required which is why I’m posting this, it’s not just me trying to get some reflected glory (and hence me linking to the original article repeatedly)

Anyway, the basic idea is the same as the original article. First put an XML file in the same directory as your iTunes library, as follows

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="albumList.xsl" type="text/xsl"?>
<wrapper>
  <incl file="iTunes Music Library.xml"/>
</wrapper>

Next is the XSLT, in a file called albumList.xsl in the same directory, which looks like this.

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
  <xsl:output method="html" encoding="UTF-8" indent="yes"/>

  <!-- match the wrapper and apply templates to the <incl> xml file -->
  <xsl:template match="/wrapper">
    <xsl:apply-templates select="document(incl/@file)/plist/dict/dict"/>
  </xsl:template>
  
  <xsl:key name="songsByAlbum" match="dict" use="string[preceding-sibling::key[1]='Album']"/>

  <xsl:template match="dict">
    <html>
      <head>
        <title>iTunes Album Listing</title>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
        <style>
          body
          {
            font-family:Arial;
          }
        </style>
      </head>
      <body>
        <table>
          <thead>
            <tr>
              <td><b>Artist</b></td>
              <td><b>Album</b></td>
            </tr>
          </thead>
          <xsl:variable name="song" select="/plist/dict/dict/dict"/>

          <!-- output the albums that aren't compilations -->
          <xsl:for-each select="$song[generate-id(.)=
        generate-id(key('songsByAlbum',string[preceding-sibling::key[1]='Album'])[1])]">
            <xsl:sort select="concat(string[preceding-sibling::key[1]='Album Artist'], string[preceding-sibling::key[1]='Artist'])"/>

            <xsl:for-each select="key('songsByAlbum',string[preceding-sibling::key[1]='Album'])
          [not(true[preceding-sibling::key[1]='Disabled'])]
          [not(true[preceding-sibling::key[1]='Compilation'])]            
          [1]">
              <xsl:call-template name="outputAlbum" />
            </xsl:for-each>
          </xsl:for-each>

          <!-- output each compilation -->
          <xsl:for-each select="$song[generate-id(.)=
        generate-id(key('songsByAlbum',string[preceding-sibling::key[1]='Album'])[1])]">
            <xsl:sort select="string[preceding-sibling::key[1]='Album']"/>

            <xsl:for-each select="key('songsByAlbum',string[preceding-sibling::key[1]='Album'])
          [not(true[preceding-sibling::key[1]='Disabled'])]
          [true[preceding-sibling::key[1]='Compilation']]
          [not(string[preceding-sibling::key[1]='Album Artist'])]
          [1]">
              <xsl:call-template name="outputAlbum" />
            </xsl:for-each>
          </xsl:for-each>
        </table>
      </body>
    </html>
  </xsl:template>

  <xsl:template name="outputAlbum">
    <tr valign='top'>
      <!-- the artist: -->
      <td>
        <xsl:choose>
          <xsl:when test="string[preceding-sibling::key[1]='Album Artist']">
            <xsl:value-of select="string[preceding-sibling::key[1]='Album Artist']"/>
          </xsl:when>
          <xsl:otherwise>
            <xsl:choose>
              <xsl:when test="true[preceding-sibling::key[1]='Compilation']">
                <i>Compilation</i>
              </xsl:when>
              <xsl:otherwise>
                <xsl:value-of select="string[preceding-sibling::key[1]='Artist']"/>
              </xsl:otherwise>
            </xsl:choose>
          </xsl:otherwise>
        </xsl:choose>
      </td>
      <!-- the album name: -->
      <td>
        <xsl:value-of select="string[preceding-sibling::key[1]='Album']"/>
      </td>
    </tr>
  </xsl:template>
  
</xsl:stylesheet>

Now open the XML file in your favoured browser (as long as your favoured browser is IE or FireFox) and the HTML should be generated. It can be slow, it might be the XSLT can be optimised somewhat but I’m not hugely experienced in optimising XSLT so haven’t investigated further. And of course the XSLT can’t cope with all rubbish data in your iTunes library, garbage in, garbage out.

One other improvement would be to improve the sorting so that artists such as The Fall came under F, rather than T, but I have no idea how to achieve that.

XslTransform not generating XML declarations

I’d just started having a look at some .NET 1.1 code using the XslTransform class and I was having a bit of trouble. The program that read the generated XML expected an XML declaration at the top of the document. No problem I thought, just add an omit-xml-declaration="no" attribute to the XSLT. I did that and the XML declaration still didn’t turn up. So I checked the XslTransform class to see if that had some property to force the declaration to appear but no. Then I tried fiddling with the other xsl:output attributes in the XSLT, but still nothing.

So what about the XmlTextWriter class then? No, nothing on that to control the XML declaration (although it looks like later versions of the .NET Framework do include support for this). But hang on, why am I using the XmlTextWriter class anyway? The XslTransform class will take a stream instead. And tada! That fixed the problem and also saved a few lines of code. So only an hour of my life wasted. Hopefully this will save you that time.

Even more on tick marks in HTML

It's always the seemingly simplest probems that have the most long-winded solutions. So my last post was wrong, although that solution worked in some HTML docs, as soon as I switched to HTML 4.01 FireFox started showing extra crud, as you'll notice if you're viewing this in FireFox. So here's a solution for tick marks in HTML 4.01, still using conditional comments, which works in IE6, IE7 and FireFox 2, probably...

<![if !IE]> &#10003;<![endif]>
<!--[if IE 7]> &#10003;<![endif]-->
<!--[if lt IE 7]> <input type="checkbox" disabled="disabled" checked="checked" /> <![endif]--> 

And to complete the whole thing, here's an XSLT template to put a tick mark in your HTML output.

  <xsl:template name="outputTickMark">
    <xsl:text disable-output-escaping="yes">
      &lt;![if !IE]&gt; &#10003; &lt;![endif]&gt;
      &lt;!--[if IE 7]&gt; &#10003; &lt;![endif]--&gt;
      &lt;!--[if lt IE 7]&gt; &lt;input type="checkbox" disabled="disabled" checked="checked" /&gt; &lt;![endif]--&gt;
    </xsl:text>
  </xsl:template>

and here's a tick mark that might work this time.

✓

By the way, I found this article on conditional comments particularly useful.

Update: The absolute final word on this subject is here

Maintainable XSLT

Doing a search on Google for 'maintainable XSLT' doesn't throw up a great deal but it seems like something that is really needed. There seem to be lots of resources out there telling me how to code in XSLT, but I haven't found any telling me how to do it elegantly or testably (is that even a word?). I've been working on and off on a project that takes an XML file and spits out a HTML representation of it. When I started off I decided to go with XSLT, rather than generating the output in C# using an XmlWriter. I still think that was the right decision. Even though XSLT is pretty verbose, generating HTML any other way isn't too concise or pretty either.

It's not a big XSLT file by any means (about 1500 lines), but I'm already finding it hard to manage and the tool support just isn't there. Visual Studio 2005 is a step up from 2003, but there are still plenty of things missing. There doesn't seem to be a way to get an overview of an XSLT file by showing what templates are in it. This means some of the possible advantages of splitting it out into separate templates are lost. But even if I was to split it out into separate templates, the markup required to call a template means the XSLT file might actually get bigger due to the calls to the templates!

Another thing I'd like to see is the ability to go to the definition of a template from a call-template call, but that doesn't seem to be available. How about regions, like in C#? OK, I want all the features of the C# editor in XSLT, am I barking up the wrong tree? XSLT is fairly different to C# and I'm coming at this with the mindset of a C# programmer but are there different ways of handling this complexity? Where should I be looking for this information?

XSLT not completely insane

I've played with XSLT before and never quite got to grips with it, but over the last few days I've finally got a basic understanding of what it's all about. It still seems like it was designed by somebody who was an XML/HTML addict and could only think in terms of tags when developing a programming language. Like the old saying goes, when the only tool you have is a hammer, every job looks like a nail. But the thing is if you have an XML document and you want to transform it to some other flavour of XML or HTML then XSLT is a fine choice. I'm still not sure if my XSLT is any good or not, I've not seen any coding standards for it anywhere. Should I be splitting things out into multiple templates? Should I be doing anything in particular to make my XSLT more maintainable? Dunnow. But I've got 1000 lines of it and it produces quite a nice HTML document so that's good enough for me at the moment.

Currently listening to Love Less by New Order from the album Technique