How I nearly wet myself.

It's almost as if an external consultant was providing measured commentary on our team's management imposed attempt to go agile. We're using Mingle as a time sheet application, amongst other things, with the sweetener that the alternative is either Microsoft Project and/or a dedicated time sheet application. So far I've not yet found where I'm able to record getting up between 3am and 4am to check the ETL process which fails on account of a fragile FTP interface to Bloomberg. Still, it's early days and the good news is that we're hurtling towards automated builds.

For all it's usefulness XQuery never really took off the way it seems it should have done. We have a query language from the W3C for XML and since XML use is pervasive... it's very much like SQL which is wildly popular and since querying data is a fairly natural thing to want to do... and there are some excellent tools from Altova, Stylus Studio and Saxonica... However, there the story appears to end without the expected outcome. Going forward (sorry) we'll certainly be making greater use of XQuery because so many of the terabytes we process are wrapped in angle brackets and it's XML all the way down to even input & output parameters on stored procedures.

To demonstrate its value by example. Here's one way to generate XML using LINQ and hash table to validate against a given schema:

using System.Collections.Generic;
using System.Linq;
using System.Xml.Linq;

namespace Wimiro.Data.Examples {
  public static class TransformXmlImperatively {
    public static string GenerateCdsSpreadOutput(this string xml, string requestXml) {
      var rootNode = xml.GetXmlDocument().FirstChild;
      var xmlWithNewDocumentRoot = rootNode.GetProcessingInstruction();

      xmlWithNewDocumentRoot += "<" + requestXml.GetResponseRootElementName()
      + " xsi:noNamespaceSchemaLocation=\"curves.xsd\" "
      + " xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">";

      xmlWithNewDocumentRoot += xml.GetCurvesPointsFromXml();
      xmlWithNewDocumentRoot += "</" + requestXml.GetResponseRootElementName() + ">";

      return xmlWithNewDocumentRoot;
    }

    public static string GetCurvesPointsFromXml(this string xml) {
      var curveDictionary = new Dictionary<string, XElement>();
      var rawDoc = XDocument.Parse(xml);
      var rawElements = from elements
                 in rawDoc.Elements("root").Elements("row")
               select elements;

      var curveRoot = new XDocument(new XElement("root"));

      foreach (var rawElement in rawElements) {
        if (!ContainsCurve(curveDictionary, rawElement)) {
          AddCurve(curveDictionary, rawElement, curveRoot);
        }

        FindCurve(curveDictionary, rawElement).Add(GetCurvePoint(rawElement));
      }

      return curveRoot.ToXml().StripRoot();
    }

    private static XElement FindCurve(IDictionary<string, XElement> curveDictionary, 
                    XElement rawElement) {
      return curveDictionary[rawElement.GetKey()];
    }

    private static XElement GetCurvePoint(XElement rawElement) {
      XElement curvePoint;
      curvePoint = new XElement("Point");
      curvePoint.SetAttributeValue("Spread", rawElement.Attribute("Spread").Value);
      curvePoint.SetAttributeValue("Maturity", rawElement.Attribute("Maturity").Value);
      return curvePoint;
    }

    private static void AddCurve(Dictionary<string, XElement> curveDictionary, 
                    XElement rawElement, 
                    XDocument curveRoot) {
      XElement curveElement;
      curveElement = new XElement("Curve");
      curveRoot.Element("root").Add(curveElement);

      curveDictionary.Add(rawElement.GetKey(), curveElement);

      foreach (var curveAttribute in rawElement.Attributes())
        curveElement.SetAttributeValue(curveAttribute.Name, curveAttribute.Value);

      curveElement.SetAttributeValue("Spread", null);
      curveElement.SetAttributeValue("Maturity", null);
    }

    private static bool ContainsCurve(IDictionary<string, XElement> curveDictionary, 
                    XElement rawElement) {
      return curveDictionary.ContainsKey(rawElement.GetKey());
    }
  }
}

And here's the way it should have been done using XQuery:

select @xml_out.query
('    for $Ticker in distinct-values(/root/row/@Ticker)
    let $Tickers := /root/row[@Ticker=$Ticker]
        for $Currency in distinct-values($Tickers/@Currency)
        let $Currencies := $Tickers[@Currency=$Currency]
            for $Date in distinct-values($Currencies/@Date)
            let $Dates := $Currencies[@Date=$Date]
    return
            <Curve    Ticker="{$Ticker}"
                    Currency="{$Currency}"
                    Date="{$Date}">{
                        for $node in $Dates
                        return 
                        <Point Maturity="{data($node/@Maturity)}"
                              Spread="{data($node/@Spread)}" />
                    }
            </Curve>')

StackOverflow! For all your hardware needs.