Saturday, January 17, 2009

M DSLs: Using DSL source line information at runtime

An interesting feature I included in my previous “M-to-C#” DSL example, was the ability to integrate the DSL scripts with the Visual Studio debugger and with exception stack traces. When an exception is thrown, I wanted its stack trace to refer to the DSL code, and not to the generated C# code or to an interpreter’s code. I also wanted to be able to set breakpoints in the DSL code and step into it using the Visual Studio debugger.

DebuggerIntegration

Unlike an internal DSL, this did not happen “by default”: a mapping between the DSL script source lines and the generated C# must be specified explicitly, using the C# “code line pragma”. For example, the following shows a generated C# class from my previous example DSL, including these pragmas:

using System;
using MAuthorizationDSL.Core;
public class IncidentReport_Comments_Edit_AuthRules : AbstractAuthorizationRule
{
#line 19 "../../../MAuthorizationDSL\IncidentReport_Comments.auth"
public void Evaluate(string user,Incident incident,Comment comment)
{
#line 21 "../../../MAuthorizationDSL\IncidentReport_Comments.auth"
if (UserIsInRole(user, "PlantSupervisor"))
{
#line 23 "../../../MAuthorizationDSL\IncidentReport_Comments.auth"
Allow("Plant supervisors can edit any comment at any time");
}
else
{
#line 27 "../../../MAuthorizationDSL\IncidentReport_Comments.auth"
if (UserIsAuthorOf(user, comment))
{
#line 29 "../../../MAuthorizationDSL\IncidentReport_Comments.auth"
if (DateTime.Now < incident.EndTime + TimeSpan.FromHours(12) )
{
#line 30 "../../../MAuthorizationDSL\IncidentReport_Comments.auth"
Allow("Comments can be edited up to 12 hours after the end of an incident.");
}
else
{
#line 32 "../../../MAuthorizationDSL\IncidentReport_Comments.auth"
Deny("The incident has ended more than 12 hours ago, its comments can't be edited anymore.");
}
}
else
{
#line 36 "../../../MAuthorizationDSL\IncidentReport_Comments.auth"
Deny("User can't edit another user's comment");
}
}
}
}
This was generated from the following DSL script:
Action Edit (user, incident, comment) {
if (User is "PlantSupervisor") {
Allow("Plant supervisors can edit any comment at any time")
}
else {
if (User IsAuthorOf comment) {
if (DateTime.Now < incident.EndTime + 12 hours)
Allow("Comments can be edited up to 12 hours after the end of an incident.")
else
Deny("The incident has ended more than 12 hours ago, its comments can't be edited anymore.")
}
else {
Deny("User can't edit another user's comment")
}
}
}
The C# was generated following a series of steps, which I explained in my previous post. In order to have the "#line" pragmas in the generated code, I had to pass along the source line information in all of these steps.
  • In the first step, parsing the DSL script to an MGraph representation, all resulting nodes implement the System.Dataflow.ISourceLocation interface. This interface allows each node to reference a specific source line and column number. (This interface is part of the M tools for parsing a DSL script to an MGraph, so all I needed to do was make use of it)
    image

  • For conversion of MGraph to XAML, I had to modify some of the MGraphXamlReader code to include ISourceLocation info in the generated XAML.
    <n2:IfThenElseStatement.ThenBranch>
    <n2:MethodCallStatement
    FileName="../../../MAuthorizationDSL\IncidentReport_Comments.auth"
    Span="(893:30,17)-(969:30,93)"

    Name="Allow">
    <n2:MethodCallStatement.Parameters>
    <n3:StringLiteralExpression
    FileName="../../../MAuthorizationDSL\IncidentReport_Comments.auth"
    Span="(899:30,23)-(968:30,92)"

    Value="&quot;Comments can be edited up to 12 hours after the end of an incident.&quot;" />
    </n2:MethodCallStatement.Parameters>
    </n2:MethodCallStatement>
    </n2:IfThenElseStatement.ThenBranch>
  • This adds some noise to the XAML code, but I don’t think it’s really a problem because it’s an intermediate format between the MGraph representation and the generated C#, and it’s therefore not intended to be human readable (even though reading it can help investigating some conversion issues).

    The two modifications required for this are:

    • Add an inputFileName parameter to methods in MGraphXamlReader.DynamicParserExtensions
    • Add two methods to the MGraphXamlReader class:
      private IEnumerable GetSourceLocation(ISourceLocation location)
      {
      var typeReference = GetTypeReference(location);

      var fileNameMemberIdentifier = GetMemberIdentifier("FileName", typeReference);
      var spanMemberIdentifier = GetMemberIdentifier("Span", typeReference);

      yield return new XamlStartMemberNode {MemberIdentifier = fileNameMemberIdentifier};
      yield return new XamlAtomNode {Value = location.FileName};
      yield return new XamlEndMemberNode {MemberIdentifier = fileNameMemberIdentifier};

      yield return new XamlStartMemberNode {MemberIdentifier = spanMemberIdentifier};
      yield return new XamlAtomNode {Value = ConvertSourceSpan(location.Span)};
      yield return new XamlEndMemberNode {MemberIdentifier = spanMemberIdentifier};
      }

      private object ConvertSourceSpan(SourceSpan span)
      {
      var context = new Context(this);
      var converter = new SourceSpanConverter();

      return converter.ConvertToString(context, span);
      }
      The GetSourceLocation method creates XAML nodes for the FileName and Span properties of the ISourceLocationInterface.

      The ConvertSourceSpan uses the System.Dataflow.SourceSpanConverter class to convert the Span to a concise string representation, for example: (893:30,17).

    • Use the GetSourceLocation to include these two properties in each AST node:
      if (node is ISourceLocation)
      foreach (var sourceLocNode in GetSourceLocation(node as ISourceLocation))
      yield return sourceLocNode;
  • For conversion of XAML to C# using my CodeGeneratingAstVisitor class, each AST node class now also needs to implement the ISourceLocation interface.


    image

    For example, in the code for generating C# for an if/then/else:

    public override void CaseIfThenElseStatement(IfThenElseStatement node)
    {
    generator.SetCurrentSourceLine(node.FileName, node.Span.Start.Line);

    generator.WriteIndent();
    generator.Write("if (");
    node.Condition.Visit(this);
    generator.Write(")");
    generator.WriteLine();
    ...
    }

    the SetCurrentSourceLine writes the "#line" pragma to generated C#

    public void SetCurrentSourceLine(string sourceFile, int sourceLine)
    {
    WriteLine();
    WriteLine(String.Format(@"#line {0} ""{1}""", sourceLine, sourceFile));
    }

Conclusion

With the "#line" pragmas in the generated C#, the compiled assembly's debugging information reference the DSL source. I believe this was an important feature and researched it for two main reasons:

  • First for the principle of developer productivity: problems are easier to investigate if the source code line responsible of the problem can be quickly identified.

  • But also to demonstrate that it could be done. An annoyance with BizTalk server is that exceptions reference lines of generated C# code, instead of lines in the code that was written by a developer. In BizTalk server, there are two options for debugging orchestrations:
    • The orchestration debugger, which allows setting breakpoints and stepping in the visual representation of an orchestration
    • The Visual Studio debugger, which works with the generated C# code (requires additional setup described in Symbolic Debugging for Orchestrations).

    It would be nice if future versions of BizTalk made use of C# "#line" pragmas to allow debugging directly in the ODX files (XLANG/s code). It could be argued that XLANG/s code was not intended to be read by developers and therefore referencing the C# instead of XLANG/s doesn’t really matter, but I disagree with that. Even though it’s hidden underneath a visual designer, XLANG/s is still a very interesting language, and in many cases it’s easier to read and understand than its visual representation.

I wrote this post primarily to report my modifications to the MGraphXamlReader example. As in my previous post, I’m not sure if the example is beyond the scope of M’s intended use, but it can still be interesting to others that may want to use M as a tool for writing imperative-style external DSLs.

0 comments:

Post a Comment