Change a TFS Work Item Definition

Let’s assume that the built-in TFS standard templates are not sufficient for you. You’re in luck: TFS allows you to create your own custom work item. Let’s imagine that, for some reason, you want a work item type called “Defect”, rather than “Bug”. Here’s the process, based on the “Bug” work item type.

First thing is to open the command prompt in administrator mode, and navigate to a work directory; for example, the “Documents” folder
.
Then export the template work item type, like so (you can export the entire definition of all work items, but it becomes unmanageable):

witadmin exportwitd /collection:http://tlaptop:8080/tfs/DefaultCollection /p:"TFSSandbox" /n:Bug /f:Defect.xml

customwi1

Check your working directory:

customwi2

Now, open the XML file. I used Notepad++, but you can use Notepad or vi, or whatever. To change the work item, change the name attribute; once you’ve done this, you have a new work item type:

customwi3

Now, you can add the fields that you want. I’m adding a field called “pcm.CustomField”:

<?xml version="1.0" encoding="utf-8"?>
<witd:WITD application="Work item type editor" version="1.0" xmlns:witd="http://schemas.microsoft.com/VisualStudio/2008/workitemtracking/typedef">
  <WORKITEMTYPE name="Defect">
    <DESCRIPTION>Customised defect</DESCRIPTION>
    <FIELDS>
      <FIELD name="Iteration Path" refname="System.IterationPath" type="TreePath" reportable="dimension">
        <HELPTEXT>The iteration within which this bug will be fixed</HELPTEXT>
      </FIELD>
	  <FIELD name="New Field" refname="pcm.CustomField" type="Integer" />
      <FIELD name="Iteration ID" refname="System.IterationId" type="Integer" />

The field name attribute (“New Field” in this case) tells TFS how to refer to this field to the user. This is important, because if you forget that you’ve called it “New Field”, you might assume this hasn’t worked and start googling to find out why.

Now you’ve told TFS that you have a field to store; the next step is to add that field to the layout:

    <FORM>
      <Layout HideReadOnlyEmptyFields="true" HideControlBorders="true">
        <Group Margin="(4,0,0,0)">
          <Column PercentWidth="90">
            <Control FieldName="System.Title" Type="FieldControl" ControlFontSize="large" EmptyText="&lt;Enter title here&gt;" />
          </Column>
          <Column PercentWidth="10">
            <Control FieldName="System.ID" Type="FieldControl" ControlFontSize="large" />
          </Column>
        </Group>
     <Group Margin="(10,0,0,0)">
          <Column PercentWidth="60">
            <Control FieldName="pcm.CustomField" Type="FieldControl" Label="Custom Field Here" ControlFontSize="large" EmptyText="&lt;Custom field here&gt;" />
          </Column>		
    </Group>
        <Group Margin="(10,0,0,0)">

As you can see, we can move this about and pretty much show anything we want here. Next, re-import:

witadmin importwitd /collection:http://tlaptop:8080/tfs/DefaultCollection /p:TFSSandbox /f:Defect.xml

customwi5

Using the new Work Item Type

And now we have a new work item type:

customwi6

And the new field is available:

customwi7

Using The Field in a Query

customwi8

References

Tutorial on modifying Work Item States

Tutorial on customising TFS 2013

Detailed definition of the FIELD tag

Turotial on modifying the FIELD tag

Programmatically List Existing Tags Using The TFS API

Having looked into this for some time; I came up with the following method of extracting team project tags. I’m not for a minute suggesting this is the best way of doing this – but it does work. My guess is that it’s not a very scalable solution, as it’s doing a LOT of work.

As it was, I couldn’t find a way to directly query the tags, so instead, I’m going through all the work items, and picking the tags. I couldn’t even find a way to filter the work items that actually have tags; so here’s the query that I ended up with:

private static IEnumerable<string> GetAllDistinctWorkItemTags(string uri, string projectName)
{
    TfsTeamProjectCollection tfs;
 
    tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(uri)); // https://mytfs.visualstudio.com/DefaultCollection
    tfs.Authenticate();
 
    var wis = new WorkItemStore(tfs);
 
    WorkItemCollection workItemCollection = wis.Query(
         " SELECT [System.Tags]" +
         " FROM WorkItems " +
         $" WHERE [System.TeamProject] = '{projectName}' ");                
 
    if (workItemCollection.Count == 0)
        return null;
 
    List<string> tags = new List<string>();
    foreach (WorkItem wi in workItemCollection)
    {
        if (string.IsNullOrWhiteSpace(wi.Tags)) continue;
 
        var splitTags = wi.Tags.Split(';');
        tags.AddRange(splitTags.ToList());                
    }
 
    return tags.Distinct();
}

From debugging, I strongly suspect that whatever you put in the “SELECT”, it returns the entire work item. I also, for the life of me, couldn’t work out a lambda query for parsing the tags.

The calling method is here:

public static IEnumerable<string> GetAllTags(string uri, string teamProject)
{
    var project = GetTeamProject(uri, teamProject);
    IEnumerable<string> tags = GetAllDistinctWorkItemTags(uri, teamProject);
 
    return tags;
}

I’ve listed GetTeamProject helper method before, but for the sake of completeness:


public static Project GetTeamProject(string uri, string name)
{
    TfsTeamProjectCollection tfs;
 
    tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(uri)); // https://mytfs.visualstudio.com/DefaultCollection
    tfs.Authenticate();
 
    var workItemStore = new WorkItemStore(tfs);
    
    var project = (from Project pr in workItemStore.Projects
                       where pr.Name == name
                       select pr).FirstOrDefault();
    if (project == null)
        throw new Exception($"Unable to find {name} in {uri}");
 
    return project;
}

Here’s the output:

tags1

Notes on Tags

A couple of points on tags: firstly, tags seem to exist in a kind of transient state; that is, while something is tagged, the tag exists, but once you remove all instances of a tag (for example, if I removed “Tagtest1” from all work items in my team project, TFS would eventually (I believe after a couple of days) just delete the tag for me. Obviously, in my example, as soon as I did this, I would no longer find it. This might leave you thinking that there is a more efficient way of removing tags (that is, you should be able to access the transient store in some way).

The existence of this Visual Studio plug-in lends support to that idea. It allows you to maintain the tags within your team project. If you’re using tags in any kind of serious way then I’d strongly recommend that you try it.

Performance

This is doing a lot of (IMO) unnecessary work, so I tried a little performance test; using this post as a template, I created a lot of bugs:

tags2

As you can see, I created a random set of tags. One other point that I’m going to put here is that a TFS database with ~30K work items and no code whatsoever increases the size of the default collection DB to around 2GB:

tags3

Now I ran the GetAllTags with some timing on:

tags4

19 seconds, which seems like quite a reasonable speed to me for 13.5k tags.

Programmatically Create Test Case Steps

In this earlier post, I discussed how to create a test case via the TFS API. For my next trick, I’m going to create some test case steps.

The first thing you need is the TestManagement Client:

testcasestep1

Once this is installed, the rest is quite straightforward; here’s the method to create the steps:

private static void AddTestCaseSteps(string uri, Project project, int testCaseId, string[] steps)
{
    TfsTeamProjectCollection tfs;
 
    tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(uri)); // https://mytfs.visualstudio.com/DefaultCollection
    tfs.Authenticate();
 
    ITestManagementService service = (ITestManagementService)tfs.GetService(typeof(ITestManagementService));
    ITestManagementTeamProject testProject = service.GetTeamProject(project);
    ITestCase testCase = testProject.TestCases.Find(testCaseId);
 
    foreach (string step in steps)
    {
        ITestStep newStep = testCase.CreateTestStep();
        newStep.Title = step;
        // newStep.ExpectedResult = "Expected result";
        testCase.Actions.Add(newStep); 
    }
 
    testCase.Save();
 
 
}

As you can see, the ITestManagementService does a lot of the work for you, and once you have the test case, the rest is straightforward. Here’s what the CreateNewTestCase looks like now:


private static ActionResult CreateNewTestCase(string uri, Project project, WorkItem testedWorkItem, string description, string assignee, string[] reproductionSteps)
{
    WorkItemType workItemType = project.WorkItemTypes["Test Case"];
 
    // Create the work item. 
    WorkItem newTestCase = new WorkItem(workItemType);
    newTestCase.Title = $"Test {testedWorkItem.Title}";
    newTestCase.Description = description;
    newTestCase.AreaPath = testedWorkItem.AreaPath;
    newTestCase.IterationPath = testedWorkItem.IterationPath;
    newTestCase.Fields["Assigned To"].Value = assignee;
 
    // Copy tags
    newTestCase.Fields["Tags"].Value = testedWorkItem.Fields["Tags"].Value;
 
    ActionResult result = CheckValidationResult(newTestCase);
    if (result.Success)
    {
        CreateTestedByLink(uri, testedWorkItem, result.Id);
        AddTestCaseSteps(uri, project, result.Id, reproductionSteps);
    }
 
    return result;
}

Finally, here’s the test case:

testcasestep2

References

https://msdn.microsoft.com/en-us/library/microsoft.teamfoundation.testmanagement.client.itestmanagementteamproject.aspx?f=255&MSPPError=-2147217396

http://blogs.microsoft.co.il/shair/2013/10/07/tfs-api-part-51-adding-test-step-amp-shared-step/

https://blogs.msdn.microsoft.com/visualstudioalm/2011/12/30/how-to-get-the-test-case-associated-with-the-unit-test/

GetFilesAsync is not returning all the files in the “My Documents” folder

The following code should return a list of all files in the “My Documents” folder:

    var files = await KnownFolders.DocumentsLibrary.GetFilesAsync();
    List<FileInformation> filesList = new List<FileInformation>();
    
    System.Diagnostics.Debug.WriteLine($"File count: {files.Count()}");

The returned count from this code is (in my case) 11. However, excluding folders, there are 61 files in that directory for me. When I iterate through the collection, it does indeed find 11.

After scratching my head for a while, I finally realised that the answer to my question was my own blog post. It only lists the allowed types.

Programmatically creating a test case for a work item using the TFS API

Previously I’ve covered how to programmatically create a bug in TFS. In this post, we’ll create a test case to cover it.

Set-up

What we’re going to do here is to create a new test case based on the bug that we created in the linked post, then we’re going to copy key values across, and link the two.

First, we need to do some re-factoring; the section of code the saves the work item can be extracted into something like this:

private static ActionResult CheckValidationResult(WorkItem workItem)
{
    var validationResult = workItem.Validate();
 
    ActionResult result = null;
    if (validationResult.Count == 0)
    {
        // Save the new work item.
        workItem.Save();
 
        result = new ActionResult()
        {
            Success = true,
            Id = workItem.Id
        };
    }
    else
    {
        result = ParseValidation(validationResult);
    }
 
    return result;
}

ActionResult is here for reference:


public class ActionResult
{
    public bool Success { get; set; }
    public List<string> ErrorCodes { get; set; }
    public int Id { get; set; }
}

Let’s re-visit what the code to create the bug looks like now:


public static ActionResult CreateNewBug(string uri, string teamProject, string title, string description,
                string area, string iteration, string assignee, string reproductionSteps)
{
    var project = TFSUtilLibrary.TeamProjectHelper.GetTeamProject(uri, teamProject);
    return CreateNewBug(project, title, description, area, iteration, assignee, reproductionSteps);
}
private static ActionResult CreateNewBug(Project teamProject, string title, string description, 
    string area, string iteration, string assignee, string reproductionSteps)
{
    WorkItemType workItemType = teamProject.WorkItemTypes["Bug"];
 
    // Create the work item. 
    WorkItem newBug = new WorkItem(workItemType);
    newBug.Title = title;
    newBug.Description = description;
    newBug.AreaPath = area;
    newBug.IterationPath = iteration;
    newBug.Fields["Assigned To"].Value = assignee;
 
    newBug.Fields["Repro Steps"].Value = reproductionSteps;
 
    newBug.Fields["Tags"].Value = "Tagtest1, tagtest2, tagtest3";
 
    ActionResult result = CheckValidationResult(newBug);
 
    return result;
}

You’ll notice that I’ve added some tags. Why will become apparent later.

Finding a Work Item

We’re creating a test case for a work item; consequently, we need to be able to retrieve a WorkItem, given an ID:


private static WorkItem GetWorkItem(string uri, int testedWorkItemId)
{
    TfsTeamProjectCollection tfs;
 
    tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(uri)); // https://mytfs.visualstudio.com/DefaultCollection
    tfs.Authenticate();
 
    var workItemStore = new WorkItemStore(tfs);
    WorkItem workItem = workItemStore.GetWorkItem(testedWorkItemId);
 
    return workItem;
 
}

Get a Team Project

The next step is that we’ll need to be able to find a team project (we just will):


public static Project GetTeamProject(string uri, string name)
{
    TfsTeamProjectCollection tfs;
 
    tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(uri)); // https://mytfs.visualstudio.com/DefaultCollection
    tfs.Authenticate();
 
    var workItemStore = new WorkItemStore(tfs);
    
    var project = (from Project pr in workItemStore.Projects
                       where pr.Name == name
                       select pr).FirstOrDefault();
    if (project == null)
        throw new Exception($"Unable to find {name} in {uri}");
 
    return project;
}

Create the Test Case

We now have everything that we need to create the test case. Here’s the parent code that will create the bug:


var result = WorkItemHelper.CreateNewBug(TFSUri, "TFSSandbox",
    "Test new bug", "New bug description", @"TFSSandbox\Team 12", @"TFSSandbox\Iteration 1", "Paul Michaels",
    "Click the screen");
 
if (result.Success)
{
    // Now create the test case
    var resultTestCase = WorkItemHelper.CreateNewTestCase(
        TFSUri, "TFSSandbox", result.Id, "Test case description", "Paul Michaels",
        "reproduction steps here");
}

So, we’re creating a bug and, if it’s successful, we’re creating a test case for it; the CreateNewTestCase code looks like this:


public static ActionResult CreateNewTestCase(string uri, string teamProject, 
        int testedWorkItemId, string description, string assignee, string reproductionSteps)
{
    var project = GetTeamProject(uri, teamProject);
    var workItem = GetWorkItem(uri, testedWorkItemId);
    return CreateNewTestCase(uri, project, workItem, description, assignee, reproductionSteps);
}

private static ActionResult CreateNewTestCase(string uri, Project project, WorkItem testedWorkItem, string description, string assignee, string reproductionSteps)
{
    WorkItemType workItemType = project.WorkItemTypes["Test Case"];
 
    // Create the work item. 
    WorkItem newTestCase = new WorkItem(workItemType);
    newTestCase.Title = $"Test {testedWorkItem.Title}";
    newTestCase.Description = description;
    newTestCase.AreaPath = testedWorkItem.AreaPath;
    newTestCase.IterationPath = testedWorkItem.IterationPath;
    newTestCase.Fields["Assigned To"].Value = assignee;
  
    // Copy tags
    newTestCase.Fields["Tags"].Value = testedWorkItem.Fields["Tags"].Value;
 
    ActionResult result = CheckValidationResult(newTestCase);
    if (result.Success)
    {
        CreateTestedByLink(uri, testedWorkItem, result.Id);
    }
 
    return result;
}

Couple of things to note here; the first is the tags – we’re copying them from the bug (see references). The second is that we are linking the two.

Links

Here’s how I created the link:


private static void CreateTestedByLink(string uri, WorkItem testedWorkItem, int newTestCaseId)
{
    TfsTeamProjectCollection tfs;
 
    tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(uri)); // https://mytfs.visualstudio.com/DefaultCollection
    tfs.Authenticate();
 
    var workItemStore = new WorkItemStore(tfs);
 
    var linkTypes = workItemStore.WorkItemLinkTypes;
 
    WorkItemLinkType testedBy = linkTypes.FirstOrDefault(lt => lt.ForwardEnd.Name == "Tested By");
    WorkItemLinkTypeEnd linkTypeEnd = testedBy.ForwardEnd;
 
    //Add the link as related link.
    testedWorkItem.Links.Add(new RelatedLink(linkTypeEnd, newTestCaseId));
    var result = CheckValidationResult(testedWorkItem);
}

It feels like there might be a slicker way than referencing “Tested By” by name, but this is the only way I could find.

Here’s the created bug with a linked test case:

tfsapi_bug_test

Conclusion

As with the previous post, I’m not trying to re-write TeamCity or anything here; this was just born out of some pain with manually setting these things up.

You’ll also notice that I’ve left the Test Steps; I’ll come back to them shortly (and by ‘shortly’, I mean in a later post).

References

http://blogs.microsoft.co.il/shair/2010/02/27/tfs-api-part-22-create-link-between-work-item-parent-child-etc/

https://social.msdn.microsoft.com/Forums/vstudio/en-US/8d8bfc70-4da7-40ac-ad34-28ab8ef73314/add-tags-programmatically-to-work-items?forum=tfsgeneral

Programmatically create a bug in TFS

If you’re creating a TFS API program from scratch, the first thing that you’ll need is to reference the TFS API libraries. They are in extensions:

tfsbug1

Don’t worry too much about which one’s you’ll need just yet, when you start to write some code, this will be more obvious. The next stage is to create a function that creates your bug; it might look like this:

private static ActionResult CreateNewBug(Project teamProject, string title, string description, 
    string area, string iteration, string assignee, string reproductionSteps)
{
    WorkItemType workItemType = teamProject.WorkItemTypes["Bug"];
 
    // Create the work item. 
    WorkItem newBug = new WorkItem(workItemType);
    newBug.Title = title;
    newBug.Description = description;
    newBug.AreaPath = area;
    newBug.IterationPath = iteration;
    newBug.Fields["Assigned To"].Value = assignee;
 
    newBug.Fields["Repro Steps"].Value = reproductionSteps;
 
    var validationResult = newBug.Validate();
 
    if (validationResult.Count == 0)
    {
        // Save the new work item.
        newBug.Save();
 
        return new ActionResult()
        {
            Success = true
        };
    }
    else
    {
        // Establish why it can't be saved
        var result = new ActionResult()
        {
            Success = false,
            ErrorCodes = new List<string>()                
        };
 
        foreach (var res in validationResult)
        {
            Microsoft.TeamFoundation.WorkItemTracking.Client.Field field = res as Microsoft.TeamFoundation.WorkItemTracking.Client.Field;
            if (field == null)
            {
                result.ErrorCodes.Add(res.ToString());
            }
            else
            {
                result.ErrorCodes.Add($"Error with: {field.Name}");
            }
        }
 
        return result;
    }
}

Obviously, we’re not writing a new front end for TFS here, but the basics are there. The first part of the function gets the relevant fields; once the .Validate() has been called, then we have a look at the result. If there are no errors then just save; otherwise, we try and work out what they were.

In the example above, I’m returning a class of the following type:

public class ActionResult
{
    public bool Success { get; set; }
    public List<string> ErrorCodes { get; set; }
}

But that’s only because this is in its own library. The method above also accepts a Project; assuming that you know what the project is called, you could use something like this to return the correct object:


public static Project GetTeamProject(string uri, string name)
{
    TfsTeamProjectCollection tfs;
 
    tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(uri)); // https://mytfs.visualstudio.com/DefaultCollection
    tfs.Authenticate();
 
    var workItemStore = new WorkItemStore(tfs);
    
    var project = (from Project pr in workItemStore.Projects
                       where pr.Name == name
                       select pr).FirstOrDefault();
    if (project == null)
        throw new Exception($"Unable to find {name} in {uri}");
 
    return project;
}

And that’s it; here’s my calling code:

var result = TFSUtilLibrary.WorkItemHelper.CreateNewBug(TFSUri, "TFSSandbox",
    "Test new bug", "New bug description", @"TFSSandbox\Team 12", @"TFSSandbox\Iteration 1", "Paul Michaels",
    "Click the screen");

Here’s the bug to prove it works:

tfsbug2

tfsbug3

Reading and Opening a Zip File in a UWP

Some years ago, I had an idea for an application, and the functionality of that application involved extracting the contents of a zip file. I spent a while trying to work out how to do this in VB6 programmatically, and finally got bored, and my app never happened (not that there was such a thing as an app at the time).

Recently, I thought that I might re-visit my idea. Things have moved on since VB6, and this is how to work with zip files in 2016.

The following code will allow you to access the files inside an archive:

 
public async Task GetZipFileInformation(Stream stream)
{
    System.IO.Compression.ZipArchive zip = new System.IO.Compression.ZipArchive(stream);
 
    var firstFile = zip.Entries.FirstOrDefault();
    if (firstFile != null)
    {
    …

Generally speaking, this is much easier that trying to automate PKZIP, or whatever we used to use back in 2002!

UWP Accessing Documents Library

Accessing the documents library from a UWP app is frowned upon by Microsoft; however, it is possible. Here is some code that will access the library:

var files = await KnownFolders.DocumentsLibrary.GetFilesAsync(Windows.Storage.Search.CommonFileQuery.OrderByName);
 
foreach (var f in files)
{
    BasicProperties props = await f.GetBasicPropertiesAsync();

This will access the library and get the properties for each file. However, just running it will fail with this error:

uwpdoclib1

So, you’ll probably get this error and, like me (and not for the first time), go looking for it here:

uwpdoc2

Of course, you won’t find it (because it’s not there), and then you’ll turn to Google. If that brought you here then you’re next step is to open the manifest file directly:

…
  <Capabilities>
    <uap:Capability Name="documentsLibrary" />
  </Capabilities>
</Package>

If there are already Capabilities then just add the line:

    <uap:Capability Name="documentsLibrary" />

Note: you need the uap prefix.

And, that’s not all. Next you need to tell it which documents it can access:

      <Extensions>
        <uap:Extension Category="windows.fileTypeAssociation">
          <uap:FileTypeAssociation Name=".txt">
            <uap:DisplayName>Text</uap:DisplayName>
            <uap:SupportedFileTypes>
              <uap:FileType>.jpg</uap:FileType>
              <uap:FileType>.txt</uap:FileType>
              <uap:FileType>.gif</uap:FileType>
              <uap:FileType>.doc</uap:FileType>
              <uap:FileType>.xls</uap:FileType>
            </uap:SupportedFileTypes>
          </uap:FileTypeAssociation>
        </uap:Extension>
      </Extensions>
    </Application>
  </Applications>
  <Capabilities>
    <uap:Capability Name="documentsLibrary" />
  </Capabilities>
</Package>

And that’s it. I can understand why they have all these restrictions, but they can be frustrating for programmers.

Message Persistence in RabbitMQ and BenchMarkDotNet

(Note: if you want to follow the code on this, you might find it easier if you start from the project that I create here.)

A queue in a message broker can be persistent, which means that, should you have a power failure (or just shut down the server), when it comes back, the queue is still there.

So, we can create a durable (persistent) queue, like this:

var result = channel.QueueDeclare("NewQueue", true, false, false, args);

The second parameter indicates that the queue is durable. Let’s send it some messages:


static void Main(string[] args)
{            
    for (int i = 1; i <= 100; i++)
    {
        string msg = $"test{i}";
 
        SendNewMessage(msg);
    } 
    
}
private static void SendNewMessage(string message)
{
    var factory = new ConnectionFactory() { HostName = "localhost" };
    using (var connection = factory.CreateConnection())
    using (var channel = connection.CreateModel())
    {
        Dictionary<string, object> args = 
            DeadLetterHelper.CreateDeadLetterQueue(channel,
            "dl.exchange", "dead-letter", "DeadLetterQueue");
 
        var result = channel.QueueDeclare("NewQueue", true, false, false, args);
        Console.WriteLine(result);
 
        channel.BasicPublish("", "NewQueue", null, Encoding.UTF8.GetBytes(message));                
 
    }
}

Now we have 100 messages:

persist1

Let’s simulate a server reboot:

parsist2

Following the reboot, it’s gone:

persist3

Admittedly, that doesn’t sound very durable!

Why?

The reason for this, is that the durability of the queue doesn’t affect the durability of the message. At least, if the queue is durable, it doesn’t make the message so.

How can it be made persistent?

Let’s change our send code a little:


private static void SendNewMessage(string message)
{
    var factory = new ConnectionFactory() { HostName = "localhost" };
    using (var connection = factory.CreateConnection())
    using (var channel = connection.CreateModel())
    {
        Dictionary<string, object> args = 
            DeadLetterHelper.CreateDeadLetterQueue(channel,
            "dl.exchange", "dead-letter", "DeadLetterQueue");
 
        var result = channel.QueueDeclare("NewQueue", true, false, false, args);
        Console.WriteLine(result);
 
        IBasicProperties prop = channel.CreateBasicProperties();
        prop.Persistent = true;
 
        channel.BasicPublish("", "NewQueue", prop, Encoding.UTF8.GetBytes(message));                
 
    }
}

The only difference is the addition of the IBasicProperties parameter. The Persistent flag is set. Now we’ll re run the same test; here’s the messages:

persist4

And after restarting the service:

persist5

As you can see, the messages are still there, and you can see the time difference where they’re been restored to the queue after a failure.

Speed – Introducing BenchmarkDotNet

I suppose the main question here is what price do you pay for durability. This gives me a chance to play with a new tool that I heard about a little while ago: BenchmarkDotNet.

It’s quite straightforward to use, just add the NuGet package:

Install-Package BenchmarkDotNet

There’s a bit of refactoring; I effectively ripped out the send and called it from a separate class:


class Program
{        
    static void Main(string[] args)
    {
        BenchmarkRunner.Run<SpeedTest>();
    }
}
 
public class SpeedTest
{
    [Benchmark]
    public void SendNewMessagePersist()
    {
        MessageHelper helper = new MessageHelper();
        helper.SendStringMessage("Test", "NewQueue", true);
    }
 
    [Benchmark]
    public void SendNewMessageNonPersist()
    {
        MessageHelper helper = new MessageHelper();
        helper.SendStringMessage("Test", "NewQueue", false);
    }
 
 
}

I then ran this:

persist6

And it produced this:

persist7

So, it is a bit slower to persist the message. I’m not sure how helpful this information is: I probably could have guessed that persisting the message would have been slower beforehand. Having said that, I am quite impressed with BenchMarkDotNet!

Acknowledging a Message Using RabbitMQ

Conceptually, message queues work in a similar fashion: you send a message to an exchange, the exchange allows people to read the message, and you have functionality that ensures the original message arrives with the destined recipient. Acknowledgement of a message is basically a way to ensure that delivery. Having said that, the two main message brokers that I’ve been investigating deal with this in a slightly different manner (albeit, the same things happen in the end).

If you want to follow through, it might be an idea to use the code from my first post as a starting point.

Let’s change the send code first to send a few messages:

static void Main(string[] args)
{            
    for (int i = 1; i <= 100; i++)
    {
        string msg = $"test{i}";
 
        SendNewMessage(msg);        
    } 
            
}

Now we have 100 messages.

rabbitack1

Next, we’ll look at the receiving code:

public void ReceiveNextMessage()
{
    var result = _channel.QueueDeclare("NewQueue", true, false, false, null);
    Console.WriteLine(result);
 
    EventingBasicConsumer consumer = new EventingBasicConsumer(_channel);
    consumer.Received += Consumer_Received;
 
    _channel.BasicConsume("NewQueue", false, consumer);
 
}

BasicConsume() has a parameter called “noAck”. It took me a while to work this out, but noAck means that it doesn’t expect an acknowledgement; that is, it will automatically acknowledge receipt. So, noAck = True mean automatically acknowledge, and noAck = False means manually acknowledge. That not entirely uncomplicated.

The received event looks a bit like this:

private void Consumer_Received(object sender, BasicDeliverEventArgs e)
{
    var body = e.Body;
    var message = Encoding.UTF8.GetString(body);
    //if (message.Contains("3")) 
    //   throw new Exception("Error here !");
 
    //_channel.BasicAck(e.DeliveryTag, false);
 
    Console.WriteLine(message);
}

I’ve left the error and the commented out BasicAck in on purpose. If you run this, unlike with ActiveMQ, where you will get a message at a time, you will get all messages in the queue (because it’s event based). Add in the BasicAck() to acknowledge the queue and you’re good to go.

If you add in the error at this stage, you can see that, in exactly the same way as ActiveMQ, it will only acknowledge the correct message. What you will also see here is we have the poison message scenario that I discussed in this post on ActiveMQ.

IMHO, this is where RabbitMQ beats ActiveMQ hands down. The following code is the simplest version of dealing with a poison message:

private void Consumer_Received(object sender, BasicDeliverEventArgs e)
{
    try
    {
        var body = e.Body;
        var message = Encoding.UTF8.GetString(body);
 
        if (message.Contains("3"))
            throw new Exception("Error here !");
 
        _channel.BasicAck(e.DeliveryTag, false);
 
        Console.WriteLine(message);
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex);
        _channel.BasicNack(e.DeliveryTag, false, true);
    }
}

So, we have a problem, and we issue a Nack. What the Nack does is allow you to re-queue the message. The code above allows the queue to process, but just moves all the bad ones to the back. The obvious problem here is that it the problem isn’t transient, we’ll keep coming back to them. It does, however, get around the problem – the queue is no longer blocked.

The solution, as it was with ActiveMQ, is to put them in a “dead letter queue”; however, unlike ActiveMQ, this is remarkably easy. Firstly, let’s refactor our queue creation a little:

public void ReceiveNextMessage()
{
    Dictionary<string, object> args = DeadLetterHelper.CreateDeadLetterQueue(_channel);
 
    // How declare the queue and pass in the dead letter exchange
    var result = _channel.QueueDeclare("NewQueue", true, false, false, args);
    Console.WriteLine(result);
 
    EventingBasicConsumer consumer = new EventingBasicConsumer(_channel);
    consumer.Received += Consumer_Received;
 
    _channel.BasicConsume("NewQueue", false, consumer);
 
}

You’ll notice that we go to a new helper method called: “CreateDeadLetterQueue()” and return a dictionary; which is, in turn, passed through to our new queue. The CreateDeadLetterQueue() function looks like this:

public static Dictionary<string, object> CreateDeadLetterQueue(IModel channel,
    string deadLetterExchange, string deadLetterRoutingKey, string deadLetterQueue)
{
    // Declare dead letter exchange                     
    channel.ExchangeDeclare(deadLetterExchange, "direct");
    Dictionary<string, object> args = new Dictionary<string, object>()
    {
        { "x-dead-letter-exchange", deadLetterExchange },
        { "x-dead-letter-routing-key", deadLetterRoutingKey }
    };
 
    // Bind the exchange to a queue
    channel.QueueDeclare(deadLetterQueue, true, false);
    channel.QueueBind(queue: deadLetterQueue,
                    exchange: deadLetterExchange,
                    routingKey: deadLetterRoutingKey);
    return args;
}

There’s effectively two steps. Firstly we need a dead letter exchange, and this needs a routing key (in this case, “dead-letter”). Next, we declare the DeadLetterQueue with the same routing key. Finally, return the argument list, which allows the linking of the main queue to the dead letter queue.

Now we are going to change the receive code so that it doesn’t re-queue:

private void Consumer_Received(object sender, BasicDeliverEventArgs e)
{
    try
    {
        var body = e.Body;
        var message = Encoding.UTF8.GetString(body);
 
        if (message.Contains("3"))
            throw new Exception("Error here !");
 
        _channel.BasicAck(e.DeliveryTag, false);
 
        Console.WriteLine(message);
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex);
        _channel.BasicNack(e.DeliveryTag, false, false);
    }
}

And running it results in a dead letter queue full of all our dodgy data:

rabbitack2

If you start getting errors when you run this, try referring to this article.

References

RabbitMQ documentation on the subject