Xamarin Forms Woes

Xamarin Forms is one of those technologies that I keep meaning to play with… but then don’t. Anyway, this is the story of the day that I first did… the intention was to spend a Sunday afternoon getting a quick prototype of a Xamarin Forms app running… but instead, I encountered error after error.

In the end,all I ended up producing was this post!

The first error after creating a new app:

This gave me the impression that I might have the wrong version of Android; so off I went to install the latest JDK; from here.

Thus began a process that took hours to complete…

Android SDK

Launch Android SDK Manager and install the latest (or a later) version:

Amazingly, this did seem to fix (or at least, change) the error.

Next Error:

The next problem was a build error from the Android project:


Error: No resource found that matches the given name: attr 'colorAccent'

There’s two things that were necessary here; the first was to make sure that this package was installed:

Install-Package Crosslight.Xamarin.Android.Support.v7.AppCompat

And now we’re onto the next problem…

Suddenly, it decided there was an issue with some of the resources in the file ‘Tabbar.axml’:

Errors:
1>C:\Code\MyApp\MyApp.Android\Resources\layout\Tabbar.axml : error APT0000: 1: error: No resource identifier found for attribute 'tabIndicatorColor' in package 'com.companyname.WizardGame'
1>C:\Code\MyApp\MyApp.Android\Resources\layout\Tabbar.axml : error APT0000: 1: error: No resource identifier found for attribute 'tabGravity' in package 'com.companyname.WizardGame'
1>C:\Code\MyApp\MyApp.Android\Resources\layout\Tabbar.axml : error APT0000: 1: error: No resource identifier found for attribute 'tabMode' in package 'com.companyname.WizardGame'

Taking inspiration from my Spectrum programming days, and having no clue what the hell was going on at this stage, I just deleted these lines… and got a different error:

error CS0012: The type 'AppCompatActivity' is defined in an assembly that is not referenced. You must add a reference to assembly 'Xamarin.Android.Support.v7.AppCompat, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.

This didn’t make much sense, but while I was messing around in the Package Manager screen, I decided to update the Xamarin.Forms package; for some reason, the latest version seems to be (numerically) older that the older one:

Giddy with the thrill of the update working without a new, spurious error, I updated everything! This required VS to be restarted at least once.

At this stage, the project started compiling… so before Xamarin regained its footing and thought of a new error, I added back in the code I’d removed:

    app:tabIndicatorColor="@android:color/white"
    app:tabGravity="fill"
    app:tabMode="fixed" />

Now it compiles!… But won’t deploy to Android

Xamarin was catching up with me – a new error now appeared:


1>Please select a valid device before running the application.

After a little help from a well-known search engine, I selected the emulation manager:

If you find yourself in this situation, then don’t keep clicking (despite no feedback at all, the manager does open in the background)!

Select one of the devices and start it:

And it now deploys… I didn’t get anything done, but I did manage to get Xamarin running on an Android emulator… so that’s something.

Getting Started With iOS for a C# Programmer – Part One

I’ve never programmed with XCode before; I’ve never even owned a Mac, so since I now have access to one, I thought I’d see if I could create a basic game. In order to motivate me to complete this, I thought I’d document my efforts, and try to put at least one post up every two weeks. As I knew absolutely nothing about this to begin with, I’m going to start from there.

The Basics

The first thing that you’ll need is an Apple account. You can use one that you might have set-up for your iPhone; otherwise, you can set one up here. It’s free to register, and there’s no need to sign up for a developer account immediately; although, should you wish to, it will cost you $99.

The next stage is to install Xcode; which can be found here. Xcode is the IDE, so it’s the equivalent of Visual Studio – it is not a language in its own right. The language we’re dealing with here is called Swift.

Your first program

When you initially run Xcode, you’ll be faced with a screen like this:

Since we don’t know what we’re doing, let’s select “Get started with a playground”:

And let’s select a Blank playground.

We then get an (almost) blank text editor:

The examples given seem to introduce a new level of complexity by including controllers; however, this is the simplest example of a working “Hello, world” program that I could come up with:

import UIKit
import PlaygroundSupport

let x = UILabel()
x.text = "test"
x.textColor = UIColor.black
x.frame = CGRect(x: 5, y: 5, width: 80, height: 20)

let v = UIView()
v.frame = CGRect(x: 0, y: 0, width: 100, height: 100);
v.backgroundColor = UIColor.white
v.addSubview(x)

PlaygroundPage.current.liveView = v

To break down that code

import UIKit
import PlaygroundSupport

Playground support is the library that allows you to visualise the preview of what the app will look like when it’s eventually on the device.

let x = UILabel()
x.text = "test"
x.textColor = UIColor.black
x.frame = CGRect(x: 5, y: 5, width: 80, height: 20)

The UILabel object has to live within a frame; otherwise it will not appear on the screen. The default colour is black, but as I’ve specified the colour of the view, I thought it made sense to specify all colours.

let v = UIView()
v.frame = CGRect(x: 0, y: 0, width: 100, height: 100);
v.backgroundColor = UIColor.white
v.addSubview(x)

PlaygroundPage.current.liveView = v

This sets up the view; which seems to map to a form in WinForms, or a view in WPF. Again, the view needs a frame; and, in this case, it does need a colour (as the default is black).

The x, y, height and width variables are just guesses – so these may need changing later. Finally, we set the current Live View on the playground so that it knows which view to display.

Running the Code

If you run that initially, you’ll see that, whilst it compiles, nothing actually happens; the trick is here:

Summary and Target

Once you’ve established the basic syntax, the code suddenly seems much more familiar. There are some things that appear foreign to the eyes of a .Net developer; for example, “let” establishes a constant. In the case that it’s used here, it’s a constant pointer.

The target game that I have in mind is a very basic space invaders style game. I’ve already written a similar one in WinJS for the Windows Store so this is kind of a coding kata for me.

As a very loose plan, I intend to do the following:

Step 2: How does this code run on a simulator and on a device?
Step 3: Make an object move across the screen
Step 4: Allow control of said object
Step 5: Collision
Step 6: Graphics
Step 7: Score

Some of these steps may be spread over more than one post – it depends how hard they end up being.

Web API Routing – The Basics

Working with API projects, it’s easy to miss some key rules about the routing. This post is basically the result of some that I missed, and subsequent the investigation. It covers some very basic routing rules, and it certainly not intended to be an exhaustive guide.

.Net Framework

Starting with a .Net Framework Web API, let’s create a new web app:

And add a new controller:

Here’s the code for the controller; as you will see, it’s massively complex, but the good news is that you only need to pay attention to the name of the action, and the code inside it:

public class TestController : ApiController
{
    [HttpGet]
    public IHttpActionResult TestAction()
    {
        return Ok("TestAction Performed");
    }
}

Let’s run the project and navigate to the URL:

How did I know that was the URL? It’s magic, and you can buy some of that magic by sending a cheque for the low, low price of $25 to the address shown at the bottom of the screen.

Actually, it’s defined in WebApiConfig.cs:

Parameters

Where there is more than a single function, one surprising (to me) feature is that the parameters that it accepts is more important to the routing than the name of the controller. Here’s a second action with a parameter:

[HttpGet]
public IHttpActionResult TestAction2(string test)
{
    return Ok("TestAction2 Performed");
}

… and here’s it working:

However, should I not give it the parameter that it craves, it hides away, and instead, we get the first function that’s no too fussy about parameters:

It doesn’t even matter whether I just put some drivel as the controller name; the first criteria is the parameter:

This is because, according to this it follows these criteria:

The default implementation is provided by the ApiControllerActionSelector class. To select an action, it looks at the following:
• The HTTP method of the request.
• The “{action}” placeholder in the route template, if present.
• The parameters of the actions on the controller.

So, if we add the {action} placeholder, that ensures that it uses the correct method:

public static void Register(HttpConfiguration config)
{
    // Web API configuration and services
 
    // Web API routes
    config.MapHttpAttributeRoutes();
 
    config.Routes.MapHttpRoute(
        name: "DefaultApi",
        //routeTemplate: "api/{controller}/{id}",
        routeTemplate: "api/{controller}/{action}/{id}",
        defaults: new { id = RouteParameter.Optional }
    );
}

Otherwise, we get a best guess based on the parameters.

.Net Core Web API

The rules have changed since switching to .Net Core; WebApiConfig has gone and, in its place, it a localised routing system.

Here, you tell the class how to handle routing; for example, the following:

[Route("api/[controller]")]

Will result anything decorated with HttpGet being called when the controller is called. The parameters must be explicitly decorated; so passing no parameters would look like this:


[HttpGet]
public string OneTest()
{
    return "TestOne";
}

Whereas, a single parameter would look like this:


[HttpGet("{id}")]
public string aaa(int id)
{
    return "value aaa";
}

If you duplicate the signatures then they are not found. As with the framework version, you can simply tell it to look to the action name that you give it:

[Route("api/[controller]/[action]")]
public class TestController : Controller
{
    [HttpGet]
    public IEnumerable<string> TestActionOne()
    {
        return new string[] { "one value1", "value2" };
    }
 
    [HttpGet]
    public string TestActionTwo()
    {
        return "two value";
    }

But, again, it pays no attention to parameters until you decorate it correctly.

References

https://docs.microsoft.com/en-us/aspnet/core/fundamentals/routing

Mocking IPrinciple.Identity and Claims in NSubstitute

In ASP.Net, there is a concept of an identity. Built on top of this is an authentication system based on claims; allowing applications to implement a claims based authentication system. That is, I can determine if my user has “Administrator” privileges in the following syntax:

var claim = ClaimsIdentity.FindFirstValue("Administrator");

For more information about how claims work, see this excellent explanation. This post is not really concerned with how claims work, but rather, how to mock them out; which is much more difficult than you might guess.

In the references below, you’ll see a number of different strategies to mock out the claims and principle objects. There also seems to be a loose consensus that even attempting to do this is folly. However, I’ve cobbled together a set of mocks using NSubstitute that work. I’m not claiming that they work in all cases, or that they will work in any situation other than the specific one that I am trying to solve; but it did work for that, and so I thought it useful enough to share.

var myController = new MyController();
 
var mockClaim = new Claim("Administrator", "test");
 
var identity = Substitute.For<ClaimsIdentity>();
identity.Name.Returns("test");
identity.IsAuthenticated.Returns(true);
identity.FindFirst(Arg.Any<string>()).Returns(mockClaim);
 
var claimsPrincipal = Substitute.For<ClaimsPrincipal>();
claimsPrincipal.HasClaim(Arg.Any<string>(), Arg.Any<string>()).Returns(true);
claimsPrincipal.HasClaim(Arg.Any<Predicate<Claim>>()).Returns(true);
claimsPrincipal.Identity.Returns(identity);
 
var httpContext = Substitute.For<HttpContextBase>();            
httpContext.User.Returns(claimsPrincipal);
 
var controllerContext = new ControllerContext(
    httpContext, new System.Web.Routing.RouteData(), myController);           
 
myController.ControllerContext = controllerContext;
 
// Act
var result = myController.TestMethod();
 
// Assert
// . . .

Remember that this is only necessary if you are trying to access claims based on the identity within the `TestMethod()`. Also, I’ll remind the reader that I assert only that this worked in the specific situation that I needed it to, but it’s probably a good starting point for others.

References

https://volaresystems.com/blog/post/2010/08/19/Dont-mock-HttpContext

http://nsubstitute.github.io/help/set-return-value/

https://stackoverflow.com/questions/1389744/testing-controller-action-that-uses-user-identity-name

https://stackoverflow.com/questions/13579519/mock-authenticated-user-using-moq-in-unit-testing

https://stackoverflow.com/questions/14190066/is-there-any-way-i-can-mock-a-claims-principal-in-my-asp-net-mvc-web-application

https://stackoverflow.com/questions/22762338/how-do-i-mock-user-identity-getuserid/23960592

https://dotnetcodr.com/2013/02/11/introduction-to-claims-based-security-in-net4-5-with-c-part-1/

Asynchronous Debugging

Everyone who has spent time debugging errors in code that has multiple threads knows the pain of pressing F10 and seeing the cursor jump to a completely different part of the system (that is, everyone who has ever tried to).

There are a few tools in VS2017 that make this process slightly easier; and this post attempts to provide a brief summary. Obviously the examples in this post are massively contrived.

Errors

Let’s start with an error occurring inside a parallel loop. Here’s some code that will cause the error:

static void Main(string[] args)
{
    Console.WriteLine("Hello World!");
 
    Parallel.For(1, 10, (i) => RunProcess(i));
 
    Console.ReadLine();
}
 
static void RunProcess(int i)
{
    Task.Delay(500).GetAwaiter().GetResult();
 
    Console.WriteLine($"Running {i}");
 
    if (i == 3) throw new Exception("error");
}

For some reason, I get an error when a few of these threads have started. I need a tool that tells me some details about the local variables in the threads specifically. Enter the Parallel Watch Window:

Launch Parallel Watch Window

Figure 1 – Launch Parallel Watch Window

This gives me a familiar interface, and tells me which thread I’m currently on:

Parallel Watch Window

Figure 2 – Parallel Watch Window

However, what I really want to see is the data local to the thread; what if I put “i” in the “Add Watch” cell:

Add a watch

Figure 3 – Add a watch

As you can see, I have a horizontal list of watch expressions, so I can monitor variables in multiple threads at a time.

Flagging a thread

We know there’s an issue with one of these threads, so one possibility is to flag that thread:

Flagging a thread

Figure 4 – Flagging a thread

Then you can select to show only flagged threads:

Filter flagged threads

Figure 5 – Filter flagged threads

Freezing non-relevant threads

The flags help to only trace the threads that you care about, but if you want to only run the threads that you care about, you can freeze the other threads:

Freeze Thread

Figure 6 – Freeze Thread

Once you’ve frozen a thread, a small pause icon appears, and that thread will stop:

Frozen Thread

Figure 7 – Frozen Thread

In order to freeze other threads, simply highlight all the relevant threads (Ctrl-A) and select Freeze.

It’s worth remembering that you can’t freeze a thread that doesn’t exist yet (so your breakpoint in a Parallel.For loop might only show half the threads).

Manual thread hopping

By using freeze, you can stop the debug message from jumping between threads. You can then manually control this process by simply selecting a thread and “Switch To Frame”:

Figure 8 – Switch to Frame

You can switch to a frozen frame, but as soon as you try to progress, you’ll flip back to the first non-frozen frame (unless you thaw it). The consequence of this is that, it is possible to switch to a frozen frame, freeze all other frames and then press F10 – you’re program will then stop dead.

Stack Trace

In a single threaded application (and in a multi-threaded application), you can always view the stack trace of a given line of executing code. There is also a Parallel Stack trace:

Parallel Stacks

Figure 9 – Parallel Stacks

Selecting any given method will give us the active threads, and allow switching:

Active Threads

Figure 10 – Active Threads

Parallel Stack Trace – Task View

The above view gives you a view of the created threads for your program; but most of the time, you won’t care what threads are created; only the tasks that you’ve spawned (they are not necessarily a 1 – 1 relationship. You can simply switch the view in this window to view Tasks instead:

Task View

Figure 11 – Task View

Tasks & Threads Windows

There is a tool that allows you to view all active, blocked and scheduled tasks:

Tasks Window

Figure 12 – Tasks Window

This allows you to freeze an entire task, switch to a given task, and Freeze All But This:

Freeze All But This

Figure 13 – Freeze All But This

There is an equivalent window for Threads. It is broadly the same idea; however, it does have one feature that the Tasks window does not, and that it the ability to rename a thread:

Rename a Thread

Figure 14 – Rename a Thread

Flags

The other killer feature both of these windows have is the flag feature. Simply flag a thread, switch to it, and then select “Show Only Flagged Threads” (little flag icon). If you now remove the breakpoints, you can step through only your thread or task!

Breakpoints

So, what to do where you have a breakpoint that you might only wish to fire for a single thread? Helpfully, the breakpoints window has a filter feature:

Filter breakpoints on thread Id

Figure 15 – Filter Breakpoints

References

https://msdn.microsoft.com/en-us/library/dd554943.aspx

https://stackoverflow.com/questions/5304752/how-to-debug-a-single-thread-in-visual-studio

Short Walks – Instantiating an Object Without calling the Constructor

One of the things that caught my attention at DDD North was the mention of a way to instantiate an object without calling its constructor.

Disclaimer

Typically, classes have code in their constructors that are necessary for their functionality, so you may find that doing this will cause your program to fall over.

System.Runtime.Serialization

The title of the namespace is probably the first thing that betrays the fact that you shouldn’t be doing this; but we’re already halfway down the rabbit hole!

Here’s some code that will create a class using reflection the normal way:

    static void Main(string[] args)
    {
        var test = Activator.CreateInstance<MyTestClass>();
        test.MyMethod();

        Console.WriteLine("Hello World!");
        Console.ReadLine();
    }

    public class MyTestClass
    {
        public MyTestClass()
        {
            Console.WriteLine("MyTestClass Initialise");
        }

        public string test1 { get; set; }

        public void MyMethod()
        {
            Console.WriteLine("Test MyMethod.");
        }
    }

The output is:

And here’s the code that circumvents the constructor:

        static void Main(string[] args)
        {
            var test2 = FormatterServices.GetUninitializedObject(typeof(MyTestClass)) as MyTestClass;
            test2.MyMethod();

            Console.WriteLine("Hello World!");
            Console.ReadLine();
        }

And we haven’t invoked the constructor:

DDD North 2017

DDD North is held every year at around this time. This year, it was on 14th October at Bradford University. In terms of the sessions that I saw, I think it’s probably amongst the best that I’ve attended!

This is a rundown of the sessions and anything I thought I might need to look into further (which, to be fair, is most of it!)

Session 1 – Nathan Gloyn – Microservices

Nathan started this talk by saying that, if you worked in the world of Microservices, you wouldn’t learn anything. This quickly became blatantly untrue (at least for me).

Amongst the talk, he mentioned the need for an external config management system, such as ZooKeeper. The reason being that, should you try to keep the config in a web.config type file (and use transforms to … transform them), you risk having to redeploy an entire system for a configuration update.

That you should log in a system that is running in a server that you have no direct access to is obvious, but he suggested keeping correlation Ids on the messages, so that the logs can trace a message as it travels through the system.

Regarding data updates, the suggestion was to keep the data updates separate from the app updates; and he came up with the intriguing idea of versioning the messages, so that the system can effectively self-update; for example, a message arrives in a subsystem saying it’s a version 1 message; the system realises that the latest version is 2, and so the update script is run then. This allows the system to gradually update itself.

Session 2 – David Whitney – Metaprogramming

This was a bit of a strange talk – as it started off, I couldn’t really work out what to make of it, as it seemed to be a “Have you heard about this thing called reflection?” But as the talk progressed, especially when the creation of a unit test framework was demonstrated in around 20 lines of code, the talk suddenly became very interesting.

One very interesting technique was that you could use reflection to register a set of default implementations for interfaces. Obviously, whether this would work in a particular project depends on why you’re using an interface in the first place, but in most cases, you’ll just want to be able to mock out your interface for a unit test, so you’ll have a one-to-one relationship between interface and implementation.

The other point of note was a mention of a method that allowed you to reflexively create a type without calling a constructor. I noted this down as CreateUnallocatedType, but for the life of me, I can’t find a reference to it anywhere. Please add to the comments or tweet me if you know what this is actually called.

Session 3 – Stephen Haunts – Scaling Agile

I hadn’t realised this until the talk started, but Stephen Haunts was unwittingly the person that got me interested in message queuing (through his Pluralsight course). The talk was effectively a review of how Spotify has scaled an agile team.

Amongst the overview of the specific scrum terms, I was pleased to see that Stephen referred back to the agile manifesto which, IMHO, is something that is frequently forgotten about (or in some cases not even known about) in some companies that claim to implement scrum.

One interesting concept was that of a feature train: the idea being that you work on your feature and, when it’s ready, it gets shipped in the next release. Stephen introduced us to a tool called Launch Darkly – which is a paid for product that allows feature switching.

Other established methods for scaling scrum teams that he mentioned were Scrum Nexus and Scaled Agile.

Session 4 – Stuart Lang – Async

https://speakerdeck.com/slang25/async-in-c-number-the-good-the-bad-and-the-ugly

This was probably my favourite talk of the day although, after dinner is possible the worse spot to have, as everyone is ready for a little nap. However, Stuart kept the subject alive and interesting.

The main focus of the talk was the SynchronisationContext – specifically focusing on deadlocks. The principle being that, while the base implementation of the SynchronisationContext (as used in a console app or a unit test) allows multiple delegates to be executed at any one time, all of the derivitives (WinForms, WPF and ASP.NET) do not. This means that, as much as it might be tempting to try these things out in a console app, or try to cover them with Unit Tests, the behaviour will change as soon as you drop it into your app.

Any app using .Result(), .GetAwaiter.GetResult(), .Wait() will be susceptible to deadlocks.

There were some workarounds that he mentioned; they were:
– SetSynchronisationContext(null)
– .ConfigureAwait(false)
– Task.Run()

They all clear the synchronisation context, preventing a deadlock.

There is an Async analyser in Rosyln called Async006.

There is an open source impementation of a Context Free Task. It can be installed from NuGet, but at the time of writing, is still in a very early release.

The star of the show here is Microsoft.VisualStudio.Threading. It provides access to a JoinableTaskFactory, which implements the behaviour that you would expect if you did call something like .Result(). That is, it executes the asynchronous code, synchronously. This means, not only do you not block yourself, but you don’t spin up a load of pointless threads in the process.

Finally, ASP.Net Core does not have this problem, because of the way that it implements the SynchronisationContext.

Session 5 – Zinat Wali – Alexa

The last session was a talk on Alexa, and how you might write a routine that will report, and then predict the pollen count. The data training model demonstrated would be familiar to people that have seen Azure’s version of the same thing.

Adding to an Existing Azure Blob

In this post I briefly cover the concept of Storage Accounts and Blob Storage; however, there are more to blobs than this simple use case. In this post, I’ll explore creating a blob file from a text stream, and then adding to that file.

As is stated in the post referenced above, Azure provides a facility for storing files in, what are known as, Azure Blobs.

In order to upload a file to a blob, you need a storage account, and a container. Setting these up is a relatively straightforward process and, again, is covered in the post above.

Our application here will take the form of a simple console app that will prompt the user for some text, and then add it to the file in Azure.

Set-up

Once you’ve set-up your console app, you’ll need the Azure NuGet Storage package.

Also, add the connection string to your storage account into the app.config:

<connectionStrings>
    <add name="Storage" connectionString="DefaultEndpointsProtocol=https;AccountName=testblob;AccountKey=wibble/dslkdsjdljdsoicj/rkDL7Ocs+aBuq3hpUnUQ==;EndpointSuffix=core.windows.net"/>
</connectionStrings>

Here’s the basic code for the console app:

static void Main(string[] args)
{
    Console.Write("Please enter text to add to the blob: ");
    string text = Console.ReadLine();
 
    UploadNewText(text);
 
    Console.WriteLine("Done");
    Console.ReadLine();
}

I’ll bet you’re glad I posted that, otherwise you’d have been totally lost. The following snippets are possible implementations of the method UploadNewText().

Uploading to BlockBlob

The following code will upload a file to a blob container:

string connection = ConfigurationManager.ConnectionStrings["Storage"].ConnectionString;
string fileName = "test.txt";
string containerString = "mycontainer";
 
using (MemoryStream stream = new MemoryStream())
using (StreamWriter sw = new StreamWriter(stream))
{
    sw.Write(text);
    sw.Flush();
    stream.Position = 0;
 
    CloudStorageAccount storage = CloudStorageAccount.Parse(connection);
    CloudBlobClient client = storage.CreateCloudBlobClient();
    CloudBlobContainer container = client.GetContainerReference(containerString);
    CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
    blob.UploadFromStream(stream);
}

(note that the name of the container in this code is case sensitive)

If we have a look at the storage account, a text file has, indeed been created:

New Blob

But, what if we want to add to that? Well, running the same code again will work, but it will replace the existing file. To prove that, I’ve changed the text to “Test data 2” and run it again:

Test Data

So, how do we update the file? Given that we can update it, one possibility is to download the existing file, add to it and upload it again; that would look something like this:

string connection = ConfigurationManager.ConnectionStrings["Storage"].ConnectionString;
string fileName = "test.txt";
string containerString = "mycontainer";
 
CloudStorageAccount storage = CloudStorageAccount.Parse(connection);
CloudBlobClient client = storage.CreateCloudBlobClient();
CloudBlobContainer container = client.GetContainerReference(containerString);
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
 
using (MemoryStream stream = new MemoryStream())
{
    blob.DownloadToStream(stream);
 
    using (StreamWriter sw = new StreamWriter(stream))
    {
        sw.Write(text);
        sw.Flush();
        stream.Position = 0;
 
        blob.UploadFromStream(stream);
    }
}

This obviously means two round trips to the server, which isn’t the best thing in the world. Another possible option is to use the Append Blob…

Azure Append Blob Storage

There is a blob type that allows you to add to it without actually touching it; for example:

string connection = ConfigurationManager.ConnectionStrings["Storage"].ConnectionString;
string fileName = "testAppend.txt";
string containerString = "mycontainer";
 
CloudStorageAccount storage = CloudStorageAccount.Parse(connection);
CloudBlobClient client = storage.CreateCloudBlobClient();
CloudBlobContainer container = client.GetContainerReference(containerString);
CloudAppendBlob blob = container.GetAppendBlobReference(fileName);
if (!blob.Exists()) blob.CreateOrReplace();
 
using (MemoryStream stream = new MemoryStream())
using (StreamWriter sw = new StreamWriter(stream))
{
    sw.Write("Test data 4");
    sw.Flush();
    stream.Position = 0;
 
    blob.AppendFromStream(stream);                
}

There are a few things to note here:

  • The reason that I changed the name of the blob is that you can’t append to a BlockBlob (at least not using an AppendBlob); so it has to have been created for the purpose of appending.
  • While UploadFromStream will just create the file if it doesn’t exist, with the AppendBlob, you need to do it explicitly.

PutBlock

The final alternative here is to use PutBlock. This can bridge the gap, by allowing the addition of blocks into an existing block blob. However, you either need to maintain the Block ID list manually, or download the existing block list; here’s an example of creating, or adding to a file using the PutBlock method:

string connection = ConfigurationManager.ConnectionStrings["Storage"].ConnectionString;
string fileName = "test4.txt";
string containerString = "mycontainer";
 
CloudStorageAccount storage = CloudStorageAccount.Parse(connection);
CloudBlobClient client = storage.CreateCloudBlobClient();
CloudBlobContainer container = client.GetContainerReference(containerString);
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
 
ShowBlobBlockList(blob);
 
using (MemoryStream stream = new MemoryStream())
using (StreamWriter sw = new StreamWriter(stream))
{
    sw.Write(text);
    sw.Flush();
    stream.Position = 0;
 
    double seconds = (DateTime.Now - new DateTime(2000, 1, 1)).TotalSeconds;
    string blockId = Convert.ToBase64String(
        ASCIIEncoding.ASCII.GetBytes(seconds.ToString()));
 
    Console.WriteLine(blockId);
    //string blockHash = GetMD5HashFromStream(bytes);                
 
    List<string> newList = new List<string>();
    if (blob.Exists())
    {
        IEnumerable<ListBlockItem> blockList = blob.DownloadBlockList();
 
        newList.AddRange(blockList.Select(a => a.Name));
    }
 
    newList.Add(blockId);
 
    blob.PutBlock(blockId, stream, null);
    blob.PutBlockList(newList.ToArray());
}

The code above owes a lot to the advice given on this Stack Overflow question.

In order to avoid conflicts in the Block Ids, I’ve used a count of seconds since an arbitrary date. Obviously, this won’t work in all cases. Further, it’s worth noting that the code above still does two trips to the server (it has to download the block list).

The commented MD5 hash allows you to provide some form of check on the data being valid, should you choose to use it.

What is ShowBlobBlockList(blob)?

The following function will give some details relating to the existing blocks (it is shamelessly plagiarised from here):

public static void ShowBlobBlockList(CloudBlockBlob blockBlob)
{
    if (!blockBlob.Exists()) return;
 
    IEnumerable<ListBlockItem> blockList = blockBlob.DownloadBlockList(BlockListingFilter.All);
    int index = 0;
    foreach (ListBlockItem blockListItem in blockList)
    {
        index++;
        Console.WriteLine("Block# {0}, BlockID: {1}, Size: {2}, Committed: {3}",
            index, blockListItem.Name, blockListItem.Length, blockListItem.Committed);
    }
}

Summary

Despite being an established technology, these methods and techniques are sparsely documented on the web. Obviously, there are Microsoft docs, and they are helpful, but, unfortunately, not exhaustive.

References

https://stackoverflow.com/questions/33088964/append-to-azure-append-blob-using-appendtextasync-results-in-missing-data

https://docs.microsoft.com/en-us/rest/api/storageservices/understanding-block-blobs–append-blobs–and-page-blobs

http://www.c-sharpcorner.com/UploadFile/40e97e/windows-azure-blockblob-putblock-method/

https://docs.microsoft.com/is-is/rest/api/storageservices/put-block

https://www.red-gate.com/simple-talk/cloud/platform-as-a-service/azure-blob-storage-part-4-uploading-large-blobs/

https://stackoverflow.com/questions/46368954/can-putblock-be-used-to-append-to-an-existing-blockblob-in-azure

Azure Recommendations

Azure provides a number of pre-configured machine learning services out of the box. One of these (still in Beta at the time of writing this) is Recommendations. The idea being that it will try to work out what, given a list of items, you would prefer, based knowledge about your habits. There’s a lot of information on line about this, but briefly, it can work out what your preference is based a combination of your past activity, and the past activity of others that have shown an interest in the same item.

Obviously, the “items” could be products, films, aardvarks, or sheep; Azure doesn’t know anything about the content of what its recommending; if you’ve bought* “A” in the past, and 75% of everyone else that has bought “A” has also bought “B” then there’s a chance you’ll want “B”. “A” could be an apple, and “B” could be a pair of sunglasses; so obviously, you need to be careful about the data that you feed it.

Recommendations API

The first thing to note is that the Recommendations API represents an earlier attempt to implement this by Microsoft, and is due to be discontinued early next year (2018). If you try to use this to follow any of the online tutorials then you’ll get into a world of hurt.

Deploy Recommendations

The new method of creating a recommendations service is via a wizard (which, I believe behind the scenes, builds a custom ARM template). This is the start of the deployment, and gives you a screen similar to the following (once you’ve logged in):

As you can see, there’s clearly some re-branding in progress here; anyway, complete the form and create the service.

Another thing that has changed in the new version of this is that the free pricing tier has disappeared:

After a few screens, it starts the deployment process:

The next screen that is displayed shows all the connection strings and keys in one handy reference:

… they are just below this screenshot.

This should create four separate services:

Sample Project

Microsoft provides a sample project that should work out of the box (they actually provide more than one – some of which work better than others). This one uses AutoRest, but there’s another referenced at the bottom of this post.

In this project, open Recommendations.Sample.Program.cs and, at the top of the main function, enter the details that you noted after the creation of your service. If you didn’t note them, then you can still find them. You’ll notice that four separate services were created: AppService, StorageAccount, App Insights and an App Service Plan.

recommendationsEndPointUri

Is found in the URL of the App Service:

apiAdminKey

Is found in the Application Settings of the App Service:

connectionString

Is the connection string of the storage account:

If you run this now, you should find that it will process and score the recommendations:

So, for example, we can see from the results above that people that bought DHF-01159 are recommended to buy DHF-01055 (although it doesn’t seem very convinced).

Footnotes

* The term that Azure uses here is “Purchase”. Different actions have different weighting (configurable), but by default, you would assume that buying something is more important than, for example, clicking on it (“Click” is another action). These actions can mean anything you choose; in the sheep example above, “Purchase” might mean shearing, and “Click” might mean photographing.

References

http://pmichaels.net/2017/08/06/deploying-azure-recommendation-service-using-arm-template/

https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-overview-of-diagnostic-logs

https://docs.microsoft.com/en-us/azure/cognitive-services/cognitive-services-recommendations-ui-intro

https://gallery.cortanaintelligence.com/Tutorial/Recommendations-Solution

https://github.com/Microsoft/Cognitive-Recommendations-Windows

Short Walks – Running an Extension Method on a Null Item

I came across this issue recently, and realised that I didn’t fully understand extension methods. My previous understanding was that an extension method was simply added to the original class (possible in the same manner that weavers work); However, a construct similar to the following code changed my opinion:

class Program
{
    static void Main(string[] args)
    {
        var myList = GetList();            
        var newList = myList.Where(
            a => a.IsKosher());
        var evaluateList = newList.ToList();
 
        foreach(var a in evaluateList)
        {
            Console.WriteLine(a.Testing);
        }
    }
 
    static IEnumerable<TestClass> GetList()
    {
        return new List<TestClass>()
        {
            new TestClass() {Testing = "123"},
            null
        };
    }
}
 
public class TestClass
{
    public string Testing { get; set; }
}
 
public static class ExtensionTest
{
    public static bool IsKosher(this TestClass testClass)
    {
        return (!string.IsNullOrWhiteSpace(testClass.Testing));
    }
}

As you can see from the code, GetList() returns a null class in the collection. If you run this code, you’ll find that it crashes inside the extension method, because testClass is null.

A note on Linq

If you’re investigating this in the wild, you might find it particularly difficult because of the was that Linq works. Even though the call to the extension method is on the line above, the code doesn’t get run until you actually use the result (in this case, via a ToList()).

New understanding

As I now understand it, extension methods are simply a nice syntactical way to use a static method. That is, had I simply declared my IsKosher method as a standard static method, it would behave exactly the same. To verify this, let’s have a look at the IL; here’s the IL for my function above:

IL Code for extension method

And here’s the IL for the same function as a standard static method:

IL code for static method

The only difference is the line at the top of the extension method calling the ExtensionAttribute constructor.

References

https://stackoverflow.com/questions/847209/in-c-what-happens-when-you-call-an-extension-method-on-a-null-object