Debugging Recommendations Engine

Here I wrote about how to set-up and configure the MS Azure recommendations engine.

One thing that has become painfully apparent while working with recommendations is how difficult it is to work out what has gone wrong when you don’t get any recommendations. The following is a handy check-list for the next time this happens to me… so others may, or may not find this useful*:

1. Check the model was correctly generated

Once you have produced a recommendations model, you can access that model by simply navigating to it. The url is in the following format:

{recommendations uri}/ui

For example:

https://pcmrecasd4zzx2asdf.azurewebsites.net/ui

This gives you a screen such as this:

The status (listed in the centre of the screen) tells you whether the build has finished and, if so, whether it succeeded or not.

If the build has failed, you can select that row and drill into, and find out why.

In the following example, there is a reference in the usage data, to an item that is not in the catalogue.

Other reasons that the model build may fail include invalid, corrupt or missing data in either file.

2. Check the recommendation in the interface

In order to exclude other factors in your code, you can manually interrogate the model directly by simply clicking on the “Score” link above; you will be presented with a screen such as this:

In here, you can request direct recommendations to see how the model behaves.

3. Volume

If you find that your score is consistently returning as zero, then the issue may be with the volume of usage data that you have provided. 1k** rows of usage data is the sort of volume you should be dealing with; this statistic was based on a catalogue of around 20 – 30 products.

4. Distribution

The number of users matters – for the above figures, a minimum of 15** users was necessary to get any scores back. If the data sample is across too small a user base, it won’t return anything.

Footnotes

* Although this post is written by me, and is for my benefit, I stole much of its content from wiser work colleagues.

** Arbitrary values – your mileage may vary.

Leave a Reply

Your email address will not be published. Required fields are marked *