I’ve recently been talking (and blogging) a lot about App Insights for Azure. One of the questions or topics that comes up when I speak about this is cost. I think broadly, the cost is comparable to other services of the same ilk (although I may write a future post on some things that can be done to manage the cost); however, what I wanted to cover in this post was the cost of not logging
Typically, the cost of logging comes (largely) from data ingestion - if your costs are high, one option is to reduce the amount of logs that you’re writing. This will work: if the cost is (for the sake of argument) £2 / GB of log ingestion, and you’re writing 10GB / day, you’ll have to pay £20 / day for ingestion charges; if you reduce that logging to 10GB, the cost will go down.
In this post, I’m not interested in the different kinds of logs - that is, I’m only talking about outputting data regarding the behaviour and performance of your running process; not whether that is a log, a trace, a metric, or a carrier pigeon. There are dozens, if not hundreds, of articles out there that talk about tracing, logging, and carrier pigeons. I am, however, interested in what happens if you stop, or reduce that output.
This doesn’t apply only to logging either, but we’ll come back to that shortly; for now, let’s make the following assumptions:
- Logging costs are £2 / GB
- You log 100 GB of data every day
- Logging (or at least the kind that you’re doing) definitively assists in the diagnosis of a problem by 30% (that is, diagnosing a problem is 30% faster)
- We are dealing with an e-commerce website
Obviously, this is an imaginary scenario, and everything will be over simplified to make my point.
The question here is: is it worth logging, or should you remove logging from the system? Logging is costing £200 / day - that’s around £6k / month.
In order to answer this question, we need to answer a different one: what is the cost of not logging? To answer that question, we need tk know what up time is worth. We’re an e-commerce site, so that’s an easy question to answer: what’s the turnover of the business?
The annual turnover of Amazon UK for 2022 was around £6.5B. This randomly Googled article says that a new e-commerce site can have over £700k annual turnover (there’s a bit of guesswork and finger in the air, but the figure isn’t that important). They’re clearly different ends of a spectrum, but the real question is, what is that in a minute?
Amazon
~ £18M / day
~ £740k / hour
~ £12k / minute
Mythical E-commerce Store from that Article
~ £2k / day
~ £80 / hour
~ £1.30 / minute
Why does the minute value matter? Because that’s what you’re losing if the service / site is down. We said it would speed up the recovery by 30% if we had logging, so a one hour issue would take only 40 minutes to resolve. You don’t really need to do the maths to work out that, in the case of Amazon, you could log onto bars of solid gold and it would still be cheap - this would pay for itself in a single outage. In fact, that would save ~£240k - essentially the cost of logging for 3 years.
However, in the smaller example, the outage would need to be much longer than an hour to make it worthwhile. The site would need to be down for at least 3 - 4 hours before the logging was even close to being worthwhile.
This doesn’t only apply to logging; once you have the turnover per minute figure, you can use it to compare disaster recovery times, or pretty much anything where a decision has to be made between reliability and cost.
Disclaimer
There are other (non-financial) impacts of downtime, and some pseudo financial ones, too - for example, if the site was down for a week, would that affect search rankings.
Further disclaimer: I’m not trying to imply that for small businesses logging isn’t worthwhile - it is. Also, if you don’t have any logging at all, you may find yourself completely unable to solve a problem.