While working on a proof of concept, I needed to create a dummy API, and then run a stress test on it. Whilst the activity itself may seem pointless (creating a templated API and stress testing it), I felt the process might be worth documenting.
Create the API
We’ll start with creating the API:
dotnet new webapi
If you just run this, you should see the following:
The next step is to call that from a console app.
Create a console app to call the API
Add a new console app project to your solution, and replace the code in Program.cs with the following:
class Program
{
static HttpClient client = new HttpClient();
static string path = "https://localhost:44356/weatherforecast";
static async Task Main(string[] args)
{
Console.WriteLine("Press enter to start test");
Console.ReadLine();
string? data = await CallWeatherForecast();
Console.WriteLine(data ?? "No data returned");
}
private static async Task<string?> CallWeatherForecast()
{
HttpResponseMessage response = await client.GetAsync(path);
if (response.IsSuccessStatusCode)
{
string data = await response.Content.ReadAsStringAsync();
return data;
}
return null;
}
}
It’s worth noting that I switched on nullable reference types here (in the csproj file):
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.1</TargetFramework>
<Nullable>enable</Nullable>
</PropertyGroup>
To test this, you’ll need to start both the projects (either select Set Startup Projects… from the solution context menu, or run the API, and then right-click the console app and select Debug -> Start New Instance).
Once you’ve checked that this works, build the solution in release mode (remember, we’re going to be running a stress test, so debug mode will show skewed results).
Call the console app from JMeter
For the stress test, I’m going to use Jmeter. You’ll need to download it from here.
I won’t go into too much detail about how to set this up, but briefly, extract the downloaded zip somewhere, and run the jmeter.bat file. You should be presented with the following screen:
Add a thread group, so we can simulate multiple users:
Then add an OS Process Sampler to run the console app:
Remember to run the API first, then click the green play arrow. You’ll see the users ramp up:
We don’t have any listeners, so the results are, unfortunately lost. Let’s add a couple:
As you can see, we now have some information. How long these calls are taking on average, the error count, etc. The throughput we’re getting is around 3/second… In fact, running a stress test locally on the same machine, it’s difficult to break it, because as the resources get used up, the JMeter process itself suffers, too. This is a good reason to run JMeter from a VM in the cloud.
Whilst it’s quite difficult to kill the service, I certainly managed to slow it down considerably:
These figures are in milliseconds, which means that 90% of calls are taking over 2 minutes. This entire test took around 15 minutes, and around 10 requests per second was about the best it got to (I ran 10 loops of 1000 concurrent users).
There’s a few things you can do to identify where the performance starts to degrade, but I’m more interested in what happens to these figures if I add DB access.
Note: when you’re playing with this, the reports don’t automatically clear each run, so you have to select each, right-click and clear.
Add calls to a DB
Let’s add Entity Framework to our project.
We can then change the controller to allow adding new records and retrieval by date:
public WeatherForecastController(
ILogger<WeatherForecastController> logger,
StressTestDbContext stressTestDbContext)
{
_logger = logger;
_stressTestDbContext = stressTestDbContext;
}
[HttpGet]
public IEnumerable<WeatherForecast> GetNew()
{
var rng = new Random();
return Enumerable.Range(1, 5).Select(index => new WeatherForecast
{
Date = DateTime.Now.AddDays(index),
TemperatureC = rng.Next(-20, 55),
Summary = Summaries[rng.Next(Summaries.Length)]
})
.ToArray();
}
[HttpGet("/[controller]/GetByDate/{dateTime}")]
public IEnumerable<WeatherForecast> GetByDate(DateTime dateTime)
{
var forecasts = _stressTestDbContext.Forecast.Where(a => a.Date.Date == dateTime.Date);
return forecasts;
}
[HttpPost]
public IActionResult Post(WeatherForecast weatherForecast)
{
DailyForecast forecast = new DailyForecast()
{
Date = weatherForecast.Date,
Summary = weatherForecast.Summary,
TemperatureC = weatherForecast.TemperatureC
};
_stressTestDbContext.Add(forecast);
if (_stressTestDbContext.SaveChanges() != 0)
{
return Ok();
}
return BadRequest();
}
Finally, we can change the main program to call those functions:
static async Task Main(string[] args)
{
Console.WriteLine("Press enter to start test");
Console.ReadLine();
ConsoleTitle("CallGetNewWeatherForecast");
string? dataGet = await CallGetNewWeatherForecast();
Console.WriteLine(dataGet ?? "No data returned");
ConsoleTitle("AddForecast");
string? dataAdd = await AddForecast(_rnd.Next(30),
DateTime.Now.AddDays(_rnd.Next(10)), _summaries[_rnd.Next(_summaries.Length)]);
Console.WriteLine(dataAdd ?? "No data returned");
ConsoleTitle("CallGetWeatherForecast");
string? dataGetDate = await CallGetWeatherForecast(DateTime.Now.AddDays(_rnd.Next(10)));
Console.WriteLine(dataGetDate ?? "No data returned");
}
private static void ConsoleTitle(string title)
{
Console.ForegroundColor = ConsoleColor.Blue;
Console.WriteLine(title);
Console.ResetColor();
}
private static async Task<string?> CallGetNewWeatherForecast()
{
HttpResponseMessage response = await client.GetAsync($"{path}");
if (response.IsSuccessStatusCode)
{
string data = await response.Content.ReadAsStringAsync();
return data;
}
return null;
}
private static async Task<string?> CallGetWeatherForecast(DateTime dateTime)
{
string dateString = dateTime.ToString("yyyy-MM-dd");
HttpResponseMessage response = await client.GetAsync($"{path}/GetByDate/{dateString}");
if (response.IsSuccessStatusCode)
{
string data = await response.Content.ReadAsStringAsync();
return data;
}
return null;
}
private static async Task<string> AddForecast(int temperature, DateTime date, string summary)
{
var forecast = new WeatherForecast()
{
TemperatureC = temperature,
Date = date,
Summary = summary
};
HttpResponseMessage response = await client.PostAsJsonAsync($"{path}", forecast);
if (response.IsSuccessStatusCode)
{
string data = await response.Content.ReadAsStringAsync();
return data;
}
return null;
}
To get a sensible reading, you’ll need to do this from an empty database:
For the first run, we’ll do 500 users, and 3 iterations:
The output from the first run is:
And let’s just check that that’s created the records we expected:
EF async calls vs non-async
To satisfy a curiosity that I’ve had for a while, I’m now going to change the update API method to async:
[HttpPost]
public async Task<IActionResult> Post(WeatherForecast weatherForecast)
{
DailyForecast forecast = new DailyForecast()
{
Date = weatherForecast.Date,
Summary = weatherForecast.Summary,
TemperatureC = weatherForecast.TemperatureC
};
_stressTestDbContext.Add(forecast);
if (await _stressTestDbContext.SaveChangesAsync() != 0)
{
return Ok();
}
return BadRequest();
}
Again, 1500 records were created:
Here’s the report:
What an interesting result. Making the update async seems to have slightly reduced the throughput. This is running locally, and I only have a 4 core machine, but I would have expected throughput to slightly increase here, rather than decrease.