The Google Cloud Platform provides a Publish / Subscriber system called ‘PubSub’. In this post I wrote a basic guide on setting up RabbitMQ, and here I wrote about ActiveMQ. In this post I wrote about using the Azure messaging system. Here, I’m going to give an introduction to using the GCP PubSub system.
Introduction
The above systems that I’ve written about in the past are fully featured (yes, including Azure) message bus systems. While the GCP offering is a Message Bus system of sorts, it is definitely lacking some of the features of the other platforms. I suppose this stems from the fact that, in the GCP case, it serves a specific purpose, and is heavily geared toward that purpose.
Other messaging systems do offer the Pub / Sub model. The idea being that you create a topic, and anyone that’s interested can subscribe to the topic. Once you’ve subscribed, you’re guaranteed* to get at least one delivery of the published message. You can also, kind of, simulate a message queue, because more than one subscriber can take message from a single subscription.
Pre-requisites
If you want to follow along with the post, you’ll need to have a GCP subscription, and a GCP project configured.
Topics
In order to set-up a new topic, we’re going to navigate to the PubSub menu in the console (you may be prompted to Enable PubSub when you arrive).
As you can see, you’re inundated with choice here. Let’s go for “Create a topic”:
Cloud Shell
You’ve now created a topic; however, that isn’t the only way that you can do this. Google are big on using the Cloud Shell; and so you can create a topic using that; in order to do so, you select the cloud shell icon:
Once you get the cloud shell, you can use the following command**:
gcloud beta pubsub topics create "test"
Subscriptions and Publishing
You can publish a message now if you like; either from the console:
Or from the Cloud Shell:
gcloud beta pubsub topics publish "test" "message"
Both will successfully publish a message that will get delivered to all subscribers. The problem is that you haven’t created any subscribers yet, so it just dissipates into the ether***.
You can see there are no subscriptions, because the console tells you****:
Let’s create one:
Again, you can create a subscription from the cloud shell:
gcloud beta pubsub topics subscriptions create --topic "test" "mysubscription"
So, we now have a subscription, and a message.
Consuming messages
In order to consume messages in this instance, let’s create a little cloud function. I’ve previously written about creating these here. Instead of creating a HTTP trigger, this time, we’re going to create a function that reacts to something on a cloud Pub/Sub topic:
Select the relevant topic; the default code just writes the test out to the console; so that’ll do:
[code lang=“java”] /** * Triggered from a message on a Cloud Pub/Sub topic. * * @param {!Object} event The Cloud Functions event. * @param {!Function} The callback function. */ exports.subscribe = function subscribe(event, callback) { // The Cloud Pub/Sub Message object. const pubsubMessage = event.data;
// We’re just going to log the message to prove that // it worked. console.log(Buffer.from(pubsubMessage.data, ‘base64’).toString());
// Don’t forget to call the callback. callback(); };
So, now we have a subscription:
Let's see what happens when we artificially push a message to it.
If we now have a look at the Cloud Function, we can see that something has happened:
And if we select "View Logs", we can see what:
It worked! Next…
# Create Console App
Now we have something that will react to a message, let's try and generate one programmatically, in C# from a console app. Obviously the first thing to do is to install a NuGet package that isn't past the beta stage yet:
> Install-Package Google.Cloud.PubSub.V1 -Pre
# Credentials
In [this post](http://pmichaels.net/2017/12/17/google-cloud-datastore-setting-new-datastore-accessing-console-application/) I described how you might create a credentials file. You'll need to do that again here (and, I think anywhere that you want to access GCP from outside of the cloud).
In APIs & Services, select "Create credentials":
Again, select a JSON file:
The following code publishes a message to the topic:
``` csharp
static async Task Main(string[] args)
{
Environment.SetEnvironmentVariable(
"GOOGLE\_APPLICATION\_CREDENTIALS",
Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "my-credentials-file.json"));
GrpcEnvironment.SetLogger(new ConsoleLogger());
// Instantiates a client
PublisherClient publisher = PublisherClient.Create();
string projectId = "test-project-123456";
var topicName = new TopicName(projectId, "test");
SimplePublisher simplePublisher = await SimplePublisher.CreateAsync(topicName);
string messageId = await simplePublisher.PublishAsync("test message");
await simplePublisher.ShutdownAsync(TimeSpan.FromSeconds(15));
}
And we can see that message in the logs of the cloud function:
Permissions
Unless you choose otherwise, the service account will look something like this:
The Editor permission that it gets by default is a sort of God permission. This can be fine-grained by removing that, and selecting specific permissions; in this case, Pub/Sub -> Publisher. It’s worth bearing in mind that as soon as you remove all permissions, the account is removed, so try to maintain a single permission (project browser seems to be suitably innocuous).
Footnotes
* Google keeps messages for up to 7 days, so the guarantee has a time limit.
** gcloud may need to be initialised. If it does then:
gcloud init
gcloud components install beta
*** This is a big limitation. Whilst all topic subscriptions in other systems do work like this, in those systems, you have the option of a queue - i.e. a place for messages to live that no-one is listening for.
**** If you create a subscription in the Cloud Shell, it will not show in the console until you F5 (there may be a timeout, but I didn’t wait that long). The problem here is that F5 messes up the shell window.