In this blog I will demonstrate how to use Azure Event hubs to send and receive messages using python.
Event hubs in the words of Microsoft are:
“Event Hubs is a fully managed, real-time data ingestion service that’s simple, trusted and scalable. Stream millions of events per second from any source to build dynamic data pipelines and immediately respond to business challenges. Keep processing data during emergencies using the geo-disaster recovery and geo-replication features.”
So to get started you’ll need a Azure subscription or you can sign up for a free account here
Once you’ve created your Azure subscription we’ll need to start by creating an Event hubs namespace, so firstly click on the Event Hubs icon or search for this in the Azure search bar.
Once in the Event hubs click add and this will open the Create Namespace pane , in here enter the below:
Name: Anything, but this needs to be unique to Azure so test is not going to work
Pricing: Basic level is more than enough
Subscription: This will be pre-populated with the subscription your logged in under
Resource Group: These are excellent for grouping together items which are related so you’re able to remove these all in one group
Location: Uk South, but you can pick whichever one is closest to you
Once this resource has been provisioned you’ll need to click into the namespace and select Event hubs
Then select to add a new event hub by clicking on the cross at the top and the below Create Event Hub pane will appear.
Simply enter a name for this event hub
Once this is provisioned click to go back into the event hub you’ve just created and select Shared access polices in the left pane under settings
Click Add and on the Add SAS Policy pane that appears add a name and click send and listen and click create
Once this has been provisioned click into the Policy and click to copy the Connection string, paste this somewhere for later.
We’ve now completed the setup from an azure point of view, so onto the Python client and receiver part.
I’ve supplied these two python scripts in my github repo at the following link.
First we need to open the install the relevant python libraries so you’ll need to issue the below pip command in whatever command tool you use, bash or cmd Prompt
Next we need to open and edit the EventHubClient.py & EventHubReceiver.py file and edit the following values in both files like below
Using the connection string you copied earlier, here is mine as an example to help you derive whats needed
ADDRESS = This is concatenation of the endpoint part of the string and the EntityPath
USER = Is the SharedAccessKeyName
KEY = Is the SharedAccessKey
Save and close both files and open a CMD prompt window from the directory where these files are located, and execute a following commands in two different CMD Prompts in this order.
BRINGING IT ALL TOGETHER
If successful, you should start to see the data flowing between the two using the Eventhubs as its streaming queue.
You can then use this as an output for Stream Analytics, blob storage….the world is your data oyster 🙂