Tag: iot edge

NOTE:  updated on 2/14/2019 to add the note about adding the content type and content encoding to the MQTT topic if you want to route messages based on the message body in IoT Edge

Recently on behalf of a customer, I was asked if it was possible to connect Node-Red to Azure IoT Edge.  Node-Red, an IBM sponsored graphical process design tool, is increasingly popular in IoT scenarios.

My first thought was “well sure, there is an IoT Hub ‘node’ for Node-Red, surely it can be used for connecting to IoT Edge”.  Well, looks like it can’t.   For two reasons, 1) it’s not designed to do that (no way to specify the edge box/GatewayHostName) and 2) it’s not very good  .  There are a couple of other options out there, too, but none of the IoT Hub specific ‘nodes’ will work for this use case.

Secondly, and this is really cool, my Azure IoT Global Black Belt peers in Europe have developed an IoT Edge module that hosts Node-Red inside Edge itself, where you can orchestrate your IoT process, and have it drop messages on edgeHub for further edge processing and/or uploading of messages to IoT Hub.   *If* you are interested in hosting and managing Node-Red from inside IoT Edge itself, you should definitely check it out.  It’s really nice.

With that said, a lot of customers (including the one that prompted this post) either already have a Node-Red environment outside of IoT Edge, or just for a host of reasons want to keep them separate.  If so, this post is for you.   Since the IoT Hub ‘node’ won’t seem to work for this use case, we are going to use the MQTT node.  One thing to note – this only addressed device-to-cloud telemetry.  I haven’t (yet) addressed cloud-to-device management/messages/configuration because my customer’s use case didn’t (yet) need it.  Baby steps.

First of all, let’s get some pre-requisites out of the way.

Pre-requisites

For this post, I assume a number of things:

• You’ve already got a Node-Red environment up and going.  If not, follow the directions to do so from the Node-Red site.    I also assume you have some familiarity with Node-Red.  If you don’t, they have some nice tutorials on the site.  Make sure Node-Red is running   (node-red-start from the command prompt)
• You already have Azure IoT Edge setup as a transparent gateway and have confirmed that you can connect to it over the 8883 MQTT using the handy openssl command included in the docs (this is necessary because Node-Red will be connecting to IoT Edge as a “leaf” or “downstream” device).  NOTE:  make sure that whatever name you use in the hostname parameter in your config.yaml file is the same resolvable name that you will use from Node-Red to connect to the IoT Edge box.  If you have to change it in config.yaml to match, restart iot edge before proceeding.
• From your IoT Edge box, copy/download/ftp/whatever the root CA cert that was used to set up the IoT Edge box.  If you used a ‘real’ cert (i.e. from DigiCert, Baltimore, etc), then you’ll need to get the public key cert from them in the form of a pem or crt file.  If you used the convenience scripts from the transparent gateway instructions above, it’s the cert labeled azure-iot-test-only.root.ca.cert.pem in the ‘certs’ folder underneath whatever folder you generated your certs in.   Just grab that file and get it to your local desktop, we’ll need it later.
• Your IoT Edge box is reachable, network-wise, from your Node-Red box.  Can you ping it?  Can you resolve it’s DNS name?  Can you telnet to 8883?

In my case, I ran everything (Node-Red and IoT Edge both) from a single Raspberry Pi that I had laying around.  But beyond the requirements for Node-Red and IoT Edge themselves, there is no requirement for specific OS’es or co-location, as long as they can see each other on the network, and IoT Edge can get to IoT Hub.

Setup

Ok, enough caveating and pre-req’ing, let’s do some setup.

One of the things you will need is an IoT Hub device identity in the hub to represent our Node-Red flow.  If you haven’t already, create an IoT Device to represent our Node-Red “device” (the flow).  Go ahead and capture the “device id”, and URI of the Hub (something.azure-devices.net).

You will also need a Shared Access Signature, a token that authenticates the device to IoT Hub.  The easiest way to generate one is to open up a cloud shell instance in the azure portal.   To do that, click on the Cloud Shell button in the top nav part of the portal.  It looks like this:

You may have to go through some preliminary setup (select a blob storage account, etc) if it’s the first time you’ve done it.  You can use either Powershell or Bash.  I prefer Bash myself, but both work.

Once the cloud shell is created and you have a command prompt, the first thing we have to to do install the azure iot hub extension for the azure cli interface.   You can do this by running this command

az extension add –name azure-cli-iot-ext

You only have to do it the first time, it ‘sticks’ for subsequent times you use the cloud shell.  Once this is done, you can generate your SAS token with this command:

az iot hub generate-sas-token –d [device id]-n [hub name] –du [duration]

where [device id] is the device id from the device you created earlier, [hub name] is the short name of your IoT Hub (with the .azure-devices.net) and [duration] is the amount of time, in seconds, you want your toke to be valid.  Shorter values are more secure, but longer values are easier to manage (you have to re-create and re-configure the token in Node-Red when it expires).  It’s up to you to figure out your best balance between security and convenience .  I will remain silent on the topic here.

Once you generate your token, you need to copy it somewhere, as we’ll need it later.  The process will look something like this (click on the picture to enlarge):

You need the part that starts with “Shared Access Signature”, without the quotes.  Highlighted in yellow in the picture.     NOTE in the picture that ‘sdbiothub1’ is the name of my IoT Hub used in this demo.  Also note that I used a duration of 60 seconds, so you can’t monkey with my IoT Hub.  Not that you would ever do anything like that.

Ok, we have all the information we need to get started.

IoT Edge and IoT Hub debug prep

There’s really nothing else we need to do in order to get IoT Edge and IoT Hub ready, but we’ll do a couple of things to have them ready in the background to both debug our Node-Red setup and see the messages flow through if when we are successful.

On the IoT Edge box, run

iotedge logs –f edgeHub

This brings the edgeHub logs up and ‘follows’ them so we’ll be able to see our Node-Red box connect.

In the cloud shell that you have open in Azure, run this command

az iot hub monitor-events –n [hub name] –timeout 0

where [hub name] is your IoT Hub name as before.  This will monitor messages that make it to IoT Hub so we can see our messages flow in from Node-Red.

Just leave those things up and running in the background and we’ll come back to them later (you can CTRL-C to get out of them later when ready)

Let’s set up our Node-Red instance.

Node-Red configuration

On your Node-Red box, bring up the Node-Red configuration page in your browser (typically http://nameofyournoderedbox:1880)

You’ll have a blank canvas.  You can do something more sophisticated later, but for now we’ll just do a VERY simple flow to show messages flowing to IoT Edge.   We are using a very simple flow for two reasons — to show just the main concept of connecting to IoT Edge and because my Node-Red skills are every bit as awesome as my blog writing skills.

From the node palette on the left hand side, grab the MQTT node from the *output* (not input) section of the palette and drop it on the design canvas.  Also grab an Inject node from the input section and drop it on the canvas.    Connect the output of the Inject node into the input of the MQTT node, as shown  below (your MQTT node won’t say ‘connected’ yet).

Double click on the MQTT node to configure it.

• In the ‘name’ field at the bottom, just give it a name you like.  Doesn’t matter what it is
• For QOS, choose either 0 or 1 .  IoT Hub/Edge do not support QOS 2.
• For the TOPIC, enter devices/[device id]/messages/events   (where [device id] is the device id you registered earlier in IoT Hub)
• NOTE:  *if* your payload is JSON, and *if* you think you might want to do routing of the messages based on the *body* of the message, you need to send a content type of ‘application/json’ and a content encoding of ‘utf-8’  (or 16, etc).  to do that, we need to append that information (url-encoded) to the MQTT topic, which gets passed in as a system property.  So, in that case, the ‘topic’ call would look like this (note the ‘/$.ct=application%2Fjson&$.ce=utf-8’ appended to the MQTT topic)
  • devices/[device id]/messages/events/$.ct=application%2Fjson&$.ce=utf-8

Below is a screen shot of mine.  Ignore the “Server” field in the screenshot, as we haven’t configured that yet.  For my ‘device id’, I called my device nodeRedTest for this and subsequent screenshots

I have Retain set to false, but I really have no idea if that makes any difference or not.  I’m not a Node-Red or MQTT expert, although I do play one on conference calls with customers .

Ok, next we will configure our ‘server’.  Click on the little edit pencil next to the Server box

Give your ‘server’ a name (doesn’t matter what it is – I called mine iothub-mqtt in the screenshots).  On the Connection Tab,

• for the Server  box, enter the hostname of your IoT Edge box (exactly as it’s specified in config.yaml and we verified we could resolve earlier)
• Change the port from 1883 to 8883  (the MQTT over TLS port)
• CHECK the Enable Secure (SSL/TLS) connection box.  We will come back and configure that in a moment.
• For the “Client ID” box, enter your IoT device ID from earlier
• UNCHECK the “use legacy MQTT 3.1 support” box.  IoT Edge does not support legacy MQTT connections

Example values based on my setup are shown below.  ‘rpihat1’ is the hostname of my IoT Edge box.

Click on the ‘Security’ Tab

• For the username, enter  “[hub name].azure-devices.net/[device id]/api-version=2016-11-14”  (filling in your values for hub name and device id)
• For the password, paste in the SAS token, in it’s entirety that you copied earlier

A sample screenshot of mine is below

You don’t need to set or change anything on the Messages tab

Back on the Connections Tab, next to the “TLS Configuration” box, click on the edit/pencil button

Next to “CA Certificate”  (the fourth button down), click “Upload”.  Upload the root CA certificate that you downloaded earlier.

For the name at the bottom, just make up a meaningful name for this config and click Save

Click Save or Update on the main node configuration box to save all our values and close it as well

We are now ready to save our flow.  Click “Deploy” in the upper right to save and deploy your flow.

Checking progress

At this point your flow is running and you should see your MQTT node status as “Connecting” with a yellow dot next to it.  Flip over to your IoT Edge box and look at the edgeHub logs.  You should see something like this….  (it may take a minute or more to look like this)

For some unexplained reason, the TLS Handshake between the Node-Red box and my IoT Edge box fails exactly seven times before it successfully connects.  I have no idea why and haven’t (and probably won’t) put in the effort to troubleshoot.  I suspect it’s related to a a 10-second delay between tries and the 60-second refresh value in the node config..  but who knows?

Eventually you should see the successful connection in the logs as shown the bottom half of the picture above.

If you flip back over to your Node-Red canvas, you should have a green dot next to your MQTT node and it should say “connected”

We just left the Inject node configured with it’s default of “timestamp”…  it really doesn’t matter what we send our IoT Hub at this point, we just want to see messages go in, and an updating timestamp every time we send a message is handy.  NOTE that the Inject node sends ‘timestamps’ in Unix Epoch time (number of seconds since Jan 1, 1970) so it just looks like a big number.

click on the little button (circled in red below) on the Inject node in the Node-Red canvas

This will cause a message to be created and sent to the MQTT node for sending to IoT Edge.  That will, in turn, send the message up to IoT Hub.

Flip over to your Azure Portal browser window and you should see your message having made it to IoT Hub.  click on the Inject button a few more times to send a few more messages (because it’s fun)

The output should look something like this

Your Node-Red installation is now connected and sending messages through IoT Edge up to IoT Hub.  You can now make your Node-Red flow more sophisticated (reading from sensors, etc) and do any and all the goodness that comes with IoT Edge (stream processing, custom modules, machine learning, etc).

If you have any questions or feedback, please leave them in the ‘comments’ section below.

Enjoy!