From Sensor To Dashboard: The Untold Guide

3 min readMar 31, 2020

Figure 1. Typical Data Flow from Sensor To Dashboard

You will most likely use Power BI as your dashboard. It is free, if you have a company registered email address. The free version will not allow you to share the dashboard to other user. For dashboard sharing, you will need a PRO version. It costs money.

We are going to use Open Source Grafana, together with Community InfluxDB, to replace Power BI.

Let’s dig in!

From Figure 1, most likely you have managed to work out the first three processes, namely: SENSOR, EDGE DEVICE, and AZURE IoT Hub. That is you have successfully send the sensor data to Azure IoT Hub. Good job!

Message Routing, From IoT Hub To Event Hub . The next step is to route the sensor reading from IoT Hub to Event Hub. We need to set up Event Hub. What is Event Hub? If you are familiar with Kafka, Event Hub is Kafka. By routing IoT Hub to Event Hub, you are publishing sensor reading as message to Event Hub.

You can easily find step by step instructions on Azure documentation on how to create Event Hub. You need to pay attention to this few terms:

Every Event Hub has one Event Hub Namespace. It will look something like this.

For Example:

Event Hub Namespace: i3eventhub.servicebus.windows.net

[In Kafka term, it is called Cluster Address]

Each Event Hub Namespace will host one or more Event Hubs:

[In Kafka term, it is called Topic]

Event Hubs: iothubevent

Test whether Event Hub is working:

Send Message [Publisher]: What it does is sending message to Event Hub.

import asyncio
from azure.eventhub.aio import EventHubProducerClient
from azure.eventhub import EventData

async def run():
   # Create a producer client to send messages to the event hub.
    # Specify a connection string to your event hubs namespace and
    # the event hub name.
   
    producer = EventHubProducerClient.from_connection_string(
         conn_str="Endpoint=sb://i3eventhub.servicebus.windows.net/;
         SharedAccessKeyName=RootManageSharedAccessKey;
         SharedAccessKey=xxxxxx",                                                            
         eventhub_name="iothubevent")


    json_body = { "measurement":"temperature",
                   "tags":{
                     "tag1":"val1"
                   },
                  "fields":{
                        "value1": 110.0
                  }
     async with producer:
        # Create a batch.
        event_data_batch = await producer.create_batch()

        # Add events to the batch.
        event_data_batch.add(EventData(json_body))
        #event_data_batch.add(EventData('Second event'))
        #event_data_batch.add(EventData('Third event'))

        # Send the batch of events to the event hub.
        await producer.send_batch(event_data_batch)

loop = asyncio.get_event_loop()
loop.run_until_complete(run())

Receive Message [Subscriber]: Get message from Event Hub:

import asyncio
from azure.eventhub.aio import EventHubConsumerClient


async def on_event(partition_context, event):
    # Print the event data.
    print("Received the event: \"{}\" from the partition with ID: \"{}\"".format(event.body_as_str(encoding='UTF-8'), partition_context.partition_id))
    
    # Update the checkpoint so that the program doesn't read the  
    #  events
    # that it has already read when you run it next time.
    await partition_context.update_checkpoint(event)

async def main():
    # Create a consumer client for the event hub.
    #client = EventHubConsumerClient.from_connection_string("EVENT  
    #HUBS NAMESPACE CONNECTION STRING", consumer_group="$Default",   
    #eventhub_name="EVENT HUB NAME",   
    #checkpoint_store=checkpoint_store)
    
    client = 
          EventHubConsumerClient.from_connection_string(
               "Endpoint=sb://i3eventhub.servicebus.windows.net/;
                SharedAccessKeyName=RootManageSharedAccessKey;
                SharedAccessKey=xxx",consumer_group="$Default",  
                eventhub_name="iothubevent")
    async with client:
        # Call the receive method.
        await client.receive(on_event=on_event)

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    # Run the main method.
    loop.run_until_complete(main())

Each time you send a message, the receiver will get a message.

Once you have configured and tested out the Event Hub, you can setup “Message Routing” in IoT Hub to route message to Event Hub. The receiver code above can be used to see whether the sensor readings has reached the event hub.

NIFI

NIFI will grab the messages from Event Hub and write the messages to InfluxDB.