Flowthings + Initial State = Awesome

2015-10-30 FlowThings

The other day, I had an opportunity to build a project leveraging the power of streaming data manipulation in a fantastic platform called Flowthings (flowthings.io) and the amazing real-time visualization power we’ve all come to know and love at Initial State.

The goal was simple: I’ve got a couple smart devices that are sending data to an Internet gateway device which in turn is streaming via various protocols to Flowthings and now I need to make a few edits, perhaps do a little messaging and then see it in a pretty, effortless, real-time dashboard for sharing and analysis. The beauty of Flowthings in this scenario is two-fold: 1) protocol support – it doesn’t matter if data is coming in via HTTP, MQTT, or even RSS; and 2) real-time data manipulation/visual data pipe-line editing.

Implementation

Starting Point: In this particular implementation I’ve got 3 devices streaming data to Flowthings currently: a SmartSensor, a SmartSwitch (wall outlet switch) and a Water Presence Detector. They’re sending data every second to a flow in Flowthings. So, the data is flowing right along. Now time to make something useful out of the stream.

Inside Flowthings web application, I can access the flow and see that I have all my data going to a child of that flow called default. Inside here, I can see the following data streaming in:

Data Flow from Flowthings
Data Flow from Flowthings

As you can see in the middle column, I have a stream of data which in Flowthings jargon is a flow of drops. Each drop represents a payload of data that has been sent. In the right-most column, you can see a little bit about the elements in the drop payload towards the bottom. This is a visual parsing of the JSON drop payload. Inside each individual drop, I have 3 devices producing multiple data points each. This is where I need the power of Flowthings data manipulation to pull the data points out into separate tracks, enact some business logic, and send them over to Initial State. Flowthings calls this functionality a track.

Creating a Flowthings Track

Flowthings Track List

In the Tracks section, I can see that there are two tracks currently enabled for this flow. I created the Initial State Track to handle the data manipulation and forwarding along to Initial State. To create this track, I used Flowthings’ graph editor. This is a useful, visual way of manipulating the drop through this track. I wanted to be able to send different parts of the data to different Initial State buckets as well as do some fun visual things like insert emoji tokens in for certain data events. Here is a view of the final track.

Flowthings Initial State Track

The track seems to pretty intuitively explain itself, but just in case, I’ll walk you through what I’m doing here. First, the track is read left to right starting at the “source” box. The first thing I’m doing is splitting the source into three different tracks. While splitting seems like a very simple task, it provides a lot of data manipulation power in a streaming application. The “split” box is doing nothing more than making a three way copy of the stream to forward along to the next boxes where I’m actually applying some filtering logic.

In the Smart Switch Data, Smart Sensor Data, and Water Data boxes, I’m actually leveraging Flowthings’ JavaScript capabilities to message the data into a format that I can then post straight to Initial State’s streaming api. The JavaScript is really simple, I’m essentially doing the following:

A few notes here:

  1. I didn’t attempt to do any error handling here to keep the example simple
  2. I’m taking some of the data and, based on its value, I’m changing what I pass to Initial State to include some visually enticing emojis.

You can see that I have access to the `drop` object in my JavaScript snippet and I can find all my properties inside the `drop.elems` property. This is because the whole drop – when looking at the JSON payload – looks like the following:

By looking at the JSON, we can see that all the properties are accessible through `drop.elems.[json-from-here]`.

So, now that I’ve split the drop into three different sub-tracks within my track and then filtered the drop information down from the full JSON payload to a JSON serialized string, I then pass that string to the next block which performs an HTTP request to Initial State. The configuration for this block looks something like this:

Initial State Post Configuration

As you can see here, I’m configuring all the necessary details for performing an HTTP POST to Initial State’s API. I’m also passing a variable that I outputted from the previous JavaScript box in the body of the POST. Since I had access to JavaScript in the previous box, I took the easy step of performing the JSON serialization to make this step even easier. However, I could have just as well done manual JSON serialization based on the available objects in this boxes’ scope using the string format tempting available inside the request through Flowthings.

The final step is to send the response from this HTTP POST to a null terminating box. This ties off the loose ends for the track and indicates that we don’t really care about the response from the HTTP POST because we’re not doing error handling in this example and a successful response is an HTTP 204 which is empty.

The Smart Sensor sub-track has an additional split that allows me to pass the data in to two different Initial State buckets, one that provides all the sensor information as a summary and another that is sending that Sensor information into a bucket containing other environmental sensors that were being streamed from a different project. As you can see, there is a lot of power in the integration of both Flowthings and Initial State. Here is an example of the resulting Tiles view of the work performed above:

Flowthings Tiles View

Takeaways

  • When needing multi-protocol support like Websockets, HTTP, MQTT, RSS, and more to come, Flowthings provides a fantastic solution
  • The transformations in the pipeline didn’t bottleneck at Flowthings, the data flowed very smoothly from device, to flowthings, through transformations, and on to Initial State.
  • When needing to perform data manipulation, filtering, splitting, real-time calculations, etc. before sending data to Initial State, Flowthings provides a great tool.
  • If you have a flow of data and you’d like to share it, doing so within Flowthings is super simple. Additionally, if you’ve got a great track streaming data to your own Initial State account and want to share that visualization, it’s also super easy! Conclusion: data and visualizations with Flowthings and Initial State can easily leverage collaborative mindshare.

Leave a reply:

Your email address will not be published.

Site Footer

Initial State