Rapid application development on a Raspberry Pi

This unit looks at how to add a device to your solution. Creating an application on a Raspberry Pi and establishing secure, trusted communication between your cloud application and devices.

Learning Objectives

  • Describe value of the Internet of Things Platform service
  • Describe how device types, application types, and gateway devices can communicate with the platform and each other
  • Demonstrate how you can securely connect an application, device or gateway to an organization, and publish data
  • Describe how you can bind a cloud application that is running on bluemix, to a service where devices are registered
  • Demonstrate how data can be received at a gateway and send to a cloud app
  • Demonstrate how data can be received by a cloud app, analyzed, and a command sent back to the correct device or gateway.

Raspberry Pi and SenseHAT

  • A quick look at devices and sensor options (6 min)

    The Rasperry 3 is provided with a Bluetooth and a WiFi connection, so you don't need other connection tools. Other types of devices may need further connections modules. The output connection consists of a HDMI port for a monitor output.

    The categories of devices are essentially two:

    • the platform cathegory - powerful, running a complete OS, like Linux. This type has a lot of computing power in a small form factor and limited power consumption. They are ideal for a gateway or to provide some compute power at the Edge, often referred to as Edge Analytics.

    • embedded devices - smaller footprint with a limited computing power (tipically just few KB of memory, without an OS). Some devices provide very low access to hw, such as timers and different types of sensors.

    Various devices differ for the supported programming languages and for the IDE (Integrated Development Environment) used to program them. For example, you can choose a NodeMCU for an environmental monitoring solution, because it provides a temperature-humidity sensor. But you can also use a Rasperry Pi to consolidate all that data before to report them to the Cloud platform. Sometimes there are configurations consisting on more than one device connected together, for example Raspberry Pi + Arduino (or Genuino). An other important device is the Sense HAT, which is provided with a tiny processor. The Raspberry Pi talks to the Sense HAT's processor, and then that processor deals with the lower level device control and sensor control on the board.

    Sense HAT is one add-on board provided with gyroscope, accelerometer, magnetometer, temperature, barometric pressure and humidity sensors, and has a 8x8 LED output matrix and a joistick.

    For connecting the devices you can use a plug and play mechanism such as the Grove Kits(https://www.seeedstudio.com/Grove-Starter-Kit-p-709.html), without needing to weld.

    As you go to develop your own IoT solution, you have got a wide range of choice of device and sensors available. But the used techniques and APIs can be applied to any device with any sensor.

  • Setting up a Raspberry Pi and Raspberry Pi Sense Hat (8 min)

    • Why Raspberry Pi and Sense HAT?

      Sense HAT has sensors for gyroscope, accelerometer, magnetometer, temperature, barometric pressure and humidity. It can output information on the built in 8x8 LED matrix, and it even has a five button joystick. There's a specific Python library named Sense HAT that you can use to control the Raspberry Pi Sense HAT. So, with a Raspberry Pi Sense HAT combination, you can sense the environment and send and receive information to and from the cloud.

    • Required HW

      Along with your Raspberry Pi and Sense HAT, you will need:

      • power supply for your Pi: Pi 3 is powered by a +5.1V micro USB supply (2.5A power supply is ideal)

      • a micro SD card that will act as the Pi's hard drive, holding the operating systems, software and applications you create (eight GB recommended). You need to use your computer to prepare the SD card.

      • a monitor with a HDMI cable for the monitor

      • a keyboard

      • a mouse

      • power supplies for all.

    • Installing the OS on the Micro SD card

      Before you can use your Pi, you need to download an OS for it. The OS will go in the micro SD card, the OS we're using is the Linux Raspbian Jessie, which is the based on the DPM Linux. It comes with a lot of software, including LibreOffice, which is a fully featured office suite compatible with Microsoft Office files, Node-RED, which is an open source visual editor for wiring together hardware devices, APIs and online services for the internet of things, and other software. In the documentation, under installation, you will find detailed instructions to download, on downloading the latest Raspbian Jessie OS and how to write the OS image on your SD card.

      Slide the micro SD card in the Raspberry Pi. Before you power on the unit, assure that you attach the Sense HAT and that you have secured it. Attach the keyboard and mouse in the USB port, the screen on the HDMI port. Connect Internet wireless

      Power on your Raspberry Pi. The primary desktop is displayed when the boot is complete. Click on the terminal icon to open the terminal window. The rest of the setup needs to be done from the terminal window.

      You could continue to work on the Raspberry Pi or switch to using your remote connection from your laptop. This will allow you to copy and paste from the documentation, rather than having to type every command.

    • Configuration of the HW

      To set up a remote connection, you need the IP address of your Raspberry Pi. In the terminal window of the Raspberry Pi, enter the command hostname -I then follow the instructions for your OS to access your Raspberry Pi. From the terminal window start the Raspberry Pi software configuration tool, by typing sudo raspi-config and pressing enter. The Raspberry Pi software configuration tool provides a straightforward way to configure your Raspberry Pi.

      The change user password option is selected, press enter, and change your password from the default. Keep this password safe, because it is not recoverable.

      Using the up down arrow keys, select host name, press enter and set the host name for your Pi. The default is Raspberry Pi, you can use the tab keys to jump to the available options, I press enter, since I made no changes.

      Now, let's modify the localization option. Set the default for cal, use the spacebar to select and deselect what you need, and set one of the locals as default.

      Set the time zone.

      If you have the WiFi option, set it to use the correct frequency for your country.

      For interfacing options, select P2 SSH and enable it, this will allow you to connect the Raspberry Pi via SSH. Enable P3 VNC, this will allow you connect to the Raspberry Pi user interface remotely. Finally, enable P5 I2C, which will allow the Sense HAT to communicate with the Raspberry Pi. You're now done with the configuration tool, reboot your Raspberry Pi. Once the device boots, you may be asked to log in with your new credentials, navigate back to the terminal window. There may be update to the firmware and Raspbian OS, the command sudo rpi-update will assure you have a latest firmware. Once complete, use the sudo reboot -n command to reboot the Pi.

      To update the Raspbian OS, enter the following three commands:

      • sudo apt-get update to download the update

      • sudo apt-get -y upgrade to install the update

      • sudo apt-get -y autoremove to remove the file you downloaded. Update the Node.js, Confirm that you want to update.

      This command will allow node applications to run bluetooth scans. Note the use of the back quotes surrounding the which node. If you use standard single quotes, you will get an error.

      Geany is a convenient text editor, you can install it with this command.

      The Raspberry Pi is now ready for the class. Please reboot before starting, I hope you found this useful. Have fun exploring.

    • Setting up your Raspberry Pi - step by step (pdf)

Rapid Application Development with NodeRED on a Raspberry Pi

In this module, we're switching focus from programming on the backend in the cloud to programming on the Raspberry Pi. Using the Raspbian Jessie wiyh Pixel OS, we have the Node-RED preinstalled, as the Iceweasel browser we prefer because Node RED runs better on that browser than default.

On the Raspberry Pi, Node RED provides different nodes with respect to the Bluemix, in particular in Raspberry Pi section . The runtime is slightly different, i.e. here the flow lives within the file system. There is a .node-red directory under the home dir, with configuration and setting files. There are also some differences in how we install new packages to be made available to the function node and to install new nodes on the pallette. In Bluemix we have not access to the command line, so, for adding any new packages we have to modify the .json file to the environment and then rebuild the environment. But on the Raspberry Pi, we have access to the command line so we can install directly.

For installing a new package, we have to go inside the .node-red directory (another possibility is ti use the Global install option). We want to install as example the CRC function (cyclic redundancy check). In the setting.js file we add the new entry at the end of the global context. Once saved the setting.js, we use the npm install command and the package is installed. We have to stop with thw node-red-stop command and start with the command node-red-start for these changes to take effect. Now we can use the CRC function for calculating the crc of a string.

Add a new node to the palette.

We simply need to install the package so NodeRED can access it.

The random node is already installed, by default, on the raspberry pi, so we are going to install another node, in particular a node that converts a binary file to base64 encoding and viceversa, enabling the text format.

To make the node available, simply go onto the command line, make sure you are in the .node-red directory, and digit the install command: npm install node-red-node-base64. Once the node is installed, stopping and then starting NodeRED: node-red-stop snd node-red-start, then refreshing the browser screen, we'll see that node show up in the pallet.

You may find it convenient to have node red start when the pi boots up. The command on how to install NodeRED, so it boots at system startup. 1:26 And if I want to stop it starting as the ii boots, I've also got the command here on how to disable the autostart at boot. 1:35 So I'm going to start my environment, and sure it start when the pi boots, so enter the command. 1:44 Another node that's not installed by default on Bluemix, but is available at install time on the Raspberry Pi is the exec node. With the exec node, I can run a system command as if I was running it from the command line. 2:01 Any output generated by the command is unavailable to my flow. 2:05 I can very easily demonstrate the use of the node by drumming it onto the palette, 2:11 adding an inject and a debug node. 2:14 And I'm going to get it to issue the netstat command. So this prints information about the network's subsystem on the Raspberry Pi. You'll see I have the option of a pending msg.payload to the command. So this allows me to effectively pipe commands together like I would on the Linux command line. 2:38 But I'm going to select to add an additional parameter. 2:42 And I'm going to add the minus A parameter to the netstat command. Which outputs both listening and non-listening sockets. 2:53 Simply deploying the flow. 2:56 And then pressing the inject node. I see the expected output coming to the debug tab. 3:02 So we're now going to look at the IBM Internet of Things platform. In the next module, we'll go in to details, but for now we're going to set up, similar to the initial sample application with the simulated device. We're now going to show you how to connect the Raspberry Pi to the Internet of things platform using that quick start service. So the quick start service is a sandbox environment, it doesn't require any security or pre-registration of device it just sits there. And allows me to publish data on the application allows me to visualize that data. 3:44 So to do that, we're going to use an IBM IoT output node. 3:51 If we open up the Configuration, you can see we get offered the option to be a Quickstart or a Registered device. We're going to select Quickstart and it predefines a device ID. So whenever you use a device with the IBM Internet of Things platform it has to have a unique device ID. Usually, when there is a network connected device, we'll use the network adapter MAC address. Because that should guarantee uniqueness. But the device ID has to be any string that guarantees its unique within the device type and organization. You'll notice that on the configuration node, there's also a very handy link to the quick start application. 4:41 If i click this link, it will open up the quick start visualization page using the generated device ID. And we're simply going to use the random node to send the random data to the quick start application. So, I create an inject node, the random node, connect it through to the IoT node. And I'm just going to put a debug node there, so I get confirmation that I am actually sending the correct data, but now I also need to format the data. 5:14 So, I am going to put a function node in, before sending the data. 5:19 To get Quickstart to visualize the data correctly, it has to be in the right format for the platform to understand. 5:27 So within the object, we need a D property, D for device, and within that property I can have any data that I want to publish in my event. For me, I'm just going to create a random property and then can fill it in with the data coming from the random node. 5:46 So now, I've got the inject node. Publishing the data every five seconds. The random node generating a random number. And this function node is formatting the data. And then sending it to the IoT node to be sent to the cloud. And if I just deploy this application, you see in the debug tab, I'm getting the expected data. And if I switch into the Quickstart application, it now shows it's receiving data and I can now see that the random number is being sent to the IoT platform.

  • QuickStart flow - step by step (10 min)
  • Programming Assignment: QuickStart flow on Raspberry Pi (1h)
  • Learning module summary: Rapid Application Development with NodeRED on a Raspberry Pi (10 min)

Introduction to the Watson Internet of Things Platform

Watson Internet of Things platform (5 min)

The Internet of Things Platform is available as a service on Bluemix. It provides device access and the data connectivity. The platform provides a number of services to help us manage devices and the device data.

We want to ensure that all data received is from a trusted device. So, the Internet of Things Platform provides a device registry. This allows you to hold security and metadata on all your devices in one place.

The Internet of Things Platform also provides the connectivity. It uses MQTT, the MQ Telemetry Transport protocol, as the underlying mechanism to allow devices to communicate with the platform.

It also provides support for gateway devices, and we'll look at the difference between a gateway and a device in the next module.

The platform provides an API for device management, to allow you not only get data from a device but also to actively manage the device.

And lastly the Internet of Things Platform provides integration with certain external services. So, as we look through the platform in a moment, you see we have integration with services such as Jasper.

The Internet of Things Platform provides an API to allow devices, applications and gateways to communicate with the platform. It provides a REST API that allows access to configuration and historic data.

In the dashbord, go into the application, in the connection section, and click on the Internet of Things Platform service. Then lauch the Watson IoT Platform. On the left hand side there are a number of sections. The first one is the Board Overview section. Below there is the Devices section. And this is where I get access to all of the devices, to be able to manage devices, create new device types, and register new devices. And all of this can also be done using the API. Below the devices section is the Access section. The Internet of Things Platform ensures all access to the platform secured. So, whether it's a device sending data to the platform or an application sending a command or a control instruction to a device, every communication has to be authenticated by the platform. This access section allows you to manage who can send data to the platform. Next we've got some usage information where you can get a little bit more information about what's going on in the platform. And lastly, there is the Settings section and this allows you to configure different aspects of the platform. It allows you to turn on and off the different features available on the platform, such as storing historic data. The Internet of Things Platform can store all the data that goes through the platform and allows you to retain that data for a set period of time. On the settings page is where you can define and configure that behavior. The settings page is also where you enable and configure the integration with other services.

One concept that you need to understand when using the platform is that of an organization. While the previous Node-RED application didn't need any authorization or registration of devices or (we used the quick start service), now we have to refer to our organization that requires you to authenticate and register all devices and applications.

You can see the six character organization ID, up on the top right corner of the dashboard. And every time you use an API, you need to make sure that you're sending data referring to the organization ID. As we look at the API and how to use it through Node-RED or directly through the programmatic API, we'll see how to plug in your organization ID.

  • Devices, Applications and Gateways part 1 (6 min) Going a little deeper ito the Internet of Things platform, we can see that is organization that ensures that all data coming to and from a device is trusted data from a trusted source. So now let's look at who you can actually send and receive data.

Within the platform, there are three types of applications that can connect to the platform. The first one is from a device. Before, it must be registered with the platform.Before you can register a device, you need to create a device type, that allows you to organize your devices into their capabilities. Next, we have an application, that is a program that wants to communicate with the Internet of Things platform, either by receiving data from a device, or by sending a command to control a device. Unlike a device, an application doesn't need to be pre-registered to be able to communicate with the platform, but it does need to send an API key to authenticate itself. Then we have a gateway. A gateway is a special type of device. Sometimes you have sensors or sensor networks that aren't able to connect directly to the internet. So, cannot connect to the IOT platform. The Gateway acts as a bridge to connect the sensor network to the internet and the IBM IoT platform. Like a device, a gateway must be registered before it can communicate with the platform. When a device wants to send data to the platform, it creates an event, and the data flows in an event.So, devices send events to the platform, and applications receive those events.

Now we are going to look at how you create device types, and then how you create and register a device. B back in Bloomix, open the service on the dashboard. So, now in the overview section. 3:40 To create a device type, I switch to the device section, and then select to add a device. 3:48 On this screen, I can choose a pre-existing device type if I've created one, or I can press this button to create a new device type. 3:59 And then I just go through and add any data that I want to that device type. I have the option of defining attributes, I won't capture it when defining a new device, or if I prefer I can just leave this all blank. Once you've gone through and you've created your device type, you then need to go and add a device. So, while in the screen, I can go and add a device and select the previously defined device type. 4:29 If I define any attributes, I get asked to complete these when I'm registering the device, but eventually I get to this screen, where I'm asked to either select a pre-assigned key or create my own key. As a tip, when you're registering a device, I always use a common token. So as I'm modifying the code or changing the code, I always know what the token for any given device is. 4:57 And that's because, once you get to this screen, this is the last chance you get to record the token. 5:06 Once you close this screen, you cannot recover a token. 5:10 And if you lose a token for a device, the only option you have, is to delete or unregister that device, and then reregister it again with a new token. 5:21 So when developing, I always use a common token, so I don't have to keep enregistering and registering devices if I get what the token was. It's up to you how you want to classify your devices. The system places no demands on how you organize and classify your devices. However, when an application selects to receive data from a device, it has the option of specifying device type as one of the selectors on what data an application wants to receive. 5:51 So, it's useful to classify different types of device, using a different device type.

  • Devices, Applications and Gateways part 2 (7 min)
  • Quiz: Using the NodeRED flow editor with your own Watson IoT platform. (5 questions)
  • Learning module summary: Introduction to the Watson Internet of Things Platform (5 min)

Controlling a device

  • Sending commands to a device (4 min)

For sending commands to a device from NodeRED we can create a JSON package and use the API. The device must be registered before to receive commands. Using the input node, a device can be registered for all commands, or you can configure a node to receive only a specific command.

As a gateway, you can receive commands for yourself, or you can choose to receive commands for devices that connect through yourself. When I choose to receive device commands, I get to specify the device type and the device ID of the commands I'm registering for.

When you register to a sequel of commands, you need to look at what command was sent and then handle that specific command appropriately.

A device cannot send a command to an other device.

When we looked at sending device events, we have to use a specific message format so the QuickStart application could display the data correctly. For commands, there's no such recommended format. You can use any valid JSON object. However, sometimes the command name itself is sufficient information for the device to understand what to do. So there's no additional data needed for the command. In that case, you can just send an empty JSON object.

Let's create a command flow statusIndicator. I have an LED connected to my Raspberry Pi, and I wanted to send a command to either turn that on or turn that off - alert state or normal state.

We can create two different input nodes with a function node to send a payload for normal or for an alert status, and send this out through the Watson IoT output node and to the Raspberry Pi device. So I put in here, the device ID of the device to receive this.

I can use the Linux service, to get the authentication credentials for this application. Deploying it then injecting one of these two options, we can either send normal or alert message.

Switching over to the Rasberry Pi, we can use a Watson IoT input node to receive the command. We have the previously defined credentials available. So I can select those to connect to the service. We can then select either all commands or choose to select only a single command.

Then we connect a debug node to see the output of the command coming into the servers. But typically you'd have to put a flow coming out of this node to actually handle the command correctly.

So if I'm receiving all commands, I have to work out which command I received, and then switch to the appropriate flow to implement that command. And as you can see, I can now receive either the normal or the alert message.

  • SenseHAT and SenseHAT simulator nodes in NodeRED (3 min)

In the NodeRED pallet we have two nodes for SenseHAT (input and output). For the input node, you can choose one or more among the outputs: Motion events, Environment events and Joistick events. Environmental sensors publish about once every second, and they are the temperature and humidity and air pressure. The motion sensors publish about ten times a second. And that's your gyroscope magnetometer and accelerometer. The joystick events only publish when you actually interact with the joystick.

We are focusing on the environmental temperature sensor. Select the sensors you want, and then that will initiate a flow. To output to the LED matrix, you're going to use the Sense HAT output node. And again, if you look in the info tab, you'll see all of the details of how you have to format the payload to actually control the output. If you don't have a Sense HAT, there are two additional nodes that you can install that acts as the Sense HAT simulator. (Going to the .node-red directory and digiting the command npm install node-red-node-pi-sense-hat-simulator). To allow you to control what the sensor's data is, there's also a web interface where you can control the environmental sensors. So you can set the value you want published. There is also an output node for the simulated Sense HAT that will show the state of the LED panel in the web interface.

  • Programming Assignment: End-to-end scenario (2h)
  • Additional assistance for the end-to-end assignment (6 min)
  • Learning module summary: Controlling the device (10 min)