Introduction

The Internet of Things is so much more than blinking lights and reading sensors, it is also the mechanism for delivering Artificial Intelligence workloads to the masses. When we traditionally think of IoT, it is typical to consider specialized hardware with physical sensors that report readings up to a cloud service for processing. With the advent of small form-factor GPU and FPGA enabled devices, we are starting to see more and more heavy lifting happening on the device itself. This allows us to reduce data outflow, while at the same time focusing on the important results that come from processing our data on the edge.

When we are able to reduce our reliance on the cloud for processing by taking advantage of accelerated workloads on-device, we are able to obtain and make use of results much faster even if we are disconnected from the internet. Microsoft's IoT Edge platform is designed for exactly this type of use case. Under the hood IoT Edge accomplishes safe deployment of code to IoT devices through the use of containerized modules. These modules can include containerized forms of popular Azure services including Serverless Functions, Stream Analytics, Machine Learning modules, Custom Vision AI services, and even local storage with SQL Server. The IoT Edge platform basically allows you to take the cloud services you already know and love and use them in your edge environment to facilitate the same powerful features and functionality you've grown to expect from using these services in the cloud.

Getting Started

Nvidia produces a number of devices suited for IoT solutions in their Jetson line of device offerings. These include the beefy 512-Core Jetson AGX Xavier, mid-range 256-Core Jetson TX2, and the entry-level $99 128-Core Jetson Nano.

To follow along with this article, you will need one of the following devices:

• Jetson AGX Xavier
• Jetson TX2
• Jetson Nano

To ensure wide compatibility, all of the content in this article was created and tested on an NvidiA Jetson Nano device.

Installing IoT Edge on Nvidia Jetson Devices

ARM64 builds of IoT Edge are currently being offered in preview and will eventually go into General Availability. We will make use of the ARM64 builds to ensure that we get the best performance out of our IoT Edge solutions.

These builds are provided starting in the 1.0.8-rc1 release tag. To install the 1.0.8-rc1 release of IoT Edge, run the following from a terminal on your Nvidia Jetson device:

# You can copy the entire text from this code block and 
# paste in terminal. The comment lines will be ignored.

# Download and install the standard libiothsm implementation
curl -L https://github.com/Azure/azure-iotedge/releases/download/1.0.8-rc1/libiothsm-std_1.0.8.rc1-1_arm64.deb -o libiothsm-std.deb && sudo dpkg -i ./libiothsm-std.deb

# Download and install the IoT Edge Security Daemon
curl -L https://github.com/Azure/azure-iotedge/releases/download/1.0.8-rc1/iotedge_1.0.8.rc1-1_arm64.deb -o iotedge.deb && sudo dpkg -i ./iotedge.deb

# Run apt-get fix
sudo apt-get install -f

Provisioning the IoT Edge Runtime on Nvidia Jetson Devices

To manually provision a device, you need to provide it with a device connection string that you can create by registering a new IoT Edge device in your IoT hub. You can create a new device connection string to accomplish this by following the documentation for Registering an IoT Edge device in the Azure Portal or by Registering an IoT Edge device with the Azure-CLI.

Once you have obtained a connection string, open the configuration file:

sudo nano /etc/iotedge/config.yaml

Find the provisioning section of the file and uncomment the manual provisioning mode. Update the value of device_connection_string with the connection string from your IoT Edge device.

provisioning:
  source: "manual"
  device_connection_string: "<ADD DEVICE CONNECTION STRING HERE>"

# provisioning: 
#   source: "dps"
#   global_endpoint: "https://global.azure-devices-provisioning.net"
#   scope_id: "{scope_id}"
#   registration_id: "{registration_id}"

You will also want to configure the default IoT Edge agent configuration to pull the 1.0.8-rc1 version of the agent. While in the configuration file, scroll down to the agent section and update the image value to the following:

agent:
  name: "edgeAgent"
  type: "docker"
  env: {}
  config:
    image: "mcr.microsoft.com/azureiotedge-agent:1.0.8-rc1"
    auth: {}

Developing IoT Edge Modules on Nvidia Jetson Using Visual Studio Code

Technically, an IDE is not required, but to get the smoothest developer experience it is highly recommended. Visual Studio Code is an officially supported IDE which allows for the installation of IoT Edge extensions to aid in development.

It is important to note that while VS Code offers pre-built binaries for Linux, Mac, and Windows it does not provide 64-bit ARM builds suited for Nvidia Jetson devices. This is okay though because VS Code is open-source so we can build the IDE from source on the device itself. This is a bit time consuming and not recommended if you are looking to get up and running quickly. If you are interested in doing so, you may want to check out this thread on the Nvidia developer forums. Keep in mind, these instructions are a moving target and what works today may not work tomorrow.

To make installation a breeze, you can leverage a pre-built .deb package that I have published which will allow you get started immediately. To install Code-OSS (The open-source version of Visual Studio Code) with extension support, run the following commands from a terminal on your Jetson device:

curl -L https://github.com/toolboc/vscode/releases/download/1.32.3/code-oss_1.32.3-arm64.deb -o code-oss_1.32.3-arm64.deb

sudo dpkg -i code-oss_1.32.3-arm64.deb

Now you can start the IDE with:

code-oss

Sweet! Now we can begin installing extensions to accelerate our IoT Edge development. Thankfully, this is made extremely easy using the Azure IoT Tools Extension). Follow the instructions in the link to install the extension into your environment. Once you are finished you can check out the documentation for Using Visual Studio Code to develop and debug modules for Azure IoT Edge.

It is important to note that you will want to ensure that you set your IDE platform setting to ARM32v7. By default this option is set to AMD64. You can find this option in the bottom left of the IDE. Changing this will ensure that your IoT Edge Modules are built using their assocaited Dockerfile.arm32v7:

As of writing, IoT Edge module templates do not support ARM64. Upon creation of your module, you will need to modify the deployment.template.json to pull down the appropriate ARM64 systemModules. To accomplish this, open your deployment.template.json file and replace the entire systemModules section with the configuration pasted below:

"systemModules": {
          "edgeAgent": {
            "type": "docker",
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-agent:1.0.8-rc1",
              "createOptions": {}
            }
          },
          "edgeHub": {
            "type": "docker",
            "status": "running",
            "restartPolicy": "always",
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-hub:1.0.8-rc1",
              "createOptions": {
                "HostConfig": {
                  "PortBindings": {
                    "5671/tcp": [
                      {
                        "HostPort": "5671"
                      }
                    ],
                    "8883/tcp": [
                      {
                        "HostPort": "8883"
                      }
                    ],
                    "443/tcp": [
                      {
                        "HostPort": "443"
                      }
                    ]
                  }
                }
              }
            }
          }
        },

Using the IoT Edge Simulator on Nvidia Jetson

You will notice that the docs make mention of using the IoT Edge Simulator for Debugging IoT Edge modules. This is possible but it will not work without a few additional steps for now.

First we need to install the iotegehubdev package using pip and make it available to all users. This can be done by performing the following steps in the terminal:

sudo apt install python-pip libffi-dev libssl-dev
sudo -H pip install --upgrade iotedgehubdev

Next, we need to configure the iotedgehubdev tool with a compatible <edge-device-connection-string>. You can create a new device connection string to accomplish this by following the documentation for Registering an IoT Edge device in the Azure Portal or by Registering an IoT Edge device with the Azure-CLI. It is recommended to name the device "iotegehubdev" during the process.

Finally, we need to update the iotedgehubdev tool to use the appropriate Iot Edge agent and test utility. When IoT Edge ARM64 support is released to General Availability this should no longer be required, but for now it is necessary.

Edit the following:

sudo nano /usr/local/lib/python2.7/dist-packages/iotedgehubdev/edgemanager.py

Modify the following parameters:

EDGEHUB_IMG = 'mcr.microsoft.com/azureiotedge-hub::1.0.8-rc1'                                                          
TESTUTILITY_IMG = 'mcr.microsoft.com/azureiotedge-testing-utility:1.0.0-arm32v7'

Now delete, /usr/local/lib/python2.7/dist-packages/iotedgehubdev/edgemanager.pyc to force re-compilation of the python module with:

sudo rm /usr/local/lib/python2.7/dist-packages/iotedgehubdev/edgemanager.pyc

The IoT Edge Solution Simulator should now work without issue when invoked from VS Code. This is a great way to test your IoT Edge modules on a local device without creating a deployment configuration.

Conclusion

We have shown that you can get started building out IoT Edge modules using the popular tooling available on Linux, Mac, and Windows right from your Nvidia Jetson device. This opens up the possibility of being able to create GPU accelerated workloads with a local development environment on your target device. Keep in mind that this story is really only just beginning and will only get better as we see improvements from both Microsoft and Nvidia regarding their official support for ARM64 platforms. For example, July will likely bring us better IoT Edge support for ARM64 and Nvidia has announced plans to update nvidia-docker to support ARM64 in that timeframe.

As these products mature, you can expect to see more details in my upcoming articles which will cover how to take advantage of GPU acceleration in IoT Edge modules. For a look at related content on dev.to, check out Getting started with DevOps CI / CD Pipelines on Nvidia ARM64 Devices and Using Cognitive Services Containers with Azure IoT Edge.

Until next time, happy hacking!