Product Overview

This product overview takes a deeper look at Gateway Engine as a developer tool and an application hosting platform.

Internet gateways often run some flavor of Linux and, as such, have a Python environment. Leveraging this fact, Gateway Engine is a framework written (mainly) in Python for developing and hosting Custom Gateway Applications in any language on Linux gateways. IoT gateway softwares often perform the same functions regardless of the use-case because the needs are so often the same.

Common functions of gateway software include:

  • Over the Air Updating (OTAU)
  • Application version control
  • Process management
  • Logfile management
  • Exosite API libraries
  • Gateway telemetrics
  • Power failure tolerance
  • Gateway fleet management

Gateway Engine is designed to be a Custom Application Hosting framework.

@startuml
!include gwe-style.iuml
skinparam arrowThickness 4

title
    <b>A Typical Exosite Gateway</b>
end title

' network building blocks
rectangle "Linux" #EFEFEF {
    rectangle "Python" PYTHON {
        rectangle "Gateway Engine" as GWE GATEWAYENGINE {
            rectangle "device-client" as GDC DEVICECLIENT {
                rectangle gwe
                rectangle gmq
            }
        rectangle supervisord SUPERVSR
        }
    }
    rectangle "Custom\nGateway\nApplication" as CGA CUSTOMAPP {
    }
}

cloud Exosite {
    rectangle "Murano Business" ExoLightGray {
        folder Product ONEP {
            folder Device PYTHON [
                <i>resources</i>
            ]
        }
    }
}

gwe o-[SUPERVSR]- supervisord : Supervisor\nMonitoring\nand\nControl
gmq o-[SUPERVSR]- supervisord
CGA o-[SUPERVSR]- supervisord
Device <-[MURANOCOM]do- gwe : Solid\nlines\nmandatory\nroutes.

gmq <.[MURANOCOM]do. CGA
Device <.[MURANOCOM]do. CGA : Dashed\nlines\noptional\nroutes.
Device <.[MURANOCOM]do. gmq

@enduml

Over the Air Updating (OTAU)

At some point or another a software requirement changes, a bug is discovered or a new feature is added. When any of these things happen, there needs to be a way for the new executable code or binary to make its way onto the gateway and/or sensors that are attached to the gateway. Gateway Engine is the vehicle for these changes from your development or staging environment down to the gateway in a secure and scalable manner.

By design, the OTAU feature of Gateway Engine installs updates and new programs the same way that it installs and updates itself and its complementary components.

For general design information and instructions on how to create OTAU and new program installation packages, please refer to the Over the Air Updates (OTAU) section.

Application Version Control

Gateway Engine employs a light-weight version tracking system. It uses the Custom Gateway Application tarball name and version to help you keep track of the latest version of the application deployed to a gateway. Incrementing or changing the version designator in the tarball name changes the version in the next Gateway Engine report for that app in the engine_report dataport.

Process And Logfile Management

Process management in Gateway Engine is implemented with the supervisord process management tool. It is light-weight, portable and extensible. The core features of supervisor that Gateway Engine utilizes are logfile rotation and process restarting in case of bugs that can cause crashes. Gateway Engine uses supervisor to restart itself as well as all other managed processes.

Exosite Api Libraries

Gateway Engine uses and depends on the device-client python library for all Exosite HTTPS and MQTT API calls. Additional documentation as well as source code documentation can be found here on this documentation site.

Gateway Telemetrics

Diagnosing issues on any computer system either post-mortem or predictively requires basic system information. Gateway Engine provides both filesystem and OS metadata as well as modem bandwidth usage (beta). These data are reported to separate dataports device_info and usage_report, respectively.

Gateway Info

To view a given gateway’s OS and filesystem meta data. Look in the device_info resource in Murano.

Free Disk Space

$ murano device read 02:42:AC:11:00:02 device_info --json | jq -r .device_info.reported | jq -r .df
Filesystem                  1K-blocks     Used Available Use% Mounted on
none                        228432284 84295248 132463644  39% /
tmpfs                           65536        0     65536   0% /dev
tmpfs                         8013028        0   8013028   0% /sys/fs/cgroup
/dev/mapper/ubuntu--vg-root 228432284 84295248 132463644  39% /var/log
shm                             65536        0     65536   0% /dev/shm
tmpfs                         8013028        0   8013028   0% /sys/firmware

Available Memory

$ murano device read 02:42:AC:11:00:02 device_info --json | jq -r .device_info.reported | jq -r .free
              total        used        free      shared  buff/cache   available
Mem:       16026060     4029336     8556988     1815860     3439736    10870316
Swap:      16367612           0    16367612

IP Addresses

$ murano device read 02:42:AC:11:00:02 device_info --json | jq -r .device_info.reported | jq -r .ipaddrs
{
  "ppp0": ["72.87.141.57"],
  "wlan0": [],
  "eth0": []
}

uname

$ murano device read 02:42:AC:11:00:02 device_info --json | jq -r .device_info.reported | jq -r .uname
Linux 3123f1418456 4.13.0-19-generic #22-Ubuntu SMP Mon Dec 4 11:58:07 UTC 2017 x86_64 GNU/Linux

Date and Time

$ murano device read 02:42:AC:11:00:02 device_info --json | jq -r .device_info.reported | jq -r .date
Fri Dec 22 22:05:29 UTC 2017

Network Usage

All processes that utilize the device-client Python library have their requests logged to a local (on the gateway) sqlite database and at regular intervals (i.e. GWE’s update_interval``setting) the stats are reported and flushed. Below is an example of a ``usage_report from a gateway that had gwe run through a single program loop with no apps installed.

Gateway Network Statistics

$ murano device read 02:42:AC:11:00:02 usage_report --json | jq -r .usage_report.reported | jq .
{
  "eth0": {
    "": {},
    "''": {
      "read:['update_interval']": {
        "tot_rx": 194,
        "num": 1,
        "rq_time_max": 0.1,
        "tot_tx": 402,
        "max_tx": 402,
        "type": "get",
        "max_rx": 194,
        "rq_time_avg": 0.1
      },
      "read:['engine_fetch']": {
        "tot_rx": 5334,
        "num": 1,
        "rq_time_max": 0.7,
        "tot_tx": 2647,
        "max_tx": 2647,
        "type": "get",
        "max_rx": 5334,
        "rq_time_avg": 0.7
      }
    },
    "top_consumer": "''"
  }
}

The statistics collected are organized by interface, identity, API method, resource(s) and then metric.

Network Metrics
Metric Unit Description
num scalar The number of times a given request is made.
type verb The type of request made. Can be get, post, read, write, long_poll, readwrite, record, actvt, list_content, get_content, content_info.
tot_rx bytes The total (aggregate) number of recieve bytes made for this request type since the last usage_report was made.
max_rx bytes Out of all requests of this type, the maximum (peak) number of recieve bytes for this request type.
tot_tx bytes The total (aggregate) number of transmit bytes made for this request type since the last usage_report was made.
max_tx bytes Out of all requests of this type, the maximum (peak) number of transmit bytes for this request type.
rq_time_avg seconds The average amount of time this request takes to complete.
rq_time_max seconds Out of all requests of this type, the maximum (peak) amount of time a request took to complete.

Network Outage And Power Failure Tolerance

When gateways are deployed into production environments they are subject to the whims of their environments. Gateway Engine has some features that help to mitigate the risks associated with network outages and power failures.

The gwe and gmq processes use a version of Python’s ConfigParser library to atomically (i.e. AtomicConfigParser) modify configuration files over-the-air. In jffs2 and ubifs filesystems, which are ubiquitous in IoT gateways due to the fact most are based on Flash memory, an error can occur when writing to a file during a power outage whereby the file is truncated to 0-bytes. This can be catastrophic for IoT systems as this almost always means “Production Down”. To deal with this, Exosite created the AtomicConfigParser library to comply with the known workaround for these sytems.

To find out more on this behavior, please read the article on atomic writes for ubifs filesystems.

If your IoT solution can tolerate dataloss, then this section won’t apply to your gateway solution. If data originating at sensors, nodes or the gateway itself absolutely must terminate in the cloud, then the Gateway Message Queuing is an option you can leverage for a powerful, tunable, store-and-forward non-volatile queue designed specifically for network and power outages. Exosite created a scaled-down version of its HTTPS Device API made specifically for IoT gateways to protect against intermittent network outages.

Power interruptions are bound to occur and gateways must be resilient to them. Should a power failure occur, supervisord will restart all managed processes once power is restored.

Gateway Fleet Management

Murano Solutions can be designed and employed to manage small (<100) to medium (<10,000) gateway fleets. As your fleet of gateway devices grows, so likely will the complexity and customization of your IoT solution. Development of custom webapps and APIs is the solution most often adopted when scaling IoT solutions and Exosite makes this easy with Murano and Professional Services.