Dzone Refcard263 Iotinfrastructureatscale
Dzone Refcard263 Iotinfrastructureatscale
Dzone Refcard263 Iotinfrastructureatscale
257
CONTENTS
Messaging Infrastructure
MESSAGING AS A "LEVER"
MORE
DIRECT OR STORE-AND-FORWARD
MESSAGING?
öö GETTING STARTED
The Internet of Things isn't just a buzzword anymore — it's real, • A device provisioning component for delivering software
it's around us, and we are already living in the IoT era. At same updates on devices.
time, IoT isn't completely new. Before this term was coined, a lot
of companies were already developing embedded devices able to All of these "core IoT" services will use the underlying messaging
connect to a network in order to communicate each other or with infrastructure to communicate with each other and with devices in the field.
Today, IoT is the field for the revenge of a "hidden" technology that is
not always considered properly: the messaging.
IoT is always about messaging. Devices communicate each other by Figure 1: What makes an IoT platform?
exchanging messages. They send (telemetry) data to a cloud platform
and receive commands through messages. IoT can be considered a Of course, all the ingested data needs to be processed in order to bring
specific messaging use case in which the clients are just devices in the value to the entire IoT solution. For this reason, other "business" services
field and services in the cloud. can leverage the messaging infrastructure in order to receive such data for
monitoring, real-time analytics, machine learning, and so on.
The real big difference between a business platform using a
messaging infrastructure and an IoT solution is scale. The number THE NEED FOR ELASTICITY... AND MORE
of connected devices can range from a few units to millions, which When it comes to supporting a huge number of connected devices
means that the messaging infrastructure has to support a huge that produce telemetry data with high throughput or receive
number of connections and a massive message exchange while commands from different backend services, the underlying messaging
providing high throughput. infrastructure in the overall IoT solution becomes the main point of
success... or failure.
WHAT MAKES AN I o T PLATFORM? MESSAGING
AS A "LEVER" The main need is to use resources productively to handle spikes in
The core of an IoT platform is the messaging infrastructure, which has
terms of connected devices and messages exchanged from lower to
to provide all the features described in the previous section. Around
higher throughput. The messaging infrastructure needs to be elastic.
such an infrastructure, there are other "core" services focused on IoT
use cases, such as: Elasticity means that the user is able to "tune" the messaging
resources according to the traffic load so that they are allocated
• A device registration component for handling devices
only when it is necessary. For example, speaking of message brokers,
information and the related status.
it's useless to have a big cluster for data ingestion when there are
• An authentication/authorization component for providing only a few connected devices. The ability to scale resources up and
access control. down provides the elasticity we need, which could be available
1
MESSAGING INFRASTRUCTURE FOR I o T AT SCALE
automatically (based on some metrics like network I/O, message it from one provider to another with little impact on the overall IoT
throughput, connected devices, and so on). solution. Maybe just changing connection information for devices and
services could be enough.
Other than elasticity, the messaging infrastructure needs to provide
resiliency and high availability, which make such an infrastructure EnMasse supports all the well-known messaging patterns (request/
more complex. All the messaging components need to recover from reply, publish/subscribe, and competing consumers), which are really
failures, which always happen in a distributed system. We cannot avoid strictly related to the available IoT communication patterns (telemetry,
failures, but we have to handle them properly. At same time, downtime notification, command/control, and inquiry).
should be approximately zero in order to have the IoT solution always
available and "reacting" to the devices and backend services inputs.
Last but not least, whoever wants to develop an IoT solution doesn't
also want to take care of all the problems related to the messaging
infrastructure deployment. It should be really simple to provide a
higher abstraction layer so that the IoT developer can focus on using
well-defined messaging addresses to allow devices to communicate
with backend services and vice versa.
In conclusion, elasticity, scalability, resiliency, high availability, deployment, Figure 2: IoT and messaging patterns.
2
MESSAGING INFRASTRUCTURE FOR I o T AT SCALE
clients connect to such a network for exchanging messages in different deploys a "managed" cluster for you without the need to install all the
ways, as we'll see in the next section. Only MQTT clients represent Kubernetes components on the VMs.
an exception because they connect to the messaging infrastructure
It's really simple to move an IoT solution based on EnMasse from on-
through a corresponding gateway bridging MQTT to AMQP.
premise to the cloud and from one provider to another in the cloud
Going through these routers, the messages are directly delivered itself, avoiding the "vendor lock-in" problem.
from producers to consumers while, in order to store them, EnMasse
leverages the Apache ActiveMQ Artemis broker project. Behind the DIRECT OR STORE-AND-FORWARD MESSAGING?
routers, one or more brokers can be deployed in order to provide Thanks to its internal architecture based on routers and brokers,
storage. Of course, it's not only about storing but also about EnMasse provides two different messaging mechanisms: direct
forwarding messages from the brokers to the clients with the usual messaging and store and forward.
An admin console provides many different features, from deploying to-peer protocol, so you can have two clients directly connected
the needed components (routers and brokers) based on the each other. In this way, the producer is able to send a message only
addresses and messaging patterns that the user wants to a web UI when the consumer is online (providing "credits"), and it receives an
for monitoring the messaging infrastructure (i.e. connected clients, acknowledgement, which means that the consumer has received the
messages, and throughput). message. Between the two parties, there is a single contract so that
the producer knows that the message is received by the consumer
Last but not least, from a security point of view, all the internal when it gets feedback.
components are connected to each other using TLS encryption, and
the client's authentication and authorization needs are provided EnMasse provides this kind of mechanism in a more reliable
by the Keycloak project. The client needs to be authenticated and and scalable way through the router network, where routers are
authorized in order to send and receive messages through EnMasse; connected to each other making a "mesh" — the clients aren't
DZO N E .CO M/ RE FCA RDZ
even the client's connections are encrypted via TLS. connected directly but instead through this network. Every router,
unlike a broker, doesn't take ownership of the message but just
forwards it to the next stop in the network in order to reach its
destination. If a router goes offline, the network is automatically
Figure 3: EnMasse architecture.
reconfigured in order to identify a new path for reaching the
consumer; it means that high availability is provided in terms of path
What has not been mentioned yet is that the EnMasse platform runs
redundancy. Furthermore, as it happens in a real direct connection
on Kubernetes and OpenShift in order to be elastic, scalable, and
with AMQP 1.0, a producer doesn't get "credits" from a router for
resilient and to effectively handle failures.
sending messages if the consumer isn't online or can't process more
All the described components are provided as Docker images that messages. Finally, direct messaging is synchronous by nature, so it's
can run as containers orchestrated by Kubernetes/OpenShift on a really useful for RPC communication.
related cluster. Deploying EnMasse is really simple, only leaving the
The entity used for identifying how producers and consumers
developers and the operators with the need to handle higher-level
exchange messages is the address, as it's defined by the AMQP 1.0
concepts from the messaging world.
specification as just a string.
Running on Kubernetes/OpenShift gives developers the flexibility to
move the messaging infrastructure and the related IoT platform built
on top of it from an on-premise solution to the cloud. Kubernetes
and OpenShift (using the Origin project or the OpenShift Container
Platform) can be installed "locally" for development and testing
purposes or even on a few servers on bare metal.
3
MESSAGING INFRASTRUCTURE FOR I o T AT SCALE
message is immediately forwarded to the final consumer, which could In an IoT-specific use case, the main two communication patterns,
be offline in that moment. It allows asynchronous communication like telemetry, command, or control, could be implemented in a few
and time decoupling because the consumer can get the message later different ways.
and at its own pace, which can be different from the producer. There
In the telemetry scenario, a device could be enabled to send telemetry
is always a double contract between producer-broker and broker-
data only if a backend service is online and able to get such data; we
consumer, so the producer knows that the message reached the broker,
don't care about data if no one is able to process it, and we are not
but not the consumer (a new message exchange on the opposite
interested in storing the related messages. In this case, using direct
direction is needed for having acknowledgement from the consumer).
messaging is the right solution. On the other side, it's possible that we
The entities used for storing messages are queues and topics, which want to ingest data from devices even when no services are running
allow point-to-point (or competing consumers) and publish-subscribe (or maybe they are busy) for processing. In this case, store and
patterns. In any case, the name of a queue or a topic is just a string, forward is the way to go, putting messages inside queues or topics
like an AMQP 1.0 address. (depending on whether we want to distribute messages to one or
more services in parallel).
In the command and control scenario, we could have the same two
approaches. In one case, we want to send a command to a device
only if it's online and we are sure that it can execute (at least receive)
the command itself in that moment. Direct messaging can help with
this use case. On the other side, it's possible that we want to handle
situations in which the device isn't online, but we want it to execute
the command when it comes back online. In this case, the command
Figure 5: Store and forward. message needs to be stored for later delivery. In order to avoid
DZO N E .CO M/ RE FCA RDZ
sending "stale" commands to a device that comes back online too late
From a client's perspective, when they connect to EnMasse through
(for the command), the message can have a related TTL (time to leave)
the unique entry point (which is the router's network), they just
so that it disappears from the queue on expiration.
connect to an address for exchanging messages; they don't know that
a broker could be behind such an address with a corresponding queue
GETTING STARTED
or topic.
The official EnMasse website provides a great documentation section
for digging into all the features and getting started with the platform,
The EnMasse operator has to define the addresses and their semantics
but it's worth showing here how easy it is to deploy and use it locally
in order to support the different messaging mechanisms as described
on OpenShift.
above. Using the related console, it's really simple and it's a matter of
defining what kind of pattern the clients need. The supported address
The first step is to get the OpenShift client tools needed to interact
types are the following:
with the OpenShift cluster; you can download the client from the
OpenShift Origin project.
• Queue: Backed by a broker for "store and forward" and
providing point-to-point (competing consumer) patterns. If you don't have an OpenShift cluster already running, you can deploy
one just using the oc cluster up command or using the Minishift project.
• Topic: Backed by a broker for "store and forward" and
providing publish/subscribe patterns. From the EnMasse releases page, you can download the latest release
and unpack it locally. It provides a script for deploying EnMasse on top
• Anycast: Similar to a queue in terms of direct messaging. A of the OpenShift cluster with just the following command:
producer can send messages to such an address only when
one or more consumers are listening on it and the router's ./deploy-openshift.sh -m "https://localhost:8443" -n enmasse
• Multicast: Similar to a topic in terms of direct messaging, it has The deployment will take some time to have all the components
a producer publish messages to more consumers listening on up; the EnMasse platform is running in a single tenant mode and
the same address so that all of them receive the same message. without authentication.
4
MESSAGING INFRASTRUCTURE FOR I o T AT SCALE
After that, you can log into the OpenShift console, select the The receiver will block until it has received ten messages.
"enmasse" project where EnMasse was deployed, and open the web
To start the sender, just run:
console to create a new address.
You should see all the messages ssent and received on the other side.
CONCLUSION
As described, EnMasse provides a really powerful messaging
infrastructure for developing an end-to-end IoT solution with the
needed scalability. Today, one of the main open-source projects focused
Figure 6: Address "anycast" creation from the web UI console. on IoT devices connectivity is already using EnMasse: Eclipse Hono.
The way that EnMasse is exposed outside of the cluster is through Hono provides a uniform interface as a set of well-defined APIs for
OpenShift routes. telemetry, events, and command/control for connecting a large
number of IoT devices. One of the available Hono deployments relies
Finally, the simpler way to have a couple of clients exchanging
on EnMasse for the underlying messaging infrastructure. These
messages via AMQP through the created address is to use the
projects together and their integration show how open source is
following Python sender and receiver.
driving the IoT world.
DZone, Inc.
DZone communities deliver over 6 million pages each month 150 Preston Executive Dr. Cary, NC 27513