Menu

IoT FEATURE NEWS

The Role of In-Memory Computing and HTAP in High Performance IoT Architectures

By

By 2020, Gartner expects over 20 billion devices to be connected to the Internet of Things (IoT). According to the “Cisco Global Cloud Index: Forecast and Methodology, 2015–2020”, the total amount of data created by devices, driven by IoT, will reach 600 ZB per year by 2020, up from 145 ZB per year in 2015. To deliver on the promise of IoT, successful IoT platforms will need a data architecture that can deliver the performance and scale to deliver real-time insights on vast amounts of data. One of the most promising technologies to address these challenges is in-memory computing-based Hybrid Transactional and Analytical Processing (HTAP).

IoT Use Cases
Consider just a handful of the IoT use cases that demand fast data processing and the resulting real-time insights to drive actions:

  • Manufacturers are using sensors to track production, streamline industrial processes, and monitor inventory in industries ranging from food to logistics
  • In logistics, self-driving vehicles will produce and consume massive amounts of data while determining optimal routes and minimizing fuel usage
  • In transportation, both ride-sharing companies and taxi drivers are using apps to broadcast anonymous information about their travel, becoming part of a broader effort to implement intelligent traffic control
  • Healthcare organizations are implementing both hospital and residential-based patient monitoring, as well as environmental monitoring
  • Connected home products include security devices, thermostats, refrigerators, robots, and garden sensors

In most of these use cases, the analysis of IoT data must occur in real or near-real time to drive appropriate action. If analytics are not processed in time to prevent a patient emergency or food spoilage, then the benefits of IoT can be minimal.

In-Memory Hybrid Transactional/Analytical Processing for IoT
An infrastructure capable of the speed and scale required for IoT can be built on hybrid transactional and analytical processing solutions that use a single data store for transactions and analytics. Until recently, in-memory HTAP was expensive. Users were also typically concerned about the impact slow running queries could have on the ability to process transactions in real-time. Today, however, in-memory HTAP solutions are increasingly affordable thanks to reductions in the cost of memory. In addition, analytical processing, often based on streaming technology, greatly reduced concerns about analytics impacting operations. The simplified architecture of HTAP solutions also offers considerable cost savings potential compared to two side-by-side solutions – one for OLTP and one for OLAP.

There are three current approaches to IoT platforms: proprietary, cloud-based and open source. The distinctions across these solutions mirror the distinctions across other solution categories.

Proprietary
These solutions may be purpose-built and address a specific use case well but may come with high costs and result in vendor lock-in. Examples include GE Predix, the Bosch IoT Suite, or PTC ThingWorx.

Cloud-based
These solutions can be cost-effective and easy to implement for certain IoT applications. However, they can have rigid application and device integration interfaces and are unable to provide real-time analytics so are more commonly used for non-real-time IoT applications. Examples include AWS IoT and Azure IoT Suite.

Open Source
These solutions can be very flexible because the source code can be customized. They can also be cost effective and result in no vendor lock-in. Some of these solutions offer the in-memory capabilities necessary for real-time IoT applications. Examples include the SMACK stack (Apache® Kafka™, Mesos™, Akka™, Cassandra™ and Spark™) and Apache® Ignite™.

In-Memory Computing Platforms for IoT
The basic elements required for an in-memory HTAP solution for Internet of Things applications include:

  • An in-memory data grid for distributed data storage and scalability
  • An in-memory compute grid for high performance parallel processing of queries
  • In-memory streaming processing to allow analysis of incoming data in real-time
  • An in-memory SQL grid that provides in-memory database capabilities including support for SQL communications through ODBC/JDBC and may provide geospatial support for location-based IoT use cases

These interconnected modules can form a single, easily managed layer running in memory for the type of speed and scalability needed to deal with the torrent of data flowing from IoT devices. By enabling these core pieces of the IoT architecture to be implemented in an integrated, scalable, high performant, in-memory computing platform, IoT applications can manage the fast data processing required for next-generation IoT solutions.




Edited by Ken Briodagh
Get stories like this delivered straight to your inbox. [Free eNews Subscription]


SHARE THIS ARTICLE
Related Articles

Rising Edge Computing Investments to Reach $350B by 2027, According to IDC

By: Alex Passett    3/27/2024

Worldwide spending on edge computing is expected to surge (and then keep going) for the foreseeable future, according to the International Data Corpor…

Read More

ZEDEDA Adds Lisa Edwards as New Board Member, Seeks Opportunities to Strengthen Operations and Scale

By: Alex Passett    3/26/2024

Earlier this morning, ZEDEDA announced the addition of Lisa Edwards to its board of directors.

Read More

An Existing IoT Collab, Emboldened: Digi International and Telit Cinterion Transform Solutions with 5G RedCap Integration

By: Alex Passett    3/25/2024

The ongoing industry collaboration between Digi International and Telit Cinterion signals strong support for the mainstream showcasing of 5G for IoT a…

Read More

Telit Cinterion's 5G LGA Modules, Powered by Snapdragon from Qualcomm, to Create a Big Leap in IoT Connectivity

By: Alex Passett    3/25/2024

Telit Cinterion recently unveiled its FE990B34/40 LGA family of modules, powered by the Snapdragon X72 5G Modem-RF System from Qualcomm Technologies, …

Read More

Embracing Innovation in Mining: The Role of Network-Aware Applications in the Digital Transformation

By: Special Guest    3/21/2024

Shabodi leverages private 5G network capabilities and enables the development of network-aware applications to enhance operational efficiency, automat…

Read More