Analysis of Current Trends in Internet of Things Gateway and Edge Data Processing Characteristics

Download Full-Text PDF Cite this Publication

Text Only Version

Analysis of Current Trends in Internet of Things Gateway and Edge Data Processing Characteristics

Hariharan Ramalingam

Research Scholar PhD, Department of Banking Technology Pondicherry University, Puducherry, India.

Dr. V. Prasanna Venkatesan


Department of Banking Technology Pondicherry University, Puducherry, India.

Abstract With multiple sensor nodes catching up with physical world for digitization, data logging, data processing there is lot of traffic towards the internet and hitting the cloud/remote datacentres. In this paper, the IoT gateway, middleware and edge data processing characteristics are analysed by looking into current industry products in the IoT gateway space and prior research on different techniques to improve IoT gateway system in the edge. Main challenges such as network latency, throughput is discussed and modelling towards realizing an ideal IoT gateway is highlighted. Efficiency driving parameters such as data throughput, data encryption/decryption techniques in the edge that also contribute towards smart IoT gateway is discussed with recommendations.

Keywords Internet of Things (IoT), Gateway, Middleware, Protocol, Sensors, Throughput, Latency, Reduction ratio


    A Gateway is an edge device appliance which acts as a communication machine, local device hub for processing heterogenous data sources from sensors in proximity and field. It bridges the IoT device that interacts with the physical world to the applications that run-in cloud or remote servers which offer software services to the customers. IoT gateway today is offered by various Industry as a product with different features that serve the service requirements i.e. health, factory, Automotive, Office, home automation etc. The parameters such as power consumption, network latency, connectivity, processing capacity in the edge is a challenge to the gateway devices and is subjective to applications or services offered. By design the IoT gateway is dependent on the sensor devices for interacting with the physical world and for processing the compute ability is managed by cloud which adds to network bandwidth, connectivity challenges.


    IoT Gateway operates with functions such as translating the data, authenticating and later authorizing the data packets, request from devices to the software layer or middleware. In this section different papers related to IoT Gateway design are discussed.

    Francisco et al. [1] highlights the various parameters involved in designing an IoT Gateway followed by its functionality. The emphasizes is on gateway being a crucial

    element of the IoT architecture and does manage the data from various sensors using different wireless protocols in the short range (Bluetooh, Wifi) and long range (LoRAWAN, 6LoWPAN etc). It details on the communication interfaces, the middleware, gateway communications and presents opportunities in areas such as redundancy, messaging semantics, ontology, traffic optimization, middleware abstraction, atomic vs persistent connection, discovery protocols (universal Plug and Play). Practical results are obtained using the Arduino, ESP8266 based Hardware and services such as Soil moisture, Luminosity, temperature, humidity, Power and humidity are exhibited. The next paper by Mastilak et al. [2] details the protocols used by the sensors and Gateway and related challenges driving the need for customizing protocols towards smart gateway.

    Mastilak et al. [2] proposes architecture of smart gateway and application protocol for low cost devices in IoT with focus on security and reliability. It also discusses on the measured results such as packet loss and delay and brings in details of the following protocol

    • Constrained Application Protocol (CoAP) is based on Representational State transfer (REST) model and is for Low energy devices such as sensors that interact with web environment. CoAP uses four message types, i.e. message that requires confirmation, message that does not require confirmation, reset and confirmation (ack).

    • Message Queueing Telemetry Transport (MQTT) is a protocol that uses Publish/subscribe design pattern. It consists of three components subscriber, publisher and broker. There is one node called broker and this node manages message distribution. Messages are sorted into topics, the publisher device publishes messages of specific topic and subscriber receives the message. Broker distributes the message sent by the Publisher to all subscribers who are subscribed to specific topic.

    • Extensible Messaging and Presence protocol (XMPP) is a protocol that helps to build solid, secure and interoperable devices, services and application for the Internet of things. It provides several choices how to connect devices such as socket connections, bi-directional streams or XML interchange. It can be used for many types of devices like sensors, controllers etc. In XMPP-IoT the devices are restricted to have a flat model consisting of one or many nodes while each node can have several fields. The XMPP-

      IoT recalls the basic XMPP principle that can be called notion of friends i.e. to exchange data between two end points they need to be friends first meaning they need to be subscribed to each other, providing their presence status to other devices.

      The challenge for these protocols is that they are designed for usage in DTLS (data transport layer security) or TLS and their operation in many platforms is very poor especially with low power IoT environment. Hence the paper proposes a custom protocol for IoT environment with smart gateway and discusses the architecture of smart gateway that could reduce interoperability problems. The smart gateway is composed of web application, middleware and application that enables to control IoT end devices. The web applications is used to communicate with external devices, storage of data from sensors using data bases, keeping a list of policies for end devices and RESTful API to communicate with applications proposed to control IoT end devices. The application that controls the IoT devices is composed of modules such as encryption (for messages), Authentication (for end devices), Retransmission (when loss of packet in network occurs), Controller of network interfaces (sending and receiving message for different interfaces), Planning (stores list of rules and creates correct commands), Data collection & formatting to JSON (for messages sent to Web application via RESTful API). The protocol was tested using Raspberry Pi3 as smart gateway and three Arduinos as end devices. The results with smart gateway proved that its suitable for devices that are not critical, low power external devices in a secured & efficient communication. The following paper with Cu et al. [3] bring a framework about reliability, availability and serviceability for the IoT Gateway with end to end impact which talks about the lower layer communication protocols design parameters to meet the framework requirements.

      Cu et al. [3] discusses reliability, availability and serviceability (RAS) for IoT Gateway. The purpose is to protection of data integrity and make the availability longer. The paper discusses the current IoT Gateway overview and proposes the RAS based architecture for IoT Gateway. RAS events reporting from kernel in the gateway are listed as Machine check Exceptions (MCE), Error detection and Correction (EDAC) and Peripheral component interconnect express advanced error reporting (PCIe AER) which functions to detect, report, manage errors in CPU, hardware and PCI hardware. IoT gateway will have the modules such as Data collector, Handler, OS and broker for collecting, monitoring and handling RAS data reported by kernel in the IoT Gateway. The paper proposes opportunities for error pattern recognition with RAS data using with data mining techniques. The next paper of Yacchirema and Palau [4] brings in the heterogeneity aspects of IoT as a system.

      Yacchirema and Palau [4] discusses on the challenges of IoT Gateway to deal with heterogeneity in the IoT system that includes devices, technologies, hardware and communication protocols. It proposes Smart IoT gateway which can have the following features

      • Enables connectivity of different protocols and traditional communication technologies such as Ethernet (wired) and wireless protocols Zigbee, Bluetooth and Wifi.

      • Uses flexible protocol to manage data from different sensors and do analysis on it.

      • Uses a light weight and optimal protocol for devices with limited resources.

      • Provides local data storage.

    Given the Smart IoT gateway the challenge is always on the access time for data from the IoT gateway which is take up as an opportunity by Azad et al. [5].

    Azad et al. [5] proposes an access time improvement framework for IoT gateways. The framework relies on the look up table at the architecture level which supports a method where data translation is done which can be stored and retrieved on the IoT gateway. Multiple gateways are connected and managed to handle the algorithms (Hashing based) and data workload from service providers. This in turn reduces the access time based on service request coming from micro services based APIs/applications. Experimental set up is done for performance analysis on the entire framework. The framework drives improvement in access time based on distributed data publication and consumption.

    With the above review and discussion, the need for smart IoT gateway for dealing with heterogeneous data environment is justified. Middleware plays a crucial role in the smart IoT Gateway managing the incoming data from sensors and delivering it to applications. By the architecture of hardware and value add features from middleware with applications realizing a smart IoT gateway is feasible.


    Some of the products in the current market has overcome this with ability to process in the edge with subjective or focussed application use cases. For understanding various products in the Industry, the table below highlights products with edge processing features versus just Cloud leverage.




    IoT Gateway Product


    Edge data anal ytics supp ort

    Cloud support



    EIS D120 [6]

    Manages data from multiple sensors and offers devices management


    Optional MS

    Azure support

    Advantec h


    Wavelet kits [7]

    Allows edge data storage and processing.





    Bosch IoT Gateway [8]

    Supports OSGi

    middleware for remote management





    Dell Edge Gateway 5000 series


    Manages data from multiple sensors and offers support for remote management





    SGX 5150

    IoT Gateway [10]

    Sensor data management





    Meshlium Xtreme IoT Gateway [11]

    Sensor data management, local storage support





    Cloud Gateway (3rd party HW) [12]

    In any IoT HW allows to connect to MS Azure cloud





    Gateway xi Edge [13]

    Local data processing from sensors and lower bandwidth support.





    IoT Gateway (3rd party HW) [14]

    Allows to

    connect to

    IBM IoT


    Yes (dep ends on Manf






    Industrial Compute Gateway [15]

    Data management from multiple sensors and

    support for edge/fog processing





    IoT Gateway [16]

    Sensor data management, support for resiliency, redundancy, edge/fog data processing.





    Industrial IoT Gateway [17]

    Sensor data management, local data processing




    Note: The table above is referred from the various product data sheet & specs from the vendor and is subjective to feature changes with reference to vendor product update.

    And there are many more IoT Gateways in development and available online for reference. With reference to the current trends the edge processing and cloud-based products are available in the market. This architecture where the cloud and edge processing work together balancing as per the workload offers benefits in network throughput, latency and energy efficiency. Middleware architecture drives the sensor data management, security, ability to manage local storage and edge processing, later connecting to applications using APIs to offer services. As the IoT products are deployment in large numbers for a use case like smart cities or Industry 4.0 or chain of retail outlets, these features very much matter for productivity or ROI of the organization.


    IoT as a platform or system consists of Edge IoT devices, Gateway and connectivity to remote servers for application/ cloud. IoT gateway acts an edge aggregation node between the sensors and cloud/remote datacenters. IoT gateway system relies on group of devices connected over wireless/ wired

    networks that is controlled by an application using communication protocols. IoT gateway core subsystems are Physical HW/Sensor interface, Drivers/firmware for the H/W, Middleware for data handling, security and processing, Application for Service, QoS requirement. With these IoT gateway handles the data from Sensors devices and process it for internet data transmission to the remote data centers/cloud for Service level processing. Many IoT gateways are deployed for domain application such as smart cities, Smart retail etc.

    1. Achieving Data reduction using an adaptive method with LMS windowed filters

      Handling high raw data volume for processing in the IoT gateway drives high energy consumption [18]. Reducing such high energy consumption is the IoT gateway is governed by the the number of data transmission. So reducing the number of data transmission will contribute to energy efficiency. The adaptive method for data reduction discussed in the paper uses the two decoupled measured values from sensor nodes and gateway devices with a convex combination. Its a prediction technique that uses LMS adaptive filters. Here the algorithm drives for two modes to arrive at the operation that includes data reduction i.e. initialization mode, normal mode and standalone mode. With reference to figure 1 the table 2 describes the functioalities of different modes.



      Operational Devices



      Gateway and Sensor nodes

      User query received in Gateway. Sensor starts transmitting observations

      Learning and error rate are initialized at Gateway and sensor devices with same values.


      Sensor nodes

      Based on prediction values reaching good approximation, execution starts.

      Based on sensor values (immediate) convex combination filter weights must be updated.

      Based on threshold vs prediction error rate device switches to stand alone mode based on sensor data to predict the upcoming measurements.

      Stand alone

      Sensor nodes

      Nodes function based on prediction model data deviation lesser than the error rate threshold value.

      No change in weights for convex combination filter


      The above algorithm with data sets such as temperature data has proved 95% communication reduction.

    2. Achieving lower network latency using Named Data Networking (NDN)

      Raw data processing from the sensors i.e. part of Internet of things at the network edge is a challenge and opportunity for data reduction which guarantees lower latency interaction compared to traditional computing to the remote cloud [19]. Information centric networking (ICN) solutions can improve IoT data processing at the network edge. ICN communication is connectionless and driven by names, which are directly used at the network layer for data retrieval. ICN instantiation called


      The IoT NCN algorithm was simulated with heavy workloads and it outperforms the legacy approach.


      Named Data networking (NDN) supports sensing, data collection and actuation tasks in IoT scenarios like smart homes, buildings. NDN is extended to IoT named Computing

      1. Heavy workload services no. of data

        packets transmitted

      2. Light workload services no. of data packets


      networking (IoT-NCN) for IoT data processing at the edge and is designed to manage computation requests over IoT contents. To enabled in-network IoT data processing, the naming scheme needs to identify the IoT contents and the processing service to be executed on them. The service name can include a limited set of parameters as input for the execution. A novel data structure, called service table is added to the NDN architecture of computing nodes to store the name of the available services. Based on the uniquely named service function the compute code runs based on the availability of service. A Ranking engine controls the available of nodes based on computing resources and service request popularity. This updates the service table accordingly.

      The IoT NCN interest is forwarded towards IoT data sources until it reaches the IoT gateway. The Gateway is selected during the forwarding process from source cloud data centers by looking at the scope of the content names and forwarding information base entries per that name. To handle the processing load the IoT NCN strategy allows the Gateway to dynamically candidate themselves for the service execution, if they have enough capabilities to do it.

    3. Achieving network throughput with Knowledge discovery from Data of Internet of things

      Massive data generated in IoT when applied to data mining techniques leads to knowledge discovery which reveals useful data for services or applications [20]. This leverage of knowledge discovery in databases also contributes to the throughput of data communication between the IoT Gateway devices and remote cloud servers or data centres avoiding raw data encapsulated into the IP packets and contributing to network bandwidth and latency. Data mining steps plays key role in extracting interesting patterns from the raw data and then decision making does the process of making selected data into knowledge. The steps are broadly classified into two major sections

      1. Data processing

        1. Selection

        2. Pre-processing

        3. Transformation

      2. Decision making

        1. Interpretation

        2. Evaluation

          The figure 4 below highlights the IoT system where KDD is integrated for arriving at knowledge.


          Application domain areas such as Big Data processing rely on the KDD from IoT raw data. The workload balancing on selective data mining decides the edge network throughput and is subjective to services. IoT service quality and system performance will be impacted by data fusion, large volume data and decentralized computing. So for processing IoT raw data following three considerations are recommended for selecting the appropriate mining technologies

          1. Objective The problem parameters such as assumptions, limitations and measurements needs to be specified.

          2. Characteristics of data Based on application domain, data characteristics such as Size, distribution and representation parameters of data need to be known.

          3. Mining algorithm – With above two parameters the data mining algorithm can be determined.

          Different mining algorithms such as k-means for clustering, decision tree for classification and apriori algorithm for association rules can be considered for deterministic mining algorithms. For meta heuristics algorithms simulated annealing and genetic algorithms can be considered. After the data mining the information has to get into decision making steps such as interpretation and evaluation leading to knowledge. This feeds back to application which manages the service offered to consumers.

          Overall the KDD with IoT system offers system performance based on balancing the selective workload between the edge and cloud for processing. This contributes to physical parameters such as network throughput, latency etc.

    4. Achieving data reduction at network edge of IoT systems using concept of Perceptually important points (PIP)

      Huge data is expected from the growing IoT edge systems which are connected to multiple sensors with links to wireless, mobile networks [21]. The application such as smart meter, medical devices sends lots of data for processing. These are raw data from the IoT devices to be proceesed and needs applications to make into meaningful service delivery for consumers. So there is significant processing required which is the challenge to the edge systems. A solution is presented for real time data reduction that automates switching in the network edge between different data handling algorithm such as adjusted data reduction methods and three types of new algorithm (based on PIP concept). This approach guarantees

      improvement in the four parameters i.e. Network bandwidth, energy consumption, I/O throughput, cloud storage and traffic costs. The middleware deployed in the IoT Gateway provides API to the gateway applications that collects data and can instantiate, customize different kinds of data handlers from source. These data are monitored, customized for pre- processing and compute information about the re- constructability of the data that is being forwarded. Selective data handlers used in the middleware for different types of data is discussed in the table below


      Name of Handlers


      Sampling handlers

      Regular sampling is performed without looking into the data. Ex: for rate 1:2 it forwards every 2nd data item that it receive.

      Piece wise approximation handler

      Forwards an average value for every N values with consideration on size of the window and variations

      Selective Forwarding handler

      Based on specified threshold, ranges, lists or all the data is forwarded

      Important points handler

      Forwards Perceptually important points.

      Change detection handler

      Forwards based on criteria such as loss less (every data that is different from the prev. one), Threshold based.

      Given the data sets such as smart meters, data reductions up to 93.8% is achievable using the above solution.

    5. Recommended model

    Based on the above discussion the parameters such as network latency, bandwidth, throughput are vital towards performance of an IoT Gateway system. The system should have the architecture to support the performance of these parameters. Based on this goal lets model towards an ideal IoT gateway system. Figure 5 describes the IoT system architecture with sensors generating data from the physical world towards Gateway. IoT Gateway has Physical HW, drivers, middleware and applications which manages the data to remote servers/ cloud and applications/services related.


    For the modelling let us consider the IoT data raw data from the sensors, lets use the variable to represent the time series data from sensors towards the gateway i.e.

    = {x1, , x2, x3,xn}

    where x1..n represents time series data from the sensor

    devices that interacts with the physical world. The IoT gateway will sense all these data in raw form and towards the goal there should be a data reduction before its encapsulated into IP packets to the internet or cloud or remote datacenters. Lets use the variable to represent the time series data after IoT gateway processing i.e. after the data reduction

    = {y1, y2, y3 ..yn}

    If the difference between input and output is becoming zero then there is no reduction. However if there is a difference then it contributes to the latency and throughput of the system. represents the reduction parameter and can vary based on data types and domain applications. So


    In case of raw data with complete reduction

    Based on Littles law which states throughput x Latency

    = Queue size. So the equation is


    is the parameter to represent latency

    is the parameter to represent throughput

    is the parameter to represent the Queue/ data size In network transmission i.e. refers to a variety of potential processing delays affecting network data transmission and is the actual amount of data flowing through. The higher the

    the lower the will be. For any given moment the data size from the IoT Gateway has impact on throughput and Latency. The algorithms used in the IoT gateway system has influence over the size of the data transmission.

    In this context where output data is , then should be represented as

    = + , is the encapsulation parameter added to data for transmission

    i.e. | |

    = 0 , = 1


    Achieving a minimal size is the key towards achieving a

    R will be a variant based on the data types and algorithms used.

    better latency

    which also i

    mproves speed, lesser memory and

    Considering a model where Reduction ratio is applied to the input data in the ideal IoT smart gateway, the output data is expected to reduce. Table 4 and figure 6 is an illustration of the data reduction once reduction ratio is applied. Here reduction ratio is based on the encryption technique or local data processing on the edge gateway.


    Input (data in bps)

    Reduction ratio

    Output data (in bps)























    error reduction. This in turn gives the opportunity for

    improvement of network throughput .


    Internet of things system has devices such as sensors and gateway that connects to internet and cloud for processing the data from the physical world. The above models discussed highlights the way to improve the network parameters such as throughput and latency. As discussed, earlier middleware plays a crucial role in the IoT gateway for achieving performance, reliability power, latency and throughput. APIs work with middleware to enable services and later link to application layer. Recommendation is to aim for solutions on optimizing physical parameters such as latency, throughput for IoT Gateway. Here the Reduction parameter plays a key role when applied to the incoming data from IoT devices towards the gateway and the same is show in the figure 6. Any design of IoT gateway middleware has to consider this as an opportunity which could drive efficiency for the overall Gateway system and related services. The opportunities for power efficiency can also be achieved as a system by clustering the resources [22] on need based services instead keep always functional. These parameters have scope of improvement by researching on encryption technique for light weight processing and data reduction techniques for the edge. Considering the edge systems have to be momentarily autonomous / smart for selected services there is lot of scope for research on Smart Gateway [23]. Deterministic and non- deterministic algorithms [24] do have scope in edge gateway characteristics optimization. Scope of this paper is to bring in the highlights and model towards ideal IoT gateway that targets future opportunities with various design parameters such as network throughput, latency, encryption & decryption techniques towards reliable and available communication between remote datacenters and the edge system.


Compute and network processing for IoT data is increasing based on applications such as smart homes, smart cities, driver less cars etc. The medium of communication in the last mile being wireless and cellular brings in lot of innovation scope for optimized data pipe between Cloud/remote data centres and IoT gateway. The discussed models in this paper will drive products with feature enhancement in this space. As highlighted in the earlier part of the paper current trends with IoT gateway products do fall is catching to this space and in future it will be thee new normal.


I like to acknowledge Dr. V. Prasanna Venkatesan, Professor, Department of Banking technology, School of Management, Pondicherry university for his guidance and support to carry out the study in a successful manner. I also acknowledge Pondicherry University for providing me good environment and facilities to carry out my research work.


  1. Francisco L. de Caldas Filho, Lucas M.C. e Martins, Ingrid Palma Araujo, Fabio L. L. de Mendonca, Joao Paulo C.L. da Costa, Rafael T. de Sousa Junior, Design and Evaluation of a Semantic Gateway Prototype for IoT Networks, UCIoT 2017 Workshop Presentation, Austin, Texas, USA December 5-8, 2017.

  2. L. Mastilak, M. Galinski, I. Kotuliak, M.Ries, Improved Smart Gateway in IoT, IEEE International conference on Emerging eLearning technologies and Application, Nov 218.

  3. Cu PHAM, Tan LE, Yuto LIM, Yasuo TAN, An architecture for supporting RAS on Linux-based IoT Gateways, IEEE 6th Global conference on Consumer Electronics (GCCE 2017), 2017.

  4. D.C. Yacchirema and C.Palau ,Smart IoT Gateway for Heterogeneous Devices Interoperability, IEEE Latin America Transactions, Vol.14, Issue:8, Aug 2016.

  5. Asad Javed, Narges Yousefnezhad, Jeremy Robert, Keijo Heljanko, Kary Farmling, Access Time improvement Framework for Standardized IoT Gateways, IEEE International conference on Pervasive computing and Communication Workshops, IEEE, Mar 2019.

  6. Product Datasheet for EIS-D120 Multiple I/O edge intelligence server, Advantech – 4bb8-bb07-2cb06da757a1/eis-d120/mod_30dbfd63-c67b-4c7c-98a6- 71c80f385279

  7. Product catalog for Waveletkits Industrial IoT system for remote data acquisition Ayyeka –

  8. Product Bosch IoT Gateway – platform/bosch-iot-suite/homepage-bosch-iot-suite.html?ref=ot-2-inst- de-2017p-sales-contact-forms-iot-platform

  9. Product SPEC sheet for DELL edge Gateway series sheets/en/Documents/dell_gateway_5000_internet_of_things_specsheet


  10. Product Data sheet for SGX 5150- Industrial grade IoT device Gateway

    – gateways/

  11. Product details for Meshlium Xtreme IoT Gateway to connect any sensor to any cloud –

  12. Product details MSAzure IoT Gateway –

  13. Product details Xi IOT, Nutanix,

  14. Product details IBM Watson IoT gateway- gateway-to-iot-foundation/

  15. Product SPEC IC3000 Industrial IoT Gateway, CISCO – industrial-compute-gateways/datasheet-c78-741204.html

  16. Product details, Secure Industrial IoT with Edge computing, RAD,

  17. Product details, Industrial IoT Gateway, PHYTEC,

  18. Yasmin Fathy, Payam Barnaghi, Rahim Tafazolli, An Adaptive Method for Data Reduction in the Internet of things, IEEE 4th World forum on IoT, IEEE, 2018.

  19. Marica Amadeo, Claudia Campolo, Antonella Molinaro, Guiseppe Ruggeri, IoT Data processing at the Edge with Named Data Networking, 24th European Wireless Conference, IEEE, June 2018.

  20. Mudasir Shafi, Sumiran Daiya, Sumit Dayal, Data from IoT and Knowledge discovery from Data, International Journal of Innovation & Advancement in Computer Science (IJIACS), Vol 7, Issue 4, April 2018.

  21. Apostolos Papageorgiou, Bin Cheng, Erno Kovacs, Real-Time Data reduction at the Network Edge of Internet of things System, 11th International conference on Network and Service Management (CNSM), IEEE, Jan 2016.

  22. Hariharan Ramalingam, Dr.V.Prasanna Venkatesan, Ultra swarm method for resource management in IoT deployed Mesh networks, Internation Conference on Distributed Intelligent Computing, 2016.

  23. Dr.C. Punitha Devi, M.Sivaranjani, Dr.V.Prasanna Venkatesan, Exploration of Specific challenges in IoT with a smart gateway solution, Internation Conference on I-SMAC (IoT in Social, mobile, Analytics and Cloud), IEEE, Oct 2017.

  24. Hariharan Ramalingam, Dr.V.Prasanna Venkatesan, Survey of bio inspired algorithms in edge computing, Journal of Emerging Technologies and Innovation Research (JETIR), Vol 5, Issue 12, Dec 2018.

Leave a Reply

Your email address will not be published. Required fields are marked *