ABSTRACT
Efforts related to Internet of Things (IoT), Cyber-Physical Systems (CPS), Machine to Machine (M2M) technologies, Industrial Internet, and Smart Cities aim to improve society through the coordination of distributed devices and analysis of resulting data. By the year 2020 there will be an estimated 50 billion network connected devices globally and 43 trillion gigabytes of electronic data. Current practices of moving data directly from end-devices to remote and potentially distant cloud computing services will not be sufficient to manage future device and data growth.
Edge Computing is the migration of computational functionality to sources of data generation. The importance of edge computing increases with the size and complexity of devices and resulting data. In addition, the coordination of global edge-to- edge communications, shared resources, high-level application scheduling, monitoring, measurement, and Quality of Service (QoS) enforcement will be critical to address the rapid growth of connected devices and associated data.
We present a new distributed agent-based framework designed to address the challenges of edge computing. This actor-model framework implementation is designed to manage large numbers of geographically distributed services, comprised from heterogeneous resources and communication protocols, in support of low-latency real-time streaming applications. As part of this framework, an application description language was developed and implemented. Using the application description language a number of high-order management modules were implemented including solutions for resource and workload comparison, performance observation, scheduling, and provisioning. A number of hypothetical and real-world use cases are described to support the framework implementation.
THE ARCHITECTURAL MODEL
Figure 2.1: Cresco Smart Grid Hierarchy
In this context, the Agent-level hierarchy would existing within the home, potentially with a Cresco-Agent embedded within a smart power meter functioning as a smart home service gateway, using Cresco-Plugins to interface with various lower-level protocols and related devices. In such an arrangement there might exist billions of plugin-managed devices, millions of homes supported by agents, thousands of cites managed as regions, and a small number of global entities with a national view of the power grid. An example of Cresco components used in a smart grid application, arranged in the Cresco hierarchy, is shown in Figure 2.1.
CRESCO IMPLEMENTATION
Figure 3.4: Msg Event Routing process for the Cresco-Agent
Messages are either routed to a specific plugin or executed on the agent itself. The route thread terminates on route completion, which is sufficient for unidirectional messages. However, there are cases where we want a response to our messages. Typically, method execution is a blocking function, where the calling instance must wait (is blocked) for a method to complete before continuing its processing. This mode of operation is considered a synchronous operation. In a distributed system with many agents and plugins we do not want to block the execution path, while waiting on methods to complete.
Figure 3.9: Secret-Key Managed Dynamic Discovery
Complex structures of networks of agents can be constructed through dynamic discovery managed by shared discovery secrets. Figure 3.9 shows an agent structure where two regions have been created dynamically through the use of differing agent discovery secrets in the same discovery domain. During the agent discovery process the existing regional controller is unable to decrypt and respond to the discovery message, so the agent promotes itself to a regional controller. The new regional controller is able to communicate with the old regional controller since the shared regional discovery secrets are the same.
FRAMEWORK TECHNOLOGIES
Figure 4.2: Cresco Application Graph
On submission to the Cresco Global Controller, a pipeline Node is created to record the initial CADL description. The incoming CADL expression is then forwarded to the App Scheduler Engine process for scheduling, where nodes described in the pipeline are represented as vNodes in the database. If needed, iNodes are created for each vNode to represent plugin implementations of request vNodes. However, iNodes can be shared between pipelines.
CASE STUDIES OF EDGE COMPUTING
Figure 5.1: Storm Topology
In this context, a tuple is a data diagrams in the form of a key-value pair. Bolts read tuples from either Spouts or other Bolts, and also typically emit a tuple stream. Normally, tuple transformations, operations, and external data drains occur in Bolts. Our Storm topology is shown in Figure 5.1. Similar to MapReduce, Storm distributes and processes tuples of information on multiple nodes and processes. However, unlike MapReduce, Storm will process tuples until the job is manually terminated.
Figure 5.12: Pipeline DAG
Genomic processing pipelines can be represented by a directed acyclic graph (DAG). In a pipeline DAG, nodes represent data sources or transformations, while edges represent the flow of the pipeline between nodes, as shown in Figure 5.12.The simple pipeline shown in the above figure is representative of a single phase pipeline, such as the conversion of raw sequence images to Fasta records.
Nodes represent context-specific functional workloads, such as data transformation, enrichment, processing, etc., edges are points of data exchange. Specifically, nodes represent functional modules that provide sets of tools and data sources used in genomic processing. Nodes can act as both sources and destinations for flows of data, as defined by the corresponding directed edges.
BUILDING A SMART CITY APPLICATION
Figure 6.2: City Data and Cresco Topology
In this section, we describe the design aspects of a Cresco application used in the management of Smart Cities. As previously discussed, the Cresco framework operates in a hierarchy, which is configured based on application requirements and available resources. Figure 6.2, shows the hierarchical mapping of Cresco components to Smart City workload, resources, and designated data processing locations (PP, COP, CP). In the figure the data designated by dashed lines represents Cresco Plugin-to-Plugin workload communications, which is commonly referred to as the Data Plane.
Figure 6.5: CP Data and Cresco Topology
CP analysis could require significant computational, network, and storage resources, perhaps beyond what is available within existing infrastructures. As with COP, CP resources will vary based on load. However, unlike COP Cresco will provision additional infrastructure capacity as needed for CP operations. Infrastructure capacity will be increased through the provisioning of virtual machines running Cresco agents, which will in turn allow for the provisioning of additional CP work-load processors. CP workload processors will subscribe to message queues provided by distributed COPs. The Cresco components that make up CPs are shown in Figure 6.5.
CONCLUSIONS
There is no generally accepted theory for edge computing. In this dissertation the characteristics, challenges, and motivations for edge computing were presented. A number of real-world use cases were described to support claims that in some cases moving computation resource assignments to sources of data is more effective than moving data to computational resources. As the number of connected devices increase globally, so will the need for intelligent end-to-end management of computational resources, workloads, and data.
An edge computing framework named Cresco was developed. While a number of open source packages were used to develop the framework all core software development, with the exception of the Cresco libraries discussed in Cresco Plugin Library and, Cresco Library, were developed by the author. Cresco libraries were developed by Caylin Hickey using common code that was originally written by the author and repeated between components. A number of custom plugins that where not covered in this dissertation have been developed by the author and others.
In the next section we describe work related to the Cresco framework described in this dissertation.
Source:Â University of Kentucky
Author: Vernon K.Bumgardner
>> IoT based Big Data and Cloud Computing Projects for B.E/B.Tech Students
>> 200+ IoT Led Projects for Final Year Students
>> IoT Environmental Monitoring System for Smart Cities Projects for Final Year Students
>> IoT Software Projects for Final Year Students