Despite priding itself on technical inventiveness, large segments of the technology industry are propelled by marketing, a field where it's often better to be first than good and to ride hype rather than shed light. One of the latest areas in IT hampered by more fuzz than focus is 'edge computing'. I use scare quotes since, as I mentioned in discussing Red Hat's edge strategy, "the edge" isn't a uniform category, but a spectrum of application types, system designs, deployment and infrastructure models and performance requirements with enormous variety. Such heterogeneity means that there will never be an easy, succinct way to describe edge systems, but I agree with Gartner's Bob Gill (and many others) that a unifying quality is a need to directly and immediately process and act upon distributed sources of data at or very near the source.

Human nature being what it is, system and application designers tend to be lazy take the path of least resistance and apply existing mental models and implementation patterns to new problems, often with disastrous (at worst) or sub-optimal (at best) results. Edge infrastructure and applications are no different, where a common scenario — one endorsed and marketed by both enterprise systems vendors and cloud providers — entails putting small-scale replicas of an enterprise system and software stack in a remote location, whether that is a branch office, retail store, manufacturing site or managed hyper-local data center.

Facilities like Vapor IO's INZONE, which I profile here, combine low-latency local connectivity with massive cloud- and carrier-neutral backhaul that makes them ideal locations for many edge implementations and many IT architects will use them like a local appendage of a cloud environment. However, extending cloud infrastructure and application designs to a wireless base station only addresses a subset of possible edge scenarios, nor does a 'copy-downscale-and-paste' approach to edge implementation represent the best design methodology.

Over the edge - moving applications to sensors and devices

While the word 'edge' denotes a boundary or discontinuity, edge computing looks more like a fractal, with more detail and different patterns emerging the closer you look at the extremities. Adding processing intelligence to formerly single-purpose devices represents a significant technology trend that usually gets categorized as IoT, but represents an extreme form of edge computing.

What started with MPUs and other embedded processors has expanded to include composite SoCs in industrial and medical equipment, automobile control systems, entertainment devices and video cameras, DPUs (NVIDIA) and IPUs (Intel) in network interfaces and computational storage drives (CSD) that combine NAND flash, DRAM, processor cores and a PCIe switch in a single component. Each of these operates on data at the source to produce some actionable output and eliminates data movement to a traditional processor.

Although DPUs, CSDs and GPUs are initially being deployed in hyperscale cloud data centers to improve scalability and accelerate computationally intensive algorithms like encryption and deep learning, they can also be used in edge scenarios where the cost and latency associated with moving large amounts of data from thousands of endpoints, whether these are video cameras, self-service retail and point-of-sale systems, equipment sensors or industrial robots is excessive. However, none of these examples resembles the prevailing edge scenario of distributed servers running a VM or container stack in a remote location.

Thus, edge computing encompasses far more than hybrid systems like HCI products, AWS Outposts or Azure Stack Edge running traditional enterprise software, but includes small form-factor (SFF) systems like NVIDIA's Jetson, sensors embedded with a Raspberry Pi or system components such as a SmartNIC or CFD. Not only are these devices physically much different than the typical "edge server," but functionally and operationally, requiring new application designs and programming models.

New application designs

The variety of device edge applications means there are many ways of designing systems and software. A typical method embeds a functionally complete application in a smart device by programming to a domain- or device-specific SDKs such as NVIDIA Metropolis or Xilinx Storage Services, an application framework and developer tools for processing visual data with AI models. For example, Gillette Stadium (home to the New England Patriots) plans to use AI at the edge in processing thousand of security video streams to detect and alert staff to security threats and incidents. It also uses image recognition to inspect and segregate recycled material such as bundled cardboard that is categorized into seven grades before disposal.

The City of Raleigh, North Carolina plans to process signal camera feeds with image recognition AI models to count traffic, adjust signal timing during severe weather or special events and eventually provide the public with real-time data to improve trip planning. Many edge applications such as Raleigh's real-time traffic monitoring system rely on streaming data and only use persistent storage to maintain state or temporarily cache data. Swim calls these "continuous intelligence" applications and Swim's CTO Simon Crosby describes the limitations of existing designs for streaming applications this way (emphasis added):

Organizations are overwhelmed by streams of live data from their products, assets, partners, people, cloud services, their apps & IT infrastructure. 'Store then analyze' architectures struggle to meet the need for granular, context-rich, real-time insights. Most data is only ephemerally useful – but also boundless. High data volumes and distributed data sources make centralized collection expensive and slow, and applications fragile. Insights are coarse and arrive too late for decision makers on the ground who need granular, situationally relevant, continuous operational intelligence.

As Crosby indicates, streaming analytic applications have four properties making them well-suited for edge deployments at the sources of data.

1. The inputs are a sequence of state changes, where "state" doesn't imply binary, i.e. off-on, but includes quantized measurements, i.e. 1.1V→1.2V.
2. The input stream never stops, although it can include long sequences of the same values or periodic bursts of rapid, but small state changes that amount to noise. Such continuity makes "store-then-analyze" application designs unfeasible.
3. The applications must provide an "answer" (output) at any given time, whenever an information consumer, be that machine or person, needs a result.
4. The meaning of a particular output is dependent on the context, whether that is temporal, i.e. answers that recently preceded it, or proximal, i.e. the results from other sensors 'nearby' which is usually defined by a network graph.

As I first detailed in 2018, Swim's software platform enables continuous processing of streaming data at the edge using intelligent autonomous devices that mirror their state and build a network graph to share application properties, methods and data via Web Agents, a variation of digital twin designed for interrelated endpoints. Because agents locally process data and only share state changes (which amount to the output of an AI or statistical model) they can be sized to the computational difficulty of the task.

The capability of embedded SoCs with GPU or AI accelerators like Jetson, Google Edge TPU, Intel Neural Compute Stick 2 (NCS2) or Raspberry Pi 4 allows running complex deep learning and ML models at the data source. Furthermore, existing streaming applications running on the popular Kafka framework can now run on edge compute and storage devices like Samsung's SmartSDD.

My take

Like "cloud," "edge computing" is an overly broad term that leaves IT practitioners with wildly different understandings based on their pre-existing assumptions, current needs and limited knowledge. Among IT architects, edge often means a distributed cloud environment with some workloads distributed to local micro-datacenters or remote facilities. Wireless carriers typically think of edge as MEC (mobile edge computing) where a distributed radio (vRAN) and virtual services architecture allows placing previously centralized systems and services in base stations and high-traffic wiring closets.

Device or IoT edge, as I've discussed here, is an extreme form of the design paradigm in which data processing is pushed into the source by exploiting tremendous advances in semiconductor density, functional integration  (via SoCs, MCPs, etc.) and distributed software frameworks. Moving workloads as near the data source as possible has many advantages including, faster, more responsive performance (low latency), greater efficiency, higher scalability and lower cost.

As the world further bathes in a flood of generally unusable data, some of the most innovative and rewarding new services and software systems will come from developers and system architects that espouse extreme edge computing and primarily leave centralized cloud services for computationally intensive workloads and transactional systems where data that is necessarily aggregated.