The SNIA Networking Storage Forum (NSF) kicked off our “Storage Life on the Edge” webcast series with a session on managing data from the edge to the cloud and back. We were fortunate to have a panel of experts, Dan Cummins, John Kim and David McIntyre to explain key considerations when managing and processing data generated at the edge. If you missed this introductory session, it’s available on-demand, along with the presentation slides at the SNIA Educational Library.
Our presenters spent a good percentage of time answering questions from our live audience. Here are answers to them all.
Q. Could an application be deployed simultaneously at near-edge, far edge and functional edge?
A. [Dan] Many of the applications are the outcomes that customers need depending on the vertical market; whether that’s manufacturing, retail or and gas. They require processing that’s distributed at each of these locations so absolutely if we go back to that computer vision
use case you can see that part of the application or the outcome includes an IoT stack that is deployed at the far right. That’s ingesting the data from the cameras (and/or sensors) as well as an AI/ML streaming framework that in runtime is deployed to process the model at the near edge. Also, applications do the retraining up in the cloud. So, yes absolutely many of the outcomes actually are distributed applications that are deployed simultaneously across the edge, the core, and the cloud.
Q. Can the cloud ever be considered to be at the edge?
A. [Dan] “The edge” has a different meaning to many people. A cloud is an edge. One of the trends that we’re seeing is what’s called the “functional edge” or some people like to refer to that as the “IoT edge,” and then you have the “far edge” and then something we defined as a “mirror edge,” (a coordinated center in a cloud), but that cloud is an edge. Now, what we’re seeing is that the onset of private wireless or 5G for example has the ability to be able to connect that far edge or that IoT edge directly to a cloud or directly to a data center, but each of those eliminates the edge computing locations in between. You can think of it like an always-on world with the speed and the bandwidth that you would get with 5G, as it becomes ubiquitous and allows you to connect processing in a cloud or core data center directly to the IoT edge which is pretty compelling, but yes, cloud is an edge.
Q. If I’m using a tablet inside a connected car where is the edge? Is it the tablet, the car or a nearby cellular base station?
A. [John] All of those could be different parts of the edge. The tablet could be the functional edge, and the car may also be the functional edge because they both have a sort of constricted or restricted domain in terms of the hardware which is fairly fixed. You can update the software and use it as your fairly dedicated functional processes, which don’t really have any other applications going on. And then the cellular base station could probably be considered the far edge and you could also have a telco office acting as a traffic processing center or small data center that could be the near-edge. Or this information from the functional and far edge could then flow into the cloud and the cloud could be that near-edge data center before some of the data ultimately goes back to a core data center that’s run by the enterprise or hosted in a colocation facility.
[Dan] There is no one edge. What we’re really talking about is edge compute or where does processing of data take place and it takes place everywhere and the applications are the outcomes that you need for that processing. If you’re in manufacturing, you need overall equipment effectiveness. If you’re in retail you need fraud detection. Where to produce that outcome, you’re going to need to be able to operate in multiple locations and so I really would define edge as the point where data is being acted on by edge compute.
Q. How do edge storage needs differ from cloud or data center storage needs?
A. [Dan] We touched a little bit on this. I was trying to convey that depending on the location, much of the data that’s generated from the IoT edge is mostly streaming data and they have to get it to a point where they can process the data. The streaming architectures are prevalent because as you’re transporting data between locations, processed data could have an intermittent connectivity or a low bandwidth situation so typically you see either a streaming data architecture or some sort of architecture that can handle that disconnect and then re-establish and continue. Most of the data that’s coming in at the far edge is usually because of the streaming. It’s a store forward and most of it is unstructured. When you start to get into upstream, where you need to do more with the data like that near edge, I had explained that you have kind of a mix of both these edge workloads which is working on this unstructured data, as well as a mix of structured data for business applications. So, you have a mix of edge and IT workloads. All of those are in that data processing pipeline that is needed to provide a near real-time response. It’s not really out of band data, it’s in-band data and then usually with a core data center or cloud is where you’re usually going to be operating on data that is out of band that requires deep storage. So, the data center will continue to have deep storage as well as the cloud where you need to expand and you need longer term storage for in-home or for training models and things like that. Typically, the further you go up the edge, locations all the way up to the core, the more storage that you’re going to need.
[David] One needs to be concerned from an application standpoint on a balancing of compute and storage resources or memory resources as well. So, latency inherent delays especially as performance requirements increase with applications require localized compute and storage resources at the point of creation of the data. The heavy lift of processing will still be done at the cloud where you can balance out racks and racks of compute resources along with arrays and racks of solid-state drives. However, as we expand out to the edge having point compute and storage resources that are tailored for specific applications perhaps we should call them edge aggregation points, basically collecting all that IoT data and you want to provide some level of real-time processing and analytics and decision-making out in the field, having those aggregation points allows you to do so. The other part of this, is how to optimize the performance against that data through caching, through providing persistent memory layer or higher performance storage flash memory. That’s actually worth another discussion at a separate time. But you can start getting deep on how to optimize compute and storage resources and memory resources at the edge.
[Dan] One thing that is not obvious to folks trying to build out edge solutions is that customers care very much about cost. If you take a look at the amount of compute with the growth of compute at the edge, there’s in aggregate more compute at the edge than there is in a core data center or cloud and if you want to reduce transport fees or if you want to reduce infrastructure sprawl you really want to be able to act on that data near the point of generation. John talked about how you would distribute the edge processing for AI and ML out to the edge. Doing so helps you filter the data so you don’t have to move all the data to a cloud or core data center, therefore reducing your infrastructure sprawl and also reducing your cost for transport. You get the added benefit of more of a real-time response. This is why you see new technologies that are coming out that promote acting on data near the point of generation. If you take a look at AWS lambda functions, for example, or even some of the data management applications that are basically tagging the data as it comes in at the edge and they’re caching it at the edge. Then they’re using a centralized repository or distributed ledger where somebody can query that ledger and then distribute that query out to where the data exists.
Q. With that topic in mind does AI training ever happen at any of the edge points or is this only reserved for cloud?
A. [John] The traditional answer would be no. AI training only happens either in the core data center or in a cloud data center and at the edge you only have inference. However, I don’t think that’s completely true now. That’s generally still true, because the training requires the most compute power, the most CPUs or GPUs, the most storage, and networking bandwidth, but I think increasingly there is more and more AI training that can happen or is happening out there beyond the core data centers and clouds, especially at near edge or the far edge.
If you’re training your phone to recognize your fingerprint or your face, I’m not sure if all of that has to go back to the data center in order for the training models to be done. Everyone knows that the actual inference for face or fingerprint recognition once you’ve trained it, is done completely on the phone—it doesn’t need to contact the cloud or core data centers to do that. Many of these phones have small GPUs built into their chipset and they can recognize your fingerprint interface even if they don’t have any connectivity going at all – no network connectivity. But I would say traditionally, most or all the training was in the data center. But some of it has moved out to the near edge or the far edge.
[Dan] The only thing I would add on top of that is some customers are disconnected from public clouds. Especially some government and even some retailers. They have their own local data centers and they do a form of federated learning in their edges.
Q. I believe there are other applications but we’re just talking about AI here. What are some of these other applications besides artificial intelligence that are used in this environment?
A. [Dan] The big one would be streaming analytics that’s non-video to preserve safety or to preserve production or to take action. You want to take action where the data is being generated. In manufacturing we see this quite a bit where a manufacturing system is taking telemetry in from a set of sensors running analytics on it and then modifying their action based on those analytics. With respect to storage that’s very small unstructured amounts of data that’s highly compressible and doesn’t require a lot of storage processing. But yes, real-time analytics.
[David] AI can encompass video stream and camera captures for security purposes and that’s an additional application for smart cities and any deployment or mass deployment of video for security. Another one that is interesting is genomic sequencing where instead of sending the batch data up to the cloud for processing it can actually be done localized to the doctor’s office. That’s an emerging trend, where we’re seeing a tremendous amount of interest especially for analyzing in near real time patient data and keeping that data private and localized where decision making can be made to benefit that patient versus sending that data across the network.
[John] I would say there are a lot of apps. The most famous apps that we’ve talked about are AI or use AI, but there are a lot of other apps that run at the edge. A lot of them are for accessing information, sharing information, or otherwise utilizing information. For people it could be as simple as navigation which may include some AI elements, but navigation by itself does not necessarily use AI for the basic navigation, nor do the shopping apps for entering information ordering food. These are all things that can run partly at the edge. Other examples would include doing diagnostics, such as automotive diagnostics, factory diagnostics, or robot diagnostics, or social media apps. Those are things with edge apps that may or may not use AI at the edge, though they often still use AI at the core or in the cloud.
Q. We’re seeing a lot of movement towards repatriation of apps from cloud to edge data centers or even far edge compute locations. Do you think part of that repatriation is being driven by not getting the storage locations right and are we perhaps aiming at the wrong problem by repatriating the app to an edge?
A. [Dan] The fundamental reason people are repatriating their edge applications is to get that processing near the point of data generation to improve overall response. Now the second half of that, is the way that applications were developed 10 years ago is completely different than applications developed today. You have a cloud native microservices container-based application environment with rich APIs and some sort of DevOps pipeline for delivery and continuous integration. The rise of containers and virtualization in cloud native application architectures are making it easier to repatriate as well, right? So, I think it’s both those things.
Q. Do you ever see CapEx versus OpEx as part of that decision making as well?
A. [Dan] Absolutely 110% and that’s the point that I was trying to make earlier. There are two things I kept saying throughout this webcast. The first is don’t overlook generating real-time responses. Work for your outcomes is obviously important, but customers are running businesses. The second is they’re looking for ways to reduce costs. So, if you’re pushing your processing out to your edge you’re doing a number of things. You’re actually reducing CapEx because you’re now filtering the data or doing all the local processing and making decisions locally and taking actions locally and reducing the infrastructure for shipping all that data up to some centralized location. That reduces your capital expenditures because it’s reducing your infrastructure sprawl.
In terms of OpEx and making it simple, that’s a more complicated response. But reducing operational expenses comes in many forms. At the edge is one of them. It could be through how do you manage your applications through the edge. Leverage of containers and Kubernetes for example, we see the hyperscalers like Amazon, Azure and Google, AKS EKS and GKE with these managed Kubernetes services in multi-cluster management they can repatriate those container applications and move them using the same common tool sets to the edge. So, things like that help reduce OpEx.
[John] I think a lot of the movement to the cloud and then the subsequent repatriation has to do with balancing of OpEx and CapEx. A lot of people started out looking at CapEx, and move to the cloud where there’s no CapEx, it’s all OpEx, so initially it is much cheaper for your first year or two to move things to the cloud because you’re saying “I don’t have to buy these servers, I just pay monthly for what I’m using.” But then after you either build up enough data or you have enough traffic or enough compute power or enough data ingress and egress, then the monthly charges add up and grow as you grow the amount of data or compute. And then suddenly people look and say “wow, if I could repatriate this data and build a relatively efficient infrastructure on premises, the total cost of CapEx and OpEx combined would be lower than keeping it in the public cloud.” There are also some security or privacy concerns because you can say well in the cloud perhaps, I can’t control which country it’s in or which data center. I can’t prove who has access or who doesn’t have access so that’s another reason to possibly repatriate.
[Dan] This is something that’s often overlooked. I told the story earlier where we were talking to a customer who has self-driving vehicles. They took a cloud-first strategy where they were doing all of their inferencing. So, they’re collecting all the sensor information from the vehicle, including the video and shipping that to the cloud doing the infrastructure in the cloud, then they’re processing and inferencing from the cloud back down to the edge. We said to them “that’s going to cost you a lot of money” and they said “yes and it costs a lot of bandwidth for us to send all this video upstream too.” So, what they are having to do is transcode their video to a lower frame rate so that they could save on transport fees. We convinced them they should probably take a hybrid approach. You still do training and everything up in the cloud, but move your inferencing on board with something you can use to filter the data. That way you don’t have to re-transcode the data so that you can maintain the fidelity of your video images because you’re only going to be sending about 30 percent of that upstream and if you’re going to send it upstream since you’re over a cellular network why don’t you send that to some co-location provider or local data centers where you have more points of presence. We want to look at helping design solutions for their age, but really take a look at how they improve their business outcomes, while also reducing CapEx and OpEx.
Q. Is the edge inherently less secure than the data center?
A. [Dan] If you take a look at the compute outside the data that are not protected by the whole data center then inherently it is less secure. I read an article where it said 30% of all security breaches are at the far edge or through some external port. So, this is infrastructure that can be moved to a room, it could be spoofed, it could be tampered with. When you’re deploying edge compute outside the walls of an IT data center, out there in the wild, you really need to pay attention to zero trust architectures. Around how do I provide my hardware attestation as well as software attestation. Can I provide cryptographic network isolation, protected transports and everything else? It is definitely a top concern.
Remember I said this was a series? I encourage you to register for next two live Storage Life of the Edge webcasts:
- March 22, 2022: Storage Life on the Edge: Use Cases
- April 27, 2022: Storage Life on the Edge: Security Challenges
We hope to see you there!