Multimodal Generative AI on Energy-Efficient Edge Processors
EdgeCortix Inc.
Bringing the next-generation artificial intelligence processor for edge devices - Edgecortix Inc. The Edge Intelligence Revolution
An intelligent edge brings computing and storage capacity closer to users so that future technologies can operate at an energy-efficient, ultra-low latency. As such, edge computing will drastically change how data is transmitted, processed, analyzed, and stored. Optimal performance, operating cost and reliability of applications and services are crucial to the next Industry 4.0, and that is exactly what edge intelligence or edge AI, brings to the table. Edge AI is expected to play a critical role in 5G, IoT, and the overall AI Revolution. There will presumably be a sharp rise in connected devices over the next 3–5 years according to the recent, Barclays “Edge Computing” research report (Long et.al., 2020). By 2023, it is estimated that there will be 30 billion connected devices globally among various industries, and the average number of connected devices per capita is expected to increase from 2.4 billion in 2018 to 3.6 billion in 2023. At present, 90% of the data generated by enterprises is processed in centralized data centers. The massive data generation from edge devices may result in bottlenecks in current cloud architectures. This is predominantly determined by the scale-up of data connection speeds, rapidly rising video growth, and future data-consuming applications. Consider a fully autonomous vehicle which generates around 5 TB of data traffic per day: a traditional cloud-only setup cannot supply the necessary bandwidth and processing power required for processing the large amounts of data produced daily, even by autonomous vehicles alone. In this regard, Edge AI mitigates bandwidth constraints, provides high power efficiency, guarantees data security, and offers the processing power and storage necessary (Long et.al., 2020). Overall, the spread of edge intelligence is anticipated to greatly benefit enterprises and consumers alike, and the edge market looks promising with various industry segments greatly impacted by it.Edge intelligence is paving the way for the next generation's lifestyle, as the perpetually narrowing gap between artificial intelligence and humankind renders a widening horizon of opportunity.
Benefiting Businesses
The majority of enterprise workloads are currently processed in three primary domains. The first is private data centers, the second is public cloud provider-hosted data centers, and the third is a hybrid cloud — a combination of both. The current architecture suffers from latency issues, security compliance challenges and in most cases, lacks scalability. Low latency and high throughput are key drivers for future applications. Currently, 75% of enterprises presume business opportunities are impeded by mistimed data response. For instance, half a second latency in searching drops 20% of traffic for Google. Amazon also analyzed that every additional 100 milliseconds of latency results in a 1% loss in sales. In contrast, edge computing can deliver the latency of less than 10 milliseconds in most cases. This ultra-low latency enables future-oriented businesses such as real-time analytics, augmented reality, industrial robotics, and autonomous driving, and further strengthens prospects of gaining new customers for these businesses (Long et.al., 2020). The burgeoning amounts of sensor-generated data at the edge also makes edge intelligence a critical component to most industries. An additional layer of data processing and analytics at the edge facilitates better privacy control and data security, which can be applied to addresses, faces, and other sensitive personal data. In addition, data stored at the edge can be isolated from the servers if the central cloud experiences a cyber-attack. Businesses could also benefit from local data processing since it efficiently reduces operational and data transmission expenditures. This is necessary as the existing cloud networks cannot cope with the required bandwidth due to the surge in unstructured data. According to Akamai, there is roughly 72 times more data traffic at the edge than the core network can process. This could potentially produce massive congestion at the cloud level. Edge intelligence combats this by analyzing a large degree of sensor data at the edge and selectively sending back only high-value data to the cloud for further processing or training. This system of offloading from the cloud to the edge is cost-saving in regard to bandwidth (Long et.al., 2020). There are also new revenue streams for service providers in content delivery and advertising, better monetization of edge data through personalized marketing and instant response in manufacturing, and business efficiency with real-time predictive maintenance and reduced asset downtime. Considering the aforementioned advantages, edge computing opens the way for new business opportunities, and more advanced operational efficiency via low latency data processing and analytics, while enabling novel revenue streams and monetization of data.Potential New Applications for End Users
End-Users can significantly benefit from edge intelligence as well. There will be new apps in diverse areas, from gaming to education. Autonomous driving and automated manufacturing will potentially increase free time. Real-time analytics and tracking will result in a better customer experience. Remote surgery and real-time health analytics can be actualized, further advancing healthcare. Safety can be improved, with surveillance processed locally. Moreover, it may lead to a greener environment, with sensor level monitoring and high energy efficiency of computation. For example, around 25% of a drone’s battery is consumed by the GPU (general processing unit) along with the added weight from the onboard chips. Making significantly more energy-efficient chips for onboard processing, or offloading the processing from drones to nearby edge servers (telco-edge) could vastly increase the battery life, without significantly sacrificing latency.
Prior to sending data to the cloud for deep learning and inference, low latency and local analytics are necessary for use cases such as intelligent video surveillance and smart traffic management. This will expedite the elimination of redundant, repetitive tasks as automation is further incorporated. Correspondingly, benefits such as higher efficiency, safer working environments, reduced traffic congestion, speedier package deliveries, cleaner air, and lower crime rates can be envisaged as well (Long et.al., 2020).
The low latency of data processing will improve user experience across a wide range of applications. Furthermore, edge intelligence is complementary to 5G, which is bound to trigger a surge of new applications in the next three years.
Tapping into the Edge Market
McKinsey's 2019 report, speculates that the joint market of edge and data centers combined is likely to expand from roughly US $6 billion in 2018 to over US $90 billion in 2025, with a 45% CAGR. This growing trend in artificial intelligence also presents semiconductor companies with the golden opportunity to capture 40–50% of the total value from the technology stack. McKinsey concludes that most value and development will be in memory, computing, networking, and storage. According to Cisco Cloud Index, there will be a large gap between the amount of data generated annually and data center-managed data traffic; this is a key driver for edge revenue opportunities across the spectrum (2020).
There is also potentially $20 billion US in opportunities spanning across services, hardware, and software that could be deployed at the edge by 2023. Edge computing will enable better user experience for both enterprises and consumers fundamentally, due to its energy-efficient, low-latency data processing (Long et.al., 2020).
The edge intelligence evolution presumably has a time span of 5–10 years, first emerging from populated metropolitan areas. Telecom companies and cloud providers started to take notable action in the edge computing market only in recent years. Most players in the ecosystem are likely to gain from this greenfield opportunity. Hardware, infrastructure, software, services, and applications are to name a few. Regarding hardware and infrastructure, the Total Addressable Market (TAM) for edge computing hardware could potentially expand to $5.2 billion US from now to 2023, representing a 22% CAGR (Long et.al., 2020).
Edge intelligence-specific software revenue could stem from data management and processing, analytics, security for edge devices, and a myriad of other applications. Furthermore, a TAM increase from an estimated $2.5 billion in 2018 to over $10 billion in 2023 is expected across mobile, cable, wireline, enterprise IoT, and public cloud (Long et.al., 2020).
The benefits of the edge AI market are perspicuous, but it is important to note that the innovation adoption curve for edge intelligence differs across industries. Some early-stage beneficiaries of the edge market include utilities, industrial, transportation, agriculture, and healthcare. These business niches share qualities that make edge computing an urgent and immediate need, as they are more latency dependent and have high precision measurement requirements.
In terms of businesses, AI/ML cost savings, and AI efficiency are key growth factors for edge computing. Machine learning, better automation, and analytics are used to enhance operational efficiency in an industrial IoT world. Companies can economize by reducing transmission bandwidth costs to send all the data back to central data centers.
Impacted End Markets
As applications increase to utilize the ultra-low latency, a sizable upside to the market can be forecasted over the longer term. This trend is likely to affect multifarious market ends including telecom services, security, IT hardware/communications equipment, communication infrastructure, and semiconductors.
Telecom companies understand the necessity for increased investment at the edge, with high demand for low-latency, optimized service. Telco’s also recognizes the profit to be captured in monetizing real estate for edge computing by converting central offices into data centers. In security, more edges create more network segments and possibly more firewall appliances. Moreover, endpoint security could become more critical as devices proliferate. The edge could facilitate more security-as-a-service adoption and competition.
On the flip side, the shift to the cloud has resulted in an economic downturn for many traditional IT Hardware/communications vendors. Nonetheless, the development of a smarter, more robust edge network could offset the capital loss to some extent. The tower, fiber vendors, and data centers are anticipated to be at the front line of the edge intelligence revolution, with tower companies and wireless service providers contributing the most.
COVID-19 Boosts Edge Demand
An increasing number of companies are integrating remote working guidelines. Working from home could lead to a 10–20% structural reduction in aggregate demand for office space. Twitter and Foursquare extending working from home (WFH) to “forever” are some of the multiple cases of companies enforcing WFH regulations during the pandemic. This adjustment has further inflated the need for edge computing. For instance, Cisco Webex hosted 20 billion meeting minutes in April according to the Data Economy’s live COVID-19 page. Edge data center deployments are what make this type of network traffic possible. Akamai, Cloudflare, and other edge computing companies have all recorded soaring data loads at their edge locations accordingly. Edge computing supplies the required infrastructure that reduces latency and alleviates bandwidth constraints.
Future with the Edge
The spread of edge intelligence is bound to drive significant changes for both businesses and consumers, while impacting several end markets. Edge computing can enable companies to level up their IoT applications. Smart technology powered by edge intelligence could also help expand existing markets with its ability to collect, interpret and immediately act on vast amounts of data critical to data-heavy applications. From cloud providers to end-users, device manufacturers to application developers, nearly everyone in the ecosystem will be impacted by this edge AI revolution. The future depends on what edge intelligence makes possible.
About the author:
Jasmine Woodruff is a Business Development Analyst at Edgecortix Inc. (www.edgecortix.com)
© Copyright 2020, EDGECORTIX
References:
Long et. al., 2020. Edge Computing. The Edge is Tomorrow. Barclays Capital Inc.
Batra et. al., 2018. Artificial-intelligence hardware: New opportunities for semiconductor companies. Available at: Batra, G., 2018. Artificial-Intelligence Hardware: New Opportunities for Semiconductor Companies. McKinsey report [online]
