Remote Rendering for Real-time AR Applications at AWS Edge

Augmented Reality (AR) applications, spanning fields such as 3D machine design collaboration, equipment repair with digital twin models, and medical surgery training using 3D body images, necessitate real-time 3D object rendering. Remote rendering is a key technology that addresses this need by processing 3D content on a server in response to AR headset movements and streaming the rendered output back to the headset in real time. This approach offers several advantages over on-device rendering, including enhanced performance, superior AR experience quality, and reduced costs, as the server can leverage more powerful graphics processing resources and support higher-resolution streaming platforms compared to the limited capabilities of individual headsets. Furthermore, remote rendering improves data protection by centralizing data on the server, mitigating risks associated with storing sensitive information directly on individual AR devices. However, remote rendering for real-time AR applications presents specific technical challenges. To deliver an interactive and immersive user experience, it must meet stringent requirements for low-latency response times, minimal jitter, and high throughput. Typically, real-time AR remote rendering demands less than 20 milliseconds of XR device round-trip latency, often referred to as “motion-to-photon latency.” Network bandwidth requirements involve approximately 10 Kbps uplink throughput for transmitting motion sensor data from the XR device to the server and 20 Mbps downlink throughput for streaming rendered images back to the device. These throughput demands scale upward with an increasing number of concurrent XR devices, such as in multi-user 3D design collaboration scenarios. This article explores how AWS edge computing services, specifically AWS Wavelength and AWS Snowball Edge, can effectively support remote rendering in both public and private 5G environments. It uses Holo-Light's XR engineering application, AR 3S, as a practical example to illustrate the remote rendering requirements and workflows within these diverse edge computing settings. Holo-Light's AR 3S, an Augmented Reality Engineering Space, enables engineers to visualize and interact with complex, full-scale 3D CAD data in AR/VR, merging digital models with physical components. The application leverages proprietary streaming technology (ISAR SDK) to offload rendering from less powerful XR devices to high-performance servers, allowing for the processing of over 100 million polygons at 40-60 frames per second, a significant improvement over the sub-1 million polygon limit of self-sufficient AR data goggles. ISAR utilizes the WebRTC standard for XR-Streaming, often requiring a STUN (Session Traversal Utilities for NAT) or TURN server to establish direct peer-to-peer connections between the server and AR devices across network address translators and firewalls. For remote rendering in a public 5G environment, AWS Wavelength integrates AWS compute and storage services directly within Communication Service Providers' (CSPs) public 5G networks, providing ultra-low-latency mobile edge computing infrastructure. This setup is ideal for applications like gaming, engineering design review, and immersive training where XR users are geographically dispersed. The solution architecture involves extending an AWS Virtual Private Cloud (VPC) into an AWS Wavelength Zone within a CSP network. A virtualized remote rendering server, equipped with GPUs, can be deployed on AWS EC2 instances in this zone. Connectivity between the AWS Wavelength Zone and the CSP network is managed by a Carrier Gateway (CGW), which facilitates inbound and outbound traffic. A Carrier IP address is assigned to the EC2 instance, and the CGW performs network address translation to route rendering stream traffic. A STUN server is essential for WebRTC peer-to-peer connections between the XR streaming server and client devices. In contrast, for private 5G environments, AWS Snowball Edge devices offer a robust solution. These ruggedized devices feature integrated cloud compute and storage services, including Amazon EC2 with GPU support, and can operate autonomously without constant connection to an AWS Region. This makes them suitable for sensitive private environments such as factories, hospitals, or defense installations, where data must remain on-premises and network connectivity to the external world is restricted. For instance, in automotive manufacturing, AR can be used for 3D design review and collaboration, with all data and applications securely hosted on-site. The Snowball Edge device must be a “Compute Optimized with GPU” type, utilizing “sbe-g” EC2 instances. It supports separate management and data planes through different network interfaces (RJ45 for management, SFP+/SFP28 for application traffic), with a Direct Network Interface (DNI) facilitating Layer-2 connections between the EC2 instance and the private 5G network, eliminating the need for a STUN server in this path. Management of the Snowball Edge is performed via AWS OpsHub, and secure connections to an AWS Region for tasks like data uploads are established using AWS Site-to-Site VPN. The private 5G network on-premises typically includes a 5G Radio Access Network (RAN) and a 5G Core network. XR devices connect to this private network via 5G modems, tethered phones, or Wi-Fi hotspots, or even wireline connections to the Snowball Edge, accessing the remote rendering service via the EC2 DNI interface IP address. In conclusion, remote rendering significantly reduces the computational burden on XR devices, leading to cost and energy savings, while enhancing data security. AWS edge services, including AWS Wavelength and AWS Snowball Edge, effectively meet the low-latency and high-throughput demands of real-time AR applications. AWS Local Zones also provide an alternative edge solution for remote rendering applications, managed directly by AWS. Holo-Light's AR 3S exemplifies how these AWS solutions facilitate advanced XR experiences. #AugmentedReality #RemoteRendering #AWSEdge #AWSWavelength #AWSSnowballEdge #5GTechnology #EdgeComputing #XRApplications #LowLatency #AugmentedReality #RemoteRendering #AWSEdge #AWSWavelength #AWSSnowballEdge #5GTechnology #EdgeComputing #XRApplications #LowLatency