The frugal HPC architect – ensuring effective FinOps for HPC workloads at scale

The adoption of cloud computing for High Performance Computing (HPC) workloads offers significant benefits over traditional on-premises infrastructure, particularly in terms of elasticity and scalability. However, this flexibility also introduces cost variability, which can be challenging for organizations with strict budgets. This article explores best practices for managing costs in cloud-based HPC environments, drawing insights from 'The Frugal Architect' principles, which emphasize integrating cost as a non-functional requirement. The core idea is to find a balance between flexibility, efficiency, and effectiveness, especially for large-scale HPC operations. High Performance Computing, encompassing both tightly-coupled systems like computational fluid dynamics (CFD) and massive loosely-coupled systems for financial modeling, frequently generates substantial costs. Therefore, careful design, measurement, and observability of these systems are crucial. The article primarily focuses on opportunities for loosely-coupled workloads, though many principles apply broadly to all HPC tasks. The first law of 'The Frugal Architect' — "Make cost a non-functional requirement" — highlights the necessity of considering cost alongside other critical factors such as security, accessibility, compliance, and maintainability to ensure system success. In a pay-as-you-go cloud model, costs are determined by unit cost and consumption volume. Optimizations can be achieved by either reducing the unit cost, decreasing total units consumed, or both, using various AWS services and features. Opportunities to reduce unit costs on AWS involve leveraging services like Amazon EC2 Spot instances, which can offer up to 90% savings compared to on-demand provisioning. The trade-off is that these instances can be reclaimed by AWS with a two-minute warning, making them suitable for HPC workloads that can handle interruptions or checkpoint their progress. For consistent demand, Amazon EC2 Savings Plans provide up to 72% savings with a one or three-year commitment. Often, a combination of Spot instances, Savings Plans, and on-demand capacity is used for optimal cost management. Technical optimizations include provisioning precise instance types from over 750 available EC2 options, ensuring that only necessary CPU, GPU, memory, storage, and network performance are paid for, avoiding the costs of unused capabilities. For data storage, services like Amazon S3 with Mountpoint or S3 Intelligent-Tiering can significantly reduce unit costs, especially for infrequently accessed files. Reducing unit consumption is another key strategy. The units of consumption vary by AWS service, with some having multiple units for different utilization types. For Amazon EC2, reducing core-hour consumption involves increasing efficiency by ensuring cores are actively computing and not idle, and tracking effectiveness by ensuring the value of the work outweighs the cost. This aligns with the second law of 'The Frugal Architect' — "Systems that last align cost to the business." Unlike static on-premises infrastructure, every increment of cloud capacity incurs a cost, which should be justified by business value. Minimizing the billable lifecycle of instances by reducing overhead activities like OS boot time, binary installation, and idle time, can significantly improve efficiency. Techniques include optimizing boot processes, using long-running On-Demand Instances to reduce frequent reboots, or diversifying instance selection to mitigate Spot instance interruptions. Arm-based AWS Graviton instances offer up to 40% better price/performance and 60% less energy consumption than comparable x86 instances for HPC workloads. While porting applications to Arm64 may be required, tools like AWS Porting Advisor for Graviton can assist. Graviton adoption can lead to faster workload completion at the same cost or reduced costs for the same duration. Monitoring CPU utilization is crucial; high periods of under-utilization indicate potential for reducing instance count. For large datasets, using Amazon FSx for Lustre linked to Amazon S3 allows for high-performance access without storing the entire dataset in Lustre, thereby reducing filesystem size. Data compression and Amazon File Cache for hybrid environments can further decrease storage costs. Shutting down FSx for Lustre and Amazon File Cache when not in use and recreating them on demand also contributes to savings. Finally, observability is essential, aligning with the fourth law of 'The Frugal Architect' — "Unobserved Systems lead to unknown costs." Robust observability systems are necessary to inform HPC managers about overall efficiency (computation as a percentage of capacity) and effectiveness (business value as a percentage of compute cost). Timely and detailed metrics enable stakeholders to identify improvement opportunities, measure changes, and detect unexpected outcomes. AWS provides tools like Amazon CloudWatch and Amazon Managed Service for Prometheus for metrics, logs, and traces. For cost visibility, AWS Data Exports allows exporting cost data in the FinOps Open Cost and Usage Specification (FOCUS) schema, which can be analyzed with Amazon QuickSight or AWS Cost Explorer to identify trends and cost drivers. AWS Compute Optimizer further assists by recommending ways to optimize underutilized resources, such as adjusting memory provisioned, with projected savings. Integrating these FinOps practices ensures that HPC workloads in the cloud are not only powerful but also economically sound. #HPC #FinOps #CloudComputing #CostOptimization #AWS #EC2Spot #Graviton #Observability #Scalability #HPC #FinOps #CloudComputing #CostOptimization #AWS #EC2Spot #Graviton #Observability #Scalability