Engineer’s Guide to Cloud Cost Optimization: Engineering Resources in the Cloud
As companies search for ways to optimize costs, they want — and need — data about many different elements, including the amount of money they are spending in the cloud, the savings potential and resources they can eliminate.
It can get complicated fast. To help teams prioritize, it’s a good idea to break down cloud cost optimization into two different universes: engineering and finance.
Resource Optimization vs. Rate Optimization
Engineering teams are actively building and maintaining cloud infrastructure. For this reason, they tend to own resource optimizations.
Rate optimization is about pricing and payment for cloud services. Depending on the organization’s structure and FinOps maturity, a person in IT, finance or FinOps may manage the cloud budget and levers used to achieve cloud savings.
Resource Optimization Happens in the Engineering Universe
Any company can get caught in the vicious cycle of cloud costs. It’s important to understand the implications of resource decisions (made by engineering teams), whether companies are new to the cloud or born in the cloud. Otherwise, they scramble to adjust and react to costs with potentially low visibility and governance.
Companies taking advantage of AWS Migration Acceleration Program (MAP) credits can fast-track cloud migrations with engineering guidance and cost-cutting service credits. Without forethought, resource and cost strategy, and visibility and accountability, there can be sticker shock once the assistance runs out.
Cloud-native companies may understand the cloud and even have costs fairly contained. But sudden, rapid growth or a significant business change means that they have to adjust to unknowns. Costs can rise even with optimization mechanisms in place.
Both situations involve an issue around scale. It takes time for companies to monitor, report and determine how to best optimize cost and resources. As they figure this out, they may be paying higher on-demand rates, which is not cost-effective.
With regards to cloud compute, one area that engineers may feel in command of because of the nature of their work is resource waste. They may use resource deployment and monitoring techniques like the following to use resources more efficiently and manage waste:
- Right-sizing: Matching instance types and sizes to workload performance and capacity requirements. This is challenging to implement in organizations that have rapidly changing or scaling environments because it requires constant analysis and manual infrastructure changes to achieve results.
- Auto-scaling: Setting up application scaling plans that efficiently utilize resources. This optimization technique might be effective for highly dynamic workloads, but it doesn’t offer as much cost savings for organizations with static usage.
- Scheduling: Configuring instances to automatically stop and start based on a certain schedule. This might be practical for organizations with strict oversight and monitoring capabilities, but most basic usage and cost analysis tools make regular predictions challenging.
- Spot Instances: Leveraging EC2 Spot Instances to take advantage of unused capacity at a deep discount. These spot instances might only be useful for background tasks or other fault-tolerant workloads because there’s a chance they can be reclaimed by AWS with just a two-minute warning.
- Cloud-native design: Building more cost-efficient applications that fully utilize the unique capabilities of the cloud. This can require a significant overhaul if an application is migrated from on-premises infrastructure without any redesign.
It takes time and effort to define, locate and actively manage waste. But the surprising thing is that reducing waste doesn’t affect cloud costs as much as one might expect. There is another lever that produces more immediate, expansive cloud savings: rate optimization.
Cloud Rate Optimization: the Financial Side of Cloud Computing
Cloud providers offer various discount instruments to incentivize the use of cloud services. AWS, for example, offers:
- Spot Instance discounts
- Savings Plans
- EC2 Instance
- Compute
- Reserved Instances
- Standard Reserved Instances
- Convertible Reserved Instances
- Enterprise Discount Programs
- Promotional Credits
Spot Instances
Spot Instances provide a deep discount for unused EC2 instances. Spot is a good option for applications that can run anytime and be interrupted. When appropriate, consider Spot Instances for operational tasks like data analysis, batch jobs and background processing.
Savings Plans
Compute Savings Plans offer attractive discounts that span EC2, Lambda and Fargate. They require an hourly commitment (in unit dollars) for a one-year or three-year term and can be applied to any instance type, size, tenancy, region and operating system.
It’s possible to save more than 60% using Compute Savings Plans, compared to on-demand pricing.
While the regional coverage is attractive, commitment terms are fixed with Savings Plans; they can’t be adjusted or decreased once made. This rigidity often leads to under- and over-commitment (and suboptimal savings).
EC2 Savings Plans only offer savings for EC2 instances. They require a one-year or three-year commitment with the option to pay all, partial or no upfront. They are flexible enough to cover instance family changes, including alterations in an operating system, tenancy and instance size. But these kinds of changes aren’t very common, and they can negatively affect the discount rate.
Overcommitment is common with EC2 Savings Plans. Standard RIs are viewed as a better option since they produce similar savings and are easier to acquire and offload on the RI marketplace.
Reserved Instances
Reserved Instances (RI) provide a discounted hourly rate for an EC2 capacity reservation or an RI that is scoped to a region. In either scenario, discounts are applied automatically for a quantity of instances:
- When EC2 instances match active RI attributes
- For AWS Availability Zones and instance sizes specific to certain regions
It’s possible to save more than 70% off of on-demand rates with Reserved Instances. They offer flexibility: instance families, operating system types and tenancies can be changed to help companies capitalize on discount opportunities. RIs don’t alter the infrastructure itself, so they are not a resource risk.
There are two different types of Reserved Instances: Standard RIs and Convertible RIs:
Standard RIs can provide more than 70% off of on-demand rates, but flexibility is restricted to liquidity in the RI marketplace and there are restrictions regarding when and how often they can be sold. They are best used for common instance types, in common regions, on common operating systems.
Convertible RIs can offer up to 66% off of on-demand rates, but their flexibility can actually generate greater savings than other discount instruments due to higher coverage potential. Convertible RIs are:
- Flexible. It’s possible to apply discounts to EC2 instances before rightsizing or executing another resource optimization.
- Offer Greater Amounts of Savings. Even with three-year commitment terms, CRIs can be exchanged. So if EC2 usage changes, it is still possible to maintain EC2 coverage while maximizing an Effective Savings Rate.
- Produce Faster Savings. Commitments can be made in aggregate for expected EC2 usage, instead of by specific instances that may have been forecasted. CRIs enable a more efficient “top-down” approach to capacity planning so that companies can realize greater savings faster without active RI management.
Enterprise Discount Program/Private Pricing Agreements/Promotional Credits
AWS Enterprise Discount Program (EDP) offers high discounts (starting at $500,000 per year)
for commitments to spend a certain amount or use a certain amount of resources over a period of time. Generally, these agreements require a commitment term for higher discounts, typically ranging from two to seven years. They are best for organizations that currently have, or plan to have, a large footprint in the cloud.
It’s common for companies to double down and commit fairly aggressively for discounts using an EDP, but they may not realize how the agreement may limit their ability to make additional optimizations (using additional discount instruments, reducing waste or changing resources).
Overcommitting to high spend or usage before making optimizations is common, so it’s always recommended to negotiate EDPs based on a post-optimization environment. Discount tiers may be fixed, but support type, growth targets and commitment spanning years (instead of a blanket commitment) can — and should — be negotiated.
