Learn some simple changes you can make to better monitor, manage, and maintain more efficient costs with AWS cloud services.
Two technical edge cases when migrating AWS Lambda to ARM/Graviton2: floating-point comparison precision and undefined C behavior with negative float casting. Learn migration considerations.
A fresh perspective on the capabilities of serverless technologies and what that means in terms of cost, services, and required knowledge for your business.
Reduce AWS VPC costs with these tips on data transfer pricing, NAT Gateway optimization, and using VPC Gateway Endpoints for S3 and DynamoDB.
We’re often amazed by how “The Cloud” scales efficiently to keep up with our demands and we try to tell ourselves that we only pay for what we use, which is partially true, but we’ve all seen the occasional rogue spend slip through the cracks only to be noticed when the bill arrives. Small companies have run into big problems by not understanding and estimating the costs of the solutions they build. On the other hand, large enterprises can find themselves paying thousands of dollars per month for resources they don’t use and don’t need. Neither of the two types of companies mentioned above had taken into consideration that having a healthy cost structure could make their business not only more profitable but also easier to manage and plan for future growth.
This article will provide the reader with a basic understanding of their options for becoming a healthier organization with respect to their Cloud Cost Management and which strategies are the ones to start implementing towards this path. Establishing a FinOps practice in any company is a long process that will take time and training. Do not attempt to fly before learning to walk or crawl!
The first step towards a healthy cloud cost structure is to create an AWS Organization that contains all the accounts under a Master Payer Account that has no resources in it. This will help us have a better understanding and segregation of costs. Beyond that, it will help us manage reservations and savings plans in an even and organized way.
Don’t be afraid to create as many accounts as needed in order to segregate our resources and manage permissions efficiently. There’s little to no incremental cost for AWS accounts.
Very often we tend to use only one or two accounts, like Production and the rest of the environments in the other one, which leads to very common mistakes that can impact other tasks like Business Support, QA, and development.
Even having a playground account for holding our company’s hackathons and non-business activities is a great idea, as it leads to healthier resource management. Here are some best practices for designing a healthy account management plan.
An effective tagging policy will provide our organization with increased granularity visibility and management over our allocated resources but it will require standardization and grouping into categories.
The following tag categories can help us achieve these goals:
Enforcement of tagging during resource creation, will help our organization maintain a solid cost profile. There are 2 types of tagging policies:
Another recommended strategy regarding tagging is to have a list of fixed values each tag can take in order to avoid typos or multiple values for the same type of resource (for example: DataLake, data-lake, dataLake).
Once the account setup is done, from the Master Payer Account, we must enable the Cost Allocation Tags. This will help us identify and group all the resources created in the accounts linked to the one mentioned before. This is done under the Billing Dashboard option, and each tag needs to be selected individually. If more tags are enabled at a later stage, this will only be reflected in the costs from the moment they were enabled, not prior to the time they were enabled.
Once Cost Allocation Tags is enabled, the first step in this journey is to enable the Cost and Usage Report in AWS.
CUR can be enabled by any administrator, it will output parquet files with our resources usage in the destination folder specified. Our suggestion is to use the hourly granularity to be able to make a deeper analysis.
Amazon Athena will provide us with a tool to perform ad hoc queries over our cost structure.
At this point we are probably wondering why using CUR if we have AWS Cost Explorer, right? Well, that’s a great question!
Let’s understand the benefits of both as they complement each other:
AWS provides us with tools for automating the monitoring of our costs and usage for our resources. Organizations can easily set up daily, weekly and monthly reports that are automatically delivered to the corresponding users, besides being able to set thresholds for sending alerts.
AWS provides us with an outstanding tool out of the box that can be easily configured with an AWS CloudFormation template that will create several dashboards that can be used for cost monitoring and forecasting.
These Amazon QuickSight dashboards are known as CUDOS mostly. follow the link below for setup and usage tips – Cloud Intelligence Dashboard with Amazon QuickSight.
It is never too late nor too early to start with our cloud cost management strategies, whether we are a small business just trying to get started or a big company that needs to have a healthier cost allocation strategy for cloud resources. The decision of not following these strategies can greatly affect our monthly bill and the discovery of any issues related to it. This is not only a technical challenge that has to be implemented but also a cultural change where people training is critical for an effective strategy.
Yes, it will take time to have it done, but it is better to start as soon as possible!
If you’d like to leverage our expertise to accelerate your AWS initiatives, while ensuring that your environment is optimized for costs, scale and efficiency, get in touch with our team.
Jorge Goldman is an Engineering Manager with over 12 years of experience in diverse areas from SRE to Data Science. Jorge is passionate about Big Data problems in the real world. He graduated with a Bachelors degree in Software Engineering and a Masters degree in Petroleum Engineering and Data Science. He is always looking for opportunities to improve existing architectures with new technologies. His mission is to deliver sophisticated technical solutions without compromising quality nor security. He enjoys contributing to the community through open-source projects, articles, and lectures, and loves to guide Caylent's customers through challenging problems.
View Jorge's articles
Jorge Goldman
