• Stars
    star
    283
  • Rank 146,066 (Top 3 %)
  • Language
    Go
  • License
    MIT License
  • Created about 4 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Cloud cost estimation for Terraform in your CLI

TerraCost logo

TerraCost

PkgGoDev

Go library for estimating Terraform costs using ingested cloud vendor prices. It is meant to be imported and used by programs (API's or standalone) with access to a MySQL-compatible database and the Internet.

Installation

go mod edit -replace github.com/hashicorp/terraform=github.com/cycloidio/[email protected] && go get github.com/cycloidio/terracost

We need to do a -replace because of golang/go#30354 (comment). We have a custom fork of Terraform and in order to use TerraCost it needs to be replaced when importing also.

Requirements

  • Go 1.16 or newer
  • MySQL database

Provider support

Currently Terracost supports the following providers and with an specific subset of resources from them:

Google Credentials

To be able to ingest the pricing data from Google the credentials needed have to have access to Compute Engine API and have also billing enabled. This is needed to be able to fetch the Machine Types, for the SKUs we would only need an normal set of credentials (or even API Key) but as we need both we have to use the more complex one. If you do not know how to activate those permissions, just use the credentials and the import will fail and on the logs will inform of what is missing and how to activate it.

Usage

Migrating the database

db, err := sql.Open("mysql", "root:password@tcp(IP:3306)/databasename?multiStatements=true")

// Can be called on every start of your program, it does nothing if the migrations
// have been executed already.
err = mysql.Migrate(context.Background(), db, "pricing_migrations")

Ingesting pricing data

db, err := sql.Open("mysql", "root:password@tcp(IP:3306)/databasename?multiStatements=true")
backend := mysql.NewBackend(db)

// service can be "AmazonEC2" or "AmazonRDS"
// region is any AWS region, e.g. "us-east-1" or "eu-west-3"
ingester, err := aws.NewIngester(service, region)
err = terracost.IngestPricing(context.Background(), backend, ingester)

Tracking ingestion progress

We're using the github.com/machinebox/progress library for tracking ingestion progress.

  1. Create a channel that will receive progress updates and set up a goroutine (it will print the bytes ingested out of bytes total and remaining time each time progress update is sent on the channel):
progressCh := make(chan progress.Progress, 0)

go func() {
	for p := range progressCh {
		// Check the docs for all available methods: https://pkg.go.dev/github.com/machinebox/progress#Progress
		fmt.Printf("%d / %d (%s remaining)\n", p.N(), p.Size(), p.Remaining().String())
	}
}()
  1. Initialize an ingester capable of tracking progress (in this example the channel will receive an update every 5 seconds):
ingester, err := aws.NewIngester(service, region, aws.WithProgress(progressCh, 5*time.Second))
  1. Use the ingester as in the previous section.

Estimating a Terraform plan

Plan estimation is possible after all the relevant pricing data have been ingested and stored in the database.

  1. Generate a plan using terraform and convert it to JSON:
terraform plan -out update.tfplan
terraform show -json update.tfplan > tfplan.json
  1. Read the plan file, estimate it and show the resource differences:
db, err := db.Open("mysql", "...")
backend := mysql.NewBackend(db)

file, err := os.Open("path/to/tfplan.json")
plan, err := terracost.EstimateTerraformPlan(context.Background(), backend, file)

for _, res := range plan.ResourceDifferences() {
  priorCost, err := res.PriorCost()
  plannedCost, err := res.PlannedCost()

  fmt.Printf("%s: %s -> %s\n", res.Address, priorCost, plannedCost)
}

Check the documentation for all available fields.

Usage estimation

Some resources do cannot be estimated just by the configuration and need some extra usage information, for that we have some default on usage/usage.go which are also all the resources and options we support currently and can be overwritten when estimating if passing a custom one instead of the custom Default one.

Examples

For more examples, please check examples.

Contributing

For Contributing Guide, please read CONTIBUTING.md.

License

This project is licensed under the MIT license. Please see LICENSE for more details.

Meet Cycloid

Cycloid is a hybrid cloud DevOps collaboration platform providing end-to-end frameworks to accelerate and industrialize software delivery.

As of now, we have three open-source tools:

  • TerraCognita: Read from your existing cloud providers and generate IaC in Terraform
  • InfraMap: Reads .tfstate or HCL to generate a graph specific for each provider
  • TerraCost: Cloud cost estimation for Terraform in the CLI

...and the functionality of each is also embedded in our DevOps solution, which you can find out more about here.

More Repositories

1

terracognita

Reads from existing public and private cloud providers (reverse Terraform) and generates your infrastructure as code on Terraform configuration
Go
2,163
star
2

inframap

Read your tfstate or HCL to generate a graph specific for each provider, showing only the resources that are most important/relevant.
Go
1,697
star
3

cycloid-cli

Source code of Cycloid Command Line to interface Cycloid APIs
Go
17
star
4

raws

[UNMAINTAINED] AWS Reader
Go
15
star
5

cyclosible

Python
8
star
6

vuex-type-const-generator

Generate a file with constants for getters, actions and mutations from a Vuex store definition
JavaScript
6
star
7

ansible-backup

Shell
5
star
8

tfdocs

Generated structured documentation for Terraform Providers
Go
5
star
9

auth0-rules-runtime

Wrapper of webtask runtime for helping to write tests for Auth0 rules
JavaScript
3
star
10

import-index-generator

Autogenerate a JS modules which exports all the modules present on a directory
JavaScript
3
star
11

aws-billing

Import detailed billing from AWS
Go
3
star
12

goat

Go SDK for auth0 API
Go
2
star
13

mxwriter

MXWriter is a small library that emulates a Multiplexer and Demultiplexer for io.Write.
Go
2
star
14

cycloid-hooks

Gathering of hooks used at cycloid
Shell
2
star
15

gopa

GO Client for OPA
Go
2
star
16

flatmap

This lib is a "fork" from of "terraform/flatmap" v0.14.10
Go
2
star
17

ansible-ecs

Install ECS-agent on servers
1
star
18

docker-cycloid-toolkit

Docker image which contain tools and a scripts for cycloid.io
Python
1
star
19

ansible-php

Jinja
1
star
20

docker-image-mailslurper

Dockerized MailsSlurper
Dockerfile
1
star
21

cycloid-resource

Concourse resource to use Cycloid features inside pipelines
Go
1
star
22

ansible-telegraf

Cycloid wrapper who call dj-wasabi/ansible-telegraf role and add some additionnal plugins
Python
1
star
23

ansible-systemd

Complement of https://github.com/tumf/ansible-role-systemd-service.git to provide features like mount and timer
1
star
24

ansible-backup-snapshot

Jinja
1
star
25

sqlr

Set of useful interfaces to work with the GO 'database/sql'
Go
1
star
26

approval-resource

Python
1
star
27

ansible-cyclosible

1
star
28

bootstrap-stacks

Collection of stack templates to be used by customer to create simple functioning stack
HCL
1
star