• Stars
    star
    2,163
  • Rank 21,320 (Top 0.5 %)
  • Language
    Go
  • License
    MIT License
  • Created over 5 years ago
  • Updated 6 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Reads from existing public and private cloud providers (reverse Terraform) and generates your infrastructure as code on Terraform configuration

TerraCognita

GoDoc Gitter chat AUR package Homebrew

Imports your current Cloud infrastructure to an Infrastructure As Code Terraform configuration (HCL) or/and to a Terraform State.

At Cycloid, Infrastructure As Code is in the company DNA since the beginning. To help our new customers adopting this best practice, we decided to build Terracognita to convert an existing infrastructure on Cloud Infrastructure into Terraform code in an automated way, relying on Terraform providers built by the community. We focused on AWS, GCP, Azure and Vmware.

We decided to Open Source this tool as we believe that it will help people to adopt IaC in an easy way. Cycloid provides this tool to let people import their infrastructure into Cycloid's pipelines, allow them to generate infrastructure diagram and manage all infra/application life cycle from a single interface.

If you are interested in contributing to Terracognita or simply curious about what's next, take a look at the public roadmap. For a high level overview, check out the What is Terracognita? blogpost or watch this video.

Cloud providers

Terracognita currently imports AWS, GCP, AzureRM and VMware vSphere cloud providers as Terraform (v1.1.9) resource/state. Please see the following versions as follow:

Providers:

  • AWS: v4.9.0
  • AzureRM: v3.20.0
  • Google: v4.9.0
  • vSphere: v2.2.0

Installation

Binary

Visit the releases page to select your system, architecture and version you need. To pull the latest release:

curl -L https://github.com/cycloidio/terracognita/releases/latest/download/terracognita-linux-amd64.tar.gz -o terracognita-linux-amd64.tar.gz
tar -xf terracognita-linux-amd64.tar.gz
chmod u+x terracognita-linux-amd64
sudo mv terracognita-linux-amd64 /usr/local/bin/terracognita

Development

You can build and install with the latest sources, you will enjoy the new features and bug fixes. It uses Go Modules, so GO 1.17+ is required.

git clone https://github.com/cycloidio/terracognita
cd terracognita
make install

Arch Linux

There are two entries in the AUR: terracognita-git (targets the latest git commit) and terracognita (targets the latest stable release).

yay -Ss terracognita
  aur/terracognita 1:0.3.0-1 (+0 0.00%)
      Reads from existing Cloud Providers (reverse Terraform) and generates your infrastructure as code on Terraform configuration
  aur/terracognita-git 1:v0.3.0.r27.gdfc5a99-1 (+0 0.00%)
      Reads from existing Cloud Providers (reverse Terraform) and generates your infrastructure as code on Terraform configuration

Install via brew

If you're macOS user and using Homebrew, you can install via brew command:

brew install terracognita

Usage

The main usage of Terracognita is:

terracognita [TERRAFORM_PROVIDER] [--flags]

You replace the TERRAFORM_PROVIDER with the Provider you want to use (for example aws) and then add the other required flags. Each Provider has different flags and different required flags.

The more general ones are the --hcl or --module and --tfstate which indicates the output file for the HCL (or module) and the TFState that will be generated.

You can also --include or --exclude multiple resources by using the Terraform name it has like aws_instance.

For more options you can always use terracognita --help and terracognita [TERRAFORM_PROVIDER] --help for the specific documentation of the Provider.

We also have make help that provide some helpers on using the actual codebase of Terracognita

asciicast

Modules

Terracognita can generate Terraform Modules directly when importing. To enable this feature you'll need to use the --module {module/path/name} and then on that specific path is where the module will be generated. The path has to be directory or a none existent path (it'll be created), the content of the path will be deleted (after user confirmation) so we can have a clean import.

The output structure will look like (having --module test) this where each file aggregates the resources from the same "category":

test/
β”œβ”€β”€ module-test
β”‚   β”œβ”€β”€ autoscaling.tf
β”‚   β”œβ”€β”€ cloud_front.tf
β”‚   β”œβ”€β”€ cloud_watch.tf
β”‚   β”œβ”€β”€ ec2.tf
β”‚   β”œβ”€β”€ elastic_load_balancing_v2_alb_nlb.tf
β”‚   β”œβ”€β”€ iam.tf
β”‚   β”œβ”€β”€ rds.tf
β”‚   β”œβ”€β”€ route53_resolver.tf
β”‚   β”œβ”€β”€ route53.tf
β”‚   β”œβ”€β”€ s3.tf
β”‚   β”œβ”€β”€ ses.tf
β”‚   └── variables.tf
└── module.tf

By default all the attributes will be changed for variables, those variables will then be on the module-{name}/variables.tf and exposed on the module.tf like so:

module "test" {
  # aws_instance_front_instance_type = "t2.small"
  [...]
  source = "module-test"
}

If you want to change this behavior, as for big infrastructures this will create a lot of variables, you can use the --module-variables path/to/file and the file will have the list of attributes that you want to actually be used as variables, it can be in JSON or YAML:

{
  "aws_instance": [
    "instance_type",
    "cpu_threads_per_core",
    "cpu_core_count"
  ]
}
aws_instance:
  - instance_type
  - cpu_threads_per_core
  - cpu_core_count

Docker

You can use directly the image built, or you can build your own. To build your Docker image just run:

make dbuild

And then depending on the image you want to use (cycloid/terracognita or your local build terracognita):

docker run cycloid/terracognita -h

Example:

export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
export AWS_DEFAULT_REGION=xx-yyyy-0
docker run \
		-v "${PWD}"/outputs:/app/outputs \
		cycloid/terracognita aws \
		--hcl app/outputs/resources.tf

Local

The local version can be used the same way as docker. You simply need to be build it locally.

To test

On the same folder you imported you can run the terraform init & plan commands:

terraform init
terraform plan -var access_key=$AWS_ACCESS_KEY_ID -var secret_key=$AWS_SECRET_ACCESS_KEY -var region=$AWS_DEFAULT_REGION

License

Please see the MIT LICENSE file.

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Meet Cycloid

Cycloid is a sustainable platform engineering to allow anyone to interact with tools, automation and cloud without having to become an expert and in respect of the best practices. Inside the self-service portal, you find some modules around Governance, deployment, operations, Finops and GreenOps. InfraImport inside Cycloid is the visual representation of Terracognita.

As of now, we have three open-source tools:

  • TerraCognita: Read from your existing cloud providers and generate IaC in Terraform
  • InfraMap: Reads .tfstate or HCL to generate a graph specific for each provider
  • TerraCost: Cloud cost estimation for Terraform in the CLI

...and the functionality of each is also embedded in our platform, which you can find out more about here.

More Repositories

1

inframap

Read your tfstate or HCL to generate a graph specific for each provider, showing only the resources that are most important/relevant.
Go
1,697
star
2

terracost

Cloud cost estimation for Terraform in your CLI
Go
283
star
3

cycloid-cli

Source code of Cycloid Command Line to interface Cycloid APIs
Go
17
star
4

raws

[UNMAINTAINED] AWS Reader
Go
15
star
5

cyclosible

Python
8
star
6

vuex-type-const-generator

Generate a file with constants for getters, actions and mutations from a Vuex store definition
JavaScript
6
star
7

ansible-backup

Shell
5
star
8

tfdocs

Generated structured documentation for Terraform Providers
Go
5
star
9

auth0-rules-runtime

Wrapper of webtask runtime for helping to write tests for Auth0 rules
JavaScript
3
star
10

import-index-generator

Autogenerate a JS modules which exports all the modules present on a directory
JavaScript
3
star
11

aws-billing

Import detailed billing from AWS
Go
3
star
12

goat

Go SDK for auth0 API
Go
2
star
13

mxwriter

MXWriter is a small library that emulates a Multiplexer and Demultiplexer for io.Write.
Go
2
star
14

cycloid-hooks

Gathering of hooks used at cycloid
Shell
2
star
15

gopa

GO Client for OPA
Go
2
star
16

flatmap

This lib is a "fork" from of "terraform/flatmap" v0.14.10
Go
2
star
17

ansible-ecs

Install ECS-agent on servers
1
star
18

docker-cycloid-toolkit

Docker image which contain tools and a scripts for cycloid.io
Python
1
star
19

ansible-php

Jinja
1
star
20

docker-image-mailslurper

Dockerized MailsSlurper
Dockerfile
1
star
21

cycloid-resource

Concourse resource to use Cycloid features inside pipelines
Go
1
star
22

ansible-telegraf

Cycloid wrapper who call dj-wasabi/ansible-telegraf role and add some additionnal plugins
Python
1
star
23

ansible-systemd

Complement of https://github.com/tumf/ansible-role-systemd-service.git to provide features like mount and timer
1
star
24

ansible-backup-snapshot

Jinja
1
star
25

sqlr

Set of useful interfaces to work with the GO 'database/sql'
Go
1
star
26

approval-resource

Python
1
star
27

ansible-cyclosible

1
star
28

bootstrap-stacks

Collection of stack templates to be used by customer to create simple functioning stack
HCL
1
star