Terraform Apps and Infra 5 - Terraform cloud

In

Migrating Local Terraform

CLI Migration

  1. Check version: the cloud block is available in Terraform v1.1, plus a Terraform account
  2. add the cloud block to your backend configuration 
    1
    2
    3
    4
    5
    6
    7
    8
    terraform {
      cloud {
        organization = "my-org"
        workspaces {
          tags = ["networking"]
        }
      }
    }
  3. specify one or more Terraform Cloud workspaces for the state files
  4. run terraform init

API migration

Integrations

VCS supported

  • GitHub
    • GitHub.com
    • GitHub.com (OAuth)
    • GitHub Enterprise
  • GitLab
    • GitLab.com
    • GitLab EE and CE
  • Bitbucket
    • Bitbucket Cloud
    • Bitbucket Server
  • Azure
    • Azure DevOps Server
    • Azure DevOps Services

How it works

  • Terraform Cloud needs:

    • access a list of repositories, to let you search for repos when creating new workspaces.
    • register webhooks with your VCS provider, to get notified of new commits to a chosen branch.
    • download the contents of a repository at a specific commit in order to run Terraform with that code.

  • Webhooks: monitor new commits and pull requests.

    • When someone adds new commits to a branch, any Terraform Cloud workspaces based on that branch will begin a Terraform run. Usually a user must inspect the plan output and approve an apply, but you can also enable automatic applies on a per-workspace basis. You can prevent automatic runs by locking a workspace.
    • When someone submits a pull request/merge request to a branch, any Terraform Cloud workspaces based on that branch will perform a speculative plan with the contents of the request and links to the results on the PR’s page. This helps you avoid merging PRs that cause plan failures.

  • SSH keys: required for Azure and Bitbucket, only for cloning. All other operations can be done by HTTPS.

Workspaces in Terraform Cloud

Component Local Terraform Terraform Cloud
Terraform configuration On disk In linked version control repository, or periodically uploaded via API/CLI
Variable values As .tfvars files, as CLI arguments, or in shell environment In workspace
State On disk or in remote backend In workspace
Credentials and secrets In shell environment or entered at prompts In workspace, stored as sensitive variables
  • It uses workspaces instead of directories
    • State versions: Each workspace retains backups of its previous state files. Although only the current state is necessary for managing resources, the state history can be useful for tracking changes over time or recovering from problems.
    • Run history: When Terraform Cloud manages a workspace’s Terraform runs, it retains a record of all run activity, including summaries, logs, a reference to the changes that caused the run, and user comments.
  • After creating a workspace
    • Allows to edit variables
      • terraform.tfvars or -var-file=terraform.tfvars
      • environment variables with export on shell
      • special environment variables (TF_PARALELISM sets up flag: terraform plan -paralelism<N> and terraform apply -paralelism<N>))
    • Creates a webhook with VCS

Runs in Terraform Cloud

  • Workspace has runs links.
  • CLI or API

Run workflows

  • UI/VCS-driven: primary mode of operation
  • API-driven: more flexible but requires you to create some tooling
  • CLI-driven: uses Terraform’s standard CLI tools to execute runs in Terraform Cloud

Environment

  • Terraform workersVM: single-use Linux VMs
  • Network access to VCS and Infra providers: needs access to all resources managed
  • Concurrency and run queueing: uses multiple workers
  • State access and authentication: stores state for its workspaces

Environment variables

Variable Name Description Example
TFC_RUN_ID A unique identifier for this run run-CKuwsxMGgMd4W7Ui
TFC_WORKSPACE_NAME The name of the workspace used in this run prod-load-balancers
TFC_WORKSPACE_SLUG The full slug of the configuration used in this run. This consists of the organization name and workspace name, joined with a slash acme-corp/prod-load-balancers
TFC_CONFIGURATION_VERSION_GIT_BRANCH The name of the branch that the associated Terraform configuration version was ingressed from main
TFC_CONFIGURATION_VERSION_GIT_COMMIT_SHA The full commit hash of the commit that the associated Terraform configuration version was ingressed from abcd1234...
TFC_CONFIGURATION_VERSION_GIT_TAG The name of the tag that the associated Terraform configuration version was ingressed from v0.1.0

Run States and Stages

graph TD

subgraph PENDING
A[pending]
end

subgraph PLAN
B[fetching]
C[planning]
D[confirmation]
end

subgraph POST-PLAN
E[running tasks]
end

subgraph COST-ESTIMATION
F[cost estimating]
G[cost estimated]
end

subgraph POLICIY CHECK
H[policy check]
I[policy override]
J[policy checked]
end

subgraph APPLY
K[cost estimating]
end

subgraph COMPLETED
T[applied]
U[applied errored]
V[plan errored]
W[planned and finished]
X[cancelled]
Y[plan errored]
Z[discarded]
end

A -- user discards before start --> Z
A -- 1st in queue --> B

B -- can not connect to VCS --> Y
B -- success --> C
C -- cancel --> X
C -- terraform plan failed --> Y
C -- success and no further policy stages required --> W
C -- success --> D
D -- success, run tasks enabled --> E
D -- success, cost estimation enabled --> F
D -- success, no cost estimation, sentinel policies --> F
D -- success, no cost estimation, no policies, auto-apply --> K
D -- success, no cost estimation, no policies, no get authorize manually --> D
D -- success, no cost estimation, no policies, reject authorize manually --> Z

E -- mandatory tasks failed --> V
E -- advised tasks failed --> K
E -- user cancels --> X

F -- success --> G
F -- no policy checks nor applies -->  W
G -- success --> H

H -- hard policy fails --> V
H -- soft policy fails --> I
H -- success --> J
I -- user overrides --> J
I -- user discards --> Z
J -- auto-applied or manual apply --> K
J -- duser discards --> Z

K -- success --> T
K -- fails --> U
K -- user cancels --> Z
  • Pending Stage: Terraform Cloud hasn’t started action on a run yet. Terraform Cloud processes each workspace’s runs in the order they were queued, and a run remains
    • Pending state: Terraform Cloud hasn’t started action on a run yet. Terraform Cloud processes each workspace’s runs in the order they were queued, and a run remains pending until every run before it has completed.
  • The Plan Stage: A run goes through different steps during the plan stage depending on whether or not Terraform Cloud needs to fetch the configuration from VCS. Terraform Cloud automatically archives configuration versions created through VCS when all runs are complete and then re-fetches the files for subsequent runs.
    • Fetching: If Terraform Cloud has not yet fetched the configuration from VCS, the run will go into this state until the configuration is available.
    • Planning: Terraform Cloud is currently running terraform plan.
    • Needs Confirmation: terraform plan has finished. Runs sometimes pause in this state, depending on the workspace and organization settings.
  • Post-Plan Stage: The post-plan phase executes any configured run tasks after the plan is complete, so it only occurs if there are run tasks enabled for the workspace. All runs can enter this phase, including speculative plans. During this phase, Terraform Cloud sends information about the run to the configured external system and waits for a passed or failed response to determine whether the run can continue.
    • Running tasks: Terraform Cloud is waiting for a response from the configured external system(s).
      • External systems must respond initially with a 200 OK acknowledging the request is in progress. After that, they have 10 minutes to return a status of passed, running, or failed, or the timeout will expire and the task will be assumed to be in the failed status.
  • Cost Estimation Stage: only occurs if cost estimation is enabled. After a successful terraform plan, Terraform Cloud uses plan data to estimate costs for each resource found in the plan.
    • Cost Estimating: Terraform Cloud is currently estimating the resources in the plan.
    • Cost Estimated: The cost estimate completed.
  • Policy Check Stage: This stage only occurs if Sentinel policies are enabled. After a successful terraform plan, Terraform Cloud checks whether the plan obeys policy to determine whether it can be applied.
    • Policy Check: Terraform Cloud is currently checking the plan against the organization’s policies.
    • Policy Override: The policy check finished, but a soft-mandatory policy failed, so an apply cannot proceed without approval from a user with permission to manage policy * overrides for the organization. The run pauses in this state.
    • Policy Checked: The policy check succeeded, and Sentinel will allow an apply to proceed. The run sometimes pauses in this state, depending on workspace settings.
      Leaving this stage:
  • Apply Stage
    • Applying: Terraform Cloud is currently running terraform apply.
  • Completion
    • Applied: The run was successfully applied.
    • Planned and Finished: terraform plan’s output already matches the current infrastructure state, so terraform apply doesn’t need to do anything.
    • Apply Errored: The terraform apply command failed, possibly due to a missing or misconfigured provider or an illegal operation on a provider.
    • Plan Errored: The terraform plan command failed (usually requiring fixes to variables or code), or a hard-mandatory Sentinel policy failed. The run cannot be applied.
    • Discarded: A user chose not to continue this run.
    • Canceled: A user interrupted the terraform plan or terraform apply command with the “Cancel Run” button.

Users, Teams, and Organizations

  • Users: individual members
  • Teams: group of users
    • Owner team: can manage an organization
  • Organizations: shared space of teams to collaborate on workspaces

Migration example

  1. Check terraform infra locally
  2. Generate your access keys and API keys on Terraform Cloud
  3. Set up you workspace session on https://app.terraform.io/session
  4. Create Your API Token for Terraform CLI Login
  5. Prepare your backend configuration on main.tf 
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
      terraform {
      backend "remote" {
        organization = "<YOUR ORG NAME>"
        workspaces {
          name = "lab-migrate-state"
        }
      }

      required_providers {
          aws = {
            source  = "hashicorp/aws"
            version = "~> 3.27"
          }
      }
    }
  6. On CLI 
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    # INIT
    # login
    terraform login
    # format files
    terraform fmt
    terraform init
    # verify terraform.tfstate.backup exists
    ls
    # clean up
    rm -rf terraform.tfstate

    # APPLY
    terraform apply
    # confirm state on cloud via browser