Terraform Apps and Infra 3 - Fundamentals

In

CLI

Basics

1
2
3
4
5
6
7
8
9
10
11
12
# check
terraform version
# switch to working directory
terraform -chdir=<path_to/tf> <subcommand> 
# initialize directory
terraform init
# create ane execution plan
terraform plan
# apply changes
terraform apply
# clean up
terraform destroy

Plan deploy and clean up

1
2
3
4
5
6
7
8
9
10
11
12
# output a deployment plan
terraform plan -out <plan_name>
# output a destroy plan
terraform plan -destroy
# apply an specific plan
terraform apply <plan_name>
# only apply changes to targeted resources
terraform apply -target=<resource_name>
# pass a variable via the command line
terraform apply -bar my_variable=<variable>
# get provider info used in configuration
terraform providers

Syntax

Hashicorp Configuration Language (HCL)

tf syntax (default HCL)

  • Documentation

  • Syntax

    1
    2
    3
    4
    5
    6
    7
    8
    resource "aws_vpc" "main" {
      cidr_block = var.base_cidr_block
    }

    <BLOCK TYPE> "<BLOCK LABEL>" "<BLOCK LABEL>"{
      # block body
      <IDENTIFIER> = EXPRESSION # argument
    }

  • Components

    • blocks: containers for objects like resources.
    • arguments: assign a value to a name.
    • expressions: represent a value.
    • identifers: names (types, resources, variables)
    • commnents
      1
      2
      3
      4
      5
      6
      # single line comment
      // single line comment
      /*
        multi-line comment
        multi-line comment
      */

  • Example .tf

    1
    2
    3
    4
    5
    6
    7
    variable "example": {
      default = "hello"
    }
    resource "aws_instance" "example": {
      instance_type = "t2.micro"
      ami = "ami-abc123"
    }

tf.json syntax

  • Example tf.json
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    {
      "variable": {
        "example": {
          "default": "hello"
        }
      }
    }

    {
      "resource": {
        "aws_instance": {
          "example": {
            "instance_type": "t2.micro",
            "ami": "ami-abc123"
          }
        }
      }
    }

Directories and files

  • file extension: .tf or .tf.json.
  • encoding: UTF-8 (recommended) or CLRF.
  • modules
    • collection of Terraform files on a directory.
    • module: a module is the top level config files on directory. Nested directories are not included, they are considered other modules.
    • root module: working directory wjere Terrafrom is invokled. Terraform configuration consist of root module + some child modules.

Resources

  • Definition

    1
    2
    3
    4
    resource "aws_instance" "web" {
      ami = "ami-a1b2c3d4"
      instance_type = "t2.micro"
    }

  • Resource types

    • providers: plugins for Terraform, offers a collection for resource types
    • arguments: specific to the selected resource typoes
    • documentation: which every provioder uses to describes its resource types and arguments

  • Meta arguments

    • depends_on: specify hidden dependencies
    • count: create multiple resource instances according to a count
    • for_each: create multiple instances according to a map or set of strings
    • provider: select a non-default provider configuration
    • lifecyle: set lifecycle configurations
    • provisioners and connection: take extra actions after resource creation

  • Operation timeouts

    1
    2
    3
    4
    5
    resource "aws_db_instance" "example" {
      # ...
      create = "60m"
      delete = "2h"
    }

  • How to apply configs

    • create: create resources that exist in the config, but are not associated with real infra object in the state.
    • destroy: destroy resources that exist in the state but no longer in the config.
    • update in-place: update in-place resources whose arguments have existed.
    • destroy and re-create: destroy and re-create resources whose arguments have changed, but which cannot be updated in-place due to remote API limitations.

  • Resource behaviour

    • accessing resource attributes
      • expression with Terraform modules.
      • read-only attributes with info obtained from remote APIs.
      • providers included data sources.
    • resource dependencies
      • most secure dependencies are handled automatically.
      • Terraform analyzes expressions within a resource block to find references to other objects, and treats those referencies as ordering requirements when creating, updating or destroying resources.
      • it’s usually not mecessary to manually specify dependencies cannot be recognized implicitly in the configuration.
    • local resources only
      • specialized resources types that operate only within Terraform itself, calculating some results and saving those results in the state for future use (ssh-keys, self-signed certs…)

Variables

Input

  • Reserved words

    • source
    • version
    • providers
    • count
    • for_each
    • lifecycle
    • depends_on
    • local

  • Optional arguments for variable declarations

    • default (it makes the var optional)
    • type (types accepted)
      • string
      • number
      • bool
      • list<type>
      • set<type>
      • map<type>
      • object({<attribute> = <type>, ...})
      • tuple[<type>, ...]
    • description (documentation)
    • validation (block validation rules, in addition to types)
    • sensitive (limit Terraform UI output)

  • Examples

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    variable "image_id" {
      type = string
      description = "The id of the machine (AMI) to use for the server"

      validations {
        # check access to values uin var
        condition = length(var.image_id) > 4 && substr(var.image.id, 0, 4)=="ami-"
        error_message = "The image id value must be a valid AMI id, starting with \"ami-\"."
      }
    }

    variable "user_information" {
      type = object ({
        name = string
        address = string
      })
      sensitive = true
    }

  • Assign value to variables

    • Environment variables
      1
      2
      export TF_VAR_image_id=ami_123abc
      terraform plan
    • variable definition files (.tfvars, ..tfvars.json)
      • Examples
        1
        terraform apply -var-file="testing.tfvars"
        1
        2
        3
        4
        5
        image_id = "ami-123abc"
        availability_zone_names = [
          "eu-west-1",
          "eu-west-2"
        ]
      • Automatically loaded files
        • terraform.tfvars, terraform.tfvars.json
        • any files ending with .auto.tfvars, .auto.tfvars.json
    • CLI: -var
      1
      terraform apply -var="image_id=ami-123abc" -var='image_list=["ami-456drf","image_id=ami-789ghi"]'
    • Terrafrom workspace

Output

  • Definition

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    output "instance_ip_addr"
      availability_zone_names = aws_instance.server.private_ip
      description = "The private IP od the main server instance"

      # last resort, add comment explaining why
      depends_on[
        /*
        security group rule must be created before this IP address could be
        actually used, otherwise the services will be unreachable
        */
        aws_security_group_rule.local._access
      ]
    }

  • Arguments and constraints

    • module.<module_name>.<output_names>
    • optional
      • description
      • sensitive
      • depends_on

Local

  • Like function temporary local variables
    1
    2
    3
    4
    5
    6
    resource "aws_instance" "example" {
      # ...
      # local can only be accessed in expressions
      # within the module in which they are declared
      tags = local.common_tags
    }

Modules

Definition

  • Collection of .tf or .tf.json files kept together on a directory

  • Types (encapsulated)

    • Root module (files in main working directory)
      1
      2
      3
      4
      module "aws_elb" "example" {
        # retrieve a value from child package
        instances = modules.servers.instance_ids
      }
    • Child modules (called by the root module, can declare output to export to caller)
      1
      2
      3
      4
      module "servers" {
        source = "./app-cluster"
        servers = 5
      }
    • Published modules (downloaded from private or public registry)
      1
      2
      3
      4
      5
      module "servers" {
        source = "./hashicorp/consul/aws"
        version = "0.0.5"
        servers = 3
      }
      • Arguments for calling
        • source (required)
        • version (fopr published modules)
        • input variables (defiend in the module)
        • meta_arguments (for-each, depends_on, count, providers)

  • Refactoring code

    • Terraform can see the new location of the module block as an entorely different resource.
    • Use terraform state mv to inform that the block was moved
      • When passing resource addresses, use prefix module.<module_name>
      • If called with count or for each, then use prefix module.<module_name>[index]
    • Taint (deprecated, use Replace) a resource in a module
      1
      2
      # terraform taint module.salt_master.aws_instance.salt_master
      terraform apply -replace="aws_instance.example[0]"

Module Sources

  • Used during terraform init
  • Types
    • Local path
      1
      2
      3
      module "consult" {
        source = "./consult"
      }
    • Module registry
      • Terraform public registry
        1
        2
        3
        4
        module "consul" {
          source = "hashicorp/consul/aws"
          version = "0.1.0"
        }
      • Other registry
        1
        2
        3
        4
        5
        6
        module "consul" {
          # namespace = "app.terraform.io/namespace"
          # provider = azurerm
          source = "app.terraform.io/example-corp/k8s-cluster/azurerm"
          version = "1.1.0"
        }
      • Git repository
        1
        2
        3
        4
        5
        6
        module "vpc" {
          source = "git::https:://hashicorp/consul/vpc.git"
        }
        module "storage" {
          source = "git::ssh:://username@example.com/storage.git?ref=v1.2.0"
        }
      • Mercurial repository
        1
        2
        3
        module "vpc" {
          source = "hg::https:://hashicorp/consul/vpc.hg"
        }
      • HTTP Terraform get (zxip, tar.bz, tar.gz, tar.xz)
        1
        2
        3
        module "vpc" {
          source = "https:://example.com/vpc-module.zip"
        }
    • SaaS examples (unprefixed URLs or SSH)
      • Cloud Hosted Terraform: "app.terraform.io/namespace"
      • GitHub: "github.com/hashicorp/example" or "git@github.com:hashicorp/example.git"
      • Bitbucket public repos: "bitbucket.org/hashicorp/example"
      • AWS S3: s3::https://s3-eu-west-1.amazonws.com/examplecorp/vpc-module.zip
        • Uses environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, or ./AWS/credentials file, or EC2 instance profile.
      • GCS bucket (Google): gcs::https://googleapis.com/storage/v1/modules/vpc-module.zip
        • Uses environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, or ./AWS/credentials file, or EC2 instance profile.

Expressions and Functions

  • Expressions: reference or compute values on a configuration

    • Types: string, number, bool, list/tuple, map/object, null
    • Named values available: resources, input variables, lcoal values, child module outputs, data sources, file system and workspace info, block-local values
    • Conditionals: condition ? true_val : false_val

  • Functions: transform and combine values on expressions

    • FUNCION_NAME(<ARGUMENT_1>,<ARGUMENT_2>)
    • You can test them directy on CLI via terraform console
      1
      2
      3
      terraform console
      max(3,2)
      exit

State

  • State is used by Terraform to map real world resources to your configuration, keep track of metadata, and to improve performance for large infrastructures.
  • Stored by default in a local file named terraform.tfstate (it is recommended to save a backup), but it can also be stored remotely, which works better in a team environment.

Backend Configuration

  • Storage for Terraform snapshots
  • Each Terraform config can specify a backend (local or remote) for state storage
  • Only used by Terraform CLI top determine
    • where the state is stored
    • where the operations are performed
  • Backend supports multiple workspaces: AzureRM, Consul, COS, GCS, Kubernetes, Local, Manta, Posgres, Remote, S3
  • Backend block
1
2
3
4
5
6
7
8
9
10
terraform {
  # only one backend block
  backend "remote" {
    # can not refer to named values
    organization = "corp_example"
    workspaces {
      name = ex-app-prod
    }
  }
}
  • When it changes in a config, you must re-run terraform init
  • When it changes, Terraform gives you the option to migrate to your state

Types

Local

1
2
3
4
5
6
terraform {
  backend "local" {
      path = "/path/to/terraform.tfstate"
      # workspace_dir path to non-default workspaces
  }
}
1
2
3
4
5
6
7
# datasource configuration
data "terraform_remote_state" "zland" {
  backend "local"
  config = {
    path = "${path.module}/../../terraform.tfstate"
  }
}

Remote

1
2
3
4
5
6
7
8
9
terraform {
  backend "remote" {
    hostname =  "app.terraform.io"
    organization = "company"
    workspaces {
      name = "my-prod-app"
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
# datasource configuration
data "terraform_remote_state" "foo" {
  backend "remote"
  config = {
    # required
    organization = "company"
    workspaces {
      name = "my-prod-app"
      # optional: prefix
    }
    # optional: token, hostname
  }
}

S3

1
2
3
4
5
6
7
terraform {
  backend "s3" {
    bucket = "mybucket"
    key = "path/to/my/key"
    region = "eu-west-1"
  }
}
1
2
3
4
5
6
7
8
9
# datasource configuration
data "terraform_remote_state" "foo" {
  backend "s3"
  config {
    bucket = "terraform-state-prod"
    key = "terraform.tfstate"
    region = "eu-west-1"
  }
}

Azurerm

1
2
3
4
5
6
7
8
terraform {
  backend "azurerm" {
    resource_group_name = "StorageAccount-ResourceGroup"
    storage_account_name = "abcd1234"
    container_name = "tfstate"
    container_key = "prod.terraform.tfstate"
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
# datasource configuration
data "terraform_remote_state" "foo" {
  backend = "azurerm" {
    storage_account_name = "abcd1234"
    container_name = "tfstate"
    container_key = "prod.terraform.tfstate"
    # optional
    use_azuread_auth = true
    subscription_id = "00000000-0000-0000-000000000000"
    tenant_id = "00000000-0000-0000-000000000000"
  }
}

Working with State

  • Uses of state

    • Metadata: track resources dependencies
    • Performance: as cache
    • Sync: share file between team members (remote, as read-only)

  • Lock

    • State locking happens automatically on all operations that could write state. You won’t see any message that it is happening.
    • If state locking fails, Terraform will not continue.
    • Can be unlocked terraform force-unlock LOCK_ID (not recommended)

Workspaces

  • Workspaces are separate instances of state data, that can be used from the same working directory.
  • You can use workspaces to manage multiple non-overlapping groups of resources with the same configuration.
    • Every initialized working directory has at least one workspace. (If you haven’t created other workspaces, it is a workspace named default.)
    • For a given working directory, only one workspace can be selected at a time.
    • Most Terraform commands (including provisioning and state manipulation commands) only interact with the currently selected workspace.
    • Use the terraform workspace select command to change the currently selected workspace.
    • Use the terraform workspace list, terraform workspace new, and terraform workspace delete commands to manage the available workspaces in the current working directory.