Terraform guide 1 - Introduction

In

Terraform: Introduction

Definition

  • Terraform = tool for building infrastructure
  • Allow simple control versopm
    • opensource
    • enterprise (collaboration, governance)
  • Features
    • Infrastructire as Code (IaC)
      • idempotent
      • high level syntax (Hashicorp)
      • easily resusable
    • Execution plans
      • intents of deploy
      • help ensure eveything in development is intentional
    • Resource graph
      • illustrates all cahnges and dependecies
  • Uses
    • Hybrid clouds
      • cloud agnostic
      • allows deployment to multiple providers simultaneously
    • Mult-tier architecture
      • allows deployment of several layers of architecture
      • usually able to automatically deploy in the correct order
    • Software-defined networking
      • able to deploy network architecture as well
  • Terraform is high-level infrastructire orchestration tool
    • Puppet, Chef, Ansible for configuration management
      • Providers that can tell this tools to perform CM duties

Setting up Docker Installing Terraform

  1. Prerequisires

    • Update the operating system and uninstall older versions 
      1
      2
      3
      4
      5
      6
      7
      8
      9
      sudo yum update -y
      sudo yum remove -y docker \
                        docker-client \
                        docker-client-latest \
                        docker-common \
                        docker-latest \
                        docker-latest-logrotate \
                        docker-logrotate \
                        docker-engine

  2. Install Docker CE

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    # install utils
    sudo yum install -y yum-utils \
    device-mapper-persistent-data \
    lvm2
    # add docker repo
    sudo yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo
    # install docker CE and enable it
    sudo yum -y install docker-ce
    sudo systemctl start docker && sudo systemctl enable docker
    # add user to docker group
    sudo usermod -aG docker cloud_user
    # test installation
    docker --version

  3. Configuring Swarm Manager node

    1
    docker swarm init --advertise-addr [PRIVATE_IP]

  4. Configure the Swarm Worker node

    1
    docker swarm join --token [TOKEN] [PRIVATE_IP]:2377

  5. Verify the Swarm cluster

    1
    docker node ls

  6. Install Terraform 0.11.13 on the Swarm manager

    1
    2
    3
    4
    5
    sudo curl -O https://releases.hashicorp.com/terraform/0.11.13/terraform_0.11.13_linux_amd64.zip
    sudo yum install -y unzip
    sudo unzip terraform_0.11.13_linux_amd64.zip -d /usr/local/bin/
    # test it
    terraform version

Deploying infrastructure with Terraform: basics

Terraform commands

  • Common commands

    Command Action
    apply Builds or changes infrastructure
    console Interactive console for Terraform interpolations
    destroy Destroys Terraform-managed infrastructure
    fmt Rewrites configuration files to canonical format
    get Downloads and installs modules for the configuration
    graph Creates a visual graph of Terraform resources
    import Imports existing infrastructure into Terraform
    init Initializes a new or existing Terraform configuration
    output Reads an output from a state file
    plan Generates and shows an execution plan
    providers Prints a tree of the providers used in the configuration
    push Uploads this Terraform module to Terraform Enterprise to run
    refresh Updates local state file against real resources
    show Inspects Terraform state or plan
    taint Manually marks a resource for recreation
    untaint Manually unmarks a resource as tainted
    validate Validates the Terraform files
    version Prints the Terraform version
    workspace Workspace management

  • Set up the environment

    1
    2
    mkdir -p terraform/basics
    cd terraform/basics

  • Create a Terraform script

    1
    nano main.tf
    • main.tf contents 
      1
      2
      3
      4
      # Download the latest Ghost image
      resource "docker_image" "image_id" {
          name = "ghost:latest"
      }

  • Initialize Terraform

    1
    terraform init

  • Validate the Terraform file

    1
    terraform validate

  • List providers in the folder

    1
    ls .terraform/plugins/linux_amd64/

  • List providers used in the configuration

    1
    terraform providers

  • Terraform Plan

    1
    terraform plan
    • useful flags
      • -out=path: Writes a plan file to the given path. This can be used as input to the “apply” command.
      • -var 'foo=bar': Set a variable in the Terraform configuration. This flag can be set multiple times.

  • Terraform Apply

    1
    2
    # you should confirm
    terraform apply
    • useful flags
      • -auto-approve: This skips interactive approval of plan before applying
      • -var 'foo=bar': This sets a variable in the Terraform configuration. It can be set multiple times.

  • List the Docker images

    1
    docker image ls

  • Terraform Show

    1
    terraform show

  • Terraform Destroy

    1
    2
    # you should confirm
    terraform destroy
    • useful flags
      • -auto-approve: Skip interactive approval of plan before applying.

  • Re-list the Docker images

    1
    docker image ls

  • Using a plan

    1
    terraform plan -out=tfplan

  • Applying a plan:

    1
    terraform apply tfplan

  • Show the Docker Image resource:

    1
    terraform show

HashiCorp Configuration Language (HCL)

  • Terraform code files (.tf and .tf.json)

    1
    2
    3
    4
    5
    6
    resource "aws_instance" "example" {
        ami = "abc123"
        network_interface {
            # ...
        }
    }

  • Syntax reference

    Type Syntax
    Single line comments # this is a comment
    Multi-line comments /* This is a comment*/
    Assign values key = value
    Strings "This is a String"
    Strings interpolation ${var.foo}.
    Numbers 10 # assumed as base 10
    Boolean true, false
    Lists of primitive types ["foo", "bar", "baz"]
    Maps { "foo": "bar", "bar": "baz" }

  • Style Conventions

    • Indent two spaces for each nesting level
    • With multiple arguments, align their equals signs
    • Setup the environment 
      1
      cd terraform/basics

  • Deploying a container using Terraform

    1. Redeploy the Ghost image 
      1
      terraform apply
    2. Open main.tf: 
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      # Download the latest Ghost image
      resource "docker_image" "image_id" {
          name = "ghost:latest"
      }
            
      # Start the Container
      resource "docker_container" "container_id" {
          name  = "ghost_blog"
          image = "${docker_image.image_id.latest}"
          ports {
              internal = "2368"
              external = "80"
          }
      }
    3. Validate main.tf, plan and apply 
      1
      2
      3
      4
      5
      terraform validate
      terraform plan
      terraform apply
      #check
      docker container ls
    4. Check Ghost on browser (http:://[SWAM_MANAGER_IP])

  • Cleaning up the environment

    1
    terraform destroy

Tainting and updating resources

  • Terraform commands:

    • taint: Manually mark a resource for recreation 
      1
      terraform taint [NAME]
    • untaint: Manually unmark a resource as tainted 
      1
      terraform untaint [NAME]

  • Updating resources

    • edit main.tf 
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      # Download the latest Ghost image
      resource "docker_image" "image_id" {
          name = "ghost:alpine"
      }

      # Start the Container
      resource "docker_container" "container_id" {
          name  = "ghost_blog"
          image = "${docker_image.image_id.latest}"
          ports {
              internal = "2368"
              external = "80"
          }
      }
    • script 
      1
      2
      3
      4
      5
      6
      7
      terraform validate
      terraform plan
      terraform apply
      # list containers
      docker container ls
      # check Ghost image
      docker image ls | grep [IMAGE]

  • Cleaning up the environment

    1
    2
    3
    4
    5
    6
    # reset
    terraform destroy
    # list the Docker images
    docker image ls
    # remove the Ghost blog image
    docker image rm ghost:latest

Terraform console and output

  • Terraform commands:

    • console: Interactive console for Terraform interpolations

  • Working with the Terraform console

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    # Redeploy image and container
    terraform apply
    terraform show
    terraform console
    # Type the following in the console to get the container's name
    docker_container.container_id.name
    # Type the following in the console to get the container's IP
    docker_container.container_id.ip_address
    # Break out of the Terraform console by using Ctrl+C
    terraform destroy

  • Output the name and IP of the container

    • Edit main.tf 
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      # Download the latest Ghost Image
      resource "docker_image" "image_id" {
          name = "ghost:latest"
      }

      # Start the Container
      resource "docker_container" "container_id" {
          name  = "blog"
          image = "${docker_image.image_id.latest}"
          ports {
              internal = "2368"
              external = "80"
          }
      }

      # Output the IP Address of the Container
      output "ip_address" {
          value       = "${docker_container.container_id.ip_address}"
          description = "The IP for the container."
      }

      # Output the Name of the Container
      output "container_name" {
          value       = "${docker_container.container_id.name}"
          description = "The name of the container."
      }
    • Script 
      1
      2
      3
      4
      5
      terraform validate
      # apply changes to get the output
      terraform apply
      # clean up
      terraform destroy

Input variables

  • Definition

    • Parameters for a Terraform file
    • A variable block configures a single input variable for a Terraform module
    • Each block declares a single variable.

  • Syntax

    1
    2
    3
    variable [NAME] {
        [OPTION] = "[VALUE]"
    }

  • Arguments (between { })

    Optional parameters Value
    type defines the type of the variable (string, list or map)
    default default value. If not provided, Terraform will raise an error if a value is not provided by the caller
    description human-friendly description for the variable

  • Using variables during an apply

    1
    terraform apply -var 'foo=bar'

  • Example with main.tf contents

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    # Define variables
    variable "image_name" {
        description = "Image for container."
        default     = "ghost:latest"
    }

    variable "container_name" {
        description = "Name of the container."
        default     = "blog"
    }

    variable "int_port" {
        description = "Internal port for container."
        default     = "2368"
    }

    variable "ext_port" {
        description = "External port for container."
        default     = "80"
    }
      
    # Download the latest Ghost Image
    resource "docker_image" "image_id" {
        name = "${var.image_name}"
    }
      
    # Start the Container
    resource "docker_container" "container_id" {
        name  = "${var.container_name}"
        image = "${docker_image.image_id.latest}"
        ports {
            internal = "${var.int_port}"
            external = "${var.ext_port}"
        }
    }
      
    # Output the IP Address of the Container
    output "ip_address" {
        value       = "${docker_container.container_id.ip_address}"
        description = "The IP for the container."
    }

    output "container_name" {
        value       = "${docker_container.container_id.name}"
        description = "The name of the container."
    }
    1
    2
    3
    4
    5
    6
    7
    8
    terraform validate
    terraform plan
    # apply using the variable
    terraform apply -var 'ext_port=8080'
    # change the container name:
    terraform apply -var 'container_name=ghost_blog' -var 'ext_port=8080'
    # reset
    terraform destroy -var 'ext_port=8080'

Breaking out our variables and outputs

  1. Edit variables.tf 
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    #Define variables
    variable "container_name" {
        description = "Name of the container."
        default     = "blog"
    }
    variable "image_name" {
        description = "Image for container."
        default     = "ghost:latest"
    }
    variable "int_port" {
        description = "Internal port for container."
        default     = "2368"
    }
    variable "ext_port" {
        description = "External port for container."
        default     = "80"
    }
  2. Edit main.tf: 
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    # Download the latest Ghost Image
    resource "docker_image" "image_id" {
        name = "${var.image_name}"
    }

    # Start the Container
    resource "docker_container" "container_id" {
        name  = "${var.container_name}"
        image = "${docker_image.image_id.latest}"
        ports {
            internal = "${var.int_port}"
            external = "${var.ext_port}"
        }
    }
  3. Edit outputs.tf: 
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    #Output the IP Address of the Container
    output "ip_address" {
        value       = "${docker_container.container_id.ip_address}"
        description = "The IP for the container."
    }

    output "container_name" {
        value       = "${docker_container.container_id.name}"
        description = "The name of the container."
    }
  4. Script 
    1
    2
    3
    4
    5
    6
    terraform validate
    # plan and apply
    terraform plan -out=tfplan -var container_name=ghost_blog
    terraform apply tfplan
    # clean up
    terraform destroy -auto-approve -var container_name=ghost_blog

Maps and lookups

  • Map: specify different environment variables based on conditions
  • Example
    1. Edit variables.tf 
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      #Define variables
      variable "env" {
          description = "env: dev or prod"
      }
      variable "image_name" {
          type        = "map"
          description = "Image for container."
          default     = {
              dev  = "ghost:latest"
              prod = "ghost:alpine"
          }
      }
      variable "container_name" {
          type        = "map"
          description = "Name of the container."
          default     = {
              dev  = "blog_dev"
              prod = "blog_prod"
          }
      }

      variable "int_port" {
        description = "Internal port for container."
        default     = "2368"
      }
      variable "ext_port" {
          type        = "map"
          description = "External port for container."
          default     = {
              dev  = "8081"
              prod = "80"
          }
      }
    2. Edit main.tf 
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      # Download the latest Ghost Image
      resource "docker_image" "image_id" {
          name = "${lookup(var.image_name, var.env)}"
      }

      # Start the Container
      resource "docker_container" "container_id" {
          name  = "${lookup(var.container_name, var.env)}"
          image = "${docker_image.image_id.latest}"
          ports {
              internal = "${var.int_port}"
              external = "${lookup(var.ext_port, var.env)}"
          }
      }
    3. Validate it and deploy 
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      terraform validate
      terraform plan -out=tfdev_plan -var env=dev
      terraform apply tfdev_plan
      # plan the prod deploy
      terraform plan -out=tfprod_plan -var env=prod
      # use environment variables
      export TF_VAR_env=prod
      terraform console
      # execute a lookup
      lookup(var.ext_port, var.env)
      # exit the console:
      unset TF_VAR_env

Terraform Workspaces

  • Terraform commands

    Command Type Action
    workspace Command New, list, select and delete Terraform workspaces
    delete Subcommand delete a workspace
    list Subcommand list Workspaces
    new Subcommand create a new workspace
    select Subcommand select a workspace
    show Subcommand show the name of the current workspace

  • Example

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    cd terraform/basics
    # dev workspace
    terraform workspace new dev
    terraform plan -out=tfdev_plan -var env=dev
    terraform apply tfdev_plan

    # prod workspace
    terraform workspace new prod
    terraform plan -out=tfprod_plan -var env=prod
    terraform apply tfprod_plan

    # default workspace
    terraform workspace select default

    # find what workspace we are using now
    terraform workspace show
    # select a workspace
    terraform workspace select dev

    # clean up
    terraform destroy -var env=dev
    terraform workspace select prod
    terraform destroy -var env=prod

Null resources and Local-exec

  • null_resource resource implements the standard resource lifecycle but takes no further action
  • local-exec provisioner invokes a local executable after a resource is created
  1. Edit main.tf 
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    # Download the latest Ghost Image
    resource "docker_image" "image_id" {
        name = "${lookup(var.image_name, var.env)}"
    }

    # Start the Container
    resource "docker_container" "container_id" {
        name  = "${lookup(var.container_name, var.env)}"
        image = "${docker_image.image_id.latest}"
        ports {
            internal = "${var.int_port}"
            external = "${lookup(var.ext_port, var.env)}"
        }
    }

    resource "null_resource" "null_id" {
        provisioner "local-exec" {
            command = "echo ${docker_container.container_id.name}:${docker_container.container_id.ip_address} >> container.txt"
        }
    }
  2. Null Resource -> perform local commands on our machine without having to deploy extra resources 
    1
    2
    3
    4
    5
    6
    7
    8
    9
    # reinitialize
    terraform init
    terraform validate
    terraform plan -out=tfplan -var env=dev
    terraform apply tfplan
    # view container info
    cat container.txt
    # clean up
    terraform destroy -auto-approve -var env=dev

Terraform modules

  • Module: container for multiple resource

    1
    2
    3
    4
    5
    6
    7
    8
    9
    # setup environment for root module
    mkdir -p modules/image
    mkdir -p modules/container
    # files for image
    cd ~/terraform/basics/modules/image
    touch main.tf variables.tf outputs.tf
    # files for container module
    cd ~/terraform/basics/modules/container
    touch main.tf variables.tf outputs.tf

Image module

  • environment: ``~/terraform/basics/modules/image`
  • example
    1. Edit main.tf 
      1
      2
      3
      4
      # Download the Image
      resource "docker_image" "image_id" {
          name = "${var.image_name}"
      }
    2. Edit variables.tf 
      1
      2
      3
      variable "image_name" {
          description = "Name of the image"
      }
    3. Edit outputs.tf 
      1
      2
      3
      output "image_out" {
          value       = "${docker_image.image_id.latest}"
      }
    4. Initialize Terraform: 
      1
      2
      3
      4
      5
      terraform init
      terraform plan -out=tfplan -var 'image_name=ghost:alpine'
      terraform apply -auto-approve tfplan
      # clean up
      terraform destroy -auto-approve -var 'image_name=ghost:alpine'

Container module

  • environment: ~/terraform/basics/modules/container
  • example: breaking out the container code into it’s own module
    1. Edit main.tf 
      1
      2
      3
      4
      5
      6
      7
      8
      9
      # Start the Container
      resource "docker_container" "container_id" {
          name  = "${var.container_name}"
          image = "${var.image}"
          ports {
              internal = "${var.int_port}"
              external = "${var.ext_port}"
          }
      }
    2. Edit variables.tf 
      1
      2
      3
      4
      5
      #Define variables
      variable "container_name" {}
      variable "image" {}
      variable "int_port" {}
      variable "ext_port" {}
    3. Edit outputs.tf 
      1
      2
      3
      4
      5
      6
      7
      8
      #Output the IP Address of the Container
      output "ip" {
          value = "${docker_container.container_id.ip_address}"
      }

      output "container_name" {
          value = "${docker_container.container_id.name}"
      }
    4. Script 
      1
      2
      3
      terraform init
      terraform plan -out=tfplan -var 'container_name=blog' -var 'image=ghost:alpine' -var 'int_port=2368' -var 'ext_port=80'
      terraform apply tfplan

Root module

  • environment: ~/terraform/basics/modules/
  • example: refactor the root module to use the image and container modules we created previously
    1. Edit main.tf 
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      # Download the image
      module "image" {
          source = "./image"
          image_name  = "${var.image_name}"
      }

      # Start the container
      module "container" {
          source             = "./container"
          image              = "${module.image.image_out}"
          container_name     = "${var.container_name}"
          int_port           = "${var.int_port}"
          ext_port           = "${var.ext_port}"
      }
    2. Edit variables.tf 
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      #Define variables
      variable "container_name" {
          description = "Name of the container."
          default     = "blog"
      }
      variable "image_name" {
          description = "Image for container."
          default     = "ghost:latest"
      }
      variable "int_port" {
          description = "Internal port for container."
          default     = "2368"
      }
      variable "ext_port" {
          description = "External port for container."
          default     = "80"
      }
    3. Edit outputs.tf 
      1
      2
      3
      4
      5
      6
      7
      8
      # Output the IP Address of the Container
      output "ip" {
          value = "${module.container.ip}"
      }

      output "container_name" {
          value = "${module.container.container_name}"
      }
    4. Script 
      1
      2
      3
      4
      5
      terraform init
      terraform plan -out=tfplan
      terraform apply tfplan
      # clean up
      terraform destroy -auto-approve