shUnit2 - Testing AWS CLI scripts

In

Assumptions

  1. The script-under-test is simple and short and does not require its own file system (mock it or run in Docker)
  2. Commands to be mocked are not addressed by their full path (such scripts can usually be refactored to utilise $PATH)

Set up

Project structure

  • Tree
    1
    2
    3
    4
    ▶ workspace .
    ├── script.sh
    └── shunit2
        └── test_script.sh

Running tests

  • Run

    1
    bash shunit2/delete_stack.sh

  • Locate the script-under-test, add a line like this at the start of every test file:

    1
    script_under_test=$(basename "$0")

Installing shUnit2

  • You may take it from the master branch using something like this:
    1
    2
    3
    curl \
      https://raw.githubusercontent.com/kward/shunit2/6d17127dc12f78bf2abbcb13f72e7eeb13f66c46/shunit2 \
       -o /usr/local/bin/shunit2

Mocks

  • The script can be divided into

    • commands related to the internal logic of the script
    • commands related to the external behaviour of the script (things it changes or whatever it actually does)
      • The commands_log created by the mocks can be queried to make assertions about the script’s actual behaviour

  • A simple mock that just silently intercepts and logs the inputs passed into it (copy/past style) looks like:

    1
    2
    3
    4
    5
    6
    7
    8
    # ${FUNCNAME[0]} in Bash is the name of a function
    chmod() {
      echo "${FUNCNAME[0]} $*" >> commands_log
    }

    chown() {
      echo "${FUNCNAME[0]} $*" >> commands_log
    }

  • A more complicated mocks can respond with fake responses:

    1
    2
    3
    4
    5
    6
    7
    some_command() {
      echo "${FUNCNAME[0]} $*" >> commands_log
      case "${FUNCNAME[0]} $*"
        "${FUNCNAME[0]} some_arg_a some_arg_b") ; echo some_response_1 ;;
        "${FUNCNAME[0]} some_arg_c some_arg_d") ; echo some_response_2 ;;
      esac
    }

  • The tearDown function provided by shUnit2 is later expected to clean up the commands_log

    1
    2
    3
    tearDown() {
      rm -f commands_log
    }

Learn by example

Cloudformation

  • Cloudformation script

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    #!/usr/bin/env bash

    usage() {
      echo "Usage: $0 STACK_NAME S3_BUCKET"
      exit 1
    }

    delete_all_artifacts() {
      aws ec2 delete-key-pair \
        --key-name "$stack_name"
      aws s3 rm --recursive --quiet \
        s3://"$s3_bucket"/deployments/"$stack_name"
    }

    resume_all_autoscaling_processes() {
      asgs=$(aws cloudformation describe-stack-resources \
        --stack-name "$stack_name" \
        --query \
        'StackResources[?ResourceType== \
        `AWS::AutoScaling::AutoScalingGroup`].PhysicalResourceId' \
        --output text)

      for asg in $asgs
      do
        aws autoscaling resume-processes \
          --auto-scaling-group-name "$asg"
      done
    }

    [ $# -ne 2 ] && usage
    # validates the inputs
    read -r stack_name s3_bucket <<< "$@"

    # deletes a key pair and deployment artifacts
    delete_all_artifacts
    # resumes any suspended processes in auto-scaling groups
    resume_all_autoscaling_processes

    # deletes the CloudFormation stack specified in the inputs
    aws cloudformation delete-stack \
      --stack-name "$stack_name"

  • Designing the tests

    • Test cases

      • a usage message is expected if incorrect inputs are passed
        • for a stack with no auto-scaling groups:
          • key pairs and deployment artifacts are expected to be deleted
          • aws cloudformation delete-stack should be issued
        • for a stack with multiple auto-scaling groups:
          • a resume-processes command should be issued for each auto-scaling group
      • Issue: it doesn’t try to handle a non-existent S3 bucket and a non-existent CloudFormation stack. So we can:
        • document this as a known issue
        • fixing the script to be more defensive (recommended!)
        • writing tests to test for and demonstrate the known issue

    • Structure of the tests on shunit2/delete_stack.sh

      1. the variable $script_under_test as mentioned above
      2. a mocks section to replace commands that make calls to AWS
      3. a more general setup/teardown section
      4. some test cases, being the shell functions whose names start with test*
      5. the final call to shUnit2 itself

  • Test cases

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    #!/usr/bin/env bash

    # section 1 - the script under test
    script_under_test=$(basename "$0")

    # section 2 - the mocks
    aws() {
      echo "aws $*" >> commands_log
      case "aws $*" in
        "aws ec2 delete-key-pair --key-name mystack") true ;;
        "aws s3 rm --recursive --quiet s3://mybucket/deployments/mystack") \
          true ;;

        "aws cloudformation describe-stack-resources \
        --stack-name mystack \
        --query "'StackResources[?ResourceType== \
          `AWS::AutoScaling::AutoScalingGroup`].PhysicalResourceId'" \
        --output text")
          echo mystack-AutoScalingGroup-xxxxxxxx
          ;;

        "aws autoscaling resume-processes \
        --auto-scaling-group-name mystack-AutoScalingGroup-xxxxxxxx")
          true
          ;;

        "aws cloudformation delete-stack --stack-name mystack") true ;;
        *) echo "No response for >>> aws $*" ;;
      esac
    }

    # section 3 - other setup or teardown
    tearDown() {
      rm -f commands_log
      rm -f expected_log
    }

    # section 4 - the test cases
    testSimplestExample() {
      . "$script_under_test" mystack mybucket

      cat > expected_log <<'EOF'
    aws ec2 delete-key-pair --key-name mystack
    aws s3 rm --recursive --quiet s3://mybucket/deployments/mystack
    aws cloudformation describe-stack-resources --stack-name mystack \
      --query StackResources[?ResourceType== \
      `AWS::AutoScaling::AutoScalingGroup`].PhysicalResourceId --output text
    aws autoscaling resume-processes \
      --auto-scaling-group-name mystack-AutoScalingGroup-xxxxxxxx
    aws cloudformation delete-stack --stack-name mystack
    EOF

      # diff -wu ensures that during failures, a nice readable unified diff
      # of “expected” compared to “actual” is seen
      assertEquals "unexpected sequence of commands issued" \
        "" "$(diff -wu expected_log commands_log | colordiff | DiffHighlight.pl)"
    }

    # section 5 - the call to shUnit2 itself
    . shunit2

Bad inputs

  • Test cases
    1
    2
    3
    4
    5
    6
    7
    # check if the input on method call is wrong
    testBadInputs() {
      # STDOUT is captured using command substitution $( ... )
      actual_stdout=$(. "$script_under_test" too many arguments passed)
      assertTrue "unexpected response when passing bad inputs" \
        "echo $actual_stdout | grep -q ^Usage"
    }

No auto-scaling

  • Additional mocks

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    aws() {
      ...
      # responses for motherstack
      "aws ec2 delete-key-pair --key-name myotherstack") true ;;
      "aws s3 rm --recursive --quiet s3://mybucket/deployments/myotherstack") \
        true ;;

      "aws cloudformation describe-stack-resources \
        --stack-name myotherstack \
        --query "'StackResources[?ResourceType== \
          `AWS::AutoScaling::AutoScalingGroup`].PhysicalResourceId'" \
        --output text")
        ## Manual tests show that this command returns an empty string for this
        echo ""  
        ;;

      "aws cloudformation delete-stack --stack-name myotherstack") true ;;
    }

  • Test cases

    1
    2
    3
    4
    5
    testNoASGs() {
      . "$script_under_test" myotherstack mybucket
      assertFalse "a resume-processes command was unexpectedly issued" \
        "grep -q resume-processes commands_log"
    }

Running the tests

  • Command plus output
    1
    bash shunit2/test_delete_stack.sh