Testing on Python

In

Types of tests

  • Unit tests: ensure the logical part of your code is working
  • Integration/service tests: ensure that your DAO (database access) and BO (Business objects) layers work as expected
  • UI tests: automate your UI

Behavior driven development (BDD)

  • Story: should have a clear, explicit title.
  • Scenario: acceptance criteria or scenario description of each specific case of the narrative. Its structure is:
    • Given: it starts by specifying the initial condition that is assumed to be true at the beginning of the scenario. This may consist of a single clause, or several.
    • When: it then states which event triggers the start of the scenario.
    • Then: it states the expected outcome, in one or more clauses.

Unit and Integration tests, with UnitTest

  • unittest is similar in structure to Junit.

Project structure

  • Example project structure
    1
    2
    3
    4
    5
    6
    7
    * scripts
      * process.py
    * tests
      * test_process.py
    * README.md
    * requirements.txt
    * .pylintrc
  • Naming: tests file for process.py may be test_process.py files.
  • Location: you may put your test files in a tests folder under the root folder of the project.
    • You should try to replicate under tests the same structure of your scripts folder.

File structure

  • Example, not real code

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    # imports
    import unittest
    # if we are working on AWS
    import boto3
    # external mock library for AWS
    from moto import mock_ec2
    # file to test
    from scripts.process import Process

    # activate mock from moto
    @mock_ec2
    class TestProcess(unittest.TestCase):

      # VARIABLES
      # private constants
      __LOCATION = "eu-west-1"

      # instance to test
      process = Process()

      # SET UP TEST INTEGRITY
      # test setup
      def setUp(self):
        # initalize, generic for tests, to keep integrity
        started_ec2_data = {}

      def tearDown(self):
        # clean up after process, to keep integrity
        process = ""

      # SET UP MOCKS
      def get_mock_helper(self):
        # mock code from another class
        helper = unittest.mock.Mock()
        helper.get_dict.return_value = self.__DUMMY_ROLE_DICT
        return helper
          
      # TESTS
      def test_report_ec2_has_instance(self):
        # set up input
        dict_profiles = {"demo-dev-1": "arn:demo_dev_1"}

        # patch mocks for external calls on test file
        with unittest.mock.patch(
          "scripts.process.Helper",
          return_value=self.get_mock_helper()
        ):
          # invoke method to test
          response = self.process.report_ec2(dict_profiles)
          # test assertion to check up results
          assert response["report"] != [] and response["errors"] == []

Run tests

  • You can run the tests manually via

    1
    python -m unittest discover -s "./tests/"

  • You may also check the code coverage for those unit tests.

    1
    2
    3
    4
    5
    6
    7
    8
    # install
    python -m pip install coverage
    # run
    python -m coverage run -m --source="scripts" unittest discover
    # get coverage report on CLI
    python -m coverage report
    # get html coverage
    python -m coverage html

Tests E2E on UI, with Gherkin and Python

Write Gherkin files

Project structure

  • Example project structure
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    * scripts
      * process.py
    * tests
      * test_process.py
    * features
      * features
        aplication.feature
      * images
        * application_running.png
      * steps
        * application.py
        * application_walkthrough.py
    * README.md
    * requirements.txt
    * .pylintrc
    • Naming: source files have .feature extension.
    • Location: under features folder, single Gherkin source file contains a description of a single feature.
      • feature_tests: .feature files
      • images: screenshots, sorted out by platform folder
      • steps: .py test steps

File structure

.feature files

  • Every .feature must consists of a single feature.
  • Lines starting with the keyword Feature followed by three indented lines starts a feature.
  • A feature usually contains a list of scenarios, which starts with Scenario keyword on a new line.
    • Every scenario starts with the Scenario: keyword (or localized one), followed by an optional scenario title.
    • Every scenario has steps:
      • Given.
      • When.
      • Then.
    • If scenarios are repetitive, you may use scenario outlines, using a template with placeholders.
  • You can use tags to group features and scenarios together, independent of your file and directory structure.
  • Background: it is like an untitled scenario, containing a number of steps. The background is run before each of your scenarios, but after your BeforeScenario hooks.

Examples

  • Basic example

    1
    2
    3
    4
    5
    6
    7
    8
    Feature: Serve coffee
      In order to earn money Customers should be able to buy coffee at all times

      Scenario: Buy last coffee
        Given there are 1 coffees left in the machine
        And I have deposited 1 dollar
        When I press the coffee button
        Then I should be served a coffee

  • Outline example

    1
    2
    3
    4
    5
    6
    7
    Scenario Outline: Eating
      Given there are <start> cucumbers
      When I eat <eat> cucumbers
      Then I should have <left> cucumbers  Examples:
        | start | eat | left |
        |  12   |  5  |  7   |
        |  20   |  5  |  15  |

  • Background example

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    Feature: Multiple site support

      Background:
        Given a global administrator named "Greg"
        And a blog named "Greg's anti-tax rants"
        And a customer named "Wilson"
        And a blog named "Expensive Therapy" owned by "Wilson"

      Scenario: Wilson posts to his own blog
        Given I am logged in as Wilson
        When I try to post to "Expensive Therapy"
        Then I should see "Your article was published."
      Scenario: Greg posts to a client's blog
        Given I am logged in as Greg
        When I try to post to "Expensive Therapy"
        Then I should see "Your article was published.

Get Python files

Quick how-to

  • Gherkin files can be transformed into python files using Behave and autogui

    1
    2
    3
    4
    5
    6
    # install
    python -m pip install behave
    python -m pip install pyautogui
    # use
    python -m behave
    # python -m behave features/

Code example

  • original features/features/example.feature file

    1
    2
    3
    4
    5
    6
    Feature: Showing off behave

      Scenario: Run a simple test
        Given we have behave installed
        When we implement 5 tests
        Then behave will test them for us!

  • resulting features/steps/example.py file

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    from behave import given, when, then, step

    @given('we have behave installed')
    def step_impl(context):
      pass

    @when('we implement {number:d} tests')
    def step_impl(context, number): 
      # number is converted into integer
      assert number > 1 or number == 0
      context.tests_count = number

    @then('behave will test them for us!')
    def step_impl(context):
      assert context.failed is False
      assert context.tests_count >= 0