Angeles' code notepad

Coding notes

Angeles' code notepad

In

Audit

  1. Run audit to check the packages status.

    1
    npm audit

  2. Take note of the issues report, so you know in which package meeds to be fixed.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
                          === npm audit security report ===                        


                                    Manual Review                                  
                Some vulnerabilities require your attention to resolve             

            Visit https://go.npm.me/audit-guide for additional guidance           


    High            Arbitrary Code Execution                                      

    Package         underscore                                                    

    Patched in      >=1.12.1                                                      

    Dependency of   serverless-plugin-resource-tagging                            

    Path            serverless-plugin-resource-tagging > underscore               

    More info       https://npmjs.com/advisories/1674                             

    found 1 vulnerabilitys (1 high) in 985 scanned packages
    1 vulnerability require manual review. See the full report for details.

Fix

Automated fix

  1. Run audit fix, for easy issues. 
    1
    npm audit fix

Manual fix for “dependency depth > 1” case

  • You can use the package npm-force-resolutions, but consider using the next-update before this one, as it it involves less risk.
  • This will only require some minor changes on your package.json file.
  • Bear in mind that this may introduce breaking changes, you should have proper tests to check they won’t be broken.

How to use npm-force-resolutions

  1. After the dependencies and devDependencies section, you must add the packages you want to patch. 
    1
    2
    3
    "resolutions": {
        "underscore": "^1.12.1"
    }
  2. You can automate its use with a generic preinstall script. 
    1
    2
    3
    scripts": {
        "preinstall": "npm install --package-lock-only --ignore-scripts && npx npm-force-resolutions"
    }

So, this will make package-lock.json gets get all dependencies pointing to the patched versions. However, this may introduce breaking changes, so be very careful using it.

In

Set up

  • Mock the method getFlag on myApp. 
    1
    spyOn(myApp, "getFlag");

Mocks

Return values

  • Simple

    1
    spyOn(myApp, "getFlag").and.returnValue(true);

  • Fake Functions

    1
    2
    3
    spyOn(myApp, "useFlagForSomething").and.callFake(function() {
        return "I'm replacing the real function with this";
    });

Verify code was called

  • Verify called

    1
    spyOn(myApp, "getFlag").and.callThrough();

  • Verify not called

    1
    expect(myApp.reallyImportantProcess).not.toHaveBeenCalled();

  • Assert on function call

    1
    expect(myApp.reallyImportantProcess).toHaveBeenCalledWith(123, "abc");

Async tests

  • Code example

    1
    2
    3
    4
    5
    6
    7
    8
    var flag = false;

    function testAsync(done) {
        // Wait two seconds, then set the flag to true
        setTimeout(function () {
            flag = true;
        }, 2000);
    }

  • Specs

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    describe("Testing async calls with beforeEach and passing callback", function () {
        beforeEach(function (done) {
            // Make an async call, passing the special done callback        
            testAsync(done);
        });

        it("Should be true if the async call has completed", function () {
            expect(flag).toEqual(true);
        });
    });

Async-await

  • Code example

    1
    2
    3
    4
    5
    6
    7
    function returnTrueAsync() {
        return new Promise(resolve => {
            setTimeout(() => {
            resolve(true);
            }, 1000);
        });
    }

  • Specs

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    describe("Testing async functions", () => {  
        it("Should work with async/await", async () => {
            // Arrange
            let flag = false;

            // Act
            flag = await returnTrueAsync();

            // Assert
            expect(flag).toBeTruthy();
        });
    });

In

Marking checkboxes

Issues steps

  1. Locate the element using its Xpath.
  2. Use the click method to click on it.
    • Selenium may not recognize it as clickable, so it raises and exception.

Solution

  1. Locate the element.
  2. Use an ActionChains object to move the cursor to the check box.
  3. Click on it.
1
2
3
4
5
check_box = driver.find_element_by_xpath('Xpath')

actions = webdriver.ActionChains(driver)
actions.move_to_element_with_offset(check_box, -5, 5).perform()
actions.click().perform()

Define the move_to_element_with_offset parameters

move_to_element_with_offset: moves the mouse by an offset of a specific element on the page, relative to its top-left corner.

  • Calculate the distance you want the cursor to move away from the top-left corner, in order to to have the cursor around the middle of the check box.
  • Find the distance for the move, execute the code with the size attribute of the element before running the complete code. 
    1
    2
    3
    4
    check_box = driver.find_element_by_xpath('Xpath')
    print(check_box.size)

    # output: {'height': 10, 'width': 10}

Handling frames

Issue

  1. Find an element on the webpage, but get errors.
  2. Retry using the Xpath, the class name… , but still getting errors.

Solution

Selenium may not find certain element since it is on another HTML frames. You need to switch to the correct frame before trying to interact with the page again.

  • If you know the name of the frame.

    1
    driver.switch_to.frame('mainIframe')

  • If you know how many frames are there on the page.

    1
    driver.switch_to.frame(0)

  • If you have no idea, find all the frames.

    1
    2
    3
    4
    frames = driver.find_elements_by_tag_name('iframe')
    print(len(frames))
    for frame in frames:
       print(frame.get_attribute('name'))

Switching tabs

Issue

  1. A button automatically opens a new tag
  2. You need to move to the other tab

Solution

Navigating through tabs is similar to navigating frames.

  • Rustic approach: iteration to a different tab

    1
    2
    3
    4
    5
    6
    current_tab = driver.current_window_handle

    all_tabs = driver.window_handles
    for tab in all_tabs:
       if tab!= current_tab:
          driver.switch_to.window(tab)

  • Elegant approach: keep rack of the order of the tabs you opened, keeping indexes.

    1
    driver.switch_to.window(all_tabs[i])

  • Hardcore approach: iteration through all tabs

    1
    2
    3
    all_tabs = driver.window_handles
    for tab in all_tabs:
       driver.switch_to.window(tab)

Traffic warnings

Be aware that you’re making considerably more requests to the website than usual. This is because for each link, you’re opening two or three new tabs.

  • Insert some random pauses in your code in order not to overload the server.
  • Take advantage of a proxy provider to make sure your code will keep running as long as there are pages left to scrape.

In

Required software

  • Postman (the GUI makes everything simpler)
  • NodeJS (recommended: latest LTS version)
  • Newman package, plus custom reporters (e.g. htmlextra) 
    1
    2
    3
    npm install -g newman
    # install some extra reporter if you like
    npm install -g newman-reporter-htmlextra

The Postman collection script

You will work with Postman as usual. The Postman collection runner is behind the Workspaces which require a Postman account. However, you may run it on Newman without any need for account.

Format

Item Format
Environment variables "{{MESSAGE}}"
Script variable data.message

Parameters on request data

  • You can have parameteres on Body -> Raw.

    1
    2
    3
    {
        "text": "{{MESSAGE}}"
    }

  • We are getting that parameter from an environment variable. We configure that on a request script, in order to redirect it to the external data file.

    1
    pm.environment.set("MESSAGE", data.message);

Parameters on test scripts

  • We can pass the parameters directly to this code excerpts. Postman provides snippets to make this task faster. 
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    pm.test("Status code is 200", function () {
        pm.response.to.have.status(200);
    });
    pm.test("Check if the text was properly sent", function () {
        var jsonData = pm.response.json();
        pm.expect(jsonData.contents.text).to.eql(data.message);
    });
    pm.test("Check if translation was correct", function () {
        var jsonData = pm.response.json();
        pm.expect(jsonData.contents.translated).to.eql(data.expected_translation);
    });

Full exported code

  • Go to the collection menu, and select export. You will get a json file.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    {
        "info": {
            "_postman_id": "2d07d2fd-5350-4959-8f41-a1d091826933",
            "name": "Fun translations",
            "description": "A few funny translations, like talking as Yoda",
            "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
        },
        "item": [
            {
                "name": "Yoda",
                "event": [
                    {
                        "listen": "test",
                        "script": {
                            "exec": [
                                "pm.test(\"Status code is 200\", function () {\r",
                                "    pm.response.to.have.status(200);\r",
                                "});\r",
                                "pm.test(\"Check if the text was properly sent\", function () {\r",
                                "    var jsonData = pm.response.json();\r",
                                "    pm.expect(jsonData.contents.text).to.eql(data.message);\r",
                                "});\r",
                                "pm.test(\"Check if translation was correct\", function () {\r",
                                "    var jsonData = pm.response.json();\r",
                                "    pm.expect(jsonData.contents.translated).to.eql(data.expected_translation);\r",
                                "});"
                            ],
                            "type": "text/javascript"
                        }
                    },
                    {
                        "listen": "prerequest",
                        "script": {
                            "exec": [
                                "pm.environment.set(\"MESSAGE\", data.message);"
                            ],
                            "type": "text/javascript"
                        }
                    }
                ],
                "request": {
                    "method": "POST",
                    "header": [],
                    "body": {
                        "mode": "raw",
                        "raw": "{\r\n    \"text\": \"{{MESSAGE}}\"\r\n}\r\n",
                        "options": {
                            "raw": {
                                "language": "json"
                            }
                        }
                    },
                    "url": {
                        "raw": "https://api.funtranslations.com/translate/yoda",
                        "protocol": "https",
                        "host": [
                            "api",
                            "funtranslations",
                            "com"
                        ],
                        "path": [
                            "translate",
                            "yoda"
                        ]
                    },
                    "description": "Post query to talk like Yoda.\n- It needs to fill the body of the request"
                },
                "response": []
            }
        ]
    }

The external file

CSV

Format

  • The first line will contain the name of the fields.
  • The following lines wil contain the values of those fields for each iteration.

Full Code

1
2
3
message, expected_translation
"This is a message", "A message,  this is"
"This is a dummy", "A dummy,  this is"

Json

Format

  • It is represented as a list of Json objects.
  • Each object must have its attributes defined as key-value pairs.

Full Code

1
2
3
4
5
6
7
8
9
10
[
    {
        "message": "This is a message",
        "expected_translation": "A message,  this is"
    },
    {
        "message": "This is a dummy",
        "expected_translation": "A dummy,  this is"
    }
]

Run it

You can run the Newman command from the command line.

1
2
newman run FunTranslations.postman_collection.json -d data-translate.csv -r cli,htmlextra
newman run FunTranslations.postman_collection.json -d data-translate.json -r cli,htmlextra

In

Newman

Newman is a CLI tool for running Postman collections.

Setup

  • You need NodeJS (which includes the npm package manager) or NVM (Node Version Manager).

    1
    npm install -g newman

  • Check it was properly installed

    1
    newman -v

Run it

  • You may check it works properly by exporting a Postman Collection as a json file from the Postman App.

    1
    newman run examples/sample-collection.json

  • You may also run a remote json file.

    1
    newman run https://www.getpostman.com/collections/631643-f695cab7-6878-eb55-7943-ad88e1ccfd65-JsLv

Hints

HTML Reporters

  • Basic reporter

    1
    2
    npm install -g newman-reporter-html
    newman run examples/demo-collection.json -r html

  • Extended reporter

    1
    2
    npm install -g newman-reporter-htmlextra
    newman run examples/demo-collection.json -r htmlextra

Pre-request scripts

  • Pre-processing

    • setting variable values
    • parameters
    • headers
    • body data
    • pre-request scripts for debugging code (e.g. logging output) 
      1
      pm.collectionVariables.set('methodName', 'myMethodName')

  • Learn with example:

    • Calculate hashed version of ${developerId}${methodName}${FORMAT}${authKey}${timeStamp}

    • Python solution (and manually paste on postman)

      1
      2
      from hashlib import md5
      md5(string_to_hash)

    • Postman solution

      • pre-request script tab on request

        1
        2
        3
        4
        5
        6
        7
        8
        9
        10
        11
        12
        13
        14
        15
        const moment = require("moment");
        const timeStamp = moment.utc.().format('YYYYMMDDHHmmss');

        const devId = pm.environmentVariables.get('devId');

        const authKey = pm.environmentVariables.get('authKey');

        const methodName = pm.environmentVariables.get('methodName');

        // use cryptoJS
        const signature = CryptoJS.MD5(`${devId}${methodName}${authKey}${timeStamp}`).toString();

        // set the new environment variable
        postman.setEnvironmentVariable('signature', signature);
        postman.setEnvironmentVariable('timeStamp', timeStamp);

      • On call

        1
        {{URL}}/{{methodName}}{{FORMAT}}/{{devID}}/{{signature}}/{{session}}/{{timeStamp}}/{{LANGUAGE_CODE}}

Integrations

Newman on Jenkins

Obtain graphical reports

  1. Create job/project “Postman collection”

  2. Go to configure/build

  3. You may set the value of execute batch command:

    1
    2
    3
    # there should be no spaces on the reporters arguments
    newman run https://www.getpostman.com/collections/631643-f695cab7-6878-eb55-7943-ad88e1ccfd65-JsLv \
        --reporters cli,junit --reporter-junit-export “newman/myreport.xml”

  4. When the project is built, you may go to Workspace/newman and check the xml report.

Publish the report

  1. Go to Configure
  2. Go to post-build actions
  3. Select publish Junit test result report
  4. Enter the path of the file in Test Report XMLs and save the changes
  5. Click on Build Now to build the project again
  6. In the Build history section, click on the date and time to open up the build results
  7. Click on Test Result
  1. Build the project a couple of times to set a general trend as sometimes Jenkins does not show the report with one build.
  2. Refresh the page.
  3. You will see the generalized report on the dashboard as Test Result Trend.
0%