Postman API Tests

Editing the Utopia Pipeline Postman Tests


  1. In Postman, export the collection and replace the file located at C:\Projects\Utopia\Utopia\Postman\Tests\Regression Test and create a pull request and get the necessary approvals.
  2. In Postman, export the environment variable (if any changes were made, if not, no need to re-upload it) to later upload it to the Azure Pipeline Library.
    *Once the new branch comes through and is approved, you'll want to then replace the environment file if needed following the steps below.  If not, skip to step #3.
    • (Currently Royce and Quinn have this permission.  If this changes, please update this document)  Go to Azure DevOps Utopia Library > Secure Files (https://dev.azure.com/eFileCabinet/Utopia/_library?itemType=SecureFiles)
    • Find the Utopia Regression Tests - Staging.postman_environment.json secure file and delete it.  Then upload the exported file with the same name.
    • Edit the pipeline and edit the "Run Postman Tests" job.
    • Reselect Utopia Regression Tests - Staging.postman_environment.json in the "Secure File" field and save.
    • If you need to rerun the tests that failed due to an error, You will want to edit the Release instead and then redeploy it. 
  3. Once a Utopia Release is deployed, the pipeline will then run the postman tests automatically when Staging is released.  Check for any errors in the pipeline and if there are any, you can see which ones failed by reading the report: https://internal.revverdocs.com/books/qa/page/reading-the-error-log-in-the-postman-auto-tests-in-the-pipeline


Reading the Error Log in the Postman Auto-Tests in the Pipeline

  1. Click on the Run Postman Tests job.
  2. Click on the Deployment Failed task

    image.png

  3. Click to open the Azure CLI step.

    image.png
    4. Scroll toward the bottom of the log and you'll see a Blob link for the Utopia Regression Tests.  Hover over it, and a toast message will appear with a "Follow Link" option.  Click it.

     

    5.  A Newman Run Dashboard page will appear with the results of each test.  Click on the "Failed Tests" tab and then click on each item to see the error as to why.  You may need to run these tests locally again to see why they are failing exactly and see if it is an issue with the newly released code or your tests/setup.

    image.png