Load Testing Your API with Postman

In this article, I'll explain how to use a Postman collection I have created to load test our instance of our test API. The process is pretty straightforward, as is shown below. You need to feed your exported Postman collection to our postman-to-k6 converter, and use the generated k6 script to load test your own API.

# convert postman collection to k6 test
postman-to-k6 test-api.json -e env.json -o k6-script.js

# run load test
k6 run --vus 100 --duration 5m k6-script.js

Our Test API & Its Testing Scenario

In order to demonstrate the power of k6 in different scenarios, we have created our test API with various example endpoints, which is available at test-api.k6.io. These endpoints are available in the Postman collection:

Public APIs

• List all public crocodiles
• Get a single public crocodile

Registration and authentication

• Register a new user
• Bearer/JWT token authentication

Private APIs

• List all your crocodiles
• Get a single crocodile
• Create a new crocodile (max 100)
• Update your crocodile
• Update selected fields on your crocodile
• Remove your crocodile

The scenario is to test all the public and private APIs. For the private APIs, a user is created and its token is extracted. The extracted token is used to make other API calls. The order is very important in the private APIs, since you cannot delete a non-existing resource, for example. By the way, crocodile is our mascot.

Our Test API Postman Collection

To ease testing of our test API and demonstrate the usage of our Postman to k6 converter, I've created a Postman collection with almost all of our test API requests. You'll see how you can access this Postman collection shortly.

This collection includes a set of collection variables, environment variables, pre-scripts, tests, authorization with two different mechanisms, and usages of the Postman Sandbox API.

Load Testing Our Test API with The Postman Collection

We have created a tool that converts your Postman collection to k6 script, which is called postman-to-k6. You can read more about its features in its release notes.

In order to convert your Postman collection to k6 script, you should take the following steps:

1. Optional: Clone the repository and skip to the step 5 :

I've created a repository for this article that contains the exported Postman collection, along with the converted script and related files. You can clone the repository and import the test-api.json and env.json files into the Postman application and possibly play with them if you want.

This repository contains everything that is needed for load testing our test API, so you can skip to step 4. When using your own collection, you should take all the steps to be able to have a k6 script out of your Postman collection, and to be able to run your load test with it.

$ git clone https://github.com/k6io/example-postman-collection.git

2. Install Node.js (if you haven't already done so):

For this, I highly recommend that you use something like nvm, which is a Node.js version manager that you can use to have multiple version of Node.js at the same time on your machine and be able to switch to any of them quickly.

3. Install the postman-to-k6 tool :

The postman-to-k6 tool is developed to help you convert the requests inside your Postman collections to k6 scripts, which are actually JavaScript code.

$ npm install -g postman-to-k6

4. Convert your exported Postman collection to k6 script :

Assuming your exported collection is named test-api.json, you can run this command to convert it to a k6 script. The env.json includes all your environment variables that are exported from Postman.

$ postman-to-k6 test-api.json -e env.json -o k6-script.js

If you need more fine-tuning of your test (like we did above), like adding data or changing environment variables inside your code, just take a look at the Options section of the postman-to-k6 README.

The script generated by the converter should look like below. As you see, I've manually added the duration (of the test run) to be 1 minute and also added the virtual users (VU) count. These two options let the script run for a minute with 100 virtual users. These 100 VUs try to make as many requests as they can to test the server, which you'll see in the next screenshot.

import "./libs/shim/core.js";
import "./libs/shim/urijs.js";
import URI from "./libs/urijs.js";
import {
  group
} from "k6";

export let options = {
  maxRedirects: 4,
  duration: "1m",
  vus: 100
};

const Request = Symbol.for("request");
postman[Symbol.for("initial")]({
    options,
    collection: {
        BASE_URL: "https://test-api.k6.io/"
    },
    environment: {
        USERNAME: "test@example.com",
        PASSWORD: "superCroc2020",
        FIRSTNAME: "John",
        LASTNAME: "Doe",
        EMAIL: "test@example.com",
        ACCESS: null,
        REFRESH: null,
        CROCID: null
    }
});

export default function () {
    group("Public APIs", function () {
        postman[Request]({
            name: "List all public crocodiles",
            id: "3ddd46c4-1618-4883-82ff-1b1e3a5f1091",
            method: "GET",
            address: "{{BASE_URL}}/public/crocodiles/"
        });

        postman[Request]({
            name: "Get a single public crocodile",
            id: "9625f17a-b739-4f91-af99-fba1d898953b",
            method: "GET",
            address: "{{BASE_URL}}/public/crocodiles/1/"
        });
    });

    // NOTE: The rest of the requests can be accessed
    // from the repository in step 1
});

The generated script is a little bit different from normal k6 scripts, since it includes various abstractions to support different Postman functionality, but you can mix them with regular http requests from k6. Also, there is a libs directory beside the script that includes shims and libraries needed for the Postman scripts to work correctly.

5. Install k6 :

k6 supports various platforms, including Windows, Linux, macOS and docker. In order to install it, just grab an Windows installer or a docker image and run it on your machine. On Linux distributions, you can use APT or YUM, and on macOS, you can use Homebrew.

NOTE: Regarding installation on Windows, you can also use choco k6 package.

6. Run k6 with the generated script :

Now that you have your collections converted into a k6 script, you can invoke k6 like this:

$ k6 run k6-script.js

The result of running the script is shown in the following console output:

          /\      |‾‾|  /‾‾/  /‾/
     /\  /  \     |  |_/  /  / /
    /  \/    \    |      |  /  ‾‾\
   /          \   |  |‾\  \ | (_) |
  / __________ \  |__|  \__\ \___/ .io

  execution: local
     output: -
     script: k6-script.js

    duration: 1m0s, iterations: -
         vus: 100,  max: 100

    done [==========================================================] 1m0s / 1m0s

    █ Public APIs

    █ Registration and authentication

    █ Private APIs

    data_received..............: 8.8 MB 146 kB/s
    data_sent..................: 4.8 MB 80 kB/s
    group_duration.............: avg=753.07ms min=239.15ms med=495ms    max=4.06s    p(90)=1.37s    p(95)=1.73s
    http_req_blocked...........: avg=12.31ms  min=362ns    med=1.52µs   max=3.47s    p(90)=1.83µs   p(95)=1.96µs
    http_req_connecting........: avg=1.95ms   min=0s       med=0s       max=779.59ms p(90)=0s       p(95)=0s
    http_req_duration..........: avg=211.11ms min=104.42ms med=183.12ms max=924.43ms p(90)=304.25ms p(95)=404.24ms
    http_req_receiving.........: avg=1ms      min=41.14µs  med=169.38µs max=130.94ms p(90)=328.31µs p(95)=2.22ms
    http_req_sending...........: avg=205.91µs min=38.06µs  med=163.76µs max=113.06ms p(90)=258.45µs p(95)=302.86µs
    http_req_tls_handshaking...: avg=8.69ms   min=0s       med=0s       max=2.43s    p(90)=0s       p(95)=0s
    http_req_waiting...........: avg=209.9ms  min=104.05ms med=182.22ms max=891.77ms p(90)=301.29ms p(95)=402.41ms
    http_reqs..................: 26363  439.382653/s
    iteration_duration.........: avg=2.28s    min=1.43s    med=2.01s    max=6.55s    p(90)=2.86s    p(95)=3.64s
    iterations.................: 2588   43.133267/s
    vus........................: 100    min=100 max=100
    vus_max....................: 100    min=100 max=100

Remarks about using the postman-to-k6 converter

1️. Should we base our load tests on the Postman converter and our Postman collections?

If you're using the converter as a way of onboarding, no. If you expect to convert your collection continuously and without doing a lot of manual edits afterwards, yes.

We recommend you to use the converter as an easy way to onboard and then rewrite your scripts to idiomatic k6 code, as we believe it to be more maintainable and less likely to degrade over time. If you convert from postman collections continuously, however, and run the script output as-is, it might make sense to keep it as is.

2. Is everything available out of the box in the converted script?

No. Since k6 uses Goja to run JavaScript, and it is not compatible with browsers' and Node.js APIs, hence there are some missing functionality. This can be fixed by importing bundled JavaScript modules. For a list of compatible libraries, please see jslib.k6.io.

3. What adjustments did you make to the script to make it work?

First, I removed the pre-script containing pm.sendRequest, because it is not supported by the converter. Then, I replaced the jsonData.hasOwnProperty syntax with the equivalent k6 syntax for extracting JSON response information: response.json("selector").

Postman API vs. k6 API

Here's a quick comparison of the Postman API versus the k6 API. To be fair, I have included features from Postman GUI application. Since k6 is scriptable from the start, you have the option to write the logic in JavaScript. Postman also supports javascript to do various tasks, but the focus is on exposing features via a richer set of GUI elements.

As you saw above, there are many features supported by each API, each to some extent. Some features needs external libraries, some are internal. Both APIs are scriptable in JavaScript, and not everything is supported by both, due to the various browser and Node.js APIs used in the libraries.

Yet, there are some features only available on k6, which is partially due to the fact that the Postman is catered for API testing or API functional testing, but k6 is focused more on API load testing.

Functional testing vs. load testing

Functional testing concerns with giving input to the system (as a black-box) via an API and examining the results, while load testing is basically doing the same thing as functional testing, but with additional load on the input to the system.

Functional testing provides input on each endpoint, and the returned results are verified in terms of correctness against a set of specifications. In turn, load testing provides a huge amount of load on each endpoint, and rather tries to aggregate the metadata returned by all the responses.

Load testing metrics for measuring performance

Concerning the measurements, the metadata will include the time it took for the request to settle and the response to return, which are measure by various metrics. For example you can measure the HTTP request duration of all requests and get their minimum, maximum, average, median, 90th and 95th percentiles.

Pass/fail a test with thresholds

You also have the option to pass/fail a test if it does/doesn't reach certain threshold(s). For example, you can specify that you want the average response time to be less than 500ms. If the average is below that, the test will fail, much like asserts in software testing.

Filter results with tags

Since you're dealing with lots of different results from different endpoints, your life would be easier if you could filter the results. Tags are supported by k6 to fulfill this requirement.

Load testing WebSocket servers

In terms of protocol implementation, WebSocket is one of the features available only in k6, compared to Postman, and you can load test your WebSocket server with it.

Conclusion

In this article I've tried to give a quick introduction to Postman, the postman-to-k6 converter and our k6 load testing tool. All these tools combined can help you turn your API requests in Postman into k6 script in order to load test your API. Many of the Postman features are supported by the postman-to-k6 tool.

Our ultimate goal is to streamline the process of onboarding you to our load testing tool, k6. In doing so, we have created a bunch of tools that can help you easily integrate load testing in your infrastructure.