Black box contract testing - third party API mocking

What if our application interacts with a third party API (eg. Stripe, Sendgrid, etc). Our black box tests run against a real running application, albeit with an in-memory database, but we don't want it to making real API calls - we likely want to the mock the behaviour of our third party API so our tests can run in a deterministic manner.

Updating our application to include a third party application.

Let's update our application to include some kind of third party interaction.

Our silly, arbitrary, (and most importantly, easy for me to implement) third party interaction will be with the JSON Placeholder API.

What we'll do is get the list of users, and when creating a pet, if the pet's name is included in the list of usernames provided by JSON Placeholder, we'll reject the request.

Here is our commit that adds the third party interaction.

And here we have a test that demonstrates this functionality by simply using the third party.

We don't want to use a real third party application.

There are a lot of reasons we don't want to use a third party application.

• We might get rate limited / we might annoy the third party if we make too many calls from our tests.
• We don't control the third party, we can't guarantee that it's always going to return the same results.
• In some network conditions we might not be able to access the third party, if our test environment can't access the wider environment for example.
• Real network calls could be slow.

There are several tools that I could use the achieve API mocking, some that allow for self hosting are:

• Mockbin
• MITM-proxy
• HTTPBin

Mockbin solution

I'm going to try Mockbin - the reason being because it supports defining responses using HAR files. Where possible I prefer using standardised specs to define data/contracts, because that tends to allow for tool interoperability. (I will note that, according to the Wikipedia article, W3C abandoned the spec, so 😕).

Mockbin does require also running a Redis container, which is one more container than I'd like, (HTTPBin runs a single docker container), but let's do it.

I first manually start Mockbin running, as per their docs ^(permalink), just check that it's going to do what I want.

There does seem to currently be an issue with the Mockbin docker container, as per this GitHub issue, code snippet below reflects using the alternative image.

docker run -d --name mockbin_redis redis
docker run -d -p 8080:8080 --link mockbin_redis:redis brianlow/mockbin

I can then make some requests against the Mockbin instance using Postman.

• Create bin
• Make a request against the bin
• See a log of the requests

So now lets do this programmatically.

Step 1. Create a har file

The first thing is, I need a HAR file for that endpoint, so for this case, let's generate from the browser.

1. Open browser dev tools -> Network tab
2. Navigate to https://jsonplaceholder.typicode.com/users
3. Click 'disable cache'
   • ‼️ This is an important step. Otherwise you will get a cached result (304), and client behaviour is to not stream that data.
4. Refresh page
5. Click the download icon to save the HAR file.

For many third party APIs generating HAR files from the browser will probably be a bit cumbersome, but let's cross that bridge later.

Our HAR file is here.

‼️ Issue with response headers - Above HAR doesn't work

The above HAR file doesn't work with Mockbin. It runs into some kind of decompression issue, see the following issue:

To resolve it, I manually munged the HAR file until it works - we remove the headers and set headerSize to 0. The file I use is here, this is actually just the response part we need, it's not a complete HAR file.

Possibly removing the headers is going to be problematic for your solution - we might need to identify which specific header is causing the trouble.

Step 2. Start Mockbin and configure - (manual)

Configuring mockbin is pretty straightforward.

1. Start a redis docker image.
2. Start a mockbin docker image.
3. Configure the behaviour mockbin by POSTing a request to /bin/create.

Let's do this manually now:

Start Redis

 run -d --name mockbin_redis redis

Start mockbin

 run -d --name mockbin -p 8081:8080 --link mockbin_redis:redis brianlow/mockbin

Configure mockbin

Nb. assumes the har file is on the file system.

curl -X POST \
  -H "Content-Type: application/json" \
  -d @./api-test-runner-jest/harfiles/jsonplaceholder.har.json \
  http://localhost:8081/bin/create

This returns us a bin id like:

"6ce92e34-6a60-4553-b551-a4d3a7473102"

Test that the mocked response is happening properly

curl http://localhost:8081/bin/6ce92e34-6a60-4553-b551-a4d3a7473102

And we get our response body as defined in the HAR file.

Step 2 - Start mockbin and configure (programmatic)

So now we're going to follow these steps, but make these programmatic steps that happen as part of running our tests.

The steps will be:

Before all tests:

1. Start the redis container
2. Start the mockbin container
3. Configure mockbin and get the response
4. As a sanity test, check that the mockbin is returning the expected response
5. Start the backend, configured to use the mockbin endpoint
6. Run our tests

After the tests

1. Kill mockbin
2. Kill redis

Before each test:

1. Start the server

After each test:

1. Kill the server

(We want to clear the server state between test runs.)

We do all of this here.

We write a test for it here.

I'm not going to paste the code here. The code snippets are well commented, and I'd basically just be copy pasting them here. (As an aside, I would absolutely love some kind of library or plugin that creates code snippets from Github permalinks, just it does if you paste them as a Github comment, for example see here.)

Brilliant!

What I like and don't like about this

I like that this runs, technically not a single process, but in a single terminal, and cleans up after it self. That is, I don't have to open separate terminal windows, or run separate commands before running my tests.

But, as clever as I feel doing this, I suspect it's a bad idea.

For one, the five or so second wait for Mockbin to be ready wouldn't be scalable for multiple test suits. Possibly we could configure it in Jest's globalSetup.

It also has your node script assuming things about the environment (eg. Docker existing), perhaps it is just tidier to have the 'Start mockbin running' part of the process running as a shell script.

That said, the actual configuration of how should Mockbin respond (ie. what HAR file to use) I think does belong to the test - because that's where we are going to be making assertions about the behaviour. It's an awful experience writing tests that depend on behaviour of some magic configuration elsewhere.