Introduction
Testing code that interacts with external services can be tricky. You need to make sure that your code is handling all the possible responses correctly, and that it's not making too many unnecessary requests. But how do you test this kind of code reliably and efficiently?
That's where VCR.py comes in. The idea behind VCR.py is simple: it records HTTP interactions made by your code, and then replays them when your tests run. That way, your tests can simulate HTTP requests without actually hitting the external service. Pretty neat, right?
In this article, we'll take a closer look at VCR.py and some of its features. We'll cover how to use VCR.py to record and replay HTTP interactions, how to customize its behavior with matchers and filters, and how to work with the cassette file that VCR.py uses to store recorded interactions.
Getting Started
To get started with VCR.py, you'll need to install it first. You can do this using pip:
pip install vcrpy
Once you have VCR.py installed, you can start using it in your tests. Here's a simple example:
import requests
import vcr
with vcr.use_cassette('test_api.yaml', record_mode='once'):
def test_api():
response = requests.get('https://api.example.com')
assert response.status_code == 200
In this example, we're using VCR.py to intercept a GET request to https://api.example.com
. The first time this test is run, VCR.py will record the HTTP interaction in a cassette file named test_api.yaml
. On subsequent runs, VCR.py will replay the recorded interaction from the cassette file instead of making a new request.
Record_mode that passed in with the VCR cassette determines how the interactions with an external API are recorded. There are four record modes available:
once
: This mode will record the interactions with the external API the first time the test is run. Subsequent runs of the test will use the recorded interactions instead of making new requests to the API.new_episodes
: This mode will record any interactions with the external API that haven't been previously recorded. If an interaction has already been recorded, it will be replayed instead of making a new request.all
: This mode will record all interactions with the external API, regardless of whether they have been previously recorded or not. This is useful for building up a complete set of recordings for an API.none
: This mode will turn off recording altogether and will only replay previously recorded interactions. This mode is useful when you want to ensure that your tests are only using recorded interactions and not making any new requests to the external API.
In the previous example, the record_mode
argument is set to 'once'
, which means that the interactions with the external API will be recorded the first time the test is run and subsequently replayed on all subsequent runs.
Working with the Cassette File
The cassette file is where VCR.py stores recorded HTTP interactions. By default, the cassette file is a YAML file, but you can also use JSON or SQLite. The cassette file can be easily read and edited, which makes it easy to inspect the contents of the file and make changes if necessary.
Here's an example of a cassette file:
- request:
body: !!binary |
eyJ0ZXN0IjoidGVzdCJ9
headers:
Content-Type: application/json
method: POST
uri: https://api.example.com/foo
response:
body: !!binary |
{"id": "123", "foo": "bar"}
headers:
Content-Type: application/json
status:
code: 200
message: OK
In this example, we have a single interaction that was recorded. The interaction consists of a POST request to https://api.example.com/foo
with a JSON request body, and a response with a JSON body containing an ID and a foo value.
One important thing to keep in mind when working with the cassette file is that it can become quite large if you have a lot of interactions recorded. To help with this, VCR.py provides several options for controlling how interactions are recorded and stored in the cassette file.
Matchers
By default, VCR.py matches HTTP interactions based on the HTTP method, URL, and request body. But what if you want to match on other criteria, like headers or query parameters?
That's where matchers come in. Matchers allow you to specify custom criteria for matching HTTP interactions. Here's an example:
import vcr
def custom_matcher(request1, request2):
# Your custom matching logic here
return True
with vcr.use_cassette('my_test.yaml',
match_on=['method', 'uri', custom_matcher]):
def test_my_function():
# Code that makes HTTP requests
In this example, we define a custom matcher function custom_matcher
that takes two requests and returns True
if they match based on our custom criteria. We then use this custom matcher in our use_cassette
call, along with the default matchers for HTTP method and URI.
Filters
Sometimes you might want to filter or modify requests or responses before they are recorded or played back. For example, you might want to filter out sensitive data from the response before it gets stored in the cassette file. Or you might want to modify the request headers before the request is sent.
VCR.py provides two options for filtering: before_record
and before_playback
. These options allow you to specify functions that are called before requests are recorded or played back, respectively. Here's an example:
import vcr
def filter_request(request):
request.headers['Authorization'] = 'Bearer xxxxxxx'
return request
def filter_response(response):
response.headers.pop('Set-Cookie', None)
return response
with vcr.use_cassette('my_test.yaml',
before_record=filter_request,
before_playback=filter_response):
# Your test code here
In this example, we define two filter functions filter_request
and filter_response
that modify the request and response respectively. We then pass these filter functions to the before_record
and before_playback
options of use_cassette
, respectively.
Placeholders
Sometimes you might have dynamic data in your responses that you don't want to hard-code in your tests. For example, if you have a test that creates a new resource and returns a unique ID, you might want to use a placeholder to represent that ID in the cassette file.
VCR.py supports placeholders in the cassette file. Placeholders are strings that are replaced with actual values during playback. Here's an example:
- request:
body: !!binary |
eyJ0ZXN0IjoidGVzdCJ9
headers:
Content-Type: application/json
method: POST
uri: https://api.example.com/foo
response:
body: !!binary |
{"id": "{{ ID }}", "foo": "bar"}
headers:
Content-Type: application/json
status:
code: 200
message: OK
In this example, we're using the {{ ID }}
placeholder in the response body to represent the unique ID that is generated by the API. When VCR.py replays the response, it will substitute the placeholder with the actual ID value that was returned by the API during the recording.
Basic Illustration
Let's assume that we have a simple Flask web application that acts as an external web service API. It returns a JSON response when we make a GET request to the /hello
endpoint:
from flask import Flask, jsonify
app = Flask(__name__)
@app.route('/hello')
def hello():
return jsonify({'message': 'Hello, World!'})
if __name__ == '__main__':
app.run()
Now, let's write a vcrpy test to test this API. We'll use the requests
library to make the HTTP request to the API:
import vcr
import requests
def test_hello_api():
with vcr.use_cassette('hello_api.yaml', record_mode='once'):
response = requests.get('http://localhost:5000/hello')
assert response.status_code == 200
assert response.json() == {'message': 'Hello, World!'}
In this test, we're using the use_cassette
context manager to wrap the HTTP request to our Flask app. We're specifying that we want to use the cassette file hello_api.yaml
to record the interactions with the API, and we're using the record_mode
argument to specify that we only want to record the interactions once (i.e., on the first run of the test).
When we run this test for the first time, vcrpy will record the interaction with the Flask app and save it to the hello_api.yaml
file. On subsequent runs of the test, vcrpy will replay the interaction from the cassette file instead of making a new HTTP request to the Flask app.
To run this test, we can simply execute the test file using a test runner such as pytest:
$ pytest test_hello_api.py
This will run the test and output the results of the assertions. If the test passes, we should see output similar to the following:
============================ test session starts ============================
collected 1 item
test_hello_api.py . [100%]
============================= 1 passed in 0.13s =============================
If the test fails, we'll see an error message indicating which assertion failed.
here's what the hello_api.yaml
cassette file might look like after vcrpy records the HTTP interaction with the Flask app (ie. our external API):
interactions:
- request:
body: null
headers:
Accept-Encoding: identity
Connection: keep-alive
Host: localhost:5000
User-Agent: python-requests/2.25.1
method: GET
uri: http://localhost:5000/hello
response:
body:
encoding: utf-8
string: '{"message": "Hello, World!"}'
headers:
Connection: Keep-Alive
Content-Length: '25'
Content-Type: application/json
Date: Mon, 05 Jul 2023 12:34:56 GMT
Keep-Alive: timeout=5, max=100
Server: Werkzeug/2.0.2 Python/3.9.5
status:
code: 200
message: OK
recorded_at: 2023-07-05T12:34:56.789012Z
recorded_with: vcrpy/4.1.1
This file contains a YAML representation of the HTTP interaction between the test and the Flask app. The interactions
section contains a list of all the HTTP interactions that were recorded, in this case just one. The request
and response
sections contain the request and response headers and bodies, respectively. The recorded_at
and recorded_with
fields indicate when and with what version of vcrpy the interaction was recorded.
Conclusion
VCR.py is a powerful tool for testing code that interacts with external services. Its ability to record and replay HTTP interactions makes it easy to write tests that are reliable and efficient. With VCR.py, you can customize how interactions are matched and filtered, and work with the cassette file to inspect and modify recorded interactions.
I hope this article has given you a good introduction to VCR.py and its features. If you're interested in learning more, I encourage you to check out the official documentation and start experimenting with VCR.py in your own tests.
Top comments (0)