Custom VCR matchers for dealing with mutable HTTP-requests

VCR is a powerful beast which makes HTTP-request mocking a real no-brainer. If you haven’t tried it before, just do it, it will make your life much easier.VCR works in really simple manner – when you issue request via some HTTP API, at first it records them and on second request reuse information stored from the first request. The tricky part of such behavior – how to match new request to the one stored before. Usually it just works, but when you have some mutable or random part in an URL it becomes a little bit harder.In this article you will learn how VCR request-matching works and how to customise matching strategy to deal with mutable URI.

Predefined strategies

VCR has next predefined matchers:

:method – HTTP method (i.e. GET, POST, PUT or DELETE) of the request
:uri – full URI of the request
:host – host of the URI(without port)
:path – path of the URI
:query – query string values of the URI
:body – body of the request(PUT or POST methods)
:headers – request headers

You can combine any of these methods to obtain required behavior. By default VCR matches request using :method and :uri. For plane GET request it’s enough.Strict :uri matching can be substituted with combination of [:host, :path] in case you have some dynamic query part, which doesn’t important for you(e.g. ?timestamp=123). Combination of [:host, :query] can be used if you have dynamic path in URI. But don’t use such combinations, there are much better options, which will be described later.Matcher for :body is handy for stricter matching of POST/PUT requests, and matcher on :headers when HTTP-headers convey some important data.

Variable query part

The most common case for mutable URI – is some timestamp or another random data in query-part of request. VCR provides special helper for such cases. Use:

# for ignoring one parameter
VCR.request_matchers.uri_without_param(:timestamp)

# for ignoring several parameters
VCR.request_matchers.uri_without_params(:timestamp, :session)

Using this helper VCR matches on the whole URI, but excludes :timestamp(and :session) from matching in query-part of URI. Here is an example of usage in RSpec:

describe "#fetch_info", vcr: {match_requests_on: [:method,                  
                              VCR.request_matchers.uri_without_param(:timestamp)]} do
  # ...
end

Variable path part

Sometimes, especially for REST-API, where you dynamically generate user_id in test and do request to remote server, request path can vary on ID. To skip matching on variable ID, you can write custom URI-matcher. In fact, this way can be used(and should be) when you can’t assemble required matching strategy from standard VCR-matchers.Custom matcher can be any Proc-object which accepts two VCR::Request objects. Here is an implementation for trailing ID URI:

# URI example: 
# http://example.com/users/123
# matches to
# http://example.com/users/124
uri_ignoring_trailing_id = lambda do |request_1, request_2|
  uri1, uri2 = request_1.uri, request_2.uri
  regexp_trail_id = %r(/\d+/?\z)
  if uri1.match(regexp_trail_id)
    r1_without_id = uri1.gsub(regexp_trail_id, "")
    r2_without_id = uri2.gsub(regexp_trail_id, "")
    uri1.match(regexp_trail_id) && uri2.match(regexp_trail_id) && r1_without_id == r2_without_id
  else
    uri1 == uri2
  end
end

describe "#upgrade", vcr: {match_requests_on: [:method, uri_ignoring_trailing_id]} do
  #...
end

Notice, we wrote matcher like variable, not like symbol.Request-object has next API:

#method – symbol(:put, :post, etc)
#uri – string, full non-parsed URI
#body – string, POST/PUT request body
#headers – Hash, all request headers
#parsed_uri – URI-object(with scheme, host, port, path, and query accessors)

This provides all means to write any custom matching strategy. VCR doesn’t have :port matcher and it can be easily implemented via “parsed_uri”-method:

port = lambda do |r1, r2|
  r1.parsed_uri.port == r2.parsed_uri.port
end

describe "#remote_call", vcr: {match_requests_on: [:method, :host, port]} do
  #...
end

Variable response

Custom matchers work great if response body isn’t parametrized by request information. If this is a case, you should leverage VCR callbacks and it can be a subject for the whole new article. But there is a good example of such usage at https://github.com/nbibler/vcr-284. It can hint you with required approach.Anyway, if stuff becomes too complex, it’s much easier to directly use webmock, fakeweb or any other mocking framework which is used by your configuration of VCR.

Move custom matchers into configuration

In case, you want to reuse custom matcher, it can be put in RSpec configuration:

VCR.configure do |c|
  c.register_request_matcher :uri_without_timestamp, &VCR.request_matchers.uri_without_param(:timestamp)

  c.register_request_matcher :port do |r1, r2|
    r1.parsed_uri.port == r2.parsed_uri.port
  end

  c.register_request_matcher :uri_ignoring_trailing_id do |request_1, request_2|
    uri1, uri2 = request_1.uri, request_2.uri
    regexp_trail_id = %r(/\d+/?\z)
    if uri1.match(regexp_trail_id)
      r1_without_id = uri1.gsub(regexp_trail_id, "")
      r2_without_id = uri2.gsub(regexp_trail_id, "")
      uri1.match(regexp_trail_id) && uri2.match(regexp_trail_id) && r1_without_id == r2_without_id
    else
      uri1 == uri2
    end
  end
end

Now you can use :uri_without_timestamp, :port, and :uri_ignoring_trailing_id in tests:

describe "#fetch_info", vcr: {match_requests_on: [:method, :uri_without_timestamp]} do
  #...
end

describe "#remote_call", vcr: {match_requests_on: [:method, :host, :port]} do
  #...
end

describe "#upgrade", vcr: {match_requests_on: [:method, :uri_ignoring_trailing_id]} do
  #...
end

Debug

Sometimes, things go wrong and you need a way to sneak into matching process. Drop this code into any test and it enables debug-mode:

VCR.configure do |c|
  c.debug_logger = STDOUT
end

Now you are fully armed and ready to use VCR at its fullest.