vcrpy breaks when using an authenticated proxy for an HTTPS request

[lagenar]

I'm trying to generate some tests cases for requests that need to use an http proxy and I get an HTTP 407 from the proxy server.

After inspecting what's being sent when using vcrpy I found that it doesn't perform the CONNECT to start tunneling to the https site. It appears to send some binary data instead. I assume it's skipping the process of performing the initial setup of the proxy request and sending the proxy authentication.

How to replicate:

1. Run netcat -l -p 4444
2. Run the following script

import vcr
import requests
import logging

with vcr.use_cassette('headers.yml'):
    requests.get('https://httpbin.org/ip', proxies={"https": "http://user:pass@localhost:4444"})

3. In netcat's output you'll see that there's no CONNECT command or proxy authorization

If you run the same request without using vcrpy the CONNECT and proxy authorization are sent correctly.

[kgraves]

There are quite a few other issues discussing proxy support. I believe #214 is where most people are being directed.

[neozenith]

A lot of changes have happened to VCRpy since this ticket was opened. As this ticket has become stale and we have a lot of old tickets that need to be groomed, I'm closing this for now to make it easier to see what is still relevant.

However if it is still needed, please feel free to re-open or create a new ticket.

Thanks! 🙏

[philiptzou]

I would like to point out that #809 actually resolved this issue finally. Although it's not yet get merged, you can write a simple monkeypatch script to update your vcrpy package:

from vcr.stubs import VCRConnection, log


class VCRConnectionMP:
    """
    Below methods are copied from vcrpy#809
    """

    def _port_postfix(self):
        """
        Copy the method code from #809 diff
        """
        port = self.real_connection.port
        port = (
            self.real_connection.port
            if not self.real_connection._tunnel_host
            else self.real_connection._tunnel_port
        )
        default_port = {"https": 443, "http": 80}[self._protocol]
        return f":{port}" if port != default_port else ""

    def _real_host(self):
        """Returns the request host"""
        if self.real_connection._tunnel_host:
            # The real connection is to an HTTPS proxy
            return self.real_connection._tunnel_host
        else:
            return self.real_connection.host

    def _uri(self, url):
        """Returns request absolute URI"""
        if url and not url.startswith("/"):
            # Then this must be a proxy request.
            return url
        uri = f"{self._protocol}://{self.real_connection.host}{self._port_postfix()}{url}"
        uri = f"{self._protocol}://{self._real_host()}{self._port_postfix()}{url}"
        )
        log.debug("Absolute URI: %s", uri)
        return uri

    def _url(self, uri):
        """Returns request selector url from absolute URI"""
        prefix = f"{self._protocol}://{self.real_connection.host}{self._port_postfix()}"
        prefix = f"{self._protocol}://{self._real_host()}{self._port_postfix()}"
        return uri.replace(prefix, "", 1)


def patch():
    """
    Patch vcrpy with HTTPS proxy handling by PR809
    """
    VCRConnection._port_postfix = VCRConnectionMP._port_postfix
    VCRConnection._real_host = VCRConnectionMP._real_host
    VCRConnection._uri = VCRConnectionMP._uri
    VCRConnection._url = VCRConnectionMP._url

Then just patch() the vcrpy in the very beginning in your code. For example, I uses pytest, I patched it from conftest.py.