如何通过袜子代理使python请求工作


How to make python Requests work via socks proxy

我在Python脚本中使用了很棒的Requests库：

1
2
3
import requests
r = requests.get("some-site.com")
print r.text

我想用袜子代理。 但是请求现在仅支持HTTP代理。

我怎样才能做到这一点？

现代方式：

1
pip install -U requests[socks]

然后

1
2
3
4
5
import requests

resp = requests.get('http://go.to',
                    proxies=dict(http='socks5://user:pass@host:port',
                                 https='socks5://user:pass@host:port'))
相关讨论

从2016年4月29日发行的requests版本2.10.0开始，requests支持SOCKS。

它需要PySocks，可以与pip install pysocks一起安装。

用法示例：

1
2
3
import requests
proxies = {'http':"socks5://myproxy:9191"}
requests.get('http://example.org', proxies=proxies)

相关讨论

如果有人尝试了所有这些较早的答案，但仍然遇到诸如以下的问题：

1
2
3
4
5
6
requests.exceptions.ConnectionError:
   SOCKSHTTPConnectionPool(host='myhost', port=80):
   Max retries exceeded with url: /my/path
   (Caused by NewConnectionError('<requests.packages.urllib3.contrib.socks.SOCKSConnection object at 0x106812bd0>:
   Failed to establish a new connection:
   [Errno 8] nodename nor servname provided, or not known',))

可能是因为默认情况下将requests配置为在连接的本地端解析DNS查询。

尝试将代理URL从socks5://proxyhost:1234更改为socks5h://proxyhost:1234。请注意额外的h(代表主机名解析)。

PySocks软件包模块的默认设置是进行远程解析，我不确定为什么请求使它们的集成变得晦涩难懂，但是我们到了。

相关讨论

您需要安装pysocks，我的版本是1.0，该代码对我有用：

1
2
3
4
5
6
7
8
9
import socket
import socks
import requests
ip='localhost' # change your proxy's ip
port = 0000 # change your proxy's port
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, ip, port)
socket.socket = socks.socksocket
url = u'http://ajax.googleapis.com/ajax/services/search/images?v=1.0&q=inurl%E8%A2%8B'
print(requests.get(url).text)
相关讨论

只要将python requests与SOCKS5 pull request合并，它将像使用proxies dictionary一样简单：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
#proxy
        # SOCKS5 proxy for HTTP/HTTPS
        proxies = {
            'http' :"socks5://myproxy:9191",
            'https' :"socks5://myproxy:9191"
        }

        #headers
        headers = {

        }

        url='http://icanhazip.com/'
        res = requests.get(url, headers=headers, proxies=proxies)

请参阅SOCKS代理支持

如果您不能等待request准备就绪而又无法使用requesocks时(例如由于缺少pwd内置模块而无法在GoogleAppEngine上使用)，则可以使用上面提到的PySocks。

  • 从仓库中获取socks.py文件，并将副本放入您的根文件夹中；
  • 添加import socks和import socket

    • 此时，在与urllib2一起使用之前，请配置并绑定套接字，在以下示例中：

    1
    2
    3
    4
    5
    6
    7
    import urllib2
    import socket
    import socks

    socks.set_default_proxy(socks.SOCKS5,"myprivateproxy.net",port=9050)
    socket.socket = socks.socksocket
    res=urllib2.urlopen(url).read()
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    # SOCKS5 proxy for HTTP/HTTPS
    proxiesDict = {
        'http' :"socks5://1.2.3.4:1080",
        'https' :"socks5://1.2.3.4:1080"
    }

    # SOCKS4 proxy for HTTP/HTTPS
    proxiesDict = {
        'http' :"socks4://1.2.3.4:1080",
        'https' :"socks4://1.2.3.4:1080"
    }

    # HTTP proxy for HTTP/HTTPS
    proxiesDict = {
        'http' :"1.2.3.4:1080",
        'https' :"1.2.3.4:1080"
    }
    相关讨论

    我在urllib3中安装了pysocks和猴子补丁的create_connection，如下所示：

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    import socks
    import socket
    socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS4,"127.0.0.1", 1080)

    def create_connection(address, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
                          source_address=None, socket_options=None):
       """Connect to *address* and return the socket object.

        Convenience function.  Connect to *address* (a 2-tuple ``(host,
        port)``) and return the socket object.  Passing the optional
        *timeout* parameter will set the timeout on the socket instance
        before attempting to connect.  If no *timeout* is supplied, the
        global default timeout setting returned by :func:`getdefaulttimeout`
        is used.  If *source_address* is set it must be a tuple of (host, port)
        for the socket to bind as a source address before making the connection.
        An host of '' or port 0 tells the OS to use the default.
       """

        host, port = address
        if host.startswith('['):
            host = host.strip('[]')
        err = None
        for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
            af, socktype, proto, canonname, sa = res
            sock = None
            try:
                sock = socks.socksocket(af, socktype, proto)

                # If provided, set socket level options before connecting.
                # This is the only addition urllib3 makes to this function.
                urllib3.util.connection._set_socket_options(sock, socket_options)

                if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
                    sock.settimeout(timeout)
                if source_address:
                    sock.bind(source_address)
                sock.connect(sa)
                return sock

            except socket.error as e:
                err = e
                if sock is not None:
                    sock.close()
                    sock = None

        if err is not None:
            raise err

        raise socket.error("getaddrinfo returns an empty list")

    # monkeypatch
    urllib3.util.connection.create_connection = create_connection

    我可以在Linux上执行此操作。

    1
    2
    3
    $ pip3 install --user 'requests[socks]'
    $ https_proxy=socks5://<hostname or ip>:<port> python3 -c \\
    > 'import requests;print(requests.get("https://httpbin.org/ip").text)'

    也许这可以帮助：

    https://github.com/kennethreitz/requests/pull/478

    相关讨论