关于python：Asyncio中的请求-关键字参数


Requests in Asyncio - Keyword Arguments

我正在将asyncio与请求模块一起使用，以发出异步HTTP请求。

我可以这样发出GET请求：

1
2
3
4
5
6
7
8
@asyncio.coroutine
def do_checks():
    loop = asyncio.get_event_loop()
    req = loop.run_in_executor(None, requests.get, 'https://api.github.com/user')
    resp = yield from req
    print(resp.status_code)
loop = asyncio.get_event_loop()
loop.run_until_complete(do_checks())

但是，我需要在请求中支持基本HTTP身份验证(在此进行描述)。

根据文档，url和auth都是request.get()的命名参数。

但是，如果我运行此命令(请注意url = \\'\\'和auth = \\'\\'的添加)：

1
2
3
4
5
6
7
8
@asyncio.coroutine
def do_checks():
    loop = asyncio.get_event_loop()
    req = loop.run_in_executor(None, requests.get, url='https://api.github.com/user', auth=HTTPBasicAuth('user', 'pass'))
    resp = yield from req
    print(resp.status_code)
loop = asyncio.get_event_loop()
loop.run_until_complete(do_checks())

我收到此错误：

1
TypeError: run_in_executor() got an unexpected keyword argument 'url'

在asyncio.run_in_executor()的原型中，支持其他参数：

1
BaseEventLoop.run_in_executor(executor, callback, *args)

requests.get()明确支持命名参数(get，auth等)。怎么了？

两种方法。创建包装函数，或仅使用会话提供身份验证。

使用会话：

1
2
3
4
5
6
7
8
@asyncio.coroutine
def do_checks():
    loop = asyncio.get_event_loop()
    session = requests.Session()
    session.auth = HTTPBasicAuth('user', 'pass')
    req = loop.run_in_executor(None, session.get, 'https://api.github.com/user')
    resp = yield from req
    print(resp.status_code)

编写一个包装函数(请注意，这里我使用def是为了清楚起见，但是lambda显然也可以工作)：

1
2
3
4
5
6
7
8
@asyncio.coroutine
def do_checks():
    def do_req():
        return requests.get('https://api.github.com/user', auth=HTTPBasicAuth('user', 'pass'))
    loop = asyncio.get_event_loop()
    req = loop.run_in_executor(None, do_req)
    resp = yield from req
    print(resp.status_code)
相关讨论

这实际上是asyncio中的设计决策。从asyncio / base_events.py的文档字符串中：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
"""Base implementation of event loop.

The event loop can be broken up into a multiplexer (the part
responsible for notifying us of IO events) and the event loop proper,
which wraps a multiplexer with functionality for scheduling callbacks,
immediately or at a given time in the future.

Whenever a public API takes a callback, subsequent positional
arguments will be passed to the callback if/when it is called.  This
avoids the proliferation of trivial lambdas implementing closures.
Keyword arguments for the callback are not supported; this is a
conscious design decision, leaving the door open for keyword arguments
to modify the meaning of the API call itself.
"""

请注意最后一句话。

asyncio PEP也要注意这一点，并建议使用lambda来解决：

This convention specifically does not support keyword arguments.
Keyword arguments are used to pass optional extra information about
the callback. This allows graceful evolution of the API without having
to worry about whether a keyword might be significant to a callee
somewhere. If you have a callback that must be called with a keyword
argument, you can use a lambda. For example:

loop.call_soon(lambda: foo('abc', repeat=42))