关于python：函数调用超时

Timeout on a function call

我正在调用python中的一个函数，我知道它可能会暂停并强制我重新启动脚本。

如何调用该函数，或者如何包装它，以便脚本在超过5秒的时间内取消它并执行其他操作？

如果在Unix上运行，则可以使用信号包：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

In [1]: import signal

# Register an handler for the timeout
In [2]: def handler(signum, frame):
...: print"Forever is over!"
...: raise Exception("end of time")
...:

# This function *may* run for an indetermined time...
In [3]: def loop_forever():
...: import time
...: while 1:
...: print"sec"
...: time.sleep(1)
...:
...:

# Register the signal function handler
In [4]: signal.signal(signal.SIGALRM, handler)
Out[4]: 0

# Define a timeout for your function
In [5]: signal.alarm(10)
Out[5]: 0

In [6]: try:
...: loop_forever()
...: except Exception, exc:
...: print exc
....:
sec
sec
sec
sec
sec
sec
sec
sec
Forever is over!
end of time

# Cancel the timer if the function returned before timeout
# (ok, mine won't but yours maybe will :)
In [7]: signal.alarm(0)
Out[7]: 0

调用alarm.alarm(10)10秒后，调用处理程序。这会引发一个异常，您可以从常规的Python代码中截获它。

这个模块不能很好地处理线程(但是，谁会这样做？)

请注意，由于我们在超时发生时引发异常，它可能会在函数内部被捕获和忽略，例如这样一个函数：

1
2
3
4
5
6
7

def loop_forever():
while 1:
print 'sec'
try:
time.sleep(10)
except:
continue

相关讨论

你可以用multiprocessing.Process来实现这一点。

代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import multiprocessing
import time

# bar
def bar():
for i in range(100):
print"Tick"
time.sleep(1)

if __name__ == '__main__':
# Start bar as a process
p = multiprocessing.Process(target=bar)
p.start()

# Wait for 10 seconds or until process finishes
p.join(10)

# If thread is still active
if p.is_alive():
print"running... let's kill it..."

# Terminate
p.terminate()
p.join()

相关讨论

How do I call the function or what do I wrap it in so that if it takes longer than 5 seconds the script cancels it?

我发布了一个要点，用一个装饰器和一个threading.Timer来解决这个问题。这里有一个细目。

为兼容性导入和设置

它是用python 2和3测试的。它还应该在UNIX/Linux和Windows下工作。

首先是进口。这些方法试图保持代码的一致性，而不考虑Python版本：

1
2
3
4
5
6
7
8

from __future__ import print_function
import sys
import threading
from time import sleep
try:
import thread
except ImportError:
import _thread as thread

使用与版本无关的代码：

1
2
3
4
5
6
7
8
9

try:
range, _print = xrange, print
def print(*args, **kwargs):
flush = kwargs.pop('flush', False)
_print(*args, **kwargs)
if flush:
kwargs.get('file', sys.stdout).flush()
except NameError:
pass

现在我们已经从标准库中导入了我们的功能。

装饰工

接下来，我们需要一个函数从子线程终止main()：

1
2
3
4
5

def quit_function(fn_name):
# print to stderr, unbuffered in Python 2.
print('{0} took too long'.format(fn_name), file=sys.stderr)
sys.stderr.flush() # Python 3 stderr is likely buffered.
thread.interrupt_main() # raises KeyboardInterrupt

这就是装饰器本身：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

def exit_after(s):
'''
use as decorator to exit process if
function takes longer than s seconds
'''
def outer(fn):
def inner(*args, **kwargs):
timer = threading.Timer(s, quit_function, args=[fn.__name__])
timer.start()
try:
result = fn(*args, **kwargs)
finally:
timer.cancel()
return result
return inner
return outer

用法

下面是一个用法，可以直接回答您关于5秒后退出的问题！：

1
2
3
4
5
6
7

@exit_after(5)
def countdown(n):
print('countdown started', flush=True)
for i in range(n, -1, -1):
print(i, end=', ', flush=True)
sleep(1)
print('countdown finished')

演示：

1
2
3
4
5
6
7
8
9
10
11

>>> countdown(3)
countdown started
3, 2, 1, 0, countdown finished
>>> countdown(10)
countdown started
10, 9, 8, 7, 6, countdown took too long
Traceback (most recent call last):
File"<stdin>", line 1, in <module>
File"<stdin>", line 11, in inner
File"<stdin>", line 6, in countdown
KeyboardInterrupt

第二个函数调用将不会完成，相反，该进程应退出并进行跟踪！

KeyboardInterrupt并不总是停止休眠线程

注意，在Windows的python 2上，睡眠不会总是被键盘中断中断，例如：

1
2
3
4
5
6
7
8
9
10
11
12

@exit_after(1)
def sleep10():
sleep(10)
print('slept 10 seconds')

>>> sleep10()
sleep10 took too long # Note that it hangs here about 9 more seconds
Traceback (most recent call last):
File"<stdin>", line 1, in <module>
File"<stdin>", line 11, in inner
File"<stdin>", line 3, in sleep10
KeyboardInterrupt

它也不可能中断在扩展中运行的代码，除非它显式地检查PyErr_CheckSignals()，请参见cython、python和keyboardinterrupt-ignored。

在任何情况下，我都会避免线程休眠超过一秒钟——这是处理器时间上的一个eon。

How do I call the function or what do I wrap it in so that if it takes longer than 5 seconds the script cancels it and does something else?

要捕获它并执行其他操作，您可以捕获键盘中断。

1
2
3
4
5
6
7
8

>>> try:
... countdown(10)
... except KeyboardInterrupt:
... print('do something else')
...
countdown started
10, 9, 8, 7, 6, countdown took too long
do something else

相关讨论

我有一个不同的建议，它是一个纯函数(与线程建议具有相同的API)，并且看起来工作得很好(基于此线程的建议)。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

def timeout(func, args=(), kwargs={}, timeout_duration=1, default=None):
import signal

class TimeoutError(Exception):
pass

def handler(signum, frame):
raise TimeoutError()

# set the timeout handler
signal.signal(signal.SIGALRM, handler)
signal.alarm(timeout_duration)
try:
result = func(*args, **kwargs)
except TimeoutError as exc:
result = default
finally:
signal.alarm(0)

return result

相关讨论

我在寻找单元测试的超时调用时遇到了这个线程。我在答案或第三方软件包中没有发现任何简单的内容，因此我写了下面的装饰程序，您可以直接进入代码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

import multiprocessing.pool
import functools

def timeout(max_timeout):
"""Timeout decorator, parameter in seconds."""
def timeout_decorator(item):
"""Wrap the original function."""
@functools.wraps(item)
def func_wrapper(*args, **kwargs):
"""Closure for function."""
pool = multiprocessing.pool.ThreadPool(processes=1)
async_result = pool.apply_async(item, args, kwargs)
# raises a TimeoutError if execution exceeds max_timeout
return async_result.get(max_timeout)
return func_wrapper
return timeout_decorator

然后，简单地超时测试或您喜欢的任何函数：

1
2
3

@timeout(5.0) # if execution takes longer than 5 seconds, raise a TimeoutError
def test_base_regression(self):
...

相关讨论

有很多建议，但没有使用concurrent.futures，我认为这是处理这个问题的最清晰的方法。

1
2
3
4
5
6
7

from concurrent.futures import ProcessPoolExecutor

# Warning: this does not terminate function if timeout
def timeout_five(fnc, *args, **kwargs):
with ProcessPoolExecutor() as p:
f = p.submit(fnc, *args, **kwargs)
return f.result(timeout=5)

超简单的阅读和维护。

我们创建一个池，提交一个进程，然后在引发TimeoutError之前等待5秒钟，您可以根据需要捕获并处理它。

python 3.2+自带，后端口为2.7(pip-install-futures)。

线程和进程之间的切换就像用ThreadPoolExecutor替换ProcessPoolExecutor一样简单。

如果您想在超时时终止进程，我建议您查看Pebble。

相关讨论

Pypi上的stopit包似乎可以很好地处理超时问题。

我喜欢@stopit.threading_timeoutable修饰器，它在修饰函数中添加了一个timeout参数，这可以满足您的期望，它停止函数。

在pypi上查看：https://pypi.python.org/pypi/stopit

相关讨论

非常好，易于使用和可靠的PYPI项目超时装饰器(https://pypi.org/project/timeout-decorator/)

安装：

1	pip install timeout-decorator

用途：

1
2
3
4
5
6
7
8
9
10
11
12

import time
import timeout_decorator

@timeout_decorator.timeout(5)
def mytest():
print"Start"
for i in range(1,10):
time.sleep(1)
print"%d seconds have passed" % i

if __name__ == '__main__':
mytest()

相关讨论

1
2
3
4
5
6
7
8

#!/usr/bin/python2
import sys, subprocess, threading
proc = subprocess.Popen(sys.argv[2:])
timer = threading.Timer(float(sys.argv[1]), proc.terminate)
timer.start()
proc.wait()
timer.cancel()
exit(proc.returncode)

相关讨论

我需要可嵌套的定时中断(SigAlarm不能做到)，它不会被time.sleep阻塞(基于线程的方法不能做到)。我最终从这里复制并略微修改了代码：http://code.activestate.com/recipes/577600-queue-for-managing-multiple-sigalrm-alarms-concurr/

代码本身：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102

#!/usr/bin/python

# lightly modified version of http://code.activestate.com/recipes/577600-queue-for-managing-multiple-sigalrm-alarms-concurr/

"""alarm.py: Permits multiple SIGALRM events to be queued.

Uses a `heapq` to store the objects to be called when an alarm signal is
raised, so that the next alarm is always at the top of the heap.
"""

import heapq
import signal
from time import time

__version__ = '$Revision: 2539 $'.split()[1]

alarmlist = []

__new_alarm = lambda t, f, a, k: (t + time(), f, a, k)
__next_alarm = lambda: int(round(alarmlist[0][0] - time())) if alarmlist else None
__set_alarm = lambda: signal.alarm(max(__next_alarm(), 1))

class TimeoutError(Exception):
def __init__(self, message, id_=None):
self.message = message
self.id_ = id_

class Timeout:
''' id_ allows for nested timeouts. '''
def __init__(self, id_=None, seconds=1, error_message='Timeout'):
self.seconds = seconds
self.error_message = error_message
self.id_ = id_
def handle_timeout(self):
raise TimeoutError(self.error_message, self.id_)
def __enter__(self):
self.this_alarm = alarm(self.seconds, self.handle_timeout)
def __exit__(self, type, value, traceback):
try:
cancel(self.this_alarm)
except ValueError:
pass

def __clear_alarm():
"""Clear an existing alarm.

If the alarm signal was set to a callable other than our own, queue the
previous alarm settings.
"""
oldsec = signal.alarm(0)
oldfunc = signal.signal(signal.SIGALRM, __alarm_handler)
if oldsec > 0 and oldfunc != __alarm_handler:
heapq.heappush(alarmlist, (__new_alarm(oldsec, oldfunc, [], {})))

def __alarm_handler(*zargs):
"""Handle an alarm by calling any due heap entries and resetting the alarm.

Note that multiple heap entries might get called, especially if calling an
entry takes a lot of time.
"""
try:
nextt = __next_alarm()
while nextt is not None and nextt <= 0:
(tm, func, args, keys) = heapq.heappop(alarmlist)
func(*args, **keys)
nextt = __next_alarm()
finally:
if alarmlist: __set_alarm()

def alarm(sec, func, *args, **keys):
"""Set an alarm.

When the alarm is raised in `sec` seconds, the handler will call `func`,
passing `args` and `keys`. Return the heap entry (which is just a big
tuple), so that it can be cancelled by calling `cancel()`.
"""
__clear_alarm()
try:
newalarm = __new_alarm(sec, func, args, keys)
heapq.heappush(alarmlist, newalarm)
return newalarm
finally:
__set_alarm()

def cancel(alarm):
"""Cancel an alarm by passing the heap entry returned by `alarm()`.

It is an error to try to cancel an alarm which has already occurred.
"""
__clear_alarm()
try:
alarmlist.remove(alarm)
heapq.heapify(alarmlist)
finally:
if alarmlist: __set_alarm()

以及用法示例：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

import alarm
from time import sleep

try:
with alarm.Timeout(id_='a', seconds=5):
try:
with alarm.Timeout(id_='b', seconds=2):
sleep(3)
except alarm.TimeoutError as e:
print 'raised', e.id_
sleep(30)
except alarm.TimeoutError as e:
print 'raised', e.id_
else:
print 'nope.'

相关讨论

timeout-decorator在Windows系统上不起作用，因为Windows不支持signal。

如果在Windows系统中使用Timeout Decorator，您将获得以下信息

1	AttributeError: module 'signal' has no attribute 'SIGALRM'

有些人建议使用use_signals=False，但我没有用。

author@bitranox创建了以下包：

1	pip install https://github.com/bitranox/wrapt-timeout-decorator/archive/master.zip

代码示例：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

import time
from wrapt_timeout_decorator import *

@timeout(5)
def mytest(message):
print(message)
for i in range(1,10):
time.sleep(1)
print('{} seconds have passed'.format(i))

def main():
mytest('starting')

if __name__ == '__main__':
main()

出现以下异常：

1	TimeoutError: Function mytest timed out after 5 seconds

我们同样可以使用信号。我认为下面的例子对您有用。与线程相比，它非常简单。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

import signal

def timeout(signum, frame):
raise myException

#this is an infinite loop, never ending under normal circumstances
def main():
print 'Starting Main ',
while 1:
print 'in main ',

#SIGALRM is only usable on a unix platform
signal.signal(signal.SIGALRM, timeout)

#change 5 to however many seconds you need
signal.alarm(5)

try:
main()
except myException:
print"whoops"

相关讨论

这里是对给定的基于线程的解决方案的细微改进。

下面的代码支持异常：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

def runFunctionCatchExceptions(func, *args, **kwargs):
try:
result = func(*args, **kwargs)
except Exception, message:
return ["exception", message]

return ["RESULT", result]

def runFunctionWithTimeout(func, args=(), kwargs={}, timeout_duration=10, default=None):
import threading
class InterruptableThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.result = default
def run(self):
self.result = runFunctionCatchExceptions(func, *args, **kwargs)
it = InterruptableThread()
it.start()
it.join(timeout_duration)
if it.isAlive():
return default

if it.result[0] =="exception":
raise it.result[1]

return it.result[1]

以5秒超时调用它：

1	result = timeout(remote_calculate, (myarg,), timeout_duration=5)

相关讨论