了解pytest配置文件

pytest.ini

pytest的主配置文件,可以改变pytest的默认行为,其中有很多可配置的选项。

1、 可以用pytest --help命令查看

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
[AppledeMBP-3:~ apple$ pytest --help
usage: pytest [options] [file_or_dir] [file_or_dir] [...]

positional arguments:
  file_or_dir

general:
  -k EXPRESSION         only run tests which match the given substring
                        expression. An expression is a python evaluatable
                        expression where all names are substring-matched against
                        test names and their parent classes. Example: -k
                        'test_method or test_other' matches all test functions
                        and classes whose name contains 'test_method' or
                        'test_other', while -k 'not test_method' matches those
                        that don't contain 'test_method' in their names. -k 'not
                        test_method and not test_other' will eliminate the
                        matches. Additionally keywords are matched to classes
                        and functions containing extra names in their
                        'extra_keyword_matches' set, as well as functions which
                        have names assigned directly to them.
  -m MARKEXPR           only run tests matching given mark expression. example:
                        -m 'mark1 and not mark2'.
  --markers             show markers (builtin, plugin and per-project ones).
  -x, --exitfirst       exit instantly on first error or failed test.
  --maxfail=num         exit after first num failures or errors.
  --strict-markers, --strict
                        markers not registered in the `markers` section of the
                        configuration file raise errors.
  -c file               load configuration from `file` instead of trying to
                        locate one of the implicit configuration files.
  --continue-on-collection-errors
                        Force test execution even if collection errors occur.
  --rootdir=ROOTDIR     Define root directory for tests. Can be relative path:
                        'root_dir', './root_dir', 'root_dir/another_dir/';
                        absolute path: '/home/user/root_dir'; path with
                        variables: '$HOME/root_dir'.
  --fixtures, --funcargs
                        show available fixtures, sorted by plugin appearance
                        (fixtures with leading '_' are only shown with '-v')
  --fixtures-per-test   show fixtures per test
  --import-mode={prepend,append}
                        prepend/append to sys.path when importing test modules,
                        default is to prepend.
  --pdb                 start the interactive Python debugger on errors or
                        KeyboardInterrupt.
  --pdbcls=modulename:classname
                        start a custom interactive Python debugger on errors.
                        For example:
                        --pdbcls=IPython.terminal.debugger:TerminalPdb
  --trace               Immediately break when running each test.
  --capture=method      per-test capturing method: one of fd|sys|no.
  -s                    shortcut for --capture=no.
  --runxfail            report the results of xfail tests as if they were not
                        marked
  --lf, --last-failed   rerun only the tests that failed at the last run (or all
                        if none failed)
  --ff, --failed-first  run all tests but run the last failures first. This may
                        re-order tests and thus lead to repeated fixture
                        setup/teardown
  --nf, --new-first     run tests from new files first, then the rest of the
                        tests sorted by file mtime
  --cache-show=[CACHESHOW]
                        show cache contents, don't perform collection or tests.
                        Optional argument: glob (default: '*').
  --cache-clear         remove all cache contents at start of test run.
  --lfnf={all,none}, --last-failed-no-failures={all,none}
                        which tests to run with no previously (known) failures.
  --sw, --stepwise      exit on test failure and continue from last failing test
                        next time
  --stepwise-skip       ignore the first failing test but stop on the next
                        failing test

reporting:
  --durations=N         show N slowest setup/test durations (N=0 for all).
  -v, --verbose         increase verbosity.
  -q, --quiet           decrease verbosity.
  --verbosity=VERBOSE   set verbosity
  -r chars              show extra test summary info as specified by chars:
                        (f)ailed, (E)rror, (s)kipped, (x)failed, (X)passed,
                        (p)assed, (P)assed with output, (a)ll except passed
                        (p/P), or (A)ll. (w)arnings are enabled by default (see
                        --disable-warnings).
  --disable-warnings, --disable-pytest-warnings
                        disable warnings summary
  -l, --showlocals      show locals in tracebacks (disabled by default).
  --tb=style            traceback print mode (auto/long/short/line/native/no).
  --show-capture={no,stdout,stderr,log,all}
                        Controls how captured stdout/stderr/log is shown on
                        failed tests. Default is 'all'.
  --full-trace          don't cut any tracebacks (default is to cut).
  --color=color         color terminal output (yes/no/auto).
  --pastebin=mode       send failed|all info to bpaste.net pastebin service.
  --junit-xml=path      create junit-xml style report file at given path.
  --junit-prefix=str    prepend prefix to classnames in junit-xml output
  --result-log=path     DEPRECATED path for machine-readable result log.

collection:
  --collect-only        only collect tests, don't execute them.
  --pyargs              try to interpret all arguments as python packages.
  --ignore=path         ignore path during collection (multi-allowed).
  --ignore-glob=path    ignore path pattern during collection (multi-allowed).
  --deselect=nodeid_prefix
                        deselect item during collection (multi-allowed).
  --confcutdir=dir      only load conftest.py's relative to specified dir.
  --noconftest          Don't load any conftest.py files.
  --keep-duplicates     Keep duplicate tests.
  --collect-in-virtualenv
                        Don't ignore tests in a local virtualenv directory
  --doctest-modules     run doctests in all .py modules
  --doctest-report={none,cdiff,ndiff,udiff,only_first_failure}
                        choose another output format for diffs on doctest
                        failure
  --doctest-glob=pat    doctests file matching pattern, default: test*.txt
  --doctest-ignore-import-errors
                        ignore doctest ImportErrors
  --doctest-continue-on-failure
                        for a given doctest, continue to run after the first
                        failure

test session debugging and configuration:
  --basetemp=dir        base temporary directory for this test run.(warning:
                        this directory is removed if it exists)
  --version             display pytest lib version and import information.
  -h, --help            show help message and configuration info
  -p name               early-load given plugin module name or entry point
                        (multi-allowed). To avoid loading of plugins, use the
                        `no:` prefix, e.g. `no:doctest`.
  --trace-config        trace considerations of conftest.py files.
  --debug               store internal tracing debug information in
                        'pytestdebug.log'.
  -o OVERRIDE_INI, --override-ini=OVERRIDE_INI
                        override ini option with "option=value" style, e.g. `-o
                        xfail_strict=True -o cache_dir=cache`.
  --assert=MODE         Control assertion debugging tools. 'plain' performs no
                        assertion debugging. 'rewrite' (the default) rewrites
                        assert statements in test modules on import to provide
                        assert expression information.
  --setup-only          only setup fixtures, do not execute tests.
  --setup-show          show setup of fixtures while executing tests.
  --setup-plan          show what fixtures and tests would be executed but don't
                        execute anything.

pytest-warnings:
  -W PYTHONWARNINGS, --pythonwarnings=PYTHONWARNINGS
                        set which warnings to report, see -W option of python
                        itself.

logging:
  --no-print-logs       disable printing caught logs on failed tests.
  --log-level=LOG_LEVEL
                        logging level used by the logging module
  --log-format=LOG_FORMAT
                        log format as used by the logging module.
  --log-date-format=LOG_DATE_FORMAT
                        log date format as used by the logging module.
  --log-cli-level=LOG_CLI_LEVEL
                        cli logging level.
  --log-cli-format=LOG_CLI_FORMAT
                        log format as used by the logging module.
  --log-cli-date-format=LOG_CLI_DATE_FORMAT
                        log date format as used by the logging module.
  --log-file=LOG_FILE   path to a file when logging will be written to.
  --log-file-level=LOG_FILE_LEVEL
                        log file logging level.
  --log-file-format=LOG_FILE_FORMAT
                        log format as used by the logging module.
  --log-file-date-format=LOG_FILE_DATE_FORMAT
                        log date format as used by the logging module.

Hypothesis:
  --hypothesis-profile=HYPOTHESIS_PROFILE
                        Load in a registered hypothesis.settings profile
  --hypothesis-verbosity={quiet,normal,verbose,debug}
                        Override profile with verbosity setting specified
  --hypothesis-show-statistics
                        Configure when statistics are printed
  --hypothesis-seed=HYPOTHESIS_SEED
                        Set a seed to use for all Hypothesis tests

pytest-parallel:
  --workers=WORKERS     Set the max num of workers (aka processes) to start (int
                        or "auto" - one per core)
  --tests-per-worker=TESTS_PER_WORKER
                        Set the max num of concurrent tests for each worker (int
                        or "auto" - split evenly)

[pytest] ini-options in the first pytest.ini|tox.ini|setup.cfg file found:

  markers (linelist):   markers for test functions
  empty_parameter_set_mark (string):
                        default marker for empty parametersets
  norecursedirs (args): directory patterns to avoid for recursion
  testpaths (args):     directories to search for tests when no files or
                        directories are given in the command line.
  usefixtures (args):   list of default fixtures to be used with this project
  python_files (args):  glob-style file patterns for Python test module
                        discovery
  python_classes (args):
                        prefixes or glob names for Python test class discovery
  python_functions (args):
                        prefixes or glob names for Python test function and
                        method discovery
  disable_test_id_escaping_and_forfeit_all_rights_to_community_support (bool):
                        disable string escape non-ascii characters, might cause
                        unwanted side effects(use at your own risk)
  console_output_style (string):
                        console output: "classic", or with additional progress
                        information ("progress" (percentage) | "count").
  xfail_strict (bool):  default for the strict parameter of xfail markers when
                        not given explicitly (default: False)
  enable_assertion_pass_hook (bool):
                        Enables the pytest_assertion_pass hook.Make sure to
                        delete any previously generated pyc cache files.
  junit_suite_name (string):
                        Test suite name for JUnit report
  junit_logging (string):
                        Write captured log messages to JUnit report: one of
                        no|system-out|system-err
  junit_log_passing_tests (bool):
                        Capture log information for passing tests to JUnit
                        report:
  junit_duration_report (string):
                        Duration time to report: one of total|call
  junit_family (string):
                        Emit XML for schema: one of legacy|xunit1|xunit2
  doctest_optionflags (args):
                        option flags for doctests
  doctest_encoding (string):
                        encoding used for doctest files
  cache_dir (string):   cache directory path.
  filterwarnings (linelist):
                        Each line specifies a pattern for
                        warnings.filterwarnings. Processed after -W and
                        --pythonwarnings.
  log_print (bool):     default value for --no-print-logs
  log_level (string):   default value for --log-level
  log_format (string):  default value for --log-format
  log_date_format (string):
                        default value for --log-date-format
  log_cli (bool):       enable log display during test run (also known as "live
                        logging").
  log_cli_level (string):
                        default value for --log-cli-level
  log_cli_format (string):
                        default value for --log-cli-format
  log_cli_date_format (string):
                        default value for --log-cli-date-format
  log_file (string):    default value for --log-file
  log_file_level (string):
                        default value for --log-file-level
  log_file_format (string):
                        default value for --log-file-format
  log_file_date_format (string):
                        default value for --log-file-date-format
  faulthandler_timeout (string):
                        Dump the traceback of all threads if a test takes more
                        than TIMEOUT seconds to finish. Not available on
                        Windows.
  addopts (args):       extra command line options
  minversion (string):  minimally required pytest version
  workers (string):     Set the max num of workers (aka processes) to start (int
                        or "auto" - one per core)
  tests_per_worker (string):
                        Set the max num of concurrent tests for each worker (int
                        or "auto" - split evenly)

environment variables:
  PYTEST_ADDOPTS           extra command line options
  PYTEST_PLUGINS           comma-separated plugins to load during startup
  PYTEST_DISABLE_PLUGIN_AUTOLOAD set to disable plugin auto-loading
  PYTEST_DEBUG             set to enable debug tracing of pytest's internals


to see available markers type: pytest --markers
to see available fixtures type: pytest --fixtures
(shown according to specified file_or_dir or current dir if not specified; fixtures with leading '_' are only shown with the '-v' option
AppledeMBP-3:~ apple$

2、 更改默认命令行选项

1
2
[pytest]
addopts = -rsxX -l --tb=short --strict

-rsxX表示pytest报告所有测试用例被跳过、预计失败但实际通过的原因。
-l表示pytest报告所有失败测试的堆栈中的局部变量。
--tb=short表示简化堆栈回溯信息,只保留文件和行数。--strict选项表示禁止使用未在配置文件中注册的标记。
3、注册标记来防范拼写错误

1
2
3
4
[pytest]
markers =
smoke: Run the smoke test functions
get: Run the  test functions that test task.get()

标记注册后,可以通过命令pytest–markers来查看:

1
2
3
(venv2) AppledeMacBook-Pro-3:tests apple$ pytest --markers
@pytest.mark.smoke: Run the smoke test functions
@pytest.mark.get: Run the  test functions that test task.get()

没有注册的标记不住出现在—markers列表里。如果使用了--strice选项,遇到拼写错误的标记或未注册的标记就会报错。

4、指定pytest的最低版本号
Minversion选项可以指定运行测试用例pytest的最低版本。例如,测试两个浮点数的值是否非常接近,approx()函数功能直到pytest3.0才出现。为了避免混淆,在使用approx()函数的项目中增加一行配置

1
2
[pytest]
minversion = 3.0

5、指定pytest忽略某些目录
pytest执行测试搜索时,会递归遍历所有子目录,包括某些你明知道没必要遍历的目录。遇到这种情况,可以使用norecurse选项简化pytest的搜索工作。norecurse的默认设置是.builddistCVS_darcs{arch}和.egg。因为有.*,所有以.开头的目录都不会被访问。如果虚拟环境命名为venv,那么需要把它加入norecursedirs里。

1
2
[pytest]
norecursedirs = .* venv src *.egg dist build

更改一个已经设置好的值时,可以先找到原来的默认值,并把它放在新添加内容的后面,就像*.eggdistbuild。

6、指定测试目录
norecursedirs告诉pytest哪些路径不用访问,而testpaths则指示pytest去哪里访问。Testpaths是一系列相对于根目录的路径,用于限定测试用例的搜索范围。只有在pytest未指定文件目录参数或测试用例标识符时,该选项才会启用。集成测试和tox很有用,定义搜索范围可以节约测试搜索的时间。

7、 更改测试搜索的规则
pytest根据一定的规则搜索并允许测试。标准的测试搜索规则如下:

  • 从一个或多个目录开始查找。你可以在命令行指定文件名或目录名。如果未指定,则使用当前目录。
  • 在该目录和所有子目录下递归查找测试模块。
  • 测试模块是指文件名为test_.py或_test.py的文件
  • 在测试模块中查找以test_开头的函数名。
  • 查找名字以Test开头的类。其中,首先筛选掉包含init函数的类,再查找类
    中以Test_开头的类方法。

如果把测试类命名为Test或Suite可以通过python_classes, python_files, python_functions更改测试搜索规则:

1
2
[pytest]
python_classes = * Test test* *Suite

8.禁用XPASS
设置 xfail_strict=true 将会使那些被标记为 @pytest.mark.xfail() 但实际通过的测试用例也被标记为失败。

1
2
[pytest]
xfail_strict = true

conftest.py

是本地的插件库,其中的hook函数和fixture函数将作用于该文件所在的目录以及所有子目录。

init.py

每个测试子目录都包含该文件时,那么在多个测试目录中可以出现同名测试文件。
举例如下:
目录a和目录b都有一个test_foo.py文件,目录结构如图:

目录.png

单独运行每一个文件:

图片.png

让pytest从dups目录运行:

图片.png

错误信息并没有说明哪里出错了。要解决这个问题,只需要在2个子目录里各添加一个空的init.py文件,再试着从dups目录执行:

图片.png