FWIW it was added back in 2.8.0 (though only as a short `--lf` options). 3.2.0 makes it smarter by improving the first-failed/last-failed clear behaviour: it now only clears tests which have succeeded.
You can see the use case in the PR: make a change, it breaks a lot of tests, then fix module par module e.g. after an initial `pytest` (failed lots) fix `pytest core --lf`, then fix `pytest controller --lf`, … the problem pre-3.2 is that `pytest core --lf` would reset the cache with only its own failures, so the subsequent `pytest controller —lf` would start with an empty cache and run every single test rather than only those which had previously failed.
This is problematic when you have a large expensive testing base (e.g. so much so that you xdist it, as the original PR did)
If for no other reason, pytest requires a lot less boilerplate.
Pytest fixtures are also quite a bit more flexible than unittest's setUp and tearDown methods, although they take a little while to really understand (at least they did for me) since they use some magic.
It has an excellent runner and makes much better use of Python features:
* you can define tests as free functions (you don't have to, it also supports classes and even regular unittest cases, you can use pytest to run your existing suite)
* exceptions aside it uses the standard `assert` statement
* fixtures are DI'd, independent from tests and much more flexible than the setUp/tearDown dance
* tests are configured and fixtures are defined using decorators & generators
* excellent support for data-driven tests (parametrized tests and fixtures)
* there is a wealth of reusable third-party plugins[0] ranging from framework helpers[1] to test parallelisation and distribution[2]
> fixtures are DI'd, independent from tests and much more flexible than the setUp/tearDown dance
If this were the only difference, it'd still be enough. Inevitably I end up with a unittest.TestCase with 30 tests, all of which mock out some resource. Except for 1. That one wants to mock out a different resource while leaving the first intact.
* With unittest: either split that test out into its own TestCase (duplicating most of the original, except for the part I don't want, or coming up with an inheritance tree with is always ugly with unittest), or
* Remove all the unwanted mocks inside that test.
* With pytest: don't pass the mocking fixture into that test.
Yup. The ability to compose fixtures too, a test can depend on a fixture but so can an other fixture.
Combined with flexible scopes and the ability to mark and select/deselect tests based on name and marks it makes for genuinely great stuff e.g. some of your tests depend on the database, you can have a session-scoped fixture handling a connection, then a function-scoped one depending on it and getting a cursor, and if you don't select any test depending on the latter no connection will be created at any point.
Parametrized tests are quite powerful and you can be creative with them. For example, I am using them in integration testing (for this test, I need process A and process B as well as a string from set C): https://vincent.bernat.im/en/blog/2016-testing-pytest-linux-...
You can define the lifetime of each parameter (just the test, the class or the whole testing session). This makes it very flexible.
The use of "assert" is magic in the sense that in case of failure, you get helpful values. "assert a == b" will show you the values of "a" and "b" in case of failure.
Pytest is more pythonic than the xUnitTest suite in the standard library. It is authored by the same genius who authored pypy, hence the wow about it. Nosetest was forked off pytest as a way to maintain the project several years ago, but pytest came back to life and eventually superseded nose. The fixtures in pytest is a very clever way to structure your tests around dynamic "given" set of inputs.
I've been a long time nosetests user, but it's become unmaintained, so I'd like to migrate something else. pytest seems to be an obvious choice, but I really dislike the way it formats test failures. Is there a way to make it use a more traditional format?
I really like Pytest but I didn't see a way to easily customize the output format (not just traceback but overall output). If they can add documentation with examples of the hooks available (if present) to customize the output, I won't have anything further to complain about Pytest.
So I'm not the only one :) Do you have any suggestions about alternatives? The Python code base I work with isn't too big so I'm open to exploring other unit test frameworks.
Oh boy! `PYTEST_CURRENT_TEST` let me take out a lot of hacks from my code. We're doing a lot with logging and parallel tests, so this was always a sore point.
I've been pretty disappointed with PyCharm's support for Pytest in a Django project -- seems to be incapable of overriding the `manage.py` test runner consistently.
pytest-django has some docs[1] for how to plug into `manage.py`, but they are broken as of Django 1.10.
Yeah, see part one of my comment -- I've found that PyCharm is pretty bad at selecting `pytest` as the test runner. It works for test functions, but not for files/directories.
Perhaps I'm missing something obvious, but PyCharm just uses `./manage.py test` even when I've selected Py.test as the project test runner.
Any pointers on using fixtures with Django database models?
Seems like it would be useful to be able to manage the lifecycle of fixtures more explicitly than Django's TestCase allows you to, but this could get gnarly if the test case transactions were rolling back changes to fixture objects.
36 comments
[ 2.2 ms ] story [ 81.4 ms ] threadYou can see the use case in the PR: make a change, it breaks a lot of tests, then fix module par module e.g. after an initial `pytest` (failed lots) fix `pytest core --lf`, then fix `pytest controller --lf`, … the problem pre-3.2 is that `pytest core --lf` would reset the cache with only its own failures, so the subsequent `pytest controller —lf` would start with an empty cache and run every single test rather than only those which had previously failed.
This is problematic when you have a large expensive testing base (e.g. so much so that you xdist it, as the original PR did)
Pytest fixtures are also quite a bit more flexible than unittest's setUp and tearDown methods, although they take a little while to really understand (at least they did for me) since they use some magic.
* you can define tests as free functions (you don't have to, it also supports classes and even regular unittest cases, you can use pytest to run your existing suite)
* exceptions aside it uses the standard `assert` statement
* fixtures are DI'd, independent from tests and much more flexible than the setUp/tearDown dance
* tests are configured and fixtures are defined using decorators & generators
* excellent support for data-driven tests (parametrized tests and fixtures)
* there is a wealth of reusable third-party plugins[0] ranging from framework helpers[1] to test parallelisation and distribution[2]
[0] http://plugincompat.herokuapp.com
[1] https://pypi.python.org/pypi/pytest-tornado
[2] https://pypi.python.org/pypi/pytest-xdist
If this were the only difference, it'd still be enough. Inevitably I end up with a unittest.TestCase with 30 tests, all of which mock out some resource. Except for 1. That one wants to mock out a different resource while leaving the first intact.
* With unittest: either split that test out into its own TestCase (duplicating most of the original, except for the part I don't want, or coming up with an inheritance tree with is always ugly with unittest), or
* Remove all the unwanted mocks inside that test.
* With pytest: don't pass the mocking fixture into that test.
Combined with flexible scopes and the ability to mark and select/deselect tests based on name and marks it makes for genuinely great stuff e.g. some of your tests depend on the database, you can have a session-scoped fixture handling a connection, then a function-scoped one depending on it and getting a cursor, and if you don't select any test depending on the latter no connection will be created at any point.
You can define the lifetime of each parameter (just the test, the class or the whole testing session). This makes it very flexible.
The use of "assert" is magic in the sense that in case of failure, you get helpful values. "assert a == b" will show you the values of "a" and "b" in case of failure.
https://docs.pytest.org/en/latest/usage.html#modifying-pytho...
It still annoys me that
takes the whole screen width. I can pipe stdout through "cat" to fix it, but it's not very convenient.Testing Python Applications with Pytest https://semaphoreci.com/community/tutorials/testing-python-a...
pytest-django has some docs[1] for how to plug into `manage.py`, but they are broken as of Django 1.10.
Anyone else had this problem?
[1]: https://github.com/pytest-dev/pytest-django/blob/master/docs...
[0]: https://pytest-django.readthedocs.io/en/latest/#why-would-i-...
Perhaps I'm missing something obvious, but PyCharm just uses `./manage.py test` even when I've selected Py.test as the project test runner.
Seems like it would be useful to be able to manage the lifecycle of fixtures more explicitly than Django's TestCase allows you to, but this could get gnarly if the test case transactions were rolling back changes to fixture objects.