When I've been programming in Java I got used to write various tests. It's good to be able to play with the API you just wrote and tests save you a lot of time as programmed tests take less time than manual testing. Especially it's handy when you have some external service your app talks to and this service is not very fast, e.g. a response could take several minutes. So if you're testing stuff manually, it can take you several minutes, and the other approach to stub this service out and use this stub in tests to test the logic and stuff.
So, returning back to Java, I must admit that Java world has a plenty of great testing/mocking/etc frameworks. And the situation in the Python world really disheartens me. First of all, testing frameworks. Maybe I've missed something, but looks like the only used frameworks are doctest and unittest, both from standard library.
I don't get this doctest framework at all. Putting code in the comments to your code seems to be awkward to me. I don't think inline code documentation is a good place to place some examples, if your code requires examples, it's better to write up a tutorial and place all the examples here. So, it's not the way to go (IMHO).
The other framework is unittest -- a clone of some old JUnit with very limited feature set. Misses lots of vital features that could be found in TestNG for example. Specifically, I miss the following features:
* Test groups/dependencies. I want to make one test depend on other. It's clear that if some basic thing fails (e.g. user auth) it makes no sense at all to test all the stuff on top of that (e.g. actions for logged in user). And all these 'EEEEE' for lots of failed tests just make it harder the reason of error
* Lack of beforeSuite etc methods
... and many more, lazy to recall right now.
Testing in Django itself is also quite cumbersome. First of all, the concept of keeping tests in app/tests.py is not very flexible. This app-centric approach is probably good when your project consists only from Django, but I really doubt it's possible in real life: most likely you will have some kind of helpers, misc. classes that don't fit into apps and so on, and all of them require to be tested. What's the good place to place these tests? If you won't place them into tests.py of one of the apps, you will most likely have to implement your own test runner.
More examples:
* You cannot run an individual test using django test runner. Imagine you managed to write, say, 100 tests and it takes about 3 minutes to run all of them. Then you find that one of the tests failed. Instead of 'try to fix -> run failing test' iterations you will have 'try to fix -> run all the tests -> wait N minutes'.
* To plug some simple things, like code coverage reports, you have to override/implement own test runner.
Now it's time to get back to stubbing again. Django doesn't provide any easy way to replace real libs with the stubbed ones. The only good thing about it is that it's quite easy to get separate database instance for tests and automatically upload fixtures into it.
But I don't see a clean way to stub out an external lib, the trick with overriding sys.modules['modulename'] doesn't work if you do it in setUp, because the lib could be already improted, etc.
The testing situation is not as bad as you think. There are a number of great test runners outside the stdlib, for example, nose. Nose has a Django plugin that will let you run your Django tests from Nose, and then you can use all the nose features, like a coverage plugin, selecting tests, etc.
ReplyDeleteThe Testing In Python list is very good, as is the Django-Users list. Either would be able to help.
Ned, thanks for the comment. And for the great coverage.py tool :-)
ReplyDeleteActually, I've looked at nose and does seem to be more interesting than unittest, but from the first try I wasn't able to get it work with Django.
However, provided the project will not finish soonish and we will still be using Django, I will try to move all tests to nose and see what happens, maybe I will like it.
This comment has been removed by the author.
ReplyDeletePortugal ist ein aufstrebendes Ziel für IT-Nearshoring. Das Land bietet hochqualifizierte IT-Experten, eine stabile politische und wirtschaftliche Umgebung sowie eine moderne IT-Infrastruktur. Unternehmen, die ihre IT-Projekte nach Portugal auslagern, profitieren von einer schnellen Markteinführung, einer hohen Servicequalität und attraktiven Kostenstrukturen. Die portugiesische Regierung hat verschiedene Maßnahmen ergriffen, um den IT-Sektor zu fördern und ein positives Geschäftsumfeld zu schaffen. Mit Portugal als IT-Nearshoring-Partner können Unternehmen ihre Agilität steigern, ihre Kosten senken und von erstklassigen IT-Lösungen profitieren. gute seite
ReplyDelete