This looks hella good, I particularly am intrigued by the nested setup/teardown system. The natural assert syntax is interesting, too, amazing what macros can accomplish.
Appropriately, the first question that came to mind was, "what's the catch?" :)
Yup. I found it interesting after a co-worker pointed it out to me because it looks and works kind of like RSpec. We are currently considering using it for a new project and it seems mature enough to give it a go.
I'm currently toying with writing a JSONReporter so we can use that as an additional test result output format and I agree, so far it seems to be a really well done project.
The biggest turnoff to me is the sheer quantity of code (regardless of the fact that it can all be crammed into a single header file). It seems much more difficult to fix a bug in it than in something like UnitTest++.
That said, inline setup/teardown and parsing expressions in asserts seem like such natural ideas, I can't believe I haven't run across frameworks doing them before. I guess the shadow of JUnit is extremely long.
The macro trick is at https://github.com/philsquared/Catch/blob/785db43bb2cd64bfe7...: planting the tested expression to the right of an operator ->*, which is overloadable and allows for changing the type of the left hand side in the comparison. The rest is a "simple matter of programming". Clever but probably a bit cute - errors would look weird if the expression isn't set up as expected.
I'm in the same boat (having used UnitTest++ for years) and I wonder the same, though I do see some points already:
- adding a custom main() to unittest++ was pretty messy
- unittest++ is not single-header, meaning you have to recompile all flavours everytime you switch toolsets
- that nested sections stuff looks good and readable, so even though it might be doable with unittest++, this really attracts me more
That being said, I would like to hear from the author or anyone else who can compare them. But likely I'm going to use this for the next project just to check it out.
I use Catch over UnitTest++ because I found it first and it was dead simple to get started with.
We don't do unit-testing as part of our official development process where I work, so I was looking for something that I could quickly try out and get started with in my spare moments. I read about several frameworks and Catch seemed to fit the bill best.
Looking over the UnitTest++ web site just now it appears to be nearly as easy as Catch, so if I had found it first I probably would have given it a try.
For those not currently doing any unit testing who want to get a feel for what it is all about, I highly recommend Catch as a "my first unit testing framework."
To be honest, I prefer bandit[1], it uses lambdas instead of macros which I consider more modern and it has a nice syntax to it. Anyone interested in catch should also check out bandit.
I really like the look of the "Sections" for managing setup/teardown of tests. I'm envious and next time I'm hacking on our in-house C# test framework I'm going to try very hard to implement them! Neat!
It looks like a nice test framework, but it doesn't appear to help at all with the really hard problems you face with testing C++ -- isolating the unit under test for separate compilation, and the development and inclusion of mocks/stubs.
Anyone know of a really good (preferably simple and pragmatic) C++ mocking/stubbing framework to go with this?
Because that's what makes it a unit test -- you only want the unit, the tests and any stubs/mocks in the executable. Anything more and you have an integration test on your hands.
Pulling a single unit out of a legacy codebase can be difficult, especially when you don't want to actually move, copy, or modify the source-files; stubbing out the dependencies and writing mock objects to verify your expectations of what the code should be doing are also very time-consuming.
Those are the hard parts of unit testing C++, and I wonder if anyone has found a good way of doing it (with a framework, or just a technique).
It's lightweight, easy to setup for separated tests or built in tests, and simple to use. Doesn't add anything to help with mocking, but you can do that yourself.
EDIT: To followup, I created a C++ template that leverages TUT and binfmtc that I could use to rapidly iterate with. I used it for working out solutions to problems in "Thinking in C++"; it's posted at http://hardcorehackers.com/~npsimons/Template.hh
30 comments
[ 4.4 ms ] story [ 76.3 ms ] threadAppropriately, the first question that came to mind was, "what's the catch?" :)
I'm currently toying with writing a JSONReporter so we can use that as an additional test result output format and I agree, so far it seems to be a really well done project.
That said, inline setup/teardown and parsing expressions in asserts seem like such natural ideas, I can't believe I haven't run across frameworks doing them before. I guess the shadow of JUnit is extremely long.
https://github.com/philsquared/Catch/blob/785db43bb2cd64bfe7...
UnitTest++ was written by Noel Llopis, is small, tasteful, fast, and supports fixtures (and nested fixtures) via structs and struct inheritance.
Honest question: Why would I use Catch instead?
- adding a custom main() to unittest++ was pretty messy
- unittest++ is not single-header, meaning you have to recompile all flavours everytime you switch toolsets
- that nested sections stuff looks good and readable, so even though it might be doable with unittest++, this really attracts me more
That being said, I would like to hear from the author or anyone else who can compare them. But likely I'm going to use this for the next project just to check it out.
We don't do unit-testing as part of our official development process where I work, so I was looking for something that I could quickly try out and get started with in my spare moments. I read about several frameworks and Catch seemed to fit the bill best.
Looking over the UnitTest++ web site just now it appears to be nearly as easy as Catch, so if I had found it first I probably would have given it a try.
For those not currently doing any unit testing who want to get a feel for what it is all about, I highly recommend Catch as a "my first unit testing framework."
I started working on something similar[1] a few days ago just as a proof of concept.
[1] https://github.com/cEhlen/CUT/blob/master/example.cpp
[1] http://banditcpp.org/index.html
CPP reference has a nice overview of it: http://en.cppreference.com/w/cpp/language/lambda
See https://github.com/joakimkarlsson/snowhouse#assertions
Definitely gonna give this a try.
Any recommended test runner to go with it?
I loved the project logo
Anyone know of a really good (preferably simple and pragmatic) C++ mocking/stubbing framework to go with this?
Pulling a single unit out of a legacy codebase can be difficult, especially when you don't want to actually move, copy, or modify the source-files; stubbing out the dependencies and writing mock objects to verify your expectations of what the code should be doing are also very time-consuming.
Those are the hard parts of unit testing C++, and I wonder if anyone has found a good way of doing it (with a framework, or just a technique).
https://mrzechonek.github.io/tut-framework/
It's lightweight, easy to setup for separated tests or built in tests, and simple to use. Doesn't add anything to help with mocking, but you can do that yourself.
EDIT: To followup, I created a C++ template that leverages TUT and binfmtc that I could use to rapidly iterate with. I used it for working out solutions to problems in "Thinking in C++"; it's posted at http://hardcorehackers.com/~npsimons/Template.hh