I created helper functions in my test that create a full user directrory with whatever concents I want. then my tests can write whatever they want, the dir is deleted at the end of the test. Tempdir isn't safe againts an attacker, but it is good emough for tests. in the real world file systems are fast enough to use in tests.
we use sqlite as our datatbase so it is simple enough for helpers to apply a schema to an in memory database and again the database can be used in tests.
i'he become against mocks in tests. A usefull tool, but rarely needed. You can make tests reliable and fast without them.
Yep, common use case for what I work on is BLAH_BLAH_THINGY_CONFIG_FILE in the environment being a path to some file that the library will open to read config, instead of having the config itself in the environment (maybe it's a large unwieldy JSON thing).
It's not secure, but I'm not worried about bad actors monitoring /tmp in my test container so they can make my tests fail anyway.
It would be nice if there were a "file paths that start with a null character belong to an ad hoc in memory file system specific to the process," or something. /tmp/tmp.eJaVgxtV0x/ is close enough.
Open to code golfing suggestions on the code linked above.
Like I said there is a place for mocks. I haven't worked with the 3 tools you named, but I find that sqlite in-memory databases are easy to setup in my tests.
I don't consider the distinction between unit and integration tests useful. If the bug is in Postgress then I want to find that out, and I care if the test is technically an integration test, I care that my software doesn't work.
The platforms I work on function solely on user defined input files. For unit tests we have example configuration files stored in the application as strings (and #ifdef'd away for non unit test builds). Then we split the file loading to allow loading from memory (string/stream).
This works so well, we actually package a library with our binary that contains the unit tests so the end user can run them locally without having to download/build anything. This has been super useful in identifying calculation differences between Linux/Windows/MacOS and processors. When a user reports an issue, we can ask them to run the unit tests just by running (./binary --test) and then give us the results.
Java has in-memory file systems that are essentially geared for this exact use case, eg jimfs[0]. You create your filesystem and any files you need when your tests are starting up, and your classes talk to them rather than the “real” ones. Maybe a similar project exists for the C++ ecosystem?
I usually just set the working directory to ${CMAKE_CURRENT_SOURCE_DIR}¹ for tests that only read files, and use configure_file()² for files that could be modified or contain variables expanded by CMake. Note that you don't need to add a dependency, as configure_file() runs during configuration, and if you change the input, will copy the file again. CMake also has the notion of text fixtures for more complicated things as well, like setting up a db or other kind of service that your tests may need to run. See property FIXTURES_SETUP³ for that.
A function that both opens a file and does something with its contents should be refactored. It’s painful to find code like this and happen to have a string or pointer to some buffer that already contains the data that needs to be converted but the conversion function is buried inside a function that tries to open a file. Go’s io.Reader is so elegant at this.
As others have suggested the code should be refactored for testing so that the opening and reading of the file contents is separated from the processing of those contents, and futher separated from opening and writing out the converted text. Then each of these can be tested in isolation, with the conversion tests just specifying the input and output text inline in each test.
If you're relying on external files or an external environment then you're not writing unit tests but integration tests, since you're integrating multiple components together.
Edit:
Another option to test this sort of code is to just write a simple command line program where you pass the input and output file as parameters. Then you can use an external testing framework (like CTest) or tool (like a simple script) to run through the input text files and compare them to expected output text files.
Refactoring so that the class gets an istream or some other abstraction would be better, but when you can't change the code easily, "/dev/fd/<n>" can be an option on Linux and some other unices.
I have also seen horrific but effective test harness code that override open/close/read/write from libc.
This is a problem that Bazel (https://bazel.build) solves in a very convenient way. You can just keep using the paths relative to the repository root, and as long as you properly declare your test needs that file it will access it without problems. Or you can use the runfile libraries to access them too.
15 comments
[ 3.1 ms ] story [ 47.8 ms ] threadwe use sqlite as our datatbase so it is simple enough for helpers to apply a schema to an in memory database and again the database can be used in tests.
i'he become against mocks in tests. A usefull tool, but rarely needed. You can make tests reliable and fast without them.
https://github.com/DataDog/dd-trace-cpp/blob/6dacac2922d08f4...
It's not secure, but I'm not worried about bad actors monitoring /tmp in my test container so they can make my tests fail anyway.
It would be nice if there were a "file paths that start with a null character belong to an ad hoc in memory file system specific to the process," or something. /tmp/tmp.eJaVgxtV0x/ is close enough.
Open to code golfing suggestions on the code linked above.
Good luck with that when you’re dealing with external APIs. And numerous systems such as Postgres, redis, Kafka.
Stil possible to boot a complete environment up with docker test containers, but then your doing an integration test.
In my experience they are useful but there is no replacements for fakes inside unit tests.
I do not mock stuff (with things like mock into). I use fake implementations and interfaces.
I don't consider the distinction between unit and integration tests useful. If the bug is in Postgress then I want to find that out, and I care if the test is technically an integration test, I care that my software doesn't work.
This works so well, we actually package a library with our binary that contains the unit tests so the end user can run them locally without having to download/build anything. This has been super useful in identifying calculation differences between Linux/Windows/MacOS and processors. When a user reports an issue, we can ask them to run the unit tests just by running (./binary --test) and then give us the results.
[0] https://github.com/google/jimfs
Great for quick fixes though, it's crazy to be able to locally mount a disk image that sits on the other side of the world on an HTTP server.
1. https://cmake.org/cmake/help/latest/prop_test/WORKING_DIRECT...
2. https://cmake.org/cmake/help/latest/command/configure_file.h...
3. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_SETUP...
If you're relying on external files or an external environment then you're not writing unit tests but integration tests, since you're integrating multiple components together.
Edit: Another option to test this sort of code is to just write a simple command line program where you pass the input and output file as parameters. Then you can use an external testing framework (like CTest) or tool (like a simple script) to run through the input text files and compare them to expected output text files.
I have also seen horrific but effective test harness code that override open/close/read/write from libc.