Snapshots diff current with previous output and I only have to accept or reject the diff. I don't have to write the expected response myself. Snapshots can also stub out parts of the response that are not determistic.
> Snapshots can also stub out parts of the response that are not determistic.
TIL! The way I knew to do it was to have a mock implementation that behaved like the real thing, expect for data/time/uuids/..., where there was just a placeholder.
Snapshot tests being able to "mask" those non-deterministic parts sounds cool!
tbh, that seems pretty close to what I would call snapshot testing already. What people usually do with it is using it for more broadly compared to API testing (for example, I currently use it to test snapshots of a TUI application I am developing) - i.e. you can use it whenever your test is of the form "I have something that I can print in some way and that should look the same until I explicitely want it to look differently in the future". There are a bit more bells and wizzles - For example, it is nice that one does not have to write the initial snapshots oneself. You write the test, execute, it creates the resulting file, then you review and commit that - handy little workflow!
Obviously better IDEs integration, support for gRPC, Websocket would be very cool.
A favorite of mine is to be available through official `apt`: there has been some work but it's kind of stuck. The Debian integration is the more difficult integration we have to deal. It's not Debian fault, there are a lot of documentation but we've struggled a lot and fail to understand the process.
So, myself and many folks I know have taken to writing tests in the form of ".http" files that can be executed by IDE extensions in VS Code/IDEA.
Those basically go in the form
POST http://localhost:8080/api/foo
Content-Type: application/json
{ "some": "body" }
And then we have a 1-to-1 mapping of "expected.json" outputs for integration tests.
We use a bespoke bash script to run these .http file with cURL, and then compare the outputs with jq, log success/failure to console, and write "actual.json"
Can I use HURL in a similar way? Essentially an IDE-runnable example HTTP request that references a JSON file as the expected output?
And then run HURL over a directory of these files?
You can use hurl in this way. I have projects with a test directory of hurl files, one hurl file per test case. The cases can run one or more http requests. The hurl file can reference external files, capture values from responses for subsequent requests, validate status and outputs. Hurl has various test runner modes and will optionally output overall test results in various parsable formats if you have a larger reporting framework that you would like to hook into.
(After I have seen the IntelliJ one from a colleague I was searching for one like that in neovim. That's the best one I found. It's not perfect, but it works.
Edit: The tool from OP looks very neat though. I will try it out. Might be a handy thing for a few prepared tests that I run frequently
https://github.com/mistweaverco/kulala.nvim is an another restish (it can do gRPC to) plugin for neovim. It is intended to be compatible with a Jetbrains as much as possible.
It’s not nearly as lightweight, and one of the major dealbreakers for postman and equivalents (even ignoring all the drama with postman) is that you have to import and export the data in the client in order to get some text file you can just commit to repo. For my team, that’s a dealbreaker, because it means that people write entire suites of stuff, and never commit them, meaning other people end up doing the same work over and over.
Ahhh, ok. I think I’m thinking about Insomnia, which is basically (in fact?) a fork of Postman. Anyways, that fact is what made Postman a dealbreaker for me, even before the drama. Another thing I like about Rest Client is that the configuration is just a text file, so bearer token etc can be updated via script that runs in a loop.
Rest Client has a few cons though, like request chaining.
I was using Rest Client and was very happy with it, but once I needed Rest Client to use my computer's NO_PROXY env variable to avoid using the proxy for a certain url, and I found it was not possible to do that with Rest Client. That's the only reason I had to look for an alternative tool. After an analysis, I liked Bruno and Hurl. I didn't try hurl yet.
yep, I've played with Hurl and find it nice but recently have been leaning into the .http stuff more. IntelliJ has it built in, there's the plugin you linked, and then for CLI i've used httpYac. No "vendor lock in", really easy to share with copy & paste or source control.
Is that POST in the readme sending the password in the query params? Is this shorthand or literally adding them to the params?
I don't really feel the need for a curl replacement. In the past I've used httpie which is pretty slick but I end up falling back to writing tests in python using requests library.
Maybe I'm not the target audience here, but I should still say something nice I guess. It's nice that it's written in Rust, and open source tooling is in need of fresh projects ever since everyone started bunkering up against the AI monolith scraping all their work. We should celebrate this kind of project, I just wish I had a use for it.
I see it more as a Postman replacement than curl. When I’m working on a set of APIs, I can quickly write a Hurl file with different combinations that I’m working on. There are usually editor integrations to run individual requests. Then I can share the same Hurl file to my team or commit it in the repo.
The POST in the README is going to send the params in the request body "url form encoded" like a form in a web page. There are more samples on the doc site [1].
Regarding curl, Hurl is just adding some syntax to pass data from request to request and add assert to responses. For a one time send & forget request, curl is the way, but if you've a kind of workflow (like accessing an authentified resource) Hurl is worth a try. Hurl uses libcurl under the hood and you've an option `--curl` to get a list of curl commands.
> The POST in the README is going to send the params in the request body "url form encoded" like a form in a web page.
Is there a different POST request in the readme or are you saying that this example is going to send the "user" and "password" params in the request body?
That seems really surprising to me - how would you then send a POST request that includes query string parameters? The documentation on form parameters [1] suggests there's an explicit syntax for sending form-encoded request parameters
If I want to open a modal & check something, could it simulate clicking on open modal button? Or is it first level for now (without any support for interactions / javascript magic)
Hurl works only on the HTTP layer, there is no JavaScript engine. If your modal open a form that will trigger some kind of XHR, you'll "simulate" the HTTP traquets that the form modal could have done. Hurl is not a kind of Playwright for instance.
Yeah love Hurl, we stared using it back in 2023-09.
We had a test suite using Runscope, I hated that changes weren't versioned controlled. Took a little grunt work and I converted them in Hurl (where were you AI?) and got rid of Runscope.
Now we can see who made what change when and why. It's great.
Can you share "fixtures" amongst tests? For example, I wouldn't want to have to copy/paste the signup and login process for each type of user across hundreds of tests.
What about test isolation? Are people using something else to "prime" the service before/after running these tests?
This looks interesting. Longtime user of the Vscode-restclient, but have been moving over to httpyac lately for the scripting and cli use. Will take a look to see if hurl is a good fit.
One annoying thing I've found in testing these tools is that a standard hasn't emerged for using the results of one request as input for another in the syntax of `.http` files. These three tools for instance have three different ways of doing it:
* hurl uses `[Captures]`
* Vscode-restclient does it by referencing request names in a variable declaration (like: `@token = {{loginAPI.response.body.token}}`).
* While httpyac uses `@ref` syntax.
From a quick round of testing it seems like using the syntax for one might break the other tools.
Guilty to have created yet-another-format for HTTP client! To "mitigate" this issue, you can use `hurlfmt` (distributed along `hurl`) that would allow you to export a Hurl file to JSON. You could then go from this JSON to another... It's not magic but it can help if you're going to change from Hurl to another thing.
No worries, it's also interesting to see different peoples approaches to the best syntax for this. Exporters/importers do make life a bit easier I suppose.
I don't know what the mechanism/incentive for getting a standard would be either. Probably most likely would be if there was one clear "winner" that everyone else felt the need mirror.
In any case, appreciate the reply and the tool. Good luck with it.
This is interesting. I’m wondering how programmable this is. Would this project (or any similar ones) be able to POST a json payload with a field set to “now()”?
Is there any program code that is not based on plain text? Punch cards maybe? For the value of "plain text" that includes a programming language with its own syntax and grammar, like Hurl.
118 comments
[ 2.5 ms ] story [ 187 ms ] threadTIL! The way I knew to do it was to have a mock implementation that behaved like the real thing, expect for data/time/uuids/..., where there was just a placeholder. Snapshot tests being able to "mask" those non-deterministic parts sounds cool!
Where do you see hurl in the next 2 years?
A favorite of mine is to be available through official `apt`: there has been some work but it's kind of stuck. The Debian integration is the more difficult integration we have to deal. It's not Debian fault, there are a lot of documentation but we've struggled a lot and fail to understand the process.
[1]: https://github.com/Orange-OpenSource/hurl/issues/366
If I find time I could throw a spec file + ci/cd workflow to get you going that way too
Those basically go in the form
And then we have a 1-to-1 mapping of "expected.json" outputs for integration tests.We use a bespoke bash script to run these .http file with cURL, and then compare the outputs with jq, log success/failure to console, and write "actual.json"
Can I use HURL in a similar way? Essentially an IDE-runnable example HTTP request that references a JSON file as the expected output?
And then run HURL over a directory of these files?
Is your expected.json the actual response body, or is it an object containing body, status, header values, and time-taken, etc?
I really like it because it serves 3 purposes:
- API docs/examples that you can interact with
- Test cases
- Manually invoking API endpoints when working on the underlying code, in an iterative loop
If that's possible, I guess the only thing I'd request is interopability with the REST Client ".http" files that VS Code/JetBrains IDE's support then.
UPDATE: Found it, looks like you can do it via the below
So that just leaves the IDE integration bit.https://marketplace.visualstudio.com/items?itemName=humao.re...
Which is a banger VS Code extension for all sorts of http xyz testing.
(After I have seen the IntelliJ one from a colleague I was searching for one like that in neovim. That's the best one I found. It's not perfect, but it works.
Edit: The tool from OP looks very neat though. I will try it out. Might be a handy thing for a few prepared tests that I run frequently
It is targeted toward more postman crowd though. May not be as lightweight.
Rest Client has a few cons though, like request chaining.
I was using Rest Client and was very happy with it, but once I needed Rest Client to use my computer's NO_PROXY env variable to avoid using the proxy for a certain url, and I found it was not possible to do that with Rest Client. That's the only reason I had to look for an alternative tool. After an analysis, I liked Bruno and Hurl. I didn't try hurl yet.
[1]: https://hurl.dev/docs/manual.html#netrc
https://blog.jetbrains.com/idea/2022/12/http-client-cli-run-...
I don't really feel the need for a curl replacement. In the past I've used httpie which is pretty slick but I end up falling back to writing tests in python using requests library.
Maybe I'm not the target audience here, but I should still say something nice I guess. It's nice that it's written in Rust, and open source tooling is in need of fresh projects ever since everyone started bunkering up against the AI monolith scraping all their work. We should celebrate this kind of project, I just wish I had a use for it.
Regarding curl, Hurl is just adding some syntax to pass data from request to request and add assert to responses. For a one time send & forget request, curl is the way, but if you've a kind of workflow (like accessing an authentified resource) Hurl is worth a try. Hurl uses libcurl under the hood and you've an option `--curl` to get a list of curl commands.
[1]: https://hurl.dev/docs/samples.html
Is there a different POST request in the readme or are you saying that this example is going to send the "user" and "password" params in the request body?
> POST https://example.org/login?user=toto&password=1234
That seems really surprising to me - how would you then send a POST request that includes query string parameters? The documentation on form parameters [1] suggests there's an explicit syntax for sending form-encoded request parameters
[1]: https://hurl.dev/docs/request.html#form-parameters
[1]: https://github.com/Orange-OpenSource/hurl/issues/4151
We had a test suite using Runscope, I hated that changes weren't versioned controlled. Took a little grunt work and I converted them in Hurl (where were you AI?) and got rid of Runscope.
Now we can see who made what change when and why. It's great.
Loved Runscope it served it's purpose until something came along that that offered the same + version control.
première fois que je vois qqch de cool sortir d'orange.
The deficiencies in huel with client state management is not easy to fix.
What I'd like is full client state control with better variable management and use.
For my last project I used Python to write the tests, which appears to work well initially. Dunno how well it will hold up for ongoing maintenance.
It gives you full control of constructing requests and assertions because test scenarios may include arbitrary JavaScript.
What about test isolation? Are people using something else to "prime" the service before/after running these tests?
One annoying thing I've found in testing these tools is that a standard hasn't emerged for using the results of one request as input for another in the syntax of `.http` files. These three tools for instance have three different ways of doing it:
* hurl uses `[Captures]`
* Vscode-restclient does it by referencing request names in a variable declaration (like: `@token = {{loginAPI.response.body.token}}`).
* While httpyac uses `@ref` syntax.
From a quick round of testing it seems like using the syntax for one might break the other tools.
[1]: https://hurl.dev/docs/capturing-response.html
[2]: https://github.com/Huachao/vscode-restclient
[3]: https://httpyac.github.io/guide/metaData.html#ref-and-forcer...
I don't know what the mechanism/incentive for getting a standard would be either. Probably most likely would be if there was one clear "winner" that everyone else felt the need mirror.
In any case, appreciate the reply and the tool. Good luck with it.
Conway's Law in action, ladies and gentlemen.