HN Explorer

Show HN: A Swiss army knife for testing HTTP from the terminal

110 points by hassy ↗ HN
Hiya HN!

Just released Artillery Probe - a Swiss army knife for testing HTTP from the CLI. Think mini-curl with better UX for common use-cases, plus a couple of extra features.

Would love for you to try it and give some feedback!

https://www.artillery.io/blog/swiss-army-knife-for-http-testing

and:

  npm install artillery@latest
What does it do?

- First of all, it's a HTTP client! It does all the usual stuff you'd expect from a HTTP client... HTTP methods, request bodies, custom headers, forms, Basic Auth etc.

- Got some JSON or XML back? It'll pretty-print it, and syntax highlight it for you.

- It'll show you request waterfalls like this one: (inspired by httpstat)

   DNS Lookup | TCP Connection | SSL Handshake | Time to First Byte | Content Transfer
    56ms      |  14ms          |     19ms      |     181ms          |   88ms          |
              |                |               |                    |                 |
              56ms             |               |                    |                 |
                            70ms               |                    |                 |
                                             89ms                   |                 |
                                                                  270ms               |
                                                                                 total:358ms
- JSON responses can be queried and sliced and diced with JMESPath (same syntax as AWS CLI) - no need to reach for jq. XML & HTML may be queried with a jQuery-like syntax too.

- You can set expectations on the response, e.g. have the CLI check that the response is a 200, or that a certain header is set, and exit with non-zero code if not. Super handy for quick acceptance testing.

We've got lots of ideas for improvements, but would love to hear what you think!

46 comments

[ 1.8 ms ] story [ 108 ms ] thread
jsd1982
> npm install

Sorry, lost me at this part. Have a better installation story like a single executable.

Beyond that, what does this do that curl, wget, et al do not? How is your interface better than those? Is it robust in the face of errors? What http protocol versions does it support?

hassy
if you’re on a Mac, Artillery has a Homebrew formula too. We have an official Docker image too. Other ways of installing are on the roadmap too (self contained tarballs and binaries).

As to what it does better than curl/wget. Probe is geared towards interactive use with HTTP APIs. So you get syntax highlighting and pretty printing for JSON, built-in querying, request waterfall visualizations, and ability to set expectations on every response.

HTTP/2 is supported.

jsd1982
https://www.artillery.io/blog/swiss-army-knife-for-http-test...

Does artillery properly escape the environment variables into json strings? I don't see how it could if you're using bash expansion in double quotes. This feature would only work with string values without spaces or special characters. Don't really see any improvement here.

https://www.artillery.io/blog/swiss-army-knife-for-http-test...

So by embedding a json query evaluator in your http client, this means that every time I iterate on my JMESPath query I'm making a new request? Is there an offline mode? What if my Json file is 200MiB? Or if the request was not idempotent?

Just saying there's good reasons why json query features are not typically built into one-shot http request tools. However, if your tool was not one-shotand you had an interactive session and cached/stored the entire result and then allow the user to interactively query it, then I could see some better value here.

hassy
Yeah so the JSON quoting part is something I’m pretty pleased with. We use a YAML parser (JSON is a subset of YAML) to parse those values, which is what allows for double quotes to be omitted.

Good point on iterating on the query! We already save the body into a temp file, so we can make Probe be able to run queries on a file. Adding it to the todo list. :)

(In my own workflow I use gron a lot for getting an overview of the shape of unfamiliar JSON, super handy tool)

As to an interactive shell… yes, 100%. Kicking ideas around something like that as well!

mdaniel
> We use a YAML parser (JSON is a subset of YAML)

I believe that has more sharp edges than one might suspect

      --json "{username: testuser, password: testpassword, birthdate: 2022-05-10}"

    ...,"birthdate":"2022-05-10T00:00:00.000Z"}
The user may be less surprised if the flag were named "--yaml-as-json" or whatever DX friendly variant one wishes, in order to more clearly mark that string as yaml. The user is actually told about the "JSON or YAML", but only if they try to include some scalar value that isn't legal yaml

Also, I appreciate that every service should strive to respond in 10s, but there wasn't an obvious way to get it to be more patient

    RequestError: Timeout awaiting 'request' for 10000ms
        at ClientRequest.<anonymous> (/home/node/node_modules/got/dist/source/core/index.js:962:65)
followed by a ginormous stacktrace and a console.error dump of some event object
hassy
thanks for trying it out! adding “make timeouts configurable” to the todo list.

you’re right on that YAML-as-JSON thing. If everything is quoted as JSON, those type conversions shouldn’t kick in. Otherwise there’s room for surprises - perhaps we can do something to make those cases more obvious.

guacom
> Have a better installation story like a single executable.

Holy guacamole this is an entitled sentiment. It’s an open source project and it’s free.

jsd1982
Feedback was explicitly requested. That was my feedback on installation. It's not entitled and has nothing to do with the project being free or open source. Where'd that come from?
Kaze404
Probably from the way it was worded.
Eduard
What makes this a worse "installation story" than e.g.

apt install, pip install, git clone && ./configure && make && make install?

ushakov
> what does this do that curl, wget, et al do not

they don’t get VC-funded by YCombinator, do they?

/s

[deleted]
(comment deleted)
dj_mc_merlin
Nice project. How are the request waterfalls done?
ovebepari
It would take me something extraordinary to move away from curl
blacksmith_tb
Sure, if you've got muscle memory for all the flags (plus it lets you reason about all the other programs that pipe stuff to curl). I personally find myself using httpie [1] at least as much.

1: https://httpie.io/

[deleted]
(comment deleted)
ratorx
I think curlie is the middle ground and the best (IMO) in the space. A nicer CLI for interactive use and can use and output curl flags for scripting.
jicea
As a maintainer, I feel obligated to mention Hurl [1], a tool for testing HTTP with plain text and curl. It’s a wrapper around libcurl, in a single binary, with syntactic sugar for asserts. You can use it also like curl to give you an HTTP output.

[1] https://hurl.dev

hassy
I haven’t come across Hurl before. It looks cool!
hassy
curl is amazing! I love curl. This is not meant to replace it completely. Probe just makes those most common use-cases friendlier, like looking at headers, or inspecting JSON retuned by an API with syntax highlighting, or querying without needing to reach for jq etc.
chrismarlow9
The big win with curl is the growing adoption of "copy as curl" in browser dev tools as well as "example of API call in curl" for third party API documentation.

I don't think I could move to anything else for that reason alone.

[deleted]
(comment deleted)
arinlen
Other than pretty printing the output, what does this offer that curl doesn't already do?
hassy
A few things :)

Syntax highlighting

Request performance waterfalls

Built-in JMESPath queries for JSON (same syntax as in AWS CLI). XML and HTML can be queried too.

Ability to set checks on the response. This one is super handy for quick acceptance tests.

ushakov
does this require Node.JS?

edit: it looks like you need to install 1276 npm dependencies to use this tool

https://github.com/artilleryio/artillery/network/dependencie...

i'll pass, the risk is unacceptable for value it offers, especially in corporate environments

jsd1982
This is essentially what I was referring to with my comment below. If you can improve the installation experience such that I don't have to care that you developed with node.js, so much the better. If I were a node.js developer I surely wouldn't want my global package name space polluted like that.

npm is not for end users.

ushakov
the solution is not write CLIs in JavaScript, unless you have a JS-library that you also want to make available as a CLI

packaging Node in a binary won't solve the problem of JavaScript's plagued ecosystem and in fact i'd rather prefer to be able to mitigate the issues/update packages myself rather than waiting for a new binary to release

jsd1982
I agree with your conclusion to not write cli tools in Javascript.

However, packaging node in a binary is pretty much what electron does today, albeit with a GUI.

Regardless, it's not a problem I wish to spend any effort to solve since it's not my problem and I've (we've) already dismissed the premise (writing CLI tools in js) lol.

spoiler
Another option is using Deno!
ushakov
honest question to the authors: why cannot this tool not be written in JavaScript?

what technically prevents you from designing the tool independent of the Node.JS and JavaScript ecosystem?

throwawayboise
Maybe it's what they know. Common advice for startups is to not get mired in language/framework debates: use what you already know.
hassy
sure, that’s a valid concern in some environments. fwiw we use Snyk.io for dependency scanning
ushakov
no, this is a valid concern in all environments, but in some environments this is more of a concern

> Snyk.io for dependency scanning

so what? does the CLI update itself automatically?

hassy
Yes, those dependency trees can be large. Yes, supply chain attacks are a real threat. But Node isn’t that different than Python or Ruby in that regard. How far down the stack do you personally choose to go? I trust you’re familiar with that famous paper published by a certain mr Thompson in the mid-80s?

The world is a big place. There’s a lot of software out there written in Node.js, used happily and productively by millions of developers, many of them in corporate environments.

Given the opinions you expressed elsewhere in the thread here I think it’s clear that this tool is not for you. I hope no one is forcing you to use it.

[deleted]
(comment deleted)
heywoodlh
> Think mini-curl with better UX for common use-cases, plus a couple of extra features.

> it looks like you need to install 1276 npm dependencies to use this tool

Sincere question (probably highlights my lack of understanding Node package management): how can this be a "mini" curl with that many dependencies?

hassy
“mini” in the sense that it doesn’t do everything that curl does. curl does a whole lot. this tool focuses on more common use cases and makes them friendlier. plus with request waterfalls and assertions it does things curl can’t do.

the number of packages is a bit of a misnomer anyway. Artillery Probe is part of Artillery which does load testing, with multiple protocols, support for multi-step scenarios, publishing to a variety of monitoring systems (Datadog, Prometheus etc) and more. That’s what most of those packages enable.

heywoodlh
I see. Thanks for the info and thanks for creating/sharing a great tool!
Uptrenda
1276 deps... how is that even possible for a simple HTTP fetcher. Does node have no standard library? actually amazing.
numlock86
Nice idea. NodeJS and thousands of NPM dependencies are a big no-go with a red flag on top, though. One of the things something like Go probably would have been a better choice for. See "bombardier" for example.
hassy
Bombardier is cool, but serves a very different use case.

I mentioned elsewhere in the comments that we have a Docker image, and are working on other methods of installing the CLI to alleviate some of these dependency-related concerns.

Getting side tracked here, but there seems to be a common sentiment when it comes to Node.js that it's uniquely insecure. Node.js has indeed had some unfortunate press when it comes to supply-chain security, but every other runtime is susceptible to those attacks (PiPy, Gems, Maven, Rust Crates). Ultimately of course, if you choose to avoid using any software built on top of those stacks, that's your choice.

Artillery specifically is no different to any other Node.js-based project in how large the dependency tree is. VSCode for instance is used by millions of developers has 1.6k dependencies [1].

1. https://github.com/microsoft/vscode/network/dependencies

numlock86
> Bombardier is cool, but serves a very different use case.

Yes, of course. I was just taking an arbitrary example from my common HTTP toolbox.

> VSCode for instance is used by millions of developers has 1.6k dependencies [1].

VSCode is maintained by Microsoft plus a huge community and they are transparent about their process in regards of auditing/freezing dependencies.