k6 is our attempt at creating a modern, developer/devops-centric load testing tool suitable for automation. It is released as open source on Github, scriptable in JavaScript (ES6), and built on a Golang core that provides good speed and concurrency/scalability.
No, there is currently no support for websockets. We have been asked for this a couple of times though, and are considering bumping up the priority for a websockets implementation.
As of now, k6 can do standard http, https (and it also supports HTTP/2)
This looks incredible for work I was previously using Tsung / Gatling for.
One request: please allow the output of either a) a CSV log of every request and it's timing or b) a configurable histogram of request timing, not just some percentiles and averages.
That info is pretty much required for doing any more advanced analysis beyond "the service is fast enough".
https://github.com/tsenart/vegeta does this really well (but doesn't have the scripting abilities of this tool).
That looks like what I'm looking for, thanks! Is there documentation of the format of those JSON logs? CSV would probably be a useful addition just for ease of ingestion into analysis tools and databases.
Once I get a chance to test this out, I'll definitely report any issues through GH.
I'll update the docs with some info on the JSON format first thing on monday!
If you should happen to try out k6 in the meantime, most of the JSON contents should be recognizable/understandable just by looking at the output, if you know what metrics k6 collects (which is documented on docs.k6.io). You can try running with e.g. 2 VUs for 2 iterations, to get a tiny amount of output data but something that looks like a real load test: k6 run --vus 2 --iterations 2 --output json=output.json script.js
The single biggest difference is that k6 lets you write test cases in real code (JavaScript), while Vegeta will only let you supply a static list of URLs to hit. Vegeta has some things k6 does not have, to be fair (like constant RPS-rate load), but k6 was created to address what we saw as missing in Vegeta and most of the other existing tools. We think developers want to write code to test things, so we created a tool that lets you use JavaScript + a scripting API designed for load testing and automation.
But you can actually aggregate results already, if you use InfluxDB as results storage: you can have multiple k6 instances push results data to the same InfluxDB instance. Results are just time series samples, so there are basically no bad effects if you do this - you get all the results in a single db, with correct time stamps. The only difficulty lies in orchestrating the multiple k6 instances so they're in sync and you get the traffic levels you want (i.e. they ramp up/down at the same time). But for a simple ramp configuration, or a config where you run at a stable traffic level, it shouldn't be a problem.
I knew it would be unpopular. I get a few u popular posts a year.
But I am very, very irritated by Golang while I am working with it on a daily basis.
It's entire design is informed by assuming that no developer should understand the codebase, but should instead read it over and over. Symmetrically, no insight can be offered in the code, only a rote transcription of an inductive design.
The net result is a language hostile to both those writing and those reading the code. No shortcuts are offered or taken. All error handling must be done manually. All data structures must be reimplemented over and over with different test suites for each one.
Thanks, I'll make sure the "params" object gets documented also. Currently, there is not so much you can set per HTTP request. You can basically add HTTP headers and you can also add tags that lets you single out specific results based on those tags (e.g. when using thresholds or when plotting graphs of results).
So - HTTP request timeouts can not be changed at the moment, but this is something I think would be very simple to implement. If you open an issue about it I'll see what we can do to get it implemented quickly.
25 comments
[ 1.7 ms ] story [ 98.0 ms ] threadAs of now, k6 can do standard http, https (and it also supports HTTP/2)
We'd really appreciate any comments/feedback you may have on exactly how it should be implemented.
One request: please allow the output of either a) a CSV log of every request and it's timing or b) a configurable histogram of request timing, not just some percentiles and averages. That info is pretty much required for doing any more advanced analysis beyond "the service is fast enough". https://github.com/tsenart/vegeta does this really well (but doesn't have the scripting abilities of this tool).
Is JSON useful to you, or do you prefer CSV?
Once I get a chance to test this out, I'll definitely report any issues through GH.
If you should happen to try out k6 in the meantime, most of the JSON contents should be recognizable/understandable just by looking at the output, if you know what metrics k6 collects (which is documented on docs.k6.io). You can try running with e.g. 2 VUs for 2 iterations, to get a tiny amount of output data but something that looks like a real load test: k6 run --vus 2 --iterations 2 --output json=output.json script.js
Had anyone used both to point out the main differences from k6?
The single biggest difference is that k6 lets you write test cases in real code (JavaScript), while Vegeta will only let you supply a static list of URLs to hit. Vegeta has some things k6 does not have, to be fair (like constant RPS-rate load), but k6 was created to address what we saw as missing in Vegeta and most of the other existing tools. We think developers want to write code to test things, so we created a tool that lets you use JavaScript + a scripting API designed for load testing and automation.
But you can actually aggregate results already, if you use InfluxDB as results storage: you can have multiple k6 instances push results data to the same InfluxDB instance. Results are just time series samples, so there are basically no bad effects if you do this - you get all the results in a single db, with correct time stamps. The only difficulty lies in orchestrating the multiple k6 instances so they're in sync and you get the traffic levels you want (i.e. they ramp up/down at the same time). But for a simple ramp configuration, or a config where you run at a stable traffic level, it shouldn't be a problem.
But I am very, very irritated by Golang while I am working with it on a daily basis.
It's entire design is informed by assuming that no developer should understand the codebase, but should instead read it over and over. Symmetrically, no insight can be offered in the code, only a rote transcription of an inductive design.
The net result is a language hostile to both those writing and those reading the code. No shortcuts are offered or taken. All error handling must be done manually. All data structures must be reimplemented over and over with different test suites for each one.
So - HTTP request timeouts can not be changed at the moment, but this is something I think would be very simple to implement. If you open an issue about it I'll see what we can do to get it implemented quickly.