IIRC, ab (ApacheBench) is still vulnerable to Coordinated Omission which makes its 95%ile latency measurements useless. wrk2 solves that
JMeter has a lot more features like following links instead of just pulling the same URLs over and over, but it’s also very resource intensive. On a typical dev machine (32G 5800x3d) it is very slow and crashes when running at an absurdly low rate like 1000 requests/second. So unless your website is hosted on a Raspberry Pi, you’d have to spin up 100 cloud VMs to do a real load test. At that point running full browsers with Selenium starts to sound reasonable.
At BrowserUp we built the first DRY (don't repeat yourself) load testing tool. The rough idea:
* Most orgs take a sprint to write load tests in a separate tool/language (Jmeter, for example).
* Re-using integration and end-to-end tests or even Postman requests for load lets you release fully tested software sooner
* Let your teams use the language and tool they know to write the tests, then we containerize and scale that in your EC2 account, because by-the-minute hardware is cheap, dev-hours spent in Jmeter are expensive.
* The dev teams/QA folk can do testing rather than waiting for the "Jmeter" specialist
you can check the premise of EaaS to help you with on-demand, isolated testing environments where you can break things with load testing. the cool thing is that it can replicate production-like setups so tests are predictable. Hope this helps. (disclaimer, I work for an EaaS company, www.bunnyshell.com)
7 comments
[ 84.1 ms ] story [ 320 ms ] threadUnfortunately, haven't used it beyond demo. Yet.
JMeter has a lot more features like following links instead of just pulling the same URLs over and over, but it’s also very resource intensive. On a typical dev machine (32G 5800x3d) it is very slow and crashes when running at an absurdly low rate like 1000 requests/second. So unless your website is hosted on a Raspberry Pi, you’d have to spin up 100 cloud VMs to do a real load test. At that point running full browsers with Selenium starts to sound reasonable.
* Most orgs take a sprint to write load tests in a separate tool/language (Jmeter, for example).
* Re-using integration and end-to-end tests or even Postman requests for load lets you release fully tested software sooner
* Let your teams use the language and tool they know to write the tests, then we containerize and scale that in your EC2 account, because by-the-minute hardware is cheap, dev-hours spent in Jmeter are expensive.
* The dev teams/QA folk can do testing rather than waiting for the "Jmeter" specialist
Let me know if you would like a demo
https://k6.io/
The load tests are written in JavaScript, the api is super clean. It’s command line only, but the command line output is really readable.
I’m sure it doesn’t cover all the edge cases of Jmeter, but for testing normal apis it seems pretty perfect.