Show HN: Calibre Test Profiles – Test your sites under real world conditions (calibreapp.com)
Hi HN! I'm Ben, the solo founder of Calibre Analytics: https://calibreapp.com. Calibre monitors websites and applications in a clean-room testing environment from all over the world. Up until January Calibre has been an "inbetween other things" / evenings / holidays kind of project. (Only, oops—I did that for four years.). Last year I knuckled down to get into the position where I could work on it full time.
I've spent the last 8 weeks working on "Test profiles" (The launch post is here: https://medium.com/get-calibre/announcing-test-profiles-and-...) — It allows you to emulate hardware devices (cpu speed, user-agents and screen dimensions), as well as limit the bandwidth (latency, download and upload speeds), and set cookies (you could use this to log into your app, test a staff-shipped feature, or maybe turn off advertising and see the performance impact.) I think I've finally got the basis of a really powerful tool, and I'm keen to share it with… everyone! I do everything myself, so if you've any questions, I'm here to answer them!
18 comments
[ 6.5 ms ] story [ 72.1 ms ] threadLast year I knuckled down to get into the position where I could work on it full time. I've spent the last 8 weeks working on "Test profiles" (The launch post is here: https://medium.com/get-calibre/announcing-test-profiles-and-...) — It allows you to emulate hardware devices (cpu speed, user-agents and screen dimensions), as well as limit the bandwidth (latency, download and upload speeds), and set cookies (you could use this to log into your app, test a staff-shipped feature, or maybe turn off advertising and see the performance impact.)
I think I've finally got the basis of a really powerful tool, and I'm keen to share it with… everyone! I do everything myself, so if you've any questions, I'm here to answer them!
Btw congrats. This looks very well done.
With Calibre the aim is to make it a part of your toolchain (read: integrations) — so it should be visible to your entire team, or build system, or slack channel.
The App itself is boring old rails, with postgres (get me started on how much I like postgres and I will literally bore you to tears.) The original views were all haml and sass delivered via the asset pipeline. I sprinkled in angular here or there as the application grew.
Today, I'm slowly tearing away the angular. I've moved to webpack for JavaScript, and React for all new UIs. I've been really enjoying both the speed of development, and how solid everything feels.
Over time, I'm writing less and less ActiveRecord, and more and more sequel… still learning, but getting better.
The app is run on Heroku, and its been A++ would recommend to everyone.
The Calibre agent is node and websockets talking to Chrome®, theres other bits of ruby, and even some golang now.
I use terraform and ansible (although, the ansible may as well have been bash scripts) through a buildkite (https://buildkite.com — the best fucking CI service on the planet) pipeline. AMIs are built, shipped to different regions, then based on some config are rolled out.
Papertrail for logs - omg, logging! Essential. I run thousands of tests per day (I don't know this number off the top of my head, but its a lot)… and the nature of the internet is THERE IS ALWAYS FAILURE. Having logging at my fingertips has always made my job building Calibre easier. I think I pay $20 per month right now, and its the most important thing I pay for.
--
Ok. Whew. Could probably keep going, but I'm not sure how much you wanna know. Happy to answer specific things though!
I do everything. No contractors. Although @fox has been recently helping me with my writing (her native language is Polish, and she writes waayyy better than me - and most others).
The biggest thing that sets Calibre apart is that its entirely built using the Chrome developer tools APIs. I like to think about it as someone on your team who opened devtools after every deploy, and kept a neat spreadsheet of all the changes that had occurred. In short, all the data is ready and available for the taking.
This also has the added benefit that I have the power of the Chrome browser to play with. Projects like Google's Lighthouse (https://developers.google.com/web/tools/lighthouse/ — Which I'm currently integrating into Calibre!) have meant that a lot more powerful tools/metrics/resources/etc are coming along strongly, and this will only help make Calibre more powerful too.
Calibre is firmly focused on what your users actually experience, and helping you get to the bottom of 'why' quickly and efficiently. So while technically focused tools are imperative for any team, many of them aren't focused on the most important people in the equation.
Also, thanks — Good question.
The site looks really good. This is a very impressive product for an "in-between other things"/evenings/holidays kind of project. My only issue is that the animation of the application window coming up can sometimes be laggy.
0: https://calibre-ebook.com/
Edit: Thank you! :-)
I quite enjoyed watching this brain storm session with Paul Irish around Lighthouse and new metrics for measuring website performance in a meaningful way.
Edit: Forgot to include the link -> https://www.youtube.com/watch?v=IxXGMesq_8s