HN Explorer

91 comments

[ 4.1 ms ] story [ 109 ms ] thread
schickling
I'm really excited to finally open-source Chromeless. We've used NightmareJS and similar tools before to run integration tests but these basically added ~20min to each build. With Chromeless we were able to reduce this time to under a minute!

Here is btw a demo playground to try it out: https://chromeless.netlify.com/

Let me know if you have any questions :)

ivan_ah
How difficult would it be to also support google cloud functions and the azure offerings? This seems like a really useful standard tool that lots of people might want to use. CI jobs on pull requests that take seconds instead of minutes = big win!
schickling
This should actually be pretty easy. There is no reason that this is bound to AWS Lambda. We're more than open to accept PRs that enable multi-cloud support.
djhworld
How did you manage to get Chrome headless compiled and running on AWS?

I spent a while looking at this a few months ago to do a similar project (run chrome in headless mode on AWS Lambda) but couldn't get the binary to work.

yamafaktory
Looks super promising, the API is really neat and running the tests in parallel is a big plus! Awesome work!
shortj
This would have been awesome to have back when I was heavy in to the UI test automation game. Our best option at that point was a spot instance EC2 fleet and analyzing commits to determine which tests would be the most valuable to run. It's awesome being able to easily run hundreds or thousands of tests in parallel, completely segmented, and pay only on demand. A fantastic use of AWS Lambda! It suddenly becomes reasonable to do full integration tests on every merge request, or even commit, and get feedback to the developer in seconds.
schickling
This sounds very familiar. I'm amazed every time to see what serverless infrastructure can be used for!
toomuchtodo
Do the math. If you're suggesting running all the tests, drastically increasing your compute volume, it's probably cheaper to use a dedicated instance running at max CPU utilization.
schickling
It's of course based on what's more important for you: Running tests non-stop and perfectly utilizing a compute instance vs having the tests executed when you need the results as soon as possible. I might argue that in most cases the latter is what you'd actually want. Especially given that you're getting billed on a millisecond basis.
[deleted]
(comment deleted)
sorenbs
I would love to have something like this for my scala API tests. We have 3k tests running on 10 ec2 instances and the entire test-suite takes 10 minutes. If some infrastructure would allow me to pay a little more and run all tests in parallel, that would be a game changer for our team.
shortj
It absolutely depends on how burst driven your utilization pattern is, but don't forget the cost to actually manage the test cluster, and the engineering cycles that go into optimizing and sharding those tests across the cluster.
toomuchtodo
For sure, lots of considerations to take into account. If you're already running your own CI pipeline though, tests on top of it are trivial (in my experience running a Jenkins CI pipeline for a python monolith). Tweak your test runner, upsize the EC2 instance, and go grab a mojito.
jotto
(Shameless promoting): for an API version of the prerendering functionality, with no warm-up latency, we're running a large cluster of Chrome headless instances here: https://www.prerender.cloud/
schickling
Sounds like a great service and very useful for frontend developers.

We've built something similar internally and will shortly migrate it to Chromeless. We basically use it to pre-render our websites and docs: https://github.com/graphcool/prep

polskibus
Can I use this somehow in a container? Farm out headless chrome + selenium on a local data center? I would be grateful for any hints.
[dead] polskibus
Can I use this somehow in a container? Farm out headless chrome + selenium on a local data center? I would be grateful for any hints.
victorhooi
So to clarify - this is basically a Node.JS wrapper around Chrome headless, right? =_)

Seems pretty awesome.

My use case is to take screenshots of various pages - the docs don't mention the default viewport dimensions, btw.

schickling
Yes exactly + this can run (in parallel) on AWS Lambda, so you don't need to worry about provisioning & running servers. That's actually the part I'm most excited about :)
purvis
It's parallel because AWS Lambda is inherently parallel? Or are you referring to within JavaScript?
adieuadieu
Parallel in that you can invoke many Lambda functions at the same time and have them run independently of each other
dlisboa
So basically if I had 200 automations I could run them on 200 lambdas and have them finish by the time the slowest one finishes? That pretty awesome, specially for testing. For many cases this would also fall under the free tier since it not that many requests/usage...it kinda seems too good to be true. Am I missing something?
schickling
Nope, welcome to the serverless future! :)
dlisboa
You guys should put up an example on how to convert a (small) test suit to use it, like you said you did in your top comment. The examples you have are cool but don't really help visualize that parallelism gain.
schickling
Sounds like a great idea. Would you mind creating an issue here so we can track this? https://github.com/graphcool/chromeless/issues
dlisboa
Done.
kensoh
There's a section in the readme which I'm not sure of - "Running integration Tests for example is much faster." I guess that gains is from the parallel executions. But for the integration testings, there is still work to be done to make a central controller / runner to distribute various tests to different Lambda instances, correct? If that's the case, is loadtesting using Chromeless be another use case? Just initiate in parallel and blast the app infrastructure. It won't be efficient in terms of costs as using non-real browsers but this is probably as realistic as simulating real users load through real browsers.
adieuadieu
Yep. That's one of the main reasons why we're so excited about this project!
bald
> For many cases this would also fall under the free tier since it not that many requests/usage...it kinda seems too good to be true

(since I've ran into the same trap a couple of weeks ago and ended up with USD 650 of unanticipated charges): the free Lambda tier includes

* 1M requests

* 400k GB-seconds

--> the GBsec can be a serious bottleneck. Imagine you're running each Lambda instance with 512MB RAM and each instance takes 2 mins to complete your test. This means that one Lambda instance is ~61.5 GBsec, meaning you can execute ~6,500 of these instances per month to remain in the free tier.

Depending on how extensive your tests are/how often you run them, you might run out of free GBsec well before you'd run out of the requests quota.

penagwin
Granted that means running ~216 instances per day, or 9 instances per hour (taking 18 minutes per hour to run total). Now you're right, if you're running a screenshot service then this will kill you real fast.

However assuming a 8 hour work day you then get ~27 instances per hour. Each test takes two minutes to run, so for a single user testing, assuming a code - test - code - test routine, you'd be able to do that nearly continuously, for 8 hours a day, every day of the month (no weekends or days off). Seems safe to assume that wouldn't occur.

coredog64
The last time I tried headless Chrome, file downloads were a PITA. Has anyone tried downloads with Chromeless?
adieuadieu
Other than taking a screenshot and evaluating JS code in the context of Chrome and returning JSON, we haven't yet implemented any file-download features. But it might be possible for us to implement something. Would you mind creating an issue here describing your use case so we can discuss it further? https://github.com/graphcool/chromeless/issues/new
kensoh
By design, headless Chrome disables file downloads. This is being tracked at this issue to offer a way to enable that, and the issue seems to be moving along =) https://bugs.chromium.org/p/chromium/issues/detail?id=696481

EDIT - above is assuming downloading a file by simulating a click event to perform the download. there may be other workarounds by script injection etc to use XMLHttpRequest() for downloading a resource directly.

languagehacker
We've been using chrome-remote-interface for test automation in a project that makes heavy use of Lambda for a distributed event processing infrastructure. I'm looking forward to seeing whether we can implement this for running our test automation suite!
biscarch
I've been super excited about Chrome headless but haven't had a chance to dig into using it yet. The api here looks amazing for getting started without getting lost in the weeds. It'd be fairly trivial to hook this up to a Slackbot and to get on-demand screenshots of various pages on my websites, etc.
mabbo
A month ago I was trying to migrate some tests from phantomjs to chrome headless. The big blockers I ran into were that chromedriver automation has certain functionality that only works with a special chrome extension that it installs- but headless chrome doesn't support extensions.

There's work being done to fix this, but it's still in progress, last I checked. Until then, resizing windows, taking screenshots and a handful of other things simply don't work.

titel
@schickling - When will the PDF support arrive? https://github.com/graphcool/chromeless/blob/master/docs/api...
megamindbrian
Cool. Now can you use it with webdriver functions instead of re-inventing the API?
afandian
I've got the impression that lots of sites block AWS IP addresses. I wonder if this would hamper the practical use of this on Lambda.

I'm doing something similar, and this concern was one motivation for running in our datacentre vs EC2.

Does anyone have concrete info on rates of bots blocked from AWS IPs?

extra88
I assume the number one use of this would be test automation for one's own sites so blocking would not be an issue.

What are sites' motivations for blocking AWS IPs? I bet there are some reasons I would agree with even though the somewhat crude method of blocking ip range would have some unintended consequences (e.g. blocking people running a personal VPN).

pravda
>What are sites' motivations for blocking AWS IPs?

I block AWS. So many crawlers up to so much nonsense! I don't block by IP, but by hostname.

  $block='.amazonaws.com';
  $ua = @$_SERVER['HTTP_USER_AGENT'];
  
  if (stripos($rh,$block)!==false &&
  	stripos($ua,'Silk')===false &&
  	stripos($ua,'Safari')===false){
  
    	$block_visitor=true;
  	$message="Blocked Host:Amazon Web Services";
  }
hackits
Just curious what have you seen crawlers do to make you conclude they're up to nonsense?
pravda
Well, from amazonaws.com, there are so many requests for wp-login.php!

And then all the off-brand scraping companies use amazonaws.com.

ingenuous2
Fairly simple (if this is like the other programmatic headless browsers) to make a request through a proxy.
afandian
And where do you run the proxy?
bdcravens
In my experience this isn't happening.
mrmondo
Indeed, so many of AWS’s IP ranges are used for DDoS and malicious behaviour they end up getting blacklisted due to their poor reputation. It’s a bit like using one third party resellers shared IP ranges in your mail relay - you’re asking to end up on reputation based lists. There’s a lot to be said for your IP reputation on the internet and when you outsource that - you outsource your freedom to maintain your reputation risk.
colordrops
Can any headless browser solution be considered complete without GPU support? A large percentage of sites use GPU enabled and accelerated features and without GPU support, headless options are worthless to many applications.
Vuneu
This is pretty nifty! I've been keeping an eye on Phantomium for a while, I wonder what's come out of it.
kensoh
Looks like Google releasing headless Chrome is really shaking up the test automation domain. This is a really cool project :) The other day I saw another interesting and refreshing implementation using GraphQL - https://github.com/joelgriffith/navalia (not affiliated with the project, just got to know from a PhantomJS issue).
cjr
This looks great! As the developer of an automated screenshot solution (https://urlbox.io), one of the major pain points when taking screenshots is font-rendering. I wonder how you could install/configure fonts on lambda?
gowan
shamless plug: i've also written a high level api on top of the chrome remote debugger chrominator [1]

similar idea. chrominator use promises instead of a fluent api. it also follows the selenium w3c spec where possible. it does cool stuff with evaluate and evaluateAsync where it resolves the remote object to something usable.

to be fair there are a few other projects i know about that wrap chrome remote debugger with a high level api:

* autogcd [2]

* ghostjs [3]

[1] https://github.com/jesg/chrominator

[2] https://github.com/wirepair/autogcd

[3] https://github.com/KevinGrandon/ghostjs

kensoh
[1] Chrominator looks interesting. jesg/gowan is the maintainer for PhantomJS GhostDriver, so the API looks extensive. Another project that I heard recently is Navalia [4], which uses GraphQL type of queries to run headless Chrome. Also worth noting is Chromy [5] by a Japanese developer; it's one of the earliest to market (released in early May). BackstopJS maintainer also looks to be in active development using Chromy.

[4] https://github.com/joelgriffith/navalia

[5] https://github.com/OnetapInc/chromy

smithclay
Shameless plug: I've been hacking on headless chrome in AWS Lambda but with selenium webdriver support [1], also using the binaries from the serverless-chrome [2] project.

To echo another comment on this thread, headless chrome seems well-positioned to shake up the automated browser testing market. The price—especially with the AWS Lambda free tier—is very, very compelling for a number of projects.

[1] https://github.com/smithclay/lambdium [2] https://github.com/adieuadieu/serverless-chrome

hartator
Really awesome. Will definitely be usefull for a project I am working on.

Any plan to support other languages beside JS?

craptocurrency
Nice tool

Some API documentation says:

pdf() - Not implemented yet

Not implemented yet

How about you deprecate the API for now but reveal the purpose please.

1024core
How do cookies work in Chromeless? Can I specify a cookiejar to use? Can I keep cookies separated?
inertial
I'll just add some related projects I've used / tried in the past.

The promise of fast execution time in parallel is tempting with Chromeless. Thanks for sharing.

- https://github.com/webdriverio/webdriverio

- https://github.com/nightwatchjs/nightwatch

- https://github.com/assaf/zombie

- https://github.com/dhamaniasad/HeadlessBrowsers

schickling
Thanks a lot for bringing this up. We've tried all of the projects listed above before we began to implement Chromeless.

Ultimately it was the combination of using headless Chrome and the ability to execute code in parallel on Lambda, which made us invest in Chromeless.

kough
How hard do you think it would be to update Nightmare/etc. to use headless Chrome under the hood? Asking as someone with some interest in the space but little experience with the codebases.