18 comments

[ 3.0 ms ] story [ 17.5 ms ] thread
This is very impressive (the way that is implemented on top AWS services).

I would be curious to see how accurate it is when it comes to rendering. I have a Screenshot as a Service side project (https://getscreenshot.rasterwise.com) that is also build with Puppeteer, but accuracy is a completely different animal.

Out of the box Puppeteer is really not that accurate so you need fine tune it to address a huge amount of rendering edge cases.

This is the area where most services come short imo. I have put a lot of effort into optimizing my service accuracy but it’s a never-ending task. There always seems to be one website that doesn’t render the way it’s supposed to be rendered.

I really don’t understand why headless browsers seem to be developed with UI rendering as a low priority. Simulating a viewport is not equal to simulating an actual screen.

I hear you. I run Urlbox (https://urlbox.io) and accuracy is one of the hardest parts, as well as scaling hundreds or thousands of instances of chrome since it’s a real memory hog.

I also managed to get a version of the urlbox renderer working on lambda but it fails on some websites due to the /tmp space limits, for example trying to do a full page screenshot of a heavy page like dailymail would use up all of the available disk space in the lambda environment with chrome’s media cache.

Love what you have done with URLBOX dude! That's my goal standard when it comes to improving accuracy for GetScreenshot. I have tested with your website preview and I think both of our solutions nail accuracy, but you have been longer in this game so you probably have way more edge cases covered. I still find myself trying to address new edge cases every certain time.
Isn’t there also a memory limit with using Puppeteer? Last time I checked, there was a hard coded memory value for screenshots within Chrome.
Was looking at building something similar on top of Lambda but I'm so happy that I found this!

Quickly browsed through the code and I don't think it supports local pages but might throw a PR your way to add support if you don't mind. Thanks!

One thing from the readme that concerns me:

S3_ACL "public-read"

It will be great if you can explain why and how is this used in the ReadMe file.

This is for making the uploaded images in S3 accessible by anyone. The rationale is that the screenshots can be viewed by browsers without any sort of authorization.

But since this is just an environment setting, you can change it if your use case is different.

Is there any javascript library to do the same? It would be useful to have a library as a dependency and just initialize it in the code and then you have screenshot in meta of each page
Isn't a problem with lambda to have a potential CPU waiting time of 30 seconds ?
I'm not sure I get what you meant. One AWS Lambda function can run for up to 15 minutes.
I think they meant run time which means they are concerned about how much it will cost since lambda charges by run time.
Definitely a noob at npm, when I run 'npm install', it freezes at:

[ ......] / extract:aws-sdk: sill extract aws-sdk@2.595.0 extracted to /home/foo/serverless_apps/chromda/node_modules/.staging/...

Any tips or tricks would be much appreciated?

Never mind. No idea why but installed after multiple tries.