HN Explorer

Ask HN: How to wget pages with javascript?

4 points by baconface ↗ HN
I need to wget a url, allow the javascript to execute, and capture the result in a file. I've tried wget, curl, httrack,plugins like lget, and links(2) and none of them execute JS.

I want something like $: firefox <url> -s <file>

Does anyone know how to accomplish this behavior without using JS?

8 comments

[ 3.1 ms ] story [ 27.2 ms ] thread
profitbaron
Personally, I'd use http://www.phantomjs.org/ and create a script which loads the page like a browser. Once you've done that you can get it to take screenshot or use JS to pull data off the page.
baconface
Thanks. I really want to get this done with the command line and it looks like Phantom JS will do the trick.
dstein
You can write a NodeJS script using the JSDom module to do this.
baconface
Thanks for the suggestion. Any idea how to accomplish this task without using JS?

I'm surprised chromium, konqueror and firefox don't offer this functionality for offline browsing.

dstein
Selenium (http://seleniumhq.org/) might work but it's not a simple command line script.
bartonfink
Selenium will do this, although it's not pretty. I spent about 4 months on a contract doing exactly that, and it sucked.