This is really slick. It's tempting to think of this as a potential route towards headless functional testing — probably no substitute for something as heavy as selenium but for light tasks and DOM-based inspection of returned payloads (and screenshots, baked in!) it seems like a plausible foundation.
I think the main difference for me between this and Selenium would be that Selenium offers some macros to click an element, or simulate a drag and drop between two elements. If you added those in here--wouldn't be too much more, I don't think--you'd have a competitive testing framework, albeit Webkit-only.
Assuming you could get jQuery to run inside this it should be pretty straight forward to translate the Selenium IDE's Selenese commands into jQuery statements.
I'm going to have to give this a run tomorrow in the AM.
I think the main differentiating factor between phantomjs and the above mentioned ones is that phantomjs actually uses WebKit and has full support for everything that WebKit supports. Zombie.js uses the DOM API due to jsdom, which might have its own parsing intricacies.
I don't know that any of those support SVG. If you want to render e.g. Raphael.JS drawings on the server, headless JS with SVG DOM support is pretty useful.
That actually sounds like a really neat idea - throwing together a PhantomJS script that scrapes the content of a particular div containing Raphael or Flot and then renders as png.
The first two applications that come to mind are automated testing and screen scraping, I'm a little surprised they didn't include examples of those. Looks interesting though.
They are completely different things. Rhino is a JS engine. envjs as a script that creates a mock window object and can run in an engine like Rhino. Htmlunit comes close, implemented in Java, tries to SIMULATE some popular browsers. PhantomJS is... Webkit.
It looks like a Javascript thread-of-execution that can write to local filesystem paths (to render graphics, at least) may call into network-loaded DOM/code. Is there any assurance that page contents can't discover and use phantom API operations? (Or perhaps read local 'file:' URIs?)
Wow, that's a good insight, seems like a possible attack vector. Maybe something like Adobe Air's security model of putting local access and network access in different iframes with a message-passing API between them would work. That's always felt like a bit of a hack to me but at least the separation between different frames in WebKit has been well-tested.
GreaseMonkey has to contend with these same issues. It has a security model, which will be broken if developers ignore the list of things you can't do.
This is great indeed! I just tried it out. For people who are running Ubuntu 10.10, and have Qt < v4.7 can change line 34 and comment line 164 (to atleast test out the examples on the website).
Anyone know if this could be used for generating heatmaps? I'd basically need to identify the x,y offsets of elements on a page. I realize that these can be effected by the browser's width/height, but I'm hoping I can set those to generate the data.
If you really want a headless webkit browser you would need to write a new webkit port to a graphics library that doesn't require a windowing system (maybe cairo).
Strictly speaking all this needs is a working QtGui component in the Qt library.
There are a couple of ways to compile Qt to operate against an in-memory framebuffer rather than a "real" windowing environment, although neither is a well supported part of modern Qt:
Not headless, many other projects, many other ways to do the same even without programming/compiling. I'm still longing to see a true headless browser that renders to Cairo or something else.
This might be useful for html sanitization. You can allow anything as input (including scripts, styles, etc), render it to a page in phantom, apply your whitelist on the effective DOM, and render it out as output. Of course, this might be resource intensive, and you have to be careful about phantomjs being sandboxed and having cpu/memory/timeout limits. The nice thing about this though is you that 1) you get your serializer/deserializer for free, 2) it's very forgiving on malformed input, 3) the output is WC3 valid since webkit corrects the DOM, and 4) you can support styles and scripts that affect the DOM.
44 comments
[ 3.1 ms ] story [ 93.1 ms ] threadI'm going to have to give this a run tomorrow in the AM.
Zombie.js: http://news.ycombinator.com/item?id=2038663
HTMLUnit: http://htmlunit.sourceforge.net/
Celerity (JRuby wrapper around HTMLUnit): http://celerity.rubyforge.org/
I've used Selenium with Firefox and Xvfb to do headless scraping in the past; looks like my toolkit just got simpler.
Screen Scraping: http://code.google.com/p/phantomjs/wiki/QuickStart#Rendering
phantomjs rasterize.js 'http://en.wikipedia.org/w/index.php?title=Jakarta&printa... jakarta.pdf
But, have security issues been considered?
It looks like a Javascript thread-of-execution that can write to local filesystem paths (to render graphics, at least) may call into network-loaded DOM/code. Is there any assurance that page contents can't discover and use phantom API operations? (Or perhaps read local 'file:' URIs?)
http://wiki.greasespot.net/Security
(Indirection over my blog in case I need to switch the file away from my webspace)
Node.js is an asynchronous I/O framework
Edit: The code for that is on http://code.google.com/p/phantomjs/wiki/QuickStart
While this cool, its really not that different from projects like http://code.google.com/p/wkhtmltopdf/.
If you really want a headless webkit browser you would need to write a new webkit port to a graphics library that doesn't require a windowing system (maybe cairo).
There are a couple of ways to compile Qt to operate against an in-memory framebuffer rather than a "real" windowing environment, although neither is a well supported part of modern Qt:
http://qt.gitorious.org/qt/pages/GettingStartedWithLighthous...
http://doc.qt.nokia.com/qtextended4.4/qvfb.html