It uses Cheerio for scraping, but it also lets you use selector queries to grab a subset of documents, whether the document is HTML, JSON or another format. It also lets you request multiple documents in one HTTP request, and a few other things.
I didn't submit this, but I did oversee the project.
Noodle is a Node-based web scraper that also handles JSON, XML and other file formats. It was initially built as a hack project to replace a core subset of YQL.
All responses can be served as JSONP, to allow for cross-domain scraping from a website's front-end.
Selector queries can be used to grab a subset of a document - e.g. CSS selectors for HTML documents and dot-notation for JSON documents. It lets you request multiple documents in a single HTTP request, and a few other things.
I helped to guide an intern, Aaron Acerboni, at my company, Dharmafly, when he built it last year.
20 comments
[ 3.1 ms ] story [ 47.2 ms ] threadNpm package npm install noodle
Npm tarball curl -kO https://registry.npmjs.org/noodlejs/-/noodlejs-0.2.0.tgz
http://npm.im/noodlejs is the NPM package
https://registry.npmjs.org/noodlejs/-/noodlejs-0.2.0.tgz is the package source
The code is pure-JS (not a C++ addin)Noodle is a Node-based web scraper that also handles JSON, XML and other file formats. It was initially built as a hack project to replace a core subset of YQL.
All responses can be served as JSONP, to allow for cross-domain scraping from a website's front-end.
Selector queries can be used to grab a subset of a document - e.g. CSS selectors for HTML documents and dot-notation for JSON documents. It lets you request multiple documents in a single HTTP request, and a few other things.
I helped to guide an intern, Aaron Acerboni, at my company, Dharmafly, when he built it last year.
Like most it ignores page requiring state or anything dynamically generated on the client.
the selector: 'h3.r a' allows only to get the list of names