Any ideas how to pivot a web crawler/scraper project?

2 points by alexander-g ↗ HN
There is a quite powerful crawler and scraper engine. We developed it for internal needs at happystayapp.com. We used it as a hotels reviews meta search and hotels aggregator from sites like booking.com, expedia, etc.

But I'm feeling it's a bit underused and I'm thinking how else it can be used?

It supports many of features modern web scraping tools have. The engine is scalable and easy extendable.

The technologies: - MongoDB - C# - Microsoft Azure - JSON

Where and how else the technology can be used? Any ideas?

0 comments

[ 0.21 ms ] story [ 11.8 ms ] thread

No comments yet.