repost from deleted:
Wicked cool concept, I can definitely see this being useful on several fronts. Just out of curiosity, how are you determining the "links to your competitors but not yours"?
The example query is """ (www.nike.com OR nike.com) NOT (www.adidas.com OR adidas.com OR redirect:nike OR domain:nike) """
I assume this is assuming that "you" are Adidas and "your competitor" is Nike?
Correct. Adidas would be able to see sites talking about their competitor Nike that are not linking to Adidas and also do not contain Nike as part of the domain. This makes sure you screen out all the Nike sub-sites like runwithnike.com which obviously don't link to Adidas.
The layouts don't really look anything alike and I think they may be using an off-the-shelf theme for their landing page. Also their site works... have you launched yet?
I apologize for my previous comment, I was reacting out of anger.
This is the page they copied http://nerdydata.com/home2.php. It was our wait list for the last 2 months while we developed our site, which now has a new homepage we put up last week. http://nerdydata.com
But no hard feelings, maybe you had the idea independently and happened to use the same exact theme and messaging.
Wow very impressive - I can see so many diverse uses of this (the least commercial but fun one would be up to the minute javascript framework popularity contests!).
I've heard from their engineers that Haskell has been their secret to being web scale while having a sane ops and engineering load. :-)
Considering I'm doing my own Haskell based tech in my business (and I'm somewhat actively in the core community), It pleases me greatly to see Haskell/ghc used in a rich array of interesting businesses!
This is true :) been a great tool to write a spider in, fast native code, a strict type system and very cheap threads has made it much faster than it otherwise would have been. ZeroMQ has also been a great help in modularising it. I'll try to get a piece out on the technical architecture later. (I'm the CTO, btw.)
And a bit of old Python glue to hold everything together.
It's funny how long things can last when you think you would have outgrown them pretty quickly. A tiny round robin domain server that maxes out at around 60k domains per second on just a single core to distributed spider bots though is just about easy with Python + ZMQ. Plenty to keep a swarm of far more complex and well engineered Haskell crawlers humming away and throwing the results to Golang indexers to push into the Elasticsearch core for ...
Well, I'll leave the rest of it to mwotton's blog post, suffice to say it's been one of those life changingly awesome projects of doom. I'm sold on the benefits of embracing polyglot.
12 comments
[ 0.25 ms ] story [ 40.3 ms ] threadYou stole our idea and our theme verbatim? How can you live with yourself?
This is the page they copied http://nerdydata.com/home2.php. It was our wait list for the last 2 months while we developed our site, which now has a new homepage we put up last week. http://nerdydata.com
But no hard feelings, maybe you had the idea independently and happened to use the same exact theme and messaging.
edit: (this is the theme https://wrapbootstrap.com/theme/beaker-responsive-working-la...)
All the best with the continuing launch.
Considering I'm doing my own Haskell based tech in my business (and I'm somewhat actively in the core community), It pleases me greatly to see Haskell/ghc used in a rich array of interesting businesses!
Elasticsearch
ZMQ
Golang & Haskell
Unbound DNS
And a bit of old Python glue to hold everything together.
It's funny how long things can last when you think you would have outgrown them pretty quickly. A tiny round robin domain server that maxes out at around 60k domains per second on just a single core to distributed spider bots though is just about easy with Python + ZMQ. Plenty to keep a swarm of far more complex and well engineered Haskell crawlers humming away and throwing the results to Golang indexers to push into the Elasticsearch core for ...
Well, I'll leave the rest of it to mwotton's blog post, suffice to say it's been one of those life changingly awesome projects of doom. I'm sold on the benefits of embracing polyglot.