Hi, this is what I would like the dotnet core community * to start using instead of looking for the Lucene team to port several years old java code to core. We already have a search engine on dotnet core. It's called Resin. I'm only a few minor optimizations away, plus one or two features I'd also like to include, from having a robust and full-featured Lucene replacement, leaner, faster, with better compression and with readable code (sometimes even object-oriented). Time to bury Lucene.Net people. Come help out.
*) Is there such a thing? I mean, you're awfully quiet and compared to the java community, which I know exists, you seldom help out in open-source projects. What's up with that? Haven't found the right project yet? Search, i.e. information retieval and NLP, is about as about as interesting as it gets, at least to me. General AI will be bourne out of IR and natural language processing. Not interested yet? There's a really cool Trie and some pretty sweet tree traversing algorithms that can perhaps be optimized. Have a look at my poor-mans [Levenshtein automaton over in-memory trie](https://github.com/kreeben/resin/blob/master/src/ResinCore/I...) and the same [over a disk-based trie](https://github.com/kreeben/resin/blob/master/src/ResinCore/I...). Sentiment analysis comes later on. It's great fun!
> Is there such a thing? I mean, you're awfully quiet and compared to the java community, which I know exists, you seldom help out in open-source projects.
Of course there is such a thing, although Microsoft did it's best to kill it about a decade ago, it's never recovered from that. As for this project, I suspect the contributors to be small, it's simply the sort of thing that's outside the skill set of most .net developers.
>Microsoft did it's best to kill it about a decade ago
A decade ago I started learning C#. I did not notice this event. What was it in particular that they did?
>outside the skill set of most .net developers
But javaland has many of those devs? I'm really wondering if anyone outside of the Lucene team and Google even has a clue about how to build a modern search engine.
If it is true that .net devs are unskilled then this project is the perfect starting point for all those who wish to transcend from being web and business application developers making tons of money for their employers, to ML devs, thus (hopefully) securing for a while their hireability, since businesses in fact have realized they need to become smarter than their compertitors and that they need ML skills (and big data capabilities).
It's the perfect starting point because in this project, I've taken all ugly, wierd and awkwardness out of IR, because there is no place for such things in IR, because IR is not ugly, wierd or awkward, it's very, very straightforward. The code needed to express an IR system needn't be complex, at all. Half the fun of doing this project is showing everybody how easy the domain of information retieval is.
> A decade ago I started learning C#. I did not notice this event. What was it in particular that they did?
There was a fairly vibrant OSS community around .net at the time filling many of the gaps. ORM (nHibernate), MVC (monorail), build (nant), IOC (Autofac and many others), unit testing (nUnit and others).
MS decided to go all NIH and create their own, almost always worse version of all of these tools rather than work with the community. The message they effectively sent was "don't do OSS on our platform, we'll crush you're successful project with an inferior clone".
>But javaland has many of those devs? I'm really wondering if anyone outside of the Lucene team and Google even has a clue about how to build a modern search engine.
I think you're right, there probably aren't that many with the know how. The even bigger problem is there aren't many with the awareness that tools like lucene exist, it's database queries or nothing to most developers.
> it's database queries or nothing to most developers
Perhaps for some but not for most, I think.
I believe dotnet devs suffer from a lack of confidence and from identifying themselves as dotnet devs, instead of just devs.
I want to be on dotnet core because I want to be cross platform and code in C#. My options, if I cannot get a community around this piece of code, is to go to golang or java, because I really want to be cross-platform and code in a decent high-level lang.
But it seems silly that I should move from the preferred platform to one where I am less productive because Microsoft were being the Microsoft they were back then, when they put so much effort today into copying the success of java. Prior to an open-sourced dotnet core, that vision was unreachable. Now it is reachable, for real.
I don't trust Oracle or Google any more than I trust Microsoft. I trust they will be what they are.
I do think dotnet core, technically, outshines java, by miles and miles. Haven't tried go yet.
I need my tech to be built on the best tech in the world. Because I'm out to beat those who are not.
We'll see how much traction this project gets. Maybe none. I'll beat the Lucene team single-handed if I have to. That's not my vision, it's just the first step towards creating what I'm actually here to create: tech, open for everyone, powerful enough to beat Google with, if you just add the right business strategy to the mix.
This is not a Lucene-killer. It's a Google-killer.
Yeah, I'm megalomanic sometimes. What a dull world to live in though, where Google is unbeatable ;)
7 comments
[ 329 ms ] story [ 963 ms ] thread*) Is there such a thing? I mean, you're awfully quiet and compared to the java community, which I know exists, you seldom help out in open-source projects. What's up with that? Haven't found the right project yet? Search, i.e. information retieval and NLP, is about as about as interesting as it gets, at least to me. General AI will be bourne out of IR and natural language processing. Not interested yet? There's a really cool Trie and some pretty sweet tree traversing algorithms that can perhaps be optimized. Have a look at my poor-mans [Levenshtein automaton over in-memory trie](https://github.com/kreeben/resin/blob/master/src/ResinCore/I...) and the same [over a disk-based trie](https://github.com/kreeben/resin/blob/master/src/ResinCore/I...). Sentiment analysis comes later on. It's great fun!
Of course there is such a thing, although Microsoft did it's best to kill it about a decade ago, it's never recovered from that. As for this project, I suspect the contributors to be small, it's simply the sort of thing that's outside the skill set of most .net developers.
A decade ago I started learning C#. I did not notice this event. What was it in particular that they did?
>outside the skill set of most .net developers
But javaland has many of those devs? I'm really wondering if anyone outside of the Lucene team and Google even has a clue about how to build a modern search engine.
If it is true that .net devs are unskilled then this project is the perfect starting point for all those who wish to transcend from being web and business application developers making tons of money for their employers, to ML devs, thus (hopefully) securing for a while their hireability, since businesses in fact have realized they need to become smarter than their compertitors and that they need ML skills (and big data capabilities).
It's the perfect starting point because in this project, I've taken all ugly, wierd and awkwardness out of IR, because there is no place for such things in IR, because IR is not ugly, wierd or awkward, it's very, very straightforward. The code needed to express an IR system needn't be complex, at all. Half the fun of doing this project is showing everybody how easy the domain of information retieval is.
There was a fairly vibrant OSS community around .net at the time filling many of the gaps. ORM (nHibernate), MVC (monorail), build (nant), IOC (Autofac and many others), unit testing (nUnit and others).
MS decided to go all NIH and create their own, almost always worse version of all of these tools rather than work with the community. The message they effectively sent was "don't do OSS on our platform, we'll crush you're successful project with an inferior clone".
>But javaland has many of those devs? I'm really wondering if anyone outside of the Lucene team and Google even has a clue about how to build a modern search engine.
I think you're right, there probably aren't that many with the know how. The even bigger problem is there aren't many with the awareness that tools like lucene exist, it's database queries or nothing to most developers.
Perhaps for some but not for most, I think.
I believe dotnet devs suffer from a lack of confidence and from identifying themselves as dotnet devs, instead of just devs.
I want to be on dotnet core because I want to be cross platform and code in C#. My options, if I cannot get a community around this piece of code, is to go to golang or java, because I really want to be cross-platform and code in a decent high-level lang.
But it seems silly that I should move from the preferred platform to one where I am less productive because Microsoft were being the Microsoft they were back then, when they put so much effort today into copying the success of java. Prior to an open-sourced dotnet core, that vision was unreachable. Now it is reachable, for real.
I don't trust Oracle or Google any more than I trust Microsoft. I trust they will be what they are.
I do think dotnet core, technically, outshines java, by miles and miles. Haven't tried go yet.
I need my tech to be built on the best tech in the world. Because I'm out to beat those who are not.
We'll see how much traction this project gets. Maybe none. I'll beat the Lucene team single-handed if I have to. That's not my vision, it's just the first step towards creating what I'm actually here to create: tech, open for everyone, powerful enough to beat Google with, if you just add the right business strategy to the mix.
This is not a Lucene-killer. It's a Google-killer.
Yeah, I'm megalomanic sometimes. What a dull world to live in though, where Google is unbeatable ;)
I don't think the megolomania/arrogance is a bad thing, most projects start with an "I can do it better" approach.