HN Explorer

Ask HN: What are you working on (hacking)?

69 points by samratjp ↗ HN
Thought it would be helpful for HN'ers to see what others are up to and potentially find cool projects to collaborate on.

Bonus points for keeping it under a tweet + demo (or github,etc).

193 comments

[ 0.24 ms ] story [ 218 ms ] thread
buro9
Reporting for SharePoint... trying to federate multiple instances and make sense of the fact that every site/list has a different schema of potentially the same data. So it's a foray into reporting against federated semi-structured data.
transatlantic
Journalism, with a focus on early-stage technology companies. Entrepreneurs are doing amazing things with small teams and relatively little capital, but the stories and lessons seem mostly lost in the stream of tech news about iPhones, the oldest Twitter user, or Facebook's privacy policies. I think there's a small but significant market for passionate, well-researched, educational content like this. I'll find out soon enough.
Concours
This sounds good, care to share your website url and may be the kind of startups/tech area (ideal profile) you'd like to write about?
transatlantic
A simple version 1 comes out Sunday. I'll post it to HN then. The focus is on companies no older than two years, and generally on those with consumer-oriented products. But beyond that, they could be funded or bootstrapped, located in Silicon Valley or Pittsburgh or anywhere else, and led by startup veterans or total newcomers. Amazing stuff is happening everywhere; you just need to look for it.
user24
Another reader right here. This is what I used to read techcrunch for.
vosper
My ongoing quest to make money predicting horse races. So far I've not made a dime, and don't really expect to, but the faint hope of future monetary rewards keeps me going.

The real payoff has been that in the process I've learned all sorts of things: about Python, data mining, working with large datasets, machine learning...

user24
I've been thinking of trying to write a machine learning app to do just that. Any hints?
vosper
I didn't implement the machine learning algorithms for myself, because there are some really good packages out there and I know I don't have the smarts to better them.

Keep in mind that I didn't really have any success:

There seem to be two main ML packages, Weka and Orange. I personally preferred Orange, it has a nice graph-based UI for linking various components together; when you've figured that out it can script in Python. Also Orange makes it easy to test your data set against various different learning systems, and compare the performance. Standard testing procedures like n-fold cross-validation are built-in and really simple to use.

Also you need data. I'm pretty sure more is always better. I actually started with greyhounds* and skimmed mine (in Python use BeautifulSoup) from a website. I tried to come up with various statistics about the recent performance of the dogs. Unfortunately nothing I tried made the ML algorithms predict better than a random choice. A friend who's into gambling suggested greyhound racing was quite random by nature, so I've switched to horses recently. I'm still building that dataset, now trying out MongoDB just for fun.

I think the trouble is that you can have as much raw data as you like, but generating the predictive statistics requires a lot of knowledge of the problem domain. I'm not actually into gambling at all so I don't know if the track conditions are important, how much breeding or the age of the animal really matters etc... This made it hard to pick likely stats (and rebuilding datasets and retraining learners can take some time).

For horses there's a lot more information in forums and racing guides etc, so I'd start with horses. Just make sure you've tested your predictions with pretend bets before you commit any real money :)

Good luck!

*I began with greyhounds because of a dissertation posted on reddit where the authors suggested they'd had some success with a neural network and gave quite a lot of detail. That piqued my curiosity, and my initial version just re-implemented their work.

EDITED: For clarity

user24
Thankyou for the detailed reply.

Yeah I hated using weka at uni. I'll look into Orange.

"I don't know if the track conditions are important, how much breeding or the age of the animal really matters etc."

Yeah, feature selection is a tough one. I'd thought that the system would pick up on good indicators by itself, but it might well be that that has to be a manual decision.

"Just make sure you've tested your predictions with pretend bets before you commit any real money"

haha, yeah absolutely. My plan was to train/test until the accuracy seemed good enough (using monte carlo) and then run the system on live data with pretend money for a few months to see what the actual performance is like, before actually investing real cash.

Do you have a link to the greyhound topic? I searched on google but couldn't find it.

noizd
Genetic algorithms can be used to determine the (close to) best combination of features from all the features you have access to.
vosper
I don't have a problem so much with having myriad statistics and picking the right ones, but not knowing which stats to generate in the first place from my database of results.

For example, I assume that a dogs past performance must be some indicator of its chances in the next race, but how do I account for the chances of dogs who didn't complete their last race? What weighting is the last race worth, compared to the ones before (perhaps it had a bad race, but on the whole is running well).

I just don't know how to optimise for those sort of things. I have a rough idea that some combination of genetic programming and GA could help - it would be an interesting challenge to builds software that knew how to apply a selection of mathematical functions to my data, and then breed the results like a GA. But it's tricky, I'd have thought.

I've been treating the ML classifiers and learners as something of a black box, perhaps a more rigorous approach is required.

user24
"software that knew how to apply a selection of mathematical functions to my data, and then breed the results like a GA"

yeah, I'd envisaged using the accuracy of the neural net as the fitness function for a GA that mutates input parameters. It's another layer of complexity, and I've no clue how you'd start, but it seems like it would work.

In other words - use a GA to select features, using how well the NN trained on that set of features performs as the fitness function.

notahacker
I've heard that the easiest way of predicting greyhound racing is to ignore the form book and monitor the odds changes following bets being placed at the very last minute by those with insider information...
microcentury
Here's a random greyhound racing tip which a man in a bar told me, so it must be true: before the race, leave it as long as possible to bet and watch the dogs. The one that is quivering and dancing and looks must wound up generally wins. Wait until you get a race where only one dog looks that way.
captaink
Thank you so much for pointing me into the direction of orange.

One of my long-standing projects is predicting soccer bets.

vosper
You're welcome. One of the things I'm looking at now is adapting ranking systems from other sports or competitions. For example I know that the Elo system from chess has been applied to other sports (I don't know the details, though, or what success they had)
tocomment
This sounds like an awesome project!

Why did re-implementing the dissertation for greyhounds not work? Was the dissertation flawed?

If you're putting this much effort into it, why not stop by a horse racing track a few times and pick up some domain knowledge? Maybe you could even talk to race horse owners, jockeys, breeders?

I wonder if you could turn this into a product for breeders? Or maybe for people buying/selling race horses? Or people hiring Jockeys, or even marketing an offshoot of this to the gamblers? Just some wild thoughts.

mian2zi3
I met a guy (I can give more details offline) who worked for an international gambling syndicate based in HK. Software written by teams in Shenzhen running on servers in HK fed by data from handicappers in Australia directed bettors to place bets at tracks all over the world. They were making something like 15-20% returns. As you can imagine, he had some crazy stories. At the time I was thinking about something similar (not horses) but eventually moved on to more respectable things (grad school.)
vosper
I'd love to hear more about this, what's the HN-sanctioned way to do PMs or non-public messaging? Throwaway email?
mian2zi3
I have an email in my profile.
fortes
Paged, multi-column layouts for in-browser publications using HTML/CSS/JS
_b8r0
In my spare time, working on a game for the 8-bit ZX Spectrum.
SteveC
How do you do your development? Do you work on the actual hardware or use an emulator?
user24
computer vision JSON webservice - you supply image, it returns tags/keywords super fast. Spare time project.

Hoping to build a freemium model out of it for image libraries to use. Happy to speak to anyone with any kind of CV / object detection knowhow.

tocomment
How far have you gotten on this? I didn't know that level of CV technology existed to tag arbitrary images. It sounds like a really cool project.
user24
thanks, it is a really fun project to work on :)

At the moment I can return colour names and faces.

Example:

input: http://cm1.theinsider.com/media/0/91/63/mini-kim-kardashian-...

output: face, yellow, blue, black.

So just colours and faces at the moment. You're right that arbitrary images are ridiculously complicated, I'm hoping to start off on a smaller domain and build up :)

(edit: obviously I can return the coords of the face too, as well as coords of empty parts of the image etc, but tagging is really what I'm focusing on at the moment)

paulgeo
A company in Berkeley is doing this: www.iqengines.com (see Developer API), and demonstration app www.omoby.com. HTML Post image and JSON/XML return label (also face, barcode, ocr, etc).
user24
thanks! Hopefully I can kick their asses ;)

edit: which doesn't seem like it'll be too difficult. On their developer test I just get {"data": {"error": 0}} back for any image I try to upload.

devonrt
Harvesting as much sheet music as possible from free sources (Icking, Petrucci, etc) converting it to MusicXML and LilyPad and then doing... something with it. I have a few ideas, but harvesting it all is a start. I'd like to put it online wiki style since it will most likely need editing after being run through OMR.
hnote
Sounds very interesting! How do you want to do the OMRing? It will probably take a lot of resources...
singular
http://www.codegrunt.co.uk/terse.html

A programming language where syntax and semantics are manipulable at run-time as well as compile-time and where you can define grammars in-line and use them immediately. I also intend to integrate the concepts of pattern calculus - http://lambda-the-ultimate.org/node/3695 - to permit extreme levels of flexibility in the language.

It's a huge project and I'm right at the start of it. But no matter how hard I try I can't get away from the desire to work on language design and compiler development. It's just my thing, and the wonderful thing about hacking is you can just do it :-)

akkartik
I see you switched from racket to Go. Care to talk about your experiences?

I've been playing with making the implementation of arc more timeless (e.g. http://arclanguage.org/item?id=12057, http://arclanguage.org/item?id=11864)

singular
I just found namespaces in Racket too painful (see [this blog post][1]), I spent hours and hours trying to do something I felt ought to be very simple, i.e. sharing a namespace between different files, yet Racket just utterly refused to do it. I RTFM but found it utterly confusing and nothing I tried, including their examples, worked. I asked on the freenode IRC channel, and even then nobody could help me.

After a while I gave up, maybe I am simply not a good enough coder to understand how Racket namespaces work, but either way I worried that if this one aspect of the language is extremely difficult, what else am I going to uncover in the course of the project? On that basis I decided it'd be wise to switch.

Initially I was going to switch to C for portability as I am also working on a parser generator, singular[2], which I thought could be useful to people even before I write it in Terse (I intend to self-host and bring singular into that too), however I worried that the many pitfalls that C brings to the party, e.g. the ease of segfaulting, null pointers, etc. and its lack of abstractions would overly slow me down, so I thought Go would be a better option, especially as it seemed tastefully designed.

My experience of Go so far is one of great admiration and enjoyment, it really is a lovely language, nicely low-level and low-key yet still providing many useful abstractions including proper interfaces, i.e. by implementing the methods of an interface you can treat it as that interface without having to explicitly inherit from it.

To be honest my decision to switch to Go is probably not that defensible as not many people are using it so the initial reason (portability) for switching to a lower-level language is less of an excuse now, so if I'm being honest I have to admit that I wanted a fast language that played nice with Linux (not that Racket wasn't either of these), and I wanted to play with Go, which kinda overrode other considerations.

Most recently I've been very interested in implementing [pattern calculus][3] in the language somehow, as it provides enormous flexibility and offers a formal underpinning to a more fundamental means of expressing abstractions than oo, functional, etc. - in fact my ambition is to have an abstraction which can encompass these paradigms in itself if you want, i.e. you can implement oo or functional or whatever you want. Obviously I am very inspired by lisp in this and many other regards.

The main thing is getting stuff done, this idea has been floating around in my mind for at least a couple years and I've changed my mind about things many times (and will carry on of course when necessary) causing me to throw away work more than once, so obviously I am somewhat focused on actually writing code and getting closer to actually having something rather than just the idea.

Luckily I am pretty damn certain about the core ideas in the language (flexible syntax, the use of pattern calculus, etc.) so that looks to be quite likely.

Anyway, it's really early days, but I am utterly committed to getting this done as I want the language for myself, want it to not be a toy language, and want it to actually do these things I think would be awesome, even if (as is most likely as with any personal language project) no one else uses it :-)

I know I'm digressing from your question, but have to say that I really think one of the most wonderful aspects of programming is the ability to just hack on stuff, no matter how crazy, with just a cheap computer, some coffee and a willingness to put in the time. So glad I was born in a time where that was possible.

[1]:http://www.codegrunt.co.uk/2010/06/28/Racket-Namespaces-Suck...

[2]:http://www.codegrunt.co.uk/singular.html

[3]:http://lambda-the-ultimate.org/node/3695

akkartik
No defense necessary. All the best! Let me know if I can help. I find talking to someone helps me avoid paralysis, that great obstacle to doing.

(email in profile)

herrherr
http://www.getmetricmail.com

In one sentence: Get your Google Analytics data straight to your inbox.

Build as part of my dissertation. Build on Google App Engine.

balac
pretty cool! I just signed up.
duck
Great looking site and very easy to use. Obviously the report is a lot nicer than that Google one that you can get emailed in PDF format, but what other advantages are there?

To me the "best" setup would be to avoid the PDF and get the report directly in my mailbox. I know that would require unique graphics for each email, but is that the only barrier from going with that approach?

herrherr
Thanks for the feedback. The email-only option is definitely on our list, it will be a bit tricky due to the different mail clients but still doable.
pstinnett
I agree. I recently unsubscribed from Metric Mail because I didn't want to look at PDFs of my analytics. I want them right in the email. Once you push that feature I'll likely resubscribe!
clstrfckr
just got the last beta key :) thanks!
timme
fortunately for everyone else, we just added another 50 beta accounts. thanks!

:)

Vivtek
http://search.cpan.org/~michael/Class-Declarative-0.06/lib/C...

A declarative framework for Perl that lets you set up complex data structures and code in an easy-to-read format.

A complete working example GUI program, using a not-yet-published set of semantic classes:

  use Class::Declarative qw(Wx::Declarative);

  dialog (xsize=250, ysize=110) "Wx::Declarative dialog sample"
    field celsius (size=100, x=20, y=20) "0"
    button celsius (x=130, y=20) "Celsius" { $^fahrenheit = ($^celsius / 100.0) * 180 + 32; }
    field fahrenheit (size=100, x=20, y=50) "32"
    button fahrenheit (x=130, y=50) "Fahrenheit" { $^celsius = (($^fahrenheit - 32) / 180.0) * 100; }
koenigdavidmj
Is that...significant indentation in Perl?

Will you marry me?

Vivtek
Well, actually it's kind of like an unholy Python/Tcl mutant thing, since the one-word tag determines the parsing of its line and children - but yes, significant indentation using a Perl filter.
cabalamat
I did something similar to that once -- a program that compiled GUI descriptions into a runable Python class, which could then be subclassed to provide functionality. The syntax looked like this:

  window @MyBigWindow "My Big Window" {
   menuBar {
      menu "File" {
         menuItem @New "New"
         menuItem @Open "Open..."
         menuItem @Save "Save"
         menuItem @Exit "Exit"
      }
      menu "Edit" {
         menuItem @Copy "Copy"
         menuItem @Cut "Cut"
         menuItem @Paste "Paste"
      }
   }
   rowLayout {
      button "Button 1"
      button "Button 2"
      button "Button 3"
   }
  }
Laying out GUI components was done with the rowLayout, colLayout and table containers. The subclass would contain code to be executed for @New, @Open, @Save, etc.
Vivtek
There's some similar stuff for wx in Perl and Python, too - I've just been taking the time to (try to) be systematic about making things fast and easy to specify. I really, really get tired of coding all that stuff by hand every damn time - it's one of the major things that made me get out of GUI work in the first place, back in the Stone Age.

I've got big plans, and since it's not expected to pay any bills for a while, I can afford to think things through sufficiently. I just hope I won't drop it entirely.

It's been fun so far, though.

cabalamat
> I really, really get tired of coding all that stuff by hand every damn time - it's one of the major things that made me get out of GUI work in the first place

Absolutely. Having sensible and configurable defaults for everything is the way to go.

njl
My Django hosting service. I looked at Heroku and thought, "I want that for Python." Yesterday, I got most of the http request path finished. There is some node.js work left, and I still need to do some more work on Varnish. I'm hoping to get to the Postgresql stuff next week, then on to the website and API...
_3ex7
In a discussion at PyOhio this past weekend, a lot of people were mentioning how they wanted a Heroku-like service for Python.
atomical
Is PiCloud unattractive compared to Heroku?
usaar333
(I'm one of the PiCloud devs)

PiCloud (http://www.picloud.com) is not a web hosting service. Rather, it is a highly scalable compute service.

(In other words, you can't host a website on PiCloud. But your site can run all of its background tasks on PiCloud).

dannyr
njl,

I hear people asking for a Django-equivalent of Heroku. You definitely have a market for this.

usaar333
What were there opinions of AppEngine? (the closest thing there is currently)
dannyr
AppEngine supports Django but just partially. You cannot use the ORM & also very few Django apps/plugins would work.
jordanmessina
You have me sold, please give me the heads up when you're done. I've been looking for this for a long time.
krisneuharth
Yes. Please post it on HN when its ready for some testing.
dangoldin
Working on some facebook integration for my social calendar site: http://www.scenepeek.com
ritonlajoie
Hi, I'm interested in your website, is it live ? I have an error when trying to access it : Solr service not responding.
dangoldin
Yea - had some random issues with Solr indexing. That's the startup life =)

It should be up and running now.

owkaye
In light of its recent meltdown I'm hacking the real estate industry to help stabilize the market, re-establish lost value, and avoid future foreclosures.
duck
http://www.hackernewsletter.com

The past week I have been working on a back end tool to fully automate the creation of the newsletter (the selection of content is done by me). It has been fun working with the MailChimp API.

Zak
I'm working on text classification. I have a decent classifier that's especially suited to author identification. I can think of a few good uses for it; the first one I'm trying to commercialize is academic anti-cheating.
mirkules
You are probably aware of it already, but a lot of universities (mine included) use MOSS (Measure Of Software Similarity) to detect plagiarism in CS classes. Link: http://theory.stanford.edu/~aiken/moss/
Zak
I am. There are also a large number of services that detect plagiarism in essays, but most (all?) only detect direct copying from published sources and sometimes re-use of an essay previously turned in by another student.

I'm targeting the custom essay - services like http://essaymill.com ("our writing, your success"), as well as students paying other students to write their papers.

cabalamat
Detecting and punishing cheating in those circumstances sounds like a Hard Problem. In particular, when your software says 2 essays were probably written by the same student, but both students deny it, how can they reasonably be punished, since there is no proof?
Zak
Questions like "why did you say X?" usually reveal whether a person is actually familiar with what they claim to have written. It's imperfect, certainly, and I would never recommend punishing a student based entirely on an algorithm's result, but I think I can provide a tool to drastically cut down on this sort of academic fraud.

I intend to make it very clear to customers that they should not punish students based only on information provided by my software.

cabalamat
> Questions like "why did you say X?" usually reveal whether a person is actually familiar with what they claim to have written.

That's true.

If an algorithm flags someone as a possible cheat multiple times, it may be worth inspecting that person further.

> I intend to make it very clear to customers that they should not punish students based only on information provided by my software.

Good idea. Customers should however publicise that anti-cheating software is in use.

tocomment
That sounds fascinating. How did you come up with the algorithms to use?
[deleted]
(comment deleted)
Zak
I experimented with existing text classification algorithms for an author identification project I was doing for fun. What I'm currently using is somewhere between KNN and SVM, but I'm not done tweaking it yet. I'm also working on boosting results using different feature sets.
user24
you might try looking at the BLEU metric. It's designed to test similarity between a machine-translated text and a human-translated one, but it could be a good starting point for detecting plagiarism too.
atomical
I ran a site called that crawled Gnutella/Limewire for student papers. That's something you could consider adding to your database and quite easy since the Limewire code and RFC are opensource. You could write your own client or modify Limewire.
k0ban
Security for online banking
smiler
Creating it or breaking it? :)
k0ban
It is similar things actually :) Once you know how to break it, you could improve it.