Ask HN: Is my web app idea worth the effort?

Does Joe Data analyst exist? and if so, is he able to gather enough data, and conceptualize what he is actually looking for? I put this in the realm of wolframalpha, where most people (even hackers & programmers) can't word a query well enough to get the data they want back.

From my experience non-programmers aren't able to manage large enough volumes of data to require data analysis. At the company I used to work for, there were people in IT who's job was specifically to build and run reports at the request of retail managers. The problem would not be resolved if you could just give the retail managers a data analysis tool, as they didn't know how to get the data out of the databases. In some ways, is this the market that pipes or dapper.net are trying to do?

I Like this idea. If you were looking for a target audience you should also include students learning statistics. In college you get to use some of the software but this would be more ideal, hopefully working cross platform. When I took statistics it was a pain running windows on my Mac just for statistics software.

I believe that you will need to make a quantum leap in usability. As pedalpete mentioned, very few people have the skills to process meaningful datasets, even those working in industries where we might expect such skills to be commonplace. The people who do have the mathematical knowledge know how to use the existing software tools.

If this were my idea, I would be looking at smaller niches where I could offer a boneheadedly simple way of doing tasks that are commonplace in a particular industry. I'd focus on accepting the dirtiest, nastiest inputs - excel spreadsheets, tables copied from word documents, tabulated lists - and giving the user a big set of simple preset operations, organised by industry and job role. I'd look further down the food chain to people who might deal with data as part of their job, but have no mathematical background whatsoever.

This might be an odd analogy, but I'm reminded of software synthesisers. Nowadays it's relatively rare that musicians have the technical understanding of synthesis to create their own sounds, even with the most basic tools. Those that do understand synthesis have usually invested a lot of time in learning the interfaces of particular synthesisers and are loathe to discard that investment. Developers are focussing on the unskilled mass market with products that make it easy to access, tweak and combine a big library of preset sounds.

Have you seen this? http://yeroon.net/

It's a bunch of web-based graphical wrappers around R created by a UCLA stats prof, Jeroen Ooms. Slick stuff, for a hobby project.

I agree with jdietrich: In order to be more than a hobby project, you need a "quantum leap in usability." But I'm optimistic. I think someone will definitely figure out how to do this in the next 5 years.

Justin also has a good point that a tool aimed specifically at students would be great. Most intro stats courses are taught using SPSS or Stata, and most of the kids in the course have never done programming before; as if stats weren't a perplexing enough subject in itself!