Ask HN: What can't you do in Excel?
One of the startups in the current YC cycle is making a new, more powerful spreadsheet. If there are any Excel power users here, could you please describe anything you'd like to be able to do that you can't currently? Your reward could be to have some very smart programmers working to solve your problem.
200 comments
[ 2.2 ms ] story [ 988 ms ] threadExcel isn't a database but a lot of people use it like one anyways.
For example, getting the sum bill of every person who lives in England looks like this in one workbook I have:
{SUM(IF($C1:$C$10000 = "England",$E$1:$E10000,0))}
(If I want to see, say, the top 10 outstanding bills in England I have to use a pivot table).
Excels biggest problem is that the workbooks produced with it are really hard to maintain. Looking at the code above, for example... going back to that (which is only a trivial example) in a month is going to be pure pain. Updating data requires cutting and pasting, which can be error prone. Unit testing is only possible with sample known-good data sets, and copying in new data tends to make it less than certain that the version you are using is the same as the one you tested.
Oh, and sharing workbooks between users is really tough. I can usually figure out other people's Java - but I have yet to be able to reverse engineer a non-trivial Excel work flow.
Slightly offtopic, but I just wanted to point out that excel syntax looks a lot like lisp, with biggest different being the first item is outside the parens.
This is true, you can do a lot with this. And if you're prepared to make a lot of columns with the various flags you need, you can build up quite complex queries - having an easy way to do this would be nice though.
I've spent a lot of time in Financial Services and it's surprising how much of the industry is run on spreadsheets, especially investment banking - and I mean online - they'll have Excel running all day, receiving real time price feeds, running a calculation and republishing. Excel is pretty much the glue that holds the whole industry together.
So I also agree on the maintenance issue - These sheets can get quite complex and it's almost impossible for someone to understand coming in cold. It would be very useful to have a workflow model on top - which I guess is really adding the algorithm aspect - but I've never seen a speadsheet metaphor that does this well.
You could do something like this:
{SUM(IF(country_name = "England",bill_amt,0))}
http://www.ozgrid.com/Excel/DynamicRanges.htm
Then why not build database-like functionality into the system?
I'd like v and hlookups to be turned into SQL (and vice versa) with animations explaining what's happening in each.
Or: http://www.exceluser.com/explore/msquery1_1.htm
1. Build predictive models from data I've entered in Excel. I find myself exporting from Excel into R a lot to satisfy this need. Microsoft partially addresses this need with the Data Mining Add-ons for Excel.
2. Have more than ~1 million rows (which is Excel 2007's limit).
3. More easily clean up data in a large spreadsheet.
4. Reversibly anonymize data -- if I download some logs with usernames or IPs, I don't want those in my analysis (I just hide those columns), but I do need to have a unique identifier for each row. And later get the names back if I want.
And a couple things I think Excel is great at already:
1. Making data look pretty. The charts are great, and there's (http://www.officelabs.com/projects/chartadvisor/Pages/defaul...) for non-power users.
2. Making data easily portable.
3. Filtering/sorting data -- VLOOKUP has saved the day so many times.
Grow up and use an actual database.
Slides 4 & 5 give some insight: http://www.scribd.com/doc/2191289/yaron-minskycufp-2006
While I am not a big user of excel, the google docs version of excel is very similar to the desktop version + collaboration (think Oddpost). The other approach taken by some people I have spoken with is viewing it more as data and tying to do more (think gmail to some extent).
(that's the maximum number of rows in Excel before 2007). I'd like something similar to SPSS, which is more convenient for tables with lots of rows and separates the data from the formulae.
A tool for working with streaming data.
Also, charting that works for large amounts data. Try having Excel chart 65,000 rows and you'll have time to make coffee while you wait. There are no ways to zoom or analyze Excel charts, either.
In fact, charting for data analysis alone is a big enough problem that needs solving (and you won't have to do all that catching up with Excel). Short of using HTML/Flash or .NET/Java components, there is nothing an enterprise worker can use. Get access to a Bloomberg terminal from somewhere and play with its charts to see what I mean.
It's a charting and reporting tool over SQL Server (and I thinkg any other ODBC/OLEDB source), it works over large datasets, has both Web UI and desktop UI, and comes in the box with SQL Server itself.
FD: I work in SQL Server, athough not on Reporting Services. FWIW, Reporting is a huge success with our customers.
I have used SQL Reporting Services as well as Business Objects tools and I am pretty impressed with the latest releases from MS.
A round() function that isn't stupid, i.e. it rounds to a certain number of significant figures, rather than relative to the decimal point. Workarounds exist, but I want a no-work-around.
I suppose it's not a problem worthy of very smart programmers, but it still makes me wish for bullets-over-SMTP.
http://www.microsoft.com/mac/itpros/default.mspx?MODE=ct&...
No Linux support of course.
Some kind of auditability of Excel spreadsheets would save enormous amouns of money and time.
Of course it's not as nice as "blame", which I think is what you want here?
When investigating bugs (and more than a few times, we're turning into real code something cobbled out of an Excel spreadsheet, so the reference standard is the old spreadsheet), it helps to know why something is different from another, and why the formula in E5 is different than the formula in E4 or E6. It is incredibly easy to screw up formulas with inserting/removing rows with cut & paste.
Two sample foul ups: http://thedailywtf.com/Articles/The-Great-Excel-Spreadsheet.... http://thedailywtf.com/Articles/The-Revealing-Spreadsheet.as...
I've encountered worse situations in the past than those 2 dailyWTF episodes (as well as seen one old employer's code on that site).
Word has a feature where you can see the changes, what was previously present, and who changed it when. Something like.
http://www.resolversystems.com
Sometimes I miss similar type of scripting. With Excel, I often finish copy-and-pasting raw data into Python string, upon which I then do something procedural.
BTW, I do my spreadsheets with a Von Neumann derivate of a finite state automaton.
This paper claims to prove that the authors' spreadsheet is Turing-complete in contrast to Excel:
http://web.engr.oregonstate.edu/~burnett/Turing/TuringMachin...
Actually, it would be more interesting if spreadsheets were not Turing-complete, given how much people are able to do with them.
Edit: Spreadsheets have no state. This is hard. If only I could delay evaluation :P
I have a few sheets that I use which pull from a non-ODBC URL using http. It populates a range of cells with new data automatically each time the spreadsheet is opened. (on a Mac too).
Yes. This feature implemented well would create a really cool app. What would be even more interesting would a situation where several different sheets could pull data from each using a clever protocol to avoid the churning of values. Imagine different enterprises which each had online sheets describing their current production abilities and current supply needs. With a clever protocol, their production processes be semi-automatically coordinated (and remember, semi-automatic, some throttling is necessary to prevent self-referential cells from feeding back in an unhelpful way).
#5 can be done in Excel by implementing an RTD server (http://office.microsoft.com/en-us/excel/HP030662371033.aspx). An interesting example is here: http://fransking.blogspot.com/2007/03/yfquotertd-real-time-d...
2. Well, Google does this :)
3. Follows from 1.
4. Good idea, but why not just use this: http://www.google.com/search?q=spreadsheet+widget
Or http://docs.google.com/support/bin/answer.py?answer=55244...
5. Already easily doable in Excel http://articles.techrepublic.com.com/5100-10878_11-6115870.h...
In Google Spreadsheet, just use: =GoogleFinance("GOOG"; "price")
------
And I hope Google is working on a REPL for Google Docs where you can interactively run any language on your spreadsheet. :-)
[ credit : http://news.ycombinator.com/item?id=430129 ]
http://projects.gnome.org/gnumeric/doc/sect-extending-python...
Resolver One is a spreadsheet that has that: http://www.resolversystems.com/products/programmability.php
OpenOffice Calc has some kind of Python programmability too, though I don't know how good it is.
My two cents: add support for arbitrary precision math. Sometimes you need it, and when you do, you really need it.
A1 = 0.1*A1
Can be useful if you need to convert values, etc., but don't want to create an entire new row/column
I should be able to make a row of "oil prices by month since 2005" that updates on its own. Or a cell with "current value of the DOW".
If I'm going to make a scatter plot style plot for anything besides an internal memo (even an internal presentation) I always plot with something else.
I know almost nothing about Excel and it's a deficiency I feel quite keenly. The program is almost completely undiscoverable to me and I don't even know where to start with it. I know that there are powerful uses and features of Excel but the model (or at least the bits of it I've been exposed to) hasn't clicked for me yet.
I suspect I'm not alone in my bafflement; maybe there is the possibility this new spreadsheet could be useful to us Excel-ignorant customers who nonetheless want access to the benefits of this class of software.
As a hacker and programmer who is comfortable with all sorts of paradigms I feel really strange admitting this weakness but I suspect that I'm not the only one who is in this boat.
I may not be answering your question since I'm talking about usability instead of more powerful features, but I can't but imagine that there'd be a market for simple and easy to use, even if it turns out it's not going to be addressed by your particular startup...
I have a table, some data that I've laid out in rows and columns. Something simple. How much money I've been paid on my invoices to clients, for example, one invoice per row.
Then I want to sum the column, to get how much I've been paid in total. (Yes, I'm talking about a very simple spreadsheet. But that's my point, that something so simple is still messed up!) So I type in a formula: =sum(C2:C10)
Now I add a row, to put in another entry. Does my sum change, to include the new row? (C2:C11) No, it does not.
So I do not want to be saying sum(C2:C10). I want to say, here is my simple table, and give me the sum of this column. Which, I don't know what the language would look like, but if I named my table "invoices" maybe it would be sum(invoices.C) or sum(invoices.amount) or something.
Every time someone comes out with a new spreadsheet (Excel, OpenOffice, Google Docs...) I look to see if it is easier to use. Nope! Everyone is too busy being compatible with the last guy.
if you use insert row - then formulas respond and will go from C2:C10 to C2:C11 (tested in excel2003 at least)
Of course, it's not perfect if you look at the proprietary format, at the smaller number of formulas than Excel, and so on. But it's a nice piece of software as far as I'm concerned.
It's a moot point since you don't have a Mac but from what you describe, Numbers does what you're missing.
I saw in one of your later comments that you were also talking about multiple tables on the same page. Numbers actually manages tables as independent objects of a page. So, in a table you can ask for the sum of a whole column without getting the numbers from another unrelated table on the same page. That's something that always bothered me in Excel.
For your data C2:C10 - Do a field: sum(C2:C11) Then when you want to add more data - right-click on the row (11) and "insert" That will update your sum calculation.
Also - you can do named fields - so that if you select the fields C2:C11 - then you can name them as "invoices" (in excel 2003 it's in the top left corner - there's a selection box you can type in. Just select and type a name in there). The lets you do the command sum(invoices)
Also - don't forget you can do something like sum(C:C) which will just give you everything in C column..
I.e. leave a blank row at the bottom of my table, and have the sum include that blank row? I actually know about that trick (thanks :)... what I want is a spreadsheet that does what I want without my tricking it.
You can do what you're wanting with dynamic named ranges.
From reading some of your other responses it seems like you don't want to sum the entire row, maybe because you have the sum listed at the bottom of the dataset or something. With a dynamic named range you can add rows to the bottom of the range, and you also get a nice name to reference it by. It works by using offset and count/counta to deliver a range based on how many occupied cells there are (depending on if you use count or counta).
There are a few ways of doing it listed here: http://www.ozgrid.com/Excel/DynamicRanges.htm
In my experience you can do an incredible amount of things in Excel before you even break into doing stuff in VBA. You just have to look at any of the numerous resources out there that have tricky formulas available.
I'm assuming that (in terms of your example) the invoice amounts that you're adding up would form a contiguous range of numbers, and that this range would be bounded by whitespace. That is, you might have some other range that used column C -- say "expenses" -- but it would be lower down, say starting at C15, and there would be at least one blank cell between the two. Is that correct? If it weren't for that lower range, you could just take the sum of the whole column and you'd be good. But it's too inconvenient (and not the "spreadsheet way") to force everything into separate columns.
If the above is correct, how would you feel about being able to define a range with a notation like this: "C2:C✱", meaning "the range of cells that starts at C2 and goes down until it hits whitespace"? Then as you add numbers to C11, C12, etc., the range would automatically expand to include them. But you'd still have to be careful to ensure there was a "moat" of whitespace around your invoice range. If you filled in the last non-whitespace cell before your other table, you'd now have connected the two tables in such a way that "C2:C✱" would leap down to the end of the second table. In other words you'd be lumping "invoices" together with "expenses" which is probably incorrect.
With the list builder you are always given an extra row at the bottom to continue adding to the list. Any formulas below the list builder will be pushed down and expanded.
The list builder also turns on Auto Filters for the list as well.
When I said, "I'm frustrated every single time I use a spreadsheet", it's not that I can't do whatever it is that I need to get done, I just get annoyed when products are made hard to use when they don't have to be. It's not so much a personal frustration as that I've spent a lot of time at non-profits helping non-computer people use computers, and it's a huge waste of their time and of my time to have to train them how to manipulate the software to get what they want instead of the software just doing it.
C2:C11 was a tremendous advance in 1979 when personal computers had 48K of memory and 40x25 character screens, but goodness gracious, it's thirty years later!
Making something easier to use is a tremendous amount of hard work, but it isn't conceptually all that hard to understand: you look at what people are doing, and you write software to implement that, instead of making them manipulate the software to do the implementation themselves.
I haven't looked at it myself so I don't know if Apple got it right or not, but from Timothee's comment that "Numbers actually manages tables as independent objects of a page", it sounds like they're at least trying.
On the other hand, having defined tables with in a workbook is great idea, but it makes the whole application a wee bit more complicated. I'd like to see it in something of a hybrid between Access and Excel, where you can mix structured and tabular data.
The formula should describe the calculation I want performed and it should continue to work even when I enter new data. For example, if I have a "defined table" as you say, I should be able to ask it for a sum of a column in the table, and have it continue to work even if I add new data to the table, with hacks or trickery or invoking obscure commands.
A few years ago I worked at a large financial consulting firm, and I was amazed at how often accountants would implement what basically amounted to an 'inner join' using nested iteration over columns in vba.
This was a few versions of Excel ago, so I don't know if this feature is available in recent versions or not. I imagine not, since then Excel would really start to encroach on Access's domain.
So, in a word, encapsulation.
Also, the line between databases & spreadsheets is fairly thin, how about some relational calculus? Some import/export with SQL? Or a query language?
Of course, if you want it to be successful, you need to be able to import Excel spreadsheets as is, including macros and formula, and you also may need to maintain the linking to/from other microsoft artifacts.
That gives the added security of allowing people to play with the data without changing the data in the database.
There is a spreadsheet interface metaphor. Parts of it are quite useful. Other parts are less convenient given the ways that people program/use spreadsheets. Some of them are better addressed with parts of the database metaphor.
http://ignitedsoftware.com/products/thinkdigits/