"A spreadsheet is then the simplest, most organically natural way of organizing discrete units of information, the easiest way to perform CRUD operations on any type of data."
So... you've basically answered the question here.
"It’s been 40 years since the original Visicalc spreadsheet program was released, and no one has been able to beat them"
There is a reason that columnar workbooks and spreadsheets have been around since humans started writing down numbers and manipulating them... it works.
A "grid of cells" does exactly what it is supposed to do in the most efficient manner possible. The only "innovation" opportunities are making the underlying product suck less, or providing analytics/reporting functionality.
I agree. There may be no better UX alternative than columnar spreadsheets. Excel shines in scenario analysis when, for example, the data comes from your head. I believe Excel has an opportunity to improve when data comes from external sources like CSV dump from a CRM. I would like to see Excel improve the way it allows you to extract data from other systems which would reduce redundant effort and decrease the likelihood of errors.
The key is that you can examine the state of most (not all) "intermediate" variables so not only do you know the inputs, the outputs, but also everything that happened in between in all these different ways.
Your calendar app adds 2 days for tasks? You can just see the cell where it says 2 to make sure that variable is set right. Its amazing, and we are just catching up to it with "always on" variable inspectors in IDEs.
Edit:
That's not the only trick: cells don't have variable names but locations. You don't have to remember the type (they are all cells) or the name of a variable. Its just a location and you click on it to select it.
I think about this a lot. There is a need for more data-driven applications to create graphics that are more “track able” than excel. I have a series of charts that need to be sent out to multiple people (internal and external) each week. Charts, logos, tables, etc could benefit from a pipeline but without needing a data engineer. Right now, we have a standard data download from Bloomberg or another data provider, macros to change the format, access to slice, back to excel where the charts are 75% standard but might need some tweaking. Bloomberg might not classify one company as healthcare but I want to include them so access keeps a list or companies I want. That same Bloomberg output could include companies some of my colleagues would classify as tech companies so relevant for multiple people. No way to see what was done last week or changes made to the chart / table output from one person to another besides new versions of the same excel file (but still no way to see the logs)
Long winded description but hopefully helps describe how important excel is but there is still room to grow.
>No way to see what was done last week or changes made to the chart / table output from one person to another besides new versions of the same excel file (but still no way to see the logs)
Why are multiple columns/pages with the various stages of the data insufficient?
Isn't this just a programming problem? Build an application that accepts the data, produces the result, and has a GUI for managing all exceptions? I've build many of these things.
What seems to be described is a UI design problem. Once the UI is designed, it also needs programmed in some form, but that doesn't seem to be the problem, just a not-particularly-interesting task required in the implementation.
Not all problems that require programming are programming problems. (In fact, most are not.)
I meant "isn't this a problem that requires software development?"
There are always complaints that Excel is a terrible solution to some very specific set of repeated tasks. Every few months, there's a posting about how all we need is a "Better Excel" that would perfectly solve everyone's totally unique 12-step problem.
Such a tool is never going to exist. Thankfully we can build tools specifically for those problems when generic tools aren't good enough. Not every problem is worth a programming solution; there's only so much time and money and paying a bunch of people to cut and paste might just be the better option.
It sounds like you are looking for Excel's PowerQuery functionality? You should not need to bring the data into Access, or write Macro's to do what you are doing anymore (Macro's were required in older versions of Excel, but not newer ones that have the data modelling capabilities).
This effectively is a data pipeline built within Excel that can either be edited visually or in 'M-Code'.
Bettering a table for data is going to be hard. Calculated fields (instead of $L$123, $Total) could be represented in other ways to bound to cells. As could code. But honestly, with the amount of time people already waste formatting spreadsheets, less options is probably a productivity enhancement.
Working with jupyter, I'm seriously contemplating around how to bring the best of Excel into a pandas or jupyter workflow. Mostly for exploration but also making reports, statistics, aggregations
(I'm pretty biased since I wrote a book on this, Effective Pandas.)
My take is that if you embrace the limitation of chaining in Pandas it will force you to write better (easier to read, debug, deploy, share, collaborate) code.
I can't look very deeply into your link and see what the courses actually say, from here, so I'm not sure. I think you want to teach me to be better at pandas. That's fine and I learn pandas constantly, but not what I'm talking about. I want to mix in a different way of working with data, getting some of the best features of excel into the notebook workflow.
I also believe that getting to know your data means reading it in a grid, not just looking at aggregations. Both are important, and with aggregations you can miss important things! Sometimes the simplest solutions are the best.
I liked the way Apple did it in their Numbers clone of Excel - you can have multiple grids on a single page. It makes it a lot easier to have related data on the same page without fiddling with the row/column sizes to suite multiple types of data.
When I was in university I was working on some biology homework and didn't have Excel, but I did have Numbers! I quickly became annoyed with how smart Numbers tried to be with formatting. It knew better than me what data type some cells were and iirc it was impossible to inform it otherwise. I bought Excel after losing an hour or so of my time.
Are you using it on Apple Silicon? If so, then Rosetta is the reason why Excel are not performant on macOS. Office365 for macOS are using x86 code at the moment, Office team is rewriting Excel to work on Apple Silicon/ARM. They have a preview build out, I'm not sure when they will release the stable version. I recalled they said it should be release in Spring or Summer.
If you have Excel running, open the Activity Monitor and find the app in the list. Then look at "Kind" column, you will see "Intel" listed. So that's why Office365 are sluggish on it.
Running on a 2.4 Ghz Intel chip with 8 cores. The Excel code on windows got a lot more love than the mac code. Probably due to apple's OS and processor architecture changes over the last 20 years, whereas windows maintained great backwards compatibility the whole time.
Took me a while why Office is still using Intel code. Turns out they have "Open with Rosetta" enabled. I disabled Rosetta on those apps and it went to use Apple Silicon code. Now it is snappier than using Rosetta.
Apple's productivity and utility software generally tends to be my favorite around. It's a big part of what keeps me on the platform.
Safari? Lightest-weight usably-well-supported browser around, by a long shot. Preview? Outstanding for a bunch of reasons, including that using it is the only time I've been happy to receive PDF files. Pages, Numbers, Keynote? More than enough for everything I do, stable, and I like that I can leave them open in the background for weeks and they're light enough that I forget they're there. Notes? Not having a built-in export function is annoying and I wish I could use markdown formatting, but it's so good at everything else that those haven't been enough for me to switch to something else. Hell, I even like the calculator better than most others.
Exporting all sheets as individual csvs in one go…
Recently I’ve just been using numbers because it’s there on the odd occasions I need to access an xslx file and, while everything is a little different, it’s just better.
Why would you claim Numbers is a clone of Excel? Spreadsheets existed before Excel, including spreadsheets with dominant market share: VisiCalc on Apple II, MultiPlan on CP/M, and Lotus 1-2-3 on MS-DOS. Excel was essentially a rewrite of MultiPlan specifically for Macintosh.
I've shifted to Airbase for pretty much anything data sheet related. The ability to have more typical data-flow joins with an easier to manage UX that teaches folks along the way is pretty phenomenal. It grows as you need, but still remains relatively simple.
It's one of my favorite pieces of software from the past decade.
> The ability to have more typical data-flow joins
This is available in Excel too - you can import data with PowerQuery, perform joins, configure relationships between multiple tables, then output that into a chart, table or pivot table. You can even put slicers on to let users interact with the data.
This is entirely automatable too, so if the underlying data changes you can just run the pipeline again.
in the metaverse, you will have 3d tables, x,y and z axis, you would be able to twist and turn those 3d tables to get different views, join with other 3d tables
Excel is seriously killing humanity. No version control, no debugging, brittle support for automation. Excel, to me, is the single most obvious sign that we must get rid of the giant tech monopolies to re-enable innovation in software.
Umm, put your excel file in git repo if you really need to? That's like saying C++ doesn't have version control.
>no debugging
Debug what exactly? If you have vb scripts, then yes you can debug. If you just have formulas.. there is literally nothing to debug.
>brittle support for automation
Ok, but that's also kind of the point. Excel should not be some insane thing where people do way too much. It's a spreadsheet with formulas- and that is what it's goal is. People who use excel with thousands of lines of VB code should literally be using something else 99% of the time.
>Excel, to me, is the single most obvious sign that we must get rid of the giant tech monopolies to re-enable innovation in software.
There is basically nothing to "innovate" related to excel. It's exactly what is needed. If you want to innovate then write your own damn spreadsheet that does some new magic you think of.
Why are you trying to overcomplicate a simple 2d spreadsheet?
>Excel is seriously killing humanity
Excel is doing the exact opposite. You are a fool saying that- excel is so freaking simple yet so powerful. Sounds like you might live in academic lala land and have never worked at a business where there are a billion different types of things to do and simple spreadsheets can generally cover most cases.
You're interesting when talking about Excel's deficiencies. But you ruin your comment with talk about "killing humanity" and "giant tech monopolies". HN is not a place to get swept away by emotion.
Yes, Excel spreadsheets frequently contain errors that are hard to spot. Sometimes those errors end people’s careers and damage companies. Tools like Quantrix that use multi-dimensional models provide a formula syntax that’s radically less error prone while providing far superior reporting.
When NeXT launched their first machine back in the late 1980's, it came with a spreadsheet 2.0 called Lotus Improv. It used what we now call pivot tables as the first class data representation. I never used it but the demos looked very cool.
If your data consists of a long complicated pipeline, there might be some useful UX coming out of shader graphs systems from computer graphics. Shaders transform data in a functional way and you can build large graphs that also clearly show the inputs and outputs of a transformation. This might be easier to debug than a spreadsheet. Any point in the pipeline can be output to a visualizer in this way.
It would make debugging, understanding and inspecting the dataflow easier, but it would probably make browsing the output a bit harder so I can't say it's an obvious slam dunk. Might be interesting though.
I have just dabbled with Houdini and what you say (it is not exactly a shader but its procedural interface is probably similar to what you are saying) is spot-on.
Lots of data incoming, a graph of operations, lots of data (and plots, and what not) outgoing.
Thanks for posting! I love the topic - I've written before [1] about how I think spreadsheets are the most popular programming paradigm ever, we just don't talk about it much.
I personally think that the evolution of spreadsheets is less about changing the UI, and instead making it possible for spreadsheet users to easily transition to more powerful programming tools in a natural and easy way. So I've spent the past 2 years building Mito [2].
Mito is a spreadsheet extension to your JupyterLab environment. You can display any Pandas dataframe as a spreadsheet, and edit it in a very similar way to Excel. For each edit you make, it generates the corresponding Python code below for those edits. Practically, you can think about Mito as recording a macro, but instead of generating scummy-crummy VBA code, it generates Python.
We currently have two types of users. 1) Excel users from a huge variety of industries who are somewhere in their journey to learning Python - and Mito helps them write Python scripts quickly and make that transition easier. 2) Python users who prefer using Mito because of it's visual interface. I pretty much only use Mito when I'm trying to pivot or graph data - some things really just are better visually, especially when you get code out that you can edit if you want!
We're open core [3], and also sell a Pro and Enterprise versions of the tool with advanced functionality. We've been steadily growing for the past year or so, as the product has improved (first time founder here!).
I might have missed the point but the Mito demo just shows importing a CSV file into a common table/grid of cells? Seems like a semi-shamless plug that 100% avoids the topic of the post: is there a better visual data model than a grid of cells?
The question of "what is a better visual data model than VisiCalc" is just one of the questions we can ask ourselves about how to create Excel 2.0. In the authors original post, they point out that Notion and Coda answer this question not with an evaluation on "cells" and the data model, but rather by extending the model to include a word processor. This isn't a purely visual change, but a functionality/integration one.
There are a bunch of different angles to consider the evolution of a spreadsheet, and, as I say in my response above, I personally think focuses on changes to the UI/display of data miss the point: what's missing in Excel 1.0 isn't a better display of data - IMO, it's giving the modern, powerful analysis tools that us programmers have access to the beginner-end of the programmer spectrum!
Different spreadsheet startups certainly have different theses on this. Subset [1] (the OP) seems to focus on side-by-side grids on an infinite canvas. Monday [2] (also referenced by OP) seems to focus on different "views" for a spreadsheet for project tracking, etc. Mito focuses on allowing you to integrate Python and spreadsheets as easily as possible. Clay [3] seems to focus on spreadsheet integrations into APIs/other data.
(Disclaimer: all the above are just my understandings of these tools, but I haven't used most of them directly mostly am just going of marketing materials... I highly recommend you check them out, though - they all look quite cool!)
My post def was a plug for Mito - I'll try and make my response to the post/thesis more clearly delineated in the future. I think this post is an awesome chance to get feedback on our spreadsheet thesis (and potentially hear back from OP on this thoughts!).
I don't feel that a grid of cells is the best visual representation for data. However, it is a natural representation for tabular data.
Tables are horrible for "visualizing". We didn't evolve looking at tables with hundreds of columns and millions of rows.
Once you learn some pivoting tools and charting, you can visualize things that you would never be able to find in a table of data. (Or if you did it would take a lot longer.)
(Sample size 1, but I teach Python, Pandas, and visualization (Jupyter w/ matplotlib/seaborn/bokeh). I've had clients tell me that one chart they came up with during a class on visualization more than paid for the training. They would have never seen that in the table of data. I've also found bugs in code by visualizing failure patterns.)
> I personally think that the evolution of spreadsheets is less about changing the UI, and instead making it possible for spreadsheet users to easily transition to more powerful programming tools in a natural and easy way. So I've spent the past 2 years building Mito [2].
Following the article idea of spreadsheet as the best paradigm, why you think users should abandon it in favor of other?
Spreadsheets are the best way to represent data, but there is a very long-tail of tasks one might to perform with their data. Requiring all of them to be built into a visual UI pretty much insists you end up with Excel - what feels like 1 billion features, where each user knows <1% of them (and costs millions of programmer-hours to build).
I don't think users can/should leave spreadsheets for the tasks spreadsheets make sense for (basic data munching, pivoting, many formulas, etc). But being able to easily transition your spreadsheet to other tools in a easy/native way is a huge win - and why at least half of our users are actually just Python programmers who use Mito because it makes that transition back and forth to spreadsheet/code so easy!
I don't think anyone should abandon tools if they are working for them :-)
I always encourage and applaud efforts to improve the stats quo, I suspect however that if there were a more efficient way for power users to quickly interact with data we would have found it by now.
In fact, to interact with large sets of data we have found it and it’s SQL. Again for power users. You data geeks are of course an exception and have a completely different set of tools.
But it differs a lot by task. The power of the spreadsheet model is that it is minimally acceptable for a wide array of tasks, not that it is usually optimal.
I think that main strength is familiarity and low learning curve. You can easily build and MVP with a spreadsheet. Understand your data, relations, patterns. The trap occurs later when you need to scale it
Spreadsheet grids are marvelous for presenting many kinds of data relevant to business. Specifically, tabular numerical data (rows of tuples with well-characterized columns). Spreadsheets are great for condensing and summarizing the minute details and aggregates spread across huge numbers of rows, and for highlighting patterns and trends.
And there are other very effective ways to present data. Hypertext, Gantt charts, and pie-charts for example, which Excel also supports.
But we don't use spreadsheet-grids for general programming. Programs (as we write them) are concerned with dependencies and control-flow and semi-structured hierarchies and naming lots of things. Programs are organized as a hierarchy - directories containing files containing the nested-pieces of the program, as text. And some parts of a program (state-machines, data-schemas, GUI layouts, date/control flows) are visualized as boxes containing labels with lines and arrows between them (and more labels). I'm surprised we don't have generic tools for that yet. Attempts have been made.
This is not to mention geometric/photographic/aural data.
I don't think programs are necessarily a hierarchy, though that is a useful way to look at them. There are definitely mutually recursive elements, though these are admittedly limited in size of scope. However, most of the problems with looking at programs as hierarchies come from forcing elements to have a single parent containing element. Instead of a tree, you get a lot more flexibility by allowing elements to have multiple parents (so it is still a DAG, but not a tree since a path from root to the element is not necessarily unique).
For humans, hierarchies are easier to think about and easier to format. But you're right: sometimes you really want to express your structure in a more general graph. And format it more graphically.
For me the most obvious examples are: state-machines and entity-rln diagrams (and, for example, showing how your C-structs point to each other and how you're using std containers and ownership).
Thanks for pointing out that what I really want is a way to create/edit a graph.
You can argue that spreadsheet formulas in cells are also a type of programming, including all the hierarchy you mentioned. Only in this case the hierarchy is not just the functions nested in each other, but also the cells calculating data based on other calculations in other cells.
And while I see the OP explanation on how cells are a great way to arrange data, I would argue that the existing way of programming in Excel is pretty horrible.
Anyone who have ever tried doing something slightly complex with Excel functions soon realized that it is pretty impossible to do certain things without a lot of "magic" involved. Which is why MS added the whole VBScript thing, and even Google Sheets have their JS App Script (or whatever it is called) to provide additional options to program based on data beyond the basic formulas.
It would be great to see more visual programming languages tested in an Excel like data entry environment. Some of the PLC [1] languages come to mind, or even languages like MIT Scratch.
> But the data isn’t persistent enough, we only get recent commands, and the location is always moving and we can’t easily reference older calculations.
You need a better shell, my friend. To quote my .zshrc
I scanned the comments to see if anyone had mentioned Causal[1] yet. I’d describe it as a spreadsheet specifically for modelling.
I used it to build a cost model for the startup I work at, consisting of around 100 different inputs, and it was rather enjoyable.
I was able to (reasonably easily) insert the different AWS costs for (for example) SD vs. HD video transcoding and see how that affected the costs of encoding and storing video 12 months from now.
I guess some Excel users just outgrow the app. It's time to write some code in R or Python, along side with Jupyter-like notebooks. Another option is to use more specialized tools like Tableau.
The table or spread sheet is not very "visual" after all.
HTTPS://ObservableHQ.com is rows of cells where cells can be code or DOM, input widgets. Cells update themselves automatically like spreadsheets. It's is fairly insane what can done and the information content possible with these interactive notebooks that run normal JavaScript (in a non-linear fashion)
Even though current VR/AR interfaces are completely useless for Excel or other data-management tasks (no keyboard support, unreliable controls, lack of development interest), I think in the future there are more embodied / spatial treatments of data access that could feel like an improvement on '2D' Excel.
As the author mentions, the original moniker for Excel was VisiCalc - a visual calculator. There's no inherent reason why 3D spatial representation would be a worse medium for a calculator.
There has been a lot of work looking into 3D visualization, but it really seems like the benefits are pretty minimal compared to the drawbacks. Even 2D visualizations seem to do better when limited to a single spatial dimension for carrying information (i.e., how pie charts are inferior bar charts in almost every way).
I should disclaim that I work on a 3D capture app called Polycam, but in that work I've grown used to the idea that 3D captures are inherently better at conveying some kinds of visual information than photographs are. Like a room with graffiti on the walls. The opposite is also true - 2D photos are way better at sunsets & portraits.
So I guess what I'm saying is that I'd bet there are some undiscovered cases where 3D is going to be better for data representation / manipulation.
That's a great point. I had a VR headset that worked with phone a few years ago, and it was absolutely incredible how 3D still images have the ability to make you feel like you are someplace that you are not, at least compared to 2D still images and video. 3D will certainly give designers more tools to work with in making memorable visualizations which is an important feature for many visualizations.
> As the author mentions, the original moniker for Excel was VisiCalc
VisiCalc wasn't actually a moniker for Excel. It was a predecessor. It was the first spreadsheet program, which was made by a different company, VisiCorp, and released in 1979. Excel was developed by Microsoft and released in 1985. Prior to Excel, Microsoft had released an earlier spreadsheet called Multiplan in 1982.
> There's no inherent reason why 3D spatial representation would be a worse medium for a calculator.
The main inherent reason why 3D isn't as great as it seems is that human vision can't see through solids. We don't perceive an entire 3D volume, we just perceive the part of its surface that faces us. We can obviously get more information from stereoscopic vision compared to 2D, but it's not a full other dimension of complete volumetric data. We mostly see a 2D surface with some depth information.
If the argument is that you need the third dimension to reflect of the shape of the data, you're not going to want to stop at three dimensions when working with stuff like multi-dimensional tensors for machine learning, etc. So any 3D display system will have the same problem displaying a 4D grid as a 2D display system has displaying a 3D grid.
Of course any >2D spreadsheet or data viewing / editing / programming language (i.e. Python / Numpy / TensorFlow / Dwarf Fortress / Minecraft / etc) needs to project and slice high dimensional data onto the 2D screen somehow, because displays and human retinas are 2D by nature.
But if it's a practical question of optimizing for human perception (retinas are 2D), engineering (screens are 2D), usability (you can't see or click on something that's hidden behind something else), and user interface design, then 2D wins hands down over 3D.
Dave Ackley, who developed the Moveable Feast Machine, had some interesting thoughts about moving from 2D to 3D grids of cells, suggesting finite layering in z (depth), but unlimited scaling in x and y (2D grid):
DonHopkins on Oct 1, 2019 | parent | context | favorite | on: Wolfram Rule 30 Prizes
Very beautiful and artistically rendered! Those would make great fireworks and weapons in Minecraft! From a different engineering perspective, Dave Ackley had some interesting things to say about the difficulties of going from 2D to 3D, which I quoted in an earlier discussion about visual programming:
David Ackley, who developed the two-dimensional CA-like "Moveable Feast Machine" architecture for "Robust First Computing", touched on moving from 2D to 3D in his retirement talk:
"Well 3D is the number one question. And my answer is, depending on what mood I'm in, we need to crawl before we fly."
"Or I say, I need to actually preserve one dimension to build the thing and fix it. Imagine if you had a three-dimensional computer, how you can actually fix something in the middle of it? It's going to be a bit of a challenge."
"So fundamentally, I'm just keeping the third dimension in my back pocket, to do other engineering. I think it would be relatively easy to imagine taking a 2D model like this, and having a finite number of layers of it, sort of a 2.1D model, where there would be a little local communication up and down, and then it was indefinitely scalable in two dimensions."
"And I think that might in fact be quite powerful. Beyond that you think about things like what about wrap-around torus connectivity rooowaaah, non-euclidian dwooraaah, aaah uuh, they say you can do that if you want, but you have to respect indefinite scalability. Our world is 3D, and you can make little tricks to make toruses embedded in a thing, but it has other consequences."
Here's more stuff about the Moveable Feast Machine:
222 comments
[ 3.1 ms ] story [ 255 ms ] threadSo... you've basically answered the question here.
"It’s been 40 years since the original Visicalc spreadsheet program was released, and no one has been able to beat them"
There is a reason that columnar workbooks and spreadsheets have been around since humans started writing down numbers and manipulating them... it works.
A "grid of cells" does exactly what it is supposed to do in the most efficient manner possible. The only "innovation" opportunities are making the underlying product suck less, or providing analytics/reporting functionality.
But yeah, it would be cool if you could pull directly from an API in to Excel. Or directly parse JSON input without having to mangle it in jq first.
Your calendar app adds 2 days for tasks? You can just see the cell where it says 2 to make sure that variable is set right. Its amazing, and we are just catching up to it with "always on" variable inspectors in IDEs.
Edit: That's not the only trick: cells don't have variable names but locations. You don't have to remember the type (they are all cells) or the name of a variable. Its just a location and you click on it to select it.
Long winded description but hopefully helps describe how important excel is but there is still room to grow.
Why are multiple columns/pages with the various stages of the data insufficient?
What seems to be described is a UI design problem. Once the UI is designed, it also needs programmed in some form, but that doesn't seem to be the problem, just a not-particularly-interesting task required in the implementation.
Not all problems that require programming are programming problems. (In fact, most are not.)
There are always complaints that Excel is a terrible solution to some very specific set of repeated tasks. Every few months, there's a posting about how all we need is a "Better Excel" that would perfectly solve everyone's totally unique 12-step problem.
Such a tool is never going to exist. Thankfully we can build tools specifically for those problems when generic tools aren't good enough. Not every problem is worth a programming solution; there's only so much time and money and paying a bunch of people to cut and paste might just be the better option.
This effectively is a data pipeline built within Excel that can either be edited visually or in 'M-Code'.
It's made by the team behind rmarkdown and is easily one of the best ways to make reporting easier.
My take is that if you embrace the limitation of chaining in Pandas it will force you to write better (easier to read, debug, deploy, share, collaborate) code.
Also works for exploration, reports, stats, aggregations. (Many examples here https://store.metasnake.com/ )
I also believe that getting to know your data means reading it in a grid, not just looking at aggregations. Both are important, and with aggregations you can miss important things! Sometimes the simplest solutions are the best.
If you have Excel running, open the Activity Monitor and find the app in the list. Then look at "Kind" column, you will see "Intel" listed. So that's why Office365 are sluggish on it.
Thanks for letting me know!
Safari? Lightest-weight usably-well-supported browser around, by a long shot. Preview? Outstanding for a bunch of reasons, including that using it is the only time I've been happy to receive PDF files. Pages, Numbers, Keynote? More than enough for everything I do, stable, and I like that I can leave them open in the background for weeks and they're light enough that I forget they're there. Notes? Not having a built-in export function is annoying and I wish I could use markdown formatting, but it's so good at everything else that those haven't been enough for me to switch to something else. Hell, I even like the calculator better than most others.
* Freeze header rows & columns.
* Naming header rows & columns.
* Graphs that don't overlap the sheet.
Things that I find Excel does better than Numbers from a data perspective:
* Data validation
* Large tables
* Formula Error checking
But anyway...
* You can freeze header rows and columns in Excel.
* You can place charts outside the sheet with data.
* I'm not even 100% sure you can't name columns, but let's say you can't.
So, Excel has 2 out of its 3 "missing features". Just saying.
Recently I’ve just been using numbers because it’s there on the odd occasions I need to access an xslx file and, while everything is a little different, it’s just better.
It's one of my favorite pieces of software from the past decade.
This is available in Excel too - you can import data with PowerQuery, perform joins, configure relationships between multiple tables, then output that into a chart, table or pivot table. You can even put slicers on to let users interact with the data.
This is entirely automatable too, so if the underlying data changes you can just run the pipeline again.
No VR needed. Heck, you could do that even with a 486 and a terminal attached to it.
https://github.com/samuelludwig/teapot
GNU teapot intro from the old K.Mandla:
https://kmandla.wordpress.com/2010/08/11/how-to-use-teapot-l...
Umm, put your excel file in git repo if you really need to? That's like saying C++ doesn't have version control.
>no debugging
Debug what exactly? If you have vb scripts, then yes you can debug. If you just have formulas.. there is literally nothing to debug.
>brittle support for automation
Ok, but that's also kind of the point. Excel should not be some insane thing where people do way too much. It's a spreadsheet with formulas- and that is what it's goal is. People who use excel with thousands of lines of VB code should literally be using something else 99% of the time.
>Excel, to me, is the single most obvious sign that we must get rid of the giant tech monopolies to re-enable innovation in software.
There is basically nothing to "innovate" related to excel. It's exactly what is needed. If you want to innovate then write your own damn spreadsheet that does some new magic you think of.
Why are you trying to overcomplicate a simple 2d spreadsheet?
>Excel is seriously killing humanity
Excel is doing the exact opposite. You are a fool saying that- excel is so freaking simple yet so powerful. Sounds like you might live in academic lala land and have never worked at a business where there are a billion different types of things to do and simple spreadsheets can generally cover most cases.
* VBA language and debugger is obviously obsoleted. Please don't truncate values on variable pane, etc...
* Aside from VBA, inspecting each expression would help debugging.
Formula and macro debugging is available for years.
Lambs functions are a thing
Power BI
It would make debugging, understanding and inspecting the dataflow easier, but it would probably make browsing the output a bit harder so I can't say it's an obvious slam dunk. Might be interesting though.
Lots of data incoming, a graph of operations, lots of data (and plots, and what not) outgoing.
I personally think that the evolution of spreadsheets is less about changing the UI, and instead making it possible for spreadsheet users to easily transition to more powerful programming tools in a natural and easy way. So I've spent the past 2 years building Mito [2].
Mito is a spreadsheet extension to your JupyterLab environment. You can display any Pandas dataframe as a spreadsheet, and edit it in a very similar way to Excel. For each edit you make, it generates the corresponding Python code below for those edits. Practically, you can think about Mito as recording a macro, but instead of generating scummy-crummy VBA code, it generates Python.
We currently have two types of users. 1) Excel users from a huge variety of industries who are somewhere in their journey to learning Python - and Mito helps them write Python scripts quickly and make that transition easier. 2) Python users who prefer using Mito because of it's visual interface. I pretty much only use Mito when I'm trying to pivot or graph data - some things really just are better visually, especially when you get code out that you can edit if you want!
We're open core [3], and also sell a Pro and Enterprise versions of the tool with advanced functionality. We've been steadily growing for the past year or so, as the product has improved (first time founder here!).
Feedback greatly appreciated!
[1] https://naterush.io/blog/Spreadsheets-are-the-Ultimate-Progr...
[2] https://trymito.io
[3] https://github.com/mito-ds/monorepo
There are a bunch of different angles to consider the evolution of a spreadsheet, and, as I say in my response above, I personally think focuses on changes to the UI/display of data miss the point: what's missing in Excel 1.0 isn't a better display of data - IMO, it's giving the modern, powerful analysis tools that us programmers have access to the beginner-end of the programmer spectrum!
Different spreadsheet startups certainly have different theses on this. Subset [1] (the OP) seems to focus on side-by-side grids on an infinite canvas. Monday [2] (also referenced by OP) seems to focus on different "views" for a spreadsheet for project tracking, etc. Mito focuses on allowing you to integrate Python and spreadsheets as easily as possible. Clay [3] seems to focus on spreadsheet integrations into APIs/other data.
(Disclaimer: all the above are just my understandings of these tools, but I haven't used most of them directly mostly am just going of marketing materials... I highly recommend you check them out, though - they all look quite cool!)
My post def was a plug for Mito - I'll try and make my response to the post/thesis more clearly delineated in the future. I think this post is an awesome chance to get feedback on our spreadsheet thesis (and potentially hear back from OP on this thoughts!).
Rock on, spreadsheets :-)
[1] https://subset.so
[2] https://monday.com
[3] https://www.clay.run
Tables are horrible for "visualizing". We didn't evolve looking at tables with hundreds of columns and millions of rows.
Once you learn some pivoting tools and charting, you can visualize things that you would never be able to find in a table of data. (Or if you did it would take a lot longer.)
(Sample size 1, but I teach Python, Pandas, and visualization (Jupyter w/ matplotlib/seaborn/bokeh). I've had clients tell me that one chart they came up with during a class on visualization more than paid for the training. They would have never seen that in the table of data. I've also found bugs in code by visualizing failure patterns.)
Following the article idea of spreadsheet as the best paradigm, why you think users should abandon it in favor of other?
I don't think users can/should leave spreadsheets for the tasks spreadsheets make sense for (basic data munching, pivoting, many formulas, etc). But being able to easily transition your spreadsheet to other tools in a easy/native way is a huge win - and why at least half of our users are actually just Python programmers who use Mito because it makes that transition back and forth to spreadsheet/code so easy!
I don't think anyone should abandon tools if they are working for them :-)
In fact, to interact with large sets of data we have found it and it’s SQL. Again for power users. You data geeks are of course an exception and have a completely different set of tools.
But it differs a lot by task. The power of the spreadsheet model is that it is minimally acceptable for a wide array of tasks, not that it is usually optimal.
And there are other very effective ways to present data. Hypertext, Gantt charts, and pie-charts for example, which Excel also supports.
But we don't use spreadsheet-grids for general programming. Programs (as we write them) are concerned with dependencies and control-flow and semi-structured hierarchies and naming lots of things. Programs are organized as a hierarchy - directories containing files containing the nested-pieces of the program, as text. And some parts of a program (state-machines, data-schemas, GUI layouts, date/control flows) are visualized as boxes containing labels with lines and arrows between them (and more labels). I'm surprised we don't have generic tools for that yet. Attempts have been made.
This is not to mention geometric/photographic/aural data.
For me the most obvious examples are: state-machines and entity-rln diagrams (and, for example, showing how your C-structs point to each other and how you're using std containers and ownership).
Thanks for pointing out that what I really want is a way to create/edit a graph.
And while I see the OP explanation on how cells are a great way to arrange data, I would argue that the existing way of programming in Excel is pretty horrible.
Anyone who have ever tried doing something slightly complex with Excel functions soon realized that it is pretty impossible to do certain things without a lot of "magic" involved. Which is why MS added the whole VBScript thing, and even Google Sheets have their JS App Script (or whatever it is called) to provide additional options to program based on data beyond the basic formulas.
It would be great to see more visual programming languages tested in an Excel like data entry environment. Some of the PLC [1] languages come to mind, or even languages like MIT Scratch.
[1] https://en.wikipedia.org/wiki/Programmable_logic_controller#...
> But the data isn’t persistent enough, we only get recent commands, and the location is always moving and we can’t easily reference older calculations.
You need a better shell, my friend. To quote my .zshrc
HISTSIZE=100000
SAVEHIST=100000
setopt inc_append_history share_history HIST_IGNORE_DUPS
bindkey '^R' history-incremental-pattern-search-backward
The results?
wc -l ~/.zsh_history
55576 /home/chx/.zsh_history
and I can pattern search it.
I used it to build a cost model for the startup I work at, consisting of around 100 different inputs, and it was rather enjoyable.
I was able to (reasonably easily) insert the different AWS costs for (for example) SD vs. HD video transcoding and see how that affected the costs of encoding and storing video 12 months from now.
[1] https://www.causal.app/
It just needs more… AirTable style functionality to be all on one like Excel.
The table or spread sheet is not very "visual" after all.
https://observablehq.com/@tomlarkworthy/notebooks2021
As the author mentions, the original moniker for Excel was VisiCalc - a visual calculator. There's no inherent reason why 3D spatial representation would be a worse medium for a calculator.
So I guess what I'm saying is that I'd bet there are some undiscovered cases where 3D is going to be better for data representation / manipulation.
VisiCalc wasn't actually a moniker for Excel. It was a predecessor. It was the first spreadsheet program, which was made by a different company, VisiCorp, and released in 1979. Excel was developed by Microsoft and released in 1985. Prior to Excel, Microsoft had released an earlier spreadsheet called Multiplan in 1982.
The main inherent reason why 3D isn't as great as it seems is that human vision can't see through solids. We don't perceive an entire 3D volume, we just perceive the part of its surface that faces us. We can obviously get more information from stereoscopic vision compared to 2D, but it's not a full other dimension of complete volumetric data. We mostly see a 2D surface with some depth information.
Of course any >2D spreadsheet or data viewing / editing / programming language (i.e. Python / Numpy / TensorFlow / Dwarf Fortress / Minecraft / etc) needs to project and slice high dimensional data onto the 2D screen somehow, because displays and human retinas are 2D by nature.
But if it's a practical question of optimizing for human perception (retinas are 2D), engineering (screens are 2D), usability (you can't see or click on something that's hidden behind something else), and user interface design, then 2D wins hands down over 3D.
Dave Ackley, who developed the Moveable Feast Machine, had some interesting thoughts about moving from 2D to 3D grids of cells, suggesting finite layering in z (depth), but unlimited scaling in x and y (2D grid):
https://news.ycombinator.com/item?id=21131468
DonHopkins on Oct 1, 2019 | parent | context | favorite | on: Wolfram Rule 30 Prizes
Very beautiful and artistically rendered! Those would make great fireworks and weapons in Minecraft! From a different engineering perspective, Dave Ackley had some interesting things to say about the difficulties of going from 2D to 3D, which I quoted in an earlier discussion about visual programming:
https://news.ycombinator.com/item?id=18497585
David Ackley, who developed the two-dimensional CA-like "Moveable Feast Machine" architecture for "Robust First Computing", touched on moving from 2D to 3D in his retirement talk:
https://youtu.be/YtzKgTxtVH8?t=3780
"Well 3D is the number one question. And my answer is, depending on what mood I'm in, we need to crawl before we fly."
"Or I say, I need to actually preserve one dimension to build the thing and fix it. Imagine if you had a three-dimensional computer, how you can actually fix something in the middle of it? It's going to be a bit of a challenge."
"So fundamentally, I'm just keeping the third dimension in my back pocket, to do other engineering. I think it would be relatively easy to imagine taking a 2D model like this, and having a finite number of layers of it, sort of a 2.1D model, where there would be a little local communication up and down, and then it was indefinitely scalable in two dimensions."
"And I think that might in fact be quite powerful. Beyond that you think about things like what about wrap-around torus connectivity rooowaaah, non-euclidian dwooraaah, aaah uuh, they say you can do that if you want, but you have to respect indefinite scalability. Our world is 3D, and you can make little tricks to make toruses embedded in a thing, but it has other consequences."
Here's more stuff about the Moveable Feast Machine:
https://news.ycombinator.com/item?id=15560845
https://news.ycombinator.com/item?id=14236973
The most amazing mind blowing demo is Robust-first Computing: Distributed City Generation:
https://www.youtube.com/watch?v=XkSXERxucPc
And a paper about how that works: