Ask HN: Why does visual programming suck?
When I asked Alan Kay this question, his first response was "well there are various types of visual programming, some of them have taken off". The type of visual programming I'm thinking about here is a WYSIWYG drag and drop interface for building UIs. Imagine having prebuilt views (tables, buttons, dropdowns, etc.) and being able to drag them and lay them out how you want. Then you interact with their API: this table pulls in data from this SQL statement; this button executes this HTTP query; this dropdown publishes this value, which this SQL statement uses via string interpolation. These "widgets" publish their data, which can be used by other widgets (w1.value), and inside SQL queries. The tables can pull in data from SQL queries. You can use string interpolation inside SQL queries, so dropdowns can influence the query being run. So selecting a drop down value would cause the query to change, which would automatically propagate changes to the table.
324 comments
[ 5.1 ms ] story [ 293 ms ] thread1. asking why visual programming sucks;
2. Talking about your experience with Alan Kay;
3. talking about your idea of an ideal visual programming method.
1. I have a theory on why VP sucks -- is it right? What do other people think?
3. Yes -- I'm looking for feedback, but in the context of why visual programming is no good. Also because of below:
2. I brought that up because visual programming is big, which is a good point that Alan Kay made. So I provided an example, or idea that I had. Is visual programming bad in this case? Why, or why not?
It's entirely possible to have complex conversations here where more than a single topic is being discussed at one time. This isn't Stack Exchange.
This is a good introduction and discussion about whether this is a fad or it's here to stay: https://medium.com/softwareimprovementgroup/low-code-wave-of...
[0] http://low-code.com
Anyway, i gave a link above, with good info, including names of leading companies. Here's one:
https://www.mendix.com/
Interface Builder was great for me at first, when learning iOS-specific controls and how everything worked, but as I've gotten better I've found it's always more of a hindrance. Things really start to break down when you need to move fully laid-out controls to a new container, for example, whereas Android's XML layouts let me do that by deleting/changing a few lines of text.
VP also hides away a lot of implementation details from you, and assumes the computer will be able to figure out something better than the programmer. For example, when would the dropdown run the SQL query? on page load? on drop down open? As you scale the app bigger, you may want to run a single SQL query and then chop up the results and hand them out to several dropdowns. If your VP interface doesn't support this then your done for, but with code there is always a way.
There is always Clojure of course, which lets you edit your GUI while the app is running, and upload code changes without having to rebuild, allowing you to tweak stuff as you go. REPL driven development is great as it gives you both the flexibility of written code, as well having close to the interactivity that VP offers.
You seem to be dancing around what I came to comment on, and that is event driven programming. It's OK for simple UIs but it tends to break down as complexity increases.
one of the main issues with labview is that people simply don't take it seriously as a programming language. if people watched how i work everyday, i work just the same as, if not more efficient than, a good text-based programmer.
Yup. It's not unlike the -insert whatever language here (C++ being a good choice)-haters ranting but not actually having used it a lot, or at all.
Moreover: depending on the type of UI, I don't think there is any language out there which can get me the results I need as fast, when I need something 'process control'-like, i.e. lot of display of e.g. electrical signals and a bunch of control buttons/textboxes/comboboxes. Then there is just no match for Labview. And I have quite some experience with writing UIs in WinForms/WPF/Qt... (typical example: good luck writing a pannable, zoomable, cursor-able live multisignal 2D graph in any of those - with labview you just feed data into the premade component and you're done) It even goes so far that even though the backend is in C++ or Python, I might take the overhead of writing IPC between backend and Labview frontend over doing the UI in the backend language.
view-source:https//www.google.com/
If you are a good enough programmer you want the power of ex regular expressions and you can solve almost any problem out there.
If you aren't you aren't much better off as you still need to learn how to program.
As a scripting language however it can be great because you can be much more opinionated about what should be prioritized.
What would be much more interesting and useful IMO was if normal languages created a visual environment akin to the visual programming languages we see sometimes.
- Visual programming tools usually only work at one abstraction level (your first point)
- Often centered around a gimmick and cannot encode "normal" detail-oriented imperative code efficiently
- Only a fraction of a given program is amenable to flow charts, Visual Basic-style forms, etc. Unfortunately, most visual programming environments are all-encompassing.
- So VPs tend to become blunt, awkward tools that lack power of expression, ergonomic editors, scalability to real usecases, etc.
FWIW, I do believe that a better code editor is possible through non-plaintext programming. (Serialize token trees and ASTs, instead of plaintext.) But such an editor would likely be used in text-editor-mode >50% of the time, just due to the high information content and sheer readability of text.
Funnily enough, this is itself another of those mirages, like visual programming, that have been chased for years and failed to gain traction (outside specialised applications).
For my money, the reason for this is that a human editing code needs to write something invalid - on your way from Valid Program A to Valid Program B, you will temporarily write Invalid Jumble Of Bytes X. If your editor tries to prevent you writing invalid jumbles of bytes, you will be fighting it constantly.
The only languages with widely-used AST-based editing is the Lisp family (with paredit). They get away with this because:
1. Lisp 'syntax' is so low-level that it doesn't constrain your (invalid) intermediate states much. (ie you can still write a (let) or (cond) with the wrong number of arguments while you're thinking).
2. Paredit modes always have an "escape hatch" for editing text directly (eg you can usually highlight and delete an unbalanced parenthesis). You don't need it often (see #1) - but when you need it, you really need it.
Bags of characters <-> Unstructured trees of tokens <-> ASTs
(BTW paredit is super cool and I'd like to see more of its kind!)
This is a well-trodden road. It starts with "wouldn't it be awesome if we could manipulate everything as ASTs", then usability intervenes and we fall back to "well, we need to be able to selectively edit as text", which means you need to be able to convert everything to bags of characters and back. And now you've built that conversion, you might as well represent the "source of truth" as bags of characters like everyone else does.
(And heaven forbid you should want to make a checkpoint commit that doesn't parse...)
- Using Merkle trees, we can assign every node a hash-based ID. So now we can refer to other nodes by ID. This lets us store a graph whose vertices are all the tree nodes.
- With the graph, we can now reference bindings not by string literal, but by ID. This eliminates shadowing problems and missing imports.
- There is now one source of truth for the names of variables and functions. As a result, in source control, a commit that renames something is a one-line change.
In other words, VP itself is a red herring. The leap that needs to first to be made is that from text to data.
*https://www.emaze.com/@AWOCZQLL/Text-is-for-Novels
In reality, programming has been stuck in the text approach for many years. Perhaps some people take that as evidence that there is no more progress to be made. Another factor that may be undermining research in programming, is the AI boom. Perhaps some think that AI will displace programmers in a not too distant time.
I agree with you that programs are data. And would benefit from a more specialized representation. I have been researching the aspect of program logic. For example, one thing that I think is terrible in programming, is deep nesting levels. That makes it complex to follow programs. I have been exploring ways to improve that. In my opinion graphical visualizations can provide ways to improve that.
I believe the solution lies in coming up with a way to compose pre-built components into complete applications...I think the existing efforts involving Angular,React,VueJS,Vaadin..etc are all pointing in the right direction....I have been working on Solvent (www.crudzilla.com) for sometime now and solving the UI building (drag-drop) is something that I am still actively thinking about...one of these days someone will crack the problem :)
True. That's why a proper visual programming environment should build on existing text-based environments and offer the same capabilities while adding power. Not disregard all progress, conventions, and achievements and start form scratch offering some unique features, but at the same time taking away stuff that's already been figured out.
Most of us don't do that. Most of us make due with off the rack suits with some tailoring for 1/10th the price.
With programming we have an expectation of bespoke design, so anything less, even if it's much easier, seems lame.
A potentially useful reframing of the question is, "What are all the common use cases that could be solved sufficiently with significantly less effort using visual programming?"
the big examples i have in mind are: * spreadsheets -- very visual, everything's in a grid. the relationships are spatial. * electronic music -- people lay out their various effects in a flowchart format and chain them from one to another * video games -- consider a game like RimWorld. you're clicking on all kinds of things and specifying what you want done with them. the behavior of the actors in the environment are modified by your specifications.
So, it's all in various optimized subdomains. As it should be!
Most of the things you do are achieved through a visual programming environment specialised for that particular task. Maybe text-based source files in complex directory trees, managed through a structured text editor with UI composition helpers represents one kind of specialised visual programming environment as well.
[0] https://www.theverge.com/2012/5/21/3032765/minecraft-emulate...
Visual programming efforts have tried to visualize code and operate at the wrong abstraction. Programming is just the act of creating programs. In a visual programming environment, the author / programmer should use visual interactions to "write the program". This means a few things needs to be considered:
- the representation - how is a computer program represented? how is state represented? - direct manipulation - imagine using a tool like Photoshop having to compile / build to see your changes between every edit. - input devices - mouse / keyboard works for writing lines of code. Touchscreen is a better input device / interface for visual programming but still lacking (no tactile response).
In my opinion, not enough is reinvented when visual programming efforts are tried. I think the entire stack (human input --> program state) needs to be reinvented and reconsidered. Bret Victor has a great section on visual program in his FAQ for one of his talks - http://worrydream.com/DrawingDynamicVisualizationsTalkAddend...
xoL is a graphic based programming language. It represents programming concepts with graphics instead of text. It is product of a long, painstaking design effort, to get an optimal graphical presentation of programs. A good description of it is available in this blog post: http://lignixz.github.io/blog/posts/17/xoL_graphics_based_pr...
A partially working prototype from a previous version is available online. The newer current design has fundamental improvements over that previous prototype. The way some program elements work was redesigned. The control method was also redesigned. It is now aimed primarily at touch based controls. Here is the prototype for the previous version: https://github.com/lignixz/xra9 . The prototype is not optimized for performance so excuse that. It is functional to some extent, if you can figure how to use it. You can add and modify programming elements.
Would welcome any opinions/feedback. Also interested in finding partners/investors/sponsors that may be interested in this project.
I thought about using something like a prototype called Hydra (1) as a sort of runtime for evaluating graphs. Anyway, I really want to get back to it at some point but other cool things that are slightly more low hanging tends to get in the way. :)
Another nice inspiration for these things is Ecolanguage (2), not really programming related but a diagram language for visualizing economic transactions.
(1) - https://github.com/giorgidze/Hydra (2) - https://www.youtube.com/watch?v=-QI1iuAvTKE
Let me start with a quote from Brian Kernighan "Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?"
When you make a graphic based programming language the debugging better be twice as good as the programming and this includes when things get really difficult with lots of threads, or when the over arching abstraction breaks at some low level and you are exposed to the underbelly of the CPU. At least C++ lets you gracefully traverse from the very high to the very low which is why it is favored in the end, you are never stuck you can always go lower.
Also, for anyone interested it looks like he just posted on HN for the first time under user 'worrydream'.
Watch his 2 talks that highlight the thinking and inspiration behind the RealTalk project:
https://www.youtube.com/watch?v=klTjiXjqHrQ https://www.youtube.com/watch?v=agOdP2Bmieg
1. It's hard to make a scheme in editor without spaghettifying it. (Partially mitigated by blocks incapsulating parts of scheme in themself, some discipline is required)
2. Usually VP makes it hard to work with complex data structures. The only solution is to basically write the same code as text losing all VP in the process.
http://hiasm.com/ : Very usable constructor for small programs, the only problem with it is that website is in russian.
There are 3 areas that I am aware of that VPLs have appeared a) modular audio synthesis b) 3d graphics c) game programming
(a) has been the first with origins back to 1995 , there are like a ton of programming apis for building modular synthesizers , there even programming languages that specialize in this that have existed for decades now like CSOUND and Supercolider but Reaktor and Max/MSP have literally mopolised the market. Reaktor itself has been such a massive success that made Native Instrument as the dominant company on audio software.
(b) 3d graphics is no secret that is probably the richest software industry with the generation of the well knows CGI for films and games its a goldmine for developers and has been dominated by VPLs. You are not aware of this most likely because we never refer to them as VPL but rather as procedural material, textures, animation etc. Not using a VPL for todays standards is considered insane even if you try to do it via a very lovable language like Python. Text languages like POV RAY that specialize in 3d graphics have died a miserable death. 3d is where TPLs go to die and are used strictly for generating the software alone and thats about it.
(c) computer games there VPLs are something new but Unreal has definitely changed that with Blueprints. Probably the best and most powerful VPL. If you have played any kind of AAA game chances are that the game you played uses Unreal which in turn makes heavy usage of Blueprints and C++ code. The rule of thumb among Unreal developers is "Do it with Blueprint and if you cannot avoid it do it with C++". Unity has smelled the roses and has implement its own VPL as well very similar to Blueprints.
So if VPLs are so amazing why we do not see more of them ? They are actually very hard to make. If an implementation of lisp , a basic one, needs a few hundreds lines of code a VPL needs thousands and that imposes a high barrier for language designers. Hence why we have thousands of TPL and only a handful of VPLs.
The majority of them are crappy obviously because that is how quality works , you have to have high quantity for high quality.
But a high quality VPL will kick high quality TPL ass any day and this is why VPLs are massively more popular than TPLs for the same task.
One that does not work as well as it looks and it depends on drag and drop is a crappy VPL . Yes a crappy VPL will suck, not as much as a crappy TPL (see C++ , Javascript etc) but it will suck none the less.
you can also read my quora answer here for more details why VPLs dominate so easily and why they will replace TPLs in the not so distant future to a very large extend
https://www.quora.com/Why-did-graphical-programming-language...
Contrary to what you may believe a VPL is not the evolution of TPL , its actually the evolution of GUI. Reaktor for example is an evolution of VSTs , synthesiser and audio effect plugins that are GUIs. VPLs in 3d are an evolution of the gui properties for material , textures etc (they are not even referred to as VPL). Even in Unreal even though Blueprint has replaced Unrealscript , its an extension of the editor GUI.
VPLs are successful specifically because the information density is so large that a linear GUI is not enough.
The success of TPL is legacy, its just has a lot of libraries. Mainly because its easier to make and has been around a lot longer.
However to be sincere here when we talk about TPLs we also cherrypick , we do not think of Assembly, Pascal, Cobol and Fortran but rather C/C++ , Java, C#, ruby, Python and Javascript. The truth is that 99.9% of TPLs have failed to gain approval of not just the users but also the coders too.
But I am not denying the fact that a TPL will target mostly pro coders than users and that VPL will target mostly users than coders. They are just different kinds of beasts making the comparison more than challenging.
I don't agree with parent but they are contributing to the conversation.
If you disagree why don't you provide some counter-points instead.
Here, let me start.
> Every time a VPL (Visual Programming Language) has gone against a TPL (text programming language) , VPL not only has come out a winner it has massacred its opponent.
That's an unsubstantiated claim. Point to some sources please.
> There are 3 areas that I am aware of that VPLs have appeared a) modular audio synthesis b) 3d graphics c) game programming
These are not the only ones [1], and VPL first appeared earlier than 1995. For example, the first version of LabView was released in 1986 [2] and certainly LabView was not the first one either.
> 3d is where TPLs go to die and are used strictly for generating the software alone and thats about it.
Even if this is true in the case of VFX (which I don't know that it is but I don't know that it's not either), it certainly is not true in video game development, which is very much an area of 3d graphics.
> computer games there VPLs are something new but Unreal has definitely changed that with Blueprints
Blueprints is awesome but you could not TODAY make an arbitrary game implemented using ONLY blueprints and have the amount of control and freedom you have if you use Unreal or another game engine with C++ or another TPL. Thus falls your initial claim, which (quoted above also) was
> Every time a VPL (Visual Programming Language) has gone against a TPL (text programming language) , VPL not only has come out a winner it has massacred its opponent.
And I bet that others can point to a wide range of areas where an existing VPL will come up short.
Some people love LabView. I certainly don't. Some prefer it and for them it's a thousand times better than using a TPL but for me 99% of the time I would go with a TPL over LabView.
[1]: https://en.wikipedia.org/wiki/Visual_programming_language
[2]: https://en.wikipedia.org/wiki/LabVIEW
"That's an unsubstantiated claim. Point to some sources please."
Reaktor and Max/MSP vs CSOUND and Supercollider , Blender python vs Material/Texture/Animation/Compositor Nodes, Blueprints vs Unrealscripts are all examples I mentioned already. Why they are not enough ?
"These are not the only ones [1], and VPL first appeared earlier than 1995. For example, the first version of LabView was released in 1986 [2] and certainly LabView was not the first one either."
This is why I said "that I am aware of" I am not an expert on all things just a few areas.
"Even if this is true in the case of VFX (which I don't know that it is but I don't know that it's not either), it certainly is not true in video game development, which is very much an area of 3d graphics."
I already mentioned Unreal and Blueprints. But yes what I meant was on the fact of actually a VPL competing with a TPL. Obviously when no VPL exists you have no competition. In the case of 3d apps efforts have been placed into using scripting languages but the users have picked VPLs instead as much easier to use and learn.
"Blueprints is awesome but you could not TODAY make an arbitrary game implemented using ONLY blueprints and have the amount of control and freedom you have if you use Unreal or another game engine with C++ or another TPL. Thus falls your initial claim, which (quoted above also) was"
What you say its true but its not because VPL is inferior its because its not fully supported. Unreal started as a C++ game engine so of course the support will be more extensive than for Blueprints . Blueprint have been around for few years and its already growing faster than the C++ api. It was only lately that Epic added the ability to compile a Blueprint to C++ code. C++ API also make it easy to wrap any C++ code and expose it to Blueprints. So if you want to make your entire game in Blueprint its not possible but also easy.
"Some people love LabView. I certainly don't. Some prefer it and for them it's a thousand times better than using a TPL but for me 99% of the time I would go with a TPL over LabView."
VPLs are made for users not coders, its their ease of use / idiot proof mentality that have made them succesful. So yes if you are a coder in most cases you will prefer and should prefer a TPL , especially a popular one.
But again I repeat in the vast majority of case you will be extremely lucky to find a VPL to even consider moving to. VPLs are extremely rare.
If I go to the last page of a Google search for C++, it says that it showed me 180 results. 18 pages.
It used to be that Google would tell you an approximation of how many matches they actually had in total for a search of their index but they no longer do anywhere that I am aware of. If there were only two pages of results I might have agreed but when there are 12 pages it will show to me I don't think there is any useful conclusion to be made about the actual total amount of sites that contain these keywords.
However, there is another site that still gives us an approximate count of results. YouTube.
Searching for C++ on YouTube I am told that there are "about 1,010,000 results".
Searching for Unreal Blueprints on YouTube I am told that there are "about 57,800 results".
Searching for UDK Blueprints on YouTube gives "about 18,600 results".
There will be some amount of overlap between the latter two.
I don't know how many of those videos matched only blueprints and was included though.
But let's do another couple of searches.
UDK C++ on YouTube gives "about 11,000 results".
Unreal C++ on YouTube gives "about 60,900 results".
Well, it seems that the words UDK and Unreal are significant when searching on YouTube, since otherwise both of those searches should have given about a million results like the search for just C++ alone did.
So comparing "Unreal C++" and "Unreal Blueprints" on YouTube actually gives a surprisingly high amount of results about blueprints compared to C++ for Unreal, certainly a lot higher than I would have expected. I would have expected at least 25 times as many for C++ but it seems to be almost the same amount of videos for each of those.
Of course one might wonder if even though the words Unreal and UDK might have matched correctly, did the words blueprints and C++? Or do the results include a lot of videos that are about neither blueprints nor C++?
Additionally some amount of videos will have both these words but will not be about the Unreal Development Kit. Instead one video might be something like "Unreal! Secret blueprints confirming conspiracy theory so-and-so confirmed!" You get the idea.
YouTube's search is wonky, after a few pages it starts to omit words. I got a video for a blueprint of a building (no "unreal" mentioned anywhere).
I am about to "work" in the industry I have a Blender plugin and Unreal editor plugin in the works , none of the use a VPL by the way because simply what I am trying to do is too technical for VPLs that target users and not coders. But of course I will be offering VPL support because users love VPLs.
I never said Blueprints are the default in Unreal , C++ is still king. However if you want proof , you can take a visit at the official Youtube channel of Unreal by Epic and see for yourself the rise of Blueprints. Most tutorials nowdays for making games with Unreal target Blueprints.
I do agree that Google's not evidence of anything, I just use it as a first approximation when I have no better sources. That said, the marketing channel of a company trying to push its tech doesn't really satisfy me either.
Of course how much Blueprints will be used will depend on the game. There are games made entirely with Blueprints.
Also Epic recommends the use of Blueprints as first choice again because of ease of use. They have invested a great deal to it after all. It even has a debugger with break points , watching values and visualization of execution flow.
Audio VPLs are far more powerful than what you describe.
Reaktor I do not know if it supports 3d physics particles , but you could do what you ask for from blender. Blender supports midi for controlling animation.
Blueprints on the other hand are a generic VPL and can do anything you describe. It's also the best choice because Unreal excels at real time graphics.
The other thing to note is that hardware had influenced programming languages, which in turn have influenced hardware, in the direction of SISD on CPUs and later SIMD on GPUs. But textual languages are poorly suited, and visual languages well-suited, for representing dataflow. In the 1970s and 1980s, there was active research on dataflow hardware as well as work on (mostly textual) dataflow languages. Now, CPU speeds are no longer improving exponentially as they were in the past, so dataflow should be re-examined.
I've been working on my own visual dataflow language (http://web.onetel.com/~hibou/fmj/FMJ.html). I haven't updated the pages for some time, but big changes are under way.
For example what I did was to code a server part in golang and use vue.js on the frontend and for the app builder. Using the web-based app builder (also 100% vue) the user can design forms. The forms are stored as serialized JSON in the database and the server renders dynamically vue.js components. It's super fast. And there's a complete freedom to design any form with a 16-column responsive grids and a lot of widgets and styling options. Forms are also nestable (thanks vue!) and completely extendable with custom JS code so it's possible to create a custom widget and then simply import it in other parts of the app.
As I said I think the shadow DOM and frameworks like vue really made the difference. I started without using such frameworks and it was not only tedious, but the resulting app wasn't very interactive and felt like a customized Excel spreadsheet attached on top of a PostgreSQL database. Now it's a completely different story - it's easy to build fully reactive apps that feel like a "normal" cloud app and are actually pleasant to use. Of course instead of spending few months designing and building it, it was created in a few hours.
So yeah... I think that drag-n-drop builders are definitely coming and will probably eliminate the need to write such apps in coming years.
https://news.ycombinator.com/item?id=12092527
For anybody interested -- email is in profile. Happy to give you a sneak peek!
That's why I don't target primarily developers, but the end-users of such apps. Of course knowing Javascript wouldn't hurt, because it opens a lot of more possibilities to further extend the entire app, but my idea was to make it possible even for non-developers to create apps for CRM, invoicing, accounting or inventory management for their needs. So in my case it's not really visual programming, but a drag-n-drop builder, which also has an integrated code editor in case the user needs it: if they know SQL, awesome - write a custom SQL and it is automatically mapped and available in the forms.
This sweet spot that you talk about between ease of use & flexibility is definitely key.
I'm honestly more concerned with maintenance costs more than building costs. In my experience the cost of the alpha release goes down with visual editing, but the cost of subsequent releases ramps up over time.
People were saying that 10 years ago, they were saying it 20 years ago, the general attitude was even the inspiration for COBOL (that business people could write their own apps). Every single time it ends up being a disaster when it comes to slightly non-trivial use cases.
It's been tried and failed by companies large and small for decades. So what do you know or do differently that they didn't?
https://www.allaboutcircuits.com/worksheets/ac-motor-control... is the link I quickly found.
It's typical to see a handful of symbols handle what would take a hundred of lines of code for the level of user interface usability, validation, and core functionality.
For physical systems it is pretty true.
Alan Kay's mistake is that a lot of his original work was with Papert on educating children. For early learning, having a visual/physical analog adds value. For professional work, when you already have the concepts down, it just gets in the way.
Expressing the connections isn't always easy, but this is more of a problem of the underlying programming language, not of the visual elements.