Ask HN: Why isn’t visual programming a bigger thing?
Visual programming seems to unlock a ton of value. Difficult concepts can more easily be grokked in a visual form. Programming becomes more approachable to first-timers. Since text is difficult to manipulate without a physical keyboard, visual programming opens the doors to doing development on mobile devices. And yet it only seems to be mainstream in education (i.e. Scratch). Why?
303 comments
[ 3.3 ms ] story [ 304 ms ] threadI suspect visual programming is more common than we realize though. I had an acquaintance at Workday who claimed a lot of work was done there in a visual programming language.
Also, arguably website/app builders are a visual programming "language" and they are extremely common.
To my mind, this is analogous to textual writing itself vs drawing where text is an excellent way to represent dense, concise information.
I have written and maintained LabVIEW applications that exceed a 1,000 VIs. I would argue that such well-architected applications are actually easier to maintain for all the reasons that people expound functional languages for. The reason is that LabVIEW is a dataflow language, and so all the benefits of immutability apply. Most data in LabVIEW is immutable by default (to the developer/user). So the reasons why people prefer languages like F#, Elixir, Erlang, Clojure, Haskell, etc. overlap with visual languages like LabVIEW. I can adjust one portion of the program without worry of side effects because I'm only concerned with the data flowing in and then out of the function I am editing.
Somehow people form the opinion that once you start programming in a visual language that you're suddenly forced, by some unknown force, to start throwing everything into a single diagram without realizing that they separate their text-based programs into 10s, 100s, and even 1000s of files.
Poorly modularized and architected code is just that, no matter the paradigm. And yes, there are a lot of bad LabVIEW programs out there written by people new to the language or undisciplined in their craft, but the same holds true for stuff like Python or anything else that has a low barrier to entry.
That’s very insightful, and I think nails part of my bias against specifically LabVIEW, as well as other novice-friendly languages. My few experiences with LabVIEW were early on in my EE undergrad. At that point I had been programming for about a decade and had learned a ton about modularity, clean code, etc. The LabVIEW files were provided to us by our professors and, still being a naive undergrad, I assumed our profs would be giving us pristine examples of what these tools could do; instead, to my programmer brain, what we had was an unstoppable dumpster fire of unmanageable “visual code” that made very little sense and had no logical flow. Turns out it’s just because our profs might be subject matter experts on a particular topic, and that topic wasn’t “clean long-term LabVIEW code”. Later on I ran into similar problems with MATLAB code that our profs handed out, but by that time I had clued into my realization. At one point I was accused by my Digicom prof of hardcoding answers because there’s no way my assignment should be able to run as quickly as it did. (I had converted a bunch of triply-nested loops into matrix multiplications and let the vectorization engine calculate them in milliseconds instead of minutes)
Just like LabVIEW, my bias against PHP comes from the same place: it’s obviously possible to write nice clean PHP, but every PHP project I’ve had to work on in my career has been a dumpster fire that I’ve been asked to try to fix. (I fully admit that I haven’t tried to do a greenfield PHP project since about 2001 or so and I’m told the ecosystem has improved some...)
I lucked out with Python and started using it “in anger” in 2004, when it was still pretty niche and there were large bodies of excellent code to learn from, starting with the intro tutorials. Most of the e.g. PHP tutorials from that era were riddled with things like SQL injections, and even the official docs had comment sections on each page filled with bad solutions.
This is important because it shows that such things can exist. So the common complaint is more about people forgetting that text-based code review tools originally didn't exist and were built. It's just that the visual ones need to be built and/or improved. Perforce makes this easier than git in my opinion because it is more scriptable and has a nice API. Perforce is also built to handle binary files, which is also better than git's design which is built around the assumption that everything is text.
I think there's a lot of nice features to be had in visual programming languages with visual compares. Like I said in another comment, I view text-based programming as a sort of 1.5 dimensional problem, and I think that makes diff tools rather limited. If you change things in a certain way, many diff tools will just say you completely removed a block and then added a new one, when all you did was re-arrange some stuff, and there's actually a lot of shared, unchanged code between the before and after blocks. So it's not like text-based tools don't have issues.
We're just used to applying our software engineering skills to our textual languages, so we take it for granted, but there exists plenty of terribly written textual code that is just as bad or sometimes worse than the terribly written visual code. That's why sites like thedailywtf.com exist!
Why? Can you give concrete examples of this? I see this sentiment a lot but never any particular examples.
When you start off with a prototype in a text-based language for something "simple", you don't extend it by continually adding in more and more input arguments and more and more complex outputs. You breakout the functionality and organize it into modules, datatypes, and functions.
Same thing goes in a visual programming language like LabVIEW. When you start adding in more functionality to move beyond simple designs, you need to modularize. That reduces the wires and boxes and keeps things simple. In fact, I liken well-done LabVIEW to well-done functional-code like in Racket, Scheme, F#, SML, etc., where you keep things relatively granular and build things up. Each function takes in some input, does one thing and returns outputs. State in LabVIEW is managed by utilizing classes.
The issue is that when you get into complex logic and number crunching, it quickly becomes unwieldy. It is much easier to represent logic or mathematics in a flat textual format, especially if you are working in something like K. A single keystroke contains much more information than having to click around on options, create blocks, and connect the blocks. Even in a well-designed interface.
Tools have specific purposes and strengths. Use the right tool for the right job. Some kind of hybrid approach works in a lot of use cases. Sometimes visual scripting is great as an embedded DSL; and sometimes you just need all of the great benefits of high-bandwidth keyboard text entry.
Regarding your point about complex logic and number crunching can be alleviated by using a 'Math Expression' node... which allows math to be written in textual form in a node.
https://blueprintue.com/blueprint/jt69wz7e/
Compare that with this bit of pseudocode:
Which one is more readable for someone who's even a little bit experienced in programming? Which one is faster to create and edit?1) It took me longer to create blueprints than it would have taken to write the equivalent code. I kept finding myself thinking "I could do this in 5 seconds with a couple of lines of code"
2) The blueprints quickly became unwieldy. A tangle of boxes and wires that I couldn't decipher. I find code easier to read, as a rule.
3) I didn't find it any simpler than writing code. A Blueprint is basically a parse tree. It maps pretty much 1:1 to code; it's just a visual representation of code.
I'd argue that 90% of it is just that they work out of the box. You download the engine bundle and you can start doing blueprints. Being able to write normal code requires you to compile UE from source, which is a non-trivial thing for any large C++ project.
I'm pretty sure if they embedded Lua or Lisp directly into the engine, and provided a half-decent editing environment in-editor, that it would meet with success just as well.
Blueprints are great for the material editor where everything is going to terminate in a specially designed exit node and there is little branching logic, but even for basic level scripting it is more messy than it used to be in the industry.
In the 1970s and 1980s I did many projects with huge schematics that you would spread out on large tables and focus on details while seeing the whole thing at the same time. It is an experience that can't be captured at all by a series of small pages, each with one to four boxes and lots of dangling labeled wires going to the other sheets. At that point you might as well just have a textual netlist, which made structural VHDL/Verilog become popular replacements for schematics.
Perhaps VR/AR will bring back visual hardware design?
And then I set out to replace them with something "better". And during the process of replacing, I came to understand and form a mental map of the original system in my mind. And at some point I realized that the original system was actually workable. And it was easier to just use the old system now that I understood it enough.
Beyond that, for real engineering? I've worked with EEs who write massive applications in Labview -- their codebases are all impossible to maintain masses of pain and suffering.
Fixed that for you :). EE/CS dual here. Some EEs can code, some CS folks can design circuits, but if I had to bet... I wouldn’t expect great work to come from either discipline working on the other side of the software/hardware line. I’m decent at both, but I pretty much squeezed two degrees into a five year programme and have put a lot of effort into maintaining competency in both disciplines throughout my career. These days I’m mostly doing embedded and am loving life :D
Though I have seen some very nice C/C++ from some of them -- those that want to write good code.
And as a CS guy, I know damn well I can't do circuit design to save my life.
I've been trying to implement something with Power Automate, and presumably that's "mainstream", but it strikes me as falling into the classic pattern of appealing to buyers rather than users. I feel 10-100 times less productive than with, say, VBA, for no advantage.
One thing that is particularly frustrating to me is that it's so slow and buggy I am afraid of losing my work at any moment. You can't save your work unless it passes preliminary validation, but sometimes reopening it makes it invalid by removing quotes or whatever. Copying something out and pasting it back often fails to validate too, as the transformations are not inverses like they should be. Sometimes it just gets corrupted entirely. I'm not aware of any way to manage versions, or undo beyond a few trivial actions.
But the more fundamental reason I hate this is because it seems not to be designed to let you take a chunk of logic and use it in a modular way. At least this style of "visual programming" seems to apply the disadvantages of physically building things out of blocks, where it's entirely unnecessary. You've got some chain of actions A->B->C, but the stuff inside those actions is on a different level; you can't take that chunk of stuff and use it as a tool to do more sophisticated things. As far as I can tell. I keep thinking "it can't be as simplistic as it seems" and thinking I'm about to find a way to create general functions.
See: https://flow.microsoft.com/en-us/
Today I was trying to figure out if I could work around some of my problems by converting everything to XML and using XPath to manipulate it but I didn't get far and apparently Microsoft only does XPath 1.0.
No syntax or type bugs, just logical or more like physical bugs. Because you are modelling physics, and sometimes the model is just not good enough. Still vastly better than traditional C++ models.
Problems: No diff tool. You can hardly see what changed. That is like shipping updated lisp images or binaries without source code to the devs. You also get a lot of windows, like 30 for a typical small model.
I can't imagine being able to write maintainable, well tested, scalable software (cough, software engineering, cough) with some version of drag and drop. I'd love a visual element added for helping navigate code. I like system diagrams, flowcharts, etc. But I'd like these to be generated by my code, not generate my code. I feel like this would be trying to write a book with only emoji and/or gifs.
That said - working with data is an area that lends itself well to visual programming. Data pipelines don’t have branching control flow and So you’ll see some really successful companies in this space.
Alteryx has a $8b market cap. Excel is visual programming as well.
Either of these is a consistent choice in itself, but they don't play well together as part of the same diagram unless rigorously separated.
A different point is that some high-level abstractions do have a fairly natural visual representation. This even includes the "functors" and "monads" that are well-known in FP. In general, one should not be surprised if category-theoretical abstractions turn out to be representable in this way. (Many visual languages actually turn this into a source of confusion, by conflating operations on an atomic piece of data with operations on a stream or sequence of singular data points. A stream is a kind of functor, which is why it (with its peculiar operations) is endowed with a simple representation in dataflow. But there are other functors of interest.)
One type of control structure I have not seen a good implementation of is pattern matching. But that doesn't mean it can't exist, and it's also something most text-based languages don't do anyway.
Not quite. You'd need to draw two parallel boxes, each of which is strictly single-entry/single-exit, and draw a while loop in each box. This is because a while loop
depicts parallel flows that do not represent stuff being done concurrently! Once you acknowledge that, pattern matching actually becomes easy: just start with a "control flow" pattern and include a conditional choice node with multiple flowlines going out of it, one for each choice in the match statement. You're drawing control flow so it's easy to see that the multiple flowlines represent dispatch, not something happening in parallel.https://i.imgur.com/AgmF87b.png
Now, the two while loops, as shown here, have no dependencies between each other and are indeed processing in parallel. However, there are various mechanisms in LabVIEW to exchange data between the two loops, the most common being queues, in which case they process concurrently.
You can also have a for loop iterating on an array.
https://i.imgur.com/nRgyckx.png
In LabVIEW, it's nice because it's trivial to configure the loop iterations to occur in parallel (if there are no dependencies between iterations), using all available cores in the computer.
And by pattern matching I meant something like the pattern matching and type destructuring you find in SML, Ocaml, F#, and Elixir.
Using labVIEW over C did have some benefits. It seemed like streamlined concurrency is a major advantage.
You don't have to have all your code inside a single diagram. Any professional LabVIEW programmer has a rule that a single VI (basically function or method) should only require a single modestly sized monitor to view it, aside from a few exceptions. This is akin to text-based languages having a sweet spot of 500-1,000 lines of code per file and keeping a function within a single screen without needing to scroll. Anything above starts to become unwieldy.
The size of a LabVIEW diagram isn't a limitation of the medium just like the size of a text-based programming language's file isn't a limitation of the medium. It all boils down to the programmer needing to modularize appropriately.
Simulink was much preferred.
You need to architect your application using classes and functions just like in any other language with whatever appropriate data abstractions present.
It's useful to allow more "citizen devs" (regular folk with little exposure) to come up with prototype high level proof of concept apps,including UX design. It is a big deal in the corporate arenas I've had exposure too,but I think widespread adoption is still years away.
You will always need non-visual languages to do things in a featureful and scalable way.
> You will always need non-visual languages to do things in a featureful and scalable way.
What is an example of a system that you have developed where things broke down with a visual programming language?
For example, in LabVIEW, you have TCP/IP, HTTP, UDP, CAN, serial, MODBUS, and more protocols and can build things out of them. If there's a missing protocol, then you can write your own, call a C/C++ DLL, .NET assembly, Python script, or a command line application, just like any other language (actually more than most languages).
It can convert Visual to/from TypeScript/ JavaScript. And it works on mobile.
It's for games, but you'd think the technology would be applicable for any kind of program.
It is in fact useful for these things. For example, LabVIEW is wonderful for asynchronous code, is automatically multi-threaded, and supports the actor model. Dataflow languages are excellent for functional programming, and this area is relatively unexplored.
The advantages you advocate come from the semantics, not the form.
These properties come from the constructed data flow graph, which can be analyzed by an execution environment and potentially prepared by a compiler. They don't come from visual vs symbolic representation. They come from modelling a computation as data flow.
And yes, I have implicitly bundled the semantics with the syntax here. I suppose in doing so, I am arguing both that dataflow is a powerful and interesting way of modeling computations that allow for a lot of neat features and that visual representations are more natural and give additional useful benefits over text-based ones. I have not seen a compelling text-based dataflow representation, but that doesn't mean one doesn't or can't exist. LabVIEW actually compiles its block diagrams to what's known as DFIR, dataflow intermediate representation, which is then compiled to LLVM IR. The DFIR is helpfully visualized as a graph, although the dataflow graph is represented by text underneath.
I first learned to program FPGAs with LabVIEW. Now I am learning HDLs, and it's a bit painful. Understanding VHDL is not hard, but I find it to be a poor representation of what it is describing. The text-based representation hides the nature of the actual computation it's representing.
If you have any references to dataflow or any links to things you've done, I'd love to take a look. :)
I wrote a server-side framework for data entry AJAX apps in 2004-2006 time frame, canonical application was entering details for an insurance quote. The UI was data bound on the back end, and a minimal update over the wire could be calculated from the bindings.
The bindings were stored in an XML document pulled from a metadata server referenced by id from the session state, meaning that we could do live rollover of new versions. Previous pages with existing sessions would use the old logic, whereas new logins would use the new logic.
This gave rise to a number of nifty properties. The session state was serialized - in fact, it was never deserialized, it was a byte array that was traversed as various attributes were accessed and updated. If there was a bug, you could rehydrate the session state, and display all session variables etc. and evaluate the current value of all data bindings, meaning that you could figure out what the user was seeing on screen. A lot like Smalltalk or Lisp images, but much smaller, because the code was static and stored externally.
Everything from disable state on controls to tab order around the screen was controlled via data binding, dynamically updating in response to values entered. The UI logic was completely declarative.
The framework was called Topoix, there are fragmentary references to it on the interwebs. I wrote an HTTP server & debugger for it too.
The data binding language was called Gravity, structurally it was a subset of C# in .NET 2 era, but it was declarative because it was not merely executed, but analyzed for data flow purposes.
The experience of creating a language amongst other things lead me to Borland, where I worked on the Delphi compiler, and partook in initial design of the new version of the data binding for controls feature. I can't claim any credit or otherwise for it, I was focused on other things, like rich reflection, which in itself helps data binding work well with a native language.
More recently, amongst many, many other things, I designed the web side data binding approach for the company I'm currently at, Duco. We're gradually replacing it all with React now, but it was useful to make pages reactive and automatically updating in response to new data at a time when people were doing jQuery from their event handlers.
An interesting academic paper at the time for me was "Functional Programming with Bananas, Lenses, Envelopes and Barbed Wire" by Meijer et al. He's a decent chap to follow if you're interested in that seam: https://www.researchgate.net/profile/Erik_Meijer
I picked up one of my favourite jargon terms from his usage, "bijective", e.g. with respect to lenses. In a data flow context this is reversible computation, a nice thing to have if you're doing data binding against an editable control (rather than something like a label). Not only does the value you're binding to change in response to upstream changes in the model, but edits to the value can be propagated back to the model. Bijective is the technical term for a function which is mappable both ways, and lens is the CS term for the abstraction which encapsulates a data flow transforms required for a scenario like data binding.
I used it to easily connect to a piece of lab equipment, reset it, set whatever settings I want, run a test, and then log the output to a file. I could setup a test then walk away and return to data. Doing the tests manually would take many months.
Both have labels as remarks/comments and you can easily put in a switch statement to test new code or use highlighting to see exactly where the program is running albeit slowly.
One of the fun things to do was circle a repetitive task then turn it into a function. A large program requires a large screen to see it all. Widescreens are terrible for it.
After basic settings and availability in libraries, it is better to move to a text language. Visual programming is a quick and dirty solution.
My only experience with Labview was in another job writing a DLL that it could import so the poor sucker working on the code could do a bunch of complex state machine stuff without having to drag "wires" all over the place in Labview. That ended up with a design pattern that was "route inputs to DLL function, route output to next stage" that turned out to be much easier to maintain. Partly because it enforced modularisation, and partly just because a series of if statements and function calls is easier to read than diagram.
https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z000...
Also, Lego have a whole series of graphical languages for their electronics, and those are very cool but extremely limited. Once you get past "when this switch is pressed make this motor go" it is easier to hit the textual language.
It was semi common to see engineers using the “big boy” tool, Labview to play with some Lego mindstorms actuators etc
http://vis.cs.brown.edu/docs/pdf/Upson-1989-AVS.pdf
The idea spawned many imitators (VTK, IBM DX, SGI Iris Explorer). The product was spun out of Stellar shortly afterwards, and the company is still in existence:
https://www.avs.com/avs-express/
Visual scripting is growing but it’s better for some things than others
Visual programming works very well for data flow problems.
LabVIEW has a diff capability, and while working for National Instruments, my team actually had a quite capable custom code-review system built out of this diff. This is an area that is brought up often, but text-based diff tools weren't magically found in the universe. They were built, and some of them are good and some of them not. It's not a paradigm's fault that tools like git were built around the idea that code must be text.
I do agree that better tools need to exist though, but there isn't a reason for why they can't. They just need to be built. There are hard and interesting problems in that space, both technically and design-wise.
> Some visual languages can turn an area into a 'subroutine', I have not seen any solution to build libraries of reusable 'subroutines'.
LabVIEW has a third-party developed package manager in the JKI VIPM. And LabVIEW has features where you can put your VIs and classes into various LabVIEW specific containers, most usually source libraries, that can then be referenced and re-used in projects. Just treat the libraries as modules like you would in any other language. They should contain classes and functions that have a shared or particular purpose.
FWIW you can have custom diff and merge drivers in git. One could probably hook up LabVIEW's diff tool as a git diff driver, I don't know about merge.
Merge is harder in LabVIEW and isn't where it should be. However, that doesn't mean it can't exist.
There are some really amazing tutorials and examples here: https://youtu.be/wubew8E4rZg
[1] https://derivative.ca
[2] https://derivative.ca/community-post/made-love-touchdesigner...
[3] https://derivative.ca/community-post/making-go-robots-intera...
TouchDesigner really showcases the enabling nature of visual programming languages. You can see your program working and inspect and modify it while it is working. These are very powerful ideas, and visual programming languages are much better platforms for ideas like this.
People, i.e. traditional programmers, are really hard and down on visual programming languages. Meanwhile, people who use LabVIEW, TouchDesigner, vvvv, Pure Data, Max, and Grasshopper for Rhino are all extremely effective and move quickly. People who are experts in these environments cannot be kept up with people using text-based environments when doing the same application.
Text-based programming is limited in dimensionality. This can become very constraining.
In contrast, I've often thought that visual programming tools are much more limited dimensionally, and that's why they can become difficult to manage beyond a low level of complexity: you only have two dimensions to work with.
With the visual programming tools, the connections between components need very careful management to prevent them becoming a tangled and overlapping web. In a 2D tool (e.g. LabVIEW), you could make a lot of layouts simpler by introducing a third dimension and lifting some components higher or lower to detangle connections - but then you'd face similar hard restrictions in 3D.
Text based programs suffer from no such restrictions; the conceptual space your abstractions can make use of is effectively unlimited, and you can manage connections and information flow between pieces of code to maximize readability and simplicity, rather than artificially minimizing the number of dimensions.
How does this not apply to a visual language like LabVIEW? Just because you draw the code on a 2D surface doesn't prevent abstraction and arbitrary programs. The way I program LabVIEW and the way it executes is actually very similar to Elixir/Erlang and OTP. Asynchronous execution and dataflow are core to visual languages. You are not "bound" by wires.
When you write text-based code, you are also restricted to 2 dimensions, but it's really more like 1.5 because there is a heavy directionality bias that's like a waterfall, down and across. I cannot copy pictures or diagrams into a text document. I cannot draw arrows between comments to the relevant code; I have to embed the comment within the code because of this dimensionality/directionality constraint. I cannot "touch" a variable (wire) while the program is running to inspect its value. In LabVIEW, not only do I have a 2D surface for drawing my program, I also get another 2D surface to create user interfaces for any function if I need. In text-languages, you only have colors and syntax to distinguish datatypes. In LabVIEW, you also have shape. These are all additional dimensions of information.
> When you write text-based code, you are also restricted to 2 dimensions, but it's really more like 1.5 because there is a heavy directionality bias that's like a waterfall, down and across.
These are two really good points. Text-based code is just a constrained version of a visual programming environment. So far most attempts at VP have attempted to represent code in terms of nodes and lines (trees), but that does not necessarily need to be the case.
The interesting about VP is that is presents a way to better map the concept and structure of programming as an abstract concept to how we physically interface with our coding environments.
It's far from likely that character and line-based editing is the mode of the future. Line-editing maps to the reality of programming in, to my eyes, such a limited way that is seems the potential for new interfaces, and modes of representation is wide open.
It's not that, as some people stubbornly say, there's no better alternative to text-based programming, but I think we just haven't conceived of a better way yet. We're biased to think in a certain way because most of us have programmed in a certain way almost solely by text and most of our tools are built to work with text. But that doesn't necessarily mean that the way we've done things is the best way indefinitely.
There's so much unexplored territory. VR opens up new frontiers. What if the concepts of files, lines, workspaces were to map to something else more elemental to programming as an abstract concept. What if we didn't think so much in terms of spacial and physical delineations, and instead something else? Blind programmers have a different idea of what an editor is. Spreadsheet programs "think" in terms of cells in relation to one another. What about different forms of input? Dark Souls can be beaten on a pair of bongos. Smash Bros players mod their controllers because their default mode of input isn't good enough at a high level. Aural and haptic interfaces are unexplored. Guitars and pianos are different "interfaces" to music. Sheet music is not a pure representation of music.
I think there's the mistaken belief that text == code, that text is the most essential form of code. Lines, characters are not the essential form of code. As soon as we've assigned something a variable name, we've already altered our code into a form to assist our cognition. Same with newlines, comments, filenames, the names of statements and keywords. When we program in terms of text, we're already transforming our interpretation of code and programming; we've already chosen and contrained ourselves to a particular palette.
What is the most essential form of code are (depends on the language, but generally) data structures and data flow. So far, our best interpretation of this is in the form of text, lines, characters, inputted by keyboard onto a flat screen - but this is still just one category of interpretation.
All this is to say is that text is not necessarily the one and only way, and it's too soon to say that it's the best way.
The way I see it is that we've had an evolution of how to program computers. It's been:
circuits -> holes in cards -> text -> <plateau with mild "visual" improvements to IDEs> -> __the future__
I think many programmers are just unable to see the forest through the trees and weeds, but visual languages see a lot of power in specific domains like art, architecture, circuit and FPGA design, modeling tools, and control systems. I think this says something and also overlaps with what the Racket folks call Language Oriented Programming, which says that programming languages should adapt to the domain they are meant to solve problems in. Now all these visual languages are separate things, but they are a portion of the argument in that domain-specific problems require domain-specific solutions.
So what I believe we'll have in the future are hybrid approaches, of which LabVIEW is one flavor. Take what we all see at the end of whiteboard sessions. We see diagrams composed of text and icons that represent a broad swath of conceptual meaning. There is no reason why we can't work in the same way with programming languages and computer. We need more layers of abstraction in our environments, but it will take work and people looking to break out of the text=code thing. Many see text as the final form, but like I said above, I see it as part of the evolution of abstraction and tools. Text editors and IDEs are not gifted by the universe and are not inherent to programming; they were built.
This has already happened before with things like machine code and assembly. These languages do not offer the programmer enough tools to think more like a human, so the human must cater to the languages and deal with lower-level thought. I view most text-based programming languages similarly. There's just too many details I have to worry about that are not applicable to my problem and don't allow me to solve the problem in the way that I want. Languages that do provide this (like Elixir, F#, and Racket) are a joy to use, but they begin to push you to a visual paradigm. Look at Elixir, most of the time the first thing you see in an Elixir design is the process and supervision tree. And people rave about the pipe operator in these languages. Meanwhile, in LabVIEW, I have pipes, plural, all going on at the same time. It was kind of funny as I moved into text-based languages (I started in LabVIEW) to see the pipe operator introduced as a new, cool thing.
In general, I have many, many complaints about LabVIEW, but they do not overlap with the complaints of people unfamiliar with visual programming languages, because I've actually built large systems with it. Many times, when I go to other languages, especially something like Python, I feel I've gone back in time to something more primitive.
Text-based representations have tremendous upsides (granularity, easy to input, work with existing tools, easy to parse), but they also have downsides I think people tend to overlook. For example, reading and understanding code, especially foreign code, is quite difficult and involved; involves a lot of concentration, back and forth with API documentation, searching for and following external library calls ad nauseum. Comments help, but only so much. Code is just difficult to read and is expensive in terms of time and attention.
> Text editors and IDEs are not gifted by the universe and are not inherent to programming; they were built.
Bret Victor has some good presentations that addresses this idea. One thing he says is that in the early stages of personal computing, multitudes of ideas flowered that may seem strange to us today. A lot of that was because we were still exploring what computing actually was.
I don't dislike programming in vim/emacs/ides. Is it good enough? Yes, but... is this the final form? I think it'll take a creative mind to create a general-purpose representation to supersede text-based representations. I'm excited. I don't really know of anyone working on this, but I also can't see it not happening.
LabVIEW has subVIs which are effectively no different from any subroutine or method in another language. LabVIEW has dynamic dispatch so it can run code with heterogeneous ancestry. You can launch code asynchronously in the background, which isn't even necessary to accomplish multi-threaded execution in LabVIEW (though there are plenty of other gotchas for those used to manual control of threading along with a couple of sticky single threaded processes that might get into your way when trying to high level reusable code). You can even implement by-reference architectures adding yet another way to break out of the 2D-ness of its diagrams. Perhaps a new development for most here will be that LabVIEW is now completely free for personal use (non-commercial & non-academic). Still, like some have pointed out, LabVIEW really shines with its hardware integration. It's the Apple of the physical I/O world. The only reason I avoid it for anything larger than small projects is it needing it's not tiny run-time engine which isn't any different from .Net distributables just more.... niche?
With text you get top to bottom lines of text (a single dimension) and any additional dimensionality has to be conceptualized completely in your mind... Or in design tools like UML... which display relations in a 2D manner. SQL design tools these days provide 2D visualizations in a graph-like manner to relate all the linkages between tables. User stories, process flow, and state diagrams are (or at least should be) mapped out in 2D in a design document before putting down code. How does the execution order of functions and the usage of variables provide any more freedom?
All I want to establish is that LabVIEW is another tool in the toolbox. People used to text are used to SEEING a single dimension and thinking about all the others in their head or outside the language. LabVIEW places two dimensions in front of you which changes how you can/have to think about the other dimensions of the software. With skilled architecture of a LabVIEW program the application will already resemble a UML, flow, or state chart. I do agree that some stuff that feels much simpler in text languages such as calculations are much more of a bear in LabVIEW; tasks that are inherently single dimensional in their text expression suddenly fan out into something more resembling the cmos gate logic traffic light circuit I made at uni.
I do embedded uC development with C/C++, I do industrial control systems and automated test in LabVIEW, and I even subject myself to the iron maiden of kludging together hundreds of libraries known as configuration file editing with a smattering of glue logic AKA modern web development (only partially sarcastic, if I never have to look at a webpack config file again I'll die happy). I (obviously by now) have the inverse view of most in this thread. For simple stuff I use C#. Microcontroller based projects I use C/C++. For larger projects I'll use LabVIEW.
Then, when something has to run in a browser I stick my head in the toilet and smash the seat down against my head repeatedly. Then I'll begin to search google for the 30 tabs I'll need to open to relearn how to get an environment setup, figure out which packages are available for what I'm trying to do, learn how to interact with the security of the backend framework I'm using, learn the de facto templating engine for said framework, decide which of the 4 available database relation packages I want to use for said backend, spend a week starting over because I realize one of the packages I based the architecture around was super easy to start with but is out of date; has expired documentation; conflicts with this other newer library I was planning on using for some other feature... Now I need a cold shower and a drink.
Cheers mates!
P.S. I do find a lot of modern web development...
https://hackernoon.com/on-kloudtrader-and-visual-programming...
It wasn't the most novel idea or anything but most existing systems were either clunky, expensive, or had UIs from the previous century. Ours was a hip SaaS inspired by Robin Hood, Bubble.is, and all the new Bloomberg terminal clones. We built an initial prototype using Google Blockly, did a ton of UI/UX research (studied every visual programming system from Sutherland's Sketchpad, to the Soviet DRAKON, to modern day MindStorms, Scratch, and the various PLC control systems and LabView derivatives) and slowly built out the rest of the algorithmic trading stack. It was tremendously difficult mainly because of lack of labor, our small startup only had 3 people. We were essentially translating by hand entire trading frameworks, backtesting tooling, and blotters into virtual Lego bricks. It was my first startup and I was inexperienced. We were fresh faced and between trying to raise funding, sales and marketing, product development, progress was slow. Patio11's (Kalzumeus) blog posts on Stockfighter were highly inspirational and we saw what they managed with only a small team and we tried to replicate. But between Patrick and the Ptaceks, they had several decades more engineering and business experience than us, something we completely discounted. The tooling around Google's visual programming system was like early Android development, works in theory but tremendously difficult to use. Microsoft's Makecode (which is also built on Blockly) had a magnitude more engineering manpower than us. Visual programming was not easy to build quickly — a production quality system wasn't something that you can clone in a weekend. We looked towards automation, around the same time, a code synthesis YC company called Optic appeared and we strongly considered leveraging them to allow us to build out faster.
https://news.ycombinator.com/item?id=17560059
However, a couple months later, YC funded a similar company called Mudrex who had a prettier UI and a founding team with a stronger fintech track record.
https://news.ycombinator.com/item?id=19347443
At that point we crossed the rubicon and pivoted to DevOps/PaaS, launching a Heroku-style product.
https://KloudTrader.com/Narwhal
Did the whole tour of Docker Swarm, Kubernetes, KVM etc. Built out our own cloud almost from scratch. Signed a contract with a broker to offer comission-free trading and everything. But it was a difficult product to sell in a crowded enterprise market with only a few (but big) customers and we were playing catch-up with companies like Alpaca, our product was being eaten from the corners by new features launched by companies like Quantopian and Quantconnect. Quantopian was where I cut my teeth on computational finance and automated trading, it was what inspired me to build a fintech startup in the first place, so in many aspects, our product being displaced was a validation of product-market fit if nothing else. In retrospect, at that time we should have switched to the ML Ops market instead, which is booming right now. Algorithmic trading, or at least the consumer focused variety that we were trying to sell, had a stack that is very similar to your usual ML stacks. In the two years I learnt about enterprise sales, the various shenanigans involved in FINRA, SEC compliance, and was tremendously valuable in terms of growth.
These days however we are mainly doing productivity software for voiceovers an...
What is your opinion on visual programming and diagramming libraries? Would you recommend particular libraries/services for visual programming if looking to build this type of UX/UI into a new product?
I'm interested in building a service that utilises a basic visual programming interface but I have no interest in building out a complex visual programming / diagramming UI toolset myself.
Do you consider the visual programming aspect a commodity or a selling feature? Or is that dependent on the market you are selling into?
Building a visual programming language in 2020 is fairly trivial and straightforward if your environment is the browser. There is a ton of libraries and open source reference projects out there: Xod, n8n, Makecode, Rete.js, Blockly. On the engineering side, as long as you do not try to map a mainstream programming language with its associated frameworks directly into visual programming, one-to-one on the syntax level, but instead build wrappers and simplified interfaces around it then it should work out pretty well.
Where I work (structural engineering firm) some of the engineers do use it for general purpose programming. I see the appeal of it, but keeping the layout tidy and all the clicking is just too much effort. However for generating geometry it's quite a useful tool.
[1] https://www.rhino3d.com/6/new/grasshopper
1) takes up a lot of space
2) each subroutine (sub-VI) has a window (in the case of LabVIEW, two windows), so you rapidly get windows spewed all over your screen. Maybe they've improved that?
3) debugging is a pain. LabVIEW's trace is lovely if you have a simple mathematical function or something, but the animation is slow and it's not easy to check why the value at iteration 1582 is incorrect. Nor can you print anything out, so you end up putting an debugging array output on the front panel and scrolling through it.
4) debugging more than about three levels deep is painful: it's slow and you're constantly moving between windows as you step through, and there's no good way to figure out why the 20th value in the leaf node's array is wrong on the 15th iteration, and you still can't print anything, but you can't use an output array, either, because it's a sub-VI and it's going to take forever to step through 15 calls through the hierarchy.
5) It gets challenging to think of good icons for each subroutine
6) If you have any desire for aesthetics, you'll be spending lots of time moving wires around.
7) Heavy math is a pain visually (so you use the math node that has it's own mini-language, and then you're back to text)
The problems you have with LabVIEW are. This seems LabVIEW specific, not visual programming specific. Let me contrast your experience with LabVIEW to my experience with Max/MSP:
> takes up a lot of space
Yeah, but, my Max code only took up a bit more space than my C++ code ever did. Sure, I now use Clojure which is a lot denser, but I didn't find the space it took up to be a problem when I used Max. I did aggressively factor code into subroutines though.
> each subroutine has a window
Agreed, but I think there is room for experimentation here. Synthmaker/Flowstone has a visual map thing at the top where you can see the zoomed in/out (iirc) views and subroutines opened in the main window, which then navigated this map. That is, subroutines didn't open in new windows and the map let you quickly navigate up and down the abstractions. It worked well in my brief time with Synthmaker, so I could envision LabVIEW or Max do something similar.
> debugging is a pain
I found debugging in Max, where you add probes to connections and can see the data that flows through them, much nicer than debugging C++, Java or Python (three languages I've used extensively).
> debugging more than about three levels deep
The trace/probe window let me see all probes, no matter the depth, so I didn't have this problem.
> It gets challenging to think of good icons for each subroutine
Max used words, but I didn't have to give them names if I didn't yet know what they would do: I could write the code without subroutines to experiment, then select any areas I wanted to abstract (to simply or for DRY) and it would put it in a subroutine and then I could name it or not as I felt necessary.
> If you have any desire for aesthetics
I did, but I didn't find I spent too much more time than I o aligning text, maybe a little. I factored things into subroutines aggressively and generally just kept things aligned for nice straight lines.
> Heavy math is a pain visually
True, I agree with this point, except that I don't think using the math/expression node is a problem and that going back to text for that is a problem. Right tool for the job: visual for dataflow, text for math expressions. I thought it worked really well.
I'm not saying visual languages are perfect, but I do think that there is lots of room for improvements and just because someone has had a bad experience with one visual language, doesn't mean the whole paradigm has those problems. That's like trying Haskell and complaining that all textual languages are hard because thinking in monads was challenging.
For an interesting take of something I'd love to see explored more: https://vimeo.com/140738254 (Jonathon Edwards' Subtext demo), there are many ideas left to explore and not enough visual languages to explore them, compared to the many textual languages we see that try new ideas.
You say visual programming seems to unlock a ton of value. What can you do with a visual language that is much easier than text? Difficult concepts might be easier to understand once there is visual representation, but that does not imply creating the visual representation is easier. And why should pictures be more approachable than text? People might understand pictures before they can read, but we still teach everyone to read.
Think coordinates, graphs, nodes, edges, flows, and nested diagrams.
“Visual” is especially meaningful in that many relationships are shown explicitly with connected lines or other means.
So yes, for many things a diagram, tree or table structure actually layer out in 2-dimensions to match what it represents is easier to understand.
Surely you appreciate diagrams in educational material despite the text. Surely you have drawn graphs or other kinds of diagrams when you need to visualize (spatially) relationships between parts of something you are designing?
If not, you just have a different style of thinking than many other people.
That contrasts with text code where connections are primarily discovered by proximity of code or common symbols.
Of course text is visual in that it’s a visible representation.
Spreadsheets are a good example of a combination of text “code” embedded in a visual table representation.
> What can you do with a visual language that is much easier than text?
Experiment an order of magnitude faster than you with text. What might take me 2s with a UI might take you 20s in text ore more. You also don't have to care about coding style, naming a ton of variables (just your nodes) so it removes lots of boilerplate.
Visual programming is usually domain specific so the UI fine tuned towards a certain problem. So comparing C++ for example to Max/Msp is missing the point. Visual programming is about solving domain specific problems while text based programming is general (to an extent... don't write device drivers in PHP).
Its actively designed to both prevent several typical problems of visual programming:
First, the overlapping wire mess getting out of hand. It doesn't let you overlap wires, if your program gets this complicated it forces you to break it down into a smaller unit. It takes some adjustment, but you get benefits from it.
Second, the "cant use it for general purpose stuff" problem is solved by the fact DRAKON can output to multiple different programming languages, meaning you can use it to build libraries and other components where you want without forcing you to use it for everything.
Labview does have a visual-diff tool, but when I was using labview regularly on a complex project, no-one used the diff system. They just checked out the entire file and compared it visually to another version.
Another thing: you can’t ctrl-f for control-flow structures. You end up mousing around for everything.
Another problem, all major graphical languages I’ve used are proprietary (labview, simulink, Losant workflow system).
If you label structures (you can turn on a structure's label and then give it some unique text) it becomes searchable. You can also create # bookmark comments that link to structures/nodes/anything on the block diagram.
NI just released a completely free for personal use version of LabVIEW (unlike their former "cheap" home edition that was watermarked and lacked features like compiling executables) though my belief is that they don't have an outwardly evident plan of long term strategy which is only likely to further the majority opinion of the platform. Also this ultimately doesn't help people that may be interested in using it in a work / academic environment or anyone that's used to a less expensive hardware platform. Ladder logic programming feels more akin to chiseling into stone, and the platform support in the NI ecosystem can be quite nice for a lot of applications.