I enjoyed reading this. I knew of quartz composer but I never did anything with it.
I love visual tools and I think they are underutilized today. I cut my teeth in ~2005 with Houdini[0] and Fusion[1] which are both heavily graph / node based (and procedural).
Most recently I have been rekindling my love for visual programming and flow based programming and plan to spend some time in January and February doing more research around flow based programming for infrastructure management.
I plan to get this sort of info published on my website which I have neglected for half a decade or more but if you are interested in visual programming you might enjoy checking these out:
I played around with Quartz Composer when I had my Macintosh. It was neat finding an obscure app deep in the directory structure that you could use to create interesting visuals. It was like my little secret.
I found out you could use it to create screen savers and iMovie effects. It provided a lot of opportunities to tinker.
The one thing that was annoying was that it didn’t offer a lot of control over the final visuals. I was used to Flash giving you frame by frame control, which you did not have in Quartz Composer.
You could create some cool effects, have some interactivity, but it seemed very limited and niche.
Quartz Composer was IMHO one of the coolest things to come out of Apple ~15 years ago in the early days of their Core Animation layer-based rendering framework. When I was first freelancing, I got a gig working on some displays for a large law firm and chose QC. Was able to do things like fancily display customer testimonials and the like, but have a Mac mini running things which had a shared folder that staff could just drop scans of letters in. It was the best of both worlds — amazing programmatic 3D rendering and Mac ease of use for the non-technical people.
I also used it briefly for some small about box effects in my apps, which allowed me to programmatically insert text or imagery based on the user’s registration or other variable data and just plug it into the inlets in the QC animation that was used.
It was basically a totally programmer-exposed motion graphics package. Too good.
The basic idea was that after launch, the app would use the NSWorkspace API to catch subsequent apps launching and fullscreen your Mac with an animation of the new app playing a guitar on stage, meanwhile the other apps already running had their icons in the crowd, dancing. The more apps you had launched, the rowdier the audience would get.
I used QC to create the animations with the placeholders for app icons and names, and then some variables related to crowd intensity. Then, at runtime, the app could plug in results from the NSWorkspace API to fill out those graphics and dial in the crowd response level.
Iris Explorer. I would probably still be blown away today. The examples that came with it made you think you were driving a supercomputer. It turned out to be rather hard for me to get it to do anything it wasn't already set up to do, but maybe I just missed the point.
IBM's open-sourced OpenDX is similar in scope and appears to run fine on Linux. I haven't used it in anger.
There have been various attempts to replace general purpose text-based languages (such as C++ or Python) with visual programming. I don't think any of them have been very successful. The level of abstraction is too low to be a good fit with visual programming. You end up with massive spaghetti diagrams. For something like data processing the level of abstraction can be much higher and is a much better fit for visual programming.
> There have been various attempts to replace general purpose text-based languages (such as C++ or Python) with visual programming. I don't think any of them have been very successful.
Their lack of success can have many reasons other than their general umfitness. IMO, their lack of success has much more to do with the ubiquity of text oriented tools. The breadth of tools and platforms a new programming paradigm would habe to support to truly be a replacement technology for text is staggering.
> You end up with massive spaghetti diagrams.
That's a matter of code quality which is a problem in text code too. Who hasn't encounter a codebase of many multi-thousand line C++ files. Who hasn't encountered an inheritance tree where various levels of the tree partially implement the interface and therefore results in code which bounces up and down the tree executing at all levels of the inheritance.
Generally, I find that most arguments favoring Text over visual programming are cart before the horse type arguments of the form "this feature is better in text". Usually that thing is better in text because we've had the time to make it better. The tools and platforms are mature not intrinsically superior. Text code is "clean" because we have used it long enough to understand what is "good" code quality. Text code is "durable" because we've made text standards and rendering ubiquitous across all platforms. And so on.
You make some good points. But I think part of it come down to the fact that text is a more compact representation than nodes and arrows. I wrote a visual programming tool for data and I think the visual approach is superior for many use cases in data processing. But I chose not to do it using a visual programming language. I did it in C++. Even if the visual programming tooling improved significantly, I still don't think I would choose it for general purpose programming.
I also came at some of this stuff from an early background in electronic music and modular synthesis (which is all about making the connections between functional blocks visual/visceral).
I've used Max/MSP and Grasshopper. I find them really great for building up interactive tools which have a bunch of inputs/controls.
The one down side of these tools is that they can turn into visual spaghetti code which is hard to trace.
I strongly recommend the last two links in the article for their visual catalogs of visual programming environments.
Visual spaghetti gets real bad real quick, but I find the process of despaghettification to be incredibly gratifying. Well routed graphs make my brain produce happy chemicals.
Surprised nobody's mentioned LabView yet! Anybody have experience/opinions using this? Their versioning is one of the worst I've ever seen, but they have a impressively comprehensive ecosystem of industrial test equipment.
I haven’t used LabView much myself but did work at a company whose product replaced it with .Net tooling. I got a sense that some common programming idioms become spaghetti. I want to say maybe that there wasn’t a proper loop in LabView?
Take this with two grains of salt. I’m not actually a LabView user and what I know comes from a company replacing it.
LabVIEW is a trap. If you are considering using LabVIEW for making product software, consider systems with better developer ergonomics, such as having your engineers write Python while being water boarded.
The problem is that LabVIEW, the language and IDE, comes with LabVIEW, the integration library for your NI sensors and test equipment and other doohickies. So slapping a program together to gather some data and display it is trivial. But over time, you're going to start to add business logic. And at first it might be bearable: You're code will end up spaghetti periodically, but you'll refactor a little bit and it'll be fine. But your codebase will expand and expand, well beyond the relatively trivial amount of machine interface code and eventually you'll just be developing software, but in LabVIEW.
And that's where LabVIEW's fundamental, inescapable flaws begin. A sample, in no particular order:
1. LabVIEW has a 1:1 correspondence between functions, files, and user interfaces. Every time you see a function (subVI in LabVIEW parlance), and want to know what it does, you'll end up opening 2 windows: the user interface, which for 90+% of vis is never actually shown to the user, or holds any relevant information whatsover, but has to be opened nonetheless), and the code, which may be trivial, or may be complex, but you had no way of knowing before you dove in. Because it takes up a whole window, you get far and beyond less information density than in a textual programming language, which pushes towards inlining a lot of repeated code, but because the actual experience of writing LabVIEW is so miserable, you'll want to implement a lot of stuff as subVIs. It's a lose-lose.
2. LabVIEW's pretty-path for software development is untestable. At some point, you'll say "Ah, I can make user interfaces actually work the way anybody wants them to using Events!". Events are lovely, except you cannot test them. Period. Despite LabVIEW being uniquely situated to allow for a system for inserting mocking code between all those user interfaces you've been making and your code, they refuse to do so, and since LabVIEW is proprietary garbage, you can't even fix this yourself.
3. Pixel art for subVI icons. Here, you get make a terrible choice: When looking at some LabVIEW code, do you want to A) have basically no way to distinguish between different functions without hovering over them (imagine if all your function names were func1() func2() etc, until you selected them), or B) spend an obnoxious amount of time designing little hieroglyphics for future programmers to decipher. The only silver lining is that you're fucked either way.
4. Laying out wires. Have you ever thought to yourself: "You know what I need? More freedom of choice when it comes to expressing myself through data flow"? Well good news! LabVIEW will give you an endless amount of personal freedom when it comes to expressing how data goes from the output of one function to the input of the next. Make sure to lay it out in a nice sane way, otherwise you will progressively go mad. If you get bored, you can always try and use the built in layout engine which, besides being written by an engineer who's idea of a tidying up is pushing all his crap against one wall and marveling at his newfound floor space, will also creatively route your wires all over the place. If you really enjoy formatting your code, LabVIEW might be a good choice.
5. On the topic of wires: You never really appreciate how much context gets encoded in variable names until you don't have any. Compared to other issues it's not THAT big a deal, but you'll miss it.
6. Version control: The diff tool works most but not all of the time, and the merge tool rarely can even load vis and when it can, it's utterly worthless anyway. Resolve all merge conflicts with "theirs" or "ours", or just abandon a branch based workflow altogether. Also, those binary blobs spontaneousl...
Any reason why we don’t have a visual tool that builds an AST?
I know that there have been a lot of insultingly bad visual tools. And I totally get why we often use text. And most of us are comfortable in our own IDE or what not setups.
But if you honestly take a step back, we’re trying to mimic a more graphical approach in text- curly braces, syntax highlighting, identation, etc.
I think the world is missing out on entire categories of folks who could be programming while we debate curly braces vs indentation and whether semi-colons are a good thing or not.
In the near future, will they look back and laugh?
I prototyped several user OSX user interface animations using Quartz Composer during my time at Apple. It was a great tool for rapid iterations. We would demo to Steve, get feedback and quickly make changes. Before Quartz Composer, Bas Ording would create prototypes using Macromedia Director. He had a nice collection of Lingo routines for various types of animations.
27 comments
[ 2.3 ms ] story [ 83.7 ms ] threadI love visual tools and I think they are underutilized today. I cut my teeth in ~2005 with Houdini[0] and Fusion[1] which are both heavily graph / node based (and procedural).
Most recently I have been rekindling my love for visual programming and flow based programming and plan to spend some time in January and February doing more research around flow based programming for infrastructure management.
I plan to get this sort of info published on my website which I have neglected for half a decade or more but if you are interested in visual programming you might enjoy checking these out:
Unit from Samuel Timbó:
https://github.com/samuelmtimbo/unit
https://ioun.it/
A video of me exploring what I figured out about it (while also learning to stream) https://www.youtube.com/watch?v=vwknTfGVDq8
Behave-Graph from Ben Houston:
https://github.com/bhouston/behave-graph
And the products I learned so long ago
[0] Houdini https://www.sidefx.com/products/houdini/
[1] Fusion https://www.blackmagicdesign.com/products/fusion
I found out you could use it to create screen savers and iMovie effects. It provided a lot of opportunities to tinker.
The one thing that was annoying was that it didn’t offer a lot of control over the final visuals. I was used to Flash giving you frame by frame control, which you did not have in Quartz Composer.
You could create some cool effects, have some interactivity, but it seemed very limited and niche.
I also used it briefly for some small about box effects in my apps, which allowed me to programmatically insert text or imagery based on the user’s registration or other variable data and just plug it into the inlets in the QC animation that was used.
It was basically a totally programmer-exposed motion graphics package. Too good.
Ars Technica write-up: https://arstechnica.com/gadgets/2007/11/ic-7/
My page: https://codesorcery.net/tag/ironcoder.html
The basic idea was that after launch, the app would use the NSWorkspace API to catch subsequent apps launching and fullscreen your Mac with an animation of the new app playing a guitar on stage, meanwhile the other apps already running had their icons in the crowd, dancing. The more apps you had launched, the rowdier the audience would get.
I used QC to create the animations with the placeholders for app icons and names, and then some variables related to crowd intensity. Then, at runtime, the app could plug in results from the NSWorkspace API to fill out those graphics and dial in the crowd response level.
Visual programming works great as long as you restrict it to a particular domain (e.g. data processing, visualization, audio processing etc).
IBM's open-sourced OpenDX is similar in scope and appears to run fine on Linux. I haven't used it in anger.
why?
Their lack of success can have many reasons other than their general umfitness. IMO, their lack of success has much more to do with the ubiquity of text oriented tools. The breadth of tools and platforms a new programming paradigm would habe to support to truly be a replacement technology for text is staggering.
> You end up with massive spaghetti diagrams.
That's a matter of code quality which is a problem in text code too. Who hasn't encounter a codebase of many multi-thousand line C++ files. Who hasn't encountered an inheritance tree where various levels of the tree partially implement the interface and therefore results in code which bounces up and down the tree executing at all levels of the inheritance.
Generally, I find that most arguments favoring Text over visual programming are cart before the horse type arguments of the form "this feature is better in text". Usually that thing is better in text because we've had the time to make it better. The tools and platforms are mature not intrinsically superior. Text code is "clean" because we have used it long enough to understand what is "good" code quality. Text code is "durable" because we've made text standards and rendering ubiquitous across all platforms. And so on.
I've used Max/MSP and Grasshopper. I find them really great for building up interactive tools which have a bunch of inputs/controls.
The one down side of these tools is that they can turn into visual spaghetti code which is hard to trace.
I strongly recommend the last two links in the article for their visual catalogs of visual programming environments.
https://www.sidefx.com/products/houdini/
Then there is https://cables.gl that is all web based and pretty neat to play around with.
/best regards yours truly node junkie
https://www.ni.com/en-us/shop/labview.html
Take this with two grains of salt. I’m not actually a LabView user and what I know comes from a company replacing it.
The problem is that LabVIEW, the language and IDE, comes with LabVIEW, the integration library for your NI sensors and test equipment and other doohickies. So slapping a program together to gather some data and display it is trivial. But over time, you're going to start to add business logic. And at first it might be bearable: You're code will end up spaghetti periodically, but you'll refactor a little bit and it'll be fine. But your codebase will expand and expand, well beyond the relatively trivial amount of machine interface code and eventually you'll just be developing software, but in LabVIEW.
And that's where LabVIEW's fundamental, inescapable flaws begin. A sample, in no particular order:
1. LabVIEW has a 1:1 correspondence between functions, files, and user interfaces. Every time you see a function (subVI in LabVIEW parlance), and want to know what it does, you'll end up opening 2 windows: the user interface, which for 90+% of vis is never actually shown to the user, or holds any relevant information whatsover, but has to be opened nonetheless), and the code, which may be trivial, or may be complex, but you had no way of knowing before you dove in. Because it takes up a whole window, you get far and beyond less information density than in a textual programming language, which pushes towards inlining a lot of repeated code, but because the actual experience of writing LabVIEW is so miserable, you'll want to implement a lot of stuff as subVIs. It's a lose-lose.
2. LabVIEW's pretty-path for software development is untestable. At some point, you'll say "Ah, I can make user interfaces actually work the way anybody wants them to using Events!". Events are lovely, except you cannot test them. Period. Despite LabVIEW being uniquely situated to allow for a system for inserting mocking code between all those user interfaces you've been making and your code, they refuse to do so, and since LabVIEW is proprietary garbage, you can't even fix this yourself.
3. Pixel art for subVI icons. Here, you get make a terrible choice: When looking at some LabVIEW code, do you want to A) have basically no way to distinguish between different functions without hovering over them (imagine if all your function names were func1() func2() etc, until you selected them), or B) spend an obnoxious amount of time designing little hieroglyphics for future programmers to decipher. The only silver lining is that you're fucked either way.
4. Laying out wires. Have you ever thought to yourself: "You know what I need? More freedom of choice when it comes to expressing myself through data flow"? Well good news! LabVIEW will give you an endless amount of personal freedom when it comes to expressing how data goes from the output of one function to the input of the next. Make sure to lay it out in a nice sane way, otherwise you will progressively go mad. If you get bored, you can always try and use the built in layout engine which, besides being written by an engineer who's idea of a tidying up is pushing all his crap against one wall and marveling at his newfound floor space, will also creatively route your wires all over the place. If you really enjoy formatting your code, LabVIEW might be a good choice.
5. On the topic of wires: You never really appreciate how much context gets encoded in variable names until you don't have any. Compared to other issues it's not THAT big a deal, but you'll miss it.
6. Version control: The diff tool works most but not all of the time, and the merge tool rarely can even load vis and when it can, it's utterly worthless anyway. Resolve all merge conflicts with "theirs" or "ours", or just abandon a branch based workflow altogether. Also, those binary blobs spontaneousl...
I know that there have been a lot of insultingly bad visual tools. And I totally get why we often use text. And most of us are comfortable in our own IDE or what not setups.
But if you honestly take a step back, we’re trying to mimic a more graphical approach in text- curly braces, syntax highlighting, identation, etc.
I think the world is missing out on entire categories of folks who could be programming while we debate curly braces vs indentation and whether semi-colons are a good thing or not.
In the near future, will they look back and laugh?
https://scratch.mit.edu/
https://snap.berkeley.edu/