Pipes was ahead of its time. Built a scraper and aggregator back in 2008-2010, fetching content from over 2000 sources, pumping all of it into a Drupal website. The amount of regex transformations and other things I had in that pipeline - is in hindsight - more complex, larger scale, and more convoluted that any other nifi or airflow pipelines I built later in my career. And all of it was free. What was possible was limited only by one's creativity!
It is surely only a matter of time until someone says 'yes, but node based editors are crap, you should do everything in code'. This has been discussed ad nauseum in HN, so I have tried to summarise the arguments here:
I'm not sure if I've ever seen a rigorous comparison between visual and text based editing before, and I'm really interested.
I disagree with some of your points:
- Higher level abstractions - I don't think that's a usage of high level abstraction I'm familiar with, or else it's not true. You can have very very high level abstractions in text code, you can have very low level abstractions in visual programming (max msp). This also goes for the point about optimization in text languages.
- Less hidden state - this really depends on the visual programming language. The geometry node editor in Blender has tons of hidden state - what are the inputs to your program, what are the outputs, how is it applied, etc.
- Less configuration - I have no idea, but my guess is you're thinking of something like Rust where library A uses image type X and library B uses image type Y and you need to write code to convert between X and Y yourself. I don't think this is inherent to visual programming - it's just a consequence of having a large, complex ecosystem with independently evolving parts.
- No syntax to remember - There's still syntax, it's just visual syntax. What does this squiggle mean? What does this color mean? What about this box around things? And unlike text, where you can just write what you see, even if you know what something is with visual programming you might not know how to replicate it (which menu button, what conditions are required, what selection do I have to make, etc), google it, etc.
- Looping and recursion - This is a failing point in visual programming languages I'm familiar with, but I don't think there's anything stopping a visual programming language from offering functions, looping, and recursion.
- Optimization - I touched on this above, but higher level abstraction makes _more_ opportunity for compilers to optimize. Specifically, in visual programming languages, there's no inherent concept of a "sequence" of calculations as there are in imperative languages, so compilers can freely parallelize.
Then: errors and run-time feedback - yes, this is true of most visual programming languages and is super great. It's equally true for many non-visual programming languages (e.g. rust).
The point about discoverability is good, and I agree. Just knowing what you can do in a programming language is very hard (the first steps). Generally in visual programming languages all the parameters are shown right there on the screen.
Here are my extra advantages to visual programming languages:
- I think it's good to make a distinction between control flow graphs and data flow graphs. IMO representing state machines in text based languages is pretty bad and understanding complex call flows etc can be very difficult. Most visual programming languages are purely data flow, but I think a control flow graph programming language would be very good at this.
- Another advantage of visual programming languages is you don't need to come up with names in order to reuse values. Having to name things isn't a joke, and seeing lines and arrows is easier to grok then trying to 1. understand what a complex name means (and maybe forcing yourself to ignore it if the name is bad) and 2. remembering that name so you can mentally connect uses.
That said, there's no need to force these to be mutually exclusive. You could mix them (allow text within the graph, or refer to the graph from text, allow making some files graphs or some text, etc), or even freely convert between them (luna lang, which appears to be thoroughly dead atm but here's some remnants: https://github.com/Luna-Tensorflow/luna).
I was an avid user of Pipes and blogged a bit about the experience of using it to build an aggregated set of feeds from various employee blogs to feed into our company site back in 2009 [0]. It holds a special place in my memory alongside early internet greats like del.icio.us [1]
I do a fair amount with n8n now, and this feels like n8n way way way before. I generally hate the term "ahead of its time", but I think this might apply.
14 comments
[ 2.4 ms ] story [ 33.6 ms ] threadA similar experience is Apache Nifi for the curious.
https://successfulsoftware.net/2024/01/16/visual-vs-text-bas...
I disagree with some of your points:
- Higher level abstractions - I don't think that's a usage of high level abstraction I'm familiar with, or else it's not true. You can have very very high level abstractions in text code, you can have very low level abstractions in visual programming (max msp). This also goes for the point about optimization in text languages.
- Less hidden state - this really depends on the visual programming language. The geometry node editor in Blender has tons of hidden state - what are the inputs to your program, what are the outputs, how is it applied, etc.
- Less configuration - I have no idea, but my guess is you're thinking of something like Rust where library A uses image type X and library B uses image type Y and you need to write code to convert between X and Y yourself. I don't think this is inherent to visual programming - it's just a consequence of having a large, complex ecosystem with independently evolving parts.
- No syntax to remember - There's still syntax, it's just visual syntax. What does this squiggle mean? What does this color mean? What about this box around things? And unlike text, where you can just write what you see, even if you know what something is with visual programming you might not know how to replicate it (which menu button, what conditions are required, what selection do I have to make, etc), google it, etc.
- Looping and recursion - This is a failing point in visual programming languages I'm familiar with, but I don't think there's anything stopping a visual programming language from offering functions, looping, and recursion.
- Optimization - I touched on this above, but higher level abstraction makes _more_ opportunity for compilers to optimize. Specifically, in visual programming languages, there's no inherent concept of a "sequence" of calculations as there are in imperative languages, so compilers can freely parallelize.
Then: errors and run-time feedback - yes, this is true of most visual programming languages and is super great. It's equally true for many non-visual programming languages (e.g. rust).
The point about discoverability is good, and I agree. Just knowing what you can do in a programming language is very hard (the first steps). Generally in visual programming languages all the parameters are shown right there on the screen.
Here are my extra advantages to visual programming languages:
- I think it's good to make a distinction between control flow graphs and data flow graphs. IMO representing state machines in text based languages is pretty bad and understanding complex call flows etc can be very difficult. Most visual programming languages are purely data flow, but I think a control flow graph programming language would be very good at this.
- Another advantage of visual programming languages is you don't need to come up with names in order to reuse values. Having to name things isn't a joke, and seeing lines and arrows is easier to grok then trying to 1. understand what a complex name means (and maybe forcing yourself to ignore it if the name is bad) and 2. remembering that name so you can mentally connect uses.
That said, there's no need to force these to be mutually exclusive. You could mix them (allow text within the graph, or refer to the graph from text, allow making some files graphs or some text, etc), or even freely convert between them (luna lang, which appears to be thoroughly dead atm but here's some remnants: https://github.com/Luna-Tensorflow/luna).
- [0] https://blog.davemo.com/posts/2009-04-06-yahoo-pipes-at-vend...
- [1] https://en.wikipedia.org/wiki/Delicious_(website)
December 2023 - 129 comments - https://news.ycombinator.com/item?id=38650878
I do a fair amount with n8n now, and this feels like n8n way way way before. I generally hate the term "ahead of its time", but I think this might apply.