I guess this is turning into another "visual programming" post.
On my side I'd like to ask your opinion about Blueprint. It really turned me off back in the day and I always wonder why they couldn't optimize UnrealScript further instead of getting something totally new.
Yeah for sure eventually everyone get used to it and it becomes the new standard ("How can I NOT do Blueprint? It's so splendid" type of posts will show up) because UE is an extremely competetent engine.
But why did we bring it forward in the first place? Is there enough reason to do so aside from that some teams get to have their own marks in the world and some managers can grab more power?
I assume you work in unreal? Unity is more popular though and doesn't have visual scripting. Unreal Engine just having visual scripting and C++ with nothing in between is probably its biggest weakness, people prefer C# over those.
It’s nice for asynchronous programming. Promises or callbacks take some getting used to, but a block diagram is as transparent of an abstraction for asynchronous flow as it gets.
> But why did we bring it forward in the first place?
Speculation: visual programming is a bit more beginner friendly, and more compatible with the brains of people designing stuff in UE. I wish blueprint was around when I was 13, messing around in UT’s version of UE and having no idea what unreal script could do. Visual language can break out all possible components into UI menus, just like everything else in UE. Visual oriented people really click with tools like visual programming.
Personally I think it’s cynical / arbitrary to pin the decision to corporate power struggles. Sometimes people propose new things out of interest or as a side project and then it grows from there.
I'm a programmer who does hobbyist gamedev and is a big fan of blueprints despite writing in text all day, for a few reasons.
* A lot of game engine functionality is extremely helpful but not discoverable. Visual scripting languages make the full spectrum of the engine's API more easily discovered
* Games are very stateful, and I find it is much easier to conceptualize this with blueprints than with code, which is better for functional stuff
* Collapsing nodes in blueprints is much more powerful as a documentation tool than collapsable lines of code. When you collapse nodes, you can show the execution running through long text high level descriptors of what hidden blocks are doing, and without needing to create a function for that task specifically.
e.g. code might do something like
def update_target():
target = find_new_target() # set target to nearest enemy unit
attack_pattern = choose_attack_pattern() # choose next attack pattern
execute_attack() # perform attack montage and execute effects
____________________
whereas the bp would be
Set next target to nearest enemy ->
Choose next attack pattern ->
Perform attack montage and execute effects
It becomes pseudocode that is bright and on the forefront of your attention, vs. most code editors which will highlight code and lowlight comments.
I worked on a visual programming tool from the late 90s to the early 2000s. The same problems apply then as they do now.
There's an unwinnable war between keeping things simple and being complex-enough to do useful things. Visual stuff is great for simple things, but simple things aren't very useful. When you really need to do more complex things, you reach a limit very quickly. It becomes pretty unmaintainable very quickly and then people will "graduate" from it and go to something more convenient.
My kid is learning Scratch right now, and it's taught him a lot of great stuff. But after about a year, he's ready to move onto Python. Scratch has taught him some very valuable concepts and he was able to jump into some Python concepts with ease (others still escape him). But you can only go so far with Scratch vs other programming languages.
The same goes for other visual tools. In the end, people graduate from the visual style because it ironically becomes too complex because they want to keep things simple, and then you've lost a customer.
I'm curious if you've tried Simulink? Because my experience with visual programming tools maps largely to yours. Especially the RAD tools of the late-90's and early-00's. Eventually you get to a place where the "simple" thing is a dizzying array of radio buttons and check boxes to manage that are all hidden away in their own little silos that you can only get to by drilling down through the visual hierarchy.
This becomes a huge problem with your "pivot" for changing things is plumbing up and down, up and down, up and down the hierarchy. Instead of being able to flip the whole thing on its head and seeing all the settings/flags/options for whole classes & categories of similar things and changing them from that perspective.
It's like there's a transition point where you know enough about how things are all put together that being forced to interact with multiple things at once only through the top of their individual silos starts to fall apart pretty dramatically.
The reason I ask about Simulink, is that it's very common in the toolchain of our customers (I'm CEO of https://www.auxon.io) and we're building an integration to it for our product. For the most part it seems to do a much better job of being an IDE for more complicated models/programs compared to the bevy of RAD tools I've used in the past. I have certainly encountered limitations reminiscent of those old RAD ways, but I can also see how much better a job it seems to do before you hit those snags.
I've used Simulink a lot in the past, and the things that it gives you (as well as it's limitations) are a big driver for my attempts to build tools in this space (I think I'm trying to do a similar thing to you -- causal reasoning using data flow information and other graph-structured engineering data such as requirements traces).
Yeah, that's the first half of our stack. Ultimately it exists in service of the second half of our stack which is for automatically stressing and analyzing system behavior to localize root causes, uncover emergent properties, and auto-optimize system parameters.
On the way toward building everything we needed for the analysis we wanted to do we realized we had a distributed tracing system suitable for continuous system lifecycles & embedded systems (as opposed to transactional lifecycles & IT systems) and a spec/query mechanism over what amounts to a logic model derived from system executions. That sort of thing tends to be valuable to folks who aren't as far along in their development or use case maturity to need bleeding out all the corner cases & "unknown unknowns" from their systems, so we exposed many of our building blocks as features themselves.
Somewhat ironically, given the thread this is all related to, everything we build today is for CLI consumption on Linux and Windows. We will build out a kind of IDE/Workbench UI this year to fulfill some of our vision around the category of CAE tools we're angling toward, and to access additional kinds of customers, but predominantly folks have preferred CLI shaped tooling because it's easier for them to bake our continuous verification & validation capabilities into their existing processes when we don't force them into a siloed GUI.
I'd love to have a chat at some point. I'm trying to (slowly) bootstrap some tools for the embedded space, and we're trying to get a grasp on (a) what our MVP should look like, and (b) how to go about approaching customer #1.
He's been in the games industry for some years now, and as a side-project he teaches kids how to code — I'm pretty certain that he builds such crazy projects mostly as a fun hobby, perhaps also partly as a showcase for his teaching.
IMO, for these kinds of projects, I suspect Scratch doesn't offer many advantages (if any at all). He likes a challenge :)
— More details, including an interview, can be found here:
He's pretty responsive to comments on his Scratch homepage — and can also be found under the same name on Twitter/FB/etc. He's always been happy to help others. Give him a shout if you have any specific questions.
(FWIW: I worked with him — Cliff — for a short while, back in the 90s)
People have also made neural networks using Redstone in Minecraft. You can make anything you want in Scratch; the question is whether it is wise to do so.
He seems to code an interpreter in scratch and then do the other stuff. Coding an interpreter is a lot easier than coding a game, and after that you really aren't coding in scratch anymore.
Complex minecraft works similarly, instead of working in redstone they make the logic gates and then the bit adders in redstone and then used programming as normal on those constructs.
In the case of his Beeb project, the 'interpreter' you speak of here is in fact a fairly full featured emulator. Although yes, the z-machine (Zork) is an interpreter of sorts.
But all of his other games - including the clones of arcade games - aren't emulators nor interpreters. They're just games, written from scratch (pun unintended), from the ground up, in pure Scratch.
After all, his day job is in fact games dev - and with 25+ years experience (I worked with him mid-90s), one could say he's a veteran in that field.
In reality, it depends on the what the game actually is, as to whether it is harder to implement than an emulator or an interpreter. e.g. I strongly suspect that Battleships was much easier to implement than the Beeb emulator was.
In my experience, there are three things that visual programming tools make difficult:
* abstraction
* version control
* test automation
Those are also (in my opinion) the three techniques which separate proper "software engineering" from coding/scripting, which is why visual programming tools are pretty much universally unsuited for complicated projects.
The one asterisk I would put on that is tools in which the visualization is based upon an actual, human readable, programming language. I did some work with XSLT back in the day using a visual programming tool, and it worked reasonably well.
I do wonder why some concept likes building chips doesn't make it into visual environments? I should be able to pass a chip with inputs and outputs to others.
It does in some. Visual programming environments are common for setting up the giant sound systems (think hundreds of channels of high-quality audio) that run everything from movie theaters to conference venues to parliamentary halls to amusement parks. Some of them support encapsulation of audio or logic elements into reusable packages.
Another technique they use for abstraction before that point include one-to-n on-page connectors and multi-signal buses when the visual wires get to be too cumbersome.
Snap! supports user defined blocks, first class functions, lexical closures, eval/apply, and even call-with-current-continuation. See the summary and description I posted in reply to the GP.
The really funny bit is that the classic visual programming tool LabView is designed mostly for test automation. But it ends up practically requiring the creation of unmaintainable spaghetti (this[1] being a classic example) and there's no test automation possible for it. It's a test automation platform so complex it needs (but doesn't have) test automation for its automation of your tests.
No one should ever write code like that and no experienced LabVIEW developer would find that anywhere close to acceptable. And there are no doubt analogs in the text-based world, except you may not even know it because the structure isn’t as visible.
Also, there are certainly test frameworks for testing LabVIEW code. NI has a few but then JKI, a third party company, has multiple testing frameworks including Caraya, which is similar in philosophy to something like FsUnit for F# or Elixir’s ExUnit.
An interesting thing about things like version control and test automation is that they are almost like "meta interpreters" that interpret your code, but do so for purposes other than executing it. Syntax checking and linting come to mind too. Those things are possible, not so much because the program is in text format, but because it's in an open format that anybody can parse.
If a visual programming language stored its programs in a human readable or at least open format, then it would be possible for people to create those tools for it. But then, people would start writing and editing the program directly in its storage format, and it would cease to be visual programming.
Yup, I've been working on a visual browser automation tool (https://browserflow.app/) for the past year, and it's been quite the fun design problem to balance simplicity (so that it can be used by non-technical users) with flexibility (so that it can handle more than the simplest cases).
An approach that worked for me was to provide an escape hatch (in my case, giving the user a way to run arbitrary Javascript on the page being automated) so that the built-in commands could be designed for the most common scenarios and users would still have a way to handle gnarly edge cases.
There is most definitely an unwinnable war between simplicity/ease-of-use and more power/customizability.
That being said, Alteryx is an example of a company that has made a ton of money via a data munging tool that is a visual programming language. The users are fanatical as well.
It is not really a war though is it? Some people prefer visual tools like Alteryx, Knime or Easy Data Transform. Other people prefer R or Python. And some people will alternate approaches, depending on the problem at hand. Also, many 'visual' tools include the option of text based programming, for flexibility.
I think visual programming tools are great when they are aimed at a specific domain. but not when they try to do too much. While Scratch is great for teaching kids to code, trying to come up with a visual replacement for C++ is almost certainly a bad idea.
Also it is possible to blend the two approaches. For example by allowing users to script some of the boxes in a text based language. This is the approach I have taken with my own visual-based data wrangling tool. The standard transforms cover 95% of cases people need (re-order columns, pivot, filter etc) and you can script a Javascript transform for anything else.
Snap! is not simply all the usability and functionality of Logo, but also all the functionality and power of Scheme! Without any of the dumbing down of Scratch or Logo. Visual block programming. Turtle Graphics. Sprites. Lexical scoping. Lambda. Closures. Call/cc. Plus JavaScript integration and web stuff. With extensions for networking, AI, machine learning, speech synthesis and recognition, graph theory, robotics, Lego, Arduino, 3d graphics, 3d design, 3d fabrication, and 3d printing, embroidery, etc. ;)
History of Logo. Proc. ACM Program. Lang., Vol. 4, No. HOPL, Article 79. Publication date: June 2020.
6.2 Brian Harvey’s Personal Narrative on Snap!: Scheme Disguised as Scratch (pp. 49-50)
In 2009, the University of California, Berkeley, was one of several universities developing a new kind of introductory computer science course, meant for non-CS majors, to include aspects of the social implications of computing along with the programming content. Scratch wasn’t quite expressive enough to support such a course (it lacked the ability to write recursive functions), so Prof. Daniel Garcia and I thought "What’s the smallest change we could make to Scratch to make it usable in our course?" After 20 years teaching Structure and Interpretation of Computer Programs [Abelson et al. 1984], the best computer science text ever written, I knew that the answer to "what’s the smallest change" is generally "add lambda." I joined forces with German programmer Jens Mönig, who had developed BYOB (Build Your Own Blocks), an extension to Scratch with custom (user-defined) blocks, including reporters and predicates. At that time we were hoping to convince the Scratch Team to adopt our ideas, so we took "smallest change" very seriously. BYOB 3.0 [Harvey and Mönig 2010], with first class procedures and first class lists, added only eight blocks to Scratch’s palette. (The code is almost all Jens’s. My contribution was part of the user interface design, plus teaching Jens about lambda.) Version 3.1 added first class sprites with Object Logo-style inheritance. The Berkeley course, The Beauty and Joy of Computing (BJC) [Garcia et al. 2012], is also used by hundreds of high schools, especially since the College Board endorsed it as a curriculum for their new AP CS Principles exam. Unfortunately, some teachers have no sense of humor, and so BYOB version 4.0, a complete rewrite in JavaScript, was renamed Snap! [Harvey 2019]. [18]
Since Scratch seemed to be positioned as the successor to Logo, it was a goal for Snap! to restore the features from Logo that are missing in Scratch. The most important missing feature, the ability to define functions (and therefore to use recursive functions), is at the core of the new language. (Scratch introduced user-definable command blocks in version 2.0, but still doesn’t support user defined reporters.) Scratch had also replaced the structured text (word and sentence) functions with a flat text string data type. We wanted to be backward compatible with Scratch, so we implemented words and sentences as a library, defining first, last, butfirst, and so on. (Since block languages allow multi-word procedure names, and you don’t have to type the long name in order to use the procedure, the library names are, e.g., all but first letter of.)
Lists are first class and can be arbitrarily deep in sublists. The usual higher order functions on lists are provided; the graphical representation of lambda is built into the blocks representing higher order functions, and so beginning users can use higher order functions in simple cases without thinking hard about function-as-data at all, but th...
Yeah, I think the endgame of visual programming tools is to just eliminate syntax errors.
You could probably conceive of C as a visual programming language if you spent long enough time. All of your keywords can be reduced to blocks, forcing people to insert the appropriate parameters. And once you allow people to define their own blocks, you're like a good portion of the way there.
The problem is that after a while, it is simply faster to type. The breadth of options available becomes too much to manage from menus and drag and drop interfaces. Already you're probably typing in some values and names. So you're constantly switching between keyboard and mouse.
At some point, the benefit of perfect syntax doesn't outweigh the loss of productivity from not being able to immediately use any construct the language makes available to you.
It's definitely allowed non-programmers to make complex AI/game state for their creations, without having to deal with syntax errors or other issues. We choose Scratch/Blocky since this would be used for more managing states/logic, rather than flowing data (where node based makes more sense)
Something I'd like: a modern maintained DRAKON version: https://en.wikipedia.org/wiki/DRAKON . Tried it once, generates reasonably good C code. It would be great to have a modern FLOSS version.
That's exactly how I feel when reading code. Recently, I explained the problem to a dear colleague when working on a code base (with about 600K lines of code). I compared the situation to working in a large office building with all lights turned off and you have to skim through thousands of documents with a flashlight.
It's comforting to know that other people feel the same way.
Side note: interactive debugging is kinda inherent with the flow based programming paradigm. The program is continuously executing with every new action and immediately reflecting the current state. To program with it is to debug. If there’s an error with a component, it turns red and everything down stream breaks as well.
Flow paradigm doesn’t claim scale though. It’s about being a DSL for specific environments, often visually-oriented software like CAD or interactive art, which doesn’t have heavy 10K person team requirements.
For example, Programming a security system for my house would be great visually. Programming all of the houses on my block? Not so much. Even though the textual rules would be roughly equivalent.
To further your point. Copy/paste. Email. Templating. Highlighting. Rendering that is not a part of the visual, such that accessibility tools work with it.
There is an impressive list of things your enter a race to parity with.
Yeah, this is going to be yet another "visual programming" topic.
I sort of disagree with the intro to LabVIEW:
> There are a large number of visual programming tools that are roughly in the paradigm of ‘boxes with some arrows between them’, like the LabVIEW example above. I think the technical term for these is ‘node-based’, so that’s what I’ll call them.
There are NOT a "large number" of visual programming tools that are roughly in the paradigm [...] like LabVIEW. I am sure there are some but, really, how many active ones?
Also, "node-based" is not the main concept. The most essential concept of LabVIEW is known as "data-flow" (https://labviewwiki.org/wiki/Data_flow). Data generated by a node "flows" to the other nodes it is connected to and when any node has data on all it's inputs, it executes and produces data which flows to whatever nodes it's connected to. This happens asynchronously. You get concurrency "out of the box" with LabVIEW. That's extremely powerful and much easier to reason about, IMHO, than the concurrency constructs in virtually any mainstream computer language.
I think there's plenty of room for all kinds of languages (both text and visual) as DSL's. If they're well-designed and intuitive, they're like a hot-knife through butter for a domain expert. The nice thing about a really good DSL is that you use it more like a library, whereas with a general purpose language you would be reaching for a framework. It gives you a lot of creative latitude and it feels right. The problems start when people start using them as golden hammers, or use them for things which they're not good at.
>There are NOT a "large number" of visual programming tools that are roughly in the paradigm [...] like LabVIEW.
They list half a dozen or so. Most of the node based systems are also flow oriented, aka nodes are functional and stateless which allows them to be mapped across any amount of input.
It's a neat pattern but it's really not as uncommon as you think. The mainstream shader languages work this way, for example.
> Data generated by a node "flows" to the other nodes it is connected to and when any node has data on all it's inputs, it executes and produces data which flows to whatever nodes it's connected to.
I think you just described flow programming. There are at least 10+ visual languages / environments that work like this.
In my opinion, LabVIEW is the most general purpose and powerful visual programming language. Others I know of are vvvv gamma where the recent update basically makes it a .NET language (although LabVIEW also has .NET integration) and Pure Data. Other tools are Simulink, TouchDesigner, Max/MSP, and Grasshopper for Rhino. After that, it drops off pretty rapidly to a collection of barely used or even known about tools pr highly specialized node-based tools.
If not a "large number", then more like a "huge number".
Even the single application Blender itself has several different visual node based programming systems (most but not all built on top of the same framework, and applied to different kinds of programming), for shader programming, CPU image processing, video compositing, lighting, animation and constraints, particle systems, physics simulations, procedural mesh generation, and the Blender Foundation and third parties are developing even more, like procedural city generation:
>Blender has a few nodes systems. The first and obvious one is Blenders shading system for Cycles and Eevee. This is the node system that we will focus on in this article.
>But we also have nodes for compositing, lighting and textures, even if the use case and future for texture nodes are uncertain at this point.
>We can also extend Blender with other node systems through add-ons. The most well-known is probably animation nodes that come bundled with Blender.
>There are also other node systems available for Blender. AMD ProRender for example, a third-party render engine that has its own shader node system. Another example is Luxrender. There is also mTree for generating trees with nodes and Sverchok that can manipulate all kinds of data with nodes.
There's so much interesting prior work!
I really enjoyed this paper “A Taxonomy of Simulation Software: A work in progress” from Learning Technology Review by Kurt Schmucker at Apple. It covered many of my favorite systems.
It reminds me of the much more modern an comprehensive "Gadget Background Survey" that Chaim Gingold did at HARC, which includes Alan Kay's favorites, Rockey’s Boots and Robot Odyssey, and Chaim's amazing SimCity Reverse Diagrams and lots of great stuff I’d never seen before:
I've also been greatly inspired by the systems described in the classic books “Visual Programming” by Nan C Shu, and “Watch What I Do: Programming by Demonstration” edited by Alan Cypher.
Also, spreadsheets are certainly one of the most popular, widely used, easily accessible, and important visual programming languages. It's not an understatement to say that the economy would grind to a halt and civilization as we know it would collapse if spreadsheets suddenly disappeared tomorrow.
The article quoted one of my earlier posts on that subject, but I've written more about spreadsheets as visual programming, and cited Brad Myer's work and articles about visual programming languages, and there's more in the HN discussion of his classic 1989 paper "Visual Programming, Programming by Example, and Program Visualization; A Taxonomy," Proceedings SIGCHI '86: Human Factors in Computing Systems. Boston, MA. April 13-17, 1986. pp. 59-66:
DonHopkins 3 months ago | on: Spreadsheet is a software development paradigm
Spreadsheet certainly are visual programming languages: by any measure, by far one of the most common most widely used types of visual programming languages in the world.
>DonHopkins on July 13, 2019 | on: I was wrong about spreadsheets (2017)
>Google sheets (and other google docs) can be programmed in "serverless" JavaScript that runs in the cloud somewhere. It's hellishly slow making sheets API calls, though. Feels like some kind of remote procedure call. (Slower than driving Excel via OLE Automation even, and that's saying something!) Then it times out on a wall clock (not cpu time) limit, and breaks if you take too long.
>A CS grad student friend of mine was in a programming language class, and the instructor was lecturing about visual programming languages, and claimed that there weren't any widely used visual programming languages. (This was in the late 80's, but some people are still under the same impression.)
>He raised his hand and pointed out that spreadsheets qualified as visual programming languages, and were pretty darn common.
>They're quite visual and popular because of their 2D spatial nature, relative and absolute 2D addressing modes, declarative functions and constraints, visual presentation of live directly manipulatable data, fonts, text attributes, background and foreground colors, lines, patterns, etc. Some even support procedural scripting languages whose statements are written in columns of cells.
>Maybe "real programmers" would have accepted spreadsheets more readily had Lotus named their product "Lotus 012"? (But then normal people would have hated it!)
I've been using LabVIEW for 15 years, first in an academic and scientific setting, then for machine vision in surveillance, then as the HMI for industrial robots. It is, for all intents and purposes, a general purpose programming language, and I think it would be more popular if it wasn't so stupid expensive.
I was never interested in or good at coding, but LabVIEW made sense to me. Now I'm using programming principles I learned in LabVIEW and re-implementing them in C++ and C#. It's been a journey.
I think what the author did here hits hard on why hacker news is such a great resource. To quote:
> Most fields have a problem with ‘ghost knowledge’, hard-won practical understanding that is mostly passed on verbally between practitioners and not written down anywhere public. At least in programming some chunk of it makes it into forum posts.
I'd be incredibly interested in seeing a series of posts like this, where authors mine hacker news and other forums for insights that they consolidate (then repost here).
What other topics do y'all think HN provides unique insight on?
I agree that this type of compendium/synthesis is useful. I was amused to open this thread and see comments focused on visual programming rather than the meta-aspect.
As for what HN could be mined for, a big topic is software languages. There are ways to subdivide that: by language, by optimization aspect (e.g., comparative performance). Geographic topics like cost of living and quality of life. Remote work pros and cons.
Another way of identifying topics worth mining would be identify posts where @dang provides a link to previous posts and those previous posts have a sufficient number of comments.
> What other topics do y'all think HN provides unique insight on?
Career managment I think (not sure if it's the good term for it). I see lots of people in their 40's or more that quit some FANG job for another job that earns less but is more aligned with how they want to live their life. I see lots of people advocating to quit jobs as soon as you see red flags. I think both of these are good wisdom.
On the other hand, a point that's still a mystery to me is "programming language choice". I've learned useful heuristics like you can do web backend in pretty much anything, if you have no libraries for something that needs them like an AWS SDK you're going to suffer, what helps in the long run slows you down at first, but that's about it. I don't know if it's because I'm too young in this field, because I don't understand/missed something, or because there is no big truth to find.
Personally I think programming language is about a compromise between performance and convenience.
For instance, I program in Python if speed is irrelevant. Rust if it is very relevant. C# if I want a large project where performance is somewhat important.
I think there are a wide range of languages for each "category" where I put Python, C# and Rust above, but those 3 are my current favorites.
Certain languages are very well suited to specific tasks. You learn these with experience. Initially I loved python because it was a joy to slice and dice strings in it. I understand that Ruby is popular for the same reason. Closure is great if you want lots of agents processing stuff on a network stack.
If I were you, I'd just focus on making fun projects in your favorite language.
I think concern about “ability to hire” is a bit overblown: I’ve worked at a couple places now that have taken the “hire people that know what they’re doing and they’ll figure out the language” approach, and it’s generally successful for most business applications: this is true even when the team is using unusual languages like Clojure.
The problem is really about tooling, you also need people who can fix bugs/port to new arch in the compiler and related tools. For a very popular language you can always find the right people for this task or even offload this task like they do with oracle.
Those are layman’s concerns. Programmers love to learn new things and are typically good at it. If a given tech has merit and some unique advantages.
Edit: I didn’t read the comment right. Productivity and being able to solve precise problems are essentially why you’re choosing a technology in the first place?
> Those are layman’s concerns. Programmers love to learn new things and are typically good at it. If a given tech has merit and some unique advantages.
I don't think that's true, I believe most programmers are either dark matter developers [1] or people doing Java (or equivalent) all their life for a software service company. There is nothing wrong with that, but that would explain why less popular languages and ecosystem still have lots of stuff. For example, Rust has ~60k crates on crates.io, while Maven Central has ~400k packages. I believe Java is used by more than 10 times the number of people that use Rust. But the people writing the libraries in the first place are usually the people that are open to trying out new languages.
I concur, though most of the traits you mention are not problem-specific.
When putting a language into "your toolbox" you indeed SHOULD focus on "popularity, maintainability, being able to hire."
However, when picking which of the languages you have learnt to apply to a given problem you need to know their relative strengths.
Most of the strengths you mention help you pick the "best in class" language. But only the problem can tell you which subset of langauges you are picking from. Sometimes the best possible tool for your problem is going to be unpopular.
That’s true for conventional languages, you can trade off some performance for a bit of productivity. However, if you want significantly more productivity, we hit walls quickly in PL design where trading off performance is no longer effective. So ya, Kotlin is slightly better and slower than C, but both languages are fundamentally still in the same ballpark that we don’t know how to get out of yet.
The author alludes to problems related to complexity, modularity, and formatting, of visual code. I think a hard problem for visual languages is the sheer physical labor of creating code using drawing tools.
I used LabVIEW extensively for a few months, and ended up with crippling wrist fatigue and eyestrain headaches from all of the fine mouse work and menu selections needed to write even small programs. This is an ergonomic problem with all graphical software, but is particularly acute with programming because of the amount of code that gets written.
Another problem is innovation. You can quickly spin up an experimental language and try it out because the infrastructure -- editor and command shell -- already exist. This is one reason why we enjoy such a proliferation of new language ideas. New ideas in graphical software require intensive effort to try out.
On the other hand, there does seem to be something about data flow programming (LabVIEW, Excel) that is attractive to beginners, that can't be overlooked. Excel is probably the number-one programming tool in use right now.
I’m not sure that a few months is enough to understand the true pros and cons of a language. I’m using Elixir in my day job now and have used F# for side projects, and it’s not clear to me any one is faster to program in, especially for GUIs. LabVIEW actually shines, contrary to popular but inexperienced opinion, for large projects. I have used LabVIEW rather extensively, and it seems any project would have seen massive delays if I used something else, which is why I didn’t.
But I can indeed see an argument for strain when using the mouse so much. I have experienced that before and improved my ergonomic setup. As an aside, it is curious to me that many programmers are heavy PC gamers as the one true way to game and don’t complain about the mouse use there but will throw out all sorts of complaints when using it to program or use tools.
> New ideas in graphical software require software intensive effort to try out.
This “problem” isn’t inherent to graphical programming though. It’s because legions of programmers are convinced that text=programming and have spent decades writing text-only tools. You see this friction when using LabVIEW with GitHub, whose tooling is incompatible with the diff and merge tools that come with LabVIEW, which makes GitHub look outdated and not LabVIEW.
I’m actually working on ideas to build a toolkit for graphical programming languages. There are certainly challenges though, such as the desktop GUI ecosystem being a mess and reliance upon graph data structures and algorithms (like automatic layout).
> On the other hand, there does seem to be something about data flow programming (LabVIEW, Excel) that is attractive to beginners
Dataflow is attractive to advanced users as well, which is part of the draw to languages like F#, Elixir, Racket, Clojure, etc. Although, the dataflow in LabVIEW is actually more advanced, which is basically a multithreaded and branched version of the single threaded linear pipelines found in the languages I mentioned. Working in F# and Elixir with pipes can be pretty irritating sometimes given my knowledge of LabVIEW.
That’s true to a degree, but I think the discrepancy is still there. Experts in LabVIEW build up key commands to do things for them, and there’s even an ability to custom implement these using LabVIEW itself. Also, for some reason other fields seem to love these visual tools like TouchDesigner and Grasshopper. Programmers can be quite myopic when it comes to their programming preferences and perspectives.
For tools that require precise placing of wires like LabVIEW and vvvv, mouse strain and also the time it takes is indeed an issue. I am personally interested in sane but beautiful automatic layouts for visual languages, but this is most certainly a difficult problem. For example, if I could make an algorithm that basically makes LabVIEW (or a similar language) reorganize diagrams the way that I do, that would be great.
I think one inspiration are editing bays that professional video editors and also audio professional use. They have heavy use of software intermixed with hardware interfaces to perform their jobs.
> Also, for some reason other fields seem to love these visual tools like TouchDesigner and Grasshopper.
It's very simple - if you work with artists and you make them a presentation of your software, and at any moment there is some amount of code displayed on the screen, some of them will get up and leave the presentation.
I worked on my software with someone who would literally refuse to use the keyboard - everything had to be done by mouse, and I know a fair amount of like-minded people in th e same field.
> It's very simple - if you work with artists and you make them a presentation of your software, and at any moment there is some amount of code displayed on the screen, some of them will get up and leave the presentation.
Surprisingly, this is also the case for mechanical engineers. The proportion might be smaller, but they still exist. I know people in freshman classes that are retook the intro to programming class which was stuff like variables, conditionals and loops in Matlab.
Then there's people who did OK in their Fortran classes in the 1990s but never touched a line of cosey since.
The people have a good chance of being intimidated by code. This is where LabVIEW and Simulink come in.
That's probably true once once hits the workforce, but I'm pretty sure I gamed 40+ hours/week back in school. That starcraft ladder doesn't climb itself...
You may know of it already and it may not solve all of or any of your problems because I don't have experience with Labview outside of Eng 101 in college, but using something like Flow in elixir may make your experience with pipes less frustrating from the description you gave here. Flow is basically a GenStage backed data processing library that allows you to build out some parallel pipelines with various timings, triggers and stages.
Thanks for the mention of Flow. I have heard of it but haven’t used it. I am admittedly still getting my feet wet with Elixir, so I’m still learning the ecosystem, which has a pretty steep learning curve. The language itself is quite nice and the tooling is pleasant. You just “mix <this>” and “mix <that>” to working programs.
Part of my issue is the reverse shock of what text based programmers experience when moving to a visual language since I’m coming to Elixir from LabVIEW. I missing seeing certain things but am becoming accustomed to it. I’ve done text based programming a decent amount but never day to day on large code bases (and I’m new to Elixir having primarily used F# and Racket for projects and some Python for work against my will).
> As an aside, it is curious to me that many programmers are heavy PC gamers as the one true way to game and don’t complain about the mouse use there but will throw out all sorts of complaints when using it to program
Are those definitely the same people? Mouse use is certainly one of the reasons I gave up gaming. (The other being that an X hour working day is quite enough time to spend in front of a computer already).
Maybe not. It wasn’t a well thought out aside and is maybe fraught with bias. Haha.
I don’t mind visual programming, but I don’t enjoy PC gaming at all. There have been times I’ve experienced fatigue in my mouse hand with consecutive heavy days of LabVIEW programming. I think there are definitely ways to improve it though.
> But I can indeed see an argument for strain when using the mouse so much. I have experienced that before and improved my ergonomic setup. As an aside, it is curious to me that many programmers are heavy PC gamers as the one true way to game and don’t complain about the mouse use there but will throw out all sorts of complaints when using it to program or use tools.
I wonder also how using visual languages like LabVIEW compares to other mouse-heavy computer users like photo editors/graphic designers, CAD users etc.
> Dataflow is attractive to advanced users as well
I quite enjoyed using Max/MSP a few years ago when I did some stuff with it for a few months. I've always wished I had it available as a general purpose language in my programming toolbox since. As for other advanced users who use (visual) dataflow, my go to example is Ansys' SCADE Suite[1][2][3], which is used for critical systems like trains, helicopters and power plants.
>>> I’m not sure that a few months is enough to understand the true pros and cons of a language.
No doubt. My problems were mainly with the ergonomics of the editing environment, not the language. And I had problems with other graphical software as well, including games.
Yea, that’s fair. I only meant to point out that the LabVIEW paradigm is powerful once you really get into it.
I can totally understand the ergonomics issue. I think that automatic layout is the way to go to reduce pixel-perfect manual layout and thus strain from heavy mouse usage.
There are certainly many many instances where visual programming doesn't do the job. One instance where I found it incredibly useful, though, was with Node-RED. I used it to build a hobby-grade process automation solution, including a number of software PID control loops, that ran current limiters, valves and pumps, several load cells and half a dozen temperature sensors. All of this was orchestrated from Node-RED running on a Raspberry Pi communicating with the Tasmota esp8266 sensor/controller nodes over MQTT and I built it from scratch, learning along the way, over the course of 2-3 days.
Something about the visual pipelines and processing nodes really helped with handling concurrency, priority and maintaining tidy boundaries between systems. It was the first time I really had fun programming something in quite some time.
One thing missing here and in most of the visual programming discussions since it sits in a somewhat adjacent SCADA industry is Ladder Logic[1].
It was pretty illuminating how many parallels ladder has with the visual and scripting solutions I'd worked with in GameDev. Hot-reload under running processes, visual debugging very reminiscent of node based visual editors. I ended up automating our greenhouse with a P100[2] from Automation Direct and was pretty impressed with how straightforward it was to use.
Structured text is making in-roads in some places from what I can tell but ladder still seems to dominate because it's easily understood and maps well to a "relay based" mental model. It seems to be very sticky, at least more than most visual scripting languages I'm familiar with.
Ladder logic is powerful stuff. I learned it at Westinghouse in the 1980s. It's a non-Von Neuman based programming methodology well suited to real time environments. Every ladder gets executed once per cycle, and there are usually a few hundred cycles per second.
Special functions are implemented in boxes, like delays, counters, etc.
My task back then was to feed the entire I/O state into a ring buffer, and when a "crash" of the steel mill happened, a host PC could then download the entire buffer, giving access to information about what happened in the seconds BEFORE the crash.
It would be fairly easy to port to the Arduino, if someone hasn't done it already. Yep - it's been done https://cq.cx/ladder.pl
Because Ladders are text based, you could use Teco/EDT/Vi/Emacs/WordStar macros on them.
Yeah, it's a pretty neat ecosystem. PLCs are kinda like the Legos of automation, incredibly easy to mix and match with a fair number of (insecure) standard protocols.
I briefly toyed with the idea of building out some open source hardware/software but I didn't really seem like there was enough of a userbase to make it viable. Anyone using it professionally wants support and 10+ years of parts, most maker/hobby projects want direct access to code.
Still some pretty neat concepts for putting together highly customized processes with extensible and interchangeable hardware.
You don't understand how industry works. If it works, you don't replace it, you get it fixed when it breaks. Nobody is going to willingly risk their factory, plant, etc. on some new supposedly better hardware. They'll keep what works going for 100+ years if they can.
It is totally logical for them to do so. The "security" threat can be band-aided over with a new firewall at the internet access. Replacing any component of the control system introduces an element of risk. Downtime is horrendously expensive, and can ruin a business. Only a fool would do anything to introduce the risk of downtime in the name of "efficiency".
I use ladder for Boolean logic but transfer functions and block diagrams and numerical flow is better visualized with function block diagrams. It helps one gain an intuitive understanding of the effect of each term in a transfer function as you watch it operate in a way that structured text doesn’t offer.
Indeed, it's still very much alive in European industrial automation scene.
The ladder logic lends itself well for majority of industrial automation problems (e.g. turn on the motor until the part triggers a sensor, then activate a pneumatic cylinder to push it sideways). The control flow can be easily followed for debugging.
Since it's also easily interfaced with structured text modules, one can write procedural code for problems that are more easily expressed in procedural code (in my experience: statemachines, network communication, loops), but still use ladder logic that's more understandable by other people who may be maintaining the machine down the roead.
I believe that diagrams are at their most useful when they are deployed as a communications tool. When discussing our software, we commonly draw a diagram to help illustrate how functionality is divided up among components, and to show which components are impacted by particular functional chains. They are a powerful tool for communicating and coordinating the work of multiple people or multiple teams. If our software does not provide us with diagrams itself, then we will commonly create them ourselves in our technical documentation or on whiteboards as an aide to collaboration.
However, as we descend in our system towards finer and more detailed levels of granularity, two things happen. First of all, fewer people are impacted by very ‘local’ design decisions, so the importance of communication diminishes. Secondly, the frequency of change increases, so the importance of mutability and ease-of-editing increases.
The requirements that we place on our engineering tools depend largely on what we are trying to do with them. Communicating and coordinating work across a large engineering organization places fundamentally different requirements on our engineering tools than does, say, working alone to solve a challenging problem that requires the creation of new concepts and abstractions.
There are many different ways to work with software, and although each share much of the same flavor (dealing with informational complexity), the nature of these challenges is often distinct enough that the different requirements may lead to vastly different solutions.
For my part, I have a lot of faith in the general utility of tools which allow for the easy generation of diagrammatic representations of high-level structure, but which naturally decay to a more conventional textual representation at finer levels of granularity.
Yeah, to make sense of complex systems they need to be well structured and modularized, and visuals have a lot to contribute here. If you can't fit your structure comfortably in a diagram, your structure is too complex for most people to manage.
Further down the line you stop needing to know where you are (global view, understanding), and you only need to deal with implementation: actually doing what you need to do. Implementation can be often very complex on its own too, that's why algorithms are so scary at the beginning... and text is a better tool here. There's not much more structure to distill, and some parts are tricky, and you have edge cases and you often need to get your hands dirty. You need to keep all this complexity in mind at the same time anyway, and after you are done you can treat the implementation as a black box. So text is superior here.
That's how we manage complexity. Now we only need tools that can represent it better. But maybe it's not even visual programming languages, but rather visual system "structurers"? There are other parts of programming that could use better visual cues, and visual editors for visual stuff (colors, shapes, borders, etc.) will always be relevant, but those are all very different concerns.
It's always a good sign when a debate moves beyond 'x' is better than 'y' to a more nuanced discussions of the circumstances and conditions when 'x' provides more value, and the circumstances and conditions when the converse is true.
I wrote a spreadsheet based rules engine at Honeywell in around 2003 which enabled the business to maintain the business rules for all sales contracts that ran through the automation and control systems division. This gave them the ability to define forms, subsets of forms (legal or otherwise), notifications and approvals. I had its own natural language based expression language which the super users and business analysts could easily understand.
The spreadsheets, were change controlled by the business an imported into the system when they were approved. I considered this design a form of a visual language and the real beauty was that it did not require any re-coding of business rules, which did change quite frequently. The developers did not have to be re-writing any business logic as the rules changed.
The back end was Java and TCL based built on a graph database. Not super fast but super easy to understand and train up new developers.
It was quite successful and ran all sales contracts for for several years with billions of dollars of contracts passing through per year. I lost contact with the teams so I'm not sure how long it survived but I know that SAP charged millions of dollars trying to build a solution in their systems but could not get one working at a reasonable price.
I'd love to hear how you integrated a Java and TCL backend with spreadsheets. I'm assuming this isn't something you can do with excel?
(Logistics at the small business I work at, is run on a mess of excel spreadsheets with complicated formulas and occasionally VBA for some email functionality. I have a feeling we're doing things horribly wrong, but I don't know of any better way to do things with our zero IT budget.)
I did not leverage any VBA or VB. I learned to code in VBA and written way too much code in that space and was well aware of the limitations.
I'm pretty sure the way it went was, on check-in of a new version, I had a Java program that read the spreadsheet validated that all the right sheets were there and that the columns on each sheet conformed with the spec and also validated rows of data. The sheets were translated on each front end node for easy access.
Regarding the expression engine, the the expressions themselves were then translated into TCL boolean expressions. The attributes for the related objects where already loaded into memory into dictionaries and then the expressions were just evaluated to get a boolean result.
Neat! Seems like what's old is new again. I'm up to the same shenanigans (sans java, TCL and a graph database). spreadsheets are a great medium to provide non-technical people access to adjusting production with nuanced precision.
Agreed. I was amazed to see how much coding time is wasted by people trying to do some data cleanup job when it can be done so much easier and safer with no coding by:
1) Exporting he data to a spreadsheet
2) Massaging the data in the spreadsheet
3) Use a general purpose program to convert the spreadsheet into scripts that can be run against the DB.
I actually wrote a VBA program that does this where you have a query with ~1, ~2, ~3 macros and you just paste in a table, click a button and it spits out scripts for you. I wrote it 20 years ago and still use it to this day.
I like the line "There’s a comforting familiarity in reading the same internet argument over and over again."
The problem I think with visual programming tools is people get stuck. I watched my dad's career die stuck in Visual Basic. It's because people get stuck in thinking the way those tools force you to think. Code in text has a magnificent universality.
I have to write scripts for Cisco UCCX (call center software) at work sometimes. Never heard of it before this job. I was horrified that uses a visual ‘drag-n-drop’ IDE to create the call handling scripts, no other supported method. It’s the only ‘visual’ coding I’ve ever used. And in the 2 years I’ve used it, I actually don’t hate it. As other posters have mentioned, a visual approach can be decent DSL.
Btw, you can run still run arbitrary Java (1.7) in UCCX scripts too, which is kinda cool. Used that to make email alerts in case of certain situations in the call queue
My feedback, based on my both long but limited LabVIEW experience
> IDEs for text-based languages normally have features like code folding and call hierarchies for moving between levels, but these conventions are less developed in node-based tools. This may be just because these tools are more niche and have had less development time, or it may genuinely be a more difficult problem for a 2D layout — I don’t know enough about the details to tell.
NI demoed a system to function zoom in/out feature at one point but never fully developed it.
LabVIEW does have a call hierarchy. Like with Doxygen, my problem with these is they show everything rather than finding ways to highlight important relationships (e.g. you don't want to show every array append).
> Input
I suffered from RSI when using LabVIEW. Maybe I hadn't mastered enough but there is a lot more than just dropping that needs to be improved, but the wiring.
> Formatting
I think LabVIEW suffers from giving people so much pixel-perfect control, it encourages people to waste time on it because its "so close".
At one point, they demoed dynamic formatting (reflows the diagram as you add/move nodes without any manual management). This really needs to be made the default.
> Version control and code review
LabVIEW NXG has a text-based GVI format. It is not easily reviewable, mergeable, etc because it has all of the graphical minutia in it.
My proposal would be
- Remove block diagram formatting (see above)
- Separate top-level hand-designed UI from VI auto-placed front panel
- Reduce reliance on icons
- Encourage naming wires
At that point, a text-based VI format would be simple enough to be manageable within existing tools.
I had to suffer through a semester of LabVIEW. It seems like it might be okay if you have high enough level blocks of functionality available that you're never hooking many together, but for what I was trying to do it was a massive pain (doable, but tedious and hard to organize and follow)
> At one point, they demoed dynamic formatting (reflows the diagram as you add/move nodes without any manual management). This really needs to be made the default.
Do you by any chance have a link to a video or article where this is shown or discussed? I’m a heavy user of LabVIEW, and I am also convinced in my own explorations of creating visual languages that dynamic automatic layout is the way to go, but it is a very hard problem.
For completeness, I feel compelled to mention GNU Radio. It is a node based visual programming tool. It has its quirks, but you can do fairly complex digital signal processing with it, and it will do I/O from your sound card, as well as RF stuff.
Years ago I bounced around from one visual tool to another, never mastering any of them but getting deep enough to see the flaws. Hypercard, Authorware, Director, Flash. They were all very approachable, but they were all incredibly flawed in the end. You'd spend most of your time jumping from one limitation to the next, trying to hack your way out of the spaghetti jungle.
I think the better approach is to use conventional languages with UI tools layered on top that at worst generate readable code. (Like I assume is often done in game development.) Source control and diffing are indispensable.
The lure of these tools is that they are approachable, which sometimes is exactly what you need to get started. It's just very rarely what you need to finish. So let these tools be used to get people interested in programming, or to give people an idea of how to accomplish a project. But more importantly let's make sure to continue taking steps to demystify coding. Not because we want everyone to be programmers, but because when people do choose to program we want them to be able to collaborate with the rest of the world, and when they leave we don't want to inherit some visual pasta dish.
I loved MacroMind (later Macromedia) (later Adobe) Director.
Delivered dozens of educational CD's, corporate CD's and many of my own 'experiments'.
One of those was a map for EverQuest to help find dead bodies (it was a big thing at the time), and help swimmers who fell off a ship to get back to land alive.
Another was a project to scan every user-authored book in Ultima Online (using basic OCR)
Director was extensible in that you could not only import 3rd party libraries, but write your own modules in Pascal.
Much more powerful and extensible than Flash IMHO.
Flash won that battle after the buy-out, and Adobe extinguished Director.
Still miss it to create proof-of-concept, simple or amazing cross-platform apps.
We need both. And more. Visual projection and editing is valuable when looking at a zoomed-out "components and wires" sort of architectural view. It's also good for showing high-level overviews of a process, like various UML diagrams should have been. It can sometimes be used well for the nitty-gritty implementation details of a single unit, although this seems to be where a lot of it breaks down.
But it's terrible for doing all of these things at once. And also, if you can control your level of detail/zoom well enough to do it right, you'd get similar benefits projecting to simpler text formats as well.
What's most important is that all this stuff needs to be declarative. Not in the sense that you hide the order of operations within a block of code, in a way that final performance characteristics can become a surprise, but that the more complex structures of your project that organises the blocks into modules, classes, functions, records, etc need to be declarative, rather than an imperative set of instructions that build a structure at run-time (usually difficult or impossible to explore and instrument) which is later executed to achieve your actual goals.
167 comments
[ 4.2 ms ] story [ 291 ms ] threadOn my side I'd like to ask your opinion about Blueprint. It really turned me off back in the day and I always wonder why they couldn't optimize UnrealScript further instead of getting something totally new.
Yeah for sure eventually everyone get used to it and it becomes the new standard ("How can I NOT do Blueprint? It's so splendid" type of posts will show up) because UE is an extremely competetent engine.
But why did we bring it forward in the first place? Is there enough reason to do so aside from that some teams get to have their own marks in the world and some managers can grab more power?
In short: There are many game designers and artists who prefer visual scripting tools over traditional code.
https://forums.unrealengine.com/t/blueprints-are-a-toy-which...
BTW I do agree visual programming makes sense in certain area, but to use it for a general scripting language? Hell no...
https://www.youtube.com/watch?v=rYQQRIY_zcM
Speculation: visual programming is a bit more beginner friendly, and more compatible with the brains of people designing stuff in UE. I wish blueprint was around when I was 13, messing around in UT’s version of UE and having no idea what unreal script could do. Visual language can break out all possible components into UI menus, just like everything else in UE. Visual oriented people really click with tools like visual programming.
Personally I think it’s cynical / arbitrary to pin the decision to corporate power struggles. Sometimes people propose new things out of interest or as a side project and then it grows from there.
Whatever, it's already done and text scripting language is not coming back for UE.
* A lot of game engine functionality is extremely helpful but not discoverable. Visual scripting languages make the full spectrum of the engine's API more easily discovered
* Games are very stateful, and I find it is much easier to conceptualize this with blueprints than with code, which is better for functional stuff
* Collapsing nodes in blueprints is much more powerful as a documentation tool than collapsable lines of code. When you collapse nodes, you can show the execution running through long text high level descriptors of what hidden blocks are doing, and without needing to create a function for that task specifically.
e.g. code might do something like
def update_target():
target = find_new_target() # set target to nearest enemy unit
attack_pattern = choose_attack_pattern() # choose next attack pattern
execute_attack() # perform attack montage and execute effects
____________________
whereas the bp would be
Set next target to nearest enemy -> Choose next attack pattern -> Perform attack montage and execute effects
It becomes pseudocode that is bright and on the forefront of your attention, vs. most code editors which will highlight code and lowlight comments.
There's an unwinnable war between keeping things simple and being complex-enough to do useful things. Visual stuff is great for simple things, but simple things aren't very useful. When you really need to do more complex things, you reach a limit very quickly. It becomes pretty unmaintainable very quickly and then people will "graduate" from it and go to something more convenient.
My kid is learning Scratch right now, and it's taught him a lot of great stuff. But after about a year, he's ready to move onto Python. Scratch has taught him some very valuable concepts and he was able to jump into some Python concepts with ease (others still escape him). But you can only go so far with Scratch vs other programming languages.
The same goes for other visual tools. In the end, people graduate from the visual style because it ironically becomes too complex because they want to keep things simple, and then you've lost a customer.
This becomes a huge problem with your "pivot" for changing things is plumbing up and down, up and down, up and down the hierarchy. Instead of being able to flip the whole thing on its head and seeing all the settings/flags/options for whole classes & categories of similar things and changing them from that perspective.
It's like there's a transition point where you know enough about how things are all put together that being forced to interact with multiple things at once only through the top of their individual silos starts to fall apart pretty dramatically.
The reason I ask about Simulink, is that it's very common in the toolchain of our customers (I'm CEO of https://www.auxon.io) and we're building an integration to it for our product. For the most part it seems to do a much better job of being an IDE for more complicated models/programs compared to the bevy of RAD tools I've used in the past. I have certainly encountered limitations reminiscent of those old RAD ways, but I can also see how much better a job it seems to do before you hit those snags.
On the way toward building everything we needed for the analysis we wanted to do we realized we had a distributed tracing system suitable for continuous system lifecycles & embedded systems (as opposed to transactional lifecycles & IT systems) and a spec/query mechanism over what amounts to a logic model derived from system executions. That sort of thing tends to be valuable to folks who aren't as far along in their development or use case maturity to need bleeding out all the corner cases & "unknown unknowns" from their systems, so we exposed many of our building blocks as features themselves.
Somewhat ironically, given the thread this is all related to, everything we build today is for CLI consumption on Linux and Windows. We will build out a kind of IDE/Workbench UI this year to fulfill some of our vision around the category of CAE tools we're angling toward, and to access additional kinds of customers, but predominantly folks have preferred CLI shaped tooling because it's easier for them to bake our continuous verification & validation capabilities into their existing processes when we don't force them into a siloed GUI.
A friend has built some pretty amazing stuff in Scratch, including a BBC Micro emulator (runs 60+ games), a few arcade game ports, and lots more [1].
As a developer myself, I find it quite surprising how far one can go with Scratch. (FWIW I've never coded with it myself)
[1] https://scratch.mit.edu/users/RokCoder/
IMO, for these kinds of projects, I suspect Scratch doesn't offer many advantages (if any at all). He likes a challenge :)
— More details, including an interview, can be found here:
https://www.coderkids.com/blog/who-is-rokcoder
He's pretty responsive to comments on his Scratch homepage — and can also be found under the same name on Twitter/FB/etc. He's always been happy to help others. Give him a shout if you have any specific questions.
(FWIW: I worked with him — Cliff — for a short while, back in the 90s)
Complex minecraft works similarly, instead of working in redstone they make the logic gates and then the bit adders in redstone and then used programming as normal on those constructs.
But all of his other games - including the clones of arcade games - aren't emulators nor interpreters. They're just games, written from scratch (pun unintended), from the ground up, in pure Scratch.
After all, his day job is in fact games dev - and with 25+ years experience (I worked with him mid-90s), one could say he's a veteran in that field.
In reality, it depends on the what the game actually is, as to whether it is harder to implement than an emulator or an interpreter. e.g. I strongly suspect that Battleships was much easier to implement than the Beeb emulator was.
* abstraction
* version control
* test automation
Those are also (in my opinion) the three techniques which separate proper "software engineering" from coding/scripting, which is why visual programming tools are pretty much universally unsuited for complicated projects.
The one asterisk I would put on that is tools in which the visualization is based upon an actual, human readable, programming language. I did some work with XSLT back in the day using a visual programming tool, and it worked reasonably well.
I do wonder why some concept likes building chips doesn't make it into visual environments? I should be able to pass a chip with inputs and outputs to others.
Another technique they use for abstraction before that point include one-to-n on-page connectors and multi-signal buses when the visual wires get to be too cumbersome.
[1] https://thedailywtf.com/articles/Labview-Spaghetti
Also, there are certainly test frameworks for testing LabVIEW code. NI has a few but then JKI, a third party company, has multiple testing frameworks including Caraya, which is similar in philosophy to something like FsUnit for F# or Elixir’s ExUnit.
https://github.com/JKISoftware/Caraya
If a visual programming language stored its programs in a human readable or at least open format, then it would be possible for people to create those tools for it. But then, people would start writing and editing the program directly in its storage format, and it would cease to be visual programming.
An approach that worked for me was to provide an escape hatch (in my case, giving the user a way to run arbitrary Javascript on the page being automated) so that the built-in commands could be designed for the most common scenarios and users would still have a way to handle gnarly edge cases.
That being said, Alteryx is an example of a company that has made a ton of money via a data munging tool that is a visual programming language. The users are fanatical as well.
Also it is possible to blend the two approaches. For example by allowing users to script some of the boxes in a text based language. This is the approach I have taken with my own visual-based data wrangling tool. The standard transforms cover 95% of cases people need (re-order columns, pivot, filter etc) and you can script a Javascript transform for anything else.
https://news.ycombinator.com/item?id=27397375
Snap! is not simply all the usability and functionality of Logo, but also all the functionality and power of Scheme! Without any of the dumbing down of Scratch or Logo. Visual block programming. Turtle Graphics. Sprites. Lexical scoping. Lambda. Closures. Call/cc. Plus JavaScript integration and web stuff. With extensions for networking, AI, machine learning, speech synthesis and recognition, graph theory, robotics, Lego, Arduino, 3d graphics, 3d design, 3d fabrication, and 3d printing, embroidery, etc. ;)
https://dl.acm.org/doi/pdf/10.1145/3386329
History of Logo. Proc. ACM Program. Lang., Vol. 4, No. HOPL, Article 79. Publication date: June 2020.
6.2 Brian Harvey’s Personal Narrative on Snap!: Scheme Disguised as Scratch (pp. 49-50)
In 2009, the University of California, Berkeley, was one of several universities developing a new kind of introductory computer science course, meant for non-CS majors, to include aspects of the social implications of computing along with the programming content. Scratch wasn’t quite expressive enough to support such a course (it lacked the ability to write recursive functions), so Prof. Daniel Garcia and I thought "What’s the smallest change we could make to Scratch to make it usable in our course?" After 20 years teaching Structure and Interpretation of Computer Programs [Abelson et al. 1984], the best computer science text ever written, I knew that the answer to "what’s the smallest change" is generally "add lambda." I joined forces with German programmer Jens Mönig, who had developed BYOB (Build Your Own Blocks), an extension to Scratch with custom (user-defined) blocks, including reporters and predicates. At that time we were hoping to convince the Scratch Team to adopt our ideas, so we took "smallest change" very seriously. BYOB 3.0 [Harvey and Mönig 2010], with first class procedures and first class lists, added only eight blocks to Scratch’s palette. (The code is almost all Jens’s. My contribution was part of the user interface design, plus teaching Jens about lambda.) Version 3.1 added first class sprites with Object Logo-style inheritance. The Berkeley course, The Beauty and Joy of Computing (BJC) [Garcia et al. 2012], is also used by hundreds of high schools, especially since the College Board endorsed it as a curriculum for their new AP CS Principles exam. Unfortunately, some teachers have no sense of humor, and so BYOB version 4.0, a complete rewrite in JavaScript, was renamed Snap! [Harvey 2019]. [18]
Since Scratch seemed to be positioned as the successor to Logo, it was a goal for Snap! to restore the features from Logo that are missing in Scratch. The most important missing feature, the ability to define functions (and therefore to use recursive functions), is at the core of the new language. (Scratch introduced user-definable command blocks in version 2.0, but still doesn’t support user defined reporters.) Scratch had also replaced the structured text (word and sentence) functions with a flat text string data type. We wanted to be backward compatible with Scratch, so we implemented words and sentences as a library, defining first, last, butfirst, and so on. (Since block languages allow multi-word procedure names, and you don’t have to type the long name in order to use the procedure, the library names are, e.g., all but first letter of.)
Lists are first class and can be arbitrarily deep in sublists. The usual higher order functions on lists are provided; the graphical representation of lambda is built into the blocks representing higher order functions, and so beginning users can use higher order functions in simple cases without thinking hard about function-as-data at all, but th...
You could probably conceive of C as a visual programming language if you spent long enough time. All of your keywords can be reduced to blocks, forcing people to insert the appropriate parameters. And once you allow people to define their own blocks, you're like a good portion of the way there.
The problem is that after a while, it is simply faster to type. The breadth of options available becomes too much to manage from menus and drag and drop interfaces. Already you're probably typing in some values and names. So you're constantly switching between keyboard and mouse.
At some point, the benefit of perfect syntax doesn't outweigh the loss of productivity from not being able to immediately use any construct the language makes available to you.
It's definitely allowed non-programmers to make complex AI/game state for their creations, without having to deal with syntax errors or other issues. We choose Scratch/Blocky since this would be used for more managing states/logic, rather than flowing data (where node based makes more sense)
Shameless plug of my take on rethinking visual programming couple of years ago: https://divan.dev/posts/visual_programming_go/
That's exactly how I feel when reading code. Recently, I explained the problem to a dear colleague when working on a code base (with about 600K lines of code). I compared the situation to working in a large office building with all lights turned off and you have to skim through thousands of documents with a flashlight.
It's comforting to know that other people feel the same way.
And there are a lot of them.
Diff, interactive debugging, keyboard-accelerated inputs and shortcuts, search, version control... The list goes on.
If you are programming a singular thing, this probably has good legs. As soon as you are programming many things, though?
This is why cad systems can be seen as visual programming of parts, but are never considered for this debate.
There is an impressive list of things your enter a race to parity with.
I sort of disagree with the intro to LabVIEW:
> There are a large number of visual programming tools that are roughly in the paradigm of ‘boxes with some arrows between them’, like the LabVIEW example above. I think the technical term for these is ‘node-based’, so that’s what I’ll call them.
There are NOT a "large number" of visual programming tools that are roughly in the paradigm [...] like LabVIEW. I am sure there are some but, really, how many active ones?
Also, "node-based" is not the main concept. The most essential concept of LabVIEW is known as "data-flow" (https://labviewwiki.org/wiki/Data_flow). Data generated by a node "flows" to the other nodes it is connected to and when any node has data on all it's inputs, it executes and produces data which flows to whatever nodes it's connected to. This happens asynchronously. You get concurrency "out of the box" with LabVIEW. That's extremely powerful and much easier to reason about, IMHO, than the concurrency constructs in virtually any mainstream computer language.
I think there's plenty of room for all kinds of languages (both text and visual) as DSL's. If they're well-designed and intuitive, they're like a hot-knife through butter for a domain expert. The nice thing about a really good DSL is that you use it more like a library, whereas with a general purpose language you would be reaching for a framework. It gives you a lot of creative latitude and it feels right. The problems start when people start using them as golden hammers, or use them for things which they're not good at.
They list half a dozen or so. Most of the node based systems are also flow oriented, aka nodes are functional and stateless which allows them to be mapped across any amount of input.
It's a neat pattern but it's really not as uncommon as you think. The mainstream shader languages work this way, for example.
I think you just described flow programming. There are at least 10+ visual languages / environments that work like this.
Even the single application Blender itself has several different visual node based programming systems (most but not all built on top of the same framework, and applied to different kinds of programming), for shader programming, CPU image processing, video compositing, lighting, animation and constraints, particle systems, physics simulations, procedural mesh generation, and the Blender Foundation and third parties are developing even more, like procedural city generation:
Procedural city generation:
https://www.youtube.com/watch?v=jb_jwsyfQc4&ab_channel=derbe...
Everything nodes:
https://code.blender.org/2020/12/everything-nodes-and-the-sc...
Nodes Workshop - 22 - 25 June 2021
https://devtalk.blender.org/t/nodes-workshop-22-25-june-2021...
>“It is like compositor but for physics”.
The complete beginners guide to Blender nodes, Eevee, Cycles and PBR:
https://artisticrender.com/the-complete-beginners-guide-to-b...
>Blender has a few nodes systems. The first and obvious one is Blenders shading system for Cycles and Eevee. This is the node system that we will focus on in this article.
>But we also have nodes for compositing, lighting and textures, even if the use case and future for texture nodes are uncertain at this point.
>We can also extend Blender with other node systems through add-ons. The most well-known is probably animation nodes that come bundled with Blender.
>There are also other node systems available for Blender. AMD ProRender for example, a third-party render engine that has its own shader node system. Another example is Luxrender. There is also mTree for generating trees with nodes and Sverchok that can manipulate all kinds of data with nodes.
Reposting this from a few years ago:
https://news.ycombinator.com/item?id=18496880
There's so much interesting prior work! I really enjoyed this paper “A Taxonomy of Simulation Software: A work in progress” from Learning Technology Review by Kurt Schmucker at Apple. It covered many of my favorite systems.
http://donhopkins.com/home/documents/taxonomy.pdf
It reminds me of the much more modern an comprehensive "Gadget Background Survey" that Chaim Gingold did at HARC, which includes Alan Kay's favorites, Rockey’s Boots and Robot Odyssey, and Chaim's amazing SimCity Reverse Diagrams and lots of great stuff I’d never seen before:
http://chaim.io/download/Gingold%20(2017)%20Gadget%20(1)%20S...
I've also been greatly inspired by the systems described in the classic books “Visual Programming” by Nan C Shu, and “Watch What I Do: Programming by Demonstration” edited by Alan Cypher.
https://archive.org/details/visualprogrammin00shu_2pf
https://archive.org/details/watchwhatido00alle
Brad Myers wrote several ar...
The article quoted one of my earlier posts on that subject, but I've written more about spreadsheets as visual programming, and cited Brad Myer's work and articles about visual programming languages, and there's more in the HN discussion of his classic 1989 paper "Visual Programming, Programming by Example, and Program Visualization; A Taxonomy," Proceedings SIGCHI '86: Human Factors in Computing Systems. Boston, MA. April 13-17, 1986. pp. 59-66:
https://news.ycombinator.com/item?id=26645253
DonHopkins 3 months ago | on: Spreadsheet is a software development paradigm
Spreadsheet certainly are visual programming languages: by any measure, by far one of the most common most widely used types of visual programming languages in the world.
HN discussion:
https://news.ycombinator.com/item?id=26057530
Taxonomies of Visual Programming (1990) [pdf] (cmu.edu)
https://www.cs.cmu.edu/~bam/papers/VLtax2-jvlc-1990.pdf
https://news.ycombinator.com/item?id=26061576
Brad Myers' paper answers the age-old argument about whether or not spreadsheets are visual programming languages!
https://news.ycombinator.com/item?id=20425821
>DonHopkins on July 13, 2019 | on: I was wrong about spreadsheets (2017)
>Google sheets (and other google docs) can be programmed in "serverless" JavaScript that runs in the cloud somewhere. It's hellishly slow making sheets API calls, though. Feels like some kind of remote procedure call. (Slower than driving Excel via OLE Automation even, and that's saying something!) Then it times out on a wall clock (not cpu time) limit, and breaks if you take too long.
>A CS grad student friend of mine was in a programming language class, and the instructor was lecturing about visual programming languages, and claimed that there weren't any widely used visual programming languages. (This was in the late 80's, but some people are still under the same impression.)
>He raised his hand and pointed out that spreadsheets qualified as visual programming languages, and were pretty darn common.
>They're quite visual and popular because of their 2D spatial nature, relative and absolute 2D addressing modes, declarative functions and constraints, visual presentation of live directly manipulatable data, fonts, text attributes, background and foreground colors, lines, patterns, etc. Some even support procedural scripting languages whose statements are written in columns of cells.
>Maybe "real programmers" would have accepted spreadsheets more readily had Lotus named their product "Lotus 012"? (But then normal people would have hated it!)
I Was Wrong About Spreadsheets And I'm Sorry:
https://www.reifyworks.com/writing/2017-01-25-i-was-wrong-ab...
HN Discussion:
https://news.ycombinator.com/item?id=26668885
Excerpt from "Taxonomies of Visual Programming and Program Visualization&qu...
I was never interested in or good at coding, but LabVIEW made sense to me. Now I'm using programming principles I learned in LabVIEW and re-implementing them in C++ and C#. It's been a journey.
> Most fields have a problem with ‘ghost knowledge’, hard-won practical understanding that is mostly passed on verbally between practitioners and not written down anywhere public. At least in programming some chunk of it makes it into forum posts.
I'd be incredibly interested in seeing a series of posts like this, where authors mine hacker news and other forums for insights that they consolidate (then repost here).
What other topics do y'all think HN provides unique insight on?
As for what HN could be mined for, a big topic is software languages. There are ways to subdivide that: by language, by optimization aspect (e.g., comparative performance). Geographic topics like cost of living and quality of life. Remote work pros and cons.
Another way of identifying topics worth mining would be identify posts where @dang provides a link to previous posts and those previous posts have a sufficient number of comments.
Career managment I think (not sure if it's the good term for it). I see lots of people in their 40's or more that quit some FANG job for another job that earns less but is more aligned with how they want to live their life. I see lots of people advocating to quit jobs as soon as you see red flags. I think both of these are good wisdom.
On the other hand, a point that's still a mystery to me is "programming language choice". I've learned useful heuristics like you can do web backend in pretty much anything, if you have no libraries for something that needs them like an AWS SDK you're going to suffer, what helps in the long run slows you down at first, but that's about it. I don't know if it's because I'm too young in this field, because I don't understand/missed something, or because there is no big truth to find.
For instance, I program in Python if speed is irrelevant. Rust if it is very relevant. C# if I want a large project where performance is somewhat important.
I think there are a wide range of languages for each "category" where I put Python, C# and Rust above, but those 3 are my current favorites.
Certain languages are very well suited to specific tasks. You learn these with experience. Initially I loved python because it was a joy to slice and dice strings in it. I understand that Ruby is popular for the same reason. Closure is great if you want lots of agents processing stuff on a network stack.
If I were you, I'd just focus on making fun projects in your favorite language.
That's one dimension, but what about popularity, maintainability, being able to hire, productivity, being able to tackle precise problems, etc.
Edit: I didn’t read the comment right. Productivity and being able to solve precise problems are essentially why you’re choosing a technology in the first place?
I don't think that's true, I believe most programmers are either dark matter developers [1] or people doing Java (or equivalent) all their life for a software service company. There is nothing wrong with that, but that would explain why less popular languages and ecosystem still have lots of stuff. For example, Rust has ~60k crates on crates.io, while Maven Central has ~400k packages. I believe Java is used by more than 10 times the number of people that use Rust. But the people writing the libraries in the first place are usually the people that are open to trying out new languages.
[1]: https://www.hanselman.com/blog/dark-matter-developers-the-un...
When putting a language into "your toolbox" you indeed SHOULD focus on "popularity, maintainability, being able to hire."
However, when picking which of the languages you have learnt to apply to a given problem you need to know their relative strengths.
Most of the strengths you mention help you pick the "best in class" language. But only the problem can tell you which subset of langauges you are picking from. Sometimes the best possible tool for your problem is going to be unpopular.
I'm pretty young in the field as well, but my impression so far is that there aren't really any big truths, just a lot of little ones
E.g. Productive person-hour scaling as a function of team headcount
I used LabVIEW extensively for a few months, and ended up with crippling wrist fatigue and eyestrain headaches from all of the fine mouse work and menu selections needed to write even small programs. This is an ergonomic problem with all graphical software, but is particularly acute with programming because of the amount of code that gets written.
Another problem is innovation. You can quickly spin up an experimental language and try it out because the infrastructure -- editor and command shell -- already exist. This is one reason why we enjoy such a proliferation of new language ideas. New ideas in graphical software require intensive effort to try out.
On the other hand, there does seem to be something about data flow programming (LabVIEW, Excel) that is attractive to beginners, that can't be overlooked. Excel is probably the number-one programming tool in use right now.
I’m not sure that a few months is enough to understand the true pros and cons of a language. I’m using Elixir in my day job now and have used F# for side projects, and it’s not clear to me any one is faster to program in, especially for GUIs. LabVIEW actually shines, contrary to popular but inexperienced opinion, for large projects. I have used LabVIEW rather extensively, and it seems any project would have seen massive delays if I used something else, which is why I didn’t.
But I can indeed see an argument for strain when using the mouse so much. I have experienced that before and improved my ergonomic setup. As an aside, it is curious to me that many programmers are heavy PC gamers as the one true way to game and don’t complain about the mouse use there but will throw out all sorts of complaints when using it to program or use tools.
> New ideas in graphical software require software intensive effort to try out.
This “problem” isn’t inherent to graphical programming though. It’s because legions of programmers are convinced that text=programming and have spent decades writing text-only tools. You see this friction when using LabVIEW with GitHub, whose tooling is incompatible with the diff and merge tools that come with LabVIEW, which makes GitHub look outdated and not LabVIEW.
I’m actually working on ideas to build a toolkit for graphical programming languages. There are certainly challenges though, such as the desktop GUI ecosystem being a mess and reliance upon graph data structures and algorithms (like automatic layout).
> On the other hand, there does seem to be something about data flow programming (LabVIEW, Excel) that is attractive to beginners
Dataflow is attractive to advanced users as well, which is part of the draw to languages like F#, Elixir, Racket, Clojure, etc. Although, the dataflow in LabVIEW is actually more advanced, which is basically a multithreaded and branched version of the single threaded linear pipelines found in the languages I mentioned. Working in F# and Elixir with pipes can be pretty irritating sometimes given my knowledge of LabVIEW.
For tools that require precise placing of wires like LabVIEW and vvvv, mouse strain and also the time it takes is indeed an issue. I am personally interested in sane but beautiful automatic layouts for visual languages, but this is most certainly a difficult problem. For example, if I could make an algorithm that basically makes LabVIEW (or a similar language) reorganize diagrams the way that I do, that would be great.
I think one inspiration are editing bays that professional video editors and also audio professional use. They have heavy use of software intermixed with hardware interfaces to perform their jobs.
It's very simple - if you work with artists and you make them a presentation of your software, and at any moment there is some amount of code displayed on the screen, some of them will get up and leave the presentation.
I worked on my software with someone who would literally refuse to use the keyboard - everything had to be done by mouse, and I know a fair amount of like-minded people in th e same field.
Surprisingly, this is also the case for mechanical engineers. The proportion might be smaller, but they still exist. I know people in freshman classes that are retook the intro to programming class which was stuff like variables, conditionals and loops in Matlab.
Then there's people who did OK in their Fortran classes in the 1990s but never touched a line of cosey since.
The people have a good chance of being intimidated by code. This is where LabVIEW and Simulink come in.
Part of my issue is the reverse shock of what text based programmers experience when moving to a visual language since I’m coming to Elixir from LabVIEW. I missing seeing certain things but am becoming accustomed to it. I’ve done text based programming a decent amount but never day to day on large code bases (and I’m new to Elixir having primarily used F# and Racket for projects and some Python for work against my will).
Are those definitely the same people? Mouse use is certainly one of the reasons I gave up gaming. (The other being that an X hour working day is quite enough time to spend in front of a computer already).
I don’t mind visual programming, but I don’t enjoy PC gaming at all. There have been times I’ve experienced fatigue in my mouse hand with consecutive heavy days of LabVIEW programming. I think there are definitely ways to improve it though.
I wonder also how using visual languages like LabVIEW compares to other mouse-heavy computer users like photo editors/graphic designers, CAD users etc.
> Dataflow is attractive to advanced users as well
I quite enjoyed using Max/MSP a few years ago when I did some stuff with it for a few months. I've always wished I had it available as a general purpose language in my programming toolbox since. As for other advanced users who use (visual) dataflow, my go to example is Ansys' SCADE Suite[1][2][3], which is used for critical systems like trains, helicopters and power plants.
[1] https://www.ansys.com/products/embedded-software/ansys-scade...
[2] https://www.ansys.com/content/dam/product/embedded-software/...
[3] https://www.youtube.com/watch?v=5z0h-WeScqw&list=PL0lZXwHtV6... (the third video introduces the visual language)
No doubt. My problems were mainly with the ergonomics of the editing environment, not the language. And I had problems with other graphical software as well, including games.
I can totally understand the ergonomics issue. I think that automatic layout is the way to go to reduce pixel-perfect manual layout and thus strain from heavy mouse usage.
Has anyone tried yet, to build a compiler, that compiles code into visual programming structures - and back?
I know scratch does this in a very limited way(code to scratch), but are there other tools around?
I am not aware of any, except for mine (which is unpublished as of yet, but if I get the website right, that might change by this weekend).
Back in 1991 I worked on Labview at a university lab. It seemed amazing for the time.
For a teenager migrating from Basic - Labview seemed a fantastic improvement.
It felt incredibly powerful to draw boxes and wires to make multiple scientific instruments gather data.
I do remember spending a lot of time on alignment, plus there was no code revision at the time...
However, the lab moved to custom C based workflow. It was great on a personal level - writing low level drivers etc.
Part of the reasoning was that at some point the scientists needed a custom UI and it was actually easier to create one with C than with Labview.
Something about the visual pipelines and processing nodes really helped with handling concurrency, priority and maintaining tidy boundaries between systems. It was the first time I really had fun programming something in quite some time.
It was pretty illuminating how many parallels ladder has with the visual and scripting solutions I'd worked with in GameDev. Hot-reload under running processes, visual debugging very reminiscent of node based visual editors. I ended up automating our greenhouse with a P100[2] from Automation Direct and was pretty impressed with how straightforward it was to use.
Structured text is making in-roads in some places from what I can tell but ladder still seems to dominate because it's easily understood and maps well to a "relay based" mental model. It seems to be very sticky, at least more than most visual scripting languages I'm familiar with.
[1] https://en.wikipedia.org/wiki/Ladder_logic
[2] https://www.automationdirect.com/adc/shopping/catalog/progra...
Special functions are implemented in boxes, like delays, counters, etc.
My task back then was to feed the entire I/O state into a ring buffer, and when a "crash" of the steel mill happened, a host PC could then download the entire buffer, giving access to information about what happened in the seconds BEFORE the crash.
It would be fairly easy to port to the Arduino, if someone hasn't done it already. Yep - it's been done https://cq.cx/ladder.pl
Because Ladders are text based, you could use Teco/EDT/Vi/Emacs/WordStar macros on them.
I briefly toyed with the idea of building out some open source hardware/software but I didn't really seem like there was enough of a userbase to make it viable. Anyone using it professionally wants support and 10+ years of parts, most maker/hobby projects want direct access to code.
Still some pretty neat concepts for putting together highly customized processes with extensible and interchangeable hardware.
if it supports the old protocol but secure it should be transparent to them
It is totally logical for them to do so. The "security" threat can be band-aided over with a new firewall at the internet access. Replacing any component of the control system introduces an element of risk. Downtime is horrendously expensive, and can ruin a business. Only a fool would do anything to introduce the risk of downtime in the name of "efficiency".
The ladder logic lends itself well for majority of industrial automation problems (e.g. turn on the motor until the part triggers a sensor, then activate a pneumatic cylinder to push it sideways). The control flow can be easily followed for debugging.
Since it's also easily interfaced with structured text modules, one can write procedural code for problems that are more easily expressed in procedural code (in my experience: statemachines, network communication, loops), but still use ladder logic that's more understandable by other people who may be maintaining the machine down the roead.
However, as we descend in our system towards finer and more detailed levels of granularity, two things happen. First of all, fewer people are impacted by very ‘local’ design decisions, so the importance of communication diminishes. Secondly, the frequency of change increases, so the importance of mutability and ease-of-editing increases.
The requirements that we place on our engineering tools depend largely on what we are trying to do with them. Communicating and coordinating work across a large engineering organization places fundamentally different requirements on our engineering tools than does, say, working alone to solve a challenging problem that requires the creation of new concepts and abstractions.
There are many different ways to work with software, and although each share much of the same flavor (dealing with informational complexity), the nature of these challenges is often distinct enough that the different requirements may lead to vastly different solutions.
For my part, I have a lot of faith in the general utility of tools which allow for the easy generation of diagrammatic representations of high-level structure, but which naturally decay to a more conventional textual representation at finer levels of granularity.
Further down the line you stop needing to know where you are (global view, understanding), and you only need to deal with implementation: actually doing what you need to do. Implementation can be often very complex on its own too, that's why algorithms are so scary at the beginning... and text is a better tool here. There's not much more structure to distill, and some parts are tricky, and you have edge cases and you often need to get your hands dirty. You need to keep all this complexity in mind at the same time anyway, and after you are done you can treat the implementation as a black box. So text is superior here.
That's how we manage complexity. Now we only need tools that can represent it better. But maybe it's not even visual programming languages, but rather visual system "structurers"? There are other parts of programming that could use better visual cues, and visual editors for visual stuff (colors, shapes, borders, etc.) will always be relevant, but those are all very different concerns.
The spreadsheets, were change controlled by the business an imported into the system when they were approved. I considered this design a form of a visual language and the real beauty was that it did not require any re-coding of business rules, which did change quite frequently. The developers did not have to be re-writing any business logic as the rules changed.
The back end was Java and TCL based built on a graph database. Not super fast but super easy to understand and train up new developers.
It was quite successful and ran all sales contracts for for several years with billions of dollars of contracts passing through per year. I lost contact with the teams so I'm not sure how long it survived but I know that SAP charged millions of dollars trying to build a solution in their systems but could not get one working at a reasonable price.
(Logistics at the small business I work at, is run on a mess of excel spreadsheets with complicated formulas and occasionally VBA for some email functionality. I have a feeling we're doing things horribly wrong, but I don't know of any better way to do things with our zero IT budget.)
I'm pretty sure the way it went was, on check-in of a new version, I had a Java program that read the spreadsheet validated that all the right sheets were there and that the columns on each sheet conformed with the spec and also validated rows of data. The sheets were translated on each front end node for easy access.
Regarding the expression engine, the the expressions themselves were then translated into TCL boolean expressions. The attributes for the related objects where already loaded into memory into dictionaries and then the expressions were just evaluated to get a boolean result.
I actually wrote a VBA program that does this where you have a query with ~1, ~2, ~3 macros and you just paste in a table, click a button and it spits out scripts for you. I wrote it 20 years ago and still use it to this day.
The problem I think with visual programming tools is people get stuck. I watched my dad's career die stuck in Visual Basic. It's because people get stuck in thinking the way those tools force you to think. Code in text has a magnificent universality.
Btw, you can run still run arbitrary Java (1.7) in UCCX scripts too, which is kinda cool. Used that to make email alerts in case of certain situations in the call queue
> IDEs for text-based languages normally have features like code folding and call hierarchies for moving between levels, but these conventions are less developed in node-based tools. This may be just because these tools are more niche and have had less development time, or it may genuinely be a more difficult problem for a 2D layout — I don’t know enough about the details to tell.
NI demoed a system to function zoom in/out feature at one point but never fully developed it.
LabVIEW does have a call hierarchy. Like with Doxygen, my problem with these is they show everything rather than finding ways to highlight important relationships (e.g. you don't want to show every array append).
> Input
I suffered from RSI when using LabVIEW. Maybe I hadn't mastered enough but there is a lot more than just dropping that needs to be improved, but the wiring.
> Formatting
I think LabVIEW suffers from giving people so much pixel-perfect control, it encourages people to waste time on it because its "so close".
At one point, they demoed dynamic formatting (reflows the diagram as you add/move nodes without any manual management). This really needs to be made the default.
> Version control and code review
LabVIEW NXG has a text-based GVI format. It is not easily reviewable, mergeable, etc because it has all of the graphical minutia in it.
My proposal would be
- Remove block diagram formatting (see above)
- Separate top-level hand-designed UI from VI auto-placed front panel
- Reduce reliance on icons
- Encourage naming wires
At that point, a text-based VI format would be simple enough to be manageable within existing tools.
Do you by any chance have a link to a video or article where this is shown or discussed? I’m a heavy user of LabVIEW, and I am also convinced in my own explorations of creating visual languages that dynamic automatic layout is the way to go, but it is a very hard problem.
I think the better approach is to use conventional languages with UI tools layered on top that at worst generate readable code. (Like I assume is often done in game development.) Source control and diffing are indispensable.
The lure of these tools is that they are approachable, which sometimes is exactly what you need to get started. It's just very rarely what you need to finish. So let these tools be used to get people interested in programming, or to give people an idea of how to accomplish a project. But more importantly let's make sure to continue taking steps to demystify coding. Not because we want everyone to be programmers, but because when people do choose to program we want them to be able to collaborate with the rest of the world, and when they leave we don't want to inherit some visual pasta dish.
One of those was a map for EverQuest to help find dead bodies (it was a big thing at the time), and help swimmers who fell off a ship to get back to land alive. Another was a project to scan every user-authored book in Ultima Online (using basic OCR)
Director was extensible in that you could not only import 3rd party libraries, but write your own modules in Pascal. Much more powerful and extensible than Flash IMHO.
Flash won that battle after the buy-out, and Adobe extinguished Director. Still miss it to create proof-of-concept, simple or amazing cross-platform apps.
But it's terrible for doing all of these things at once. And also, if you can control your level of detail/zoom well enough to do it right, you'd get similar benefits projecting to simpler text formats as well.
What's most important is that all this stuff needs to be declarative. Not in the sense that you hide the order of operations within a block of code, in a way that final performance characteristics can become a surprise, but that the more complex structures of your project that organises the blocks into modules, classes, functions, records, etc need to be declarative, rather than an imperative set of instructions that build a structure at run-time (usually difficult or impossible to explore and instrument) which is later executed to achieve your actual goals.