I agree. I haven't read the article, but the first thing that came to my mind was: "it doesn't suck". Rather, I find it a fun activity.
There's so much creativity one can put into programming. Especially when programming for oneself and not for Super Evil MegaCorp (not the actual game dev company [0]!). Sometimes it feels to me like art, for example when creating games.
My older brother, at some point in his live, he thought programming sucked as well. But then he stopped doing programming professionally and now just as a hobby and it became fun for him once again.
Regarding programming as art, I like the quote from Fred Brooks comparing it to poetry:
"The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of imagination ... yet the program construct, unlike the poet’s words, is real in the sense that it moves and works, producing visible outputs separate from the construct itself"
According to the author programming sucks for the following reasons:
* costs and time, difficult to maintain
* difficult to hire developers, difficult to learn
* no diversity in industry
* users have no influence on product
Well, yes. Software is a labour intensive task. But given the number of jobs digitization has destroyed and the immense gains it provides, I don't think it so bad. Despite the authors criticism it's still an improvement.
I find his argument hard to follow, the author wants to solve 'the problem', but I don't quite get it. The all purpose machine was a good idea, having an evolving eco system, competition, forks, etc. is a good thing.
Programming is not designed to solve those things - they're not problems created by programming, but by:
* Incompetent management
* Lack of communication
* Rush to market, rather than integrity of solution
* No solid requirements
The unfortunate irony here is that the startup mentality promotes the above problems because it's generally not trying to solve a problem, but make money.
Unfortunately, well-established companies have to (or certainly feel they have to) compete with startups.
Many reports show how time-to-market is more and more important, and 'agile' is the only way.
Even companies well-versed in proper engineering practices move towards the modern 'best practices' and acquire all of the best problems at the same time.
Read the article first. (The author is in fact himself a programmer who loves what he does.) The reason he says it sucks is not because it isn't interesting, but because modern programming is using a machine (the computer) to solve problems that that machine was never meant to solve and is actually rather ill-fitted to solve. He says that all the pay-offs and work-arounds we have had to use to get computers (originally a purely mathematical machine) to do what we want is the cause of many, if not most, problems in programming.
I really don't think that programming sucks. I think some software sucks, and this is usually because of a number of reasons.
* There is always a deadline and pressure to deliver that exceeds quality controls.
* Software development shifted from engineering to crafting a long time ago, mostly because of the prior pressure to deliver.
* Poorer quality code that works and building new features on top are easier to sell, than rewrites and paying off technical dept.
I think it comes down to building in pieces that you don't mind throwing away when a better piece comes along.
The article is a long rant, but wrong on more premises and conclusions. And yet, by quoting the people who were actually right in the cases they wrote about, it appears more correct than it is.
There are too many wrong "conclusions" and premises in the text, just an example, here's the author writing about Babbage's and his Analytical Machine:
"he was focused on doing math fast and without errors. He didn’t care how complex the machine was as long as it accomplished that."
Of course, not true, Babbage had to care. He was designing a purely mechanical machine for years at the time when nobody had electrical logical circuits. He designed the simplest "universal enough" machine in the limits of the technology of his time. And the design was never built. But to claim that his design was "too complex" the author would have to design a significantly simpler one under the same limitations and goals.
The whole article is, unfortunately, like this. Uses something that actually happened or somebody did say, "concludes" something else. And the claims that aren't that kind of "conclusions" also sound like from somebody who only very partially understands what he writes about.
This doesn't mean that the author shouldn't attempt to write more articles, he can learn something from it (although I'd suggest him to base his writing on his own, specific, experiences) just that this article as it is is a quite bad potential basis for a serious discussion, IMHO.
He asks "Is there another hardware, general-purpose or not, that would fit our needs more precisely while being cheaper to program?"
The answer is a simple "no." The author just doesn't understand how much of everything was invested up to now for him to pay so little for the CPU's he uses:
Note just the "plant stat-up costs in billions" column. And that's just a piece in the whole puzzle. There are big gains for those who'd improve a current state, so the humanity as the whole is much more motivated for the real improvements, where they are possible, than the author assumes.
When we remove wrong conclusions after quoting others, the rant can be summed up to just a few claims, one of major being "I'm confronted with different frameworks and libraries, "everybody" solves similar problems, it "sucks" to work this way, maybe the hardware is the main problem." No.
reference? In any case, the main point is that he was making tradeoffs for his math machine, not for modern software development. Not blaming him, just pointing out it was suboptimal for us.
> The whole article is, unfortunately, like this
I'd love more examples. I have definitely a lot to learn about writing.
> The author just doesn't understand how much of everything was invested up to now for him to pay so little for the CPU's he uses
I don't see how "a lot of money was invested" proves how there is no alternate hardware that would make programming more productive.
> I don't see how "a lot of money was invested" proves how there is no alternate hardware that would make programming more productive.
Couldn't maybe the reason for not understanding be not knowing enough about the theoretical and practical aspects of the hardware development? How about suggesting any reasonable direction for improvement? Practically everybody who's involved with hardware and software constantly tries to make something better, somehow you posit that all of them are wrong for doing what they are doing?
It's simple: people propose, sometimes also implement the "alternate hardware." Hardware evolves to the demands, what we have is a result of the evolution. Unsuccessful fail, successful survive. Success is measured in people being ready to invest billions just to be able to produce a new generation of the CPUs, because that's how much it costs. There's no cheaper known way. If you invent something better than the existing state, and it's provably and significantly better, it is going to be accepted. That's how we have today ARM, that's how we don't use Itanium, that's why Intel uses some technology developed by AMD. The leading position of Intel x32 and AMD x64 architectures on the desktops and servers was decided exactly because it was "easier" to program them compared to some other solutions (and the computations were also faster). ARM dominates for having some specific other advantages. All aspects, including these you recognized as important, are actually factored in the success of the current hardware.
> the main point is that he was making tradeoffs for his math machine, not for modern software development
And this doesn't mean that people just blindly reproduced what he did during the following 150-something years. Instead, just like he did, people constantly questioned what they did and decided based on the limitations they had, just like he. His design was more "mentioned" than actually used. It's the opposite, once more or less independent development recognizes which ideas are the "best" in the given time and the given limitations, people who study the old research recognize that the same ideas were considered or even followed in some older designs. Now we have enough experience that we can study the failed and successful attempts that already answer some questions somebody new to the field can have.
> I have definitely a lot to learn about writing.
If you write about the possibility of a "better hardware" familiarizing more with the computer hardware and the theoretical and practical basis and limitations would help not sounding uneducated. Rules are to be broken, the "well known truths" are to be questioned to make a change, but the questioning has to be based on some knowledge, when it's in the field where so much is dependent on the knowledge.
Taking the creativity out of an inherently creative process.
Imagine you're a painter and then comes some moron (who is probably not a painter) yelling at you:
- Why are you holding the brush that way? Also, I don't know that brand.
- You should make thinner strokes, and always start coloring the surroundings so you don't go over by mistake.
- Do not bother with the first brushes until you have a clear pencil sketch. Actually, here, use this one that I paid someone else to make. Limit yourself to add these colors.
- The painters at X draw ultra realistic portraits, that doesn't even look like a cow!
In pure software engineering there is much less creativity needed than most people want to admit.
While the UI/UX people gather requirements and work with the client to work out how the app should look and behave, sure, they can use their imagination. But once the requirements are set then in many cases a software engineer can sit down and build the software. There is no creativity needed to decide whether to use a vector or a double linked list if you need to prioritise fast append and prepend. After all, if you break down software to its basic building blocks, it's all mathematics and logic. And there is no creativity in Mathematics.
I know. I am so fucking angry no one told me this. I am a creative person I become very depressed without creative control. I had no idea programming was going to be this way and now I am stuck in a life of boring drudgery, wasting my life doing the thinking of some incompetent higher up the food chain. But there is no way out - I have to feed myself and my family. FML
I think you confuse creativity (to create something new) with self-actualization and visual art.
Programming is a craft, often used to create things which are really boring.
Coders often like their craft, they like creating things, even if the final product is not important to them on a personal level.
The result of the work is not boring (hopefully, for the sake of its users). But the process of getting there is rarely creative (in the sense of inventing something new).
Most software engineers rarely need to come up with new data structures or fancy algorithms. Most of these things are already there, we just need to know which ones to pick in a particular situation and how to glue them together.
It's perfectly fine to produce boring products. Data entry has never been fun, but it's needed.
Creating new algorithms or data structures may be scientifically creative. I don't know, maybe it's boring for the scientists and they think reasoning and writing about these is much more fun and creative than implementing them. Anyways it just one kind of creativity.
But there is creativity in 'just gluing' stuff together. It's not that simple and there is more choice involved than you suggest.
If a requirement is 'creative' - that is, it involves doing something that has never been done before, as opposed to a standard design pattern - then it will also require a creative implementation. Sometimes very simple requirements require great creativity in the implementation.
Most software requirements leave the question of how the thing will be achieved entirely unstated. It's very easy to write a single page of requirements that could require hundreds of pages of code to implement.
Shouldn't the title refer to Software Engineering or Software in general instead of Programming?
The word programming (for me) represents a more abstract meaning, like getting your thoughts in the physical world.
I'd be sympathetic with someone saying that software development sucks...in the same way that writing doesn't "suck", but having to be a published writer, e.g. someone who has to churn out something that people notice and will pay for, could be a sucky existence (until you're JK Rowling)
But programming? It's an expression of the mind, and as a nice side benefit, the ability to communicate explicit commands to tireless machines to act on our behalf. There have been a couple of long, recorded speeches that I need to transcribe (for searchability and research purposes) and so I've been checking out the various speech-to-text APIs (Watson and Oxford) as a first step (e.g. before sending it to Mechanical Turk) in automating the process. The APIs are so easy to tinker with that I thought it'd be fun to pass in clips of Malcolm Tucker to see how well the APIs did. Then I remembered the IDEA project (a collection of audio of people all around the world with various accents attempting to read English) and thought it'd be interesting to gauge which accents are the hardest for machines to figure out...and all it takes is some scripting to automate the feeding of these recorded clips into the APIs (the analysis is tricky because of the high variance in audio quality, never mind the accents themselves). All of this is useful in figuring out a solution to my original problem, except it's a lot more fun and I get to think a lot more about how language works.
If I weren't a programmer? I'd set aside a day, brew some coffee, and manually do the tedious work of transcription by re-listening to those speeches several times over just to get them into a text format. How boring.
Yeah, programming can be difficult, but it is completely the right level of difficulty for the power it gives us. Even the things that make me nearly throw my laptop into the wall -- such as string encoding -- are a result of something kind of wonderful...the attempt at having an automated, standardized way to translate every squiggle among the hundreds of human languages, and cultural quirks, too, such as a pile of poo.
I don't think I've said anything the OP would disagree with. But I kind of checked out after he got into ranting about JavaScript, as if the world needed more bundles of words that insinuate that programming == web development. Things could be smoother, and they are getting that way because of how programming coordinates the flow and storage of information and assertions...but a lot of the OP's essay is not much more useful than someone ranting about how literacy sucks because we all can't agree what makes for great literature these days.
[meta] The title of this article sucks, really, it's a bait, which is sad because the article is actually interesting. Why do blog post titles about programming have to suck in order to attract readers ?
> Lots of rework happens because of miscommunications with the user. Other times, the user doesn’t fully understand their problem (there’s not much we can do about this)
Agreed. That's where "lean programming" comes into play. If you have the capacity to make the code change as requirements change then I think the issue is pretty much solved. However, not all languages/solutions are equal in that matter. Obviously TDD, Agile techniques help.
> If we assume that typing code is the way to program, we will never look for better alternatives.
Typing code is a medium. The fact is, for an engineers it usually the most convenient way to create a program.
> But without an end goal, of course you have millions of developers releasing Javascript frameworks every day
How many of these are still maintained 1,2,5 years after?
So I don't think that is really a problem, there is an handfull of well maintained frameworks and libraries in the JS world. Most are not. Most are flashy with a nice "flat" page then you never hear about them again.
It's easy to produce shitloads of code, it's harder to maintain it and build a real community around it.
> Building software is 90% a human problem
Agreed, but like everything humans do.
> What would programming be to you in an ideal world?
There is no general answer to that question. That's why we have all these languages, frameworks and libraries yet we keep on creating languages,frameworks and libraries.
What drives me as a "programmer" isn't the medium, but the goal, the product.
Creating something useful. Languages, frameworks and development methods are the necessary constraints.
> [meta] The title of this article sucks, really, it's a bait,
It was not intended. The title just tries to summarize the post. Still, so much to learn about writing!
>> Lots of rework happens because of miscommunications with the user. Other times, the user doesn’t fully understand their problem (there’s not much we can do about this)
> Agreed. That's where "lean programming" comes into play. If you have the capacity to make the code change as requirements change then I think the issue is pretty much solved. However, not all languages/solutions are equal in that matter. Obviously TDD, Agile techniques help.
I have a note on Agile. As I mentioned, I believe Agile or lean programming to be solutions. They don't question anything about the status quo, they just try to cope with it. It's good, it's an improvement. But an incremental one from the perspective of the stack of poker chips.
>> If we assume that typing code is the way to program, we will never look for better alternatives.
> Typing code is a medium. The fact is, for an engineers it usually the most convenient way to create a program.
>> Building software is 90% a human problem
>Agreed, but like everything humans do.
The first rule of the tautology club is the first rule of the tautology club. Of course! the idea is to make programming something that humans don't do, or at least, reduce how much human involvement you need. Humans are the weaker link in software development, we should do like with other things that humans don't do anymore, figure out a solution and put our time to do something else.
> Humans are the weaker link in software development, we should do like with other things that humans don't do anymore, figure out a solution and put our time to do something else.
Here's the thing: everything sucks. Every industry, profession, even hobby is full of seemingly arbitrary limitations, poor tooling, quirky hacks you have to work around, little bits of required know-how that seem ancillary to the central goals. Ask your friends in other engineering disciplines, politics and public policy of all kinds, finance, medicine, and energy. Everything sucks because the world is complicated and path dependency calls the shots. The question to ask is whether things are getting better or worse.
And things in programming are getting emphatically better! For all the incidental complexity and pointless thrashing about [1], almost everything about writing software is better than it was just 10 years ago, which is not a very long time. The people who say things like "nothing ever changes" or "we're just reinventing old technology" have some seriously rose-tinted glasses. It's so much easier to do just about everything. The tools are better, the hardware is better, the quantity and quality of information is better. And we have better, bigger software doing more important things, which is why the industry is doing so well. The sky is doing the opposite of falling.
That isn't to say there isn't room for improvement, or that complaining about broken things is wrong. That lets us keeping fixing things. It's more that "everything is broken and stupid" isn't a very useful lens, and leads to fool's errands like this:
> It’s time to kill the aimless, bottom-up, ad-hoc incrementalism in programming in favor of purposeful, top-bottom and planned innovation.
I'll stick to my massive pile of working software that solves real problems, warts and all. And I'll continue to help improve it wherever I can so that it works better.
[1] I joked recently that Javascript library developers would be more efficient if they had their CI servers deprecate their APIs automatically before releasing them
>> It’s time to kill the aimless, bottom-up, ad-hoc incrementalism in programming in favor of purposeful, top-bottom and planned innovation.
Agree. Let the author put his money where his mouth is first, then we can discuss where he's right.
He wants programming to be "easier." Well everybody wants everything easier. It's easy to have just one thing easy scarifying other dependencies. But the limitations exist everywhere. And it was damn hard to even come this far and maintain the state we are here now:
> Are any of his shows based somewhat around this topic
Not that I know. It wouldn't be appropriate for a general audience he targets. Anyway, there's a pattern, as soon as almost every stand up comedian becomes reasonably successful he starts to travel a lot by plane and the gags start to revolve about that. We just got lucky with his take to make it meta enough, recognizable and hilarious at the same time.
What are you going to make or you're making already to change the "suckiness" of programming? Or was your article (and the question to others) your contribution? You've got at least a lot of comments here (and a lot of disagreement too) now you have something to proceed.
as a tinkerer, I love programming. I enjoy optimizing routines, fighting complexity, exploring the solution space for hard problems.
the problem is, that doesn't pay for a roof and a meal.
so I spend my day fighting users that requires twenty seven pop-ups not to delete their own data accidentally, browsers that have each their own idea of css3, enterprise services that are a landmine of old documentation, bugs, workarounds, workarounds for bugged workarounds and bugs on top of these.
as my mentor once said, if work were fun they wouldn't have to pay you to do it.
It's worth mentioning that Chris Granger wrote an excellent blog post[0] that's similar to this one, but far more succinct. While I don't use Light Table, I'm still a huge fan regardless.
>What would programming be to you in an ideal world?
More visual, especially when observing execution.
When you're debugging anything non-synchronous—be it events, messaging, distributed systems—it usually sucks, even in the rare case that you do have good tooling.
I'd like to create, view and debug my code in an almost entirely visual fashion. I want to see events and messages perfuse through the structure of what I'm building, with complete temporal control of the view.
Pretty much. One ideal might be for a flow chart to come alive and actually be the structure of your application itself, rather than just a reference or an aid.
I suppose that while visual programming excels at conveying topology and structure, it can be weak at a low level due to the tedium involved. Traditional code is arguably better suited for low-level grunt work, at least for most people.
A hybrid solution might be best. For example: a message travels from one high-level visual entity to another, and you'd easily be able to drill down into the code comprising those entities, locating the exact lines responsible for emission or receipt.
"Even if UML is viewed as the ‘de facto’ standard, it is
by no means universally adopted. The majority of those
interviewed simply do not use UML, and those who do use it
tend to do so selectively and often informally."
"They also
highlight some of the fundamental tensions within UML,
resonating with arguments [8], [25], [17] that UML’s intended
strengths (i.e., generality, accommodating different levels of
abstraction) are intimately associated with its observed
weaknesses (e.g., latent complexity, issues of transformation
and coordination between views) and arise from fundamental
properties of UML (e.g. lack of formal semantics, separation of
expressions of structure and behavior)."
Whatever we do next, we have a nice set of examples and case studies of "what can go wrong."
The specific software tools based on UML are especially good examples of the problems.
While I'm not terribly familiar with UML, I was under the impression that it's primarily a modeling language, not necessarily a visual programming implement.
As far as I'm aware, most visual programming today is actually implemented in the form of node-based editors, which gained popularity in the film and visual effects industries well over a decade ago.
The best modern examples I can think of are probably Nuke[0] and UE4's Blueprint[1] system.
"The development of Rose 2.0 combined a Windows-based Booch notation editor called Object System Designer (acquired from Wisconsin-based Palladio) with a new intermediate representation, and with new semantic analysis, code generation, and reverse engineering capabilities. The latter, which allowed prospective customers to analyze existing C++ code to produce "as-built" navigable class diagrams, helped overcome Rational's late re-entry into the market for object-oriented modeling tools. Rose 2.0 ran on Windows PCs and on several Unix-based workstations."
It wasn't usable, except to be used in the projects doomed to fail. Or spend a lot of money. Which is something a goal of the managers. But it still can't be named a success.
UML didn't pick up for whatever reason and visual programming fell into the valley of deception. I expect that with more UI/UX research we can get to the plateau of productivity in the next decade.
If you believe it's a good way, you should certainly go that way. Please write about your experiences afterwards.
Rose 2.0, I've mentioned and quoted explicitly, produced the code from the diagrams (that's the "code generation" from the quote). And it was tragedy. Maybe you find somebody's experiences on the web.
Wrong. Here is a thing, soviet spaceship Buran, which was performing fully automated landing in 1991, was programmed with this stuff - https://en.wikipedia.org/wiki/DRAKON. Teams of software engineers created this language so rocker scientists would be able to programm spaceship without software developers.
I don't have a problem with generating diagrams from configuration and/or code (e.g. with Graphviz) but I'm not keen on diagrams actually being edited and used to generate stuff. However, I would say that my aversion isn't based on a fundamental problem just on my experience that "graphical" tools are often awful from a usability and practicality perspective so that the benefits never seem to clearly outweigh the significant issues (e.g. compare diffs of text files with diffs of diagrams).
>However, I would say that my aversion isn't based on a fundamental problem just on my experience that "graphical" tools are often awful from a usability and practicality perspective ...
Yeah. For any truly novel visual programming approach to succeed it'd probably require some next-level UI/UX work for sure.
Visual programming could be very well an alternative. Or fucking oculus vr or god knows why. I 100% agree with rl3, it proposes a very interesting and challenging problems for the UI/UX folks.
You might want to look at https://en.wikipedia.org/wiki/DRAKON. Teams of soviet software engineers created this language in 1980-s so rocket scientists would be able to program spaceship without software developers. Buran was programmed to perform automatic landing using this.
Labview has some interesting ideas (I think a subset made it into Lego Mindstorm)
Also many people don't use what is available to them. Modern IDEs support single-stepping and painless runtime introspection, and many have supported it for 20 years.
Still people resort to trying to trace out in their head what happens runtime.
One of the premises of this article - that "competing standards" exist primarily because of political reasons or shallow decision making - demonstrates a fundamental misunderstanding of software engineering.
Sure, some engineers (especially junior ones) enjoy reinventing the wheel more than using something that already exists to solve users' problems. We all like to trick ourselves into thinking we're unique snowflakes (this problem extends into our personal lives as well). But more often than not, when multiple popular standards emerge, it's because there are legitimate engineering tradeoffs that are being made.
JSON vs. XML? XML is far more capable but complex. JSON is far simpler but less capable. Sometimes when you're delivering value to customers, you need that extra complexity, so you use XML. Other times you don't need it so you use JSON. It has nothing to do with ego or politics. Multiple different JSON libraries? One may prioritize ease of use to get up and running, the other may prioritize strong typing for serialization speed. One may prioritize strict compliance with the spec, the other might prioritize speed above all else. JSON vs. a binary format? JSON is more readily compatible and easier to debug. Binary is faster but more complex to setup. It goes on and on.
Again, sometimes competing standards or libraries or languages emerge because of political or capitalistic concerns. But usually when you're talking about competing open standards, there are multiple because there are legitimate engineering tradeoffs being made because engineering is not a one-size-fits-all science. (Few sciences are, otherwise they wouldn't still have people working on them.)
When an open source project rejects a contribution, or five projects exist to solve the same problem, they're often making legitimate engineering tradeoffs that are in no way arbitrary, and any one project would suffer to try to be every thing to every person. The article in question doesn't even point out a single example of a set of standards or libraries that are entirely arbitrary in their differences or could be collapsed into one solution, which further highlights how this is a theoretical argument, not a practical one.
Another thing to remember is that our collective understanding of the problems evolve over time and often subsequent standards reflect this. XML vs JSON is definitely a case of this, as is the progression of CORBA -> SOAP -> JSON based REST services.
I clarified that "Sometimes" the causes are political, it used to say "Usually".
But if after a 30 minute read you picked up on a stupid example and you started a discussion on XML vs JSON you are just proving my point. The expectation in the 60s was that machines will figure out protocol, even make protocol, on the fly, by asking about each other. Yet, you want to have another discussion about XML and JSON.
Machines do figure out the protocol on the fly by asking each other; see the Accept and Upgrade headers in HTTP, for example. Of course, HTTP is itself a protocol, because you can't "ask each other", or do any kind of communication at all, without a common protocol to start with.
As for making protocols on the fly, that makes roughly as much sense as two people inventing their own language to talk to each other.
That was the great time of belief in "4GL". I think what we've learned from that is that the hard part is not so much implementing what we want, but making all the decisions. Coping with all the interacting possibilities and working out what we want in each case.
> Of course we have never questioned any of this. We don’t have a direction, a path or an overall goal for programming!
> Let me ask you this: What would programming be to you in an ideal world?
> We don’t know! We’ve never asked ourselves this question!
When at the start of the "essay":
> Turns out, we are horrible at predicting the future and we end up discarding that extra complexity.
Too bad outsiders think (and will for a long time) it's a done thing. Society thinks software is the best thing since sliced bread so it's all around us. It's both a curse and a blessing.
Compared to other fields, CS or engineering is relatively new. It's exciting, it will keep breaking and evolving. And that should be it. It should keep going places without restrictions, precisely because we don't know.
Similarly to scientists ripping open humans on anathomical theatres in the 17th, we are having fun.
You hit the spot. Yes, who is it to say that we reinvent programming tomorrow and it useless by next week. Great point.
My answer is, don't look at it from a black and white perspective. I'm not aiming for a 100% future proof solution, that is impossible as you pointed out. But we can make something much more future proof that what we have right now.
We don’t have a direction, a path or an overall goal for programming!
We (who do we mean when we say "we", anyway?) don't have a direction, path or overall goal for life, society, the human race and so on. I don't even have a plan for what I'm going to have for lunch next Tuesday.
Programming is contingent and incremental because rationality is bounded and the future is uncertain. We don't know what will be successful in the next few years or how the world will want to use it or what it will be prepared to pay for.
We can build jewel-like systems that are perfectly adapted to their use case using technology to the utmost. This nearly always results in a stranded evolutionary dead-end. The BBC Domesday Project (live video database on an 8-bit micro using PAL laserdisc!) is my favourite example, but there are many.
Personally, I don't think we could reinvent programming and make it 100% future proof. But if we reinvented programming it will be much more future proof that what we have now. The reason is that our current stack was not invented in the first place, kind of evolved upwards organically. If we did something it will at least be purposeful.
I would say that everything the author describes as facts that have led us to this "mess" are actual testments to how fucking amazing human ingenuity is.
Look at everything we built out of arbitrary conventions of boolean logic that has real impact and is shapping the modern world!
>>The computer shifted the complexity of our solutions from the world of atoms to the world of ideas.
And that's fucking awesome! Thanks everyone that was part in making that a reality!
Complexity can pile up in this new world, sure, but we are only starting to get our feet wet. Outside the boundaries of processing and storage, we are our only limitation. We can create things as complex as we can logically describe them.
It's also incredibly difficult to create something that will acomodate everyone else's solutions.
Software development is in its infancy. The fact that we can create absurd structures that still work is great because we can use them even if our skills are not mature enough to make truly elegant structures.
Saying programming sucks and wishfull thinking does not help anyone. Not happy? Start hacking on the next thing!
>>The computer shifted the complexity of our solutions from the world of atoms to the world of ideas.
>And that's fucking awesome! Thanks everyone that was part in making that a reality!
I glad you are entertained with programming. I am too! but we should judge a tool like programming by its productivity, not by it's entertaining powers.
> We can create things as complex as we can logically describe them.
One of my points is that we can't. To make a single app facebook needs hundreds of engineers and it is still buggy. Humans are, yet again, the weaker link in software engineering. I believe we need fundamentally better tools.
> It's also incredibly difficult to create something that will acomodate everyone else's solutions.
Agreed. I'd very happy with a tool that is not general purpose that could churn though CRUD systems (vast majority of systems our there) in a matter of hours without errors. If you want to something else like AI, or games or whathaveyou, use other tools built specifically for that or a programming language if none are available.
That CRUD tool (gross oversimplification) could be expanded with new capabilities as long as we understand those new capabilities properly.
> Saying programming sucks and wishfull thinking does not help anyone. Not happy? Start hacking on the next thing!
No one is going to listen to this - but what the hell:
The solution is totally obvious. The code we write has a form defined by it's input->output mapping over all possible inputs.
Write a function and you immediately define this input output mapping. All you have to do at this point is search through all previously written functions to find the one that most resembles the one you are trying to write.
Trying to write a nested for loop to pull out pixel values from a jpeg? Start writing it, shit you get it wrong - but the input output mapping is close enough to the cluster of previously written solutions that the correct code pops onto the screen.
Structure aware code search across all open source code would make a lot of the pain just go away.
> All you have to do at this point is search through all previously written functions to find the one that most resembles the one you are trying to write.
This magical repository you seek is called "stackoverflow" :-D
In the ideal world, i would program purely at the level of a sugared lambda calculus, free of trivial things like runtime,memory use,network latencies,serialization,platform incompatibilities,dependency management,string encoding/decoding issues <so many others>. Basically everything that makes programming different from math.
However, programming may be lambda calculus at a theoretical level, but execution is a turing machine. All these little real world things that can be abstracted away in abstractions(!!), have to be unabstracted to meet a bunch of very real NAND gates somewhere, limited memory, resources etc.
Thats one reason it "sucks", but thats why its engineering and not math. Maybe one day computers will just be so powerful that we can afford to treat memory as infinite, runtime as infinitesimal, and build all the abstractions we want for free - then programming will be done :)
In this fun world, when i want to build something ill just draw a bunch of squiggly boxes and describe an alarmingly underspecified thingy i want done. Then the program will automatically figure out all possible inputs, quickly perform a "32 questions" kind of thing and resolve any ambiguity, then produce code. Managers will once again be confused why "adding this textbox is so hard", but this time they will be right - i will just tell lambda to go make a textbox.
We are what normal people rely on to get the computer to do things that they find difficult.
Things are getting so complex for us that we find it difficult to service the requests.
Solution: We need someone to help us to solve the problems. Or more practical - we need intelligent tools which attempt to guess what we are really trying to do.
Far more intelligent autocomplete: "This looks like you are writting a postcode splitting regex, here is one that has been used by many programmers.". "Looks like you are adding a new business layer class - do you want to create an entity as well. Oh also it looks like this is going to part of the new view model you have been working on - do you want me to add a projection in for bl Class, mapping to the un populated properties on the vm?" etc.
This article more than anything underscores my feeling that there are too many people in programming who should not be doing it in the first place. It's a skilled profession, but it's increasingly dominated by people who despise the craft aspect of it. This results in both, suckage and bloat on the one end, and unrealistic expectations on the other.
> Hiring developers is an extremely difficult task. Learning to program is ridiculously difficult as well.
If programming was a solved thing, that would mean problem solving itself was a solved thing. It has by nature to be difficult. The question is more whether we manage to avoid additional difficulty on top of that. He's talking about the cost of bugs, but I posit the cost of burdensome frameworks and artificial bureaucracies in software development processes is by far bigger and harder to come to terms with, both of which exist in part to manage this inherent difficulty (often achieving the opposite).
> Lots of rework happens because of miscommunications with the user. Other times, the user doesn’t fully understand their problem
This is not unique to software development by any means. Every organization that deals with end users has to negotiate this interface.
> anywhere from 50 to 90% of the cost of building software goes towards maintaining it after the first release
Whether that figure is real depends largely on the type of software you're writing. If this fits what you're doing, the assumption that at some point the program is supposed to be "done" and every bit of time spent evolving it further must be a sign of inefficiency is simply unproductive. That kind of program is more usefully viewed as an evolving organism, and the sooner management recognizes the nature of this the better it will be handled by the organization.
> Our tools don’t make a good separation of essential and accidental complexity[2]––we are forced to crystalize the essential parts of our programs in a programming language and make them coexist with irrelevant, usually platform-specific code (accidental complexity).
This too depends on the nature of your chosen ecosystem. On the whole, interop has steadily improved throughout the history of computing. We have meaningful universal data formats now, and many options for moving code between platforms. Where interop troubles exist, they mostly exist on purpose (like the iOS example you made). You can absolutely solve most of this problem by throwing computing power at it and choosing an interpreted or cross-compiled environment if you want to avoid writing glue code.
That's the word choice made in the article, it's not mine. I agree with you, I've been programming since I was a child, too. Learning is hard, and it never stops, but that is not a problematic aspect at all as long as you enjoy doing it.
I see this kind of article posted regularly, and it has been for at least the past decade (if not longer.)
Worse, it's often accompanied with 'but profession X is so much better and they are all certified and do things properly unlike us awful cowboys!'. In the worst case you get the whole 'in the future everyone will program and programmers will become totally redundant'.
I have a feeling the developers writing these posts don't have a huge amount of experience, and have a natural tendency to want to turn the table up side down and start afresh to get rid of all this cruft. I understand it, but it's frustrating
But what they perceive to be the cruft isn't just cruft. In programming the possibility of starting afresh is there, and has been done several times - haskell, erlang, go, rust, clojure, etc. etc. etc. on the programming side. In the web world the front end is filled with the latest great solution that just pushes aside all this terrible accidental complexity, and this happens in other realms too, yet we still see these articles.
I think there are a number of things at play here:
1. Programming is hard and often in annoying, frustrating, and painful ways. Nothing you do can change that, and blaming it on tooling or legacy or whatever doesn't change the _intrinsic_ complexity at hand. People like to imagine it's hard in challenging and enjoyable ways, and sometimes it is, but more often it's just frustrating.
2. Abstractions won't save you. They help, but always involve trade offs of one kind or another. Something that is declarative enough hides imperative details which eventually leak in horrible ways unless you come to understand how they work, and something too imperative invites duplication, complexity and bugs. Having seen giant T/SQL stored procedures, Microsoft Access queries that take hours to run and excel sheets that are so complicated nobody can vet them, I have seen my fair share of collapsed abstractions which started wonderful.
3. Programmers are needed. A great example is excel - some people point to it as the philosopher's stone of the 'everyone is a programmer' world, but as soon as you start doing things it's not good at, or exceed a certain complexity, it explodes into horrible pieces and fails in all kinds of terrible ways. Then the programmers need to step in. It's not something special or conceited, personally I think only a certain % of the population are capable of thinking in a sufficiently logical way to understand what a computer needs, whether employed as 'programmers' or not, and these are the only people who can help beyond the simplest case. Nothing to do with intelligence, just a kind of thinking. The power of software + the rarity of this way of thinking combined with people actually using it is why programmers are expensive, it's not an anomaly imo.
4. Real world work is largely boring and/or painful. I think particularly here people miss this point, in fact PG wrote a great piece relating to this (http://paulgraham.com/schlep.html), but it's so important - even the most interesting problem involves a great deal of schlep to get things _nicely_. No matter how wonderful programming could possibly be made, there is no avoiding this. It's true not only of programming but any job. I think there's too much of a delusion sold by many that work can be nothing but a joy if only you pick the right one - no. It can _net_ be wonderful, but in that net are wins and losses. If you love programming, the kick of solving problems and seeing an application do something for somebody easily outweighs the pain of schlepping to that point.
5. Other professions aren't magically better. I did a civ eng degree at arguably the top civ eng school in the world, and worked in a practising office for a couple of years and believe me the insight I got from both the theo...
> In the worst case you get the whole 'in the future everyone will program and programmers will become totally redundant'.
Yeah, not going to happen because not everyone can think in the level of detail required to program.
Some people just think programming is like waving a magic wand, but when you keep asking them "so, what should it do if this happens? What about if that happens?" they slowly start to understand...
Yeah this is exactly what I mean by a certain kind of logical thinking. It's made worse by these tasks seeming a lot more simple than they actually are, before you even think about accidental complexity.
What's frustrating is when programmers mistake inherent complexity for the accidental kind and therefore label the whole enterprise a terrible mess.
In fact overall I think this distracts from ACTUAL accidental complexity since if you think programming is generally terrible you'll probably not be too interested in some small details of that :)
I think we just push our requirements until they hit boundaries where programming/tooling/whatever sucks.
These days it's trivial to create a CRUD web app that is even save against CSRF when you use a framework like Django.
But of course, nobody wants a simple CRUD web app anymore. It has to do business logic, it must have responsive design, usable on mobile, incrementally load pages, and be an enterprise integration framwork that checks with the credit card processor, the warehouse software etc.
It's just that we don't talk anymore about the things that have become trivial, because we don't spend much time on them.
So of course, in our perception, programming and tooling suck. And they always will. And that's OK, because if everything was breeze, we'd be bored.
We may be programming in our different teams, companies, organisations and countries, but the stuff we make makes an impact and affects the world around us.
This impact is an input into future similar projects and, given time, the technologies that results from wherever will be better for it. Please note that this idea does not mean that any group now is going to make these technologies, but what we do now will influence the future.
The same pattern has been observed in the industrial revolution - many tinkerers, manufacturers and inventors were all sort of learning from each other, all improving. This happened over decades. The main difference with programming is the time dimension. We've only really been doing it at a cultural level for the last twenty years - we're only just getting started in discussing and sharing what we are doing.
It's depressing to see how much effort was put into the presentation of this blog post compared to the effort in clarifying its ideas. The author's ideas aren't novel.
Programming isn't something that can be "solved" any more than math or writing is. There is inherent complexity in building apps and there are an infinite number that could be built. The fact that it hasn't been "solved" is no indication that we've "never asked ourself this question!"
Most of the pitfalls that the author identifies can be avoided by simply subscribing to the TDD workflow. A little peace of mind actually makes programming pretty fun.
89 comments
[ 2.8 ms ] story [ 127 ms ] threadVery interesting thoughts about this. I wouldn't think programming sucks, until you think about how it could be in an ideal world.
I'd love to see a follow-up article about what the OP thinks about that question.
There's so much creativity one can put into programming. Especially when programming for oneself and not for Super Evil MegaCorp (not the actual game dev company [0]!). Sometimes it feels to me like art, for example when creating games.
My older brother, at some point in his live, he thought programming sucked as well. But then he stopped doing programming professionally and now just as a hobby and it became fun for him once again.
I will read the article later today :)
---
[0]: http://www.superevilmegacorp.com
"The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of imagination ... yet the program construct, unlike the poet’s words, is real in the sense that it moves and works, producing visible outputs separate from the construct itself"
That's one of the reasons why I still love programming (after programming professionally for almost 25 years): http://henrikwarne.com/2012/06/02/why-i-love-coding/
* costs and time, difficult to maintain
* difficult to hire developers, difficult to learn
* no diversity in industry
* users have no influence on product
Well, yes. Software is a labour intensive task. But given the number of jobs digitization has destroyed and the immense gains it provides, I don't think it so bad. Despite the authors criticism it's still an improvement.
I find his argument hard to follow, the author wants to solve 'the problem', but I don't quite get it. The all purpose machine was a good idea, having an evolving eco system, competition, forks, etc. is a good thing.
Sad really.
Many reports show how time-to-market is more and more important, and 'agile' is the only way.
Even companies well-versed in proper engineering practices move towards the modern 'best practices' and acquire all of the best problems at the same time.
No agile development, just a horrible, horrible mess.
* There is always a deadline and pressure to deliver that exceeds quality controls. * Software development shifted from engineering to crafting a long time ago, mostly because of the prior pressure to deliver. * Poorer quality code that works and building new features on top are easier to sell, than rewrites and paying off technical dept.
I think it comes down to building in pieces that you don't mind throwing away when a better piece comes along.
[No ability to execute, no debugging, no compilation step, no unit tests....]
Whoever does this - how about sending a percent or 2 my way when you exit for billions :-D
There are too many wrong "conclusions" and premises in the text, just an example, here's the author writing about Babbage's and his Analytical Machine:
"he was focused on doing math fast and without errors. He didn’t care how complex the machine was as long as it accomplished that."
Of course, not true, Babbage had to care. He was designing a purely mechanical machine for years at the time when nobody had electrical logical circuits. He designed the simplest "universal enough" machine in the limits of the technology of his time. And the design was never built. But to claim that his design was "too complex" the author would have to design a significantly simpler one under the same limitations and goals.
The whole article is, unfortunately, like this. Uses something that actually happened or somebody did say, "concludes" something else. And the claims that aren't that kind of "conclusions" also sound like from somebody who only very partially understands what he writes about.
This doesn't mean that the author shouldn't attempt to write more articles, he can learn something from it (although I'd suggest him to base his writing on his own, specific, experiences) just that this article as it is is a quite bad potential basis for a serious discussion, IMHO.
He asks "Is there another hardware, general-purpose or not, that would fit our needs more precisely while being cheaper to program?"
The answer is a simple "no." The author just doesn't understand how much of everything was invested up to now for him to pay so little for the CPU's he uses:
https://en.wikipedia.org/wiki/List_of_semiconductor_fabricat...
Note just the "plant stat-up costs in billions" column. And that's just a piece in the whole puzzle. There are big gains for those who'd improve a current state, so the humanity as the whole is much more motivated for the real improvements, where they are possible, than the author assumes.
When we remove wrong conclusions after quoting others, the rant can be summed up to just a few claims, one of major being "I'm confronted with different frameworks and libraries, "everybody" solves similar problems, it "sucks" to work this way, maybe the hardware is the main problem." No.
> Of course, not true, Babbage had to care
reference? In any case, the main point is that he was making tradeoffs for his math machine, not for modern software development. Not blaming him, just pointing out it was suboptimal for us.
> The whole article is, unfortunately, like this
I'd love more examples. I have definitely a lot to learn about writing.
> The author just doesn't understand how much of everything was invested up to now for him to pay so little for the CPU's he uses
I don't see how "a lot of money was invested" proves how there is no alternate hardware that would make programming more productive.
Couldn't maybe the reason for not understanding be not knowing enough about the theoretical and practical aspects of the hardware development? How about suggesting any reasonable direction for improvement? Practically everybody who's involved with hardware and software constantly tries to make something better, somehow you posit that all of them are wrong for doing what they are doing?
It's simple: people propose, sometimes also implement the "alternate hardware." Hardware evolves to the demands, what we have is a result of the evolution. Unsuccessful fail, successful survive. Success is measured in people being ready to invest billions just to be able to produce a new generation of the CPUs, because that's how much it costs. There's no cheaper known way. If you invent something better than the existing state, and it's provably and significantly better, it is going to be accepted. That's how we have today ARM, that's how we don't use Itanium, that's why Intel uses some technology developed by AMD. The leading position of Intel x32 and AMD x64 architectures on the desktops and servers was decided exactly because it was "easier" to program them compared to some other solutions (and the computations were also faster). ARM dominates for having some specific other advantages. All aspects, including these you recognized as important, are actually factored in the success of the current hardware.
> the main point is that he was making tradeoffs for his math machine, not for modern software development
And this doesn't mean that people just blindly reproduced what he did during the following 150-something years. Instead, just like he did, people constantly questioned what they did and decided based on the limitations they had, just like he. His design was more "mentioned" than actually used. It's the opposite, once more or less independent development recognizes which ideas are the "best" in the given time and the given limitations, people who study the old research recognize that the same ideas were considered or even followed in some older designs. Now we have enough experience that we can study the failed and successful attempts that already answer some questions somebody new to the field can have.
> I have definitely a lot to learn about writing.
If you write about the possibility of a "better hardware" familiarizing more with the computer hardware and the theoretical and practical basis and limitations would help not sounding uneducated. Rules are to be broken, the "well known truths" are to be questioned to make a change, but the questioning has to be based on some knowledge, when it's in the field where so much is dependent on the knowledge.
Imagine you're a painter and then comes some moron (who is probably not a painter) yelling at you:
- Why are you holding the brush that way? Also, I don't know that brand. - You should make thinner strokes, and always start coloring the surroundings so you don't go over by mistake. - Do not bother with the first brushes until you have a clear pencil sketch. Actually, here, use this one that I paid someone else to make. Limit yourself to add these colors. - The painters at X draw ultra realistic portraits, that doesn't even look like a cow!
Programming sucks because we need to eat.
While the UI/UX people gather requirements and work with the client to work out how the app should look and behave, sure, they can use their imagination. But once the requirements are set then in many cases a software engineer can sit down and build the software. There is no creativity needed to decide whether to use a vector or a double linked list if you need to prioritise fast append and prepend. After all, if you break down software to its basic building blocks, it's all mathematics and logic. And there is no creativity in Mathematics.
What?!
Coders often like their craft, they like creating things, even if the final product is not important to them on a personal level.
Most software engineers rarely need to come up with new data structures or fancy algorithms. Most of these things are already there, we just need to know which ones to pick in a particular situation and how to glue them together.
Creating new algorithms or data structures may be scientifically creative. I don't know, maybe it's boring for the scientists and they think reasoning and writing about these is much more fun and creative than implementing them. Anyways it just one kind of creativity.
But there is creativity in 'just gluing' stuff together. It's not that simple and there is more choice involved than you suggest.
If a requirement is 'creative' - that is, it involves doing something that has never been done before, as opposed to a standard design pattern - then it will also require a creative implementation. Sometimes very simple requirements require great creativity in the implementation.
Most software requirements leave the question of how the thing will be achieved entirely unstated. It's very easy to write a single page of requirements that could require hundreds of pages of code to implement.
But programming? It's an expression of the mind, and as a nice side benefit, the ability to communicate explicit commands to tireless machines to act on our behalf. There have been a couple of long, recorded speeches that I need to transcribe (for searchability and research purposes) and so I've been checking out the various speech-to-text APIs (Watson and Oxford) as a first step (e.g. before sending it to Mechanical Turk) in automating the process. The APIs are so easy to tinker with that I thought it'd be fun to pass in clips of Malcolm Tucker to see how well the APIs did. Then I remembered the IDEA project (a collection of audio of people all around the world with various accents attempting to read English) and thought it'd be interesting to gauge which accents are the hardest for machines to figure out...and all it takes is some scripting to automate the feeding of these recorded clips into the APIs (the analysis is tricky because of the high variance in audio quality, never mind the accents themselves). All of this is useful in figuring out a solution to my original problem, except it's a lot more fun and I get to think a lot more about how language works.
If I weren't a programmer? I'd set aside a day, brew some coffee, and manually do the tedious work of transcription by re-listening to those speeches several times over just to get them into a text format. How boring.
Yeah, programming can be difficult, but it is completely the right level of difficulty for the power it gives us. Even the things that make me nearly throw my laptop into the wall -- such as string encoding -- are a result of something kind of wonderful...the attempt at having an automated, standardized way to translate every squiggle among the hundreds of human languages, and cultural quirks, too, such as a pile of poo.
I don't think I've said anything the OP would disagree with. But I kind of checked out after he got into ranting about JavaScript, as if the world needed more bundles of words that insinuate that programming == web development. Things could be smoother, and they are getting that way because of how programming coordinates the flow and storage of information and assertions...but a lot of the OP's essay is not much more useful than someone ranting about how literacy sucks because we all can't agree what makes for great literature these days.
> Lots of rework happens because of miscommunications with the user. Other times, the user doesn’t fully understand their problem (there’s not much we can do about this)
Agreed. That's where "lean programming" comes into play. If you have the capacity to make the code change as requirements change then I think the issue is pretty much solved. However, not all languages/solutions are equal in that matter. Obviously TDD, Agile techniques help.
> If we assume that typing code is the way to program, we will never look for better alternatives.
Typing code is a medium. The fact is, for an engineers it usually the most convenient way to create a program.
> But without an end goal, of course you have millions of developers releasing Javascript frameworks every day
How many of these are still maintained 1,2,5 years after?
https://www.nczonline.net/blog/2015/12/why-im-not-using-your...
So I don't think that is really a problem, there is an handfull of well maintained frameworks and libraries in the JS world. Most are not. Most are flashy with a nice "flat" page then you never hear about them again.
It's easy to produce shitloads of code, it's harder to maintain it and build a real community around it.
> Building software is 90% a human problem
Agreed, but like everything humans do.
> What would programming be to you in an ideal world?
There is no general answer to that question. That's why we have all these languages, frameworks and libraries yet we keep on creating languages,frameworks and libraries.
What drives me as a "programmer" isn't the medium, but the goal, the product.
Creating something useful. Languages, frameworks and development methods are the necessary constraints.
> [meta] The title of this article sucks, really, it's a bait,
It was not intended. The title just tries to summarize the post. Still, so much to learn about writing!
>> Lots of rework happens because of miscommunications with the user. Other times, the user doesn’t fully understand their problem (there’s not much we can do about this)
> Agreed. That's where "lean programming" comes into play. If you have the capacity to make the code change as requirements change then I think the issue is pretty much solved. However, not all languages/solutions are equal in that matter. Obviously TDD, Agile techniques help.
I have a note on Agile. As I mentioned, I believe Agile or lean programming to be solutions. They don't question anything about the status quo, they just try to cope with it. It's good, it's an improvement. But an incremental one from the perspective of the stack of poker chips.
>> If we assume that typing code is the way to program, we will never look for better alternatives. > Typing code is a medium. The fact is, for an engineers it usually the most convenient way to create a program.
Turns out that the medium has an huge effect on our understanding. https://signalvnoise.com/posts/2976-bret-victor-thinks-math-...
>> Building software is 90% a human problem >Agreed, but like everything humans do.
The first rule of the tautology club is the first rule of the tautology club. Of course! the idea is to make programming something that humans don't do, or at least, reduce how much human involvement you need. Humans are the weaker link in software development, we should do like with other things that humans don't do anymore, figure out a solution and put our time to do something else.
WALL·E would disagree, and so would I.
And things in programming are getting emphatically better! For all the incidental complexity and pointless thrashing about [1], almost everything about writing software is better than it was just 10 years ago, which is not a very long time. The people who say things like "nothing ever changes" or "we're just reinventing old technology" have some seriously rose-tinted glasses. It's so much easier to do just about everything. The tools are better, the hardware is better, the quantity and quality of information is better. And we have better, bigger software doing more important things, which is why the industry is doing so well. The sky is doing the opposite of falling.
That isn't to say there isn't room for improvement, or that complaining about broken things is wrong. That lets us keeping fixing things. It's more that "everything is broken and stupid" isn't a very useful lens, and leads to fool's errands like this:
> It’s time to kill the aimless, bottom-up, ad-hoc incrementalism in programming in favor of purposeful, top-bottom and planned innovation.
I'll stick to my massive pile of working software that solves real problems, warts and all. And I'll continue to help improve it wherever I can so that it works better.
[1] I joked recently that Javascript library developers would be more efficient if they had their CI servers deprecate their APIs automatically before releasing them
>> It’s time to kill the aimless, bottom-up, ad-hoc incrementalism in programming in favor of purposeful, top-bottom and planned innovation.
Agree. Let the author put his money where his mouth is first, then we can discuss where he's right.
He wants programming to be "easier." Well everybody wants everything easier. It's easy to have just one thing easy scarifying other dependencies. But the limitations exist everywhere. And it was damn hard to even come this far and maintain the state we are here now:
Louis C.K's take:
https://www.youtube.com/watch?v=uEY58fiSK8E
Not that I know. It wouldn't be appropriate for a general audience he targets. Anyway, there's a pattern, as soon as almost every stand up comedian becomes reasonably successful he starts to travel a lot by plane and the gags start to revolve about that. We just got lucky with his take to make it meta enough, recognizable and hilarious at the same time.
I am.
Jesus! (1)
What are you going to make or you're making already to change the "suckiness" of programming? Or was your article (and the question to others) your contribution? You've got at least a lot of comments here (and a lot of disagreement too) now you have something to proceed.
1) John 8: 58 and http://hightechministries.org/Eight%20I%20Am%20of%20Jesus.pd...
the problem is, that doesn't pay for a roof and a meal.
so I spend my day fighting users that requires twenty seven pop-ups not to delete their own data accidentally, browsers that have each their own idea of css3, enterprise services that are a landmine of old documentation, bugs, workarounds, workarounds for bugged workarounds and bugs on top of these.
as my mentor once said, if work were fun they wouldn't have to pay you to do it.
A usual misconception, my fault, is to think I'm complaining about our profession. I'm not, I'm talking about programming, the tool.
Re: I love programming. Me too! but I think tools should be measured by their outcomes, not by how much we love them.
>What would programming be to you in an ideal world?
More visual, especially when observing execution.
When you're debugging anything non-synchronous—be it events, messaging, distributed systems—it usually sucks, even in the rare case that you do have good tooling.
I'd like to create, view and debug my code in an almost entirely visual fashion. I want to see events and messages perfuse through the structure of what I'm building, with complete temporal control of the view.
[0] http://www.chris-granger.com/2014/03/27/toward-a-better-prog...
I suppose that while visual programming excels at conveying topology and structure, it can be weak at a low level due to the tedium involved. Traditional code is arguably better suited for low-level grunt work, at least for most people.
A hybrid solution might be best. For example: a message travels from one high-level visual entity to another, and you'd easily be able to drill down into the code comprising those entities, locating the exact lines responsible for emission or receipt.
http://oro.open.ac.uk/35805/8/UML%20in%20practice%208.pdf
"Even if UML is viewed as the ‘de facto’ standard, it is by no means universally adopted. The majority of those interviewed simply do not use UML, and those who do use it tend to do so selectively and often informally."
"They also highlight some of the fundamental tensions within UML, resonating with arguments [8], [25], [17] that UML’s intended strengths (i.e., generality, accommodating different levels of abstraction) are intimately associated with its observed weaknesses (e.g., latent complexity, issues of transformation and coordination between views) and arise from fundamental properties of UML (e.g. lack of formal semantics, separation of expressions of structure and behavior)."
Whatever we do next, we have a nice set of examples and case studies of "what can go wrong."
The specific software tools based on UML are especially good examples of the problems.
As far as I'm aware, most visual programming today is actually implemented in the form of node-based editors, which gained popularity in the film and visual effects industries well over a decade ago.
The best modern examples I can think of are probably Nuke[0] and UE4's Blueprint[1] system.
[0] https://www.thefoundry.co.uk/products/nuke/
[1] https://docs.unrealengine.com/latest/INT/Engine/Blueprints/E...
https://en.wikipedia.org/wiki/Rational_Software
"The development of Rose 2.0 combined a Windows-based Booch notation editor called Object System Designer (acquired from Wisconsin-based Palladio) with a new intermediate representation, and with new semantic analysis, code generation, and reverse engineering capabilities. The latter, which allowed prospective customers to analyze existing C++ code to produce "as-built" navigable class diagrams, helped overcome Rational's late re-entry into the market for object-oriented modeling tools. Rose 2.0 ran on Windows PCs and on several Unix-based workstations."
It wasn't usable, except to be used in the projects doomed to fail. Or spend a lot of money. Which is something a goal of the managers. But it still can't be named a success.
UML didn't pick up for whatever reason and visual programming fell into the valley of deception. I expect that with more UI/UX research we can get to the plateau of productivity in the next decade.
Btw, there is a variant of UML that is executable. https://en.wikipedia.org/wiki/Executable_UML
If you believe it's a good way, you should certainly go that way. Please write about your experiences afterwards.
Rose 2.0, I've mentioned and quoted explicitly, produced the code from the diagrams (that's the "code generation" from the quote). And it was tragedy. Maybe you find somebody's experiences on the web.
That sounds quite positive compared to my experiences with code generated from UML.
Yeah. For any truly novel visual programming approach to succeed it'd probably require some next-level UI/UX work for sure.
Visual programming could be very well an alternative. Or fucking oculus vr or god knows why. I 100% agree with rl3, it proposes a very interesting and challenging problems for the UI/UX folks.
Also many people don't use what is available to them. Modern IDEs support single-stepping and painless runtime introspection, and many have supported it for 20 years.
Still people resort to trying to trace out in their head what happens runtime.
<https://www.youtube.com/watch?v=VZQoAKJPbh8>
A recent iteration <https://youtu.be/VZQoAKJPbh8?t=46m56s> looks wiki-like.
Sure, some engineers (especially junior ones) enjoy reinventing the wheel more than using something that already exists to solve users' problems. We all like to trick ourselves into thinking we're unique snowflakes (this problem extends into our personal lives as well). But more often than not, when multiple popular standards emerge, it's because there are legitimate engineering tradeoffs that are being made.
JSON vs. XML? XML is far more capable but complex. JSON is far simpler but less capable. Sometimes when you're delivering value to customers, you need that extra complexity, so you use XML. Other times you don't need it so you use JSON. It has nothing to do with ego or politics. Multiple different JSON libraries? One may prioritize ease of use to get up and running, the other may prioritize strong typing for serialization speed. One may prioritize strict compliance with the spec, the other might prioritize speed above all else. JSON vs. a binary format? JSON is more readily compatible and easier to debug. Binary is faster but more complex to setup. It goes on and on.
Again, sometimes competing standards or libraries or languages emerge because of political or capitalistic concerns. But usually when you're talking about competing open standards, there are multiple because there are legitimate engineering tradeoffs being made because engineering is not a one-size-fits-all science. (Few sciences are, otherwise they wouldn't still have people working on them.)
When an open source project rejects a contribution, or five projects exist to solve the same problem, they're often making legitimate engineering tradeoffs that are in no way arbitrary, and any one project would suffer to try to be every thing to every person. The article in question doesn't even point out a single example of a set of standards or libraries that are entirely arbitrary in their differences or could be collapsed into one solution, which further highlights how this is a theoretical argument, not a practical one.
But sometimes things are complicated/messy because we're trying to solve complex problems.
That's not to say that there isn't a simple solution to a particular problem nor that we should strive to find it.
But to assume that there are always simple solutions to complex problems is itself a form of ignorance.
I clarified that "Sometimes" the causes are political, it used to say "Usually".
But if after a 30 minute read you picked up on a stupid example and you started a discussion on XML vs JSON you are just proving my point. The expectation in the 60s was that machines will figure out protocol, even make protocol, on the fly, by asking about each other. Yet, you want to have another discussion about XML and JSON.
As for making protocols on the fly, that makes roughly as much sense as two people inventing their own language to talk to each other.
Turns out that people invented their own language by talking to each other.
> Of course we have never questioned any of this. We don’t have a direction, a path or an overall goal for programming!
> Let me ask you this: What would programming be to you in an ideal world?
> We don’t know! We’ve never asked ourselves this question!
When at the start of the "essay":
> Turns out, we are horrible at predicting the future and we end up discarding that extra complexity.
Too bad outsiders think (and will for a long time) it's a done thing. Society thinks software is the best thing since sliced bread so it's all around us. It's both a curse and a blessing.
Compared to other fields, CS or engineering is relatively new. It's exciting, it will keep breaking and evolving. And that should be it. It should keep going places without restrictions, precisely because we don't know.
Similarly to scientists ripping open humans on anathomical theatres in the 17th, we are having fun.
You hit the spot. Yes, who is it to say that we reinvent programming tomorrow and it useless by next week. Great point.
My answer is, don't look at it from a black and white perspective. I'm not aiming for a 100% future proof solution, that is impossible as you pointed out. But we can make something much more future proof that what we have right now.
We (who do we mean when we say "we", anyway?) don't have a direction, path or overall goal for life, society, the human race and so on. I don't even have a plan for what I'm going to have for lunch next Tuesday.
Programming is contingent and incremental because rationality is bounded and the future is uncertain. We don't know what will be successful in the next few years or how the world will want to use it or what it will be prepared to pay for.
We can build jewel-like systems that are perfectly adapted to their use case using technology to the utmost. This nearly always results in a stranded evolutionary dead-end. The BBC Domesday Project (live video database on an 8-bit micro using PAL laserdisc!) is my favourite example, but there are many.
Personally, I don't think we could reinvent programming and make it 100% future proof. But if we reinvented programming it will be much more future proof that what we have now. The reason is that our current stack was not invented in the first place, kind of evolved upwards organically. If we did something it will at least be purposeful.
Look at everything we built out of arbitrary conventions of boolean logic that has real impact and is shapping the modern world!
>>The computer shifted the complexity of our solutions from the world of atoms to the world of ideas.
And that's fucking awesome! Thanks everyone that was part in making that a reality!
Complexity can pile up in this new world, sure, but we are only starting to get our feet wet. Outside the boundaries of processing and storage, we are our only limitation. We can create things as complex as we can logically describe them.
It's also incredibly difficult to create something that will acomodate everyone else's solutions.
Software development is in its infancy. The fact that we can create absurd structures that still work is great because we can use them even if our skills are not mature enough to make truly elegant structures.
Saying programming sucks and wishfull thinking does not help anyone. Not happy? Start hacking on the next thing!
>>The computer shifted the complexity of our solutions from the world of atoms to the world of ideas. >And that's fucking awesome! Thanks everyone that was part in making that a reality!
I glad you are entertained with programming. I am too! but we should judge a tool like programming by its productivity, not by it's entertaining powers.
> We can create things as complex as we can logically describe them.
One of my points is that we can't. To make a single app facebook needs hundreds of engineers and it is still buggy. Humans are, yet again, the weaker link in software engineering. I believe we need fundamentally better tools.
> It's also incredibly difficult to create something that will acomodate everyone else's solutions.
Agreed. I'd very happy with a tool that is not general purpose that could churn though CRUD systems (vast majority of systems our there) in a matter of hours without errors. If you want to something else like AI, or games or whathaveyou, use other tools built specifically for that or a programming language if none are available.
That CRUD tool (gross oversimplification) could be expanded with new capabilities as long as we understand those new capabilities properly.
> Saying programming sucks and wishfull thinking does not help anyone. Not happy? Start hacking on the next thing!
I am, thanks for the encouragement.
The solution is totally obvious. The code we write has a form defined by it's input->output mapping over all possible inputs.
Write a function and you immediately define this input output mapping. All you have to do at this point is search through all previously written functions to find the one that most resembles the one you are trying to write.
Trying to write a nested for loop to pull out pixel values from a jpeg? Start writing it, shit you get it wrong - but the input output mapping is close enough to the cluster of previously written solutions that the correct code pops onto the screen.
Structure aware code search across all open source code would make a lot of the pain just go away.
1. specify with enough precision your requirements 2. search through all code for matches 3. vet those matches 4. incorporate it into your code-base
and in the end you don't want a bunch of spaghetti - you want a consistent and digestible architecture.
But I do think this type of coding will become mainstream within a few years. As you say, it's obvious.
This magical repository you seek is called "stackoverflow" :-D
However, programming may be lambda calculus at a theoretical level, but execution is a turing machine. All these little real world things that can be abstracted away in abstractions(!!), have to be unabstracted to meet a bunch of very real NAND gates somewhere, limited memory, resources etc.
Thats one reason it "sucks", but thats why its engineering and not math. Maybe one day computers will just be so powerful that we can afford to treat memory as infinite, runtime as infinitesimal, and build all the abstractions we want for free - then programming will be done :)
In this fun world, when i want to build something ill just draw a bunch of squiggly boxes and describe an alarmingly underspecified thingy i want done. Then the program will automatically figure out all possible inputs, quickly perform a "32 questions" kind of thing and resolve any ambiguity, then produce code. Managers will once again be confused why "adding this textbox is so hard", but this time they will be right - i will just tell lambda to go make a textbox.
Things are getting so complex for us that we find it difficult to service the requests.
Solution: We need someone to help us to solve the problems. Or more practical - we need intelligent tools which attempt to guess what we are really trying to do.
Far more intelligent autocomplete: "This looks like you are writting a postcode splitting regex, here is one that has been used by many programmers.". "Looks like you are adding a new business layer class - do you want to create an entity as well. Oh also it looks like this is going to part of the new view model you have been working on - do you want me to add a projection in for bl Class, mapping to the un populated properties on the vm?" etc.
We
> Hiring developers is an extremely difficult task. Learning to program is ridiculously difficult as well.
If programming was a solved thing, that would mean problem solving itself was a solved thing. It has by nature to be difficult. The question is more whether we manage to avoid additional difficulty on top of that. He's talking about the cost of bugs, but I posit the cost of burdensome frameworks and artificial bureaucracies in software development processes is by far bigger and harder to come to terms with, both of which exist in part to manage this inherent difficulty (often achieving the opposite).
> Lots of rework happens because of miscommunications with the user. Other times, the user doesn’t fully understand their problem
This is not unique to software development by any means. Every organization that deals with end users has to negotiate this interface.
> anywhere from 50 to 90% of the cost of building software goes towards maintaining it after the first release
Whether that figure is real depends largely on the type of software you're writing. If this fits what you're doing, the assumption that at some point the program is supposed to be "done" and every bit of time spent evolving it further must be a sign of inefficiency is simply unproductive. That kind of program is more usefully viewed as an evolving organism, and the sooner management recognizes the nature of this the better it will be handled by the organization.
> Our tools don’t make a good separation of essential and accidental complexity[2]––we are forced to crystalize the essential parts of our programs in a programming language and make them coexist with irrelevant, usually platform-specific code (accidental complexity).
This too depends on the nature of your chosen ecosystem. On the whole, interop has steadily improved throughout the history of computing. We have meaningful universal data formats now, and many options for moving code between platforms. Where interop troubles exist, they mostly exist on purpose (like the iOS example you made). You can absolutely solve most of this problem by throwing computing power at it and choosing an interpreted or cross-compiled environment if you want to avoid writing glue code.
And this was in the CSE stream which was for the less able children.
Worse, it's often accompanied with 'but profession X is so much better and they are all certified and do things properly unlike us awful cowboys!'. In the worst case you get the whole 'in the future everyone will program and programmers will become totally redundant'.
I have a feeling the developers writing these posts don't have a huge amount of experience, and have a natural tendency to want to turn the table up side down and start afresh to get rid of all this cruft. I understand it, but it's frustrating
But what they perceive to be the cruft isn't just cruft. In programming the possibility of starting afresh is there, and has been done several times - haskell, erlang, go, rust, clojure, etc. etc. etc. on the programming side. In the web world the front end is filled with the latest great solution that just pushes aside all this terrible accidental complexity, and this happens in other realms too, yet we still see these articles.
I think there are a number of things at play here:
1. Programming is hard and often in annoying, frustrating, and painful ways. Nothing you do can change that, and blaming it on tooling or legacy or whatever doesn't change the _intrinsic_ complexity at hand. People like to imagine it's hard in challenging and enjoyable ways, and sometimes it is, but more often it's just frustrating.
2. Abstractions won't save you. They help, but always involve trade offs of one kind or another. Something that is declarative enough hides imperative details which eventually leak in horrible ways unless you come to understand how they work, and something too imperative invites duplication, complexity and bugs. Having seen giant T/SQL stored procedures, Microsoft Access queries that take hours to run and excel sheets that are so complicated nobody can vet them, I have seen my fair share of collapsed abstractions which started wonderful.
3. Programmers are needed. A great example is excel - some people point to it as the philosopher's stone of the 'everyone is a programmer' world, but as soon as you start doing things it's not good at, or exceed a certain complexity, it explodes into horrible pieces and fails in all kinds of terrible ways. Then the programmers need to step in. It's not something special or conceited, personally I think only a certain % of the population are capable of thinking in a sufficiently logical way to understand what a computer needs, whether employed as 'programmers' or not, and these are the only people who can help beyond the simplest case. Nothing to do with intelligence, just a kind of thinking. The power of software + the rarity of this way of thinking combined with people actually using it is why programmers are expensive, it's not an anomaly imo.
4. Real world work is largely boring and/or painful. I think particularly here people miss this point, in fact PG wrote a great piece relating to this (http://paulgraham.com/schlep.html), but it's so important - even the most interesting problem involves a great deal of schlep to get things _nicely_. No matter how wonderful programming could possibly be made, there is no avoiding this. It's true not only of programming but any job. I think there's too much of a delusion sold by many that work can be nothing but a joy if only you pick the right one - no. It can _net_ be wonderful, but in that net are wins and losses. If you love programming, the kick of solving problems and seeing an application do something for somebody easily outweighs the pain of schlepping to that point.
5. Other professions aren't magically better. I did a civ eng degree at arguably the top civ eng school in the world, and worked in a practising office for a couple of years and believe me the insight I got from both the theo...
Yeah, not going to happen because not everyone can think in the level of detail required to program.
Some people just think programming is like waving a magic wand, but when you keep asking them "so, what should it do if this happens? What about if that happens?" they slowly start to understand...
What's frustrating is when programmers mistake inherent complexity for the accidental kind and therefore label the whole enterprise a terrible mess.
In fact overall I think this distracts from ACTUAL accidental complexity since if you think programming is generally terrible you'll probably not be too interested in some small details of that :)
These days it's trivial to create a CRUD web app that is even save against CSRF when you use a framework like Django.
But of course, nobody wants a simple CRUD web app anymore. It has to do business logic, it must have responsive design, usable on mobile, incrementally load pages, and be an enterprise integration framwork that checks with the credit card processor, the warehouse software etc.
It's just that we don't talk anymore about the things that have become trivial, because we don't spend much time on them.
So of course, in our perception, programming and tooling suck. And they always will. And that's OK, because if everything was breeze, we'd be bored.
We may be programming in our different teams, companies, organisations and countries, but the stuff we make makes an impact and affects the world around us.
This impact is an input into future similar projects and, given time, the technologies that results from wherever will be better for it. Please note that this idea does not mean that any group now is going to make these technologies, but what we do now will influence the future.
The same pattern has been observed in the industrial revolution - many tinkerers, manufacturers and inventors were all sort of learning from each other, all improving. This happened over decades. The main difference with programming is the time dimension. We've only really been doing it at a cultural level for the last twenty years - we're only just getting started in discussing and sharing what we are doing.
Programming isn't something that can be "solved" any more than math or writing is. There is inherent complexity in building apps and there are an infinite number that could be built. The fact that it hasn't been "solved" is no indication that we've "never asked ourself this question!"
No, you have to present your reasoning before I agree with you.