That you should not believe that classic engineering techniques work in software. Also, do not believe that you, as a software developer, know how engineering works in real world. You don't.
1. We don't need to know how engineering works in real world. You need to be in a domain and work in that domain to acquire that knowledge. Knowing how to calculate stress in civil/structural engineering doesn't mean you are an engineer. You need a job. You need an internship if you were a student.
2. It is quite dumb to assume we build software in the same way that bridge is built. That's essentially what Water Fall method would do. You build the blueprint precisely, build a prototype and then build the actual bridge. I don't see why the author would assume we think we were classical engineer. I never assumed we would be using the same technique all the time.
1. Well, you don't need to, but it's certainly interesting.
2. People assume all the time that software development is engineering. That's what leads to clashes between management and developers; clients not understanding why you can't estimate precisely; neverending projects; etc.
It's a very important thing to understand and explain that software development is _not_ an engineering discipline -- at least not yet. We strive to make it another branch of engineering, that's why we have design patters, methodologies, etc. Engineering is usually boring, but predictable, and that's a very important property software lacks at the moment.
Well, the classical engineering domains are all pretty different anyway (electrical, structural, civil, etc). I remember my grandfather saying he thought it was ludicrous that they were all covered by a single qualification.
Building software with an engineering approach is a real thing. Not everyone subscribes to it, but doesn't mean it isn't a real thing.
For the record, I do a lot of gardening too, and building software has fuck all to do with that.
I also don't get how everyone who argues against software-as-engineering talk like buildings and bridges never fall over. They do.
I have to agree strongly with this. It's like a professional trainer calling themselves a professor just because they teach. You are an engineer if your diploma says so. It is an earned title, not a job description. I say this as a CS dropout that never calls myself an engineer.
Engineering is about process, not about degrees. My CS degree did not at ALL prepare me to be a software engineer. It taught be to be a programmer. Engineering is more about the process than the technology. CS teaches theory and technology.
100% agrees, I didn't say I didn't do engineering, I just don't call myself an engineer. Ironically, I do call myself a programmer though, and I don't need a degree to do that. The state of CS degrees is a whole other topic, which from the sound of it we agree upon so far. Maybe I just can't call myself a scientist ;)
"Engineering is the application of scientific, economic, social, and practical knowledge in order to design, build, and maintain structures, machines, devices, systems, materials and processes. It may encompass using insights to conceive, model and scale an appropriate solution to a problem or objective. The discipline of engineering is extremely broad, and encompasses a range of more specialized fields of engineering, each with a more specific emphasis on particular areas of technology and types of application."
Application of scientific/practical knowledge: check. Design, build and maintain systems/processes, check. Using insights to conceive, model and scale a solution to a problem, check.
Sorry, but this is just a nonsense discussion imho. Stop devaluating your own grade.
I for one really like the article. Yes it's true engineering projects aren't anywhere near as perfect or on-time as people often given them credit for, but that's exactly the point. People often expect engineering project to fit a agh exile exactly. The article is about changing that perception and dealing with the reality that as an industry we don't estimate very well at all. So we need to change the metaphors to get people to understand more clearly what is I we are doing. And the Gardner metaphors is apt. Great Article!
It's a slightly flowery (haha) way of putting it, but I agree whole-heartedly. I recently held a talk at a Python conference and I asked the question: degrees and job titles aside, are we scientists, engineers, artists or a combination of all of these. The overwhelming response was "all of the above".
My preferred term is "artisan". A software developer is applying techniques acquired primarily through experience, supplemented by training and outside knowledge. The artisan has a clear vision of the end result in high-level strokes, but the actual process and implementation is strongly dictated by experience and intuition.
Obviously there are different branches of software developer that have more characteristics of engineering, others of science, but I would hazard to guess that 80% of software development is essentially a guided artistic process.
You should have said to them: 'you're only f* programmers'. Just as a joke. As a software engineer (?), I am quite fed up with people in their early 20s calling themselves 'architects', 'artist', etc.
You're not a scientist, and not an artist. Being a scientist and a programmer is extremely different because of a simple, seemingly minor thing: the lenght of the feedback loop. As a programmer, your feedback loop is sometimes an hour, sometimes a few minutes, sometimes a second (IntelliSense/other IDE help), sometimes weeks. But for scientist, it's sometimes forever. So it's for very different personalities.
Heh, well I actually say "developer", because "artisan" sounds douchey, "engineer" is (imo) incorrect and "programmer" tends to be a very narrowly defined role. Design and operational sides of a "developer" are not really adequately covered by "programmer".
But that's semantics really. We're talking about what sort of techniques do you employ. It doesn't really help to say that programmers use "programming" techniques because it says nothing about the common attributes with other industries / professions.
Which is where "artisan" comes in. A good developer is, IMO, more like an great chef - excellent technique, a wealth of experience across many disciplines and the best tools are the (ahem) recipe for success, not "good process".
I think "Process Implementer" is best. I can see both a mathematics and engineering of processes. Processes instead of computer, because it is more general and applicable to fields like biology.
Also computer science might be the worst -- there is zero science in CS. Science has to do with nature![0]
That makes very little sense. I'm not saying all developers are scientists but the length of the feedback loop as absolutely nothing to do with the scientific method, which is the only real defining character of a scientist.
Like in most professions, there are multiple feedback loops of varying length. A lot of the accidental complexity in each profession arises from the shearing effect between the different loops.
Unfortunately "programmer" has come to be synonomous with "fancy typist" by the people who pay the bills. "Engineer", "architect", and "developer" all imply someone who codes with a focus on X, whether it be low level reliability, high level design for scalability, or general competence and requirements gathering skills.
No that's not right. I would rather state software developers are not scientists since they don't have to conform to a scientific peer review process (journals etc.). (Please don't come with code review as peer review... just don't).
Just the same software developers are not engineers since they don't have to uphold an industry certification process nor are they held to public safety and ethic standards.
I think this [silly] argument is nothing more than a special case of the argument over whether words have inherent meaning, or derive meaning from usage. I don't think 'software engineer' was ever meant to denote a new subtype of engineer. I think 'software engineer' is a compound word that means whatever it is we do. At work, whenever someone says 'get an engineer to look at that', absolutely nobody thinks that means to call one of the 'real' kinds of engineers on the phone. They know it means to get a software developer/programmer/code monkey/dev or whatever you want to call people who make software out of computer code.
It is far more than semantics, the term engineering denotes a rigid adherence to processes and protocols, whereas 90% of people that call themselves "software engineers" are just developers at best - they write code to do what they are told it should do with very little concern for the overall processes involved in an engineering discipline.
This misses my point completely. My point is that the term 'software engineering' is not meant to denote a type of engineering. It may include engineering as a loose metaphor, but the term 'software engineering' stands alone as a word in itself, with its own meaning, and that meaning is obvious to anyone who is or works with software engineers.
More analogies: Should we not have software architecture, because it isn't real architecture? Should we not refer to systems programming, unless the practitioner has studied systems theory? Should we not have windowing systems, because no real windows are involved? Almost everything in computing is termed by metaphor with 'real' things.
It's closer to the British use of the word engineer, which can mean anything from the civil engineer who designed the Channel Tunnel, to the mechanic who repairs the trains. I think I prefer the prescriptive definition, though, it's useful to have a word for someone expected to work with that level of planning and rigour.
I wish we took the engineering metaphor more seriously. In particular, I'd like to see some real consequences for malpractice on the part of practitioners -- this would help us in our struggles with management to get projects done successfully.
I've seen so many cases, for instance, where people screw up simple things, such as generating primary keys, and keep making the same mistakes over and over again. This has got to stop.
> In particular, I'd like to see some real consequences for malpractice on the part of practitioners
I agree as long as we also see some of the benefits - required registration with a professional body - along with a requirement that certain projects cannot use uncertified practitioners (in other words, better pay for Software Engineers).
> If you were building a bridge or a skyscraper and you told me, before you began, that you knew exactly how it would look when it was finished – I would believe you. If you told me that you knew to some insane degree of accuracy how long it would take to get to ‘finished’ – I would believe you again. That’s how Engineers roll
Oh neat, maybe he can explain why the new Bay Bridge was practically a decade over schedule and billions over budget.
Or the Burj Khalifa was about 2 years overdue?
Or the California High Speed rail is projected to be twice the budget they thought, and they've barely broken ground on it?
So the idea that "Real Engineers" building bridges and railroads and skyscrapers get it right in plans and schedule is total bullshit. We've seen larger software projects, with more people working on them than these examples, hit their deadlines (Windows comes to mind -- except Vista).
And by the way, those "Engineering" disciplines have only been around for, what, hundreds, if not thousands of years. What they do for a living is mostly explored territory. Software, at least at a scale to be considered engineering at all, has been around for about 3-4 decades... maybe. And we're still hitting up against major unknowns all the time at every level of the practice, big and small.
I agree with you that the author of this article doesn't know much about how engineering projects really get done, but I have to disagree with your claim that software engineering has only been around for "3-4 decades... maybe". In the 1960s, IBM was building its OS/360 operating system, the massive project that inspired Brooks' famous book The Mythical Man Month. The Multics operating system (which inspired Unix with ideas like a hierarchical file system and an OS written a high level language) was started in 1964. Lisp, Algol and Fortran were all implemented in the 1950s. So software engineering has been around for at least 6 decades.
You are correct, but I would point out that this does not detract from the argument considering the time scale. Software engineering could be 150 years old, and still not have the close to the history of traditional engineering.
Is it always? In dollar amounts, I really do not know that it always is. "Don't let the perfect be the enemy of the good" and all. As a developer, I am always going to be the one push for better tests, more efficient systems, etc. but pg's post a month or two ago about scaling without automation (very rough paraphrasing) points to the fact that this does not always hold.
> The latest trendy programming language won't make your hard problem all that much easier to solve.
Maybe not the latest, but I would think that it was way easier to develop a desktop app in C++ than Fortran, develop a website in PHP than C++, develop a web app in Ruby than PHP, etc. Tech definitely improves over time. Jumping on every third bandwagon seems to be worthwhile.
> A complex system can't be properly integrated and tested in two weeks (as recent news reports have shown).
> The CEO setting an arbitrary deadline doesn't magically alter the laws of physics.
These are not engineering problems, but political/administration problems that have been around forever and in every endeavor. I am certain that stone masons building cathedrals dealt with the same sorts of incompetent management, despite a dedication to craftsmanship and engineering.
We don't have three generations of programming, really.
The majority of people involved in making decisions and programming right now haven't been doing this professionally for 30+ years. And for those who have (I'm closing in on 20), the culture that they grew up in was not shaped by people with experience programming.
The examples of "programming" and "lessons" from IBM/etc in the 1960s hardly affected anyone outside of those few small teams with respect to learning lessons re: software development.
IMO, "modern" development didn't really start until the early-mid 90s, when networking really started to take off at the small business and consumer level. This has meant that people who'd never dealt with making software decisions before now are - most of the clients that I have who are in their 30s and 40s have had 'website' duties thrown in their laps, and it's just another item on a checklist.
The lessons and timetables are still new to these people - certainly the mentors and bosses they had 15 years ago never had to deal with this stuff, so they were never exposed to it. The last 15-20 years have been huge in terms of businesses getting crash courses in software experience (whether they wanted it or not). It'll be another 15-30 years (imo) before these maxims are taken to heart by organizations when planning, because they'll have had 2-3 generations of institutional successes and failures to draw on. MOST orgs don't have that yet.
"And we're still hitting up against major unknowns all the time at every level of the practice, big and small."
I was with you until you got to this line. Let's be brutally honest: the limiting factor for most of today's software projects (i.e. CRUD websites and iPhone apps) is not technology. It's developer competency, experience and self-discipline. I'd go so far to say that most working software developers don't even understand the fundamentals of the technologies they're using -- it's why you see people replacing their relational databases for key-value stores, then trying to re-invent database join algorithms from scratch in the application layer. Or why every new generation of "engineers" re-discovers asynchronous programming in a new language (then promptly writes a framework in Blub...because Blub is so much better than last year's Blub.)
Truthfully, it's pretty damned hard to find gigs working on the frontier of technology. Most projects are predictable, tedious slogs to fit together well-understood technologies without shooting yourself in the foot. And yes, requirements shift and clients get fussy, but that's true of any engineering discipline. You learn to pad the schedule to compensate for the shifts. It's part of being a professional.
As far as I can see, commercial software isn't limited by "major unknowns", so much as an industry-wide unwillingness to learn from experience. But hey...that tends to happen in industries when you're considered "old" at 30....
Granted, most engineers are not exactly employing quantum physics to parking lot design. Most people, in general, work on gluing together pre-built parts that they do not understand all of the fundamentals of.
Ah man, this hit slightly hard as someone who just moved from Postgres to MongoDB to avoid database migrations only to realize that I've now lost the power of a many to many relationship.
As far as I know, the best migration tool for SQLAlchemy is Alembic, and it can't magically handle migrations (which is what I wanted). I jumped technology because it seemed like MongoDB did (and still does) meet my use case better. I'm open to any opinions or advice though.
MongoDB is schemaless, so you don't have to do traditional migrations. Unless you're making a joke about me migrating to a different database to avoid migrations, in which case, good joke.
Migrations are about data, not schema. You don't need migrations with a "schemaless" database at the beginning, but as soon as you have data you care about, you're gonna need to ensure some sort of consistency.
I think you should learn the relational model (not SQL). Understand the problems it was trying to solve and how the relational model solves them. Then figure out how that applies to your app. Database in Depth was helpful for me http://www.amazon.com/Database-Depth-Relational-Theory-Pract... .
Yes, you should move back to Postgres. The reason is you can do many more operations on structured data than on structure-less data. You found that out already with the many-to-many thing. There are plenty of other intangibles that will bite you in the ass later.
You aren't wrong, MongoDB is schemaless in the sense that you don't need to define a schema to use Mongo. But to use any datastore, you'll be defining a schema somewhere - generally, in your app itself.
Depends how you define database. Perhaps it's more precise to say that Mongo is a schemaless key-value store, but in truth it provides many functions that people think of as a "database", and the differences between it and traditional RDBMS seem inessential to what distinguishes a database from other things.
MongoDB guys call a "database migration" something like "adding a column to a table". They'll tell you it's an incredibly complex and time consuming thing to so, AND THAT"S WHY YOU NEED MONGO!
Of course out here in the real world, an "alter table" statement takes no time at all even on a large table, and can be done while the system is running.
Now I am not saying that MongoDB has no uses at all; but I am saying that the people around it don't really have much of a clue about databases in general and aren't in any sort of a position to be choosing technologies.
I downvoted you before I realized that you coming at this from a position of ignorance rather than ego.
Mongo is a way of storing 'stuff'. You don't have to define the form that the stuff takes before you shove it in the database. This makes things easier by lowering the knowledge barrier to getting started, which is a double-edged sword.
What I've gathered is that the tradeoff of not having to deal with schema migrations now is something I'll have to pay for with data integrity down the road. Do you think that's a good summation?
> As far as I can see, commercial software isn't limited by "major unknowns"
I'm sorry, I needed to get in an afternoon run and actually deleted a paragraph that explained this a little more that I couldn't get just right before posting.
I completely agree with you. There aren't a lot of technical unknowns for most things people are trying to do.
Thing is, I wasn't actually talking about technical challenges with that point, rather, the requirements that are given at the beginning of a project. The requirements for the new Bay Bridge, for example, are pretty darn clear: make a new bridge from Oakland to Yerba Buena Island.
The requirements for making G+ attract Facebook users... errr... uhh...
Or how about the requirements for the next Call of Duty to be fun...
A great many of today's software projects -- especially those discussed here on HN -- aren't straight up engineering problems, they're more design and UX problems. Even for something like the software for the Joint Strike Fighter, you're more into UX than hard-hittin' avionics for a lot of it these days.
And I think that's what makes it hard to predict software development schedules. If we were all building ATM software again and again, we'd be knocking it out of the park. A competitor to Snapchat that is able to sway users over... that might be harder to define up front.
"Real" engineering has the commonality with software engineering that many projects are quite similar to previous projects and are thus fairly predictable. (This is true also of safety factors; I read a good book on it not too long ago[0]).
A bog ordinary 12 story building will probably come in close to schedule and budget because it's a well-understood problem domain and there is a large pool of existing examples to draw data from.
But the Burj Khalifa required solving some genuinely novel problems, of the sort which only turn up at that scale. When a project involves substantial R&D, schedules become much less certain.
If my business is churning out Wordpress brochureware, I can give a pretty good estimate of how long a given job will take and how much it costs.
If on the other hand my customers require me to develop novel, publishable algorithms, then any estimate I give will need to have very wide bands.
I agree with you. My post was only really trying to promote Agile over Waterfall. Never intended for it to be taken so literally. I hope people enjoy reading it regardless.
It certainly makes for good HN discussion fodder :)
FWIW, I've had the "Real Engineering" discussion before -- way more in depth than you illustrated here. My grandparent response above is probably a bit of pent-upness towards those discussions more than anything about your article in particular.
>So the idea that "Real Engineers" building bridges and railroads and skyscrapers get it right in plans and schedule is total bullshit.
A few examples do not an argument make. You don't think in the "law of long numbers" sense.
If you did, you'd find out that MOST engineering projects do end on time and budget, and even if they miss, they miss far more gracefully and with extremely better predictability of outcome that MOST software projects.
That you can find some counter-arguments doesn't mean much -- what TFA proposes holds in general, and holds much better than the alternative propositions (that engineering is as fucked or more fucked than software). Of course there will be several counter-examples, it doesn't have to be perfect to be near-perfect compared to software schedulling.
>And by the way, those "Engineering" disciplines have only been around for, what, hundreds, if not thousands of years. What they do for a living is mostly explored territory.
"Or the California High Speed rail is projected to be twice the budget they thought, and they've barely broken ground on it?"
A correction: They've broken no ground on it. An AP article today reports that the "California High-Speed Rail Authority is in escrow on just one parcel of the 370 it needs" for the first 30 miles. http://abclocal.go.com/kabc/story?section=news/state&id=9320...
And the entire effort is in trouble: "Sacramento County Superior Court Judge Michael Kenney ruled that California's high-speed rail authority had violated the letter of the 2008 ballot initiative authorizing $10 billion in bonds for the 500-mile train's construction." http://online.wsj.com/news/articles/SB1000142412788732410820...
I actually agree with the parent comment re: engineering but wanted to make sure that folks didn't think ground had been broken already on California's high speed rail plan. I wasn't sure and had to check myself.
I'm pretty sure that the kind of engineering that designs circuits to do things in hardware that you'd normally do in software can't be that much different than writing software. A title that I find more ridiculous than software engineer is software architect. Architects are partly like engineers, but they also design things to look pretty while serving their function. Making software look pretty on the inside is utterly pointless, because users only see what the software does, not what the design looks like. Designing software is a lot more like engineering than architecture.
The skyscrapers and bridges that are finished on time and on budget are the ones that have been built before. For physical structures, it makes sense to do the same work again, because then you have two bridges or two skyscrapers instead of only one. People can live in two different locations, and they can cross yet another river, which is useful.
For software, if you need another copy, that's a trivial task. It's more on schedule and budget than any skyscraper or bridge could ever hope to be. Once built, we can use it everywhere. The only thing that requires any real effort, is building that very first skyscraper.
I come from a civil engineering background. The problems you cite are not engineering problems - they're management problems. In the environments I've worked in, heads would have rolled for the over-runs/over-dues you mention. We call these NFE situations.
BTW - thank you for mentioning (indirectly) that civil engineering is the world's oldest profession. ;-)
I disagree. When I code for my own amusement I might see myself as an artist but when I do professional work I definitely want to see myself as an engineer as well as a craftsman.
To me it feels as if we are just at a very early stage. It takes huge amounts of time to learn software engineering, putting aside mastering it and humans haven't been practicing for very long. So I'd guess that we're just clueless enough to not be able to feel like engineers yet. We learn most of our practical abilities from experience so thats an argument for craft, but once we get a better understanding of what we are actually doing, and formalize it, our job will probably be very engineery. There is just millions of problem domains and no formal education on the domain of writing software yet (no a CS degree is not it). And I think there is a class of applications that are already so well understood that they do get developed in an engineering fashion already.
I was hoping that a more substantial point was going to be made than this piece of fluff. Here is the substantial point.
If you're building a skyscraper or a bridge, an engineer has to sign off on the design. If the design subsequently proves to be defective, said engineer is personally liable. In many places, calling yourself an engineer when you are not licensed to sign off on designs and be liable in this way is against the law.
We can debate whether or not extending this principle to software development is a good idea. What we can't debate is that software development does not currently work this way. You, the software developer, are not an engineer. If you think a design is bad, you do not have both the legal authority and responsibility to stand up, declare it so, and force the design to be changed to something saner.
But saying you're not an engineer because software projects don't succeed like projects done by real engineers - that's both silly and wrong.
There have been rumblings over the past decade or two about quantifying a practice and study aimed towards turning software engineering into a capital-e Engineering discipline, but evidently they never really go anywhere.
That's what makes it a stupid point: it depends on a specific definition of engineer that is not used (and is not possible to use) with software engineering, and that definition is oriented around chartering/certification of the professional themselves.
That he buttons the essay up in such a self-serving way is just icing: taking "engineering" too literally for the context, then calling himself a gardener.
It has already been extended into software development. Plenty of software projects have to be signed off in aerospace, automotive, telecoms, energy and defense.
In fact I'd be amazed if it did not extend into consumer electronics in large companies, e.g. Apple et al cannot risk bricking millions of phones with a bad update.
The point might be that there are thousands of badly run projects, just as there are in construction.
I said they didn't have the same soil. That would include the multitude of internal tools they use to facilitate their software development, most of which are not publicly available, their culture, and the aptitude of their people.
First, software engineers exist in a broader culture. The connotations of the various words in our respective languages are not in our control. "Engineer" and "gardener" have very different connotations. In a profession that sometimes struggles to gain respect, it would not behoove us to use a term that (wrongly in my opinion) is associated with lower social and professional status. If we could rewrite the English language and define "gardener" as a person with rare, hard-to-learn, and highly valuable skills, then perhaps we would do well to embrace the term "software gardener." But we can't.[1]
Second, software is engineering. This article seems to define engineering as Big Design Up Front. While that may describe part of what engineering can entail, it's not a complete or general description. If I had to describe engineering generally, I'd say it has to do with understanding and manipulating complex systems, analyzing and fulfilling requirements, distilling vague notions to concrete implementations, and building things that work. Software fits all of that.
[1] As I hinted above, I don't think it's fair, right, or accurate to denigrate actual gardeners as compared to software engineers. I'm stating the way most people perceive social status, not the way I'd like them to.
In August 1991 during the S&L recession and fresh out of Vocational School, I landed my first CAD jockey job drawing fabrication plans for a precast prestressed concrete manufacturer. The first task, after running Diazos and learning that neckties would become caught in the feed and where the reverse button was by feel as my face was pressed to the top of the machine and my air supply was rapidly constricting was drawing plans for a series of double tee members [TT]. It took a couple of weeks to generate the 100 odd variations and get them through the QA process, but then I moved on to other things.
But come October, they started to roll off the line...20+ kips of concrete a pop and then stacked three high. To this day, the sensation is fresh; "So that's what the fuck it really looks like."
The reason it's fresh is because it returns every time I am in the presence of some lines I put on paper manifested.
The picture of the bridge is not the bridge. The engineer, like the hacker, has better intuitions regarding the way in which the picture will correspond to its instantiation - but it is still no more than a [hopefully] informed intuition.[1]
[1] The idea that there is an Engineer rather than a thousand authors of a successful bridge tends to be a bit absurd in modern construction practice. Such heroics are still far more likely in software development.
The fact that the feedback loop of a software engineering process is much tighter and cheaper than that of civil engineering does not mean that it is not engineering - that's a fallacy.
Perhaps it's less applicable to civil engineers, because of the scale and safety requirements of their profession, but let's look at electrical engineering. An electrical engineer will build prototypes; they will build models; they will use computer simulations to predict how systems will behave; they will build testing and QA infrastructure. It's definitely harder to iterate—once you've built a million widgets, shifting circuits around is going to be expensive—but it's not qualitatively different to developing a software product.
As in any engineering discipline, software engineers must optimise for a system's required qualities. In some cases, reliability might not be that high up the list. In others—let's say medical systems, or those used in space travel—software engineers have rigorous systems in place to ensure quality, and in many cases substantial up-front planning and certification will be involved.
Ultimately this seems to say that software engineering is different from (specifically) civil engineering. I think that's broadly rather true – the nature of civil is such that generally there are stringent safety requirements, and indeed massive costs involved in making subsequent changes. The majority of software does not have the same requirements, and so it's no surprise that the process is somewhat different. And I agree that not all development is engineering, of course, but to dismiss the entire field is a bit of a shallow argument.
Bleh... Can we just stop with the metaphors already? Maybe I'm not an engineer or a gardner or a scientist but maybe I am. I can be all of those things or none of them. I know how to make software of varying types and that's enough, how I apply those skills is a completely different thing. Maybe I'm a biologist using python to help see the migration patterns of fish better. Maybe I'm a web developer building highly scalable web systems in java. Maybe I'm writing drivers at a kernel level. It doesn't matter, programming is a skill set not a job definition.
Engineers aren't just Engineers. They are Mechanical Engineers, Electrical Engineers, etc.. Scientists are just Scientists, they are Biologists, Chemists, and Computer Scientists. When we want programming/software development to have that same level of social standing we will use programming as a skill set not a job title.
Changing requirements and development landscape do not an analogy to a gardener make.
I do not 'grow' nor 'tend' code, trusting some unknown universal force to keep me on the right path.
I engineer buildings, bridges, roads, tunnels - all sorts. Sometimes they're very small and sometimes they're very large but they're all subject to the forces pushing and pulling on them - my job is to build them and keep them upright as well as I can subject to the laws and constraints of the environment.
I am not a gardener. I am an engineer building, modifying, and destroying complex structures daily in a controlled dance so as to keep their superstructure useful and safe.
Only to the naive eye could such complexity be reduced to such apparent simplicity.
I would like to see him “growing” a database or a compiler, perhaps a firmware for a car or the CAD software used by the architect to design a skyscraper.
To me, engineering is about solving problems, it's about taking systematic analytical approach to whatever problem arise. It is a state of mind, and that is why engineering now covers everything from software to biochemistry.
I grow tired of the endless comparisons of software development with some other practice, profession, or physical process.
All analogies eventually break down if you take them too far. (If they didn't then you wouldn't be comparing something to something else; you'd be comparing something with itself.) But software development is so different from so many other practices and processes that most comparisons to gardening, building, finances, etc. don't provide much real value.
Its especially tiresome when it's claimed that "sofware isn't like that, it's like THIS". In the article, I find that neither gardening nor engineering is an especially enlightning analogy to software development.
I'm a software hacker that gets paid an engineer's salary.
I use old tools (vim, terminals), like engineers use old tools (protractor, pencil), to make comparatively much larger artifacts (search products, management tools), like engineers (buildings, bridges).
You're not judiciously applying the metaphor. You've only rephrased what it means to be "self-taught"; it goes without saying that we don't allow unlicensed people to engineer things. It also goes without saying that no one lives or dies by a faulty HTTP request header.
133 comments
[ 2.9 ms ] story [ 114 ms ] thread2. It is quite dumb to assume we build software in the same way that bridge is built. That's essentially what Water Fall method would do. You build the blueprint precisely, build a prototype and then build the actual bridge. I don't see why the author would assume we think we were classical engineer. I never assumed we would be using the same technique all the time.
It's a very important thing to understand and explain that software development is _not_ an engineering discipline -- at least not yet. We strive to make it another branch of engineering, that's why we have design patters, methodologies, etc. Engineering is usually boring, but predictable, and that's a very important property software lacks at the moment.
Building software with an engineering approach is a real thing. Not everyone subscribes to it, but doesn't mean it isn't a real thing.
For the record, I do a lot of gardening too, and building software has fuck all to do with that.
I also don't get how everyone who argues against software-as-engineering talk like buildings and bridges never fall over. They do.
Saying it's just "whatever engineers do" is meaningless.
"Engineering is the application of scientific, economic, social, and practical knowledge in order to design, build, and maintain structures, machines, devices, systems, materials and processes. It may encompass using insights to conceive, model and scale an appropriate solution to a problem or objective. The discipline of engineering is extremely broad, and encompasses a range of more specialized fields of engineering, each with a more specific emphasis on particular areas of technology and types of application."
Application of scientific/practical knowledge: check. Design, build and maintain systems/processes, check. Using insights to conceive, model and scale a solution to a problem, check.
Sorry, but this is just a nonsense discussion imho. Stop devaluating your own grade.
My preferred term is "artisan". A software developer is applying techniques acquired primarily through experience, supplemented by training and outside knowledge. The artisan has a clear vision of the end result in high-level strokes, but the actual process and implementation is strongly dictated by experience and intuition.
Obviously there are different branches of software developer that have more characteristics of engineering, others of science, but I would hazard to guess that 80% of software development is essentially a guided artistic process.
You're not a scientist, and not an artist. Being a scientist and a programmer is extremely different because of a simple, seemingly minor thing: the lenght of the feedback loop. As a programmer, your feedback loop is sometimes an hour, sometimes a few minutes, sometimes a second (IntelliSense/other IDE help), sometimes weeks. But for scientist, it's sometimes forever. So it's for very different personalities.
But that's semantics really. We're talking about what sort of techniques do you employ. It doesn't really help to say that programmers use "programming" techniques because it says nothing about the common attributes with other industries / professions.
Which is where "artisan" comes in. A good developer is, IMO, more like an great chef - excellent technique, a wealth of experience across many disciplines and the best tools are the (ahem) recipe for success, not "good process".
Also computer science might be the worst -- there is zero science in CS. Science has to do with nature![0]
[0] R. Feyman: http://www.youtube.com/watch?v=lL4wg6ZAFIM
Just the same software developers are not engineers since they don't have to uphold an industry certification process nor are they held to public safety and ethic standards.
More analogies: Should we not have software architecture, because it isn't real architecture? Should we not refer to systems programming, unless the practitioner has studied systems theory? Should we not have windowing systems, because no real windows are involved? Almost everything in computing is termed by metaphor with 'real' things.
I wish we took the engineering metaphor more seriously. In particular, I'd like to see some real consequences for malpractice on the part of practitioners -- this would help us in our struggles with management to get projects done successfully.
I've seen so many cases, for instance, where people screw up simple things, such as generating primary keys, and keep making the same mistakes over and over again. This has got to stop.
I agree as long as we also see some of the benefits - required registration with a professional body - along with a requirement that certain projects cannot use uncertified practitioners (in other words, better pay for Software Engineers).
Oh neat, maybe he can explain why the new Bay Bridge was practically a decade over schedule and billions over budget.
Or the Burj Khalifa was about 2 years overdue?
Or the California High Speed rail is projected to be twice the budget they thought, and they've barely broken ground on it?
So the idea that "Real Engineers" building bridges and railroads and skyscrapers get it right in plans and schedule is total bullshit. We've seen larger software projects, with more people working on them than these examples, hit their deadlines (Windows comes to mind -- except Vista).
And by the way, those "Engineering" disciplines have only been around for, what, hundreds, if not thousands of years. What they do for a living is mostly explored territory. Software, at least at a scale to be considered engineering at all, has been around for about 3-4 decades... maybe. And we're still hitting up against major unknowns all the time at every level of the practice, big and small.
edit: grammar
- Accumulation of technical debt is bad.
- The latest trendy programming language won't make your hard problem all that much easier to solve.
- A complex system can't be properly integrated and tested in two weeks (as recent news reports have shown).
- The CEO setting an arbitrary deadline doesn't magically alter the laws of physics.
Is it always? In dollar amounts, I really do not know that it always is. "Don't let the perfect be the enemy of the good" and all. As a developer, I am always going to be the one push for better tests, more efficient systems, etc. but pg's post a month or two ago about scaling without automation (very rough paraphrasing) points to the fact that this does not always hold.
> The latest trendy programming language won't make your hard problem all that much easier to solve.
Maybe not the latest, but I would think that it was way easier to develop a desktop app in C++ than Fortran, develop a website in PHP than C++, develop a web app in Ruby than PHP, etc. Tech definitely improves over time. Jumping on every third bandwagon seems to be worthwhile.
> A complex system can't be properly integrated and tested in two weeks (as recent news reports have shown).
> The CEO setting an arbitrary deadline doesn't magically alter the laws of physics.
These are not engineering problems, but political/administration problems that have been around forever and in every endeavor. I am certain that stone masons building cathedrals dealt with the same sorts of incompetent management, despite a dedication to craftsmanship and engineering.
Mortgages are debt, are they bad?
Yes. Yes they are. Many people make a huge bet on the housing market with borrowed money, without even realizing what they're doing.
The majority of people involved in making decisions and programming right now haven't been doing this professionally for 30+ years. And for those who have (I'm closing in on 20), the culture that they grew up in was not shaped by people with experience programming.
The examples of "programming" and "lessons" from IBM/etc in the 1960s hardly affected anyone outside of those few small teams with respect to learning lessons re: software development.
IMO, "modern" development didn't really start until the early-mid 90s, when networking really started to take off at the small business and consumer level. This has meant that people who'd never dealt with making software decisions before now are - most of the clients that I have who are in their 30s and 40s have had 'website' duties thrown in their laps, and it's just another item on a checklist.
The lessons and timetables are still new to these people - certainly the mentors and bosses they had 15 years ago never had to deal with this stuff, so they were never exposed to it. The last 15-20 years have been huge in terms of businesses getting crash courses in software experience (whether they wanted it or not). It'll be another 15-30 years (imo) before these maxims are taken to heart by organizations when planning, because they'll have had 2-3 generations of institutional successes and failures to draw on. MOST orgs don't have that yet.
I was with you until you got to this line. Let's be brutally honest: the limiting factor for most of today's software projects (i.e. CRUD websites and iPhone apps) is not technology. It's developer competency, experience and self-discipline. I'd go so far to say that most working software developers don't even understand the fundamentals of the technologies they're using -- it's why you see people replacing their relational databases for key-value stores, then trying to re-invent database join algorithms from scratch in the application layer. Or why every new generation of "engineers" re-discovers asynchronous programming in a new language (then promptly writes a framework in Blub...because Blub is so much better than last year's Blub.)
Truthfully, it's pretty damned hard to find gigs working on the frontier of technology. Most projects are predictable, tedious slogs to fit together well-understood technologies without shooting yourself in the foot. And yes, requirements shift and clients get fussy, but that's true of any engineering discipline. You learn to pad the schedule to compensate for the shifts. It's part of being a professional.
As far as I can see, commercial software isn't limited by "major unknowns", so much as an industry-wide unwillingness to learn from experience. But hey...that tends to happen in industries when you're considered "old" at 30....
Of course out here in the real world, an "alter table" statement takes no time at all even on a large table, and can be done while the system is running.
Now I am not saying that MongoDB has no uses at all; but I am saying that the people around it don't really have much of a clue about databases in general and aren't in any sort of a position to be choosing technologies.
Mongo is a way of storing 'stuff'. You don't have to define the form that the stuff takes before you shove it in the database. This makes things easier by lowering the knowledge barrier to getting started, which is a double-edged sword.
I'm sorry, I needed to get in an afternoon run and actually deleted a paragraph that explained this a little more that I couldn't get just right before posting.
I completely agree with you. There aren't a lot of technical unknowns for most things people are trying to do.
Thing is, I wasn't actually talking about technical challenges with that point, rather, the requirements that are given at the beginning of a project. The requirements for the new Bay Bridge, for example, are pretty darn clear: make a new bridge from Oakland to Yerba Buena Island.
The requirements for making G+ attract Facebook users... errr... uhh...
Or how about the requirements for the next Call of Duty to be fun...
A great many of today's software projects -- especially those discussed here on HN -- aren't straight up engineering problems, they're more design and UX problems. Even for something like the software for the Joint Strike Fighter, you're more into UX than hard-hittin' avionics for a lot of it these days.
And I think that's what makes it hard to predict software development schedules. If we were all building ATM software again and again, we'd be knocking it out of the park. A competitor to Snapchat that is able to sway users over... that might be harder to define up front.
So in an abstract since, and in a world where engineers are the only ones involved in construction, I understand his point.
And software development is never involved in any of that?
While I suspect software is more often plagued by problems I do not see that it is another kind of problems than other forms of engineering.
A bog ordinary 12 story building will probably come in close to schedule and budget because it's a well-understood problem domain and there is a large pool of existing examples to draw data from.
But the Burj Khalifa required solving some genuinely novel problems, of the sort which only turn up at that scale. When a project involves substantial R&D, schedules become much less certain.
If my business is churning out Wordpress brochureware, I can give a pretty good estimate of how long a given job will take and how much it costs.
If on the other hand my customers require me to develop novel, publishable algorithms, then any estimate I give will need to have very wide bands.
[0] http://chester.id.au/2013/07/07/review-to-engineer-is-human-...
FWIW, I've had the "Real Engineering" discussion before -- way more in depth than you illustrated here. My grandparent response above is probably a bit of pent-upness towards those discussions more than anything about your article in particular.
A few examples do not an argument make. You don't think in the "law of long numbers" sense.
If you did, you'd find out that MOST engineering projects do end on time and budget, and even if they miss, they miss far more gracefully and with extremely better predictability of outcome that MOST software projects.
That you can find some counter-arguments doesn't mean much -- what TFA proposes holds in general, and holds much better than the alternative propositions (that engineering is as fucked or more fucked than software). Of course there will be several counter-examples, it doesn't have to be perfect to be near-perfect compared to software schedulling.
>And by the way, those "Engineering" disciplines have only been around for, what, hundreds, if not thousands of years. What they do for a living is mostly explored territory.
That just reinforces the original posts thesis.
A correction: They've broken no ground on it. An AP article today reports that the "California High-Speed Rail Authority is in escrow on just one parcel of the 370 it needs" for the first 30 miles. http://abclocal.go.com/kabc/story?section=news/state&id=9320...
And the entire effort is in trouble: "Sacramento County Superior Court Judge Michael Kenney ruled that California's high-speed rail authority had violated the letter of the 2008 ballot initiative authorizing $10 billion in bonds for the 500-mile train's construction." http://online.wsj.com/news/articles/SB1000142412788732410820...
I actually agree with the parent comment re: engineering but wanted to make sure that folks didn't think ground had been broken already on California's high speed rail plan. I wasn't sure and had to check myself.
For software, if you need another copy, that's a trivial task. It's more on schedule and budget than any skyscraper or bridge could ever hope to be. Once built, we can use it everywhere. The only thing that requires any real effort, is building that very first skyscraper.
BTW - thank you for mentioning (indirectly) that civil engineering is the world's oldest profession. ;-)
To me it feels as if we are just at a very early stage. It takes huge amounts of time to learn software engineering, putting aside mastering it and humans haven't been practicing for very long. So I'd guess that we're just clueless enough to not be able to feel like engineers yet. We learn most of our practical abilities from experience so thats an argument for craft, but once we get a better understanding of what we are actually doing, and formalize it, our job will probably be very engineery. There is just millions of problem domains and no formal education on the domain of writing software yet (no a CS degree is not it). And I think there is a class of applications that are already so well understood that they do get developed in an engineering fashion already.
If you're building a skyscraper or a bridge, an engineer has to sign off on the design. If the design subsequently proves to be defective, said engineer is personally liable. In many places, calling yourself an engineer when you are not licensed to sign off on designs and be liable in this way is against the law.
We can debate whether or not extending this principle to software development is a good idea. What we can't debate is that software development does not currently work this way. You, the software developer, are not an engineer. If you think a design is bad, you do not have both the legal authority and responsibility to stand up, declare it so, and force the design to be changed to something saner.
But saying you're not an engineer because software projects don't succeed like projects done by real engineers - that's both silly and wrong.
That's what makes it a stupid point: it depends on a specific definition of engineer that is not used (and is not possible to use) with software engineering, and that definition is oriented around chartering/certification of the professional themselves.
That he buttons the essay up in such a self-serving way is just icing: taking "engineering" too literally for the context, then calling himself a gardener.
And the lowest bidder for contracted work often sucks, or finds ways to bill more than the original bid. :)
First, software engineers exist in a broader culture. The connotations of the various words in our respective languages are not in our control. "Engineer" and "gardener" have very different connotations. In a profession that sometimes struggles to gain respect, it would not behoove us to use a term that (wrongly in my opinion) is associated with lower social and professional status. If we could rewrite the English language and define "gardener" as a person with rare, hard-to-learn, and highly valuable skills, then perhaps we would do well to embrace the term "software gardener." But we can't.[1]
Second, software is engineering. This article seems to define engineering as Big Design Up Front. While that may describe part of what engineering can entail, it's not a complete or general description. If I had to describe engineering generally, I'd say it has to do with understanding and manipulating complex systems, analyzing and fulfilling requirements, distilling vague notions to concrete implementations, and building things that work. Software fits all of that.
[1] As I hinted above, I don't think it's fair, right, or accurate to denigrate actual gardeners as compared to software engineers. I'm stating the way most people perceive social status, not the way I'd like them to.
But come October, they started to roll off the line...20+ kips of concrete a pop and then stacked three high. To this day, the sensation is fresh; "So that's what the fuck it really looks like."
The reason it's fresh is because it returns every time I am in the presence of some lines I put on paper manifested.
The picture of the bridge is not the bridge. The engineer, like the hacker, has better intuitions regarding the way in which the picture will correspond to its instantiation - but it is still no more than a [hopefully] informed intuition.[1]
[1] The idea that there is an Engineer rather than a thousand authors of a successful bridge tends to be a bit absurd in modern construction practice. Such heroics are still far more likely in software development.
Perhaps it's less applicable to civil engineers, because of the scale and safety requirements of their profession, but let's look at electrical engineering. An electrical engineer will build prototypes; they will build models; they will use computer simulations to predict how systems will behave; they will build testing and QA infrastructure. It's definitely harder to iterate—once you've built a million widgets, shifting circuits around is going to be expensive—but it's not qualitatively different to developing a software product.
As in any engineering discipline, software engineers must optimise for a system's required qualities. In some cases, reliability might not be that high up the list. In others—let's say medical systems, or those used in space travel—software engineers have rigorous systems in place to ensure quality, and in many cases substantial up-front planning and certification will be involved.
Ultimately this seems to say that software engineering is different from (specifically) civil engineering. I think that's broadly rather true – the nature of civil is such that generally there are stringent safety requirements, and indeed massive costs involved in making subsequent changes. The majority of software does not have the same requirements, and so it's no surprise that the process is somewhat different. And I agree that not all development is engineering, of course, but to dismiss the entire field is a bit of a shallow argument.
Gardener fits, but we design the plants in essence....
To me Programmer, developer, even software architect seems closer than gardener.
I do not 'grow' nor 'tend' code, trusting some unknown universal force to keep me on the right path.
I engineer buildings, bridges, roads, tunnels - all sorts. Sometimes they're very small and sometimes they're very large but they're all subject to the forces pushing and pulling on them - my job is to build them and keep them upright as well as I can subject to the laws and constraints of the environment.
I am not a gardener. I am an engineer building, modifying, and destroying complex structures daily in a controlled dance so as to keep their superstructure useful and safe.
Only to the naive eye could such complexity be reduced to such apparent simplicity.
All analogies eventually break down if you take them too far. (If they didn't then you wouldn't be comparing something to something else; you'd be comparing something with itself.) But software development is so different from so many other practices and processes that most comparisons to gardening, building, finances, etc. don't provide much real value.
Its especially tiresome when it's claimed that "sofware isn't like that, it's like THIS". In the article, I find that neither gardening nor engineering is an especially enlightning analogy to software development.
I use old tools (vim, terminals), like engineers use old tools (protractor, pencil), to make comparatively much larger artifacts (search products, management tools), like engineers (buildings, bridges).
You're not judiciously applying the metaphor. You've only rephrased what it means to be "self-taught"; it goes without saying that we don't allow unlicensed people to engineer things. It also goes without saying that no one lives or dies by a faulty HTTP request header.