It's not as simple as that though - it all depends on the context of the project. Sometimes it is perfectly OK to go crazy and use bleeding edge technology other times that would be daft - it all depends what you are trying to achieve mixed in, as the article says, with a heavy dose of judgement.
It's about how you use bleeding edge technology though, especially after playtime is over and you've established it does what you want. All too often the engineering of things like abstracting the bleeding edge tech, or documenting what is going on gets forgotten. Then some other poor soul is left to pick up the pieces when said bleeding edge tech released a breaking API change.
I think one of the big valuable traits of a good programmer is a real fear of complexity. But I don't think the biggest problem is the complexity that comes from "rockstars" and over engineering. I think it's a way bigger problem that complexity tends to grow uncontrollably as features are added and bugs fixed. Especially when the domain is poorly understood and when the team isn't focused on keeping a coherent and simple system.
This kind of creeping complexity isn't introduced by mavericks because it's fun. It arises because of a conservative tendency to never throw anything out, to avoid refactoring, to be afraid of changes and simplifications. And also because of simple incompetence in design. Special cases multiply, no fruitful abstractions are introduced, the team's language is confused.
I would disagree in that's the working definition I've most commonly heard. Think of rockstars doing the same old thing but now with a cool trendy new sound. Yup, its just "happy birthday" again, but I've got a rockin' hairdo and a wild guitar solo (its all about the solo with these guys) and I added some pyrotechnics to the show, although it does nothing productive. None the less, at the end of the day, other than an impressive show, it was still just "happy birthday".
I don't think it involves wearing leather pants to work, although it doesn't exclude it, I suppose.
Yes. Because good judgement requires understanding complex situations, and that requires skill. And judgement is improved by experience.
For example you see a bad thing in the code? If you can understand the problem and the risks and benefits, you can make a good decision. If you can possibly make it better, understanding the time available to you, and the risk to the product, you should make it better. Not put it off and let the bad thing become more entrenched technical debt.
Making that kind of decision, requires good judgement. It requires understanding your own skill, time management, risk, and more.
Being able to exercise good judgement comes with extreme rewards for any project, as there will be less costly mistakes, and more successes and better code.
Maybe it's just because the post is out of date now (2011) but I'm not sure why "new technology", "rockstar" and "produces unmaintainable code" are all conflated together.
Does "rockstar" mean "bad" in this context? Because "using the latest technology for the sake of it" is not a trait of a good developer, and I have no clue what the hell "rockstar developer" means any more (unless they're also in a major rock band, then I know what it means).
"quickly write amazingly elegant code and manage complexity"
No... the number one trait of a great developer is to reduce complexity in systems to a maintainable and cost efficient state.
Any developer that introduces complexity such as introducing tech stacks that no one else on the team comprehends is not "great" no matter how elegant their code. They are a liability to the employer.
Unless, of course, that developer is self employed and is developing a solution for themselves. But that isn't what the article was about.
Dianne's solution creates a maintenance problem if the number of devices grows too much or if network stability is a problem, or if there is an expectation down the line of, for instance, push sync.
My point is not that her solution was not valuable, but that to suggest that the maintainability of the two solutions is black and white like this is wrongheaded.
The really unfortunate implication here is that learning new technologies is either something that shouldn't happen or something that shouldn't happen on the job. I'd say that something with these requirements would have been a perfect opportunity for Diane to, "learn postgres," if she felt that it would have been better. The extra day or two of effort would likely pay off – if not on this project, then on the next.
> but she also knew that anything much more complex would be beyond her current skills.
And with this mindset, her skills for the foreseeable future.
There's a difference between technology and implementation though. If Dianne had chosen a solution that fundamentally can't scale easily then that is a poor decision, whereas a technology that can scale written in a way that can be upgraded later is much more flexible.
As for the MySQL v Postgre I see that as a sensible Agile decision - much more confidence in quality now, with the knowledge that in the future a move to a different DB is beneficial.
The REST/Sinatra/MySQL solution was chosen in contrast to protocol buffers and Cassandra - the scaling potential of the systems are very different.
If you haven't worked on a project of this nature then the complications are likely not to be obvious to you. For instance, once this ships, they will be required to maintain that version of the REST API that was designed forever, including any side effects or semantic weirdness. If they determine that some changes are required for the sake of scale, they need to cross their fingers pretty hard that they can squeeze all of those changes into the existing API.
If this system were truly designed with Dianne's current abilities in mind, we don't know what that REST API looks like. Is it possible to imagine that it sends the full listing of recipes on every request? Absolutely. Are you going to be required to do that forever once the device ships? Yes.
I guess it depends on how efficient you want to be regarding your hardware costs. I have had to replace systems built in exactly this way because of traffic from fewer users than you'd think.
MySQL can easily scale to well beyond the "500 users" unless each of those users puts an especially heavy burden on the data store. The scenario at hand is synchronizing recipes and that is not a heavy burden.
Further, with a high-performance web framework, foregoing resilience (not recommended, but just for argument's sake) a single web-application instance should be able to handle a much larger number of users as well.
But all of this isn't really relevant to the point made by the OP. He's just using these fictional data points as an illustration. Although I am personally in favor of using examples that are more representative of reality, I think we can mostly ignore the fictional data points to appreciate his point.
> I'd say that something with these requirements would have been a perfect opportunity for Diane to, "learn postgres," if she felt that it would have been better. The extra day or two of effort would likely pay off – if not on this project, then on the next.
Perhaps on a product team but for an agency dealing with clients your time is your money. You spend an extra day or two on the project and the client is getting charged for that. Now they're wondering why we're thousands over their budget when the initial perception was we could work within it.
I'm all for learning new things on the job but that opportunity is only for certain situations.
> Dianne's solution creates a maintenance problem if the number of devices grows too much or if network stability is a problem, or if there is an expectation down the line of, for instance, push sync.
None of those are maintenance problems, they weren't part of the additional spec. For custom to-spec development, if the requirements change then it's expected that the code will have to change as well, incurring a change order.
A maintenance problem would be something like MySQL being an unreliable DB or the code she wrote being unreadable. If someone needs to go in and fix inevitable bugs then there will be issues.
If the customer wants to add features later then they'll have to pay for the changes and if the new features are different enough from the original spec then the changes might be quite expensive.
The ideal solution would be somewhere between a massively scalable system and a system that just barely meets expected capacity. When an engineer is designing a bridge they over-build just enough that the bridge is solid even when packed with 18 wheelers in a snow storm. Most of the time the bridge won't be anywhere near that load. A great developer builds to the expected capacity + some additional capacity based on the uncertainty of the prediction of capacity. I don't see either Jack or Dianne as perfect developers.
I agree that the "bridge" should be built to the expected capacity + additional due to uncertainty, but that's a slippery slope and one that this article is trying to address. It's a judgement call on what amount of work is appropriate for what amount of capacity, as adding on more capacity due to uncertainty might delay the product or solution from launching.
It may be more appropriate to properly separate concerns and build a flexible architecture that can be upgraded over time to meet needs as they come, or left as-is if those needs never manifest.
I agree with the op but have a different spin on it. It depends on what you see as your core business. Is your business about the technologies used to build the application or are you using technology to let, "kitchen devices synchronize their list of recipes". If your business is the later then you are better of using tried and tested technologies your developers are familiar with and can maintain without detracting from the core business.
I think the exact opposite holds true. If you're a MySQL / Sinatra shop, being the best at MySQL is the goal - double this if you're working for clients.
If your job is to provide the best possible kitchen device sync platform, then doing on the job research into improving that is essential to the business.
I do not disagree with you. It is all relative and the most important thing is to know what your core business is. Using a car analogy, car salesmen and engineers both need to have technical knowledge of the cars they work with. One focuses on selling cars and the other making the cars. I think too often start-ups are not too clear on their core business. Engineers know the cars better than the salesmen but does that make engineers better equipped to sell cars?
The ideal way is to build for either postgres or mysql, using vanilla SQL, therefore both shipping something that is known to work (mysql, in her case) and learning something about postgres. My observation is aside from anecdotes and peculiar corner cases which should be very strongly avoided, the philosophy of mysql is always the cub scout motto "try your best". The philosophy of postgres is more like "It shall be perfection at all times or I will curl up and die horribly as the ultimate prima donna". When times are good and there's nothing unusual, there is absolutely no practical difference. Its a meaningless personality test to walk up to a dev and ask which outlook on life matches their own outlook and which makes them feel like vomiting. It tells you something about the dev and nothing about the problem. Rarely, if ever, does actual business need enter into the debate.
Postgres/mysql is a business needs decision that needs to be made at a higher pay grade than hers, most likely. Assuming anyone at a higher paygrade knows anything about databases or specifically mysql vs postgres. If no one at a higher level understands, its her job to put it into simple little one syllable words even an overpaid MBA can understand, or they simply have to trust an underling to make a higher level decision.
Something not discussed in the article specifically is this behavior comparison is classic bodger/tinkerer behavior aka Jack vs true engineer behavior aka Dianne. Most people are flexible between the two, some are notoriously inflexible... I have a backyard garden swing built to hold 4500 pounds because that reflected my junk pile at the time and desire to never maintain the swing after installation, although over-specification behavior like that at work would lead to my firing or the financial failure of the company, so I gotta know when to switch. This should have been explicitly discussed in the article.
This is a problem because writing working, maintainable code that solves the given problem rather than an imagined problem should be the trait of a competent developer.
I had the pleasure of going on a course run by a German developer recently, and his brutal focus on quality and maintainability made me rethink what we class as a good developer.
Brutally in the article Jack is not a rockstar - Jack is incompetent. A great developer solves complex problems in a way that is maintainable - the problem is that most problems aren't that complex. Dianne however didn't do anything that made her "great" - any developer who can't do the kind of engineering practices she did should be fired.
Hacking on something in your spare time is completely different, but as soon as you want to work on something that others will work on as well you need to be into that engineer mindset. It's one of the things that impresses me about Patrick McKenzie's writing, and his experience of learning that the hard way in Japan.
Maybe we should be asking why the things we make need to be maintainable. Why can't I just make program module x to input a and output b and never touch it again.
I believe modular code is the future and the sooner we get there the better.
You can believe all you want, but please develop code with your head in the present.
What you said is akin to saying: "sure, the code we wrote is slow, but we can fix this with hardware in a month - Moore's Law!". That doesn't help anyone - it's just wishful thinking.
Apply a bit of brain power and make your code simple and straight forward so that your customer doesn't waste time and effort maintaining it (or more likely in the "Rockstar" case, replacing it entirely).
Here's how I read your comment: why do we worry so much about making things maintainable - why don't we just make things maintainable? Building a system out of small modules is what makes it maintainable.
What do you do when bugs show up in that module? rewrite it from scratch, introducing god knows how many new bugs? Or only address bugs when a rewrite of the entire module is necessary?
Because x(a) -> b might need to be understood by everyone working with your module at some point. What if you leave the company and someone inherits your module? What if they decide to change their codebase and need to rewrite your module?
Writing modular code doesn't magically give you maintainable code either. It needs documentation, proper use of design patterns, and needs to be readable.
This kind of judgement is not a sufficient and not even a necessary trait of great developers. It's nice to have, but 10 such developers will not allow you to build anything better than Dianne's solution.
Many great developers have misjudged their abilities or the practicality of particular ideas and solutions. I won't name (and shame) any, but look for examples among famous game developers.
Dianne knew it wasn't the most elegant solution, but she also knew that anything much more complex would be beyond her current skills.
Since when are complex solutions elegant? I always thought the simple ones were the ones deserving to be called "elegant". Also, the cheapest, fastest, and most reliable components are those that aren't there.
Some people (you call them Rockstars) are interessted in the newest technology and they want to use them to solve problems. No matter how efficient it is or how good somebody can maintain in.
But you forgot one thing: the "Rockstars" use technology, that will be normal for the masses 2 or 3 years later.
Ruby, for the first time, was used by Rockstars too.
It depend on your company and the size, but you need some rockstars and a mass of good developers. The rockstars look for new things, they try them, they show how to use it. The mass of good programmer can look at the results and choose what make a sense to use in productive environmnet.
There is nothing bad with a rockstar programmer, only with people that think they are rockstars and talk like "i am the king baby". ;)
"But you forgot one thing: the "Rockstars" use technology, that will be normal for the masses 2 or 3 years later."
Or sometimes they pick the wrong tech, and it dies, and someone else gets to pick up the pieces 3 years later. Predictions are difficult, especially about the future.
Raw UDP+checksum might have been even better. Easier to build on a tiny microcontroller. Lighter on the network. Offers an expansion path to NAT hole punching for push updates.
The catch-22, of course, is that at some point, MySQL was the "new thing" to Diane, so how did she learn it in the first place?
The answer of course is that judgement is less about making decisions upon familiarity per se and more about knowing when unfamiliar technology seems to provide good trade offs given what is known about it. For example, PostgreSQL may be unfamiliar, but choosing it is a wildly different type of decision than choosing another less mature unfamiliar data store, since at a high level PostgreSQL is known to be a well-understood, mature technology that may provide significant leverage over MySQL in certain scenarios. Being able to understand the various dimensions of technology choices and make good ones with limited information and experience is what makes good judgement, not just an outright aversion to unfamiliar things.
Diane most likely chose MySQL because it was the most popular open-source database. Popularity is often reason enough to choose a particular tool, because it means that that tool is more likely to be supported by other good tools.
"Popularity is often reason enough to choose a particular tool"
Widely available, many eyes searching for (and patching) security bugs, many people can understand and debug it, probably the most documentation out there, its the simplest possible overall way to do it, she's not exactly doing anything new, so using a tried and true successful solution is good business and sound engineering practice.
So, the cub scouts are building birdhouses this year, again. Last year we used hammers. I found a nailgun recently invented in the black forest region of Germany which shoots a special birdhouse domain-specific-nail thats a precise metric size specialized for this kind of wood and building birdhouses, unlike local nailguns which use an air compressor or .22 blanks, this one uses compressed unicorn gas, which everyone who's cool knows it doesn't smell, and its brand new and I'm the only guy this side of the pond who knows anything about this new nailgun and the only guy who can use it or maintain it, and it cuts hammering time down from 5 minutes per birdhouse to 5 milliseconds even though theres no business case to care. Oh no wait I'll just bring some hammers again this year. But old fashioned obsolete hammers don't scale to 10B units! Oh that's OK the school gym where we have pack meetings only holds 500 and usually only about 20 scouts show up anyway and the countries population is only 400M, so I'm all good. And given other business constraints the scout pack is permanently constrained to grow slower than I can buy more hammers. Yeah, looks like a job for a hammer, its hammer time... Would I roof a house with a hammer, well no, but I'm not roofing a house so I don't much care.
I think that asking why a programmer knows any piece of knowledge is a very good glimpse into how competent she is. I don't care that somebody likes Git more than SVN but I do care why (on a technical level) somebody likes Git more than SVN.
2nd dimension: ecosystem of the tool (developers/libraries/support/etc)
3rd dimension: developer's ability to use the tool (time to learn, current ability, etc)
If you think on factors going from 0 to 10, MySql could be [4, 7, 8] and Postgres could be [7, 7, 4] for that developer.
Of course the 3rd factor depends on the developer (and on the time available), for example I know Go is very powerful, but I wouldn't use it, unless the available options for me are not good enough for a given problem (so the time to learn compensates for the loss that using other technologies would incur)
I think another important dimension is the suitability of the tool to the technical culture and composition of the organization. For example statically typed java-ish languages tend to be more attractive if you have many disconnected groups contributing to a code base, and if I'm being cynical, overall less average skilled programmers just hacking out features. Whereas a small team of top-tier developers are more well positioned to leverage the benefits of more fancy tools and languages.
Personally, as a dev, I push myself a little with each new project. I've been using Sinatra for all my web projects, with this latest one I took the jump to Rails. Otherwise the project fell squarely within my realm of knowledge, as a previous project showed me how the Facebook API works.
The time before that I tried to incorporate good OO style.
Next time I think I want to incorporate testing. Or maybe I won't feel that the next one is the right time to incorporate it, and instead I'll pick up a new Javascript library.
As devs, we have a remarkable ability to shape a project by choosing which technologies to use. So long as we exercise good judgment, we can learn new things on the job and not have to bet the farm.
As for MySQL / PostgreSQL, that decision probably didn't matter all that much for the given problem. Abstracted over with Sequel, they're pretty much interchangeable until you get to heavy, industrial-scale use. A lot of people can go an entire career without needing to figure out the use case for each.
I don't agree with the article at all. At least, not in the way it was presented.
Sure, designing such a small system to use Cassandra and Protocol Buffers might be overkill. Then again, it might not. Without more information, it would be impossible to know which one is the best developer.
For instance: how much time did the Node+Cassandra+Protobuf implementation take? How is that, compared to the junior dev that only knows MySQL? Was Jack on a deadline, or did he have time to explore alternatives? Was Jack really inexperienced, or he did have experience in the new tech, but wanted those in his resumé?
Were they billing hours? What about the coworkers, do they only know Ruby and MySQL too, or are they confortable about learning new things? Do they even have coworkers?
Was Jack's solution modular enough that components could be replaced, if they were a problem? (Cassandra, for instance)
And most importantly: what was it that made Jack's implementation unmaintainable? Sure it wasn't because they are "new"(the article is rather old) technologies. Was the code badly documented? Was the design flawed?
Jack's biggest flaws, as can be inferred from the article, is not asking enough questions before diving into the implementation, and not getting permission before using "new" tech.
I know the author did not get in depth about the personalities or the project itself because he wanted stereotypes to illustrate a point, which was taken. But the problem with it is that other people could get the impression that "new" technologies should never be considered, or that any attempt to make the application "scalable" from the beginning is a "Jack trait" and thus, wrong.
Not to mention that Jack might even be right, should the company's expectations prove to be too low.
How is Dianne solution not maintainable or scalable? I have used flask gevent and mysql to take similar load. MySQL is not bad at all if you check everything you send it with explain detailed while developing.
This is one of those "why I am a great developer" sorts of posts that allows people to pat themselves on the back and congratulate each other on how great they are, passive-aggressively sending it out to the team in hopes of denigrating some team member who likes newer things. In this case "judgement" means "uses the things that I know", and there seems to be little evaluation beyond that high-level use case.
Why was -- in this contrived scenario -- Jack's solution a "maintenance nightmare"? Just because.
> Why was -- in this contrived scenario -- Jack's solution a "maintenance nightmare"? Just because.
> The result was a combination of Google Protocol Buffers, node.js, and Cassandra. Elegant, scalable, and totally unmaintainable.
If I can venture a guess, Jack went ahead (without asking further questions, as opposed to Dianne) and took what was very hot at that time.
2 years ago, Node.js was extremely hot, in a sense that it was the new thing that somehow combined the hot javascript and made something easier. I remember my collegate excitedly talking about the amazing serverside javascript.
I don't know about Google Protocol Buffers and Cassandra but I assume they were what the author had in mind as 'very-hot-but-not-widely-used-yet', and most technology that fits such description generally falls into 'not-reliably-stable' and 'not-many-resources-yet' category.
Think how you would do Java EE without help of Javadoc.
I got the same feeling as you niuzeta, and I feel the article may be getting judged a little too harshly (although I find the assertion of this being the best quality in a developer a little too absolute for my tastes).
I think the author stated in the article that the fictional character Jack ".. makes a point of starting each project with at least three new technologies."
I personally understand this as three new technologies Jack has not previously worked with, which should be a definite indication of some risk/unknown factors. Plus given the context of this fictional scenario, it seems unlikely that other developers associated with the project/company know them.
Now if everyone were so conservative that they'd never try anything new, there would never be any progress, but I think the point of the article is that something isn't better, just because it is new. In fact, to me the rings of the "worse is better" philosophy which I didn't think was taboo in any way here on HN :-)
Finally I'd like to add that the author didn't say this was a "maintenance nightmare", he did however say the solution was elegant, scalable and totally unmaintainable.
>Finally I'd like to add that the author didn't say this was a "maintenance nightmare", he did however say the solution was elegant, scalable and totally unmaintainable.
Which is impossible, right? If it's elegant, it's maintainable. That's part of the definition of elegant software. A on liner that does amazing things, that very few people understand is not elegant. The same thought scales up to a whole solution IMO.
Yeah, I agree with you: 'just because' .. the author of the article doesn't know anything about protocol buffers, node.js and Cassandra. Probably because those things are 'too new' and 'not proven' technologies - to him.
But if I had to take over a project from someone, to maintain it, I'd MUCH RATHER have it be done using those technologies than MySQL and Sinatra. Yikes!
I recently had to make a bid on a project that would give the users the ability to share their works, produced in an embedded environment, with their communities - a "Cloud" for the users. I designed the system architecture of the embedded system to be as self-maintainable as possible, and as simple as possible. Since its an embedded OS environment that the users would be using, instead of writing code I integrated such tools as git, and designed a system topology that would allow the user to be shielded from all the git hassle, while still getting a push-button experience.
I lost the gig to someone who decided that it would be better to re-invent "topography ordering" (his words) in a custom MySQL+PHP application, and when I was given an opportunity to challenge his design, I said "great, you're going to re-invent Git, poorly".
He spent the rest of the day "pissed that you compared his project to .. GitHub", which is what he thought I meant when I said "git". Nope, I said "git", I mean "git" - not 'github'. He actually didn't know how to use git.
Either way, off they go .. re-inventing the DAG with MySQL+PHP in an embedded environment, when they could've just wrapped the feature in a git script ... yikes++!
So this article just demonstrates to me that there are a lot of blow-hards out there who just don't get why its so important to apply the NIH principle to themselves, not just others ..
But if I had to take over a project from someone, to maintain it, I'd MUCH RATHER have it be done using those technologies than MySQL and Sinatra. Yikes!
Except that isn't really an option in most cases.
The two most realistic choices are:
Take over a high-quality implementation using somewhat stale tools or an average to shoddy implementation with newer tools that the developer was learning on the fly.
.. or, take over a project that was done by a competent developer who knew what they were doing and were able to communicate that knowledge through their codebase to anyone else who might pick up the project later on - which is the real #1 Trait of a great developer, whatever technologies they're using ..
Take over a high-quality implementation using somewhat stale tools or an average to shoddy implementation with newer tools that the developer was learning on the fly
My experience has been that developers who are most interested in innovations in this industry are dramatically more likely to build better solutions -- the passion and interest cuts both ways. Versus the career developer who does the absolute minimum necessary.
But of course with any given situation and set of developers your mileage will vary (I've known on-the-edge developers who build ridiculously nice code, and hanging-two-generation-back coders who make disgusting code. And vice versa). This particular scenario bothered me because the nodejs / cassandra canards were propped up under the illusion that the only advantage they offer was nefarious "Web Scale". Only in reality for many projects they make it ridiculously efficient to build solutions. The same goes for Go right now -- there are practitioners who can build in moments what would have taken months on a, for instance, ASP.NET team.
My experience is that for every developer who adopts a new technology for its true merits, there are at least two others who use it only because it is at the top of Hacker News/Ruby Weekly/other trendy publication, and those developers take advantage of very few of the technology's benefits.
Coming from a functional programming perspective on code quality, I think new code written with a modern understanding of software engineering generally is better quality than old code backed by tech written with last decade's understanding.
The quality of an implementation is rarely tied to the tools except in extreme cases. A good Developer in general builds good solutions no matter the toolset. A good python developer is likely to become a good Objective-C developer if they invest the time to learn and because they understand the need for clean modular code the iOS code is likely to turn out pretty well.
> Why was -- in this contrived scenario -- Jack's solution a "maintenance nightmare"? Just because.
Because apparently the author didn't have the necessary skills to maintain it. Cassandra isn't easy to master, especially if you only have cheapo developers who are reluctant to switch to PostgreSQL from MySQL.
But yes, it's a highly opinionated, narrow-minded piece.
Having only used Oracle, MSSQL, and MySQL, I can't imagine a small 500-user app would be that hard to implement in in Postgre even if I'd never touched the platform before - does Postgre have some terrifying gotchas, or does it work as a reasonably well-behaved SQL server? That is, if I naively installed a PostgreSQL server on my box and started creating tables and indexes and users, would I be destroyed by some horrible gotcha as I failed to account for some critical config (like selecting the right storage engine in MySQL)?
As someone who moved from MySQL to Postgres, the only thing that took some getting used to was users being actual Unix users that you create and then grant access to. At the time it seemed odd considering what MySQL did but it ended up being a feature I like quite a bit.
PG roles are stored in the DB, not Unix. You can set up pg_hba to use ident authentication so that users are automatically granted access by their Unix login, but it isn't the default (and usually not recommended), and you still have to have PG roles in the database for them to correspond to. So I think there's been a misunderstanding.
It's my understanding that there are no config gotchas in PostgreSQL (unlike MySQL). I've definitely found it more of a pain to set up at first. This is because the default role is very restricted and defaults to safe.
They're similar, but there are some pitfalls you need to be aware of: count(*) optimisation in mysql for example. Different behaviour in the query optimiser. Most cases are easy to handle, but if you're used to mysql, there's still a learning curve.
Such an argument would be rich coming from someone building a solution on Ruby. Indeed, such arguments essentially limit your technology choices to Oracle, SQL Server, or DB2, with an app layer built on .NET or J2EE.
From a management perspective, it is easier to hire competent new employees for established, popular technologies.
Employees don't stay around for long anymore, especially dynamic, innovative employees who stay up to date on the newest technologies. People like that get recruited away all the time. Then who's going to maintain your bleeding-edge system? How long will it take to hire another employee of the same caliber?
If you're a top company who can afford to recruit and keep the best employees, then by all means, build on the latest and greatest. There is certainly nothing inherently wrong or unmaintainable about technologies just because they are new.
But if you're a medium or middling company with significant turnover in the tech ranks (i.e. most companies), then it's probably safer to build with the most popular, "standard" tech you can get away with.
I hated this perspective as a 22 year old when my (prior) company practiced it, but I can see the wisdom of it now. Not everyone should be on the bleeding edge, or even leading edge. It's important for both employers and employees to be on top of how close to the edge the firm chooses to be.
Agreed. If anything the number one trait of a developer is that they solve problems. However, that is not unique to developers and is required of non-technical people as well. What it means to be good at solving problems seems largely context-dependent to me, but if you are really good at solving important problems in a valuable domain you are great at that thing (developer or not).
Any article stating one trait as being most important for anything is probably being hyperbolically reductive, but in this case it's particularly painful because "judgement" is absolutely meaningless. Every single thing every person does requires judgement, and if you don't have it you'll probably get hit by a bus before you have a chance to get a job.
What this article is really arguing against are early-adopter developers who chase every new fad for its own sake and prioritize their own amusement over the best solution at hand. But this says about how good the developer is, only how good their actual contribution to your company will be relative to their potential. A shit developer can do their best but it still won't measure up to a great developer's trend-chasing effort.
Now even assuming some base level of competence, there are many different mindsets and personalities that lend themselves to different types of programming. I could go on and on about specific skills but we're mostly programmers here so I'll just hint at it: debugging skills, low-level skills, architecture skills, analyst (business requirements) skills, hci skills, modeling skills, clairvoyance, luck. All of these things have varying degrees of importance depending on the job and team at hand, and some combination of them will make someone the best developer for some specific job.
But if you find my answer too wishy-washy and you want specifics about what makes a great developer I'll boil it down to two things. First, to achieve competence as a developer, one needs have the tenacity and logical thinking to debug any issue no matter how arcane. Second, to achieve greatness one needs to be able to comprehend the entire problem space in such a way as to account for many more factors than an average developer let alone a non-technical person could ever hold in their head at once, and have the ability to translate those constraints into a system which is at once easier to understand and more elegant than the vast majority of competing solutions.
Learn new technologies on your own time. Don't use the client to fund you picking up the latest-greatest whatever. That's what personal projects are for. You take the knowledge gained from your personal projects (and unforeseen circumstances in paid projects that you had to code your way out of) and do what you know will work the requirements, versus what will be exciting for you.
Jack drops a couple turds into your codebase, but so does Diane. Hers just take longer to arrive. Inability to seek actual best tools based upon personal fear of the unknown does not a great developer make.
Basing one's code off the familar or the purported "best practice" without actual knowledge leads to drek down the road. A great developer chooses the right tools for the task (along with other things...) Diane's, for instance has these flaws:
- Requires a relatively heavy runtime.
- Requires HTTP stack.
- Wastes CPU/power
These are not negligible considerations for embedded devices.
Someone who knew what the crap they were doing would likely have chosen a binary protocol standard (many such exist), then pumped that information out over an appropriate topology - someone suggested UDP. The database should probably be SQLite; maybe berkDB, depending on the complexity of the data model and the CPU used. Bluntly, the proposed "great" solution isn't. It's an inelegant solution putting heavy demands on the hardware unnecessarily.
Judgement is fine. What you use as a proxy in hiring for judgement is experience. E.g., you don't hire a Ruby dev to do embedded work. Experience tells you that.
What makes a developer great is great wisdom: knowledge applied skilfully.
n.b.: I wouldn't hire either Jack or Diane based on this article; Diane is fearful and Jack picked a javascript engine too immature for embedded. Jack might be led to understanding and might be worth more; I don't want to go around trying to convince people that better is better - I have no hope for Diane if she can't spend a day learning postgres.
I thought that in this contrived example, Jack and Diane were writing the software that runs on the server, not on the kitchen devices. Also, what if the "devices" are small-form-factor PCs? That would relax the constraints considerably.
Ah, see, I read the spec as saying that it would be things stored on the kitchen devices (toaster, etc).
You do want to make sure your comms layer is easy to make reliable when talking from the device, regardless. Might be HTTP, might not be - I'd likely take a gander at ProtoBuf myself for feeding over TCP/IP to minimize comms complexity on the phone home code.
I think I'd use HTTP as the synchronization protocol, because GET and PUT are specified as being idempotent, meaning it's safe to retry them. Also, HTTP is a ubiquitous standard, meaning there are several implementations to choose from, on both the client and server sides.
The ubiquitous of HTTP is indeed a definite point in its favor(as is its popularity and level of documentation).
> GET and PUT are specified as being idempotent, meaning it's safe to retry them.
Entirely OT: Abstractly, yes, that's how they should be implemented, but they don't have to be and consequently it's entirely up to the receiving server whether they are or not. Which, in terms of assuming that someone will make unverified and bad assumptions, means that the implementer at some point will let them slip and become non-idempotent.
:grin: Yep! As the original article said, digging into the spec and clarifying is absolutely a trait of a great developer (even an average one).
Possibly being able to write a very clear requirements/spec with very few words is one of the hallmarks of the greats. (The initial RFCs for TCP/IP are amazing).
121 comments
[ 6.3 ms ] story [ 257 ms ] threadWe embrace complexity at our peril. Dullness is a virtue.
It's not as simple as that though - it all depends on the context of the project. Sometimes it is perfectly OK to go crazy and use bleeding edge technology other times that would be daft - it all depends what you are trying to achieve mixed in, as the article says, with a heavy dose of judgement.
This kind of creeping complexity isn't introduced by mavericks because it's fun. It arises because of a conservative tendency to never throw anything out, to avoid refactoring, to be afraid of changes and simplifications. And also because of simple incompetence in design. Special cases multiply, no fruitful abstractions are introduced, the team's language is confused.
I don't think it involves wearing leather pants to work, although it doesn't exclude it, I suppose.
For example you see a bad thing in the code? If you can understand the problem and the risks and benefits, you can make a good decision. If you can possibly make it better, understanding the time available to you, and the risk to the product, you should make it better. Not put it off and let the bad thing become more entrenched technical debt.
Making that kind of decision, requires good judgement. It requires understanding your own skill, time management, risk, and more.
Being able to exercise good judgement comes with extreme rewards for any project, as there will be less costly mistakes, and more successes and better code.
This is an old story, but I guess it has to be retold every few years, to inform newer developers and to remind older developers (and management).
The number one trait of a great developer is literally their ability to quickly write amazingly elegant code and manage complexity.
"Writing elegant code and managing complexity" means using proven and familiar technologies (or spending enough time to know them inside out).
If I could upvote the post by 10, I would - so true.
Exercising self-control is being the leader of one person.
No... the number one trait of a great developer is to reduce complexity in systems to a maintainable and cost efficient state.
Any developer that introduces complexity such as introducing tech stacks that no one else on the team comprehends is not "great" no matter how elegant their code. They are a liability to the employer.
Unless, of course, that developer is self employed and is developing a solution for themselves. But that isn't what the article was about.
My point is not that her solution was not valuable, but that to suggest that the maintainability of the two solutions is black and white like this is wrongheaded.
The really unfortunate implication here is that learning new technologies is either something that shouldn't happen or something that shouldn't happen on the job. I'd say that something with these requirements would have been a perfect opportunity for Diane to, "learn postgres," if she felt that it would have been better. The extra day or two of effort would likely pay off – if not on this project, then on the next.
> but she also knew that anything much more complex would be beyond her current skills.
And with this mindset, her skills for the foreseeable future.
If you haven't worked on a project of this nature then the complications are likely not to be obvious to you. For instance, once this ships, they will be required to maintain that version of the REST API that was designed forever, including any side effects or semantic weirdness. If they determine that some changes are required for the sake of scale, they need to cross their fingers pretty hard that they can squeeze all of those changes into the existing API.
If this system were truly designed with Dianne's current abilities in mind, we don't know what that REST API looks like. Is it possible to imagine that it sends the full listing of recipes on every request? Absolutely. Are you going to be required to do that forever once the device ships? Yes.
Further, with a high-performance web framework, foregoing resilience (not recommended, but just for argument's sake) a single web-application instance should be able to handle a much larger number of users as well.
But all of this isn't really relevant to the point made by the OP. He's just using these fictional data points as an illustration. Although I am personally in favor of using examples that are more representative of reality, I think we can mostly ignore the fictional data points to appreciate his point.
Perhaps on a product team but for an agency dealing with clients your time is your money. You spend an extra day or two on the project and the client is getting charged for that. Now they're wondering why we're thousands over their budget when the initial perception was we could work within it.
I'm all for learning new things on the job but that opportunity is only for certain situations.
None of those are maintenance problems, they weren't part of the additional spec. For custom to-spec development, if the requirements change then it's expected that the code will have to change as well, incurring a change order.
A maintenance problem would be something like MySQL being an unreliable DB or the code she wrote being unreadable. If someone needs to go in and fix inevitable bugs then there will be issues.
If the customer wants to add features later then they'll have to pay for the changes and if the new features are different enough from the original spec then the changes might be quite expensive.
It may be more appropriate to properly separate concerns and build a flexible architecture that can be upgraded over time to meet needs as they come, or left as-is if those needs never manifest.
Edit: grammar
If your job is to provide the best possible kitchen device sync platform, then doing on the job research into improving that is essential to the business.
Postgres/mysql is a business needs decision that needs to be made at a higher pay grade than hers, most likely. Assuming anyone at a higher paygrade knows anything about databases or specifically mysql vs postgres. If no one at a higher level understands, its her job to put it into simple little one syllable words even an overpaid MBA can understand, or they simply have to trust an underling to make a higher level decision.
Something not discussed in the article specifically is this behavior comparison is classic bodger/tinkerer behavior aka Jack vs true engineer behavior aka Dianne. Most people are flexible between the two, some are notoriously inflexible... I have a backyard garden swing built to hold 4500 pounds because that reflected my junk pile at the time and desire to never maintain the swing after installation, although over-specification behavior like that at work would lead to my firing or the financial failure of the company, so I gotta know when to switch. This should have been explicitly discussed in the article.
I had the pleasure of going on a course run by a German developer recently, and his brutal focus on quality and maintainability made me rethink what we class as a good developer. Brutally in the article Jack is not a rockstar - Jack is incompetent. A great developer solves complex problems in a way that is maintainable - the problem is that most problems aren't that complex. Dianne however didn't do anything that made her "great" - any developer who can't do the kind of engineering practices she did should be fired.
Hacking on something in your spare time is completely different, but as soon as you want to work on something that others will work on as well you need to be into that engineer mindset. It's one of the things that impresses me about Patrick McKenzie's writing, and his experience of learning that the hard way in Japan.
I believe modular code is the future and the sooner we get there the better.
What you said is akin to saying: "sure, the code we wrote is slow, but we can fix this with hardware in a month - Moore's Law!". That doesn't help anyone - it's just wishful thinking.
Apply a bit of brain power and make your code simple and straight forward so that your customer doesn't waste time and effort maintaining it (or more likely in the "Rockstar" case, replacing it entirely).
Writing modular code doesn't magically give you maintainable code either. It needs documentation, proper use of design patterns, and needs to be readable.
Many great developers have misjudged their abilities or the practicality of particular ideas and solutions. I won't name (and shame) any, but look for examples among famous game developers.
Since when are complex solutions elegant? I always thought the simple ones were the ones deserving to be called "elegant". Also, the cheapest, fastest, and most reliable components are those that aren't there.
Some people (you call them Rockstars) are interessted in the newest technology and they want to use them to solve problems. No matter how efficient it is or how good somebody can maintain in.
But you forgot one thing: the "Rockstars" use technology, that will be normal for the masses 2 or 3 years later.
Ruby, for the first time, was used by Rockstars too.
It depend on your company and the size, but you need some rockstars and a mass of good developers. The rockstars look for new things, they try them, they show how to use it. The mass of good programmer can look at the results and choose what make a sense to use in productive environmnet.
There is nothing bad with a rockstar programmer, only with people that think they are rockstars and talk like "i am the king baby". ;)
Or sometimes they pick the wrong tech, and it dies, and someone else gets to pick up the pieces 3 years later. Predictions are difficult, especially about the future.
The answer of course is that judgement is less about making decisions upon familiarity per se and more about knowing when unfamiliar technology seems to provide good trade offs given what is known about it. For example, PostgreSQL may be unfamiliar, but choosing it is a wildly different type of decision than choosing another less mature unfamiliar data store, since at a high level PostgreSQL is known to be a well-understood, mature technology that may provide significant leverage over MySQL in certain scenarios. Being able to understand the various dimensions of technology choices and make good ones with limited information and experience is what makes good judgement, not just an outright aversion to unfamiliar things.
Widely available, many eyes searching for (and patching) security bugs, many people can understand and debug it, probably the most documentation out there, its the simplest possible overall way to do it, she's not exactly doing anything new, so using a tried and true successful solution is good business and sound engineering practice.
So, the cub scouts are building birdhouses this year, again. Last year we used hammers. I found a nailgun recently invented in the black forest region of Germany which shoots a special birdhouse domain-specific-nail thats a precise metric size specialized for this kind of wood and building birdhouses, unlike local nailguns which use an air compressor or .22 blanks, this one uses compressed unicorn gas, which everyone who's cool knows it doesn't smell, and its brand new and I'm the only guy this side of the pond who knows anything about this new nailgun and the only guy who can use it or maintain it, and it cuts hammering time down from 5 minutes per birdhouse to 5 milliseconds even though theres no business case to care. Oh no wait I'll just bring some hammers again this year. But old fashioned obsolete hammers don't scale to 10B units! Oh that's OK the school gym where we have pack meetings only holds 500 and usually only about 20 scouts show up anyway and the countries population is only 400M, so I'm all good. And given other business constraints the scout pack is permanently constrained to grow slower than I can buy more hammers. Yeah, looks like a job for a hammer, its hammer time... Would I roof a house with a hammer, well no, but I'm not roofing a house so I don't much care.
I think that asking why a programmer knows any piece of knowledge is a very good glimpse into how competent she is. I don't care that somebody likes Git more than SVN but I do care why (on a technical level) somebody likes Git more than SVN.
"I was forced to use it on that project" or "brief research showed it was the optimal thing to use on that project" or "I got stuck in project Euler"
1st dimension: power of the tool
2nd dimension: ecosystem of the tool (developers/libraries/support/etc)
3rd dimension: developer's ability to use the tool (time to learn, current ability, etc)
If you think on factors going from 0 to 10, MySql could be [4, 7, 8] and Postgres could be [7, 7, 4] for that developer.
Of course the 3rd factor depends on the developer (and on the time available), for example I know Go is very powerful, but I wouldn't use it, unless the available options for me are not good enough for a given problem (so the time to learn compensates for the loss that using other technologies would incur)
The time before that I tried to incorporate good OO style.
Next time I think I want to incorporate testing. Or maybe I won't feel that the next one is the right time to incorporate it, and instead I'll pick up a new Javascript library.
As devs, we have a remarkable ability to shape a project by choosing which technologies to use. So long as we exercise good judgment, we can learn new things on the job and not have to bet the farm.
As for MySQL / PostgreSQL, that decision probably didn't matter all that much for the given problem. Abstracted over with Sequel, they're pretty much interchangeable until you get to heavy, industrial-scale use. A lot of people can go an entire career without needing to figure out the use case for each.
Sure, designing such a small system to use Cassandra and Protocol Buffers might be overkill. Then again, it might not. Without more information, it would be impossible to know which one is the best developer.
For instance: how much time did the Node+Cassandra+Protobuf implementation take? How is that, compared to the junior dev that only knows MySQL? Was Jack on a deadline, or did he have time to explore alternatives? Was Jack really inexperienced, or he did have experience in the new tech, but wanted those in his resumé?
Were they billing hours? What about the coworkers, do they only know Ruby and MySQL too, or are they confortable about learning new things? Do they even have coworkers?
Was Jack's solution modular enough that components could be replaced, if they were a problem? (Cassandra, for instance)
And most importantly: what was it that made Jack's implementation unmaintainable? Sure it wasn't because they are "new"(the article is rather old) technologies. Was the code badly documented? Was the design flawed?
Jack's biggest flaws, as can be inferred from the article, is not asking enough questions before diving into the implementation, and not getting permission before using "new" tech.
I know the author did not get in depth about the personalities or the project itself because he wanted stereotypes to illustrate a point, which was taken. But the problem with it is that other people could get the impression that "new" technologies should never be considered, or that any attempt to make the application "scalable" from the beginning is a "Jack trait" and thus, wrong.
Not to mention that Jack might even be right, should the company's expectations prove to be too low.
Why was -- in this contrived scenario -- Jack's solution a "maintenance nightmare"? Just because.
> The result was a combination of Google Protocol Buffers, node.js, and Cassandra. Elegant, scalable, and totally unmaintainable.
If I can venture a guess, Jack went ahead (without asking further questions, as opposed to Dianne) and took what was very hot at that time.
2 years ago, Node.js was extremely hot, in a sense that it was the new thing that somehow combined the hot javascript and made something easier. I remember my collegate excitedly talking about the amazing serverside javascript.
I don't know about Google Protocol Buffers and Cassandra but I assume they were what the author had in mind as 'very-hot-but-not-widely-used-yet', and most technology that fits such description generally falls into 'not-reliably-stable' and 'not-many-resources-yet' category.
Think how you would do Java EE without help of Javadoc.
I think the author stated in the article that the fictional character Jack ".. makes a point of starting each project with at least three new technologies."
I personally understand this as three new technologies Jack has not previously worked with, which should be a definite indication of some risk/unknown factors. Plus given the context of this fictional scenario, it seems unlikely that other developers associated with the project/company know them.
Now if everyone were so conservative that they'd never try anything new, there would never be any progress, but I think the point of the article is that something isn't better, just because it is new. In fact, to me the rings of the "worse is better" philosophy which I didn't think was taboo in any way here on HN :-)
Finally I'd like to add that the author didn't say this was a "maintenance nightmare", he did however say the solution was elegant, scalable and totally unmaintainable.
Which is typically pushed by worse developers. How coincidental!
Which is impossible, right? If it's elegant, it's maintainable. That's part of the definition of elegant software. A on liner that does amazing things, that very few people understand is not elegant. The same thought scales up to a whole solution IMO.
But if I had to take over a project from someone, to maintain it, I'd MUCH RATHER have it be done using those technologies than MySQL and Sinatra. Yikes!
I recently had to make a bid on a project that would give the users the ability to share their works, produced in an embedded environment, with their communities - a "Cloud" for the users. I designed the system architecture of the embedded system to be as self-maintainable as possible, and as simple as possible. Since its an embedded OS environment that the users would be using, instead of writing code I integrated such tools as git, and designed a system topology that would allow the user to be shielded from all the git hassle, while still getting a push-button experience.
I lost the gig to someone who decided that it would be better to re-invent "topography ordering" (his words) in a custom MySQL+PHP application, and when I was given an opportunity to challenge his design, I said "great, you're going to re-invent Git, poorly".
He spent the rest of the day "pissed that you compared his project to .. GitHub", which is what he thought I meant when I said "git". Nope, I said "git", I mean "git" - not 'github'. He actually didn't know how to use git.
Either way, off they go .. re-inventing the DAG with MySQL+PHP in an embedded environment, when they could've just wrapped the feature in a git script ... yikes++!
So this article just demonstrates to me that there are a lot of blow-hards out there who just don't get why its so important to apply the NIH principle to themselves, not just others ..
Except that isn't really an option in most cases.
The two most realistic choices are: Take over a high-quality implementation using somewhat stale tools or an average to shoddy implementation with newer tools that the developer was learning on the fly.
My experience has been that developers who are most interested in innovations in this industry are dramatically more likely to build better solutions -- the passion and interest cuts both ways. Versus the career developer who does the absolute minimum necessary.
But of course with any given situation and set of developers your mileage will vary (I've known on-the-edge developers who build ridiculously nice code, and hanging-two-generation-back coders who make disgusting code. And vice versa). This particular scenario bothered me because the nodejs / cassandra canards were propped up under the illusion that the only advantage they offer was nefarious "Web Scale". Only in reality for many projects they make it ridiculously efficient to build solutions. The same goes for Go right now -- there are practitioners who can build in moments what would have taken months on a, for instance, ASP.NET team.
Because apparently the author didn't have the necessary skills to maintain it. Cassandra isn't easy to master, especially if you only have cheapo developers who are reluctant to switch to PostgreSQL from MySQL.
But yes, it's a highly opinionated, narrow-minded piece.
I'll cut the author some slack here though since the article is over two years old by now.
Maybe this article will shed some light:
http://www.xaprb.com/blog/2010/05/10/the-manager-programmer-...
Employees don't stay around for long anymore, especially dynamic, innovative employees who stay up to date on the newest technologies. People like that get recruited away all the time. Then who's going to maintain your bleeding-edge system? How long will it take to hire another employee of the same caliber?
If you're a top company who can afford to recruit and keep the best employees, then by all means, build on the latest and greatest. There is certainly nothing inherently wrong or unmaintainable about technologies just because they are new.
But if you're a medium or middling company with significant turnover in the tech ranks (i.e. most companies), then it's probably safer to build with the most popular, "standard" tech you can get away with.
What this article is really arguing against are early-adopter developers who chase every new fad for its own sake and prioritize their own amusement over the best solution at hand. But this says about how good the developer is, only how good their actual contribution to your company will be relative to their potential. A shit developer can do their best but it still won't measure up to a great developer's trend-chasing effort.
Now even assuming some base level of competence, there are many different mindsets and personalities that lend themselves to different types of programming. I could go on and on about specific skills but we're mostly programmers here so I'll just hint at it: debugging skills, low-level skills, architecture skills, analyst (business requirements) skills, hci skills, modeling skills, clairvoyance, luck. All of these things have varying degrees of importance depending on the job and team at hand, and some combination of them will make someone the best developer for some specific job.
But if you find my answer too wishy-washy and you want specifics about what makes a great developer I'll boil it down to two things. First, to achieve competence as a developer, one needs have the tenacity and logical thinking to debug any issue no matter how arcane. Second, to achieve greatness one needs to be able to comprehend the entire problem space in such a way as to account for many more factors than an average developer let alone a non-technical person could ever hold in their head at once, and have the ability to translate those constraints into a system which is at once easier to understand and more elegant than the vast majority of competing solutions.
Basing one's code off the familar or the purported "best practice" without actual knowledge leads to drek down the road. A great developer chooses the right tools for the task (along with other things...) Diane's, for instance has these flaws:
- Requires a relatively heavy runtime.
- Requires HTTP stack.
- Wastes CPU/power
These are not negligible considerations for embedded devices.
Someone who knew what the crap they were doing would likely have chosen a binary protocol standard (many such exist), then pumped that information out over an appropriate topology - someone suggested UDP. The database should probably be SQLite; maybe berkDB, depending on the complexity of the data model and the CPU used. Bluntly, the proposed "great" solution isn't. It's an inelegant solution putting heavy demands on the hardware unnecessarily.
Judgement is fine. What you use as a proxy in hiring for judgement is experience. E.g., you don't hire a Ruby dev to do embedded work. Experience tells you that.
What makes a developer great is great wisdom: knowledge applied skilfully.
n.b.: I wouldn't hire either Jack or Diane based on this article; Diane is fearful and Jack picked a javascript engine too immature for embedded. Jack might be led to understanding and might be worth more; I don't want to go around trying to convince people that better is better - I have no hope for Diane if she can't spend a day learning postgres.
You do want to make sure your comms layer is easy to make reliable when talking from the device, regardless. Might be HTTP, might not be - I'd likely take a gander at ProtoBuf myself for feeding over TCP/IP to minimize comms complexity on the phone home code.
> GET and PUT are specified as being idempotent, meaning it's safe to retry them.
Entirely OT: Abstractly, yes, that's how they should be implemented, but they don't have to be and consequently it's entirely up to the receiving server whether they are or not. Which, in terms of assuming that someone will make unverified and bad assumptions, means that the implementer at some point will let them slip and become non-idempotent.
Possibly being able to write a very clear requirements/spec with very few words is one of the hallmarks of the greats. (The initial RFCs for TCP/IP are amazing).