35 comments

[ 2.7 ms ] story [ 91.5 ms ] thread
But what's the dilemma?
Believing that the nature of knowledge is dogmatic; that there exists no pattern behind anything and that anything that is knowable must be memorised by rote and stored wholesale in the mind, rather than guiding principles abstracted from diverse experience refined and applied in multiple contexts.

I humbly submit that this single idea is the root of so much of the mental incompetence in the world today. It encourages the spread of religion and it's comparison with real science, it discourages attempting to comprehend and apply deep unifying principles between all things and a breadth of thought that comes far easier when one isn't afraid of exposing their preconceived ideas and views to the light of critical enquiry.

Last but not least, it encourages the emergence of a political class that at best exists purely to coordinate and rule the "underclass" specialists, and at worst offers no independent value themselves.

That's not a dilemma for generalists, that's a critique of specialization.
Except that it's far more common for people to believe it and thus ask silly questions that attempt to ascertain the degree to which one has drilled themselves in said rote memorisation. Like a google interview I had a few years ago where they asked me what the number of blocks in an ext2 filesystem for a specific distribution of Linux was at the time. The irony of being asked this by a company who actually would've featured prominently in my answer (ie, just google it) was not lost on me, and it disabused me of the notion that I'd actually like to work there after all.
Specialization is not equivalent to rote memorization. But rote memorization is a bedrock foundation for almost any specialty, because it is the very language of that specialty. The more fluent you are in a given language, the more easily you can write beautiful prose with it.
They often go together, as you illustrate well with your comment, I agree that rote memorisation can be a bedrock foundation for any specialty, however it is not necessarily the only foundation. And the thread of my argument here is that it is not as good a foundation as a complete understanding of many topics in an interwoven fashion.

An attempt at an analogy; I assume you're familiar with jQuery as that's your nickname so I'll use that. One could memorise every single API method and syntactical quirk of the framework and be rightfully considered a specialist, but this does not innately suggest that they understand the absolute multitude of concepts that are likely to be bought together in a project making use of this specialty, or how the fabric of a jQuery object might be incorporated into a larger project using a full polyglot of development languages and spanning JRuby, Java, Groovy, Scheme, Scala etc. Knowing the underlying concepts that unite them gives a person the ability to solve problems in any one specific area potentially better than someone who had simply memorised all the API documentation for the jQuery framework.

Going through the rote memorisation of all these disparate things you're likely to eventually notice the patterns and ideas that they share, but in my opinion this is a backward way of doing things. Starting with patterns and ideas and fleshing out the specifics with reference documentation on a case by case basis has always been my preferred approach, it's much easier to find a base pattern and go through it's iterations than it is to simply memorise all the potential iterations of a given pattern. How easy is it to memorise a table of results on several thousand experiments demonstrating F=ma compared to simply applying F=ma several thousand times?

Even more so with something that moves as fast as development; when the framework changes and your memorised documentation needs to be updated, it is much easier to understand the change from the perspective of cause and effect rather than just "here's how it was including all edge cases and implications, and here's how it is now, including all edge cases and implications."

I completely agree. In fact, this is why physician specialists are required to perform basic clinical duty in many different specialties before diving deeper into their primary specialty (I hope a physician will correct me if I am wrong).

The manner of specialization also depends on the field. In programming we have the luxury of time and a search engine, so knowing basic concepts are more important than API details. A surgeon has no such luxury--your heart transplant specialist had better know everything down cold.

Surgeons often lookup how to do a surgery, and even call people for advice before they start. For complex problems they can spend hours or days deciding how to approach a complex problem using 3d images before they start. Now there are a lot of rote skills involved with fixing problems as they show up but normally they have plenty of time to prepare ahead of time and several people to assist them.

Granted, when someone shows up in the ER with a gunshot wound they don't have the time, but that's hack and slash medicine where the goal to keep the patent alive for the next few hours so they can get time to deal with any major problems.

It would take me a long time to dig up the number of language and dialects I have used, and yet I don't really know half of them. I have several times helped someone solve a problem in a language I did not know by looking at the code and saying ok it's X. And this is not just decedents of C. I have solved a problem someone with 30 years of embedded coding solve a problem he was having in an assembly language I can't even remember the name of.

I might not be a true Master, but I can tell learning the inner workings of an API that's only lasts for 10 years is not the way to get there. ASM, C, LISP, and SQL are far more important than Rails.

PS: I am all for Memorizing constants but API's are a dime a dozen. One of the few masters I have seen rewrote a system ~90% from scratch in 3 weeks in a language he had never seen that a well respected team was unable to finish in a year.

Edit: Focus on the things that let you do a month long task in a a few days not a hour long task in minutes because the little short cuts are easily found when you need them but the big stuff is far harder to stumble upon when you need it.

Some of that sounds a little dogmatic in itself. It's an interesting historical fact that the idea of there being a deep underlying unity in nature was originally a religious one. In a time before there were any examples of really successful science, the idea that the universe might actually be comprehensible to us on a deep level seemed pretty loopy unless you believed that it was deliberately constructed to be intelligible. After all, the universe isn't all that comprehensible to any other form of life on earth -- why should we be so special if we are not in fact special? So in a sense, the idea that science is possible has its roots in a particular sort of religious belief.

Anyway, that is not intended as an argument in favor of religious belief today. But it's worth asking whether any rational justification could have been given in the 13th century for a belief in the deep underlying unity of the universe, before we had available the fruits of the research based on that assumption. I think it would have been far more sensible in those days to believe that the universe is just a complicated place with a lot of different kinds of stuff in it, and a lot of different kinds of principle governing what happens to that stuff.

Interestingly, this sort of pluralistic conception of reality is coming back into fashion in some circles, under the heading of the "dappled universe". Nancy Cartwright's work on these topics is really interesting. (FTR: She is trained as a physicist and has no religious agenda.) It's actually shocking how little reason there is to believe in a single set of underlying natural laws when you consider these issues with an open mind.

edit: There's a good (somewhat critical) review of one of Cartwright's book's here [doc]: http://personal.lse.ac.uk/CARTWRIG/PDF%20various/Book%20revi...

It's actually shocking how little reason there is to believe in a single set of underlying natural laws when you consider these issues with an open mind.

What's the alternative to a single set of natural laws?

Perhaps one set of natural laws that applies around here, and a slightly different set that applies in Andromeda, but only on Tuesdays? But surely the two sets, plus the Andromeda/Tuesdays restriction, put together form one slightly more complicated set of physical laws?

Perhaps an infinite set of subtly different natural laws which apply at different points in space and time? But that's still just one very large set of laws, right?

>What's the alternative to a single set of natural laws?

I take the point that however many natural laws there are, there is a set containing all of them. However, you did delete the important adjective "underlying" when asking your question. Isn't it pretty clear what it would mean for there to be no single set of underlying natural laws? It would mean, for example, that laws of chemistry would not necessarily be reducible to laws of physics, laws of psychology would not necessarily be reducible to laws of biology, etc. etc. You may think that this is wrong, but it's a perfectly intelligible idea.

Cartwright argues that there is no "ultimate", universal set of laws to which all other true laws inevitably reduce. From her point of view, laws are relatively local and specific. She goes through a lot of physical phenomena, particularly involving lasers, and shows that although physicists are able to make very precise predictions, they virtually never make these predictions merely by special-casing general principles. Her argument (which I don't have the space or expertise to summarize here) is that there is consequently no reason to think that the general principles are really true, even though they are fantastically interesting and useful.

Also, you seem to be tacitly assuming that all natural laws are necessarily going to be laws of physics, but that is one of the questions at issue. E.g., are we right to assume that all the laws of chemistry are "in principle" reducible to physics, even though we can never hope to make such a reduction in practice? Perhaps that is just a kind of unjustified faith in the unity of reality.

Anyway, I am sort of on the fence on these issues, but I found her arguments unexpectedly persuasive. It really is surprising how non-stupid the idea of a pluralistic reality is when you look at these questions in detail.

E.g., are we right to assume that all the laws of chemistry are "in principle" reducible to physics, even though we can never hope to make such a reduction in practice?

Actually, making that reduction in practice is pretty much what I do for a living.

Yes, I'm not suggesting that there's a complete disconnect between physics and chemistry, but we're a long way from having a complete understanding of how all chemical reactions work at the physical level. That is, an actual proof that all known laws of chemistry are mathematical consequences of the laws of physics (without making too many fudges, simplifying approximations, etc. etc.)

This is not in any way intended as a criticism of physicists or chemists, in case it comes across like that. They're both incredibly successful at what they do.

Just googling around, this paper gives quite a nice overview of what sensible scepticism about the possibility of reduction looks like: http://www.springerlink.com/content/v05g613468p6xk17/

However, as you illustrate yourself, it is no longer the 13th century and we have a large amount of evidence to suggest the relationships between things extend further than a simple jumble of complicated and unrelated data. I don't think it's dogmatic to point this out at all, I think it's dogmatic to deny it, needing to roll the clock back over six hundred years to posit a situation where the evidence may indicate otherwise.

Even if that were granted, I'm still not so certain that it makes sense. Humans have been finding laws to apply for much of recorded history. I realise it's quite a long way from the elegance of something like general relativity to the simplicity of something like pythagoras' theorem, but the emergence of the former depended upon the epistemological grounding of the latter.

It was not my intent to make allusions to a unifying great spirit or any such nonsense, merely that for the vast majority of recorded history searching for the patterns and relations in material reality has given humanity most of it's most spectacular successes. To cast this all away and try to parcel everything into microscopic localities which are not thematically related at all and thus discard what can be learned by taking a wider view.

That said, I'll check out the dappled universe idea, I've not come across it before. It amuses me more than a little that this particular concept is being pushed by someone neatly straddling the disciplines of philosophy and physics though, isn't that in itself something of a defense of a broader view?

>we have a large amount of evidence to suggest the relationships between things extend further than a simple jumble of complicated and unrelated data.

Oh yes, of course. But there's a lot of middle ground between a complete jumble and complete unity. I don't think anyone is suggesting that there are no regularities at all, just that the ones that there are may be relatively local and particular. (They're still pretty damn impressive for all that, though.)

I think this is an aesthetic problem in a way. Everyone seems to have a kind of Platonic aesthetic these days -- everything has to be ultimately simple and unified and abstractly beautiful. If you'll excuse a stupid metaphor, it's sort of a question of how you visualize God's office. Does it just have a plain desk with a couple of pens and two equations written neatly on a notepad, or is it a huge jumble of dusty books and papers covering every conceivable topic at every conceivable level of analysis? I think part of the reason for my "conversion" to the dappled view of things was that I started to find the Platonic aesthetic less and less appealing. (Of course, it might be a really bad idea to let aesthetic considerations have this much influence, but maybe that's another topic.)

>It amuses me more than a little that this particular concept is being pushed by someone neatly straddling the disciplines of philosophy and physics though, isn't that in itself something of a defense of a broader view?

I think it's a bad kind of slippery slope argument to go from "there are some interesting connections between some things" (certainly true) to "everything is deeply connected and unified" (plausible, appealing in some ways, but not something a rational person has to believe).

Everytime I see a 'generalist' vs 'specialist' post like this, I think its wrong to compare generalists who are very good in a lot of different areas to specialists that are very good in just one of them.

Ideally, you'd have to compare generalists who are mediocre in lots of fields with a specialist whom is very good on a field.

Or, in your case, like you said (being an accomplished practitioner of many different fields), comparing that to being like Marie Curie or Einstein, to quote your examples. That is, if you took all the time and effort from being a writer, orchestra musician, etc, and put it into a single field.

Even if you can say your life's been very fulfilling (and you're probably right), its a hard argument for the generalist to say he's better than the specialist in the specialists' field.

As for 'you can't be good at everything', that really means 'you wont be as good as you can be at something if you keep learning everything'. But putting in lots of work on everything obviously make you good on everything.

But even mozart, being very talented a musician in a young age, had to choose (at age 4?) the violin or the piano (he chose the later)

(comment deleted)
"Ideally, you'd have to compare generalists who are mediocre in lots of fields with a specialist whom is very good on a field."

Isn't the whole point of the essay to suggest that generalists need not be mediocre at all? It's not obvious that the energy spread over several activities would be as efficiently spent on one specialist activity - if you do (without proof) assume that that's feasible, then that's actually begging the question...

No, the above argument leaves room for diminishing returns - it only assumes that spending more time learning foo is more efficient at learning foo than spending more time learning bar is at learning foo.

Suppose Alice and Bob have similar levels of talent. Alice spent 10000 hours learning the piano. Bob spent 2000 hours learning the piano, 2000 learning the violin, 2000 learning percussion, 2000 learning to sing and 2000 writing a novel. The above argues that Alice will be better at playing the piano than Bob. Probably not five times better (assuming that could be defined in a sensible fashion) - there are diminishing returns - but still better.

This comes up a lot for me with programming, so many different languages, platforms and APIs are flying past us here on HN as well as languages that have been around for a long time.

So many times I see a cool new API and say that would be great to have a play around with if I had the time.

I do the same, but then realize: by doing this, I'm specializing into programming.
A random observation. Many significant advancements came when someone who knew about one thing learned about a second and realized that the first applied to the second.

Generalists are more likely to do that than specialists. And in my experience they are more fun to hang out with.

However it is true that nobody can be good at everything. Therefore one of the most important skills there are IMO is realizing who is better than you at what. In fact I consider people who can keep track of the skills of people around them to be valuable in their own right. But my experience is that people who gain this ability inevitably learn something about a lot of different things. Therefore people with this key ability tend to be generalists!

Therefore I value specialists, but prefer a good generalist more. Particularly in a small organization where people get varied responsibilities.

Disclaimer: I am a generalist and hence am quite biased. (In case that wasn't obvious.)

Many significant advancements came when someone who knew about one thing learned about a second and realized that the first applied to the second.

This is the premise to the book "The Medici Effect" by Frans Johansson, which states that innovation is found in the intersections between disciplines. It's a good read.

Avoid the extremes. Don't do one thing, don't do hundreds of things. Find a distribution that makes you happy.

Go for depth. Go for breadth. Just don't be shallow and narrow.

Scour the earth for things you love, then plumb their depths entirely.

Go for depth. Go for breadth.

agreed! what's to prevent you from going into depth in 1 (or maybe 2-3) things, and then going for breadth on a dozen more, getting decently at good at them as hobbies but not becoming world-class? seems like a good balance. 1-2 things in which you're an expert, and 6-8 more in which you're a proficient hobbyist

Scour the earth for things you love, then plumb their depths entirely.

Beautifully articulated!

(comment deleted)
(comment deleted)
(comment deleted)
"... the blogger greatly overestimates his or her skills in other areas."

The author never claims to be great at everything; merely to be good at many things. Your point is not ridiculous, but it is mostly irrelevant. To be fair, I don't think you deserved the downvote, but I can see why it might be given.

I don't think my point was irrelevant. I think the author was arguing one can be as good as a specialist in many areas.

If the author was merely arguing that one can be good at many things, then he/she was burning down a strawman, because no one who says "Jack of all trades, Master of none" is saying that a Jack is no good.

Anyway I deleted my post because I felt it was detracting from the discussion above.

It doesn’t interest me what you do for a living. I want to know what you ache for and if you dare to dream of meeting your heart’s longing.

It doesn’t interest me how old you are. I want to know if you will risk looking like a fool for love, for your dream, for the adventure of being alive...

-Oriah Mountain Dreamer, "The Invitation"

I reckon that the best way to become a generalist is a two-step process:

1. Become a specialist

2. Become a generalist

This has two advantages. Firstly, you'll be useful and employable in your youth, since a specialist with a few years' experience is a valuable asset, but a half-baked generalist is pretty useless. Secondly, and more importantly, you'll know what it's like to really understand a subject, which should help you in your quest to partially understand all the others.

If you are the kind of person who will accomplish step 1, you are probably not the kind of person who will be inclined to take step 2.