Ask HN: How to best acquire theoretical computer science knowledge?
I ended becoming the first employee at Bay Area startup. They were a bit desperate and I convinced them I could make their Android apps. I did and they are both highly rated. I also made one of my own that ended up getting featured on Google Play and receiving a lot of press. I have a bit of knack for it but I am so bored with GUIs and UIs. I want to work on harder problems and not just use other peoples libraries. The problem is that my lack of knowledge limits my thinking. I want to fix this. I am torn between going back to school for another BS in CS, pursuing a MS in CS, or teaching myself from things like MIT Open Courseware and Udacity.
I am not really worried about getting a job. This is about gaining knowledge. Although having a "real" CS would certainly help keep me from being pigeon holed as an app developer and open a few more doors. I do think that going back to school would be the quickest route to gaining the knowledge I want. I am just worried that I wouldn't get into a top school and I don't know if I should try to do a BS, MS, or just take classes. I really don't want another $50k in debt either. Especially when things like the Georgia Tech online MS CS is coming out for $7k.
So, what is the best way for a proven and largely self taught developer to take their knowledge to the next level?
121 comments
[ 10.2 ms ] story [ 2340 ms ] threadI have myself taken half a course on Udacity(did not finish) and a Startup Engineering Course on Coursera. I've started a new course on Coursera and really enjoy the material on there.
Also, I'd suggest you read a lot, the internet is a treasure trove of information. I've learnt from long blog posts and free online books. Have a go and explore, you'll reach a point where you don't have enough time to study everything you want.
Goodluck
The above is deliberate practice and learning but also keep an eye out for CS-level posts linked on HN and http://www.reddit.com/r/compsci and relate them to what you're learning. Read a post about AI data structures in Prolog? Don't understand Prolog that sounds interesting? Get a feel for what Prolog is. Write a simple program. Get a feel for implementing the data structure in a language you do understand. Then try and bring both Prolog and the structure together. Rinse and repeat with anything that interests you. Again, be sure to TAKE NOTES or you'll forget things using this scattershot approach.
It'll take a TON of time but if you keep your sessions relatively short, you'll get a shallow knowledge of all of the major CS areas quickly, and then you can use your job or natural interest to drive you into going deep on areas that affect or motivate you.
I'm starting an MSc in software engineering later this year but my own CS knowledge has mostly been learned in the above ways (that is, I don't have a bachelors in CS).
Clarification: As cliveowen got me to realize (below), the above approach might not be for you if you have a very specific target in your career development. My experiences and recommendations are specifically around getting a broad level of experience rather than aiming at a specific role (which, perhaps, you should be doing).
In my case (and what I felt was the case for the OP) a general, broad level of knowledge is paramount. I then study deeper on an as-needed basis off the back of my broad but shallow knowledge. Being able to tie together the basics of many unrelated areas really helps in my work, but is surely not for everyone. I already have my dream job for life so my goal is more on becoming well rounded rather than getting a specific job.
From a career POV, should the OP specifically focus on a specific target rather than just aim to get a general CS understanding? Quite possibly! But that's a different discussion/recommendation that I think you could kick off in your own response to OP. So I'm voting you up not because I agree with your criticism, but because perhaps the OP does need to think more solidly about the end goal.
How do the processes behind your webapps work? Why or why can't your app support 10,000 clients hitting it at once? What's happening at the HTTP level? The TCP/IP level? Just understanding some of the algorithms behind routing and TCP congestion avoidance could give a ton of food for thought, still get you into learning algorithms, notation, and reading CS papers, but still be more directly beneficial to the day job. I wish you luck however you end up approaching it! :-) The only bit I want to repeat is take a lot of notes.. do blog posts, write articles.. anything to ensure you retain that knowledge because it's so easily lost as you get older.
Also, read actual code. Want to know how diff works? Read diff. Getting better at code forensics is the second-best way for a self-taught programmer to learn real CS.
EDIT: I am a largely self-taught programmer.
My problem with teaching myself is that it's hard for me to stay motivated after the first couple of weeks. Now this just me -- it's not you. So maybe you don't have this problem at all, but I've found school to be a good motivator.
Plus having a good professor can help get through the tough spots. I studied complexity under Papadimitriou and he was great at explaining things I struggled with.
One problem with school though is you'll have less flexibility to skip things you don't care about, and investigate those things you do. That can be a blessing or a curse.
That being said, theory plays a small role in everyday development. If your primary goal is to increase your knowledge as it relates to development you don't want to focus on theory until it becomes a pain point. Algorithm theory helps with scalability, but practice is far more useful to programming. Conferences in your field that can expose you to new things to learn and different tool chains in your language are valuable. Learning a new related language can also be valuable you might consider learning iOS and then JavaScript which would let you work on Android, iOS and Cordova (formerly PhoneGap) applications at a level where you'd be comfortable going from JavaScript to native to write unavailable features.
If you're happy and have good opportunities on Android, focus on getting a deeper understanding: what are good open source repos, what coding conventions do they use, what frameworks and approaches could you know better. Are there parts of the OS you haven't worked with and wanted to? For instance, I built a demo app for LG designed for in-store promo devices where we hacked around in the OS to intercept the display of screens that would let the user delete apps from the phone and throw up a password lock that would kick them out to the main screen. Try and get a sense of where your best (however you measure it) opportunities are and focus your learning in that direction.
Get another job that will force you to expand your knowledge. I am sure your job experience with your first start up is serving you better than a CS degree.
Similarly, a job which involves doing low level computer programming would force you to learn a lot of very practice CS, and you would be getting paid for it instead of the other way around.
There was another reading list that I remember seeing, I think it was from Stanford's TCS website, but I no longer can find it.
Self-teaching is fun because you get to choose your own curriculum, but it's often frustrating too because if you get stuck, there is no professor or TA to unstick you. This issue can somewhat mitigated via the internet.
In any case you first need to investigate what is interesting to you, free resources are far from rare and you can probably build yourself a decent curriculum from MIT OCW, Coursera, Udacity and similar websites which you can then study at your own pace with videos and books. If finding a job is not your concern I wouldn't advise putting yourself into debt. See http://www.saylor.org/majors/computer-science/ for an example of curriculum.
[1] http://www.amazon.com/Introduction-Theory-Computation-Michae...
[2] http://www.amazon.com/Introduction-Theory-Computation-Michae...
http://used.addall.com/SuperRare/submitRare.cgi?author=sipse...
It depends on how you learn. Some people will recommend Coursera et al., which are great. I tried some Coursera courses, but found that I found the pace too slow and became bored quickly.
For me, reading always works the best, since I can adjust it to my pace. You could look up a curriculum (as someone else suggests) and compose a reading list from that. Also, there are some works where you really can't go wrong. Some examples:
- Structure and Interpretation of Computer Programs, Abelson and Sussman
- Algorithms, Sedgewick, Wayne
- Introduction to Automata Theory, Languages, and Computation, Hopcroft, Motwani, Ullman
Lesser known, but incredibly fun books:
- Purely Functional Data Structures, Chris Okasaki
- The Reasoned Schemer, Friedman, Byrd, Kiselyov (or a good Prolog book).
After learning the foundations, you could branch out to a subfield that interests you.
(Or as Frank Zappa has bluntly put it: “If you want to get laid, go to college. If you want an education, go to the library.”)
http://www.cnn13.com
* Sedgewick has no proofs in his book (or at least very few). CLRS has a lot of proofs in it.
* CLRS uses pseudocode, while Sedgewick uses actual code. I think they use Java now (it was Pascal in the edition that I had used).
* Sedgewick is really a practitioners introduction. It views the algorithmic problem as the jumping off point. Whereas CLRS is more an intro for people who are going to study computer science -- so it focuses on the methods.
Both good texts. I think for most programmers I'd recommend Sedgewick, but if your goal is to be a computer scientist, then I'd recommend CLRS.
You can watch the lectures at 2x speed. ;)
I don't think the OP is in it for the certificates.
Even so you want to be smart about it, and gain the knowledge in such a way such that employers will recognize your expertise. It is for this reason that I would rather recommend the MS. It will at least get you an interview next time you want to change jobs.
First, you need to figure out exactly what you want to learn. By theoretical, what do you mean? If your goal is to be on the same level as other programmers who are CS grads, then you might consider digging into algorithms and data structures and discrete math. This is pretty much the base of anything theoretical you'll do in computer science.
If you're in it to learn, go the udacity route. If, after completing all of their courses, you still feel you have some gaps, then you can go back to school if need be. If, on the other hand, your goal is to go into research, then you'll just have to bone up and go back to grad school to be taken seriously. In research, credentials matter.
6.004: http://ocw.mit.edu/courses/electrical-engineering-and-comput...
6.042: http://ocw.mit.edu/courses/electrical-engineering-and-comput...
CLRS: http://mitpress.mit.edu/books/introduction-algorithms
Maybe better to choose a particular hard problem that is interesting and focus on that. Something that would be neat to do but isn't as simple as just calling a few library functions.
It also depends upon what you consider to be a deep level?
Are you interested in electronics and signal processing, mathematics or the structure/design of programming languages? Are you interested in individual computers or distributed systems? Are you just interested in computers or are you interested in using computers to model other things, like economics, physics or virtual worlds?
At a minimum you could brush up on algorithm / data structure understanding and some of the math behind that. Problem is that as soon you gain understanding of something it will simply expose you to deeper subjects that you don't fully understand. You will never reach the bottom of all of these rabbit holes.
Ultimately it depends on where you see your shortcomings that are preventing you from doing what you want. Do you wish you had better credentials so that you could get a job with better pay or more interesting work? Or do you find yourself frustrated because you are not able to sufficiently grok a certain subject area.
Start with a course on computer architecture / digital design. Something that'll have you put together a simple 8 bit CPU. Move your way up with a course on operating systems - processes, swapping, cache invalidation, file systems, drivers.
Then, maybe something on compilers? Or how to write your own interpreter.
Finally, a good networking course is pretty important.
Very little of this is theoretical compsci, but it's something that will get covered in a degree program. That said, knowing how to recognize what kind of class of algorithm we're looking at, and data structures all around.
You will learn everything from logic gates/transistors to software.
Subjects like that give you a firm grounding in the area of computer science but you could study them for ages especially on your own. These subjects are also not very practical in terms of everyday engineering and on top of that before you even get to that level, a top school would require you to study a bunch of mathematics to gain a certain level of rigor of your thinking. My guess is that this is hard to do on your own.
On the other hand, since you have a related degree already and since you know programming, I would heartily recommend you to study on your own combining two approaches: bottom-up to learn the basics and top-down to immediately start increasing your market value and to start opening doors and to keep yourself motivated to go forward with the more tedious study of the basics.
E.g. say you are interested in distributed systems - you can start reading up on them (top-down) while digressing here and there to learn some of the basics.
Make use of online resources like Coursera for the necessary basics (bottom-up): at least some maths heavy on proofs [1], at least some complexity theory, at least some graph theory, ....
I think a purely bottom-up approach (going to a university) could be a waste of time and money in your case. But you need to emulate it a bit because a purely top-down approach could be too superficial and wouldn't teach you some of the more rigorous thinking you might need.
Whatever you do and learn try to do it in-depth. Superficial knowledge of many things will not benefit you much in the long term while in-depth study of many computer science fields will likely lead you through a series of small enlightenments.
[1] My professor of linear algebra used to insist that we as computer scientists have to know all the proofs of the theorems we were learning - in contrast to mathematicians learning the same subject - simply because we have to know how things are done to the last detail and we have to develop that kind of thinking.
I don't know that I agree with the poster above with respect to:
"a top school would require you to study a bunch of mathematics to gain a certain level of rigor of your thinking"
Most good schools require only two courses in "pure theory": first an introduction to discrete mathematics, where you learn about logic and how to prove stuff, a smattering of number theory/crypto, graph theory, probability, and automata theory; second an algorithms course that teaches you to analyse and construct algorithms according to certain design methods. Together this is really only a year of mathematics, and both courses are quite fun.
Note that discrete mathematics as taught by a good CS program is NOTHING like the math you learned in high school or elementary school, it teaches you to think creatively about mathematical structures and then justify that creativity with logic.
If you're already a developer, you'll probably have a bunch of latent structures hanging around in your brain that will let you have an easy/fun time with the material in these two courses. I suggest that you study discrete mathematics and algorithms in depth, to the level of some decent undergrad course on each. Learning this material will give your thinking the sort of quantitative/logical "edge" you might want -- these two courses contain the basic mathematical tools common to much of computer science. I do agree with the post above that they are not the most practical things in terms of everyday engineering, though.
Addendum: if you find that you really enjoy the mathematical angle and you think machine learning is cool, also learn linear algebra, because a bunch of machine learning theory and implementation relies on linear algebraic algorithms and concepts. Linear algebra actually would be practical to know if you ever do anything with data analysis.
Even at good schools (in the US, anyway) the calc up to multivar that is required isn't done with what I would call good proofs. They'll generally do epsilon-delta definitions of the limit and a few other fun things, but will fall far short of a truly rigorous development of the calculus. Same goes for many introductions to linear algebra and stats with calc courses. Some math departments use linear algebra to introduce proofs and creative mathematical thinking, though, so this isn't always the case.
So if someone is looking to develop how they think in mathematics and they already have a good programming background, they are much better served by taking a discrete math course than any of those sorts of math courses. If they then want to learn calculus, I would recommend self-study from Spivak's "Calculus" book which is a fun and elementary but rigorous treatment of the subject. You get to see not just how calculus works, but why it works, which is sorely lacking in most undergrad calc courses, even up to multivar.
If someone wants to develop their mathematical thinking and is good at programming descrete math isn't going to help them much because they have probably already seen much of it. They need to be exposed to high level calc and linear algebra to develop their mathematical thinking.
My background: I did a BS in CS/Economics but spent most of my time playing online poker and skipping class. I managed to get a job I love on mostly potential, and I've had a great time re-learning all the stuff I was superficially exposed to but didn't work that hard at.
The most important thing is caring about what you do and taking ownership over your own development.
If you're going back to school then you should definitely consider an MS. Maybe you're worried that you wouldn't be accepted or that it would be too challenging? In that case, narrow down a list of universities/departments where you would like to enroll, and identify what you need to study in order to get accepted.
Alternatively, teach yourself, and try to join a top tech company that focus on making the kind of software you're interested in. Working with great programmers has a large impact on your learning. Perhaps you're underestimating this impact because your first job wasn't challenging enough.
Education will not teach you anything, but it will give a great opportunity to learn. Many will simply learn enough to get through exams and then promtly forget everything again.
You will also get introduced to concepts and areas you would never even think of exploring otherwise. I thought graph theory sounded boring, but now I enjoy it. Formal languages and automata theory? Physics? Electronics? Hardware design? These are topics I would never have explored if it wasn't for going to university.
Artificial intelligence, compilers and operating systems are topics I probably would have explored, but I sincerely doubt I would go as deep as I have.
Also do not underestimate the effect deadlines can have. Procrastination is a constant evil in my life somewhat negated by school forcing me to do things.
I would say pick up a single system, say database (relational or document) or routing, OS, DNS, Web server, A compiler, Filesystem, NFS, etc. and study the hell out of it.
Once you know how to understand a particular system in depth, everything else will start falling in place.
Read lots of papers. Start with easy ones to learn how to best digest highly academic technical writing then keep pushing.
CS is remarkable in how much of its academic knowledge is online, available, for free. Taking advantage of that resource is beyond key.
If you know roughly what you want to learn, search out a seminal paper in that area. If it's deep CS, you'll probably find a good one from the 80s. Seek out every citation that's proximal to a sentence in that paper that confuses or excites you. Search on Google Scholar for all the recent papers that cite the one you found and begin eating up the chain that direction. When you find a deep topic you want to learn more of—set theory, logic, formal languages, discrete math, matrix analysis, automata theory, geometry, topology, &c.—don't be afraid to look for a good book on the topic. There are resources around the internet answering "What's a good introductory book to X" for all kinds of X.
If you don't know the general area you're interested in then I recommend the exact same strategy but with greater emphasis on following citations that make connections to other areas of CS. Additionally, pick a few disjoint topics (Category theory, system architecture, graph theory) and follow them all simultaneously looking for connections.
Formal languages, automata, and process algebras form a really fundamental mathematical course of study that I'd highly recommend to anyone interested in "why" CS works. Optimization, matrix analysis/linear algebra, and diffeq form a great basis for simulation and machine learning.
Generally, for any of these routes, learn to find your boundary of knowledge. Studying source material is very challenging because any given paper will tend to assume many things about your background. Most of those assumptions will be false and lead to density of the paper. You can often power through without them, but you can also take it as an indication of a new place to study.
Finally, see if you can find a study group or journal club. I'm not there, so I don't have firsthand experience, but I'm sure there are many in the bay area.
(Oh, and if you read mathematical papers/books don't skip the proofs and exercises. That'd be like reading the iOS documentation without ever programming anything.)
[1]: http://ccr.sigcomm.org/online/files/p83-keshavA.pdf
Many of the classics are in there, such as the Plan9 paper and Paxos Made Practical.
If the images are distracting, here's the list without them: http://dl.dropboxusercontent.com/u/315/articles/index_no_ima...
Actually, looking over it, that list isn't very curated. Just Ctrl-F "unix" and that'll take you to some good papers.
What sort of projects are you interested in?
http://blog.fogus.me/2011/09/08/10-technical-papers-every-pr...
Tackle a big project: write an extensible text editor, design and implement a language and runtime, design and implement a new garbage collection system, or the currently quite popular RSS reader and feed provider.
http://www.ccs.neu.edu/home/viola/classes/gems-08/
In terms of "taking knowledge to the next level", though, it could mean anything from you wish you knew what red-black trees and big-O notation were, to you'd love to be able to design your own programming language and write a compiler, to you wish you could administer a Linux box. I would actually suggest the best approach here is to find someone with the job you want and talk with them 1:1 to figure out where your knowledge could be most improved.
Also, for Coursera, etc, there are local study groups at places like Hacker Dojo you could join to get some of the benefits of college without the expense.