Ask HN: How do I choose the right resource to learn CS fundamentals?
For example:
For intro courses:
* Computer science an interdisciplinary approach (princeton)
* CS61A - UCB
* Introduction to CS and programming (MIT)
* Stanford
* CMU
Data Structures and Algorithms:
* Princeton Algorithms
* CS61B - UCB
* Stanford Algorithms course
* MIT Algorithms
* CMU
Apart from this you have multiple books on each topic - Data Structures/Algorithms, Discrete Mathematics, Theory of Computation, Operating systems, Networks, and so on.
Apart from these you also have resources like teachyourselfcs, ossu, functionalcs.github.io/curriculum/.
I am attracted by the resources/online/books posted by courses in UCB/Princeton/MIT/Stanford/CMU. At the same time I get boggled down and overwhelmed that I have soooooooooo many materials to cover. Intro courses aren't that big of a deal since I am able to recognize/solve most questions fairly easily in multiple resources. But my next step of Data structures and algorithms is overwhelming that I am unable to start somewhere.
How do you recommend to choose the right resource (online/book) for each topic/course? Is it worth going through multiple university courses/books for the same topic?
92 comments
[ 3.3 ms ] story [ 168 ms ] thread- Programming: Learn two languages: Python and C
- Algorithms and Data Structures: Implement each data structure in the two languages above and implement a few algorithms of each type.
- Computer Architecture: For the referred excellent book, implement all assignments in any one language. Go head and burn the design on an FPGA, get the computer running on real hardware.
- OS: Having done ECS above, you should be in good shape to write your own OS: there is xv6, Xinu, Minix and many to choose from. Again have your OS running at least in a VM.
- Computer Networking: Write your own HTTP server in C.
- Math for CS: I would say focus on learning math essential for games, some linear algebra and leave it there. When you encounter a relevant field; AI or games, you should be in a position to pick up more math if required.
- Databases: Recently a book has been published on database internals, which is strongly recommended. Work through this book.
- Languages and Compilers: Learn a lisp, write a lisp interpreter (should introduce you to some FP concepts) and then working through Concepts, Techniques and Models of Computer Programming should be a good foundation.
Whether you are a student or working full time, these above are time consuming but well worth the ROI if you put in the effort. Be creative and ensure that you publish all your work as part of your portfolio. Good luck!!!
[1] https://www.amazon.com/Database-Internals-Deep-Distributed-S...
[1] https://www.amazon.com/Designing-Data-Intensive-Applications...
I had difficulty implementing data structures in C, not in python. Python I was able to think in terms of classes and attributes. But I was finding it difficult to do the same in C since there is no concept of classes. I am still trying to learn pointers properly to have an understanding how to implement data structures and algorithms effectively.
I came across the book you have recommended and it is a very nice book. I would recommend that along with Designing Data Intensive Applications.
Thank you.
As the url suggests, it's got an agenda about interview prep but the materials do not - they're straight up CS resources. Like I said, this may not be exactly what you want, but it could be helpful.
On a different note, I have completed the Stanford Algorithms Course part 1 (currently working on part 2) and I can recommend it.
https://ocw.mit.edu/courses/find-by-topic/#cat=engineering&s...
Introduction to Computer Science followed by Data Structures and Algorithms should give you a healthy start.
--
Learning these fundamentals is useful, but not necessarily immediately practical. Building and doing is the best way to learn. This is a good start, and the fundamentals will certainly give you an edge against most people graduating from a bootcamp, but after this I'd recommend finding a good tutorial, whatever the language that teaches you step by step how to build XYZ... I learned ruby/rails by doing Michael Hartl's tutorial building a microblogging platform like twitter.
1. Following the courseware is not always free, starting with the textbook
2. There are no milestones or credentials to track your progress
3. I found little community to interact with
This is just anecdotal. Some of these may have changed, be different.
Has anyone here worked through many of these courses themselves?
Did you even take courses there? Have a different experience?
nand2teris.org and cs50.net
See if you can solve all the projects & assignments in these courses.
These cover a lot of ground. Once you are done with these two courses end to end, you will get a fair understanding of many of the computing big ideas.
If you already programming, they may be too basic.
I really wish there is a course that uses C (like first 4-5 classes in CS50) and uses C ONLY and then go as deep as possible, similar to CS61A, but apparently on other topics (one system programming project?)
Compare the second homework assignments, which are supposed to be given after 2 weeks (~6 days) of instruction:
15-112: https://www.cs.cmu.edu/~rdriley/112/assignments/02/
CS 61A: https://cs61a.org/hw/hw02/
The 61A assignment forces higher order functions way too early, and it's often discouraging to people just starting out - and it's not even common (in my experience) to write code like this anyways. I also like the way that the CMU assignment writes tests as simple functions, without magic docstring tooling.
Don't skip the code tracing sections, where you read code and write the output by hand - it helps you understand and internalize what the code is doing, rather than just running code samples over and over again to see how it works.
Also, there's some nice notes on debugging https://www.cs.cmu.edu/~rdriley/112/notes/notes-debugging.ht...
[1] I took this version, which is ordered a bit differently http://www.kosbie.net/cmu/spring-16/15-112/schedule.html
Other wise you are just looking for an instruction manual for life, which doesn’t exist as such.
I think you'd be better served by doing a lot of courses from the same institution. If you choose courses that were designed to fit together, you'll have a lot less wasted time from overlapping material or holes from missing material.
> Is it worth going through multiple university courses/books for the same topic?
What is your goal exactly? Is it to evaluate and compare everyone's materials?
If your goal is your own learning, then why wouldn't you take a more advanced course from the same institution instead of repeating a slightly different version of what you've already learned?
I agree regarding choosing materials from a singular institution. Thing is, not all the institutions have all the courses and materials online. This in turn results in choosing specific courses from specific institutions which are available online. But the caveat here is that when I intermix the courses from different institutions, I find the prerequisites to be different since different institutions cover same/similar courses in different depths/breadths. For example, Databases in CMU are at a much higher standard than that of Stanford's.
> What is your goal exactly?
Goal is to learn since my undergrad is not in CS. I would not be able to afford a masters for the next 2-3 years so I am trying to use the intermediate time to get myself the undergrad curriculum knowledge I never had. I hope this would help me with enhancing my day to day job and also help me prepare when I apply for my masters/PhD.
while I generally agree with your idea, a little counterpoint: it depends on a way you consume those materials. If you do that thoroughly, actually read and analyze source codes or examples, try things on your own as instructed, complete excercises and so on there is absolutely no point in doing multiple thins on the same topic. But if your preferred method is fast skimming, skipping some excercises and so on then skimming two or three books/videos/whatever on the same subject could make some sense. The latter method is very chaotic and is probably not the best idea to apply it for learning the fundamentals, but I just wanted to point out that such a learning technique does exist and sometimes makes sense, especially if just the general understanding is your goal.
One of my favorite mentees came in to a software development job from a biochemistry background. She’s got a knack for it and did a great job self learning a lot of basics, but I really saw some awesome improvement after she did a “CS masters bridge program” through NYU- https://engineering.nyu.edu/academics/programs/bridge-progra...
I guess it’s really meant to get you into a grad program, but I know she got a ton out of it as a career programmer.
If it's not too much to ask, would you mind being my mentor online in some way? I would really like to learn from someone with some direction and hold myself with some accountability too.
My suggestion would be to enroll in a college and pay for your education. This way, you'll be more motivated because the pace will be imposed and it will cost you more than your time if you don't do the work required to pass and more importantly, learn something of value.
But I think at least one should be able to learn 2 topics by oneself: A programming language, Basic data structure and algorithm as well. Then he can learn the others leisurely.
This book has very little in the way of prerequisites, and it covers a lot of fundamental algorithms and a little bit of Math, but it is a lot more accessible and didactic than, say, the Cormen book. I don't think you need any other resource for studying the book (videos, forums, etc.) other than maybe the website for convenient access to the source code (and maybe some other Java reference, although you don't need deep Java knowledge to understand the code).
Note: I'm plenty biased for this resource... I don't know why, but I find the implementations so elegant an easy to follow, even though the source code is Java, a language I admit sometimes can look anything but elegant (in its production form with all those imports and redundant type signatures :-). Many algorithms are implemented using data structures introduced in previous chapters, so it makes sense to read it cover to cover.
For Computer Architecture, Nand2Tetris [2] is another resource that comes up often and for good reason. I only worked through half of this book but I really like it too and have it on my back burner to complete the second half of it: first part is about implementing a computer from the ground up (nand gates being the "atoms"); second part is about implementing a parser/compiler for a higher level language targeting the same computer.
1: https://algs4.cs.princeton.edu/
2: https://www.nand2tetris.org/
Its also focused on how to solve problems than Sedgwick. Algorithms explains some elementary algorithms that noone is ever going to code themselves. CLRS teaches you what to look for in algorithms for solving particular problems.
Hence while it may be nice to see the Java code, I personally prefer pseudocode.
I did the intro course from the sedgewick and wayne from coursera along with their book and loved it. Would recommend it to anyone who likes a decent challenge while learning. The book is pretty dense.
Watch the videos, then read the relevant section of the text book, and don't miss the assignments which were fantastic ways to learn about the nuances of writing efficient algorithms and data structures.
If only every programming MOOC put that much effort into their autograders...
I really do not recommend videos/online classes for algorithms. The primary reason is that if you are learning them for the first time, there is a lot of information to get through and using the book format will force you to slow down and ensure you are learning things at your own pace.
I also highly recommend "Grokking Algorithms" as a easy primer. It seems a little "basic", but it really is a great way to quickly pick up the basics of algorithms, an a great refresher. I used it in parallel to TADM and helped me get unstuck a quite a few times.
http://www.algorist.com/
https://www.manning.com/books/grokking-algorithms
https://github.com/gregors/standard_draw_tk
Thank you!!!
I'd also suggest a top-down approach where you start with a flexible end goal you'd want to achieve (e.g. be a full-stack dev), in which case you can start by babystepping a hands-on approach (e.g. learning javascript, learning client-side and server-side of things). Complimentary fundamental course outlines can also help, e.g. Comptia A+ gives you hardware fundamentals, Comptia Network+ gives you networking fundamentals, CloudAcademy can get you started on working with cloud providers like Azure/GCP/AWS, and so on and so forth.
It's easy to get lost in the theoretical side of things, being able to test them out in action as soon as you can could give you quite an ideal balance.
What I do is to create spreadsheet with 4 columns:
- "Topic": the topic name, e.g.: "self balancing trees".
- "Material": did I read/watch the material for this topic.
- "Notes": do I have my own takeaways or notes about this material.
- "Code": do I have code for this? like a Jupyter notebook or source code, etc.
This approach worked for me, and helped me get better at online tests and interviews.
If you think you already understand a concept, you can just move on to another topic.
You can enroll for free at: https://www.edx.org/course/cs50s-introduction-to-computer-sc...
I never went to University and didn't take a lot of CS courses but since I create web development video courses I wanted to be able to reference other courses to folks who asked so I took CS50 a few years ago to vet it. It was a great intro course and I even learned a little bit of C in the process. David Malan (the head instructor) is also top notch.
You can still switch to edx as soon as it's live there.
In contrast, I like the MIT/Princeton/UCB courses since they use an easier language to start with and also introduce C in the later courses for systems programming.
I'd say 50% C and 50% Functional, and remove the web part, really superficial and boring. And then go as deep as possible. Can even intertwine the C/Functional part, e.g. write an interpreter for a subset of the functional language in C as the last large project.
Going through hackerrank tasks is a great way to practice. Also Cracking the Coding Interview and online interview questions.
The good news: It probably doesn't matter that much. Different programming books will give you different strengths and weaknesses, and that's a good thing because teams need diverse skillsets. It's been years since I wrote production code in a strongly typed, statically typed language, but I did that for so long that I still think in types, and I bring that structured way of thinking to the table in Python/Javascript, which means I avoid a lot of the mistakes those languages allow you to make. I've worked with people who were very comfortable with metaprogramming, which always feels messy to me because it inherently muddles types. It's not a tool I ever reach for, and I push back when people add metaprogramming to codebases, which is a good thing, because too much metaprogramming can be bad. But sometimes, judiciously used, it can be very effective, so people with metaprogramming skill will solve problems that would be very difficult for me to solve. If we had both read all the same fundamental programming books, we'd never have these diverse skills that make our teams effective.
TL;DR: Pick the first one from a reputable organization and do it. Time spent agonizing on this decision is time wasted.
+ Bach's The Design of the Unix Operating System describes how operating systems and file systems work. Even Windows describes it's similarities and differences using the same terminology.
+ Hellerstien's Berkeley CS 186 Intro to Databases lectures. Databases get more love at Berkeley than elsewhere. https://archive.org/details/UCBerkeley_Course_Computer_Scien...
I have _never_ had any real use of TAoCP. I own all volumes and have really tried to make use of them but they feel more like a trivia book than something you can actually use in your work.
Sure, it's cool to own and recommend it but have people actually used it?
To end on a more positive note: I highly recommend "Introduction to Algorithms" by Cormen et al. Not as cool but very useful.
Per the question, the context of my answer is learning CS and in particular self-directed learning. If the question was about resources for work, I'd probably say "StackOverflow" and be done with it.
(Based on my experience as CS undergrad and grad student later)
That's what it exists for. To build the street cred of people who own it and have read it. Notice how no one who has read it comes back with a comparison of TAOCP and CLRS. For example "learning bogosort from Knuth instead of Cormen is better because Knuth gives you xyz insight into it". No, they simply think it's better because it's harder.
A rough comparison is On the Origin of Species, which is a solid foundational text without being too arcane while Philosophiæ Naturalis Principia Mathematica is not. No sane person suggests that to learn the Newton's laws of motion you need to learn Latin and then read Principia (except the people who fetishize difficulty). Just read Resnick and Halliday instead.
By no means am I downplaying Knuth's genius. He is a stalwart without whom our industry would be worse. But TAOCP is not a good foundational text because it's not approachable.
The two are hard to compare, IMO, because their approaches and intended audience are different. CLRS is an undergraduate/graduate textbook on algorithms, where the authors have selected a broad range of topics based on things like their suitability for teaching or coursework. TAOCP on the other hand treats a narrower (and rather different) set of topics in greater depth, each section being essentially one person's excellent distillation of all published literature on the topic, passed through a tasteful interestingness+usefulness filter, and presented in expository style. To expand on that:
* CLRS treats algorithms as an academic subject in its own right, while Knuth is more concerned about what is actually useful to a programmer who is interested in writing efficient programs. For example, CLRS has an entire chapter on Fibonacci heaps (and the 2nd edition also had one on binomial heaps), which are great theoretically (amortized constant time for some operations), but impractical to implement and (because of the constant factors) hardly worth using in practice. In TAOCP you'll not find them mentioned. (Though there are Fibonacci trees, the search trees that arise as a result of a simple and easy-to-implement algorithm called Fibonaccian search.)
* Compare the table of contents: https://en.wikipedia.org/w/index.php?title=Introduction_to_A... versus https://en.wikipedia.org/w/index.php?title=The_Art_of_Comput... (Knuth wrote down the table of contents in 1962 and has been writing to it ever since; the ToC of CLRS is of course a function of present-day fashions among algorithms researchers.)
* Roughly, CLRS, like algorithms researchers, treat algorithms as a branch of mathematics, following the idea-theorem-proof structure (and sometimes overdoing it: https://www.goodreads.com/review/show/155959101), writing only pseudocode, etc. It is possible to go through the entire book without ever writing a program or even feeling the urge to write one. (Not saying that's wrong.) Your thinking stays at a uniform level, roughly the "idea" level of thinking about the algorithm and what it does, proving things about it, etc. In TAOCP it's common to find an idea/algorithm described on one page informally, then a page later (rarely, when warranted: https://news.ycombinator.com/item?id=14520230) have an implementation in assembly language (MIX/MMIX), with a pointer to an exercise that asks you to analyze how many times a certain register is modified or whatever. In TAOCP, we always have mathematics in the service of algorithms -- the idea is to take an actual algorithm/program that can be implemented, then analyze it using whatever mathematical tools it takes (many of which were invented by Knuth... in fact at one point he wanted to name the books "Analysis of Algorithms", but the publishers didn't think the title would sell).
* For a concrete example: just last week I read TAOCP's section on tries (Chapter 6 Searching (Vol 3), part 6.3 "Digital Searching"). In CLRS there's a passing mention of tries (radix trees) only in one paragraph (Problem 12-2 in Chapter 12 Binary Search Trees), which asks to prove that using this data structure a bunch of binary strings of total length n can be sorted in O(n) time. This makes sense I guess because mathematically/asymptotically there may not be much more to say about them. But t...
> TeX was designed for typesetting, not for programming; so it is at best “weird” when considered as a programming language. — DEK, Digital Typography, page 235
> In some sense I put in many of TeX's programming features only after kicking and screaming. — DEK, Digital Typography, page 648
> So TeX is a programming language but I had to put in those features kicking and screaming. […] In a way I resent having every language be universal because they’ll be universal in a different way. […] I was really thinking of TeX as something that the more programming it had in it, the less it was doing its real mission of typesetting. When I put in the calculation of prime numbers into the TeX manual I was not thinking of this as the way to use TeX. I was thinking, “Oh, by the way, look at this: dogs can stand on their hind legs and TeX can calculate prime numbers.” — DEK, interviewed in Peter Seibel's Coders at Work
> Michael Plass, who was Don's student and worked with Don on TeX, told me once that "Don tried very hard not to make TeX a programming language. Unfortunately, he didn't succeed." — Norman Ramsey
I've written more about this verbosely elsewhere [2], but in any case even if TeX had been intentionally designed from the start for public consumption (rather than for just him and his secretary, as was his original intention), it hardly justifies something like "terrible computer scientist" — TeX/MF/CM was just his side-project; the computer science is different.
[1]: (Accidentally Turing-complete) https://www.gwern.net/Turing-complete http://beza1e1.tuxen.de/articles/accidentally_turing_complet... https://aphyr.com/posts/342-typing-the-technical-interview https://www.youtube.com/watch?v=uNjxe8ShM-8
[2]: https://tex.stackexchange.com/a/384881/48 , https://cstheory.stackexchange.com/a/40282/115
OP is trying to get into CS, so they need something that's clear and approachable. By your own admission, it's possible even for experienced people to be stumped by TAOCP. It's simply too vast and too difficult for a beginner to finish. This fetishizing of difficult things doesn't help anyone, least of all people trying to get into the field.
The failure mode to watch out for here is not OP getting developing a small misconception about something, but that he finds the field too intimidating to continue.
It's far more likely that a beginner has success with the other suggestions from this thread - like Nand2Tetris, which is clear, explains the fundamentals, is accurate and most importantly, fun.
Nand2Tetris is a great resource. There are lots of great resources: Let's build a compiler, SICP, Paradigms of Artificial Intelligence: Case Studies in Common Lisp, Code Complete, Code, etc. and on and on. What to choose is the OP's problem.
Nothing is as robust as Knuth. It's stood up longer than most Hacker News readers have been alive. And is still keeping up...or rather still at the sharp edge. The first edition of Volume I is more than fifty years old and still entirely relevant. Binary trees, queues, and stacks haven't changed. Neither has the math.
What has changed is that teaching computer science has become an industry. It perpetuates itself via gatekeeping. By withholding knowledge. An industry that parcels out knowledge in semesters. But CS is a lifetime field...Knuth started in 1962.
The foundation of computer science would be the work done by the likes of Alan Turing and Alonzo Church. They established foundations of CS before Knuth was born.
By engaging I mean doing exercises, implementing algorithms in code, writing pet projects.
This will ensure you understand a topic, not have an impression that you understand. In past, I made a mistake to rush read textbooks till the point I feel I understood the content and ignoring most of the exercises - the exam showed me that I was wrong :)
At the same time, I would not suggest to apply engage mode for every book and chapter - your progress will be slow, and you may lose motivation deep in the middle of some thick textbook, as you may not be able to relate very detailed information there to your current expertise.
Personally, I apply the combination of skimming and engagement.
Don't sweat the choice too much, choose one of the canonical books, and commit to mastering the material by doing, not by (solely) reading.
...or use any other book or lecture series. Stop worrying about choosing the "right" resource; just find a resource and start learning.
You end up implementing the algorithms and data structures yourself in your language of choice, which I find is a good way to learn.