Ask HN: I feel like an 'expert beginner' and I don't know how to get better

Build something that is doable but will push your current skills. Choose a project that appears slightly beyond your current capability. That is the only way to truly learn.

Solve real problems - write programs that "scratch your own itch" - and learn as you need to. It's hard to tell how well you're learning when you don't have actual results to go by.

Honestly though that set of interests doesn't seem like the sort of thing that lends itself to real problems; it sounds to me like you've already got more than enough CS theory. If you're looking to learn the kind of programming that will make you better at doing it professionally, you need to start making practical things. Find a hobby that could do with a website/app or something else in your life that can be automated.

1. Keep trying.

2. Books and papers are great, but you should also try to find a mentor / guide / teacher / expert to talk about the issues you're facing and what you're thinking about. Schools are organized the way they are for a reason, even if they often fail at their intended purpose.

Love what you do and keep doing it, after 25 years I still aknowledge that I am a beginner in some areas and an expert in other.

Above all, always have fun!

I'd suggest drilling down into ONE subject area. If you want to become the world's foremost expert on NFA to DFA conversion techniques, go for it!

Forget about trying to learn EVERYTHING. Pick one thing, learn it well, repeat with the next thing.

As mentioned, the more you can do this within the confines of projects you feel passionately about, the easier it will be.

I'm going to recommend if what you're doing isn't working for you, that you should try a different approach to see if it can get you the momentum you're looking for, in addition to what you've already been doing:

Take a break from focusing on the strategies and tactics of programming. No single algorithm, framework or language is going to unlock the panacea of potential inside of you. Most problems simply aren't that complex in the beginning.

Having the mindset of an innocent beginner always is the toughest thing to maintain to remain a problem-based thinker instead of a solution-based thinker.

The best way I've found is simply to solve problems the best I can, and when I learn what I could have done better, if the need is there, go and refactor it. No matter how great you are, or aren't today, what you write may look bad in 5 years because you have more experience.

Most things we build as developers become obsolete. It's a separate discussion but I'm not sure what you're trying to optimize, your skills, or a result in a project?

No developer is a a factory of churning out code or results at the same speed. Be less focused on the practice of programming alone and look at the results you're creating.

Software is as much a craft, an art, as much as a technical skill, and maybe it's something for you to explore the other fronts.

Simply put you have to start practicing. Try to solve problems with the techniques you've learned, and you will get a better understanding of their strengths and their weaknesses. That will help you internalize the theory you have absorbed. It does take time (this annoys most people) but it serves the exact same purpose as homework does in school, moves the understanding from the hippocampus out through the rest of your brain.

Maybe try something pretty. Information visualization with http://d3js.org/ Or make a game with cool AI.

Whatever you do, as long as there is a minimum amount of fun in it, you'll get better. And the more you know, the better you can identify what areas you have and/or want to improve in.

PS: I guess what I'm looking for is a 'work-out' type procedure. Like practicing to run a marathon, you work out and run a lot (I assume). What should I do in order to 'work-out' my CS skills? Musicians practice, athletes work-out, what do computer scientists do in order to improve problem solving skills and knowledge?

Unlike for athletes or musicians, practice in a knowledge field (like CS), seems a little ill defined..

The most important advice that comes to mind is a bit proverbial, and from a board game so appropriately titled; "Go."

Lose your first 50 games of Go as FAST as possible! Don't worry about winning or losing or finding the "right" move, just put some stones down, get used to looking at the shapes that come up, and get a good feeling for how the rules work. Of course, a consequence of that attitude is that you will lose most of those games, but it doesn't matter. Once you have a bit of experience under your belt, then you are ready to begin

I recently got out of college I'm going through almost same situation. What I've learned is you are eventually going to forget many things after some time, and you can't know it all. Secondly, knowing theory is one thing, making use of what you know is another. You can grasp all theory you want, but the main point is how are you gonna put that into use? A good job where you get to solve interesting problems may help. I couldn't do so. What I'm doing right now is, pick up a subject area and build things from scratch. I feel that this is working for me.

Also, I see a pattern that more you are used to studying more quickly you tend to understand something new. I believe its something do with reading habits. So don't leave theory completely, but keep a balance b/w theory and practical.

Either build something, or go to the academy. If you build something, you'll do a lazy evaluation of the knowledge tree, learning just what you need to get the job done, which will be a very small subset of all that you described above, depending on the domain of your problem. However if you want knowledge for the sake of knowledge, get a master degree / PhD, it sounds to me it is exactly what you are looking for.

Also you don't sound line an expert beginner, you sound like a beginning expert, I'm programming for 10 years for a living, built a couple of money making startups, and doing my MSc at the moment, I can't say I know half of all that at an expert level ;)

Finally, accept that you don't really have to know everything, and more so, you can't really be an expert in everything, it's really hard but CS is a very wide field. You can't help but being a beginner expert on a wide area of topics, and only a "real" expert in a very narrow subject. Not all CS Phds are expert in everything, actually they are most likely expert in a very narrow set of topics relevant for their research.

Bottom line, either learn for learning and do it in a place that honors it (academy or independent / commercial research) Or build something that makes money, and the subset of human knowledge needed to make it work will be defining itself (you'll have to fight the urge to learn things that "you'll probably need later", and make it more "on demand" learning)

I tend to be pretty mercenary, but I think that writing software in exchange for money will be the most educational thing you can do right now.

To become a better programmer and not a better computer scientist:

Do your own thing. Build something you want to see built and you will learn oh so much. Programming isn't about how you implement something so much as it is for what reason. Think of something you want to see built and figure it out from there.

For example, I learned Python by trying to write an app that would take my Shazam tags and convert them to a Google Music playlist so I could more easily remember songs to listen to later.

Notice that I didn't write anything about SQLite or how slow Python's HTTP was when making the queries, because in learning Python, that wasn't important; those things were just implementation details that I only started thinking about after my application was demonstrably "slow." And more than that, I really didn't consider anything about lambda calculus and I don't know a single thing about NFAs or DFAs. I just wrote an app.

And I learned something, enough to get the job done. If you really want to learn about how to apply functional programming, learn Haskell. If you want to learn about compilers, write a compiler. You'll learn enough, because there's no way you can learn "everything" on a topic in Computer Science.

The field is much too broad, and you'll be way better off lazy-evaluating it than calculating it wholesale.

Tl;dr: If you want to be a better programmer (as opposed to being a better computer scientist) build and the knowledge will follow.

Pick a Free Software project that you use regularly (possibly one without a CLA to begin with if you want to avoid paperwork), preferably one written in a language you already know, find the bug tracker, and find a bug (or wishlist item) that sounds interesting but really trivial. (No, really. The first time you do this will probably be a lot harder than you think it ought to be. If it's not, no problem, pick something less trivial second time around.)

Download the code, figure out how to build/install it, and start to find your way around the codebase to try and figure out which bit of code is at fault/needs extending. When you get stuck, ask on the dev mailing list or IRC or whatever comms channels the core dev(s) have.

A lot of programming is not about designing something elegant and new. It's maintenance work, fixing bugs, extending functionality, adding new features. Sometimes adding exciting new features is a chance to design something elegant and new, but other times it's a bunch of repurposing and refactoring some features that are mostly-there under the hood, but need a couple of tweaks, and a small amount of really new stuff (but in the same idiom as the rest of the system) and exposing in a new way.

You'll really find out how to properly spelunk into a codebase (which is a complementary skill to just reading code), how code is used, and how it solves real-world problems.

If another dev solves the bug before you do, that's not a problem. The real purpose of the exercise was for you to learn, and only perhipherally to help the project. As a bonus, you can see how the other dev solved the problem, and how their solution differs from yours. Did they solve the bug at the "same level" as you? Was their fix a bigger or smaller change than yours?

If you get there first, great! Submit a patch to their mailing list, or a pull request to their git page, or whatever they use. Do not take their criticisms of your work as a personal insult. (If they do insult you personally - which almost certainly won't happen, but very occasionally does - that's another matter. Drop it and find another project. Life's too short to waste on asshats.) Rather, listen to exactly what they don't like about the way you solved the problem, use that to fix the problem in a way they will like, and re-submit.

Repeat.

Seems like you fell into the typical CS trap. The problem is that Computer Science is not software engineering, and the dev skills you need aren't going to magically fall out from between the pages of CS books. What you need is more dev oriented knowledge and experience/practice.

A couple books that are worthwhile: "Refactoring" and "Rapid Development". That will teach you a lot of basic skills in terms of development process and how to improve the design of real systems (warts and all). Also, take a look at the Architecture of Open Source Systems, it'll acquaint you with how applications fit together.

Also, take on some projects. Pick something interesting and work on it. Pick some small stuff then move up from there. I'd suggest in your case eventually building a compiler. If you're interested in AI, build some simple games and work on building AI for them. There is nothing more important than actually writing code.

I've learnt so much over the years and barely remember any of it. I wonder whats the point of studying all this if I am going to forget 90% of it in couple of months. How do people deal with this? Should I convert everything I learn into some sort of cliff notes for rapid reacquisition of the knowledge at a later time?

When looking to improve your programming ability, it's important to ask for what end you are improving your skills. If your goal is to become a professional computer scientist, then you'll need to choose an area of focus. The field of computing simply has too many facets for anyone to master them all. If your goal is to learn, simply for its own sake, then all that matters is that you are enjoying the learning. It won't matter what technologies you use, or if what you're making actually works, so long as you enjoy it. In either case, you should choose something to learn that interests you. There are many highly sought after computer scientists, who specialize in computer vision algorithms, for example, who get recruited by companies all the time, despite knowing little to no software engineering and data structures. TL;DR: Learn anything, it will get you places.

I think some of it might just be that CS theory sounds intimidating. But if you paid attention in college (which it sounds like you did), you might find that papers really aren't that hard to read. The hairy bit comes up when you don't know enough math, but this is typically more a problem if you're trying to learn something specialized (e.g cryptography).

Also, there's a disconnect between academia and the workforce. You might invent a fancy new data structure that blows up in real use because it ignores caches. Or you might invent something that's cache-aware, but everybody's using scripting languages or the JVM and doesn't have that level of access to the hardware. And so on. If you're strictly speaking about becoming better at the work of programming (rather than CS), that stuff isn't that important. You will spend way more time, especially as you move up in seniority, interacting with people - whether it's designing your system to be understandable by humans, persuading others of your ideas, getting funding, etc.

1 read abd understand other people's code

2 build, then rebuild, your own systems

After 10,000 hours of this, assuming you keep climbing rather than just doing the same thing over and over, you will be competent.

Get an internship at a company you think you will like to work at, something with 10-15 people I think is ideal. If you can't find one yet make some small projects help some friends or build something at a hacker space and try again.

I'm always suspicious of the advise to work on open-source projects, or practising programming on your own, or online coding academies, etc. I feel these are without purpose and people soon lose interest since there is no goal (although helping out on an open-source project is certainly noble).

My advise would be to look at your own network (friends, family, etc) and find those who are in business and ask them about their pain; and there is always some pain that a business has. Then figure out a solution to their problem and program that. This serves 3 purposes: 1) It has a definable goal and purpose (solving the pain) as it's a real-life project. 2) You will learn tons about yourself, programming, and the business. 3) It could lead to either employment or a program you could sell to others and start a business.

As always, make sure you write-up a contract which states that the IP is yours. Hope for the best; plan for the worst.

Trying to learn Machine Learning stuff I realized I didn't remember much Linear Algebra from my college days. Im also a beginner in Haskell. So I decided to crack open my Linear Algebra book and start implementing what I was reading in Haskell. The result of this was that my Linear Algebra knowledge that was all hazy now feels like its etched out in my brain like a digital circuit board :-) And my Haskell calf muscles are in much better shape. Im still a beginner in Haskell but I can now walk fast in it instead of just crawling like before. Hopefully in a year I'll be running in Haskell too.

There are few choices and/or some combination of the following:

(1) Start committing to Github daily. Make your own project and just go with it until you it's finished and learn what you need too.

My example project: http://austingwalters.com/openbkz/

(2) You can go through online courses in an attempt to learn more and become an expert, many courses are online. You could also start following blogs, read papers and replicate results.

If you are interested in maximizing learning: http://www.scotthyoung.com/blog/

(3) The past month I have been writing a blog a day, it increased my productivity and forced me to really master topics:

My explanation on blogging: http://austingwalters.com/learning-through-blogging/

I recommend blogging, videos, or writing stuff down for "teaching" others because it really helps you master a subject and helps you think of ways to use those concepts.

If you want to improve at programming: follow the advice about either making a personal project or working on an open source project. The fastest way to get better at programming is to program more.

However: it sounds like your interests are actually more in the realm of Computer Science than Programming, though: type theory, theory of computation, compilers, AI, etc. are less in the realm of programming and much more in the realm of computer science. You don't really need to know how an NFA or DFA works in most day to day programming, frankly. For learning Computer Science, I would suggest finding an advanced course with all of its material online (MIT OpenCourseware is a good place to start, or any of the other free online courses places), and working through a course on the topic. If it has prerequisites that you don't know any/most of the material from, then find a course on those. In my opinion, the other common options (reading through a textbook, reading papers, working on a project related and just learning what you need for that) all have flaws: textbooks are usually designed to accompany a course, which means they usually have much more than you need to learn a lot about the subject (and, in my experience, many are unbelievably boring and poorly written; if you want to go that route, make sure you find ones that are appropriate for self study). Reading papers is really interesting (and a lot more fun than reading textbooks), but without context or knowledge about the area, it's hard to evaluate the paper's meaningfulness/claims/etc., and hard to decide which papers are important to read. Working on just a project (e.x. just writing a compiler) leads to learning just enough to make that project, and not more. If you want to improve at programming: follow the advice about either making a personal project or working on an open source project. The fastest way to get better at programming is to program more.

Frankly: it sounds like you are decidedly not as much of an 'expert beginner' as you think you are. Familiarity with basic data structures+complexity, an understanding of theory, and an understanding of math through linear, ODEs, and discrete already puts you on a very firm grounding. (Outside of certain very specific parts of CS, you /do not/ need more math than that. If you're going into graphics research, knowing differential topology+geometry might be handy. I'm having a hard time thinking of things other than that, though.) From your description, you have more or less finished the 'beginner' stage, moved well beyond the 'novice' stage, and are moving towards becoming someone with a lot of advanced knowledge. Just knowing that you're interested in things like type theory+compilers/machine learning+expert systems+AI already puts you /way/ beyond being a beginner.

Shameless plug of someone else's stuff: if you're not sure where to start on the courses front, and want to start on the programming languages side of things (compilers, etc.), this course might be a good place to start: https://www.udacity.com/course/cs262 I haven't worked through this course in particular, and I don't know its exact difficulty, but I took undergraduate programming languages from this professor and he's an /amazing/ teacher.

Edit: don't know why I had two copies of that written, but now there should only be one. Also, cryptography is another part of CS that you need more math (in this case, abstract algebra). (More multiple copies? I seem to be pretty bad at this "say things once" thing, and need to make some preference changes.)