Ask HN: Feeling grossly incompetent working in opensource. What do I do?
While in college I worked three times as hard as any other student and I felt more prepared than any other student in the school. I built toy languages, and compilers while others were working on understanding basic data types. Now I am here working on an OpenSource project and I have come to realize that the multiple layers of abstraction that exists has made it impossible to change even one line of code without it effecting the entire codebase. I have spent the last three weeks attempting to understand all the moving parts, and feel just as lost as when I was brought into the project. No matter how much documentation and how much code I read I come no closer to making any changes in the codebase whatsoever. I still have two more years of college left, and I am starting to wonder if I have what it takes to become a programmer. Any suggestions would be helpful. thanks.
26 comments
[ 0.22 ms ] story [ 62.7 ms ] threadEither you have misunderstood the meaning of "abstraction", or the code base has what people call "leaky" abstractions.
The point of abstraction is to create stable internal APIs, irrespective of implementation details. The entire point of abstraction is to create the separation that your code base seems to be lacking.
> No matter how much documentation and how much code I read I come no closer to making any changes in the codebase whatsoever
This approach is necessary, but you won't really understand the code until you start debugging. Can you tell us more about the stack (and, ideally, the project itself) so that we can give you better tips? Sometimes understanding code is a lot easier with the right tools (IDE, debugger, etc.)
> I am starting to wonder if I have what it takes to become a programmer
I'm sure you can do it, but it's also important to figure out if you want to. If you work on multiple code bases, including your own, and you don't enjoy it, or the frustration you're feeling now is not healthy for you, then this might not be the right career for you.
A lot of being a programmer is banging your head against a wall for many hours longer than a typical person would be willing to, until you finally get past an obstacle. Sometimes the obstacle is bad documentation, often it's bad configuration, other times it's a bug.
If the fun parts are not worth the not-fun parts for you, then that sounds like an unpleasant career path.
I think this appears to be the problem. The codebase appears to be very kludgey.
> This approach is necessary, but you won't really understand the code until you start debugging. Can you tell us more about the stack (and, ideally, the project itself) so that we can give you better tips? Sometimes understanding code is a lot easier with the right tools (IDE, debugger, etc.)
I think the architecture is essentially one of the reasons I am having so many problems. I'm not familiar with docker/apache solr/their custom database, etc. However, I think that's more of a tangential issue. It's more of a feeling of imposter syndrome as another poster said. For everyone else in the field it seems like this is effortless.
> I'm sure you can do it, but it's also important to figure out if you want to. If you work on multiple code bases, including your own, and you don't enjoy it, or the frustration you're feeling now is not healthy for you, then this might not be the right career for you.
It's not really frustration. So much more as feeling like I will never be good enough at this to be successful in this career.
> A lot of being a programmer is banging your head against a wall for many hours longer than a typical person would be willing to, until you finally get past an obstacle.
This is exactly my personality type, and my wife makes fun of me for you. Once I get my teeth into something I basically don't stop until I got it. It's actually created problems for me in other career fields, but it's seen as an asset for programmers. So I know on a personality level I have what it takes, but yeah... Just get the impression I'm not smart enough for this.
Deciphering large codebases takes a LONG time. Months often, to get up to speed. One thing that has always helped me is to take notes. Especially if you start getting lost in a deep set of function calls. Write all that down. Form hypotheses on what parts of the code are doing and try to come up with ways to test those hypotheses.
Soon it will start to become less murky. It just takes a lot of time and perseverence, even for veterans.
It gets easier the more you experience. I’ve been in the game for around 20 years. I still get caught on sharp edges in code bases. Things still take longer than I originally estimate. Recently, I lost hours on a dumb docker thing because I don’t understand it as much as I would like. One of the code bases I work in from time to time still takes me sometimes days to figure out something that “should be easy,” and I’ve been in that code off and on for literally years. I think you are doing just fine. There are people in this industry who can’t code themselves out of a wet paper bag. You sound like you are already beyond what some professional programmers can do.
Seek help from other contributors to the project. It could just be a shitshow of code that takes hand holding to understand.
1. At first, they always look like overcomplicated incomprehensible crap. Every little thing that you would have done in way X but was done in way Y initially looks like that.
2. But much later on, after you struggled a lot with making changes to the code base, things start to click. You understand why things were made as they were, what the constraints and requirements were. When you want to make changes, you know where they are supposed to go.
3. Then after much more time, it will look like overcomplicated crap again as you understand that people should have said "no" to the more bizarre of the requests, and that much easier solutions for some of the other things exist.
The original programmers started out somewhere between two and three, and as long as they stay involved, they can always guide people, tell them where their changes go or whether they are ill advised. They know which parts are essential to the architecture and which were never meant as more than temporary crap.
You get problems when none of the original programmers are still involved, and you only get people who have to climb the ladder by themselves. Or only have some people in step 1 left. Then the project is dead.
Also, you said that you have two more years of college yet to go. You are placing a completely unrealistic expectation on yourself. You are very unlikely to be able to, after your sophomore year of college, jump into a big code base and understand it in three weeks, even if you're better than others in your class.
As for impostor syndrome, I'm stealing this from someone, but how about Meta Imposter Syndrome: "All those other people have imposter syndrome, and yet I see how much they're able to do. So they're not like me, the real imposter."
But, in your situation, all those people who are able to get things done in that code base are not like you. I don't know what the code base is, but I strongly suspect that they are out of school, have several years working professionally, and have several years in that code base.
Your problem isn't that you're not smart enough. Your problem is that your expectations for yourself are very unrealistic.
tldr; don't give up.
Keep expanding your knowledge.
Read the new pragmatic programmer.
It took me about 5 years to get over my imposter syndrome.
Dont get hung up over one project. I have written a lot of bad code and a lot of clever code. Both in the same project.
Being a good dev is a lot like being a good doctor. Study and practice, sometimes you will get it right, sometimes you won't.
You are lucky enough to be experiencing what it’s like to be part of a real software project. It is not at all unusual to be in this position for weeks or even months depending on the complexity of the code base.
If you somehow manage to continue until graduation, you will have invaluable experience that virtually no other job candidates at your level will have. It will be a massive advantage over your competitors, and when you are asked about your experience you’ll have specific and credible stories to draw on when you talk about how you solved problems.
At the very least, congratulations on sticking to this for as long as you have. This stuff is hard. That’s why they pay us a lot of money for it.
I think it's like becoming an architect spending your career building houses. Houses are small enough that you can reasonably expect to understand every detail of one. And when all you work on is houses, you begin to expect that you'll soon understand the totality of each product you work on.
But then you get a job working on skyscrapers. It's impossible for any single person to understand the entire structure, and you don't appreciate that fact because the concept is still foreign to you. I think a lot of companies have "skyscraper" code bases, and it's not always obvious when you're looking at one. A good hint is probably, any software that's been around for decades and enjoys widespread use is likely to be a skyscraper.
Embrace it. School is designed to give you set pieces that can be understood in isolation, the outside world rairly has that luxury. You won't truly improve without discomfort, though the particulars will be personal and multidimensional.
The end skill is solving problems in the wider world.
Working on someone else's codebase—especially if it's an open source project with many different contributors over a long period of time—is its own distinct thing that has to be learned. Everyone has to learn how to do it, regardless of where their abilities as a programmer are. If you are adept (as it sounds like you are) at building your own software, then it can feel like the rug has been pulled out from under you ("Am I actually any good at this?") when you try to contribute and simply can't grok what's going on, but if you persist, you will pick it up. Don't take your struggles now as a reflection of your abilities.
This is a virtue and a fault. After some successes and some failures, you are likely to learn to be more wise when choosing which problem to deep dive into. The wrong deep dive will damage you.
It also takes time to learn that cowboy duct-tape solutions are sometimes correct: this is best learnt by being in a team with someone who is productive in that manner (the downsides of it are obvious to engineer types; learning the positives of hacks is harder for engineers; judging when to apply fast solutions needs a good engineer). If you can only do anal retentive engineering, then your skills are still super valuable, but perhaps try to veer towards projects where beauty and correctness are valued.
For many engineers, I think it is worthwhile to learn the dark side: how to deliver a quick and dirty compromise that meets a need e.g. most web development. I personally find this difficult: google “yak shaving”. Some engineers never learn how to make good compromises, and they sometimes end up stuck in a dead end.
“but yeah... Just get the impression I'm not smart enough for this.“
Keep your humility, use it to connect with clients and users. They feel it 10x.
Finally I’ll add that the best developers I’ve ever worked with have been technically mediocre. Their other skills have made them great - abilities to concentrate on the right things and not worry about being technical goddesses or gods. I have worked with technically great people, that just do a shit job because they can’t get on with others or they produce something that only they can understand.
Do `git show COMMIT_HASH` for the first ones and you'll see the genesis.
You're looking at the result of thousands of commits, each one having required countless hours of thinking and coding. This can be overwhelming.
Here[0] are some tips that help getting to know a codebase. Some may be irrelevant.
[0]: https://news.ycombinator.com/item?id=19924100
A concrete suggestion that came to mind is isolate the simplest unit of functionality the project implements and try to implement it yourself fromsscratch using the existing code as a reference; stripping out all the error handling, edge cases etc.
The feasibility of this depends on the project but it can be a useful way to understand both the basics and the rest of the code that seems like spaghetti which is probably due to error handling, edge cases, etc. As an example I implemented part (a simple passwordless connection and select statement) of a database connection library from scratch in c# and it really helped me understand the existing solution better.