Ask HN: What did you learn the hard way?

241 points by donkoz_ab ↗ HN

321 comments

[ 2.9 ms ] story [ 168 ms ] thread
The only useful policy is the one that is followed
The people you spend the most time with affect how you think and what you do.
I originally posted this separately, but then saw your post and am adding it here. What you said and the below were some of the most profound realizations for me, even though it seems so obvious, but the extremely high degree to which this is true isn't that obvious.

---

People are incredibly hard to change.

Spent half a year teaching someone programming. They told me they wanted to learn, etc. Ended up doing most of the bootcamp work for them so I could explain it to them, etc. This person had 5 day breaks where they didn't study a single thing. This also goes with "look at what people do, not what they say, or how they present themselves."

That also comes down to "don't waste time on shitty people." I used to spend time with everyone because of naivety, then I realized just how precious that time is and a lot of people just waste it.

Absolutely agree. One phrasing I heard that changed my mind a lot was "It's hard work keeping your own life afloat. Be careful who you tie yourself to. They are as likely to pull you down as you are to lift them up".

In one interpretation, that's a scary thought encouraging selfishness. In my view, it's a pragmatic description of reality, encouraging judiciousness. I still find great joy in helping others, sometimes at some expense to myself, but I now feel a lot less bad about making value judgements before deciding if such a thing is worthwhile, rather than feeling obligated to help everyone that might want it.

I agree, but not in the way that this statement is usually intended. It often goes along with “you become the average of the five people you surround yourself with”.

Unfortunately, sometimes I find myself surrounded by people who have vastly differing life philosophies than my own (relatives, friends, spouse). In that case, the disagreement in a way pushes me harder to achieve what I want to out of sheer stubbornness than if I actually was surrounded by people who thought the same way as me.

I would add to this the following. You are the person you spend the most time around. You have power to affect your own outlook in small ways on a day to day basis. Added up over a long period of time can result in real change in outlook. I'm talking really tiny things like instead of saying "I'm bad at X" say "I haven't done X well before". Sort of brainwashing yourself into a growth mindset.
HR is there to protect the company, not you.
I've heard things like "if you have a problem and are unhappy, just go to HR". Nope, I'm not going to rant to them, and I'm certainly not going to talk to our in office mental health professional.
I learned this one as well. One week after going to HR with a concern I was fired.
My SO is a HR Manager. I hear the constant drama first hand over dinner. My take: HR are angels after hiring and during onboarding. After that, any correspondence will be kept to an absolute minimum.
Who benefits the most from the work your SO does... employees or the corporation? As others point out, they are to be avoided at all costs! And usually play a really bad gate-keeping role when it comes to IT hiring. It's a hard thing to learn early in your career.
Are you actually asking? If so, I think you misinterpreted my post. Please re-read, we agree. If HR has anything to do with any part of the technical interviewing process, the company is already misplacing their use. I won't touch HR with a ten foot pole unless absolutely necessary.
Sadly, after more than 20 years building software, _NEVER_ trust a business person. My faith in humanity had to have me bitten by this the hard way more than once.
You are also a business person, aren't you?
+1 every time I say exactly the same. I have an axiom about that.

1. Do not trust “business” guys! 2. If they give you some “sweet” offer, look at the rule No.1

Could you elaborate a bit more? I’m not sure what to take away from this.
Here's a more charitable way of thinking about this, based on my own experiences. When you're making a deal anyone (agreeing to do work, buy/sell, partner, etc.) there are two sets of motives you have to consider: what they tell you up front, and what they actually want. In your day to day, even at work, those two things are the same. But in many contexts where you would be talking to a "business person," those two things can be very different.

Lumping it all into "business person" isn't very useful because there are a lot of business people. So maybe a few specific examples can help.

Account managers and salespeople: while their stated job is to help you with customer issues, they're actually paid to sell/upsell you, and even if they're not, they're evaluated on selling/upselling you. This means they're likely to stretch the truth about the product they're selling in order to close the deal. So when you're interacting with salespeople, you can never 100% trust what they tell you, because they're not actually trying to help you.

Business development: the company they work for has an agenda. That agenda won't be obvious to you. Maybe they're asking you to come in to do a demo because they want to consider partnering with/acquiring your startup. Actually, they're spinning up a team to just build it themselves, but they want ideas. The problem is you have no way of knowing which is which.

HR: you know the common wisdom, they're there to protect the company, not you, and often that makes a huge difference.

Someone hiring you for contract work: they do want the work done, but they're trying their hardest to avoid paying. That means every ambiguity in the agreement becomes something they might try to leverage against you.

In other words, don't trust someone until you understand their hidden motive. "Business people" just tend to have more of them than your average person.

Yeah, I'm surprised that they trust each other at all. What keeps them from bashing each other with sticks?

When I see business people smile at each other and shaking hands, I cannot make sense of this in my mind. What's the point of smiling when both people couldn't care less if the other person dropped dead on the spot.

They care whether they can get some use out of the other before either drops dead
there are many business people who don't engage in that. at least on main Street, where relationships actually matter.
Also, watch out for business people disguised as tech people. They're the most dangerous of all.
all developers in New York simultaneously quit
I might give a softer version:

1) Never assume people will respect the value of your time and the fruits of your labor. Some people in particular are looking for related arbitrage opportunities. It isn't even necessarily malevolent, it can be just a frequent human limitation in how value is perceived. Though sometimes it is predatory to the point of malevolence.

2) Assume any future promised reward is unlikely to materialize when making your decisions (if it is not specified via contract, assume it will outright not happen). If a given course doesn't make 90% sense if a possible upside doesn't happen, then it doesn't make sense.

Either I'm your customer or you're my customer. All transactions with another business should be buy/sell.
This one is great; easier to understand than the often quoted one. I shall steal!
Make things pretty.

No amount of hard work and usefulness can amount to a beautiful product.

I have strong feelings about Lean Startup who told me to create a MVP, which made me look like an amateur.

So, listen to the marketing people and wait to release a beautiful finished product.

Amen. I've shown the exact same application on a dingy thinkpad and a shiny new retina iPad, and had diametrically opposite reactions.
... MVP, which made me look like an amateur.

If you're in a domain where aesthetics and "beauty" matter, then your MVP is missing the "V" unless you have, indeed, made the thing beautiful. The Lean Startup approach doesn't tell you not to make things beautiful... it just tells you to make only what you need to validate your hypothesis. If that means making something beautiful, then you make it beautiful.

The website has a wikipedia feel, free and useful information.

There are lots of fans currently, but at the time(and to this date), I get people questioning legitimacy due to a having a white shadow in my CSS.

"...which made me look like an amateur." This made me chuckle since I can only imagine the frustration.
As a user I beg of you not to sacrifice usefulness and features for Pretty.

Seen way too may times where stuff was rewritten to look better but usability suffered.

Case in point: Walmart recently redid their self checkouts. The new system looks a lot better and I like it if it did not take 3 times as long to pay. It even took forever compared to the old one to register cash being inserted. And there were several complete lockups.

Recently it seems better so I believe they have been working on the back end but for a while I would not even touch them

Waitmart takes even longer with these systems? I generally always go to cashiers for that reason. I feel like I'm becoming the minority who would rather deal with an actual person than an automated system most of the time, and I develop automated software.
I avoid the self-checkout because I don't appreciate the (perceived) atmosphere of suspicion that you're trying to steal something.
(comment deleted)
That mysqldump adds "DROP TABLE IF EXISTS" by default. So if you want to backup and restore several rows, you are in trouble.

Not really the hard way because DROP TABLE isn't instantaneous on big tables. So I managed to kill the query before the table was gone.

Get everything in writing. EVERYTHING.
And along those lines, enforcing what you have in writing costs a lot of money.
Ha, I’ve learned the hard way that: By the time you invoke a contract, the relationship is already in trouble.
This is usually true. There's two saving graces, though:

1) Just having the contract sometimes gives people pause about what boundaries they're willing to cross.

2) A contract can help you salvage things that are not the relationship once the relationship has gone south if you choose too.

Keep in mind that #1 works, even if what’s in the contract is unenforceable. Companies use this to their advantage a lot.
I would add though:

A written agreement is only as good as the people who sign it.

How not to run a contract development business during a recession. :)
Worse is better

Or, to put it a bit more optimistically - usable now is better than perfect later. I've found that, if I disappear behind a curtain and spend a long time trying to make something really well-polished and feature rich, that just gives the user a lot of time to build up their expectations, and also to get frustrated by the delay. By the time you ship, they'll be actively looking for ways to find fault. When I YAGNI my way into a 80% or 90% solution and turn it around quickly, though, more often than not, they'll initially just be impressed at how quickly I was able to help them. Requests for changes will come, but they're generally small, so it's usually relatively easy to turn those around quickly as well.

Thank you, I think there’s generally some perfectionism on the technical side and things that are easier to address later that don’t require delays
> Or, to put it a bit more optimistically - usable now is better than perfect later.

This is the same principle of Minimum Viable Product. In MVP you launch the first version of your product with sufficient features, but not all, to early adopters so that you can get feedback from them.

MVP and the "Build Measure Learn" feedback loop, gives you important tools to leverage your product.

Yes, absolutely.

The language around MVPs tends to be couched in terms that pretty directly implies designing a product for external customers, though, so it tends to be harder to internalize for people who are doing in-house work, and even people who are somewhat removed from the product design process, such as many junior developers. What I've seen in is that it's common for people who are eager to impress their clients or teammates to want to say "yes" to everything. So, even if they like the idea of building MVPs in principle, in practice, their MVPs end up being pretty maximal.

Which is why I think there's value in framing the principle in terms of human-human relationships, and not just in terms of stuff like finding product-market fit. It's a more personal, relatable way of thinking about it.

The magic insight for me was when I finally realized that I could give people less than what they asked for, and they'd actually be happier than if I tried to give them everything they wanted.

> The magic insight for me was when I finally realized that I could give people less than what they asked for, and they'd actually be happier than if I tried to give them everything they wanted.

I consider that a form of art/skill that we learn through multiple experiences (and errors too).

I work closely with developers and I observed that the majority try to implement always the perfect solution that normally comes with a high cost and time to develop. Clients normally prefer a solution that is quicker to implement but that responds to 80% of what he wants.

What does YAGNI stand for?
You ain't gonna need it. The idea is that developers tend to think, "what if they need $x too? I'll just add it." Or, worse, they over gerneralize their solution. "I needed to rank 3 cart items by price, so I developed a way to rate an unbounded iterable of any type."
There are (at least) 3 sides to exercise: Strength, Cardio, and Stretching. They are each at least as important as the others.

Go to the dentist. Even if you have great teeth.

Stretching is probably mostly hype. E.g., https://www.painscience.com/articles/stretching.php
Tell that to gymnasts and martial artists. I have seen the "science" of this over the years, but I have also seen the practical fact that it's absolutely unavoidable.
That article seems to aim at the hype that stretching will help with anything other than stretching. It's pretty obvious to anyone who ever tried that practice helps you stretch farther, but some people also stretch because they believe it will prevent injury or make their muscles more efficient or something like that.
I stretch because it feels good. I don’t expect a cure from cancer for it, though.
Gymnasts, martial artists, ballet dancers and some athlets (e.g. javelin throwers) stretch to increase their range of motion because the latter is beneficial for their activities. The effect is temporary (+) unless you start at a young age and ruin your joints (-> China). If you don't belong to those groups, stretching doesn't do much for you, as has been repeatedly pointed out in various publications. The other benefits attributed to stretching, e.g. a lower risk for injuries, are actually caused by the warming-up effect of the stretching movements and can be also achieved, in a much safer way!, by some simple exercises.

(Side note: When talking about the benefits of stretching, some people also mention Yoga, but that's a different story and 99.999% of Yoga teachers and practitioner don't have the slightest idea what they are doing.)

(+) That's the reason why you have the impression that you are stiff when you skip stretching for one or two days.

Try squatting deep without being flexible. Squatting --> strength --> better posture, metabolism, & bone density.
Isn't everything mostly hype? After you have a certain minimal level of strength and cardio capacity, how much do those actually affect your life?

From the article: "Although it can boost flexibility, the value of this is unclear, and no other measurable and significant benefit to stretching has ever been proven."

The thing I had to learn the hard way was the value of flexibility. You can pump out squats and go for a jog, but if your knees and ankles hurt when you go down stairs, you have a problem. If you can't sit on the ground with your back straight, you have a flexibility problem. A lot of lower back pain comes from flexibility issues.

What is the value of flexibility? Go visit a nursing home and see people who can barely move. Part of that is lack of strength, part is lack of cardio, and part is lack of flexibility.

----

Also, you should beware of lessons learned the hard way. A lot of times they are another way expressing a pain point, sometimes with a solution that worked - but that solution may not be optimal.

Being in a leadership position in a software org usually involves going to meetings all day and telling people bad news.
That all my conventional-sounding musical ideas are easily captured by basic music theory. Even though banging your head against the wall has been useful in developing the ear, avoiding music theory doesn't make any sense since you keep repeating those well-known patterns, and in the end you have to start developing that pattern against pattern type of thinking instead of composing note against note & note after note.

Edit: "in favor of" -> "instead of"

> in the end you have to start developing that pattern against pattern type of thinking in favor of composing note against note & note after note.

I think there is a typo, or this is ambiguous. Are you saying you should use "pattern against pattern type of thinking" (and what is that?) or that you shouldn't? You should use "note against note & note after note" or you shouldn't?

Sorry, meant to say "instead of" not "in favor of". I'll edit.

What I meant to say that if you haven't learned the proper classification system that we now have in general music theory (dominant & tonic chords, sus chords, and let's say basic modulation principles), you might walk a long time in the dark looking for "that feeling" or "that continuation".

Documentation is like sex. If it is good, it is good. If it is bad it is good
This is hilarious and 100 percent correct.
But if the client wants me to pretend I care enough to do it well, they gotta pay.
"Our code is self-documenting. If you need documentation then the code isn't written well enough"

-Assholes who don't know what it is like to work with mature code bases.

    "Our code is self-documenting. If you need documentation then the code isn't written well enough"

    -Assholes who don't know what it is like to work with mature code bases.
- People who don't know how to write clean code
Sorry, parent poster: Gotta agree w/ above comment here. With the minor warning that the unit tests are included in this. In a good environment, I should be able to go to the tests to understand not only WHAT the code is doing but WHY it's doing it. No matter how 'mature' the code base is.
In any mature code, there is going to be unintuitive code that only exists to work around an external issue beyond your control.

Recent example: when you authenticate to OpenSSH running on macOS 10.13 or 10.14, make sure to never attempt keyboard-interactive auth after password auth, or the server will stop responding.

Somewhere you need to write this fact down, so that the next person who comes across your auth code doesn't change the order of authentication methods.

You need to document things like that.

If you have a mature code base, you'll have dozens, or hundreds of these workarounds for all kinds of issues. If you look at the code, it'll seem inefficient. If you don't document why the code looks like it does, then someone will come along (possibly you) a few years in the future and "optimise" that line.

Sure, you could have a unit test that checks whether the workaround is in place. But then you STILL need to document why that unit test is there, or someone will come and consider it pointless and remove it.

or maybe they just didn't write the code? i mean, if it's mature it likely was a result of years of effort and many hands involved. some documentation is always better than none, lack of documentation is either laziness or dogma; neither of which is a valuable mindset when working on a team for a business.
I hope you are using someone else to judge how clean your code is. (Maybe you are, maybe it is). I have come across very little self documenting code outside of a textbook. Code is a very low level medium. In the meantime a comment can help save a lot of time.
Code is harder to read than write.
I agreed with this until my current job, when it was stated cheerfully by a guy whose code is so shit lousy that I want to do violence to him (an emotion I haven’t felt in probably ten years).

Management thinks he’s brilliant. Everybody else thinks he’s an insane asshole.

Just because you know people who can say this with sincerity and gravitas doesn’t mean there aren’t assholes.

There are always assholes.

I think the book _Pragmatic Programmer_ explains the concept of this asshole type of programmer and how to have the same results as him without being an asshole yourself

        "Our code is self-documenting. If you need documentation then the code isn't written well enough"

        -Assholes who don't know what it is like to work with mature code bases.

    - People who don't know how to write clean code
- People who think that not only they figured out what 'clean code' even means, but that everyone else will agree, thus single-handedly ending some 50+ years of heated industry-wide debate, numerous billion-dollar attempts by some of the most talented people in the world, and troves of 8-dimensional mental gymnastics about design patterns by way of a $21 ebook, a code review and some blog posts
Not to mention thinking that clean sensible code is always an option. There will always be something. It could be a kernel bug, it could be a quirk of an external library, it could be just to get the bloody thing to link against static libraries, but there will always eventually be an issue that requires a workaround that doesn't make sense without context.
And it extends far beyond comments in source code. Don't be that person who people have to run to because he/she keeps everything in their head. It's not job security. Take some time and write things down for your future self and team members.
Except when the documentation is out dated and has very little to do with the actual code. At that point you are better off without bad documentation than with it. I've wasted a lot of time solving strange bugs because the API had stayed the same as when the documentation was written, but the implications were totally different.
Documentation priority is inverse of volatility:

1) Why (requirements, decision points)

2) What (use cases, interactions, external APIs)

3) How (internal APIs)

4) Who (SMEs, owners, servers)

And don't waste people's time; don't document things that are obvious.

I've heard the variant:

If it is good, it's great. If it is bad, it is better than nothing.

You can never get back the time you waste. I'm talking on the years scale.

The fact that it's never too late to make positive change in your life doesn't mitigate that.

Time is the ultimate luxury.
(comment deleted)
Always make backups. The little bit of time it takes will be a life saver if anything transpires unexpected. Oh and don't deploy on Fridays.
On this same note, test your backups occasionally.

Untested backups are worse than no backups.

Disaster recovery is hard. For startups, I finally learned to do this drill: Restore the entire business to fully operational status on clean computers from offsite backups in under 24 hours. You always hope you won't need it, but...
You're spot on! Specifically I was referring to SQL data. Been bitten more times than I'd like to admit making a hasty change.

Edit: refined.

Added corollary, Always test restore your backup, it's the best example of schrodinger's cat.
I never liked the "be sure to make backups" and have "and be sure to test" as an afterthought (or "added corollary" in this case :-). Most ppl ignore the afterthought.

I taught my students "You don't need a backup system. You need a _restore_ system." It changes the mindset from "Yes, we make backups every night" to "Yes, we can restore the database in x hours".

I've seen way too many companies take the "we're good; we make backups" only to fail when trying to restore.

Quantity has a quality all of its own.
How to successfully roll out a product/site.

Released "Version 2" which was an entire newly built system that we worked on for an entire year (we also thought we were "agile" btw). This completely crashed and around 10K users were unable to use the product for almost 2 weeks. We did plenty of things right but we also failed.

Biggest Lessons learned: 1. Release features and changes in manageable incremental chunks 2. Load test, load test, load test - just do it! Yeah, sure, your app worked with your one user when there is 1kb of data in your datastore - what about 5000 users accessing it with 10GB in your data store? 3. Get and use an APM - when it feels like everything is on fire you'll need to know where the fire is coming from 4. One way changes that can't be rolled back are the devil. Take the time needed to make sure you can roll back - manageable incremental changes help with that.

My project management muscles arr being trained to be risk averse, and my 'move fast and ship ' muscles are kicking in to make releases multipart and piece meal.

It werks

What is an APM in this context? Google just tells me about Alternative Payment Methods.
Application Performance Management/Monitoring.

New Relic, AppDynamics, and DynaTrace are a few vendors in this area.

Start a company with a friend will bite you back one day (for me 20 years after the company starts !)
A 20 years partner is a great partner.
25 years after, definitely. 20 years after it might be brutal and personal.
20 Years? That sounds good from a business stand point?

However, I guess how much of your friendship suffer during how much of that?

The story is a bit more complicated than that but long story short it came out of nowhere and left me emotionally crushed for more than a year.

Now I struggle to get the company back on track (as he « sold » me his shares without leaving me another option), and it’s a bit complicated but on the way.

Math.

Multivariable calculus and differential equations weren’t a part of my undergraduate CS curriculum. Now that I’m doing an MEng in AI, I have to learn the hard math I skirted by before.

I don’t have the time or money to take the proper math classes so I’m learning everything myself. I have to figure out my own structure and timeline.

Without the math, I wouldn’t be much more than a technician when it comes to AI. It’s worth the work.

Hell hath no fury like a woman scorned (with legal representation).
"You are not that special." Can be a shock to the ego when you leave a top school, but if you take it the right way, it takes some pressure off.
Don’t fish without a license
I'm really curious about this story.
I don't know johnwheeler's story, but I got talking to an Oregon game warden a couple weeks back. His story was that the state is handing out jail time and big fines in certain circumstances. What he described was work parties (large families, for example) coming from as far away as Seattle to pick up clams on the Columbia river. This particular type of clam is in vast numbers, but it is an invasive species. So you would think the state would be thankful to have them gone, right? What he described though is that the state is concerned about the spread to other bodies of water - so they want them left right there, rather than transported and potentially dumped somewhere else to spread. Given that these groups are collecting them for restaurant sales, I suspect that is an unlikely scenario, but there you have it. He said the biggest ticket he'd written was on the order of tens-of-thousands of dollars. So apparently this can be a hard lesson to learn.
Always roll up your window when driving past the archery range.
I really need to know the story behind this.