The answer to why this is, I’m beginning to realize after talking to dozens of Googlers working at Google as acquaintances, friends, former coworkers, HN comments and twitter personalities is - Google has an active disrespect if you’re not part of their in group. If you haven’t passed their tests, you’re basically an object of derision - a mark, an object just used for incrementing a CPM metric, or an incompetent, low-class buffoon who doesn’t deserve to have their jet setting luxury lifestyles. And unfortunately, everything trickles down from that.
Yep, Googles stuff looks nice, but I can’t afford to use their stuff. The time budget simply doesn’t allow for rewriting things more than once a decade, and even that’s a pain.
A "couple dicks"? It's endemic, dude. You can't be given everything you could ever dream of and _not_ end up like this.
By comparison, I have nothing. zilch, for my hard work. Neither do most people working on core tech at AWS. That's why they empathize with their customers (I'm guessing).
It is not endemic. I think you've met a few people you didn't like, read some tweets, and extrapolated poorly.
And I don't think people working on GCP are really treated that much better than their counterparts at AWS. The fact that you think the difference in product depreciation policy between them is caused by people getting "everything they could dream of" is funny though
Trips to Europe or to random conferences in first class? Trips to _any office_ for "face time"? Massages? Bonuses of any sort, including peer bonuses and holiday gifts? Refreshers? Society thinking that you're "smart" for passing the loop? You're rewarded just for being alive at Google, but I get nothing but scorn and condescension.
Fun fact for you - if I get promoted this year at Amazon to L5, I'll _still_ make less than a new grad at Google. It's extremely depressing to the extent that I can't get out of bed in the morning, but I'd wager that inferiority complex probably helps on the product side to empathize with customers.
Levels.fyi is good, but it's still polluted with TC's from multiple regions in the US (California + NYC have higher base and higher TC by 15% due to taxes). In addition, promos always happen at the low-band and aren't negotiated. There's very likely some intersection between high-band L4 and low-band L5, and I'd be there when/if I promo. The only way to get out of the band is to get a high raise during evaluation season, but this maxes out at 10% or so for "top tier" performance evaluated only once a year.
*The net effect is "pay for performance" isn't really a thing.
The only way for me to do a significant compensation bump is to do what's called a "dive and save", but this is very rarely done for L5's and is significantly more common for L6's. I got an offer for about $200k a year ago at a hedge fund, but that wasn't eligible for dive and save because of my level and I'm (clearly) too mentally defective to pass a Facebook or Google loop.
The other more popular approach is to "play the game" as I think of it, I.e. every 18-24 months you switch to a new job with a substantial increase in pay as result.
I wish this wasn't the case, but I guess it is what it is.
The saying goes that if you want a raise, you have to negotiate for it at your new job. Internal promotions typically make less than an outsider coming in for the same position. Some corps say that pay bumps have a max, so internal employees hit that max when getting promoted even though that max is still below what a new hire would be brought in at. I've run into that personally. I fixed it by getting a new job for a new company. When you negotiate that new job's salary, you have to make sure you're going to be okay with it for the next few years while the cycle starts over. And people wonder why job stints are so short today.
I've studied hundreds of hours, failed HC for full time once and I've concluded that I'm mentally incapable of doing so (I blame my parents due to the slight heritability of IQ).
"Just move" is something someone without any empathy for other people's circumstances would say - most people aren't capable of "just moving" as if the only concern is choosing an employer.
Man you really shouldn't be so hard on yourself. These coding interviews aren't IQ tests. If it's any help, a good recommendation from someone in a company would probably be a stronger signal than a leetcode type question.
My feeling is that getting a job is like dating, a mysterious mix of luck skill chemistry etc, there are some rules but you can follow the rules precisely and still end up losing just because of some odd misfortune. However if you try long enough and put in a solid good faith effort, long enough being unspecified and possibly after you die or go broke, you generally end up with something decent.
That long enough part is the kicker and that terrifies me in every job search (to the extent I have sometimes jumped at the first half decent opportunity, and agreed to salaries below my potential). I find it does help to apply often and everywhere and not be invested in any particular opportunity.
With dating they say every failed relationship is a step toward finding the right one, and I think that same principle applies to job interviews.
I worked at Facebook for years and did a lot of interviews and the hiring process has inconsistent results for all but the best and worst candidates. The amount of times a candidate was rejected because one iterviewer who had questionable rationale was surprising.
Stop comparing yourself to others and work on yourself. Look inward, not outward.
You make six figures at a top tech company and you’re bitterly complaining because you think you’re entitled to more, and in the same breath you’re chastising Google’s pretentious culture? There’s a lot of irony there.
Focus on improving yourself and don’t get caught up in comparing yourself to others.
I see too many people talking about their Tahoe Google offsites (jbd@ on twitter claims a bunch of her coworkers just work from Tahoe during ski season) or their Colorado skiing weekends.
Anyways, hypocrisy doesn't mean I'm wrong, and I think its valid to be mad that I'm seen as inferior daily.
> Fun fact for you - if I get promoted this year at Amazon to L5, I'll _still_ make less than a new grad at Google.
Yet you still make more than what the vast majority of engineers in western Europe will ever make in their whole career. And that's still only looking at the most developed countries.
The job market isn't fair. No use getting burnt over that.
That doesn’t negate my point that you still make more than most developers in the US outside of the HN SV bubble writing CRUD apps or bespoke internal apps that will never see the light of day outside of the company. Go to salary.com and pick any major city not on the west coast.
Give me a break. Lots of teams in Google have restrictive travel policies and never see the pointy end of a plane. Massages cost money. Holiday gifts ceased to exist years ago.
I'm also quite surprised by your wage comment: a common aphorism is that Amazon cheaps out on everything except real estate and compensation.
> Lots of teams in Google have restrictive travel policies and never see the pointy end of a plane.
Which ones do and don't? Because all of the ones I've heard of have pretty loose ones compared to my organization.
And yes, Amazon doesn't cheap out on real estate because we own so much of it. It's honestly remarkable how it's done. But I was under the impression everyone knows they pay less.
Do you not realize how much AMZN appreciated over the last 5 or 10 years compared to FB or GOOG? Many Amazon engineers came out way ahead financially compared to their FANG peers.
FB and GOOG provide more stock and more refreshers. I think its hard to say AMZN engineers have come out ahead consistently.
I only have about ~40 stocks up from about 35 before the refresh cycle (unvested) and that's unlikely to go up if I get promoted (if you have too many they don't give you more). Googler and FB refresher values are significantly higher.
It's totally endemic. If you work at Google, and can't recognize the smug, superior, yet incompetent culture, odds are you've be incultured.
It's not unique to Google. Lots of organizations intentionally build a culture of elitism. However, it means I would never, ever rely on Google for anything. Free products like search are a-okay. My email is on gmail because legacy.
Are you really complaining that you only make probably around a quarter million a year?
I don’t even work on the “core tech”, I am on the consulting side, probably make less than you do (albeit in a much lower cost of living area) and I am probably older. But, I am not throwing a pity party on HN
Something doesn’t add up. When you say you’re working on the “core tech” are you a developer located in the US working at AWS? Is that your total comp or just your base are you not including RSUs/signing bonus?
I’m not even here to shame $150K. That’s what I was making as CRUD developer pre-Covid at 45 years old (long story) and I also graduated from a no name state school - in the 90s. $150k still means you’re making more than roughly around 80-85% of American households.
It's total compensation including everything. My base is only around $109k or so. I'm not in California or NYC and I joined out of college so base isn't as good as industry hires.
I fully understand shaming $150k, new grads at Google make $180-$210k.
You graduated from college and are now making more than most developers in the US and you work at a big tech company. You have the opportunity to leverage that and either get promoted from within or change jobs.
Are you really struggling making $150K straight out of college? You graduated from a state school (as did I) you probably don’t have that much debt.
For context, I’ve been out of college almost a quarter century and I am just now making a little more than a college grad at Google. I’m not complaining and I only have a good 10-15 years to take advantage of the opportunity. You have your whole life ahead of you.
Stop complaining, get on the grind, and do what you need to do. No one owes you a quarter million just because you walked across the stage with a CS degree.
>You have the opportunity to leverage that and either get promoted from within or change jobs.
I think the real problem is I'm not capable of doing so, at least according to the fair labor market.
> Are you really struggling making $150K straight out of college? You graduated from a state school (as did I) you probably don’t have that much debt.
I don't have any debt, I just have a really pathetic savings rate and a really low net worth compared to nearly everyone I know, including folks that didn't do "the grind" that I did. I'll probably not be able to retire or own a car, much less own a house someday.
I’m really not being facetious, and this is going to come across that way in writing. You really need to talk to a mental health professional.
I hate that we as a society can say that if you have signs of cancer go see an oncologist and it’s not okay to say that if you show signs of mental health issues go see a professional.
But everything about your comments and your user name points to it.
I spend a lot of time in the Ukraine and talk with people. A friend of mine earns $140/mo ($1680/yr) as a skilled machinist. He has probably as much accumulated skill as me and sweats at his work. I sit around and type out some JS and drink coffee, and make an order of magnitude more money than him. I drove a $500 car (on vegetable oil - cheaper). Live in one bedroom appartment with a Hungarian family. Outside the window are people shooting themselves up. I worked for a couple of years out of a McDonalds with earplugs so the music didn't distract me. I maintain a complex software stack that services nearly a million users with a software they love. I don't want to be doing anything else.
I don't like the IQ/talent fetish that seems to pervade Anglophone culture. Some people have a stronger natural predisposition to sit and stare at a screen and figure out puzzles, and that happens to be sought after on the labor market. Doesn't mean others have less inherent value as human beings.
Would be nice to have a small appartment one day. I just try to offer some perspective.
>> I don't have any debt, I just have a really pathetic savings rate and a really low net worth compared to nearly everyone I know, including folks that didn't do "the grind" that I did. I'll probably not be able to retire or own a car, much less own a house someday.
Dude, please seek mental help.
You have so many working years ahead of you. Stop worrying about owning a house. Start budgeting using something like YNAB and get your savings rate under control. Also get all the basic investment vehicles going: 401k, after tax Roth, backdoor Roth etc. You’ll do just fine.
A lot of my friends are already putting money down on houses and are making hundreds of thousands in options trading on principal that I _don't have_. I'm falling behind permanently and even maxing out my 401k (which I didn't do last year because I don't anticipate living to 65 but decided to this year) won't get me closer with a low principal.
I don’t know about other locations, but in Seattle the googlers and SpaceX people walk around with a lot of swagger. And you don’t have to talk to them long until workplaces get brought up and a small jab at Amazon or Microsoft is made. Which idk maybe is fair if they used to work at Amazon/Microsoft. But still I wish we weren’t so intent on maintaining a hierarchy. Maybe we just all need to develop more of a life outside of work.
Including the most critical group of all - customers. It’s astonishing how little Google cares about its paying customers, at least when it comes to AdWords and GSuite, and GCP.
I think the Pixel division is the only exception, though I don’t own one so I can’t comment personally.
After 3 years of updates, Google drops Pixel support like a hot potato.
As far as I'm informed every Pixel version so far had battery degradation issues. After year or so the battery seems to have only a fraction of the previous capacity.
It inherits primarily from the idiotic interview loop, and an entire cottage industry which has grown up around it.
I am reminded of this funny Feynman quote (paraphrasing): "The whole purpose of the existence of the Mensa club was to decide who else was eligible to join the club". Now, in all fairness, this is true of all the big tech companies to varying degrees, but boy does Google Cloud in particular take the cake on this one!
Of course, Big Tech does produce very good products, and at scale, but that is being built by about 10% of their engineers. The rest of them work there just to keep the interview loop as exclusive as possible. :-) After all, otherwise, one of them would have actually solved long runningcustomer problems.
Plus the rest of the world is now coming to an agreement that a lot of Big Tech's growth is coming from a lot of unfair advantages - such as massive data collection before it was understood as a bad thing, ability to lobby local politicians at a much bigger scale, ability to influence regulations that hurt them somewhat but entirely kill the competition (GDPR for example), complete lack of accountability for past malpractices (e.g. Facebook's friendly fraud case, for which no one went to jail) because they are now too powerful to actually be sent to jail.
I feel like this was super true in the pre Facebook days but they were humbled A LOT by the massive flop of Google plus.
I honestly don't feel like I get the my s * doesn't stink vibe from them anymore.
Especially because everyone knows their Achilles heel now.
But for their ad revenue what would they be?
Everything else material from came from an acquisition or is infra stuff.
It's cool but they also had to learn the hard lesson of Hadoop too.
Yeah they built big table... But the rest of the industry standardized on Hadoop (at the time) because they never released an implementation and so they started losing out on good hires because they didn't want to get locked into proprietary Google infra.
I think you see them recognizing this with intiatives like all of the stuff around kubernetes.
But hey I wouldn't build anything on their (non open) stuff and get more scared by the day over my dependence on El Goog because they've shown time an again how little they care for us common plebs.
But I feel like they get better when they lose but they've only started losing more recently
> Yeah they built big table... But the rest of the industry standardized on Hadoop (at the time) because they never released an implementation and so they started losing out on good hires because they didn't want to get locked into proprietary Google infra.
Hadoop and Bigtable are not the same thing. Bigtable is a noSQL database. Hadoop is a big data processing framework. Hadoop is actually an open source implementation of MapReduce, which was developed at Google and which has since been replaced by Flume
Hadoop is a collection of things: HDFS (filesystem), Hbase (keyvalue database), MapReduce (data processing), and YARN (resource scheduling and job management).
Hbase was open-source implementation of Bigtable while HDFS was a copy of Google File System. The divergence appeared with other processing frameworks like Spark but now there's Apache Beam to unify it all.
I have worked at a few orgs and I've heard so many time the "because we are Google" to explain why a product will be used. I've never understood that mentality; I've never understood people being proud basically about a product somebody else built 20 years ago.
I do believe leadership is failing Google, and their lack of vision is starting to affect the company. There was quite a bit of inertia, but I fear search might drop in the next 5 years and there will be nothing ready to replace ads.
On the other hand, when someone expresses an opinion like this, you never know if they were contacted at some point by a Google recruiter and ended up with a case of sour grapes.
I had a phone conversation with a hiring manager once, and obviously I was not the right type or class or caste or something, but it wasn't a technical screen at all, so it's a mystery to me forever what exactly determined the "in group" as you put it.
As such, I know I'm biased towards them, but I can't really assess the overall company.
Of course. I'm well aware that I'm considered mentally defective (or maybe just garden variety low IQ) by the hiring committee. That doesn't really change my observation here.
You pop up in every post about Google or Facebook and make the conversation about yourself, wallowing in self pity because you didn't land a job there straight out of college. This might sound harsh, but you need to get over it and move on.
^^ what this guy said. You named yourself lowiqengineer because your SAT score was “only” 2200. That’s in the 98th percentile —- if it were an IQ test, it would get you into Mensa, the organization for high IQ people!
You talk as if people from FANG (your invented acronym to exclude Amazon) look down on all others, but it seems that you are the one looking down on all others, everyone who wasn’t lucky enough to get into G/FB straight out of college while also building up not just 1, but 2 or 300k or net worth.
These companies you aspire to do behavioral interviews, and I’m guessing you’re failing them.
Take a step back. You won’t “fail” life because you “only” got a job at one FAANG and not another, or because your net worth is only 100k straight out of college....even the sentence I just wrote sounds completely ridiculous.
Your problems are stemming from how you view yourself and the world, which is full of false assumptions. A lot of smart people on HN have told you this —- you should listen to them.
> These companies you aspire to do behavioral interviews, and I’m guessing you’re failing them.
Don't know why people say this. I'm usually pretty good at behavioral interviews. The problem is the algorithmic interviews (again, this is where my IQ fixation comes from).
IDK man, compared to most everyone else I seem to have failed life at this point given that I have zero accomplishments and little financial security.
Serious question: How do you know you're good at behavioral interviews? Have you ever seen the feedback written on a behavioral interview? The companies you're describing don't usually give out feedback to applicants, so you can't know, right?
It sounds to me like you have anxiety. I'm saying this because I have anxiety, so I know what it's like. One hallmark of anxiety disorders is that you think things are true about the world, even though you don't actually have evidence this is the case. Or, you fixate on some pieces of information while ignoring evidence to the contrary, which again is irrational thinking. Or you hold assumptions about how the world works that aren't proven. You should look into cognitive behavioral therapy as a potential solution.
You say "compared to everyone else". But, you have to acknowledge that the net worth and income numbers you named put you into the 99th percentile, right? So, you actually mean "compared to a very small subset of people I have failed", right? And this is just a completely irrational argument. I mean according to this vein of thinking, then since I don't have as much money as Jeff Bezos, I've failed. And the implication then is the whole world has failed. Right?
I've always passed interviews that focused on behavioral aspects, I've frequently struggled in algorithm focused loops (bombed 2/4 in my Google onsite because, again, I'm intellectually inferior). Besides, until recently G didn't include any behavioral loops.
> "compared to a very small subset of people I have failed"
Compared to nearly everyone at elite universities I have absolutely failed. I'm sure some of them will become the Jeff Bezos of 2050 as well - it's irrelevant that it's still a small number. I don't want to be compared against somebody that works at Cisco or IBM for instance for the work and mental anguish I've put in.
Even here you have many false assumptions though. For instance, the Harvard median new grad salary is 69k [1]. So you’re wrong in your assertion that compared to nearly everyone at elite universities you have failed. By your own definition of success, compared to most (by definition of median) people at elite universities, you have succeeded, and it’s they who have failed.
You also have an implicit assumption that people who work at Cisco or IBM haven’t put in work or mental anguish, and you’re wrong. I know many people who worked and studied hard to get jobs at these places.
You’re artificially restricting the set of successful outcomes so that you can say you’ve failed because you don’t fall into that arbitrary set. Not to mention that your entire definition of success as defined by things like money and perks is wrong.
The perspective is inverted. From the perspective of a Harvard/Stanford/CMU/MIT undergrad working at Amazon is failing, which is nothing to say of IBM or Cisco (this is what I've understood after talking to several). I'd wager that the median new grad TC for Harvard CS students is at least twice that.
> compared to most everyone else I seem to have failed life
Please realize how offensive and insulting this is.
You were hired by one of the most successful tech companies, which most people could never aspire to. You claim that Amazon pays you 80% of what (you assume) Facebook or Google would pay you, which puts you financially ahead of the vast majority of people. By calling yourself a failure, you’re calling nearly everyone on Earth an even bigger failure, which is offensive and makes you look extremely entitled and detached from reality.
> Don’t know why people say this.
It’s because you come off in your posts as entitled, obsessive, elitist, insensitive, and bitter. These are the only sideS of yourself that you convey here, so that’s why people assume you’re failing your behavioral interviews.
Sorry again if I’m being harsh, but you’ve been posting the same stuff here for months, and every discussion gets derailed by people consoling you or advising you, which you invariably reject. Let’s stop going in circles.
I was about to disagree with what seems like a personal attack but then without looking at the posting history of the poster you replied to and just looking at the username. I have to agree.
On another note, I’m in my mid 40’s spent all of my time bouncing between yet another CRUD job most of my career (not complaining, it’s paid for a decent lifestyle in my relatively low cost of living area) and I’m always encouraging fresh CS grads to go for broke and take the r/cscareerquestions route of “grind leetCode and work for a FAANG”, knowing they will make more fresh out of college than I did until a few months ago.
However, as a University lecturer I know a lot of people Google have employed. In my opinion, they have rejected some of the greatest students I have ever taught, and accepted some idiots who know how to speak well.
While they have employed some good people, I believe they purposefully taret the type of people who think working at Google makes you a fundamentally better person than anyone else, rather than the best programmers/researchers/AI/whatever.
> In my opinion, they have rejected some of the greatest students I have ever taught
There's no question that Google is happy to have false-negatives in interviews.
> I believe they purposefully taret the type of people who think working at Google makes you a fundamentally better person than anyone else
why would they do this? And, if they do this, how are they generally speaking so successful? (this is also an amusing comment because the only group I find more critical of Google than HN is Googlers)
Then this would imply that selecting for people who want to work at Google, instead of the "best candidate" is a valid strategy because it leads to better outcomes.
Again I think this whole line of thinking is nonsense, but even if you take it at face value, it still doesn't make sense.
> And, if they do this, how are they generally speaking so successful?
They have almost complete monopoly on online search, web ads, online video, on phone OS, on browsers. And they are not afraid to abuse those to get more, and are getting away with this.
You don't need to do anything right when you rent-seek most of the online world.
I think it's very destructive of meaningful discussion to redefine a term of jargon in order to exploit the stigma of the original definition while talking about something else.
In real estate, the value of a property is often tied to the value of the rental income it could bring in, even if it's never been offered for rent.
'Rent seeking' doesn't refer to the same kind of 'rent', of course, but I think the analogy holds up pretty well. All Google has to do is turn a few screws, and the rest of us will have no choice but to sing whatever tune they call.
> And, if they do this, how are they generally speaking so successful?
When a company has a lot of money coming in, they can be successful despite decision X, rather than because of decision X.
Microsoft can interview people asking about filling airliners with golf balls. Valve can just not bother with Half-Life 3. Google can have zero support and a self-driving car division that keeps avoiding chances to release anything.
That the companies are successful doesn't mean all their actions are smart - sometimes it's that their successes are big enough the occasional bad decision doesn't hurt them.
Maybe the ability to get along with people is more important than some people are willing to admit?
I’ve met plenty of “smart people” who couldn’t get things done to save their lives partially because they couldn’t communicate well or play well with others.
Google don't want knowledge of algorithms, Google want fluency in creating algorithms. As a lecturer you likely missed a lot of gems since college testing doesn't test peoples ability to create algorithms and instead mostly test their ability to apply known algorithms. A lot of people who are great at applying known algorithms actually suck at algorithms in practice.
> In my opinion, they have rejected some of the greatest students I have ever taught, and accepted some idiots who know how to speak well.
This has been a source of angst for friends of mine at google for over 15 years, which is almost 3/4 of google’s existence. The hiring process is just random.
Part of it is due to measures put in to avoid certain unconscious biases (hire your friends, regardless of how good they are; hire only people like yourself, etc). So I have some sympathy.
I think this is really common. I've met quite a few Google engineers whose whole identity is defined by their employer. Its striking, and a testament to the malleability of the human mind, even if the individual is "smart".
From a brief look at job openings at Google Dublin, this mentality would be particularly mind boggling as it looks like the Dublin office houses the support staff (not that there's anything wrong with working support).
So, in general, Dublin Big Tech jobs are in:
- customer support
- sales
- finance (explains itself really)
- SRE (mostly because of favourable time zones)
Most of the core engineering is in London/Zurich. Dunno what's gonna happen post Brexit, but anecdotally, the FAANG which I am most familiar with has had real problems hiring in London post-2016.
I really want to like google cloud. Every time I use it things seem better set up than in AWS. I feel like I can't even bring it up at work though because of the issues with deprecation.
I hear you. At the end of the day though a little more effort on front end setup is infinitely better than the time wasted later on when you have to rebuilt everything every time Google decides to get rid of another product / service / platform / setting. Burned by that one too many times and moved on.
Google is an absolutely fantastic company from which to get free stuff. Gmail, Maps, Docs, Youtube? Mmm-mm good!
Buy things from Google, though? You mean, give Google money and in exchange expect them to adhere to some kind of standard of behavior, support, and customer service? Go get a coffee because you're clearly not awake yet.
This is the exact opposite of the usual complaint: that if you don't pay for support you can expect Google to arbitrarily shut your account and scoop up all your data - however paying them actually allows you some control.
Many of the frequent horror stories on here about account suspensions and completely unresponsive support are coming from people or companies with paid accounts.
I do believe that there is a level of spending where you can actually get a hold of someone competent who can look into things. But that level appears to be rather high.
About 5 years ago, Hewlett-Packard tried being a paid GSuite customer, paying for tens of thousands of seats, with a path to onboarding the entire company. We could not get support, or even a TAM who would answer their phone.
After a year, HP dumped GSuite, and went to O365 instead. We got a responsive TAM, full multi-tier support, and white glove handholding for migration. And, I think it was cheaper, too.
There is NO level of spending as a customer at which Google actually gives a shit.
I used to work with Steve. He’s a good story teller. But a story teller tells good stories by having a fairly casual relationship with facts and the truth.
For example, emacs lisp suffers from bitrot. Some core things haven’t changed in forever but also you can’t pick up elisp config from 15 years ago and it’s fine today.
All live systems evolve and change. In this case emacs is just so slow that Steve thinks that it doesn’t.
I don't know why people are downvoting you. I think you're right. I agree with his general point (I expanded a bit on that here [1]), and I also agree that people often overlook that general point since it is so convenient to do so. But otherwise, the specific examples he wrote (and the other assertions he made) seem to me to be incorrect.
But, as you said, he is a good story teller. Like Taleb. Getting the moral across is more important, even if sometimes the wolf eats little red riding hood in malformed renderings.
My elisp config file dates back to 1986. It's almost entirely unchanged. It still works. I have never had to remove anything because it stopped working. I just occasionally add stuff to it every few years.
“ it’s well-known that they refuse to host (as a managed service) any third-party software until after AWS has already done the same thing and built a successful business around it, at which point their customers hold them at gunpoint. But that’s the bar, to get Google to support something.”
I had Google cloud break twice this week. First their "AI Notebooks" images broke some dependency, then some IAM change broke the cloud console interface.
Though our engg. team is on it, Pointless hours of productivity lost :-(
Besides AWS and Azure what's the third major cloud provider he's referring to? (I can think of others, just haven't really thought any were close to Amazon/Microsoft/Google.)
You didn't read the article, did you? Startup developers don't mind supporting multiple options and backwards compatibility. Often they have to because they don't have the time and the money to rewrite everything, so it's common in startups to run and support multiple versions of APIs, clients etc.
It's Google who doesn't provide backwards compatibility, upgrade paths and migration options.
> But that's your job dude. So what if it costs more money and time. Fucking pay for it anyway.
So why doesn't Google pay for it anyway?
> I feel like startup software dev needs a swift kick in the ass or three with that phrase.... "do it anyway!"
I read the whole article. My reaction is such because I feel that much of software dev is plagued by the same bullshit that the article describes about Google
I've been on the web since the mid 90s. The last 15 years have had more cool things taken away from me because of sunset bullshit than the first half of that time period. Software (even web apps) should be eternal, there should never be "features removed" or "EOL" announcements. If they don't want to run something they should open source it and let the community take over. What the hell is the point of hoarding all this wonderful IP if it serves noone?
It's odd to see an article emphasising backwards compatibility, but not a single mention of Microsoft (I even ctrl+F'd the page source to check!) They've regressed a bit in the recent years but I'd still consider MS the gold standard for back-compat. I have Win32 binaries I last modified over 15 years ago, small (or perhaps tiny - size measured in KBs) utilities that I use daily. They worked perfectly on Win95, and still do on Win10.
I wonder when/if the software industry will ever stabilise. It's possibly the only industry where frequent and disruptive change continues to happen, and is even welcomed by many in it (not me).
> I’ll also give a shout-out to our friends in the Operating Systems business: Windows, Linux, NOT APPLE FUCK YOU APPLE, FreeBSD, and so on, for doing such a great job of backwards compatibility on their successful platforms.
I share your desire for stability. I don't necessarily mind frequent updates for security reasons, but I'm getting to the point where if things are changing in big ways often, I'm just not going to use it --- I no longer find enjoyment in running on the upgrade treadmill, it's just work.
That said, I don't see a lot of hope for our outlook. Most people don't seem to have a concept of completable software. It would be nice if it were bug free, but getting to the point where the cost to fix bugs compares to the cost of leaving bugs as known issues is doable.
OK but he talks about how Android OS has had backward compat for over ten years, but Windows has been doing it far longer and with a much bigger OS and more complex hardware base.
"completable software". that's the ticket. the maturity to say no more features will be accepted; it is done.
for the past 20 years we have been teaching the agile way, that points to the constant churn of requirements, the pivots, and the idea there is always another sprint
Linux code might be backwards compatible, but nothing like windows. I can run pbrush.exe taken from Windows 95 on windows 10, and it works fine. Want to run a 10yo binary on Linux? It's easier to set up a VM with a 10yo Linux release on it. You've hot the source code and you can compile it? Well, tough, the build tools have changed, and some autoconf script has been removed, even from autoconf-archive.
Actually, I've kept an ancient statically compiled Mathematica (linux) binary from 2000 and it actually still runs. I had to make some symlinks since X11 font paths have changed. That's a full GUI app with a sophisticated internal kernel.
This should be generally true of any well-compiled static binary on linux from 20 years ago.
I went googling for some Raymond Chen examples of the great and terrible things Windows does in the name of back compat, and came up with this old thread: https://news.ycombinator.com/item?id=13450160
When IBM starts to say they are going to deprecate something in mainframe, they will deprecate it in probably 10-12 years time. There will be 3 versions in between and each version will have a much better migration path. If a corporate don't want to take risk, they will be mostly the last ones to migrate, after confirming &consulting all the IBM tech champions, they would start the migration process which would last a year, before they install the new version.
I think NetBSD, or any other BSD generally, is also pretty good with backwards compatibility. I run programs well over 15 yrs old using the latest kernels. If I need to, I can build any of the older kernels, too, because all the code is still readily available (=gold standard). Earliest one predates Win95.
Your Win32 binaries sound better than whatever software is being distributed by the companies today.
The absolute worst in my small experience with it is anything touching the node ecosystem. Try to follow any guide or use any library not updated in the last week and there will be breaking API changes and deprecation warnings everywhere.
The sad part about the js ecosystem is that even the biggest corps like Google can't keep their own js ecosystem stable on the last version. I have a small Angular app with Firebase backend and every time I come back to it to update, the libs are out of sync
Google is notoriously bad at maintaining (or even designing) their open-source stuff. I wouldn't say "even the biggest corps like Google" -- Google does considerably worse at this than many hobbyist maintainers! I just don't use Google libraries anymore, if I can at all avoid it.
See also my other comment[1] - Angular is an example of such a magical does-everything framework.
We've abandoned an app because of the inability to keep Firebase working on it (JS libraries). Between out of sync Typescript definitions, broken APIs, broken interfaces, "this version of some other dependency works with this version of Firebase thanks to some random transient dependency", it just became too much. We won't use the JS libraries at all anymore.
Angular is the only one that has remained even remotely usable, but certainly not stable. It's never as simple as running the upgrade utilities, changing the version, and being done. It ALWAYS takes at least a day to find all the "little" things they didn't feel fit to mention in the upgrade docs.
It's aggravating. I genuinely don't like using Google software, at all.
I've felt for a long time that Google is coasting on the momentum of the early web and the awe people felt for what they built early on. That hasn't been Google for a long, long time.
I'm running into this problem more and more. It seems like Node is fundamentally unusable for anything but the nimblest teams that do not mind updating their dependencies automatically, every week, and when something breaks, they are capable of fixing it within hours.
I really like Node as a language (especially with TS), but Node as an ecosystem feels like a very hard fit into 90%+ of corporations.
I think the culture of Node comes from the earlier culture of in-browser JavaScript. When running on the browser platform you have to make sure your libraries are up to date and maintained, otherwise you're at the mercy of browser updates breaking things from under your feet (this is less bad now, but it was awful in the days of IE<9, and on android before they started shipping auto-updating Chrome webview)
Consequently, JavaScript developers got used to the burden of maintenance and rapidly updating to the latest versions of libraries, and this culture carried through to Node (partly because a lot of libraries are shared between node and the web).
Having said that, if you pick your libraries well, it's not too bad these days. When I upgraded from Node 12 to Node 14 earlier this year, I had to upgrade the `pg` package to a newer version that supported Node 14 (there was one available), but I didn't have to make any code changes. And other than that I've had no forced version upgrades in a long time.
I guess if you're lookig at 5-10 year timescales with literaly no maintenance then this would be a different matter though.
I strongly feel that you must have end to end tests while using node because of the dependency hell. Not even knowing if a upgrade of a dependency breaks your system is just hard. Also testing it by hand is just not maintainable.
There are massive differences in how much of a problem this is across different platforms.
With Clojure, I can think of 2 times ever when a dependency caused an issue. It was extremely obvious since the issue was "won't compile", and the fixes were simple.
With PHP, I expect any change to potentially break something. Bump your AWS SDK which uses a different minor version of guzzle? Fatal error.
There's a world of difference between breakage being an everyday thing and a true rarity.
This is only a problem in some parts of the ecosystem - but they are the parts that most 'tutorials' are written about, because they look shiny and fancy. If I'm blunt, they're the parts of the ecosystem that HN loves to gloat about in their "how I saved my company thousands of dollars by building a todo app with GlamorousLibraryJS" type posts.
The short answer: if you avoid the shiny tools that claim to do everything, you will not have this problem. And as a special case, avoid Gulp, which is mismanaged.
The single-responsibility libraries that have a well-defined scope, on the other hand, have often been sitting on npm unchanged for 5-6 years, because they are simply done, and they do what they need to. They will very likely never deprecate anything, as there is simply nothing to change.
I would argue that these libraries are actually doing a better job of stability than their counterparts in other languages.
Edit: An additional factor is that it's easy to write a tutorial about an extensive framework with a large scope; there's plenty of stuff to write about. Writing a tutorial about "this is how you make a function call to this library to parse a geo URI", on the other hand, probably isn't going to happen.
So if you follow third-party tutorials, you are naturally going to end up at the packages that are most prone to deprecations.
The popularity of Go suprises me when even relatively simple programs measure 10-20 MB or more. It is like Java. Is it even possible to write Go programs measured in KB?
I'd be interested in some concrete examples of things GCP have depreciated. I'm an (admittedly fairly light) user of GCP and haven't come across any myself.
Personal experience: I have to support an internal tool for a company in app engine. The contract is part time and I have to prioritize what issues to fix. The answer is usually "those caused by Google deprecating something". The nosql db some hyped individual has chosen years ago - "deprecated, we don't bother with solving bugs there". Email service: "it is in python2, but not in python3, look for management approval to use external service". File storage: "this library is deprecated, use whatever service is fancy now". Task queues: "nope, we are changing the api and calling it cloud tasks now, so please refactor all of your files to use the new semantics".
Dataflow SDK libraries have updated their minor version from 2.4 to something like 2.16 in just two years. Guess what, everything below 2.14 is deprecated and not supported.
And yes. Despite these being minor version updates, stuff will break if you go from, say, version 2.9 to version 2.11.
I saw that he mentioned a few of them in the article. He writes:
I can tell you that virtually everything I’ve used, from networking (legacy to VPC) to storage (Cloud SQL v1 to v2) to Firebase (now Firestore with a totally different API) to App Engine (don’t even get me started) to Cloud Endpoints to... I dunno, _everything_, has forced me to rewrite it all after at most 2–3 years.
I work on Firebase. His characterization of Firestore is incorrect and lazy. It did not replace Realtime Database (which is the product he’s calling Firebase) they are two products which live side by side and both are being actively improved.
People assumed that because we made a new database we were killing the older one, but that was never true.
"the customer is always right" doesn't mean every individual customer is always correct and you should never disagree with any of them, it means the customer, in aggregate, is correct and you need to react to customer opinions.
But also idk what else you'd want them to say "the thing you're taking shit isn't deprecated" seems like an important response to "you mishandled this deprecation".
I haven’t been totally happy with Google’s stewardship of Firebase, but that line in the article bugged me too. I didn’t think RTDB was ever deprecated.
Thanks for taking the time to list them out, didn't spot those in the article.
Personally, I think most of them seem quite sensible. Having a 'support everything forever' approach is obviously going to impose a huge burden on the teams who maintain this stuff which is then going to limit the ability to make anything better. The depreciation notices generally seem pretty good (12-15 months notice by the looks of it, sometimes followed by degraded functionality rather than complete removal of the feature).
that’s the whole point of the article though. Why should the “huge burden on the teams who maintain the stuff” be shifted to the customer who has less resources than Google to manage that burden. 12-15 months is not a long time. If you have to rebuild things that already work every year, you’re wasting a lot of time that could be spent building value for your customers.
> Having a 'support everything forever' approach is obviously going to impose a huge burden on the teams who maintain this stuff which is then going to limit the ability to make anything better
And yet AWS is doing just that while innovating at the same time.
Deprecate [1] doesn't mean remove though. While the old style paths are officially deprecated, they will be supported indefinitely. GCP's problem is that they deprecate and remove features.
I wonder what the complaints are regarding App Engine. I've been using it for over five years without any issue.
Come to think of it, the Channels API shut down was a bit of a nuisance. But it never worked all that well, and deprecation seemed like a reasonable move to me.
In relation to this, you then have AWS who basically support everything forever until their customers stop using it. They _still_ support people on EC2-Classic even though they would rather not. Every TAM I've come into contact with brags about how they won't deprecate until the customer deprecates.
There's a reason that people trust Amazon with their compute, and that's because in regards to their technology they're trustworthy.
Search each product in docs for the "Deprecate", "Deprecated" or "Deprecations" page (it's not common across product lines), they list every single functionality and date they deprecated stuff.. And yeah, they've done a fair bit.
> [Google] contracted with a company called Bitnami to create a bunch of “click to deploy” solutions, or perhaps I should write “solutions”, because they don’t solve a fucking thing. They’re just there as checkboxes, as marketing filler, and Google never gave a shit whether any of them worked from Day One.
I've had similar experiences with broken Bitnami images that makes it seem like they aren't even tested before being released [1][2]. Their WordPress images do work, but the way file permissions are configured doesn't play well with the way they have WordPress set up [3]. Have I just had bad luck, or is this a common experience?
There are lots of random downvotes for no real reason. Sometimes perfectly ordinary posts are dead. I try to upvote gray posts if they're not worthless.
That said, my take (and that of siblings) is that I'm much better off building my own containers than serving as unpaid Q&A for Bitnami. It would be smart to use containers built by people more expert than I am. Containers that are broken by default do not meet that criteria. So, "reach out to community support" is not helpful to people that have already decided to ditch Bitnami.
> We're careful to pin versions, and hadn't anticipated someone would re-write history and break published code.
This is exactly the kind of thing that will make me ditch a company forever and never look back.
If you want to pin your image you should either use the most specific tag or just use the digest:
$ crane digest bitnami/mongodb:4.2
sha256:8650c2d92eea97732eae359a140ee86ee3923a2a19b19443e1dc01ec20d5387d
$ docker run bitnami/mongodb:4.2@sha256:8650c2d92eea97732eae359a140ee86ee3923a2a19b19443e1dc01ec20d5387d
Now, we might have introduced a regression between some version of a container and the next minor version; shit happens; it's hard to tell without more specific information though.
> > We're careful to pin versions, and hadn't anticipated someone would re-write history and break published code.
> This is exactly the kind of thing that will make me ditch a company forever and never look back.
On the other hand, assuming that instead of a misunderstanding, what you saw is actually the image behind an immutable tag such as 4.2.8-debian-10-r50 being replaced, this is a serious security issue; somebody could have hacked docker hub, or crafted a valid certificate for docker hub and MitM'd you,
I'd also ditch a company forever and never look back if they honestly don't care about that problem, which I assure you it's not the case.
We'd greatly appreciate reporting such cases to security@bitnami.com / security@vmware.com .
Reminds me when we were implementing Bitnami's OpenEdx and it was crashing due to a Python file importing newrelic (reporting SDK of that product) but it was not installed. Seems they didnt test that release ran at all.
I've used the Bitnami images to set up Wordpress a few times and won't again. It takes less time to do it from scratch once you know how, because of the exact issues you mention - you keep getting bitten by details that end up taking longer than just starting from scratch.
In fact at this point it's easier to use AWS Lightsail.
> “[...] Drop whatever you are doing because it’s not important. What is important is OUR time”
> Most NPM packages
NPM packages tend to be free and open source so morally they don’t owe us anything. So yes what is important is the time of the authors of the packages. I am grateful someone took the trouble to release them and many popular ones are actually well maintained.
But cloud services that you are paying for are a totally different ball game. The company does need to maintain backwards compatibility for a good amount time and they need to value your time.
I don't care about moral debts or obligations. I care about whether stuff works and I can get shit done.
Other ecosystems (like Python) don't have the same culture as NPM. There are clean, well-documented, maintained packages.
Google doesn't have a moral obligation to support GCE customers either. It's just that everyone I know who has used GCE would never use it in production, and loudly advises everyone they know not to do so either. They'll be bleeding money at some point, not to mention they're already bleeding reputation. Google went from smartest-people-in-the-room to smug-incompetent-arrogant-douchebags. That's hard to get over. That's about good business, not some kind of moral obligation.
I recently returned to dev work after years in a niche of systems administration in banking and healthcare. The existing product is built on npm, and after a month in, I'm simply disappointed that such a fragile set of software is this popular.
Almost every day there is an automated PR to bump one of the thousands of dependencies. LTS is ending for Angular 8, but one of the main component dependencies doesn't support 9 yet. I had to put typescript in an unsupported state so one of the dependency upgrades wouldn't break something -- the solution is an ugly hack not supported by anyone. Another required a bump to lodash, which suddenly started failing. This led me to the following comment in a github thread:
> The recursive implementation of cloneDeep was by design with the shortcoming that in some rare cases it could run into stack size exceeded issues. I'm not how cloneDeep is being used but chances are a more specialized clone tailored to their scenario will be a better option.
So, I agree with the OP: the messaging for support in the npm world can be distilled to "fuck you".
If I had ever proposed to a customer that we would build infrastructure for them written by thousands of pseudo-anonymous entities who may or may not break everything because they felt like it, I would have correctly been fired.
This leads me to believe that supportability, reliability, and security aren't important to the npm ecosystem because ultimately the products built with npm don't matter. A bank or a utility company may use them to build a customer facing application, but for the stuff that's important, a competent technology manager would never let npm through the door.
(edit: I replaced "tools" with "software" in the first paragraph, because tools don't change interfaces and effects on a whim).
> I have been inexorably pushed away from GCP, towards cloud agnosticism.
On a side note I think this is what all of us should be aiming for. Every time someone on a team I'm working for starts trying to use some very complex tool that a SaaS/cloud has built for us I always urged them to look for a simpler solution.
Do you really need AWS Lambda/kinesis/etc or can you write your code and deploy it onto our cluster as a normal service/api/control loop?
If you make it really easy to write code, deploy it, and monitor it, you'll often find it's way less devops time to write your software as apis/control loops than it is to use a large amalgamation of cloud services. I also haven't found a good way to unit test cloud services and their interlinking.
On the contrary, every time someone comes to me with a suggestion to build something in-house or use an open source solution when a aws service exists I say the hell with that. I don't want to maintain it, build features into it or operate it.
I want full in vendor lock-in, it saves an immense amount of time.
You just need to pick a vendor that gives a shit about it's customers to be locked in to - and that's simply not google.
It's tough to avoid the time savings that vendor lock-in buys you these days. I have had to grudgingly accept that, despite the fact that I absolutely hate giving in to it. Look at what Apple has done with lock-in (or Google for that matter in other areas of the ecosystem) and you can see the reasons why.
However you can take steps that even when you give in to lock in, you structure your code/projects so that switching to another standard is straightforward: include non-vendor build scripts that would work on any Docker/VPS/baremetal server, isolate the locked-in config and scripts from the rest of the app as much as possible (even if it means running separate external scripts to execute them), etc.
Once you've done it a couple times you build up a suite of small tools and it gets easier.
I got bit hard by Zeit/Vercel, and then by GCP, and then by AWS. All of them do one thing or another to manage to find a way to extract more money. My goal is to make it so that if one day I decide we simply need to leave the platform, we can at a minimum of effort.
Because this actually does happen 1-2 times a year, thanks to exactly the issues raised by OP, it's paid dividends to put in the small time it takes to keep things mostly agnostic on the code side.
No, the difference between python2 and python3 in the managed services goes much beyond unicode in the Google universe.
The most obvious example is that in app engine standard python 2 you can send and receive emails while you need a third party service in python 3.
In my other comment I've mentioned some other issues when Google start neglecting stuff.
Yes; exactly this. I'm contracted to Google, and so much of my Googler colleagues' time has been wasted in the last year porting projects off of the Python 2 APIs which are about to be shut down.
Are they porting to the AppEngine Python 3 APIs? Hell no: those are totally useless (no BigTable, no user authentication, etc.). If you have to rewrite everything to replace all the functionality that used to be "batteries included", you might as well build something can run on any old bare VM instead, and so—like any sensible external developer—my colleagues are doing everything they can to avoid depending on any Google-specific APIs, libraries or platforms.
I really though AppEngine and PaaS were the future, but evidently that future will not be written at Google.
> Are they porting to the AppEngine Python 3 APIs? Hell no: those are totally useless (no BigTable, no user authentication, etc.).
That's hilarious(ly bad): BigTable was like the main selling point of OG AppEngine, and they practically forced their user authentication scheme on you too.
The old Firebase API. No longer possible to create new databases that can be used from the old API.
Which in practice meant that "migrating to a new database under a different account" translated into "rewriting half a codebase due to a cascade of breaking changes in the surrounding tooling".
> > I know I haven’t gone into a lot of specific details about GCP’s deprecations. I can tell you that virtually everything I’ve used, from networking (legacy to VPC) to storage (Cloud SQL v1 to v2) to Firebase (now Firestore with a totally different API) to App Engine (don’t even get me started) to Cloud Endpoints to… I dunno, everything, has forced me to rewrite it all after at most 2–3 years, and they never automate it for you, and often there is no documented migration path at all. It’s just crickets.
it actually winds up being less DevOps work, on average, to support open-source systems running on bare VMs, than to try to keep up with Google’s deprecation treadmill.
I honestly feel like this is a better idea in general, at least in data science stuff.
For context, I tend to get dumped with undocumented poorly written untested DS code bases every time I move to a new job.
The difference between setups where there's a bare VM/server and it runs using standard tools (command line scripts) or at least some open-source orchestration and any kind of proprietary technology tied to infrastructure is massive.
Additionally, the server stuff can be maintained and debugged by a far larger number of people, thus reducing bus factor.
Like, maybe in the web dev world there are things that everyone knows that are AWS services (maybe S3 counts, I guess) but in the DS world, I just know that I'm in for a whole lot more pain if they haven't just used bare servers.
this is the classic advantage of Free Software and Open Source over non-free tools. you often get a mix of commercial services and inhouse custom development that is no better documented or supported than FOSS alternatives.
Keeping your stuff up and running, patched with the latest security updates takes time. And opensource systems also break compatibility at times. Sometimes the installation/upgrade process changes over time and you need to learn how to do proper upgrades even if the software technically can be made to work with the rest of the system.
It's hard to quantify, but I don't think it's so clear-cut as Yegge makes it look.
well, being forced to adapt your system because a service provider is not allowing you to keep it running can be a significant drain on your resources. especially when you can't control when you do it.
sure FOSS needs to be upgraded too, and sometimes there is change that you need to adapt to, but you are in control of the schedule, and if you are busy you can delay the upgrade work to when you have time for it. a service being shut down won't give you that flexibility
Google gives you 12 months notice before turning off the lights.
A FOSS tool "forces" you to upgrade when the old version you're using has a serious security issue that's is not backported. Which means that unless the supported version is fully backward compatible with your software, you have a quite tight schedule to deal with.
Sure, you always have the "option" of running insecure systems. Unfortunately a lot of people choose that "option" and severely underestimate its cost...
an upgrade with some incompatible changes is still a world of difference compared to potentially being forced to rewrite a whole bunch of code because the system you used is completely discontinued and the alternative that replaces it isn't even remotely similar
The mishandled Python 3 transition sure created a lot of pain (I experienced some of it, even though I only started with Python in 2016), but I never realized how much it hurt the language’s momentum. This must be one of the reasons why JavaScript is so popular today. JS benefits here from being run in browsers, which the author points out are fantastic at back compat, even as the dev culture surrounding the language doesn’t value it at all.
I really love the array.map( e -> e.doStuff() ) syntax. You can do similar in Java and other JVM languages of course. Not sure whether you can do that in Python.
But map() (in either language) has the semantics of applying a transformation to each item in a collection to produce a new collection, so you wouldn’t really want to use map() to call something like doStuff(). That's what normal loops and forEach() are for. If you aren’t transforming the collection into a new collection, map() is the wrong choice; you’re telling anybody reading it that you’re doing something that you aren’t.
But assuming doStuff() just has a bad name and is actually transformational, then the most direct translation would be to use map() and a lambda:
new_array = map(lambda e: e.doStuff(), array)
However it’s more idiomatic to use a list comprehension:
I never realized how much it hurt the language’s momentum.
Yes. The combination of a botched transition with a high arrogance level was hell. Here's something I wrote in 2015 after porting a medium-sized system from Python 2 to Python 3.[1] I listed a number of package problems encountered during the conversion. Ones that indicated those libraries weren't being used much in production. I discovered that the SQL connector broke if you loaded a database of tens of thousands of records, for example. And oh, did I get hate mail. You can see the comments below mine.
That's what led me to Go. Go is a mediocre language. That's a strength when you just need to get something done. It comes with a set of libraries which are heavily used internally within Google. So, if you're doing something that is server-side for some web-related task, the libraries for that are probably both present and well-debugged. And they don't seem to be deprecated rapidly. I'm not using them for any Google-specific services, so I can't speak to that.
Yeah I have the same thoughts about go, which I've been using every day for the last four years. It's super rare for an upgrade to break anything (has happened though). When it comes to running stuff in prod, boring is good.
> In the Emacs world [...], when they make an API obsolete, they are basically saying: “You really shouldn’t use this approach, because even though it works, it suffers from various deficiencies which we enumerate here. But in the end it’s your call.”
> Whereas in the Google world, deprecation means: “We are breaking our commitments to you.” [...] It means they are going to force you to do some work, possibly a large amount of rework, on a regular basis
It's fine if they want to go hunting for their Perfect API v10™, but just leave the old APIs alone - build abstractions under them if needed. Don't push that work onto your customers.
> Let’s say hypothetically that Apple was dumb enough to pull a Guido van Rossum, and declare that Swift 6.0 is backwards-incompatible with Swift 5.0, much in the way that Python 3 is incompatible with Python 2.
Damn, I was waiting for this. The whole essay was a setup for this paragraph. Reframes the argument using a shared traumatic experience for everyone. This humanizes the effect, we all know someone that lost someone to the Python2/3 transition. I can't count the number of Python friends that turned into Gophers, joining some cult of middle class bearded tech dad with a side business distilling pear brandy.
Best Yegge Essay Yet! Welcome back Steve.
edit, finished the essay. He smoked GCP in this one, damn damn hard. Hickory.
Apple already pulled the Python 3 stunt twice with swift. Swift 1 to Swift 2 to Swift 3 were all very much incompatible.
Of course the language wasn’t as broadly used back then and Apple is known for quicker churn than Python so the need to write code compatible across Swift versions was much less pronounced
Swift can be very painful from this perspective. I just recently had to do surgery on a project because one of my dependencies changed the tools version.
The back-compat promise is one of the reasons I'm investing in Rust for new projects, even if I find the actual programming much more pleasant in Swift.
But I'm a bit less optimistic about Rust now given the recent layoffs at Mozilla.
I haven’t used Swift heavily till recently, but didn’t they say from the beginning there would be lots of breaking changes in the first couple versions? Doesn’t make it less painful but they were at least up front about it.
Async/await currently looks like it'll be a non-breaking change. Obviously you'll have to redesign a whole bunch of code to actually take advantage of it, but you won't have to adopt it just to upgrade to the newest version of Swift. If you decide it's not worth the effort, you'll have the option to just not redesign anything.
Yes, hopefully not a breaking change, but a lot of APIs and libraries will have to be redesigned completely in order to allow people to take advantage of it. And a lot of old code will look inconsistent for quite a long while.
I'm a bit surprised that language support for such a key concurrency feature comes so late in the game.
It's not the same issue that Yegge describes though.
“So late in the game” is relative, no? Java doesn’t have it, Rust stabilised it last year after ~9 years (or around the same time period if we’re comparing 1.0 releases).
Async/await does not need to be a breaking change - C# managed fine without it being so. The breaking changes in Rust were around the standardisation of std::futures::Future rather than async and await.
That said async/await makes a world of difference to the usability of Rust for async code and I’d imagine it will for Swift too.
The only breaking change was that async because a keyword, the Futures stuff didn’t cause any breaking changes. There were three major versions of the library before it moved into the stdlib, though.
Right, by “breaking changes” I mean across the ecosystem rather than in the Stdlib. There’s still a decent amount of code based on futures 0.1 and many libraries that depended on it required rework as a result.
When did Apple “mandate” apps move to Swift? The majority of code out there is still Objective C.
The problem isn’t making an incompatible “v2”. It’s getting rid of “v1”.
If I can’t take advantage of new functionality without moving to a new version that’s a headache. But it doesn’t cause me to throw away all of my existing investment. Especially if you can combine the old and the new.
Apple has not mandated moving apps to Swift. Obj-c is still fully supported and even continuing to be improved. They have not announced any plans to deprecate obj-c either.
I don't they'd ever deprecate Objective-C when so much of their internal software and the biggest apps are still 100% Objective-C. Objective-C support is a good counter-example to Apple supposedly deprecating things.
Good luck on deprecating Objective-C and the C++ toolchain.
As an example core of Unity is written with C++ with Objective-C bindings for APIs such as Metal.
If Apple would mandate Swift then things like Unity would just cross compile into swift, causing perf regressions for users, all for no benefit for Apple.
> we all know someone that lost someone to the Python2/3 transition
This feels like a very strange perspective. Why is it a problem to "lose" someone from a language? You're an engineer (a presumption on my side); casting yourself into a subsection of a "Python engineer" seems like a move with all the downsides and no upsides.
I don't get the Python3 bw-incompatibility hate. Getting rid of cruft is literally the only way to get rid of cruft. It's literally the Pythonic to not have cruft.
I dread the day Python will be as bloated as C++ or Java, neither that dare to remove things.
Arguably worth it. Dividing 1 by 2 with "1 / 2" and getting 0 is a real 'wat' for novice programmers. And the workaround is a 'wat' for experts: instead of using a floating point division operator, cast one of your parameters to float.
Hey! Leave Java out of this, Java didn't do anything to you!
Joking mostly aside, python is way way closer to C++ in terms of language feature bloat; where Java is notoriously spartan and slow moving (moreso historically).
Ofcourse I know you're also reffering to standard lib gunk but that's relatively less impactful.
The big issue with the python 3 shift was when you touch something as substantial, subtle, and pervasive as strings you're going to cause problems. I wont speak to if it was worth it.
Probably less to do with backwards compat then how it was handled. The first few versions of python 3 didn't really provide any compelling benefit, (indeed things like performance were worse, not better), and most people didn't _care_ about the cruft it removed. The "print" statement being a great example. Was it kind of weird? Sure. Did that weirdness have any consequences at all? Not really. Was it worth it to force people to replace thousands of lines of code for something that was essentially a minor aesthetic issue, if that? No way.
Also the python 2 string type was more in line with how other unix utilities work, so while I do think the 3 string type is generally better for the web, it wasn't an obvious improvement for the other large usage of python. While it's a very versatile language, I would say the major niches of python are: web dev, shell scripting, and scientific computing. The changes in python 3 were (somewhat) helpful for web dev, but really were inconsequential and/or actively harmful for the other uses.
Mixed with how condescending the python dev team acted about it, and how it really wasn't until about python 3.6 that there was any compelling reason to move to python 3,you can see where most of the hate comes from.
And by the way, even if Python is very popular today, I do think that rift really hurt the language. I worked at least at one company where Python was considered for a project, but eventually shelved because of management's confusion over the 2/3 issue.
Agree with everything you said here, but let me riff on this
> The "print" statement being a great example. Was it kind of weird? Sure.
I am reminded of the Emerson quip "A foolish consistency is the hobgoblin of little minds". The Py2 print statement is special/weird because it is a very important and special behavior, especially for newcomers. It's not randomly special, it's special because it models something that deserves special treatment.
Py3 is more consistent, but less humane. I use it over Py2 only because the market has moved on, but every time I type print with parentheses I curse Guido.
Two simple reasons. First, "hello world" requires less explanation
print "hello world"
vs
print ("hello world")
Second, printing a line to stdout is an extremely common and special activity (especially during debugging). There is almost no programmer-defined function you could write that has a function as fundamental as that of "print".
Larry Wall of Perl thought about these things in terms of Huffman Coding, i.e. the most common thing you might want to write should have the syntax elide any easily understood context.
In the case of "print", we know we're not talking about any random function here: it's the function for getting output to the terminal, one of the most fundamental operations in all of programming. So I think it's totally fine for it to have a special representation in the language.
One nit I would call-out is that most people don't put a space between the print and the ("hello world"), so idiomatically when learning, print is 'just another function'. And since HelloWorld is usually the first thing people learn, I wouldn't want them to think print is different than any other function.
Also, the first line of The Zen of Python is "Beautiful is better than ugly", I can understand why consistency was a priority here.
We could get into a whole other discussion about whether or not a programming language should have and adhere to a "culture", but fundamentally I agree with the arguments against Python making this change. It's just not worth the effort, and like others have said, they might've lost of a lot of momentum to JS.
The funny thing is, if you type print with space it recognizes what you meant, and still wouldn't do it, raising an exception instead. It would've cost them literally nothing to support both ways to write print, since the parser has to detect statement print anyway.
That's great for new programmers. It's terrible for people in a mature ecosystem of lame duck software. Some production critical python code is 20 years old.
Most hate originates because they question whether it actually was cruft and wheter the new solutions are really better. And even more hate critics the way transition was handled. And indeed, that some removed features were added later again speaks for a certain lack of insight when decicions were made.
That the transition took nearly a decade (and for some still goes at this point) and several minor-versions is another hint at strong problems in planing. At the end we got something better, but the road toward it was just far to painful for what we got. And this fuels hate till today.
Python3's intentions were good, but the execution was abysmal. It was just a bunch of extra work on top of what should have been a pretty straightforward migration.
this is a really nitpicky thing to debate, but for me, since this essay is basically a sequel to the infamous Yegge Platforms Rant, then the platforms rant is still better, because it is much older and still correct. he could have directly copy pasted paragraphs into this essay and it would still be relevant
Apple is AFAICT the least concerned of all the major platform vendors about backwards compatibility. Programs routinely stop working on new releases of macOS or iOS, sometimes in ways that are impossible to fix. On Mac they’ve already changed processor architectures twice and look poised to do so again. That’s much more disruptive than the python 2 to 3 migration.
> That’s much more disruptive than the python 2 to 3 migration.
Yet Python 2 to 3 started in 2008 - 12 years ago, and is still not “done” in any meaningful sense.
If I think back to 12 years after the intel OSX transition - 2018 - I can’t think of a single outstanding issue, whereas I bash into python 2 vs 3 literally every time I touch it (and therefore mostly choose not to).
I am slightly disappointed that folks choose to focus on Swift in this quote. The essay was pointing out how GCP deprecates products and makes users rewriting running applications. This is akin to forcing a Python2/3 split with nearly every deprecation.
I have heard that at Google, you get promoted/rewarded for launches. To me, that seems like an obvious perverse incentive to continuously deprecate things, so new things can be "launched" to replace them. There are no "launches" if you're just maintaining backwards compatibility!
Maybe GCP should take a hard look at their internal incentives?
> One fun bit of trivia about Bigtable is that they had these internal control-plane entities (as part of the implementation) called tablet servers, which had large indexes, and at some point they became a scaling bottleneck.
Anyone understand the technical jargon here? I'm a bit lost.
> Bigtable maintains data in lexicographic order by row key. The row range for a table is dynamically partitioned. Each row range is called a tablet, which is the unit of dis- tribution and load balancing.
And later...
>The master is responsible for assigning tablets to tablet servers, detecting the addition and expiration of tablet servers, balancing tablet-server load, and garbage col- lection of files in GFS. In addition, it handles schema changes such as table and column family creations.
Each tablet server manages a set of tablets (typically we have somewhere between ten to a thousand tablets per tablet server). The tablet server handles read and write requests to the tablets that it has loaded, and also splits tablets that have grown too large.
I think Google couldn't do better, even if they wanted to.
The reason is the engineering culture and the promotion system. People expect to switch to new projects, and work towards to a promotion within 2-3 years. This shows in all products, and presumalby also the reason for the deprecation. New people come on the project, want and need to do a major overhaul, and do not have resource or incentives to support the old stuff.
The other half is that most projects are done efficiently by relatively small groups of engineers. I remember that Android Market was still a few people in MTV, with no payment possibilities outside of the US, when Apple already had significant revenue from apps. And that shows everywhere. I'd argue Google is incapable of tackling projects that require more than a few hundred people for more than 2-3 years.
But as a billion dollar company, that does not work. Throwing money at obvious new billion dollar opportunities like Microsoft did with gaming and cloud, and following through over 5 years is not possible for Google.
My armchair thought: what if the CEO said "wow, Yegge is right, we've gotten really off trail here. you need to show leadership in maintaining a 3+ year old system at Google for promotions?" Could that happen and would it work?
Yes, but there might be other downsides... Suddenly every developer turns into 'maintenance man' just refactoring existing code to make it neater without adding much.
The solution to this is more qualitative performance evaluations. People are really good at gaming quantitative systems, so once it becomes a rule that more releases == more promotions, people will prioritize that over actual value creation. Same for if maintenance work becomes the metric.
You need the people evaluating performance to really understand the work, and make informed, discretionary decisions about who to promote to maximize real value creation.
> You need the people evaluating performance to really understand the work, and make informed, discretionary decisions about who to promote to maximize real value creation.
I agree but for everyone working at / running a company without a money printing machine it is worth noting that this is really expensive. You need to take someone that’s a good engineer—-one with judgment and a modicum of people skills—-and give them mostly thankless, stressful work.
The promotion system isn't the issue, the promotion system was the issue 10 years ago. For the last 10 years Google has been hiring and promoting and keeping people who prioritize shiny new things more than maintaining old things.
If you start rewarding maintaining old things you aren't going to suddenly get what you want, you're just going to lose a tonne of people because they don't want to do what you're telling them and you won't be able to hire good people with your priorities because no one is going to believe they can have a good career at Google by prioritizing stability and long term support. It's all downstream of culture, and culture change is very difficult.
Of course if you do buy into this idea that this is the issue, then GCP has already lost. It would be years before they could make a dent in that culture, and more years after that for external customers to know and trust Google's new culture. By that time there's going to be a dominant player anyway. So maybe actually it's best to stick with the current culture - try and win on your strengths than try to fix your defficiencies.
It might be even deeper than that. I don't know if it's representative, the few Googlers I talked to were there only for the money. Their view of the company was actually quite negative.
Keeping backward-compatibility requires people that care. That have enough pride in their work to counter-balance the grind.
Obviously having people that care about the work they do is a positive thing, but there are plenty of people in plenty of industries managing to do a good job despite only being in it for the money.
It might not be a good business decision for Google, but surely if they create enough positions that are solely focussed on the boring stuff but are paid better to make up for it, there is a price point at which they'd get enough interested people.
The fact that most people are there mostly for the money (which I'm pretty sure is the case) actually works in their favor if they wanted to prioritize maintenance more. Currently people don't do it because they're not rewarded for it, not because they don't care. It's actually far easier to steer people towards certain behaviors if you have lots of money and all you need to do is give more or fewer RSUs or bonus $$s.
That would be a career suicide to become a support engineer like this. 4 years later, when his stocks dry out, he'd have to switch the company and he'd need to tell a convincing story why the new company should pay him the new market rate. And "I supported legacy code" is not such a story. There's always an option to go to Microsoft and support legacy stuff for life, but beware that MS pays peanuts (relatively speaking) and with the MS pay you'd be priced out of the housing market.
To show that you deserve whatever pay you need to show impact and complexity. I don't think you would have a problem showing either when talking about maintaining Google scale products...
Now we are talking. The thing is, only few work on those big projects. The rest are doing god knows what and have to jump ships often. If your resume says you spent 5 years maintaining a smallish noname project, your career is at risk. I don't believe that Google keeps all its 100k employees busy with complex and high impact projects.
And that's assuming people cannot find satisfaction in maintaining things. Personally I do find it satisfying to be "polishing" the same software for a long time. Iron out bugs, make performance improvements here and there, update to newer frameworks/APIs. Code is like a living thing and keeping it alive can be very rewarding work.
Seems like this is probably true for the vast majority of people. I don't know why we circle jerk on passion and it's not very healthy. Sure I love to write code and most days I don't hate my job, but if I didn't need the money I'd rather write code for myself or open source projects, or do other things.
There's nothing wrong with that, and when I'm at work, I do my best to do a good job and make reliable, maintainable systems. Those two things aren't in conflict.
Periodically I'm jealous of googlers who try new stuff. Our company started ~1975 and comes with a good bit of crappy legacy code that as good business and customer driven people we maintain while we deprecate and replace components. At times it's boring, an operational pain in the butt, and frustrating. I'm working on my second major deprecate and replace project now. But when I'm done I'm gonna move to green fields. We sometimes are too conservative.
> The promotion system isn't the issue, the promotion system was the issue 10 years ago. For the last 10 years Google has been hiring and promoting and keeping people who prioritize shiny new things more than maintaining old things.
> you're just going to lose a tonne of people because they don't want to do what you're telling them and you won't be able to hire good people with your priorities because no one is going to believe they can have a good career at Google by prioritizing stability and long term support
I don't follow. Wouldn't ridding Google of its reputation for inadequate maintenance, make it a more prestigious employer? Its reputation for paying well wouldn't change. How would this lead to an exodus?
Obviously generalizing here, but most engineers (especially fresh graduates) would rather work on a new thing than do maintenance, if given the choice. Google offers a pretty good value proposition: Lots of money and you get to work at new exiting projects and if you don‘t like it you can change to something you like. There‘s obviously more nuance to this, but that seems like a pretty good deal for many people.
Reputation is a lagging indicator. You have to deal with the fact you're trying to recruit people who care about backwards compatibility into a company with a strong reputation for not caring about that. You're going to have the bad reputation long after the underlying problem is fixed.
Maybe... Just maybe they need to let those "shiny things" engineers leave if they don't like keeping things running and continually improving them after v1.0. If you only hire people who get bored with a project after a year or egotists who want to start from a green field on every single project, then this is the effect it will have on your products.
I think that's only part of it though. I think that the only way a company can let products wither for years without a single new feature or improvement is if they don't have a product manager to represent the users, and bring a vision to the engineering team.
Culture changes real quickly with the right leadership at the top. Bezos and his "everything as a service" mandate, it didn't take long to implement and worked just fine. What you need is empowered leadership and the willingness to take a short term pain for a long term gain, that thing that Yahoo couldn't do and ended up dying of.
That could be fixed in 1 day if the CEO simply said hire people to maintain this stuff, and don't require them to get promoted. Google's a mega corp now, not a startup.
And they pay white well at entry level. Most people don't care about promo unless forced (up or out).
You grossly underestimate how long it takes to change the culture of a company, let alone one the size of Google. A CEO can’t snap his/her fingers and direct a change of this kind into existence.
416 comments
[ 4.6 ms ] story [ 287 ms ] threadBy comparison, I have nothing. zilch, for my hard work. Neither do most people working on core tech at AWS. That's why they empathize with their customers (I'm guessing).
Fun fact for you - if I get promoted this year at Amazon to L5, I'll _still_ make less than a new grad at Google. It's extremely depressing to the extent that I can't get out of bed in the morning, but I'd wager that inferiority complex probably helps on the product side to empathize with customers.
This doesn't appear to be close to true based on levels.fyi, unless you're severely underpaid for an Amazon L5.
*The net effect is "pay for performance" isn't really a thing.
The only way for me to do a significant compensation bump is to do what's called a "dive and save", but this is very rarely done for L5's and is significantly more common for L6's. I got an offer for about $200k a year ago at a hedge fund, but that wasn't eligible for dive and save because of my level and I'm (clearly) too mentally defective to pass a Facebook or Google loop.
The other more popular approach is to "play the game" as I think of it, I.e. every 18-24 months you switch to a new job with a substantial increase in pay as result.
I wish this wasn't the case, but I guess it is what it is.
"Just move" is something someone without any empathy for other people's circumstances would say - most people aren't capable of "just moving" as if the only concern is choosing an employer.
No disagreement on the good recommendation as a signal, but I doubt that'll matter to the HC if I can't pass 3/5 rounds at the minimum.
That long enough part is the kicker and that terrifies me in every job search (to the extent I have sometimes jumped at the first half decent opportunity, and agreed to salaries below my potential). I find it does help to apply often and everywhere and not be invested in any particular opportunity.
With dating they say every failed relationship is a step toward finding the right one, and I think that same principle applies to job interviews.
Anyways, that’s my two cents
Mind you, he would probably have turned us down anyway to go do a PhD, so he probably dodged a bullet.
You make six figures at a top tech company and you’re bitterly complaining because you think you’re entitled to more, and in the same breath you’re chastising Google’s pretentious culture? There’s a lot of irony there.
Focus on improving yourself and don’t get caught up in comparing yourself to others.
I see too many people talking about their Tahoe Google offsites (jbd@ on twitter claims a bunch of her coworkers just work from Tahoe during ski season) or their Colorado skiing weekends.
Anyways, hypocrisy doesn't mean I'm wrong, and I think its valid to be mad that I'm seen as inferior daily.
Yet you still make more than what the vast majority of engineers in western Europe will ever make in their whole career. And that's still only looking at the most developed countries.
The job market isn't fair. No use getting burnt over that.
I'm also quite surprised by your wage comment: a common aphorism is that Amazon cheaps out on everything except real estate and compensation.
Which ones do and don't? Because all of the ones I've heard of have pretty loose ones compared to my organization.
And yes, Amazon doesn't cheap out on real estate because we own so much of it. It's honestly remarkable how it's done. But I was under the impression everyone knows they pay less.
I only have about ~40 stocks up from about 35 before the refresh cycle (unvested) and that's unlikely to go up if I get promoted (if you have too many they don't give you more). Googler and FB refresher values are significantly higher.
It's not unique to Google. Lots of organizations intentionally build a culture of elitism. However, it means I would never, ever rely on Google for anything. Free products like search are a-okay. My email is on gmail because legacy.
Build a business on Google platforms? Nope.
I don’t even work on the “core tech”, I am on the consulting side, probably make less than you do (albeit in a much lower cost of living area) and I am probably older. But, I am not throwing a pity party on HN
I’m not even here to shame $150K. That’s what I was making as CRUD developer pre-Covid at 45 years old (long story) and I also graduated from a no name state school - in the 90s. $150k still means you’re making more than roughly around 80-85% of American households.
I fully understand shaming $150k, new grads at Google make $180-$210k.
You graduated from college and are now making more than most developers in the US and you work at a big tech company. You have the opportunity to leverage that and either get promoted from within or change jobs.
Are you really struggling making $150K straight out of college? You graduated from a state school (as did I) you probably don’t have that much debt.
For context, I’ve been out of college almost a quarter century and I am just now making a little more than a college grad at Google. I’m not complaining and I only have a good 10-15 years to take advantage of the opportunity. You have your whole life ahead of you.
Stop complaining, get on the grind, and do what you need to do. No one owes you a quarter million just because you walked across the stage with a CS degree.
I think the real problem is I'm not capable of doing so, at least according to the fair labor market.
> Are you really struggling making $150K straight out of college? You graduated from a state school (as did I) you probably don’t have that much debt.
I don't have any debt, I just have a really pathetic savings rate and a really low net worth compared to nearly everyone I know, including folks that didn't do "the grind" that I did. I'll probably not be able to retire or own a car, much less own a house someday.
I hate that we as a society can say that if you have signs of cancer go see an oncologist and it’s not okay to say that if you show signs of mental health issues go see a professional.
But everything about your comments and your user name points to it.
I don't like the IQ/talent fetish that seems to pervade Anglophone culture. Some people have a stronger natural predisposition to sit and stare at a screen and figure out puzzles, and that happens to be sought after on the labor market. Doesn't mean others have less inherent value as human beings.
Would be nice to have a small appartment one day. I just try to offer some perspective.
Dude, please seek mental help. You have so many working years ahead of you. Stop worrying about owning a house. Start budgeting using something like YNAB and get your savings rate under control. Also get all the basic investment vehicles going: 401k, after tax Roth, backdoor Roth etc. You’ll do just fine.
A lot of my friends are already putting money down on houses and are making hundreds of thousands in options trading on principal that I _don't have_. I'm falling behind permanently and even maxing out my 401k (which I didn't do last year because I don't anticipate living to 65 but decided to this year) won't get me closer with a low principal.
Not living in that area anymore, I can't say. I haven't lived there for 10 years.
I think the Pixel division is the only exception, though I don’t own one so I can’t comment personally.
As far as I'm informed every Pixel version so far had battery degradation issues. After year or so the battery seems to have only a fraction of the previous capacity.
I am reminded of this funny Feynman quote (paraphrasing): "The whole purpose of the existence of the Mensa club was to decide who else was eligible to join the club". Now, in all fairness, this is true of all the big tech companies to varying degrees, but boy does Google Cloud in particular take the cake on this one!
Of course, Big Tech does produce very good products, and at scale, but that is being built by about 10% of their engineers. The rest of them work there just to keep the interview loop as exclusive as possible. :-) After all, otherwise, one of them would have actually solved long runningcustomer problems.
Plus the rest of the world is now coming to an agreement that a lot of Big Tech's growth is coming from a lot of unfair advantages - such as massive data collection before it was understood as a bad thing, ability to lobby local politicians at a much bigger scale, ability to influence regulations that hurt them somewhat but entirely kill the competition (GDPR for example), complete lack of accountability for past malpractices (e.g. Facebook's friendly fraud case, for which no one went to jail) because they are now too powerful to actually be sent to jail.
I feel like this was super true in the pre Facebook days but they were humbled A LOT by the massive flop of Google plus.
I honestly don't feel like I get the my s * doesn't stink vibe from them anymore.
Especially because everyone knows their Achilles heel now.
But for their ad revenue what would they be?
Everything else material from came from an acquisition or is infra stuff.
It's cool but they also had to learn the hard lesson of Hadoop too.
Yeah they built big table... But the rest of the industry standardized on Hadoop (at the time) because they never released an implementation and so they started losing out on good hires because they didn't want to get locked into proprietary Google infra.
I think you see them recognizing this with intiatives like all of the stuff around kubernetes.
But hey I wouldn't build anything on their (non open) stuff and get more scared by the day over my dependence on El Goog because they've shown time an again how little they care for us common plebs.
But I feel like they get better when they lose but they've only started losing more recently
(Rant over)
Hadoop and Bigtable are not the same thing. Bigtable is a noSQL database. Hadoop is a big data processing framework. Hadoop is actually an open source implementation of MapReduce, which was developed at Google and which has since been replaced by Flume
Hbase was open-source implementation of Bigtable while HDFS was a copy of Google File System. The divergence appeared with other processing frameworks like Spark but now there's Apache Beam to unify it all.
I do believe leadership is failing Google, and their lack of vision is starting to affect the company. There was quite a bit of inertia, but I fear search might drop in the next 5 years and there will be nothing ready to replace ads.
I had a phone conversation with a hiring manager once, and obviously I was not the right type or class or caste or something, but it wasn't a technical screen at all, so it's a mystery to me forever what exactly determined the "in group" as you put it.
As such, I know I'm biased towards them, but I can't really assess the overall company.
You talk as if people from FANG (your invented acronym to exclude Amazon) look down on all others, but it seems that you are the one looking down on all others, everyone who wasn’t lucky enough to get into G/FB straight out of college while also building up not just 1, but 2 or 300k or net worth.
These companies you aspire to do behavioral interviews, and I’m guessing you’re failing them.
Take a step back. You won’t “fail” life because you “only” got a job at one FAANG and not another, or because your net worth is only 100k straight out of college....even the sentence I just wrote sounds completely ridiculous.
Your problems are stemming from how you view yourself and the world, which is full of false assumptions. A lot of smart people on HN have told you this —- you should listen to them.
Don't know why people say this. I'm usually pretty good at behavioral interviews. The problem is the algorithmic interviews (again, this is where my IQ fixation comes from).
IDK man, compared to most everyone else I seem to have failed life at this point given that I have zero accomplishments and little financial security.
It sounds to me like you have anxiety. I'm saying this because I have anxiety, so I know what it's like. One hallmark of anxiety disorders is that you think things are true about the world, even though you don't actually have evidence this is the case. Or, you fixate on some pieces of information while ignoring evidence to the contrary, which again is irrational thinking. Or you hold assumptions about how the world works that aren't proven. You should look into cognitive behavioral therapy as a potential solution.
You say "compared to everyone else". But, you have to acknowledge that the net worth and income numbers you named put you into the 99th percentile, right? So, you actually mean "compared to a very small subset of people I have failed", right? And this is just a completely irrational argument. I mean according to this vein of thinking, then since I don't have as much money as Jeff Bezos, I've failed. And the implication then is the whole world has failed. Right?
> "compared to a very small subset of people I have failed"
Compared to nearly everyone at elite universities I have absolutely failed. I'm sure some of them will become the Jeff Bezos of 2050 as well - it's irrelevant that it's still a small number. I don't want to be compared against somebody that works at Cisco or IBM for instance for the work and mental anguish I've put in.
You also have an implicit assumption that people who work at Cisco or IBM haven’t put in work or mental anguish, and you’re wrong. I know many people who worked and studied hard to get jobs at these places.
You’re artificially restricting the set of successful outcomes so that you can say you’ve failed because you don’t fall into that arbitrary set. Not to mention that your entire definition of success as defined by things like money and perks is wrong.
[1] https://www.google.com/amp/s/www.cnbc.com/amp/2018/09/14/sta...
Please realize how offensive and insulting this is.
You were hired by one of the most successful tech companies, which most people could never aspire to. You claim that Amazon pays you 80% of what (you assume) Facebook or Google would pay you, which puts you financially ahead of the vast majority of people. By calling yourself a failure, you’re calling nearly everyone on Earth an even bigger failure, which is offensive and makes you look extremely entitled and detached from reality.
> Don’t know why people say this.
It’s because you come off in your posts as entitled, obsessive, elitist, insensitive, and bitter. These are the only sideS of yourself that you convey here, so that’s why people assume you’re failing your behavioral interviews.
Sorry again if I’m being harsh, but you’ve been posting the same stuff here for months, and every discussion gets derailed by people consoling you or advising you, which you invariably reject. Let’s stop going in circles.
Let me repeat: that is thousands of millions of people who can barely even conceptualize the life you're able to live.
On another note, I’m in my mid 40’s spent all of my time bouncing between yet another CRUD job most of my career (not complaining, it’s paid for a decent lifestyle in my relatively low cost of living area) and I’m always encouraging fresh CS grads to go for broke and take the r/cscareerquestions route of “grind leetCode and work for a FAANG”, knowing they will make more fresh out of college than I did until a few months ago.
Someone else’s success is not my failure.
However, as a University lecturer I know a lot of people Google have employed. In my opinion, they have rejected some of the greatest students I have ever taught, and accepted some idiots who know how to speak well.
While they have employed some good people, I believe they purposefully taret the type of people who think working at Google makes you a fundamentally better person than anyone else, rather than the best programmers/researchers/AI/whatever.
There's no question that Google is happy to have false-negatives in interviews.
> I believe they purposefully taret the type of people who think working at Google makes you a fundamentally better person than anyone else
why would they do this? And, if they do this, how are they generally speaking so successful? (this is also an amusing comment because the only group I find more critical of Google than HN is Googlers)
The usual recipe for success in tech: competitors who are even less competent.
See also: how Microsoft dominated personal computing back in the day.
Again I think this whole line of thinking is nonsense, but even if you take it at face value, it still doesn't make sense.
By making illegal deals with PC sellers, and by vandalizing apps running on their OS.
They have almost complete monopoly on online search, web ads, online video, on phone OS, on browsers. And they are not afraid to abuse those to get more, and are getting away with this.
You don't need to do anything right when you rent-seek most of the online world.
https://en.wikipedia.org/wiki/Rent-seeking
'Rent seeking' doesn't refer to the same kind of 'rent', of course, but I think the analogy holds up pretty well. All Google has to do is turn a few screws, and the rest of us will have no choice but to sing whatever tune they call.
When a company has a lot of money coming in, they can be successful despite decision X, rather than because of decision X.
Microsoft can interview people asking about filling airliners with golf balls. Valve can just not bother with Half-Life 3. Google can have zero support and a self-driving car division that keeps avoiding chances to release anything.
That the companies are successful doesn't mean all their actions are smart - sometimes it's that their successes are big enough the occasional bad decision doesn't hurt them.
Also, even if I am right, real-world competence may involve more bluffing, less high quality knowledge of algorithms, than I would like.
I’ve met plenty of “smart people” who couldn’t get things done to save their lives partially because they couldn’t communicate well or play well with others.
This has been a source of angst for friends of mine at google for over 15 years, which is almost 3/4 of google’s existence. The hiring process is just random.
Part of it is due to measures put in to avoid certain unconscious biases (hire your friends, regardless of how good they are; hire only people like yourself, etc). So I have some sympathy.
But only some.
This was prompted by me noticing that most of the people who still had their badges visible after work appeared to have Google badges.
- sales
- finance (explains itself really)
- SRE (mostly because of favourable time zones)
Most of the core engineering is in London/Zurich. Dunno what's gonna happen post Brexit, but anecdotally, the FAANG which I am most familiar with has had real problems hiring in London post-2016.
Buy things from Google, though? You mean, give Google money and in exchange expect them to adhere to some kind of standard of behavior, support, and customer service? Go get a coffee because you're clearly not awake yet.
I do believe that there is a level of spending where you can actually get a hold of someone competent who can look into things. But that level appears to be rather high.
After a year, HP dumped GSuite, and went to O365 instead. We got a responsive TAM, full multi-tier support, and white glove handholding for migration. And, I think it was cheaper, too.
There is NO level of spending as a customer at which Google actually gives a shit.
https://en.wikipedia.org/wiki/Category:Discontinued_Google_s...
For example, emacs lisp suffers from bitrot. Some core things haven’t changed in forever but also you can’t pick up elisp config from 15 years ago and it’s fine today.
All live systems evolve and change. In this case emacs is just so slow that Steve thinks that it doesn’t.
But, as you said, he is a good story teller. Like Taleb. Getting the moral across is more important, even if sometimes the wolf eats little red riding hood in malformed renderings.
[1]: https://notgpt.com/2020/08/16/long-lived-systems/
GCP will fail, I think everyone knows it, and even if they don't, they fear it.
This isn’t true, of course. They supported managed Kafka before AWS did: https://cloud.google.com/confluent
If you have an issue it’s not GCP support or GCP’s eng job to fix the cluster. There’s real service uptime implications of this.
Though our engg. team is on it, Pointless hours of productivity lost :-(
"But that's sooooo muuuuch too teeessstt!! Whaaaaaaaaaaaaaaaa!!!!!!!!!!"
But that's your job dude. So what if it costs more money and time. Fucking pay for it anyway.
I feel like startup software dev needs a swift kick in the ass or three with that phrase.... "do it anyway!"
Who fucking cares if these founder assholes only make an 800% return instead of 900%? Do it right.
It's Google who doesn't provide backwards compatibility, upgrade paths and migration options.
> But that's your job dude. So what if it costs more money and time. Fucking pay for it anyway.
So why doesn't Google pay for it anyway?
> I feel like startup software dev needs a swift kick in the ass or three with that phrase.... "do it anyway!"
You mean Google devs.
I've been on the web since the mid 90s. The last 15 years have had more cool things taken away from me because of sunset bullshit than the first half of that time period. Software (even web apps) should be eternal, there should never be "features removed" or "EOL" announcements. If they don't want to run something they should open source it and let the community take over. What the hell is the point of hoarding all this wonderful IP if it serves noone?
You aren't a customer, you're a user who paid to bypass a rate limit. There is a difference.
I wonder when/if the software industry will ever stabilise. It's possibly the only industry where frequent and disruptive change continues to happen, and is even welcomed by many in it (not me).
> I’ll also give a shout-out to our friends in the Operating Systems business: Windows, Linux, NOT APPLE FUCK YOU APPLE, FreeBSD, and so on, for doing such a great job of backwards compatibility on their successful platforms.
I share your desire for stability. I don't necessarily mind frequent updates for security reasons, but I'm getting to the point where if things are changing in big ways often, I'm just not going to use it --- I no longer find enjoyment in running on the upgrade treadmill, it's just work.
That said, I don't see a lot of hope for our outlook. Most people don't seem to have a concept of completable software. It would be nice if it were bug free, but getting to the point where the cost to fix bugs compares to the cost of leaving bugs as known issues is doable.
This should be generally true of any well-compiled static binary on linux from 20 years ago.
No way! IBM mainframe (zSeries) is the gold standard for backward compatibility in IT.
Your Win32 binaries sound better than whatever software is being distributed by the companies today.
See also my other comment[1] - Angular is an example of such a magical does-everything framework.
[1] https://news.ycombinator.com/item?id=24168279
Angular is the only one that has remained even remotely usable, but certainly not stable. It's never as simple as running the upgrade utilities, changing the version, and being done. It ALWAYS takes at least a day to find all the "little" things they didn't feel fit to mention in the upgrade docs.
It's aggravating. I genuinely don't like using Google software, at all.
I've felt for a long time that Google is coasting on the momentum of the early web and the awe people felt for what they built early on. That hasn't been Google for a long, long time.
I really like Node as a language (especially with TS), but Node as an ecosystem feels like a very hard fit into 90%+ of corporations.
Consequently, JavaScript developers got used to the burden of maintenance and rapidly updating to the latest versions of libraries, and this culture carried through to Node (partly because a lot of libraries are shared between node and the web).
Having said that, if you pick your libraries well, it's not too bad these days. When I upgraded from Node 12 to Node 14 earlier this year, I had to upgrade the `pg` package to a newer version that supported Node 14 (there was one available), but I didn't have to make any code changes. And other than that I've had no forced version upgrades in a long time.
I guess if you're lookig at 5-10 year timescales with literaly no maintenance then this would be a different matter though.
This is true for any language/platform.
With Clojure, I can think of 2 times ever when a dependency caused an issue. It was extremely obvious since the issue was "won't compile", and the fixes were simple.
With PHP, I expect any change to potentially break something. Bump your AWS SDK which uses a different minor version of guzzle? Fatal error.
There's a world of difference between breakage being an everyday thing and a true rarity.
The short answer: if you avoid the shiny tools that claim to do everything, you will not have this problem. And as a special case, avoid Gulp, which is mismanaged.
The single-responsibility libraries that have a well-defined scope, on the other hand, have often been sitting on npm unchanged for 5-6 years, because they are simply done, and they do what they need to. They will very likely never deprecate anything, as there is simply nothing to change.
I would argue that these libraries are actually doing a better job of stability than their counterparts in other languages.
Edit: An additional factor is that it's easy to write a tutorial about an extensive framework with a large scope; there's plenty of stuff to write about. Writing a tutorial about "this is how you make a function call to this library to parse a geo URI", on the other hand, probably isn't going to happen.
So if you follow third-party tutorials, you are naturally going to end up at the packages that are most prone to deprecations.
The list goes on and on.
And yes. Despite these being minor version updates, stuff will break if you go from, say, version 2.9 to version 2.11.
I can tell you that virtually everything I’ve used, from networking (legacy to VPC) to storage (Cloud SQL v1 to v2) to Firebase (now Firestore with a totally different API) to App Engine (don’t even get me started) to Cloud Endpoints to... I dunno, _everything_, has forced me to rewrite it all after at most 2–3 years.
People assumed that because we made a new database we were killing the older one, but that was never true.
But also idk what else you'd want them to say "the thing you're taking shit isn't deprecated" seems like an important response to "you mishandled this deprecation".
Personally, I think most of them seem quite sensible. Having a 'support everything forever' approach is obviously going to impose a huge burden on the teams who maintain this stuff which is then going to limit the ability to make anything better. The depreciation notices generally seem pretty good (12-15 months notice by the looks of it, sometimes followed by degraded functionality rather than complete removal of the feature).
And yet AWS is doing just that while innovating at the same time.
[1] "Depreciate" is something else.
Come to think of it, the Channels API shut down was a bit of a nuisance. But it never worked all that well, and deprecation seemed like a reasonable move to me.
Here's a detailed list[1].
[1]: https://cloud.google.com/appengine/docs/deprecations
There's a reason that people trust Amazon with their compute, and that's because in regards to their technology they're trustworthy.
Example: https://cloud.google.com/stackdriver/docs/deprecations
I've had similar experiences with broken Bitnami images that makes it seem like they aren't even tested before being released [1][2]. Their WordPress images do work, but the way file permissions are configured doesn't play well with the way they have WordPress set up [3]. Have I just had bad luck, or is this a common experience?
[1] https://community.bitnami.com/t/cant-authenticate-to-fauxton... [2] https://community.bitnami.com/t/couchdb-server-log-gives-wro... [3] https://github.com/WP2Static/wp2static/issues/598
Can't get too upset, we don't pay them anything. Not in a rush to use them again though.
EDIT Disclaimer: I work for Bitnami; not in that team though. I reached out internally to see if this is a known issue, but it's Saturday...
EDIT 2: does the downvote mean bitnami community support is so bad or did my wording offend anybody?
That said, my take (and that of siblings) is that I'm much better off building my own containers than serving as unpaid Q&A for Bitnami. It would be smart to use containers built by people more expert than I am. Containers that are broken by default do not meet that criteria. So, "reach out to community support" is not helpful to people that have already decided to ditch Bitnami.
> We're careful to pin versions, and hadn't anticipated someone would re-write history and break published code.
This is exactly the kind of thing that will make me ditch a company forever and never look back.
We use immutable tags like "4.2.8-debian-10-r50" which we never overwrite.
Then we have "semantic versioned" moving tags, like "4.2.8", "4.2", ... which resolve to the latest image matching that prefix.
Moving tags are what they are; I'm personally not a fan; all major popular images have that (https://hub.docker.com/_/golang, https://hub.docker.com/_/python, ...)
You can read more about Bitnami tagging scheme at: https://docs.bitnami.com/tutorials/understand-rolling-tags-c...
If you want to pin your image you should either use the most specific tag or just use the digest:
Now, we might have introduced a regression between some version of a container and the next minor version; shit happens; it's hard to tell without more specific information though.> > We're careful to pin versions, and hadn't anticipated someone would re-write history and break published code.
> This is exactly the kind of thing that will make me ditch a company forever and never look back.
On the other hand, assuming that instead of a misunderstanding, what you saw is actually the image behind an immutable tag such as 4.2.8-debian-10-r50 being replaced, this is a serious security issue; somebody could have hacked docker hub, or crafted a valid certificate for docker hub and MitM'd you,
I'd also ditch a company forever and never look back if they honestly don't care about that problem, which I assure you it's not the case.
We'd greatly appreciate reporting such cases to security@bitnami.com / security@vmware.com .
In fact at this point it's easier to use AWS Lightsail.
Most NPM packages.
> Most NPM packages
NPM packages tend to be free and open source so morally they don’t owe us anything. So yes what is important is the time of the authors of the packages. I am grateful someone took the trouble to release them and many popular ones are actually well maintained.
But cloud services that you are paying for are a totally different ball game. The company does need to maintain backwards compatibility for a good amount time and they need to value your time.
Other ecosystems (like Python) don't have the same culture as NPM. There are clean, well-documented, maintained packages.
Google doesn't have a moral obligation to support GCE customers either. It's just that everyone I know who has used GCE would never use it in production, and loudly advises everyone they know not to do so either. They'll be bleeding money at some point, not to mention they're already bleeding reputation. Google went from smartest-people-in-the-room to smug-incompetent-arrogant-douchebags. That's hard to get over. That's about good business, not some kind of moral obligation.
Almost every day there is an automated PR to bump one of the thousands of dependencies. LTS is ending for Angular 8, but one of the main component dependencies doesn't support 9 yet. I had to put typescript in an unsupported state so one of the dependency upgrades wouldn't break something -- the solution is an ugly hack not supported by anyone. Another required a bump to lodash, which suddenly started failing. This led me to the following comment in a github thread:
> The recursive implementation of cloneDeep was by design with the shortcoming that in some rare cases it could run into stack size exceeded issues. I'm not how cloneDeep is being used but chances are a more specialized clone tailored to their scenario will be a better option.
So, I agree with the OP: the messaging for support in the npm world can be distilled to "fuck you".
If I had ever proposed to a customer that we would build infrastructure for them written by thousands of pseudo-anonymous entities who may or may not break everything because they felt like it, I would have correctly been fired.
This leads me to believe that supportability, reliability, and security aren't important to the npm ecosystem because ultimately the products built with npm don't matter. A bank or a utility company may use them to build a customer facing application, but for the stuff that's important, a competent technology manager would never let npm through the door.
(edit: I replaced "tools" with "software" in the first paragraph, because tools don't change interfaces and effects on a whim).
On a side note I think this is what all of us should be aiming for. Every time someone on a team I'm working for starts trying to use some very complex tool that a SaaS/cloud has built for us I always urged them to look for a simpler solution.
Do you really need AWS Lambda/kinesis/etc or can you write your code and deploy it onto our cluster as a normal service/api/control loop?
If you make it really easy to write code, deploy it, and monitor it, you'll often find it's way less devops time to write your software as apis/control loops than it is to use a large amalgamation of cloud services. I also haven't found a good way to unit test cloud services and their interlinking.
I want full in vendor lock-in, it saves an immense amount of time.
You just need to pick a vendor that gives a shit about it's customers to be locked in to - and that's simply not google.
However you can take steps that even when you give in to lock in, you structure your code/projects so that switching to another standard is straightforward: include non-vendor build scripts that would work on any Docker/VPS/baremetal server, isolate the locked-in config and scripts from the rest of the app as much as possible (even if it means running separate external scripts to execute them), etc.
Once you've done it a couple times you build up a suite of small tools and it gets easier.
I got bit hard by Zeit/Vercel, and then by GCP, and then by AWS. All of them do one thing or another to manage to find a way to extract more money. My goal is to make it so that if one day I decide we simply need to leave the platform, we can at a minimum of effort.
Because this actually does happen 1-2 times a year, thanks to exactly the issues raised by OP, it's paid dividends to put in the small time it takes to keep things mostly agnostic on the code side.
I mean, there's stuff like Python 2 support being sunset on managed services, but as Yegge himself points out that's on Guido, not GCP.
Are they porting to the AppEngine Python 3 APIs? Hell no: those are totally useless (no BigTable, no user authentication, etc.). If you have to rewrite everything to replace all the functionality that used to be "batteries included", you might as well build something can run on any old bare VM instead, and so—like any sensible external developer—my colleagues are doing everything they can to avoid depending on any Google-specific APIs, libraries or platforms.
I really though AppEngine and PaaS were the future, but evidently that future will not be written at Google.
That's hilarious(ly bad): BigTable was like the main selling point of OG AppEngine, and they practically forced their user authentication scheme on you too.
Which in practice meant that "migrating to a new database under a different account" translated into "rewriting half a codebase due to a cascade of breaking changes in the surrounding tooling".
> > I know I haven’t gone into a lot of specific details about GCP’s deprecations. I can tell you that virtually everything I’ve used, from networking (legacy to VPC) to storage (Cloud SQL v1 to v2) to Firebase (now Firestore with a totally different API) to App Engine (don’t even get me started) to Cloud Endpoints to… I dunno, everything, has forced me to rewrite it all after at most 2–3 years, and they never automate it for you, and often there is no documented migration path at all. It’s just crickets.
it actually winds up being less DevOps work, on average, to support open-source systems running on bare VMs, than to try to keep up with Google’s deprecation treadmill.
For context, I tend to get dumped with undocumented poorly written untested DS code bases every time I move to a new job.
The difference between setups where there's a bare VM/server and it runs using standard tools (command line scripts) or at least some open-source orchestration and any kind of proprietary technology tied to infrastructure is massive.
Additionally, the server stuff can be maintained and debugged by a far larger number of people, thus reducing bus factor.
Like, maybe in the web dev world there are things that everyone knows that are AWS services (maybe S3 counts, I guess) but in the DS world, I just know that I'm in for a whole lot more pain if they haven't just used bare servers.
Keeping your stuff up and running, patched with the latest security updates takes time. And opensource systems also break compatibility at times. Sometimes the installation/upgrade process changes over time and you need to learn how to do proper upgrades even if the software technically can be made to work with the rest of the system.
It's hard to quantify, but I don't think it's so clear-cut as Yegge makes it look.
sure FOSS needs to be upgraded too, and sometimes there is change that you need to adapt to, but you are in control of the schedule, and if you are busy you can delay the upgrade work to when you have time for it. a service being shut down won't give you that flexibility
Google gives you 12 months notice before turning off the lights.
A FOSS tool "forces" you to upgrade when the old version you're using has a serious security issue that's is not backported. Which means that unless the supported version is fully backward compatible with your software, you have a quite tight schedule to deal with.
Sure, you always have the "option" of running insecure systems. Unfortunately a lot of people choose that "option" and severely underestimate its cost...
an upgrade with some incompatible changes is still a world of difference compared to potentially being forced to rewrite a whole bunch of code because the system you used is completely discontinued and the alternative that replaces it isn't even remotely similar
But map() (in either language) has the semantics of applying a transformation to each item in a collection to produce a new collection, so you wouldn’t really want to use map() to call something like doStuff(). That's what normal loops and forEach() are for. If you aren’t transforming the collection into a new collection, map() is the wrong choice; you’re telling anybody reading it that you’re doing something that you aren’t.
But assuming doStuff() just has a bad name and is actually transformational, then the most direct translation would be to use map() and a lambda:
However it’s more idiomatic to use a list comprehension:Yes. The combination of a botched transition with a high arrogance level was hell. Here's something I wrote in 2015 after porting a medium-sized system from Python 2 to Python 3.[1] I listed a number of package problems encountered during the conversion. Ones that indicated those libraries weren't being used much in production. I discovered that the SQL connector broke if you loaded a database of tens of thousands of records, for example. And oh, did I get hate mail. You can see the comments below mine.
That's what led me to Go. Go is a mediocre language. That's a strength when you just need to get something done. It comes with a set of libraries which are heavily used internally within Google. So, if you're doing something that is server-side for some web-related task, the libraries for that are probably both present and well-debugged. And they don't seem to be deprecated rapidly. I'm not using them for any Google-specific services, so I can't speak to that.
[1] https://lists.archive.carbon60.com/python/python/1187081/?pa...
> In the Emacs world [...], when they make an API obsolete, they are basically saying: “You really shouldn’t use this approach, because even though it works, it suffers from various deficiencies which we enumerate here. But in the end it’s your call.”
> Whereas in the Google world, deprecation means: “We are breaking our commitments to you.” [...] It means they are going to force you to do some work, possibly a large amount of rework, on a regular basis
It's fine if they want to go hunting for their Perfect API v10™, but just leave the old APIs alone - build abstractions under them if needed. Don't push that work onto your customers.
Damn, I was waiting for this. The whole essay was a setup for this paragraph. Reframes the argument using a shared traumatic experience for everyone. This humanizes the effect, we all know someone that lost someone to the Python2/3 transition. I can't count the number of Python friends that turned into Gophers, joining some cult of middle class bearded tech dad with a side business distilling pear brandy.
Best Yegge Essay Yet! Welcome back Steve.
edit, finished the essay. He smoked GCP in this one, damn damn hard. Hickory.
Of course the language wasn’t as broadly used back then and Apple is known for quicker churn than Python so the need to write code compatible across Swift versions was much less pronounced
The back-compat promise is one of the reasons I'm investing in Rust for new projects, even if I find the actual programming much more pleasant in Swift.
But I'm a bit less optimistic about Rust now given the recent layoffs at Mozilla.
I'm a bit surprised that language support for such a key concurrency feature comes so late in the game.
It's not the same issue that Yegge describes though.
Async/await does not need to be a breaking change - C# managed fine without it being so. The breaking changes in Rust were around the standardisation of std::futures::Future rather than async and await.
That said async/await makes a world of difference to the usability of Rust for async code and I’d imagine it will for Swift too.
The problem isn’t making an incompatible “v2”. It’s getting rid of “v1”.
If I can’t take advantage of new functionality without moving to a new version that’s a headache. But it doesn’t cause me to throw away all of my existing investment. Especially if you can combine the old and the new.
Flutter and React Native both work on iOS. And obviously Objective C still works
As an example core of Unity is written with C++ with Objective-C bindings for APIs such as Metal.
If Apple would mandate Swift then things like Unity would just cross compile into swift, causing perf regressions for users, all for no benefit for Apple.
This feels like a very strange perspective. Why is it a problem to "lose" someone from a language? You're an engineer (a presumption on my side); casting yourself into a subsection of a "Python engineer" seems like a move with all the downsides and no upsides.
I dread the day Python will be as bloated as C++ or Java, neither that dare to remove things.
Arguably worth it. Dividing 1 by 2 with "1 / 2" and getting 0 is a real 'wat' for novice programmers. And the workaround is a 'wat' for experts: instead of using a floating point division operator, cast one of your parameters to float.
Joking mostly aside, python is way way closer to C++ in terms of language feature bloat; where Java is notoriously spartan and slow moving (moreso historically).
Ofcourse I know you're also reffering to standard lib gunk but that's relatively less impactful.
The big issue with the python 3 shift was when you touch something as substantial, subtle, and pervasive as strings you're going to cause problems. I wont speak to if it was worth it.
Also the python 2 string type was more in line with how other unix utilities work, so while I do think the 3 string type is generally better for the web, it wasn't an obvious improvement for the other large usage of python. While it's a very versatile language, I would say the major niches of python are: web dev, shell scripting, and scientific computing. The changes in python 3 were (somewhat) helpful for web dev, but really were inconsequential and/or actively harmful for the other uses.
Mixed with how condescending the python dev team acted about it, and how it really wasn't until about python 3.6 that there was any compelling reason to move to python 3,you can see where most of the hate comes from.
And by the way, even if Python is very popular today, I do think that rift really hurt the language. I worked at least at one company where Python was considered for a project, but eventually shelved because of management's confusion over the 2/3 issue.
> The "print" statement being a great example. Was it kind of weird? Sure.
I am reminded of the Emerson quip "A foolish consistency is the hobgoblin of little minds". The Py2 print statement is special/weird because it is a very important and special behavior, especially for newcomers. It's not randomly special, it's special because it models something that deserves special treatment.
Py3 is more consistent, but less humane. I use it over Py2 only because the market has moved on, but every time I type print with parentheses I curse Guido.
Can you expand on this, why is this more humane? And why should print be a special function?
Larry Wall of Perl thought about these things in terms of Huffman Coding, i.e. the most common thing you might want to write should have the syntax elide any easily understood context.
In the case of "print", we know we're not talking about any random function here: it's the function for getting output to the terminal, one of the most fundamental operations in all of programming. So I think it's totally fine for it to have a special representation in the language.
One nit I would call-out is that most people don't put a space between the print and the ("hello world"), so idiomatically when learning, print is 'just another function'. And since HelloWorld is usually the first thing people learn, I wouldn't want them to think print is different than any other function.
Also, the first line of The Zen of Python is "Beautiful is better than ugly", I can understand why consistency was a priority here.
We could get into a whole other discussion about whether or not a programming language should have and adhere to a "culture", but fundamentally I agree with the arguments against Python making this change. It's just not worth the effort, and like others have said, they might've lost of a lot of momentum to JS.
That the transition took nearly a decade (and for some still goes at this point) and several minor-versions is another hint at strong problems in planing. At the end we got something better, but the road toward it was just far to painful for what we got. And this fuels hate till today.
Yet Python 2 to 3 started in 2008 - 12 years ago, and is still not “done” in any meaningful sense.
If I think back to 12 years after the intel OSX transition - 2018 - I can’t think of a single outstanding issue, whereas I bash into python 2 vs 3 literally every time I touch it (and therefore mostly choose not to).
Maybe GCP should take a hard look at their internal incentives?
Anyone understand the technical jargon here? I'm a bit lost.
> Bigtable maintains data in lexicographic order by row key. The row range for a table is dynamically partitioned. Each row range is called a tablet, which is the unit of dis- tribution and load balancing.
And later...
>The master is responsible for assigning tablets to tablet servers, detecting the addition and expiration of tablet servers, balancing tablet-server load, and garbage col- lection of files in GFS. In addition, it handles schema changes such as table and column family creations. Each tablet server manages a set of tablets (typically we have somewhere between ten to a thousand tablets per tablet server). The tablet server handles read and write requests to the tablets that it has loaded, and also splits tablets that have grown too large.
The article was worth a read for that quote alone.
The reason is the engineering culture and the promotion system. People expect to switch to new projects, and work towards to a promotion within 2-3 years. This shows in all products, and presumalby also the reason for the deprecation. New people come on the project, want and need to do a major overhaul, and do not have resource or incentives to support the old stuff.
The other half is that most projects are done efficiently by relatively small groups of engineers. I remember that Android Market was still a few people in MTV, with no payment possibilities outside of the US, when Apple already had significant revenue from apps. And that shows everywhere. I'd argue Google is incapable of tackling projects that require more than a few hundred people for more than 2-3 years.
But as a billion dollar company, that does not work. Throwing money at obvious new billion dollar opportunities like Microsoft did with gaming and cloud, and following through over 5 years is not possible for Google.
Looks like someone deserves a bonus
You need the people evaluating performance to really understand the work, and make informed, discretionary decisions about who to promote to maximize real value creation.
I agree but for everyone working at / running a company without a money printing machine it is worth noting that this is really expensive. You need to take someone that’s a good engineer—-one with judgment and a modicum of people skills—-and give them mostly thankless, stressful work.
If you start rewarding maintaining old things you aren't going to suddenly get what you want, you're just going to lose a tonne of people because they don't want to do what you're telling them and you won't be able to hire good people with your priorities because no one is going to believe they can have a good career at Google by prioritizing stability and long term support. It's all downstream of culture, and culture change is very difficult.
Of course if you do buy into this idea that this is the issue, then GCP has already lost. It would be years before they could make a dent in that culture, and more years after that for external customers to know and trust Google's new culture. By that time there's going to be a dominant player anyway. So maybe actually it's best to stick with the current culture - try and win on your strengths than try to fix your defficiencies.
Keeping backward-compatibility requires people that care. That have enough pride in their work to counter-balance the grind.
It might not be a good business decision for Google, but surely if they create enough positions that are solely focussed on the boring stuff but are paid better to make up for it, there is a price point at which they'd get enough interested people.
MS pay is actually competitive for external hires, although internal raises aren't usually very high.
This is true for most people working for large multinational corporations. Good things can still be developed without passion.
There's nothing wrong with that, and when I'm at work, I do my best to do a good job and make reliable, maintainable systems. Those two things aren't in conflict.
Isn't that inherent to their interview process?
I don't follow. Wouldn't ridding Google of its reputation for inadequate maintenance, make it a more prestigious employer? Its reputation for paying well wouldn't change. How would this lead to an exodus?
Obviously generalizing here, but most engineers (especially fresh graduates) would rather work on a new thing than do maintenance, if given the choice. Google offers a pretty good value proposition: Lots of money and you get to work at new exiting projects and if you don‘t like it you can change to something you like. There‘s obviously more nuance to this, but that seems like a pretty good deal for many people.
I think that's only part of it though. I think that the only way a company can let products wither for years without a single new feature or improvement is if they don't have a product manager to represent the users, and bring a vision to the engineering team.
This is the same reason why it is near impossible to reform a police department. It would take firing everyone and starting over.