No, they won't run out of people to hire but the cost to hire them will go up. At a certain point, they may have to pay more and offer more perks to attract the level of talent they want. The culture will mellow once the payroll expense starts climbing.
I was at Amazon for four years and enjoyed my time there. I had one stint on a high profile high-pressure team that lasted about 6 months (no on-call, though). That was the busiest time I had at Amazon but it was kinda known going into that project.
I generally worked 45-50 hour weeks, with the occasional 50-60 hour weeks but that was pretty rare.
It's a big company. It really depends a lot on your manager, your team, and your org.
This is what I don't understand. When Tbray left, he said he was doing so in solidarity with the distribution center types, but he held to the line that AWS was great. This despite nightmare stories like this one regularly bubbling up, and I've heard similar from friends.
What is it about tech culture that people within it refuse to admit the level of abuse-by-design that happens at these companies? Is it that they want to believe that they are somehow fundamentally different from the distribution and grocery store workers, and fundamentally similar to Bezos? I can't help but feel it's just this kind of false consciousness that prevents developers from _getting organized_.
Sidenote, payroll tax on Amazon in Seattle trying to get revived. It's a long and winding story. Check out taxamazon.net
There is something about the human psyche that makes you partial to something that treats you well.
My work treats me exceptionally well. It does not treat the sales center employees as well. I find it hard to say it's a terrible place to work, because as mentioned earlier, it takes tremendous care of me, and is the reason I am able to have the life I have, the house I have, and support the family I have in the way I can.
It's ok to want to continue to be treated great, and also have them treat others better.
Payroll taxes are always borne by the worker. They may not make the payments, but when I hire people, I set aside an all-in budget, and their salary is whatever is left after benefits and taxes. If the payroll tax goes up, it means their raise is smaller.
That's how every business works. All per employee costs are borne by the employee one way or another.
This is only true in the sense that there is a cost to move goods, but it is not true in the sense that retailers charge more to move the good to you. If it were, I wouldn't be able to reliably purchase goods shipped to my door for equal or less than what I pay in physical stores. When shipping is included in the price, that means that the shipper bears the cost of shipping by taking a smaller margin. If I were to bear the cost, it would appear as an explicit cost or higher prices, neither of which are happening.
By the same argument, customers are paying the costs for Best Buy's internal logistics and retail floor operations to enable “free browse and instant pickup”. If those costs are higher than Amazon/NewEgg's costs for "free shipping", I'd expect to see exactly the relative prices that you're seeing.
I'm sorry, but I can't track the point you are trying to make.
I'll reiterate my point that shipping is a cost that has to be paid, but who bears that cost is not necessarily the customer. If it was the case that the customer was the one bearing that cost, there would be a difference in the price you pay for goods at a physical store vs. goods shipped to your home, implicitly or explicitly. There is generally no difference, and in many cases, ship to home is cheaper because you can select from more retailers.
This is not evidence that payroll taxes are borne solely by the worker. I'm not claiming that workers don't bear any of the burden. Just that they don't bear all of the burden. this would be especially true in an asymmetrical payroll tax system in competitive labor markets like the Seattle Software Developer market.
I'm not sure how a competitive labor market makes any difference. They all have to pay the same payroll taxes. If anything it's a great equalizer -- all companies must bear that employee cost. So they will all take it out of their employee's earnings equally.
In this context, we are talking about the Seattle Head tax, which is NOT applied equally.
In the larger discussion of payroll taxes, to disprove that workers bear the entire burden of payroll taxes, just go through the thought experiment where we eliminate the employer side of the SS tax and whether wages would rise 6.2%. They wouldn't. Workers were willing to work for $N previously, there is no reason that they would stop working/change jobs unless they were paid $N + 6.2%.
Of course salaries wouldn't go up. Because you removed the cost from the equation.
But now do the thought experiment where the employee must explicitly pay the extra 6.2%. Now of course you have to give them a 6.2% raise, because they would demand it to cover their increased costs.
Numbers: I pay the worker $10000 a month. Let's say my tax cost is $500 a month, and their tax cost is $500/mo. So they get a check for $9500, they see a $500 tax deduction, and they net $9000.
My all in cost is $10000.
Now the city says "you don't have to pay your part of the tax anymore, the worker does!"
So now I pay them $10000, they see a deduction of $1000 on their paycheck, and net $9000. My all in cost is still $10000, their net is still $9000. Nothing has changed.
I'm still paying $10000 to have that worker. And if my budget is 100000, I can still only afford 10 workers.
The divide between AWS employees and everyone else employed at Amazon seems to be "2007 Google employee" against "sweatshop"; they're run mostly-separately, from what I've read.
This was also four years ago; the gist is a copy of a post from 2015, by the above reddit poster.
> The divide between AWS employees and everyone else employed at Amazon seems to be "2007 Google employee" against "sweatshop"; they're run mostly-separately, from what I've read.
This is incredibly offensive. I work in CDO and have friends in retail and AWS and the biggest difference day-to-day is that we have different systems to manage our AWS accounts.
Equivocating me to a Wipro employee is honestly really disgusting. These sorts of microaggressions are the worst part of working at Amazon, the WLB is fine.
> The divide between AWS employees and everyone else employed at Amazon seems to be "2007 Google employee" against "sweatshop"; they're run mostly-separately, from what I've read.
This is not true. Amazon and AWS are exactly the same organizationally speaking. You could hardly tell who works in an Amazon team and who works in an AWS team. Maybe if they tell you their building.
The cultural drift happens in the teams and orgs, not the parent "company". Everyone at Amazon knows this and that’s why many people know changing teams is a gamble and prefer staying at the same team doing the same menial tasks, than venturing into finding another team.
Amazon and AWS have both great teams and shitty teams. There’s just not an easy way to tell, except for one internal tool that allows you to see how people are rating their managers but this is all bs because some teams don’t have enough data or any data at all. Also the data itself is probably skewed, since people are afraid of giving poor rates to their managers (this is a general problem in the modern workplace dynamics, not at an Amazon problem)
The other way is asking a friend but I don’t know how people do that since I only knew people from my team and teams that sat nearby. Maybe I’m too socially incompetent.
It’s worth pointing out why it needs to be “revived”: because the first iteration of the tax would have destroyed grocery stores and was protested by construction unions due to its disastrous policy.
This iteration is targeting the 800 largest employers in Seattle, many of whom aren’t currently profitable.
“Tax Amazon” is classic dishonesty — neither bill has been focused on Amazon, both would be a disaster for Seattle, and proponents try to focus on Amazon to keep people from noticing how damaging their out of control spending is.
In 2010, the Seattle budget was 3.8B; in 2020, it’s 6.5B. An increase of 2.7B and growth of 71% in a decade. (Per capita, the Seattle budget is 37% more now than in 2010.)
They increased about .5% inflation adjust per year from 2010-2020. That again does not mean that spending is out of control especially when you consider that 2010 was still in the depths of the great recession.
Must be why all those small businesses that got affected (like the long-running local Bartell Drugs chain [0]) were protesting on the streets against this tax once it was implemented (shortly before it was repealed).
Writing off the opposition as solely "pro-amazon shills" is exactly what gets those bs laws passed in the first place. I wonder how much money was wasted on all of this, given that it was repealed shortly after. Seems to be very in-line with how wastefully Seattle City Council has been running things.
As someone not from Seattle, the drama around that tax was quite ridiculous. The idea of a "head tax" is utterly nonsensical in that it's an incentive to NOT hire people, which is the entire point that companies are pulled into cities. Jobs. Jobs are one of the things cities care most about more than anything. A head tax actively disincentives hiring of new workers. A head tax can maybe make sense if you have permanently low unemployment and companies can't hire enough people, but that's not Seattle's problem.
Given how many people are moving to Seattle every year, far faster than new housing can be built, I'd say that an effectively negative unemployment rate has, in fact, been one of Seattle's problems, and that slowing hiring down would, in fact, help things.
> Jobs are one of the things cities care most about more than anything.
And look where that got cities.
One school of thought is "just keep importing jobs AND add a ton of new housing" but there's a sort of capitalistic dystopian feeling to that, like that's the path to Blade Runner. Just need to start shipping out the people without high-enough paying jobs (again, actually - Europeans moving to the Americas doesn't seem that dissimilar) and we're there!
That's from 2017 when it was introduced but it's a good non-partisan explanation, including the many problems with the proposal. https://sccinsight.com/tag/head-tax/ has newer posts.
Seattle population has grown by around 23% [1] and inflation means $1 in 2010 has the spending power of $1.18 today [2], you'd expect a 45% increase (1.18*1.23) just accounting for population change. According to seattle's open government website, their budget was $6.24Bn in 2020, and $4.13Bn in 2010, a 51% increase. So just ball park spending has grown probably less than 0.5% per annum over inflation in the last decade.
I worked at AWS. Your experience may vary significantly depending on which team you are in, either at AWS or plain-Amazon. Some people are having the time of their lives there. Some are working horrible hours, maintaining crap old code that they hate, and struggling with performance reviews that saps their morale and prevents them from seeking a better team or leaving the company. Some people genuinely like the competitiveness, the politics, and know the Leadership Principles like it's the law.
Amazon is a fascinating company, it's Big Capitalism in a way no other company is today, and as such it's writing History, for better or worse. For the average developer it's not great, though.
Every company is full of nightmares, especially anyone doing something big enough to be interesting. For instance that guy who started blogging about how janky Tesla's software was, most of us who have been around a while read all that and shrugged. Not that bad. It is easy to imagine perfection or competence but very hard to actually achieve it at large scales with humans.
So at the same time it can be true that company X is a great place and also true that company X is a nightmare.
This. Amazon maybe has thousands of developers working in various departments. The departments can be totally different from each other, have different culture, values whatever. Even in very small companies I have seen this effect.
>Is it that they want to believe that they are somehow fundamentally different from the distribution and grocery store workers, and fundamentally similar to Bezos?
I think this might be a big part of it. The idea that "I'm college educated and don't perform physical labor means I'm not like them" seems to be a pervasive societal thought not even strictly related to tech.
It's good to be on top. The reason AWS is so great for those with voices whom we pay attention to, is because of the misery of a whole lot of people below them.
It's not even programmers like OP but also all the servicepeople and staff at datacenters, and the rest of the crew which makes up the infrastructure that makes it work.
> What is it about tech culture that people within it refuse to admit the level of abuse-by-design that happens at these companies?
The Amazon example is extreme, but a lot of tech workers fit the stereotype of people who don't have a lot else to do.
I spend 100+ hours at my computer a week, so if you want long hours, I can live with that.
I am up at 1AM anyway, so if you need something done at that time, sure. I just need to switch to my remote desktop.
I have colleagues and friends who will log in on weekends or evenings out of boredom to do work. No pressure to do it, but they just want something to do.
I fill my time with internet commenting and various online contests, but if they need help at 9PM, I am fine with jumping on the VPN to help and have done that a few times myself.
Amazon still has problems beyond the long hours and the 3AM calls, but there are people ok with those things or at least ok enough with them to not put up a fuss.
> Very sad when you think about it. Wasted potential.
Eh, you leave the university environment of all its clubs, hackathons, etc, likely to another city where you know far fewer people. You need to wake up the next morning, so gaming until 5AM with your friends no longer works. Someone is also on on-call, so they can't join, so you postpose it further.
Maybe it's just the type of people amazon hires? I've been programming for 5+ years. Know what I'm doing today at 5 pm? Headed over to Lake Michigan with the fiance. Yesterday I went on a 4 mile run. Personally I never check things outside of 8-5 and my employer is A-OK with that.
The biggest problem is when people like this article's author get a job and the HR people never make it clear that they may need to have "pager duty" or otherwise work outside of 8-5. Those different expectations is what caused him to have a very bad experience.
Personal attacks and name-calling are bannable offenses here. Would you mind reviewing https://news.ycombinator.com/newsguidelines.html and sticking to the rules when posting to HN? We'd be grateful.
One thing seems to have changed since the OP wrote their post -- managers at AWS are open about the on-call requirement. At least the ones in Portland. They were proud that they owned the product all the way from the code on the back to the customer at the front. So if you didn't want to be responsible for some level of ops, you knew that going in.
I didn't move forward to an interview, this was just from a meet-and-greet they did a few months ago (they do it every month IIRC). I have other reasons for not being at all interested in working for Amazon, but I figured it would be interesting to meet with them informally anyway.
> They were proud that they owned the product all the way from the code on the back to the customer at the front. So if you didn't want to be responsible for some level of ops, you knew that going in.
There's a huge difference between being responsible for (some level of) ops and literally climbing out of bed when the pager rings. At work, I'm responsible for the operations of the services I develop, which means that I'm doing third level support during normal business hours and I write the playbooks that first/second level support executes. You certainly won't catch me working after 5 PM.
They didn't for me - it meant my SO would wake me up after they'd been woken up.
I then started wearing headphones to bed so that I would hear the phone ring and not disturb them, but when you're rolling out of bed and hopping on a call at 3AM it's difficult.
What I seriously underestimated was the strain that being on-call can put on a relationship, it's a massive regret.
I worked for a company where developers could opt in to on-call cover. Doing so meant you got some extra cash, and if you were called then you got so much for the first hour, then a different amount for each hour after that. Not being coy, I can’t remember the details.
I'm a pretty heavy sleeper, and would go to bed with my phone on silent, set to ring a long time before going to answerphone, and with my Apple Watch on my wrist. If my phone rang then my watch would buzz me awake, eventually. The only snag was if, in my sleep, I knocked the watch out of contact enough that it locked and wouldn’t buzz on calls, but in practice I was lucky enough that I never got a call when that happened.
Not that I was called often. The company had developers around the world so it was only if they needed some specific knowledge or skills. That meant that being on-call was mostly a fee to compensate for being available at short notice and not drinking, etc.
Ultimately they decided they weren’t getting enough calls to justify the cost of paying the on-call fees and scrapped the system, just relying upon whichever developers around the world were working.
If the company thinks its applications are important enough that it needs people around 24/7 to work on them then the company should hire enough people to have 3 shifts a day.
Is it abusive if you willingly sign up for it? Doctors are on-call for emergencies. We compensate for this so I'm not clear on why its abusive if employees willingly sign up. If you don't want to be on-call, find another job.
I think if you are compensated for it, and allowed to recuperate from middle-of-the-night working, it's ok. If the employee knew this was part of the job though, and not something that was thrust upon them after they were hired.
I do believe there are people out there for whom "living their best life" truly does entail spending almost all their time in front of a computer.
Nobody shamed Chopin for spending his life in front of the piano. Some people choose to dedicate their lives to one single thing, and often those are the people who end up contributing the most.
That's not the comparison as far as I understand it. The analogy is of someone that dedicates all their time to becoming the best at their craft, like Chopin. It's not about being a genius.
yeah no. as someone who has worked at Amazon for 7 years I can tell you that it’s not true across the board and there are good “pockets” where there is respect for people’s boundaries.
To add to this, in my time there I worked during the weekend 3 times (i don’t count oncall here). My work day included 8 hours at most. I would get in at around 8AM and leave at 4PM. (optimizing the commute)
I started as an SDE 2 by the time I left I was a Principal SDE.
Do good work, be in a good org.
Amazon is not a huge, homogeneous entity. It’s more like hundreds of small companies that are forced to work together, bound together by senior leadership.
I am sure it is not all of Amazon, but that is where the horror stories are coming from. So Amazon as a whole may not have this culture, but the teams with this culture seem to disproportionately be at Amazon.
so, the bigger an org, the higher the likelihood of stories like this popping up.
also, the incentives and their structure play a part in how this unfolds.
It’s also very curious to me that only a certain type of person runs into situations like these: either they don’t know better or they have accepted the conditioning that work is more important than themselves. it’s sad really.
i think it’s really really important to have a good mentor with a strong set of values to help you navigate the corporate landscape.
i also think it’s important to pull a Ghandi (ie be the change you want to see) as often as possible. for example: one think i would inoculate in young devs is that it’s not acceptable to set meetings earlier than 9am or later than 4pm. meetings during lunch were not accepted. i would be pretty vocal about morons that worked over the weekend because they had some free time. i would also push really hard against management that had their sick little practices to extract more work out of the drones. this mostly worked because i got the job done - but again: you have a voice. you are NOT a machine
law of numbers; Amazon Seattle headcount alone is 50K+ people. also, amazon has a service-oriented management model to match the general SoA architecture, where small teams are given massive leeway as to how exactly they run their services. managers and your leadership are thus very make or break.
I'm trying to interpret your words as something other than "well, yeah, my weekends were interrupted, but it wasn't my choice so it's not worth mentioning." I can't come up with another way to read it, though.
So, don't you think that, in the context of having to give up nights and weekends, you should mention on-call? Your choice or not, the weekends and nights are affected, in fact isn't it kind of worse when it's not your choice?
i agree with you that the oncall is not ideal. it is also widespread in the industry. so what’s the point to complain about something that is an essential part of the job? (advertised or not, if you don’t want to do it you will have to look for another job)
i also think there is a huge difference between being oncall and having to respond in case something breaks vs mandatory death marches all day, every day.
I guess it depends on your definition of widespread. It is relatively common, but it's also far from universal, and there are companies that use on call which have a vastly different experience that what many people at Amazon report. In fact it seems like even different teams inside of Amazon have drastically different experience of on call. Treating it like a fact of life, like the weather, leaves you in a frame of mind that makes it impossible to advocate for change. Your own phrasing "be the change you want to see in the world" could as easily apply to on call as much as it does anything else -- but you won't take that stance if you think it's inescapable.
hmm. no. you do have agency. you have agency during writing the code. you have agency during the code review. you have agency during wiring of alerts (if you do stupid shit you will receive a lot of pages. don’t so stupid shit). you have agency during the events. you also have agency after the events when you do a coe/post mortem. you can defined prioritize things to improve the life quality of other developers.
to give you an example. the team i was in at amazon had 3000 tickets in the queue when i started. anything except sev2s were basically ignored. lower severity tickets would escalate when shit hit the fan. i advocated for fixing classes of issues instead of myopically focusing on one-offs. by the time i left the queue was tens tickets and mostly feature request or higher level investigations.
to give you another example: i would basically remove all alerting that was not actionable. the worst possible thing that you can do is wake up in the middle of the night and not be able to do anything. i would ask for runbooks and the test was “if i take a developer from another team and put them oncall can they function independently 95% of the time”. i would think about what the experience of being oncall was (ie you don’t take people and throw them in the deep end of the pool and wonder why they drown)
so i guess what i’m saying is that oncall for me wasn’t that bad or stressful. it sucks having to be near a computer but I was rarely paged for stuff that broke or needed to be fixed right NOW. (once stabilized our team had 1 sev2 every other week)
Okay, at the risk of putting way too much effort into this, I'm going to assume you have 5 people on your team, therefore you're on call 10 times a year roughly. 5 weeks out of the year then you'd be oncall during a sev2. Assuming sev2's are uniformly distributed you are probably interrupted outside normal working hours ~80% of the time, so call it 4 times a year you have your work-life balance negatively affected by on-call.
you need to generate some 3d graphs and after that write a paper about using gradient descent to improve the WLB of SW developers :)
our team had 7 people and at some point we started sharing the oncall load with a sister team (ie you were oncall for the services on both teams), meaning that you would be oncall roughly once every 3 months. not ideal - but not the worst thing in the world.
this is very informative. I wouldn't have believed it, without the first person perspective. Before i read this, it was simply unbelievable to me that people could be that bored without any life outside work. Speaking as someone who really values free time and has tons of hobbies and things to do outside of work, it's just unbelievable.
Most of the people I know are fairly new graduates, so that would skew things.
My friends and I spent most of our lives since grade 9 doing school stuff or stuff to get into school stuff. In my case, couple that with introversion and an aversion to most normal activities and the easiest thing to do is be good at the get into school stuff.
In university, most of my social activities revolved around the startup world, hackathons, or other competition-oriented activities as that is what I was good at. I went to one hackathon a weekend in my final academic year.
So I became good at a lot of these types of things which all revolved around the academic world.
I am not in university anymore, but I can still fill my time with the kinds of competitions and challenges in that world. If I couldn't, I doubt that I would know how to use my time. Even with those, I spend a ton of time commenting online.
Most of the people I know are in the same kind of boat. They were good at one or two things (might be science fair, research, conference organizing, etc) back in university which consumed their time. Then they left and are now left wondering, "now what?"
Thank you for sharing. I can hear a shy cry for help. I think it's fine to spend a lot of time with computer, in general, I do it myself. There is so much to do, learn and help others. On the other hand, I think it is good to realize when it eventually becomes a replacement activity for lack of motivation/opportunities to do something harder. Then try to break that, take a challenge, do something new, either with computer (a personal project to make something cool?) or better, without it - for example, try some manual work with wood, do some science project with a friend, or get a pet and teach him tricks or something. Just don't spend yourself fully on the online/academic life, that sounds too ephemeral and can be a bit disappointing later on. I for sure regret spending so much time in academia.
If you can't sleep at 3AM and would like to, plan to do anything else afternoon than staring at the screen - like, try to walk and find a path to a moderately distant place in your city you know/like but have no idea how to get there on feet, without using a phone unless you really get lost. This can be fun and long walk and navigation will tire you down. You can think about what you're gonna do next time too.
Like taking up some challenges in life. Besides maintaining physical fitness and getting tired in the process, one thing I like is to create contraptions (does not have to be a "product") that could be useful to you or people you know. An air purifier, a charging station for a phone, a piece of wooden furniture, or anything else. Get some tools and get working.
If you prefer social activity, plan to do the above with a friend.
You move across the country to a city you don’t know, you end up there with no social skills and no real entry point it the social scene so you binge watch Netflix enroll in a masters degree and stare out of your condo window.
Then you ssh in and work to fill the hole. Now I don’t even get to see my co-workers.
Perhaps the first thing they should arrange is to have lunch breaks together with the low-wage work-force, i.e. in the same physical space. Or perhaps a company outing. That could result in some awareness and perhaps solidarity.
If these developers don't ever see or talk to the people who are suffering from the company strategy, then the problem may remain to appear imaginary or abstract.
Tbray is going to have a different experience at any org he joins.
He's not coming in as a no name peon. He's financially well off, internet famous and would (and could) probably leave at the first sign of mistreatment.
He's in Canada, do health insurance isn't tied to job.
If psychologists surveyed one of these companies, like Amazon, to look at the sorts of attachements the employees have now, and had as a child, I think you will find distinct deficit of people who had a happy childhoods with secure attachments sticking around at these companies.
The bravado and masochism so common in such places, in my opinion, is a form of self-bullying. You don't have to look hard to find managers exploiting this characteristic, and I suspect the most of the ones who thrive in such an environment do it. I worked a brief contract there and it was clear that the manager was used to making statements designed to trigger his subordinates' insecurities.
When your self worth is tied to the quality of your work, but your ability to self-evaluate is not solid. you are vulnerable to manipulation. I have that trigger too, it's just harder to hit, especially by some punk who is used to saying 'frog' and having people jump. I felt sometimes like he was waiting for me to react to some subtly passive-aggressive statement he just made and I just shrugged it off and stopped talking or kept going.
A master can be very tied to the quality of their work, and hand you your ass if you try to disagree with them. This doesn't develop overnight, and it's a winding road, but it's very undeveloped in a bunch of 25 year olds with a toxic cocktail of Impostor and Dunning-Kruger, and any 28 year old who spent more time honing their interpersonal skills in college has some ideas how to get what they want out of such people.
And those people think this is okay. Normal. That it's only what they deserve. And then we have a culture of avoiding psychotherapists, who would tell you two sessions in that this is bullshit and you shouldn't have to live this way.
The loaner thing is so true - having two bosses is THE PITS! That and not sleeping.
My first job out of college I had two bosses. HORRIBLE.
This came up recently, a manager, trying to be helpful, not sure they could manage someone said why don't we both manage them. NO!
The other things - trading sleep and shutting off friends - the first will destroy your quality of life and the second you will regret for a LONG time no matter how critical the day to day was.
I'm not perfect. Setting boundries. I focus on situations in my case - what situation lead to bad results. Also - you can sometimes trade time for reduced money - which in some of these situations is a NO BRAINER. I'm trying to do that. Society really IS NOT setup to support that.
I go home even if I'm behind. I sleep next to my wife and play with my kid. Covid has made catching back up with friends harder.
I think what folks from the outside don't realize is that folks making big money+ - you put a lot of pressure ON YOURSELF. You are given lots of money, often lots of responsibility, your work impacts lots of people (internally and externally). This is even if Jeff Bezos is not your boss.
My favorite example of having two bosses was as an intern. I was getting loaned out to a team that thought I was working full-time when I was really working 20 hours per week. I wasn't aware of this until I started getting super negative feedback from the manager. She basically thought I was the worst developer alive haha - then when I clarified I was a part-time intern and full-time student she totally lost it on my og manager. Of course he then was furious at me for blowing his scheme... not sure exactly how the internal budget/trading hours/money worked, but they were offloading 40 hours of full-time budget for my part-time efforts.
> On-call wasn’t (and isn’t) too terrible for my team. At first we averaged 1 page every 2 weeks; now we’re up to about 1 a week.
Wow, I wish I had once a week. This is way better than my previous employer/role. I was on-call for months straight, and would get paged multiple times a night, every day, even at 2am, 4am. I had to carry my laptop to dinners, date nights, game nights, etc.
I got burned out and moved careers into a role that never has on-call. I don't get to code much anymore, but I've found other things to enjoy
I don't know how absurd this sounds but on-call is the only reason I have stayed away from backend or frontend/web development. I had worked on both for few months and ran back to native app development. There might be a place where the company has a "follow the sun" policy for on-call but I haven't come across any.
Yeah On-call can vary greatly by company, although how On-call issues are handled can have a huge impact on your perception of a job.
At one previous employer, you were on a 2-week rotation for on-call.
But Every time you were on call, it was the same set of problems, always at the same inconvenient times (Mid sunday afternoon, random weekdays but always between 3-5AM, etc.) Admittedly often they were NOPs (Oh, alert came in, but when you check the system it's fine.)
But we didn't fix the false alarms. And we didn't fix the recurring real problems either; the Org's view was that was the entire purpose of the On-call person; that middle child you send to pay the loan shark not-quite enough because you don't care what happens to them.
At later employer, I actually had to deal with -more- on call issues, both in severity and frequency. And yet, I hate being on-call less, because at the very least when we find problems on-call the org puts a focus on fixing them.
'Encourage employees to be critical not just of ideas, but also of expectations.'
From my experience as a developer; I've never really been involved in the process of setting expectations. Something I find incredibly frustrating at times. It's like sure you can give your opinion but it's too late to make any difference. I think it comes back to the main point in the article about open communication.
I think open communication needs to be inclusive. However, that's rarely the case at least in terms of setting business expectations. It's only natural to want to be involved in shaping some objective or some expectation that impacts you and for which you may be one of the more qualified to discuss.
>I've never really been involved in the process of setting expectations
Nobody will ask you. You have to be upfront about it from the start.
Request time estimations and fully fledged requirement definitions from the start. Review them before accepting a project, push back against overly ambitious plans and propose changes that allow you to deliver value quickly and iteratively. It's easier to react to surprises if you are able to test the ideas as you go.
Stick to the plan as much as you can. Push back against changes to the requirements if they're not vital. Let others know about blockers clearly, as soon as you run into them. If you get behind schedule, communicate it properly. Stuff happens, unfortunately, and deadlines not being met is a part of life. You just get better at estimating with experience.
There's no way the upper echelons will adapt if they don't receive input timely and loudly enough. If your boss is any kind of decent, they will listen to you and trust your input. If they don't listen, then you should reconsider your current job.
> I mention on-call duty because it is “peculiar” in that the only other profession requiring this kind of responsiveness is doctors, literal lifesavers. When you go on-call for the first time, it’s terrifying and tells you “holy crap, this is serious.”
A key difference is that the oncall engineer is usually able to directly influence the quality and robustness of the systems being watched. Sort of like preventative medicine, but much more direct.
Crappy teams make crappy software that pages often (or not at all if their ops fails to write any meaningful alerts)
I don't trust an engineer that can't troubleshoot their own software running in production. I think the "you write it, you own it" adage is really smart and raises all tides when engineers are empowered to root cause each issue.
Well... some of us join teams that have problem software that is going to take a long time to fix, and don't have the manpower for it. And sometimes the alarms are overly sensitive.
Crappy managers stress delivery over quality, and then engineers pay for it when they're oncall. The manager doesn't care how much their team gets paged because they'll be sound asleep while their engineer bangs their head against the wall at 2am, and the manager gets their pat on the head for delivering more projects.
I appreciate this being shared, but I have some qualms.
I think talking about the Leadership Principles as a 'typical corporate motto' is unfair. Amazon lives their leadership principles far more than the other companies I've seen. They don't include things they don't actually care about, like most corporate mottos do (you'll notice the lack of 'we care about our employees' or 'do no evil' LPs).
At least in my experience, you only need to pay back Amazon proportional to (2 years minus duration employed) so it really isn't that bad (maybe that wasn't the case when this was written).
Maybe it's changed, but the signing bonuses I saw were essentially increased salary until your stock starts vesting seriously. It gets paid out and vests every paycheck so there isn't anything you need to pay back.
I think it depends on the size of the signing bonus. Smaller ones are paid upfront as a lump sum, larger ones are paid out over time. The latter of course is just an abuse of language by Amazon to disguise the fact that they're cutting your salary in year 3/4 in exchange for stock.
How so? Google is facing employee issues and protests because they sold the 'do no evil' ideal, but the employees don't think some of their projects follow that. I think you would struggle to find an equivalent portion of Amazon employees who think Amazon isn't customer obsessed.
> I mean in the sense that being customer obsessed should mean you do not evil
I think there is a real tension between customer obsession and 'do no evil'. Serving the individual needs of consumers can come at the expense of the public good. I think example of that are the amount of waste that Amazon packages generate by shipping as fast as possible and the amount of environmental damage AWS can quietly cause via their datacenters.
My signing bonus was structured in 13 separate payments, one attached to my first paycheck roughly equal to half the total bonus, and the other half distributed evenly over each paycheck in my second year of employment. If I had quit in my first year, I would have had to pay back the lump sum payment proportional to how long I had been at the company. The "bonus" paid out over the course of the second year never needs to be paid back under any circumstances.
Relocation bonus would have had to have been repaid in full if I had left during my first year.
From the perspective of not taking advantage of the financial irresponsibility of recent grads, this is an improvement over the structure described in TFA, but the right solution is make sure that recent grads know what compensation agreement they're signing and can resist the urge to spend it all, that can't be left to employers, whose every incentives push against that. Big lump sum payment right out of college are very useful as emergency slush funds even if you can't responsibly do much with them for a few months.
I can confirm that unlike most other companies' "mission" the Leadership Principles are a real thing(tm) that even those of us who are skeptical about corporate yadda yadda were caught up using and being influenced by.
Two things about it: it's not feel-good bullshit like the parent said. That makes it real. And another: so many principles are at direct odds. Do you "Insist on the highest standards" and delay the launch or do you "Deliver results" now? How do you "Think Big" while also obeying the mandate for "Frugality"? There is a lot of tension in the principles which matches real-world choices. It's not the shallow corporatese of most mission statements.
You're right. The proportional compensation is something I was very wrong about.
At the time, I was also in debt due to being straight out of college and having been dumb AF with money (student loans, car, & credit cards) and so owing anything would've been bankrupting.
It was so easy to be stupid with money when you're making an engineer's salary.
Not shifting blame, I did that debt to myself and I dug myself out of it after educating myself.
Not sure anyone will see this since the post is rather old now and off the front page, but just to add my two cents:
I was made an offer for L5 at Amazon earlier this year. For compensation, it's like you suggested: the offer is structured such that the first two years weigh more heavily toward cash (a separate signing bonus for each of the first two years, which IMO was pretty good and _does not_ need to be paid back, it's just combined with your normal bi-weekly paychecks) and then the latter two years are where you get the majority of your RSU rewards, which wind up being worth the same or more than the cash bonuses of the first two years (assuming I suppose that the stock price continues to be stable and/or increase from what it was at time of signing).
A note about the signing bonuses, the offer letter details basically do not say anything about needing to pay them back for leaving early. If your employment is terminated (by yourself or Amazon itself), then you just end with the prorated portion for the current pay cycle and that's it, you don't get any more of it (fairly obvious anyway if you leave a place).
I used to be like that and then one day I realized it's just a CS job, I'm not doing open heart surgery so I shouldn't stress about it. Since then my life has been much better.
You could be shown the door in no-time. Amazon has forced attrition and the manager decides who gets the cull. They may make a mountain out of a mole to further their objectives.
As someone who came from a similar environment (MSFT during the stack ranking days), the forced attrition and manager's arbitrarily blackballing employees are the ingredients of the toxic stew. If a manager hates your guts, there's really nothing you can do, even if you are great at your job.
Does your manager love your guts? Are you good at your job? Hopefully that paycheck can bleach the guilt of your colleagues losing their jobs because you are good at yours. That toxicity could also be because they are either selecting or creating people who are okay with this.
Yes, yes, and thankfully I left that environment. Now I work in the public sector, I guess you could say. Mostly because of the gas-lighting of reviews and performance "feedback." I've been on both sides of it, and seeing that I'm the same person, it really made it obvious how arbitrary it is.
I feel like it's common wisdom now that forced culling is not a great strategy and has lots of perverse incentives, but still not universal.
The on call stuff sounds outright dystopian to me, it's not even legal in my country. More places should think about adopting something akin to France's El Khomri law's (https://en.wikipedia.org/wiki/Right_to_disconnect) and ensure that every citizen has enough time to recuperate from work and isn't pestered by their employer 24/7.
I don't understand what's so dystopian about it. Some systems are critical and need to be kept online at all time, or at least have very minimal downtime.
The description in the post is not even that bad. A page a week is nothing. And it is clearly state that your workload is reduced during your on call period.
The dystopianism comes in because the vast majority of teams are decidedly _not_ 27/4 critical. Sure, some teams really run the 1-Click Buy button or the S3 API. Most run some DB behind some API behind another 3 APIs behind 5 other teams that run a widget.
In practice, your workload may not be reduced during on call because it's "expected" and you still have to meet your normal deliverables while the team isn't staffed with "extra" people. It also takes a toll on employees to be mentally-available at all times in case some triviality emerges and raises an alarm. You'll find many engineers not willing to leave a small area or even go out for dinner without bringing their laptop and a hotspot, just in case.
Your "in practice" doesn't match my on call experience over the past 15 years. During "on call" workload is reduced (the author of the post even mentions it). There are ways to implement sane on call policies that benefit everyone. A few I've seen:
- use a primary and secondary on call so that the primary can still go get groceries, or play in the yard with their kids, etc
- overlap on call and work hours. I did that at a job and I was fine with working 6pm - 2am just to cover this time slot. Won't work for everyone, but may be a nice compromise for some.
Like everything else, there are ways to do it right, and way to go complete bananas with it.
> Like everything else, there are ways to do it right, and way to go complete bananas with it.
Sure, agreed. Sometimes it's implemented well and sometimes it isn't. I've experienced some of both.
When it isn't well-implemented and is particularly non-critical, it can feel a bit dystopian to the engineers. If done well and for a good cause, it should be fine.
On my team, net workload is not reduced, it just becomes radically different. It's only your time spent on longer term projects that gets reduced (to zero).
What was stated seems pretty typical on small teams who manage critical systems. How are critical systems managed otherwise without having to 3x your team (assuming you want 24hr coverage with a 8hr workday)?
>How are critical systems managed otherwise without having to 3x your team (assuming you want 24hr coverage with a 8hr workday)?
They typically aren't, you need to hire more people then. In anything defined as critical I would suggest that being well rested is also a requirement rather than having a team of people who are so overworked they develop mental illness.
Doesn't make sense financially. You're gonna triple the size of your team, just to make sure that this one time of the week someone will restart the DB?
I would much rather bite the bullet, be on call a week a month or so and use the budget for something else.
This is a law to protect the mental and physical well being of citizens and workers, obviously not Amazon's bottom line, that's literally the point. If you treat people like disposable commodities then yes this does not make any sense.
You know what impact the mental and physical well being of workers? Have their company go under because they had to triple the size of the team just to implement an on call policy.
And I'm not talking about Amazon here. Of course Amazon can afford to hire everybody and their mom to cover 24 hours in a day. I'm under the impression that your original comment was really a critic of the practice in general, not just this specific example.
Some small companies build products they try to sell to bigger companies. These bigger companies have SLAs, so they need SLAs from their providers to sign a contract. Even at a small startup on call may be necessary to get customers.
>I'm under the impression that your original comment was really a critic of the practice in general, not just this specific example.
It absolutely was criticism of the practise in general. Yes, small companies that can't ensure that there workers aren't overworked don't really make sense with protections like these. But the same is true for environmental protection.
Critical infrastructure arguably ought to be handled by companies large enough and with enough staff to comply with regulation like this.
For anything else I don't see the issue. Some random smartphone app company doesn't need to fix anything at 3 am in the morning, it can be fixed the next day. That's exactly the culture I'm critizing. 24/7 readiness to work for some random product at the expense of mental health is awful.
I feel like we're just going in circles because you're not addressing any of the examples I'm providing.
I gave you examples of how to implement on call without overworking your employees and even giving them the option to run a few errands here and there. So what's the problem? You seem to be against the practice just for the principle, not even really for any bad effets due to bad implementations.
As for which products are worth implementing "on call" for, yes I agree, some companies are too quick to think that the world can't function 30 minutes without their product.
Anyways, I've seen it implemented successfully at companies with very low attrition and I do believe it is a necessary tool sometimes for small companies to grow and sign big whale customers.
You don't need to keep the primary development team as primary oncall 24/7. "Hire people to keep an eye on things off-hours, follow playbooks when necessary, and escalate to the right people if they can't resolve" is a reasonable pattern that folks have done for decades the world over.
You get less incentive alignment perhaps, but it really isn't hard to come up with ways to make creating work for another team unappealing.
More members on the team would increase the gap between 2 on call periods for team member (e.g. every 4 weeks instead of 2), it wouldn't get rid of on call entirely.
Then that's the cost of managing the service. The only reason companies don't pay that now is because of the significant leverage they have over software development labor. We may be better compensated than many other roles, but we're still just entirely expendable resources.
The only reason companies don't do that is because it doesn't make sense. What are you gonna do with a team that is 3 times the ideal size just to have enough people to cover a 7/24 on call?
It absolutely does make sense if the service is that critical. If it's not that critical, it doesn't need 24/7 on-call. There are some "in between" circumstances, sure, and those might require additional resources or occasionally work outside of normal hours.
It doesn't make financial sense because companies can get away with exploitative behavior because software engineers are quite naive (or even actively detrimental to themselves and their peers) when it comes to labor relations.
So you have 5 engineers building your product. Now you hire 10 more and organize them in 3 shifts. What work do you give to the 10 new engineers? Are you supposed to now hire 200% more PMs, designers, QA, etc and make up some new projects for these guys?
I see this comment again and again on this thread "if it's really critical spend the money", but the thing is even small companies sometimes have critical systems. Simply because they're trying to compete with bigger ones, or they're partnering with companies that do have critical systems and the SLA gets passed down the chain of dependencies.
Why are we assuming that Amazon, the third largest corporation in the world by market cap, needs to limit itself to small teams?
They could afford to 3x the teams if they wanted to. They just don't want to, because management prefers to just keep the money and burn out their subordinates. (There's always more where they came from.)
How are you proposing to run a complex system that essentially powers a big chunk of the entire internet and can go down any second without on-call?
If a big selling point of your system is the ability to be on 24/7, on-call is inevitable. But it definitely can be way less gruesome than it currently is at Amazon for a lot of teams right now. For example, my on-call schedule is extremely relaxed and low-stress, it is never a worry I actually have or something I dread ahead of time. But getting rid of it completely is simply impossible for a lot of "always online" services.
Hiring more people is a great idea for making on-call less stressful, because it allows for much sparser on-call shifts (e.g., one week every 8 months vs. one week every 2 months). But I don't see how hiring more people gets rid of on-call completely. Someone will always have to be on-call.
You cannot just hire people who do on-call-only duties and nothing else, you have to be very closely familiar with the code and features being written in order to resolve issues effectively. Which usually means that you need to be someone who actually worked on those things, not just some glorified extra-fancy customer-support worker.
According to TFA, programmers at Amazon are required to be on call for a full week at a time. More programmers could mean sparser on call shifts, but it could also mean shorter on call shifts -- i.e. instead of going from one week every two months to one every eight months, go to two or three days every two months.
Being effectively at work 24 hours a day for seven days straight sure sounds like a recipe for burnout to me. As TFA notes, these people aren't doctors, the only thing that happens if a glitch takes two hours to fix rather than one is that Amazon loses some money. Amazon naturally would prefer not to lose that money, but its workers presumably would prefer to be able to sleep too.
Where I'm at we have an on-call schedule like the one mentioned in the post. It isn't too bad, and we rarely ever get paged. Most of the issues that the on-call person has to deal with, at least on my team, will come up during the normal working hours.
> Revisit your bonus/relocation payback strategy ... if that's what you need to keep employees, that's fucked.
The reason for it isn't employee retention. It's to stop people from accepting a job from Amazon just so someone will pay for their move to Seattle and give them a free bonus.
Would people do this? Yes. I've seen them try. It also discourages people who aren't serious about the job.
> Remember when I mentioned Amazon's moving and signing bonus? Little caveat on that, if you leave or are fired within 2 years, you have to pay it back.
Paying back a signing bonus if they fire you seems super rough. Is this common elsewhere? 2 years also seems high - the time I got a signing bonus, it was "pay back if you leave voluntarily in 1 year" which was much nicer than the described.
Unfortunately, much of this seems pretty common, especially months-of-crunch-time. Being able to figure out projects to avoid (if you're at a big corp like this where such things are possible), and being able to avoid them is a very good career skill to cultivate.
On-call can work quite well if the team has sufficient ownership to improve the underlying issues, not just keep the lights on. I've done this with good success. I'm often disappointed by how many managers and companies I've seen who don't get that aspect.
Believing too much in mantras like "adding manpower to a late software project makes it later" can be dangerous, too, though. How well it works depends on the manager and the person scoping and planning the work. Another critical skill to develop to escape being the person doing the grunt work.
> Paying back a signing bonus if they fire you seems super rough. Is this common elsewhere? 2 years also seems high - the time I got a signing bonus, it was "pay back if you leave voluntarily in 1 year" which was much nicer than the described.
I actually don't think this is true. My understanding is if you're PIPed they don't really care. The signing bonus is paid out monthly the 2nd year (and for people hired at SDE2+ its first year too) so you don't have to pay it back after the first year.
I've heard that companies rarely enforce these clauses unless you really burned a bridge. If anything they won't give you severance if you quit within the first year.
I fear OP may have misunderstood how the signing bonus works and it seems to have caused him undue stress.
I've interviewed at Amazon and had a lot of conversations about compensation. I don't know about the terms of the relocation bonus, but for the signing bonus, you don't pay it back if you are fired and even if you leave voluntarily, you only pay back a pro-rated amount. For example if your signing bonus is $20k, and you leave at the 6 months mark, you owe $10k back but you get to keep the other $10k.
I'm surprised at how many people in this thread are misunderstanding this. Every tech company I've seen has had a stipulation saying you have to pay back signing bonus or relocation if you leave within 1 year.
Yea, that's something else I forgot to mention. I chalk it up to OP possibly being inexperienced/naive (I think this was his first job out of college), but I have worked with a lot of companies and all of them require you to pay back at least part of your signing bonus if you quit within a year.
When I graduated college, all of my friend group discussed it and we all took out a small chunk of the bonus to go on a small group vacation, but the rest of the bonus got put into an investment account and not touched for at least a year. That way if we did decide to leave, the money was easily accessible to be paid back. I know not everyone has that flexibility, but it's probably the best way to approach signing bonuses if you're not 100% sure you will be at the company long term.
My experience with Amazon is that my signing bonus was in two parts:
Part one is in your first year where you get a lump sum signing bonus with your first paycheck (paid monthly) with the understanding that if you leave you would pay back a pro-rated amount proportional to how many months were left in the year.
Part two is in your second year and is paid partially with every pay check and if you leave you don’t pay anything back, you just don’t get the remainder of the money to be paid out the rest of the year.
After year two stock vests tend to vastly outpace the signing bonus pay but I’d imagine this varies on the stock price.
> After being on the team for a couple of months, I was put on the on-call rotation
I don't work at Amazon but the situation is very similar to my job. On-call can be brutal, even if there are no incidents during your week (which is rare.)
Just the fact that you have to be 24/7 _ready_ during a whole week sucks, especially when you have a family. During my on-call week, I can't go get groceries, I can't take my kid to her hockey practice, we can't go out for lunch over the weekend, etc.
At my last position they tried to offer me a "promotion". More responsibility, more accountability, mentoring juniors AND on call, all for the grand pay raise of $0.
"on call" was super, super vague and I pushed back hard on it. If they require me to work outside something reasonable like 6am-6pm, they will pay me at least double for it. No exceptions.
I wish everyone would push back that hard, and that on-call was an official opt-in "extra pay" scenario.
Needless to say I turned down their "promotion" even after months of them trying to make me take it.
Note: I am NOT saying I'll work 12 hour days. I'm just saying if I have to come in early for a meeting for 6am, or stay until 6pm sometimes, that's fine. If I'm expected to come in or be on call at 2am, that's a different story.
The 1st time I worked at a company with "on call" we had this concern while trying to define the policy/rotation/etc. It was always "well, yeah I can be on call sure, but this specific Tuesday at 7pm I have this thing". We solved it in 5 minutes: just have a primary and a secondary that gets the page after 10 minutes. Solved. Go get your groceries, if it's bad luck and the page happens during the 45 minutes where you're out, your buddy will get it.
Yep. This was the solution for us. The secondary is the next-in-line for primary. Secondary is also first in line for helping out during incidents. This means communication with other teams, a second incident popping up (rare, but has happened), etc.
I’ve found that just having someone else to commiserate with at midnight, if needed, really helps the mental state of on-call.
Absolutely. We had the same policy. If primary wasn't sure what to do, they could ping secondary to regroup. We even had a third "last line of defense" which was basically always the same 3 people, pinged if nothing was acked in the first 30mn.
There was one period in my life at a 401k administration place where I was responsible for making sure the transaction data for all of the clients made it to AIG every night. For months, I had to do the transmission manually at 7pm each night, which meant not much fun for me ever. I never even got a thank you, and when I finally managed to get it automated, my position was eliminated. At least they gave me a pretty good severance.
> During the hiring process, on-call is not mentioned in any way
Then next:
> are you willing to work nights and/or weekends
To be fair I'd say this is mention enough.
If someone is willing to (or has to) look the other way at the time of hiring when something like this was explicitly said, then even a "hey, we have on-call here, you fine with that?" would be okay.
I think it goes both ways (in fact, most of this post seems to go both ways in that Amazon sounds brutal but OP could have taken actions to mitigate it). If your team is going to be requiring on-call, you should say that during the interview process. "Are you willing to work nights and/or weekends" is indicative, but is still too vague. Some might interpret that as just "sometimes you might have to work late if you are on a tight deadline", not as an on-call situation.
However on the interviewee's side, if you see something in a job description that asks if you're willing to work nights and weekends, that should be an immediate indicator to you that you should ask why you possibly would have to work nights/weekends and in what capacity. It's the interviewee's responsibility to ask questions like this even if the employer isn't immediately forthcoming.
I'm not sure where they got the notion that emergency services are the only other field where people go on-call. Plenty of industries operate 24/7.
I worked in high volume manufacturing where process engineers could be on-call 24/7-365 unless you explicitly asked a backup to cover you during a vacation or weekend. Depending on the part of the process you owned, that might mean a middle-of-the-night call a couple times a month or several times a week. No human lives on the line, just lots of money.
I have had a different experience with working at AWS. I've heard really terrible things working in retail, but AWS has been a pretty good place to work, with normal 9-5 hours. I can go more into detail about how my experience has been different from the article if people want.
I'm a software engineer at Amazon in Retail. (This is just my opinion, I don't speak for Amazon in any official capacity, etc.)
Different software products have different severity levels. For my team, the oncall burden is not so bad. We have an internally facing tool. When we do get paged, it's usually requires acknowledging a ticket, a few minutes to validate something, sending an email out, and that's it. For teams with more externally visible products, a particular page can be stressful, but my understanding is they also spend a lot of time in ensuring system robustness in order to limit the number of pages. The "15 minute" response time the post referred to is generally to acknowledgement - the expected time to resolution can be quite variable.
The working hours are generally pretty good as well. The general expectation is that you're not working crazy hours.
I've ran into "project crunch time" at Amazon, but the experience at Amazon was like everywhere else I've worked. This also will vary by team: teams that need to make changes to support Black Friday obviously have a pretty hard deadline. Or if your team is getting ready for a re:invent announcement, there's no way to work around that. But from what I've heard, teams that have such a hard deadline also have a corresponding quiet period in December or January.
Most people I know within Amazon that get burned out on one team and feel like there's no resolution within that team (or just feel like they've had enough with one area) solve that by switching teams. This is generally pretty easy, since most teams are hiring at any time, and they would rather take an internal transfer than an outside hire.
Amazon has a pretty good system around having a senior individual contributor track (I haven't worked for any of the other big tech companies, my understanding is that they're similar). Part of the benefit of this system is that SDE1's/SDE2's have someone to reach out to for helping navigate management. Anyone at Amazon who reads this who is experiencing something similar to the OP should feel free to reach out to one of the senior engineers for help.
Overall, I don't doubt that the post in the original article is that person's experience, but it has not been my experience.
Anyone working over their contracted hours needs to realise they're being exploited by their employer. Sometimes this is justified: we pay more than the going rate because we expect you to step up occasionally. Maybe it's kind of expected with Amazon in particular.
But when it descends into perma death march and your manager flings you "a few" days holiday in recompense... does that make up for it? If it doesn't, to whom has the profit gone?
They have a contract, but the contract will specify them as "exempt salaried" employees. That's pretty standard across the board in the tech world (as well as many other professional roles).
Exempt salaried employees aren't paid by the hour, and don't have a set number of hours they are expected to work. There is no overtime, but there's also no (contractual) penalty for working fewer than 40 hours in a week.
>Be upfront and as precise as possible with what a position entails. "Working alongside smart and passionate people" is lazy hiring.
I understand that this person was coming straight out of school, but the onus is on the applicant to figure out more about the job during the interview process.
Interviewers, if they're coached properly by the organization, will lie. "Sell the candidate on the company" is a thing. Human beings aren't conditioned to have their skepticism alarms ramped up to 100% constantly in every interaction. At some point a certain level of trust to be honest with one another is required for society to have any cohesion. In my view businesses, especially software/tech companies, are quite poor at this on a good day and frequently work in ways that undermine it.
> I mention on-call duty because it is “peculiar” in that the only other profession requiring this kind of responsiveness is doctors
Uh, not even close. I used to get up at 4am EST to trade European markets for a hedge fund. I was on call 24/7/365. I had my laptop and Bloomberg fingerprint scanner with me on any trip I took.
And this is just my experience. I have had many friends in engineering, management consulting, fin tech, etc who have similar demands made of them.
I’m not saying it’s right, but it is part of the rules of the game that we all sign up to. There’s a reason all of the aforementioned jobs pay so well.
In the military, we had a 2 hour recall for a large chunk of our battalion, about 200 people for a few months at a time. If you got a call, you were expected to be ready to go in less than 2 hours. Social norms meant the expectation was closer to 45 minutes than 2 hours.
I think oncall is a common requirement for anything involving large sums of money, treatment of high risk injuries, or geopolitical power struggles.
One I hadn’t even considered (typical stereotype here: no friends who are in the armed forces). But yeah I imagine loads of others (military, EMT, police) are in a similar situation.
280 comments
[ 4.6 ms ] story [ 273 ms ] threadYet to know anyone who lasted more than a year or two there. Obviously some do, but they go through an enormous number of people.
I was at Amazon for four years and enjoyed my time there. I had one stint on a high profile high-pressure team that lasted about 6 months (no on-call, though). That was the busiest time I had at Amazon but it was kinda known going into that project.
I generally worked 45-50 hour weeks, with the occasional 50-60 hour weeks but that was pretty rare.
It's a big company. It really depends a lot on your manager, your team, and your org.
What is it about tech culture that people within it refuse to admit the level of abuse-by-design that happens at these companies? Is it that they want to believe that they are somehow fundamentally different from the distribution and grocery store workers, and fundamentally similar to Bezos? I can't help but feel it's just this kind of false consciousness that prevents developers from _getting organized_.
Sidenote, payroll tax on Amazon in Seattle trying to get revived. It's a long and winding story. Check out taxamazon.net
My work treats me exceptionally well. It does not treat the sales center employees as well. I find it hard to say it's a terrible place to work, because as mentioned earlier, it takes tremendous care of me, and is the reason I am able to have the life I have, the house I have, and support the family I have in the way I can.
It's ok to want to continue to be treated great, and also have them treat others better.
That's how every business works. All per employee costs are borne by the employee one way or another.
By the same argument, customers are paying the costs for Best Buy's internal logistics and retail floor operations to enable “free browse and instant pickup”. If those costs are higher than Amazon/NewEgg's costs for "free shipping", I'd expect to see exactly the relative prices that you're seeing.
I'll reiterate my point that shipping is a cost that has to be paid, but who bears that cost is not necessarily the customer. If it was the case that the customer was the one bearing that cost, there would be a difference in the price you pay for goods at a physical store vs. goods shipped to your home, implicitly or explicitly. There is generally no difference, and in many cases, ship to home is cheaper because you can select from more retailers.
In the larger discussion of payroll taxes, to disprove that workers bear the entire burden of payroll taxes, just go through the thought experiment where we eliminate the employer side of the SS tax and whether wages would rise 6.2%. They wouldn't. Workers were willing to work for $N previously, there is no reason that they would stop working/change jobs unless they were paid $N + 6.2%.
But now do the thought experiment where the employee must explicitly pay the extra 6.2%. Now of course you have to give them a 6.2% raise, because they would demand it to cover their increased costs.
Numbers: I pay the worker $10000 a month. Let's say my tax cost is $500 a month, and their tax cost is $500/mo. So they get a check for $9500, they see a $500 tax deduction, and they net $9000.
My all in cost is $10000.
Now the city says "you don't have to pay your part of the tax anymore, the worker does!"
So now I pay them $10000, they see a deduction of $1000 on their paycheck, and net $9000. My all in cost is still $10000, their net is still $9000. Nothing has changed.
I'm still paying $10000 to have that worker. And if my budget is 100000, I can still only afford 10 workers.
https://www.reddit.com/r/cscareerquestions/comments/4eu909/i...
The divide between AWS employees and everyone else employed at Amazon seems to be "2007 Google employee" against "sweatshop"; they're run mostly-separately, from what I've read.
This was also four years ago; the gist is a copy of a post from 2015, by the above reddit poster.
This is incredibly offensive. I work in CDO and have friends in retail and AWS and the biggest difference day-to-day is that we have different systems to manage our AWS accounts.
Equivocating me to a Wipro employee is honestly really disgusting. These sorts of microaggressions are the worst part of working at Amazon, the WLB is fine.
This is not true. Amazon and AWS are exactly the same organizationally speaking. You could hardly tell who works in an Amazon team and who works in an AWS team. Maybe if they tell you their building.
The cultural drift happens in the teams and orgs, not the parent "company". Everyone at Amazon knows this and that’s why many people know changing teams is a gamble and prefer staying at the same team doing the same menial tasks, than venturing into finding another team.
Amazon and AWS have both great teams and shitty teams. There’s just not an easy way to tell, except for one internal tool that allows you to see how people are rating their managers but this is all bs because some teams don’t have enough data or any data at all. Also the data itself is probably skewed, since people are afraid of giving poor rates to their managers (this is a general problem in the modern workplace dynamics, not at an Amazon problem)
The other way is asking a friend but I don’t know how people do that since I only knew people from my team and teams that sat nearby. Maybe I’m too socially incompetent.
This iteration is targeting the 800 largest employers in Seattle, many of whom aren’t currently profitable.
“Tax Amazon” is classic dishonesty — neither bill has been focused on Amazon, both would be a disaster for Seattle, and proponents try to focus on Amazon to keep people from noticing how damaging their out of control spending is.
In 2010, the Seattle budget was 3.8B; in 2020, it’s 6.5B. An increase of 2.7B and growth of 71% in a decade. (Per capita, the Seattle budget is 37% more now than in 2010.)
https://www.geekwire.com/2018/seattle-socialists-constructio...
https://www.seattletimes.com/seattle-news/politics/seattle-c...
That's a bold assertion without any citation.
Writing off the opposition as solely "pro-amazon shills" is exactly what gets those bs laws passed in the first place. I wonder how much money was wasted on all of this, given that it was repealed shortly after. Seems to be very in-line with how wastefully Seattle City Council has been running things.
0. https://www.king5.com/article/news/local/seattle/enough-is-e...
If your goal is to reduce rents by reducing the number of people living nearby, then it can meet the goal.
And look where that got cities.
One school of thought is "just keep importing jobs AND add a ton of new housing" but there's a sort of capitalistic dystopian feeling to that, like that's the path to Blade Runner. Just need to start shipping out the people without high-enough paying jobs (again, actually - Europeans moving to the Americas doesn't seem that dissimilar) and we're there!
That's from 2017 when it was introduced but it's a good non-partisan explanation, including the many problems with the proposal. https://sccinsight.com/tag/head-tax/ has newer posts.
Seattle population has grown by around 23% [1] and inflation means $1 in 2010 has the spending power of $1.18 today [2], you'd expect a 45% increase (1.18*1.23) just accounting for population change. According to seattle's open government website, their budget was $6.24Bn in 2020, and $4.13Bn in 2010, a 51% increase. So just ball park spending has grown probably less than 0.5% per annum over inflation in the last decade.
[1]: Acutally 2010-2018 - https://www.google.com/search?client=firefox-b-d&q=seattle+p...
[2]: https://www.in2013dollars.com/us/inflation/2010?amount=1
[3]: https://openbudget.seattle.gov/#!/year/2020/operating/0/serv...
> This iteration is targeting the 800 largest employers in Seattle, many of whom aren’t currently profitable.
Citations needed. Citing a few articles quoting a few politicians does not count.
Creating an alt account to make this one comment not really trust-inspiring either.
Amazon is a fascinating company, it's Big Capitalism in a way no other company is today, and as such it's writing History, for better or worse. For the average developer it's not great, though.
So at the same time it can be true that company X is a great place and also true that company X is a nightmare.
I think this might be a big part of it. The idea that "I'm college educated and don't perform physical labor means I'm not like them" seems to be a pervasive societal thought not even strictly related to tech.
It's not even programmers like OP but also all the servicepeople and staff at datacenters, and the rest of the crew which makes up the infrastructure that makes it work.
The Amazon example is extreme, but a lot of tech workers fit the stereotype of people who don't have a lot else to do.
I spend 100+ hours at my computer a week, so if you want long hours, I can live with that.
I am up at 1AM anyway, so if you need something done at that time, sure. I just need to switch to my remote desktop.
I have colleagues and friends who will log in on weekends or evenings out of boredom to do work. No pressure to do it, but they just want something to do.
I fill my time with internet commenting and various online contests, but if they need help at 9PM, I am fine with jumping on the VPN to help and have done that a few times myself.
Amazon still has problems beyond the long hours and the 3AM calls, but there are people ok with those things or at least ok enough with them to not put up a fuss.
Very sad when you think about it. Wasted potential.
Eh, you leave the university environment of all its clubs, hackathons, etc, likely to another city where you know far fewer people. You need to wake up the next morning, so gaming until 5AM with your friends no longer works. Someone is also on on-call, so they can't join, so you postpose it further.
The end result is a lot more free time.
The biggest problem is when people like this article's author get a job and the HR people never make it clear that they may need to have "pager duty" or otherwise work outside of 8-5. Those different expectations is what caused him to have a very bad experience.
Yes, these expectations should be made clear from the beginning.
I suspect they don't as they would be even more desperate to hire.
I didn't move forward to an interview, this was just from a meet-and-greet they did a few months ago (they do it every month IIRC). I have other reasons for not being at all interested in working for Amazon, but I figured it would be interesting to meet with them informally anyway.
There's a huge difference between being responsible for (some level of) ops and literally climbing out of bed when the pager rings. At work, I'm responsible for the operations of the services I develop, which means that I'm doing third level support during normal business hours and I write the playbooks that first/second level support executes. You certainly won't catch me working after 5 PM.
I then started wearing headphones to bed so that I would hear the phone ring and not disturb them, but when you're rolling out of bed and hopping on a call at 3AM it's difficult.
What I seriously underestimated was the strain that being on-call can put on a relationship, it's a massive regret.
I'm a pretty heavy sleeper, and would go to bed with my phone on silent, set to ring a long time before going to answerphone, and with my Apple Watch on my wrist. If my phone rang then my watch would buzz me awake, eventually. The only snag was if, in my sleep, I knocked the watch out of contact enough that it locked and wouldn’t buzz on calls, but in practice I was lucky enough that I never got a call when that happened.
Not that I was called often. The company had developers around the world so it was only if they needed some specific knowledge or skills. That meant that being on-call was mostly a fee to compensate for being available at short notice and not drinking, etc.
Ultimately they decided they weren’t getting enough calls to justify the cost of paying the on-call fees and scrapped the system, just relying upon whichever developers around the world were working.
If the company thinks its applications are important enough that it needs people around 24/7 to work on them then the company should hire enough people to have 3 shifts a day.
Nobody shamed Chopin for spending his life in front of the piano. Some people choose to dedicate their lives to one single thing, and often those are the people who end up contributing the most.
And maybe, just maybe, universally acclaimed musical geniuses are not a good point of comparison for common folk.
To add to this, in my time there I worked during the weekend 3 times (i don’t count oncall here). My work day included 8 hours at most. I would get in at around 8AM and leave at 4PM. (optimizing the commute)
I started as an SDE 2 by the time I left I was a Principal SDE.
Do good work, be in a good org.
Amazon is not a huge, homogeneous entity. It’s more like hundreds of small companies that are forced to work together, bound together by senior leadership.
also, the incentives and their structure play a part in how this unfolds.
It’s also very curious to me that only a certain type of person runs into situations like these: either they don’t know better or they have accepted the conditioning that work is more important than themselves. it’s sad really.
i think it’s really really important to have a good mentor with a strong set of values to help you navigate the corporate landscape.
i also think it’s important to pull a Ghandi (ie be the change you want to see) as often as possible. for example: one think i would inoculate in young devs is that it’s not acceptable to set meetings earlier than 9am or later than 4pm. meetings during lunch were not accepted. i would be pretty vocal about morons that worked over the weekend because they had some free time. i would also push really hard against management that had their sick little practices to extract more work out of the drones. this mostly worked because i got the job done - but again: you have a voice. you are NOT a machine
Why wouldn't you count it? Do those days not-count somehow?
So, don't you think that, in the context of having to give up nights and weekends, you should mention on-call? Your choice or not, the weekends and nights are affected, in fact isn't it kind of worse when it's not your choice?
i also think there is a huge difference between being oncall and having to respond in case something breaks vs mandatory death marches all day, every day.
to give you an example. the team i was in at amazon had 3000 tickets in the queue when i started. anything except sev2s were basically ignored. lower severity tickets would escalate when shit hit the fan. i advocated for fixing classes of issues instead of myopically focusing on one-offs. by the time i left the queue was tens tickets and mostly feature request or higher level investigations.
to give you another example: i would basically remove all alerting that was not actionable. the worst possible thing that you can do is wake up in the middle of the night and not be able to do anything. i would ask for runbooks and the test was “if i take a developer from another team and put them oncall can they function independently 95% of the time”. i would think about what the experience of being oncall was (ie you don’t take people and throw them in the deep end of the pool and wonder why they drown)
so i guess what i’m saying is that oncall for me wasn’t that bad or stressful. it sucks having to be near a computer but I was rarely paged for stuff that broke or needed to be fixed right NOW. (once stabilized our team had 1 sev2 every other week)
our team had 7 people and at some point we started sharing the oncall load with a sister team (ie you were oncall for the services on both teams), meaning that you would be oncall roughly once every 3 months. not ideal - but not the worst thing in the world.
How many promotions was that? 4? How did you manage to do that in 7 years? That doesn’t sound easy at all.
My friends and I spent most of our lives since grade 9 doing school stuff or stuff to get into school stuff. In my case, couple that with introversion and an aversion to most normal activities and the easiest thing to do is be good at the get into school stuff.
In university, most of my social activities revolved around the startup world, hackathons, or other competition-oriented activities as that is what I was good at. I went to one hackathon a weekend in my final academic year.
So I became good at a lot of these types of things which all revolved around the academic world.
I am not in university anymore, but I can still fill my time with the kinds of competitions and challenges in that world. If I couldn't, I doubt that I would know how to use my time. Even with those, I spend a ton of time commenting online.
Most of the people I know are in the same kind of boat. They were good at one or two things (might be science fair, research, conference organizing, etc) back in university which consumed their time. Then they left and are now left wondering, "now what?"
Like taking up some challenges in life. Besides maintaining physical fitness and getting tired in the process, one thing I like is to create contraptions (does not have to be a "product") that could be useful to you or people you know. An air purifier, a charging station for a phone, a piece of wooden furniture, or anything else. Get some tools and get working.
If you prefer social activity, plan to do the above with a friend.
Then you ssh in and work to fill the hole. Now I don’t even get to see my co-workers.
“It is difficult to get a man to understand something, when his salary depends on his not understanding it.”
― Upton Sinclair
If these developers don't ever see or talk to the people who are suffering from the company strategy, then the problem may remain to appear imaginary or abstract.
He's not coming in as a no name peon. He's financially well off, internet famous and would (and could) probably leave at the first sign of mistreatment.
He's in Canada, do health insurance isn't tied to job.
If psychologists surveyed one of these companies, like Amazon, to look at the sorts of attachements the employees have now, and had as a child, I think you will find distinct deficit of people who had a happy childhoods with secure attachments sticking around at these companies.
The bravado and masochism so common in such places, in my opinion, is a form of self-bullying. You don't have to look hard to find managers exploiting this characteristic, and I suspect the most of the ones who thrive in such an environment do it. I worked a brief contract there and it was clear that the manager was used to making statements designed to trigger his subordinates' insecurities.
When your self worth is tied to the quality of your work, but your ability to self-evaluate is not solid. you are vulnerable to manipulation. I have that trigger too, it's just harder to hit, especially by some punk who is used to saying 'frog' and having people jump. I felt sometimes like he was waiting for me to react to some subtly passive-aggressive statement he just made and I just shrugged it off and stopped talking or kept going.
A master can be very tied to the quality of their work, and hand you your ass if you try to disagree with them. This doesn't develop overnight, and it's a winding road, but it's very undeveloped in a bunch of 25 year olds with a toxic cocktail of Impostor and Dunning-Kruger, and any 28 year old who spent more time honing their interpersonal skills in college has some ideas how to get what they want out of such people.
And those people think this is okay. Normal. That it's only what they deserve. And then we have a culture of avoiding psychotherapists, who would tell you two sessions in that this is bullshit and you shouldn't have to live this way.
you're 100% right
My first job out of college I had two bosses. HORRIBLE.
This came up recently, a manager, trying to be helpful, not sure they could manage someone said why don't we both manage them. NO!
The other things - trading sleep and shutting off friends - the first will destroy your quality of life and the second you will regret for a LONG time no matter how critical the day to day was.
I'm not perfect. Setting boundries. I focus on situations in my case - what situation lead to bad results. Also - you can sometimes trade time for reduced money - which in some of these situations is a NO BRAINER. I'm trying to do that. Society really IS NOT setup to support that.
I go home even if I'm behind. I sleep next to my wife and play with my kid. Covid has made catching back up with friends harder.
I think what folks from the outside don't realize is that folks making big money+ - you put a lot of pressure ON YOURSELF. You are given lots of money, often lots of responsibility, your work impacts lots of people (internally and externally). This is even if Jeff Bezos is not your boss.
Wow, I wish I had once a week. This is way better than my previous employer/role. I was on-call for months straight, and would get paged multiple times a night, every day, even at 2am, 4am. I had to carry my laptop to dinners, date nights, game nights, etc.
I got burned out and moved careers into a role that never has on-call. I don't get to code much anymore, but I've found other things to enjoy
It was horrible and in retrospect a really bad decision with the expected outcomes, mostly broken mental health.
At one previous employer, you were on a 2-week rotation for on-call.
But Every time you were on call, it was the same set of problems, always at the same inconvenient times (Mid sunday afternoon, random weekdays but always between 3-5AM, etc.) Admittedly often they were NOPs (Oh, alert came in, but when you check the system it's fine.)
But we didn't fix the false alarms. And we didn't fix the recurring real problems either; the Org's view was that was the entire purpose of the On-call person; that middle child you send to pay the loan shark not-quite enough because you don't care what happens to them.
At later employer, I actually had to deal with -more- on call issues, both in severity and frequency. And yet, I hate being on-call less, because at the very least when we find problems on-call the org puts a focus on fixing them.
'Encourage employees to be critical not just of ideas, but also of expectations.'
From my experience as a developer; I've never really been involved in the process of setting expectations. Something I find incredibly frustrating at times. It's like sure you can give your opinion but it's too late to make any difference. I think it comes back to the main point in the article about open communication.
I think open communication needs to be inclusive. However, that's rarely the case at least in terms of setting business expectations. It's only natural to want to be involved in shaping some objective or some expectation that impacts you and for which you may be one of the more qualified to discuss.
Nobody will ask you. You have to be upfront about it from the start.
Request time estimations and fully fledged requirement definitions from the start. Review them before accepting a project, push back against overly ambitious plans and propose changes that allow you to deliver value quickly and iteratively. It's easier to react to surprises if you are able to test the ideas as you go.
Stick to the plan as much as you can. Push back against changes to the requirements if they're not vital. Let others know about blockers clearly, as soon as you run into them. If you get behind schedule, communicate it properly. Stuff happens, unfortunately, and deadlines not being met is a part of life. You just get better at estimating with experience.
There's no way the upper echelons will adapt if they don't receive input timely and loudly enough. If your boss is any kind of decent, they will listen to you and trust your input. If they don't listen, then you should reconsider your current job.
A key difference is that the oncall engineer is usually able to directly influence the quality and robustness of the systems being watched. Sort of like preventative medicine, but much more direct.
Crappy teams make crappy software that pages often (or not at all if their ops fails to write any meaningful alerts)
If you work on a buggy-enough system, and don't do the root causing, then you will have incredibly bad on-call experiences.
I think talking about the Leadership Principles as a 'typical corporate motto' is unfair. Amazon lives their leadership principles far more than the other companies I've seen. They don't include things they don't actually care about, like most corporate mottos do (you'll notice the lack of 'we care about our employees' or 'do no evil' LPs).
At least in my experience, you only need to pay back Amazon proportional to (2 years minus duration employed) so it really isn't that bad (maybe that wasn't the case when this was written).
Maybe it's changed, but the signing bonuses I saw were essentially increased salary until your stock starts vesting seriously. It gets paid out and vests every paycheck so there isn't anything you need to pay back.
On the other hand at the end of year 2 when you have gotten used to the monthly payments I guess the we told you so is it bit harder to agree with.
Customer Obsession is "do no evil".
I think there is a real tension between customer obsession and 'do no evil'. Serving the individual needs of consumers can come at the expense of the public good. I think example of that are the amount of waste that Amazon packages generate by shipping as fast as possible and the amount of environmental damage AWS can quietly cause via their datacenters.
Relocation bonus would have had to have been repaid in full if I had left during my first year.
From the perspective of not taking advantage of the financial irresponsibility of recent grads, this is an improvement over the structure described in TFA, but the right solution is make sure that recent grads know what compensation agreement they're signing and can resist the urge to spend it all, that can't be left to employers, whose every incentives push against that. Big lump sum payment right out of college are very useful as emergency slush funds even if you can't responsibly do much with them for a few months.
Two things about it: it's not feel-good bullshit like the parent said. That makes it real. And another: so many principles are at direct odds. Do you "Insist on the highest standards" and delay the launch or do you "Deliver results" now? How do you "Think Big" while also obeying the mandate for "Frugality"? There is a lot of tension in the principles which matches real-world choices. It's not the shallow corporatese of most mission statements.
You're right. The proportional compensation is something I was very wrong about.
At the time, I was also in debt due to being straight out of college and having been dumb AF with money (student loans, car, & credit cards) and so owing anything would've been bankrupting.
It was so easy to be stupid with money when you're making an engineer's salary.
Not shifting blame, I did that debt to myself and I dug myself out of it after educating myself.
Just giving more context.
I was made an offer for L5 at Amazon earlier this year. For compensation, it's like you suggested: the offer is structured such that the first two years weigh more heavily toward cash (a separate signing bonus for each of the first two years, which IMO was pretty good and _does not_ need to be paid back, it's just combined with your normal bi-weekly paychecks) and then the latter two years are where you get the majority of your RSU rewards, which wind up being worth the same or more than the cash bonuses of the first two years (assuming I suppose that the stock price continues to be stable and/or increase from what it was at time of signing).
A note about the signing bonuses, the offer letter details basically do not say anything about needing to pay them back for leaving early. If your employment is terminated (by yourself or Amazon itself), then you just end with the prorated portion for the current pay cycle and that's it, you don't get any more of it (fairly obvious anyway if you leave a place).
I want it to happen. I'd like a word.
You could be shown the door in no-time. Amazon has forced attrition and the manager decides who gets the cull. They may make a mountain out of a mole to further their objectives.
I feel like it's common wisdom now that forced culling is not a great strategy and has lots of perverse incentives, but still not universal.
Do you know how they implement it? Is it the old Microsoft model with obligatory bell curve distribution of performance reviews?
Each manager of, say, 10 people rates their own reports.
The managers in the org compare notes, and count up how many poor performers they have between them.
If the number is below X, managers who did not downrate enough people are told to go back, and find more people to give bad ratings to.
If they can't find those people, that's fine - then those managers will get downrated.
OK, that's what I meant. Thank you.
I was part (as a manager) of a smallish org that started doing that after being acquired by a big corpo. I just left :)
The description in the post is not even that bad. A page a week is nothing. And it is clearly state that your workload is reduced during your on call period.
In practice, your workload may not be reduced during on call because it's "expected" and you still have to meet your normal deliverables while the team isn't staffed with "extra" people. It also takes a toll on employees to be mentally-available at all times in case some triviality emerges and raises an alarm. You'll find many engineers not willing to leave a small area or even go out for dinner without bringing their laptop and a hotspot, just in case.
- use a primary and secondary on call so that the primary can still go get groceries, or play in the yard with their kids, etc - overlap on call and work hours. I did that at a job and I was fine with working 6pm - 2am just to cover this time slot. Won't work for everyone, but may be a nice compromise for some.
Like everything else, there are ways to do it right, and way to go complete bananas with it.
Sure, agreed. Sometimes it's implemented well and sometimes it isn't. I've experienced some of both.
When it isn't well-implemented and is particularly non-critical, it can feel a bit dystopian to the engineers. If done well and for a good cause, it should be fine.
They typically aren't, you need to hire more people then. In anything defined as critical I would suggest that being well rested is also a requirement rather than having a team of people who are so overworked they develop mental illness.
I would much rather bite the bullet, be on call a week a month or so and use the budget for something else.
And I'm not talking about Amazon here. Of course Amazon can afford to hire everybody and their mom to cover 24 hours in a day. I'm under the impression that your original comment was really a critic of the practice in general, not just this specific example.
Some small companies build products they try to sell to bigger companies. These bigger companies have SLAs, so they need SLAs from their providers to sign a contract. Even at a small startup on call may be necessary to get customers.
It absolutely was criticism of the practise in general. Yes, small companies that can't ensure that there workers aren't overworked don't really make sense with protections like these. But the same is true for environmental protection.
Critical infrastructure arguably ought to be handled by companies large enough and with enough staff to comply with regulation like this.
For anything else I don't see the issue. Some random smartphone app company doesn't need to fix anything at 3 am in the morning, it can be fixed the next day. That's exactly the culture I'm critizing. 24/7 readiness to work for some random product at the expense of mental health is awful.
I gave you examples of how to implement on call without overworking your employees and even giving them the option to run a few errands here and there. So what's the problem? You seem to be against the practice just for the principle, not even really for any bad effets due to bad implementations.
As for which products are worth implementing "on call" for, yes I agree, some companies are too quick to think that the world can't function 30 minutes without their product.
Anyways, I've seen it implemented successfully at companies with very low attrition and I do believe it is a necessary tool sometimes for small companies to grow and sign big whale customers.
It doesn't make financial sense because companies can get away with exploitative behavior because software engineers are quite naive (or even actively detrimental to themselves and their peers) when it comes to labor relations.
I see this comment again and again on this thread "if it's really critical spend the money", but the thing is even small companies sometimes have critical systems. Simply because they're trying to compete with bigger ones, or they're partnering with companies that do have critical systems and the SLA gets passed down the chain of dependencies.
They could afford to 3x the teams if they wanted to. They just don't want to, because management prefers to just keep the money and burn out their subordinates. (There's always more where they came from.)
If a big selling point of your system is the ability to be on 24/7, on-call is inevitable. But it definitely can be way less gruesome than it currently is at Amazon for a lot of teams right now. For example, my on-call schedule is extremely relaxed and low-stress, it is never a worry I actually have or something I dread ahead of time. But getting rid of it completely is simply impossible for a lot of "always online" services.
hire more people?
Hiring more people is a great idea for making on-call less stressful, because it allows for much sparser on-call shifts (e.g., one week every 8 months vs. one week every 2 months). But I don't see how hiring more people gets rid of on-call completely. Someone will always have to be on-call.
You cannot just hire people who do on-call-only duties and nothing else, you have to be very closely familiar with the code and features being written in order to resolve issues effectively. Which usually means that you need to be someone who actually worked on those things, not just some glorified extra-fancy customer-support worker.
Being effectively at work 24 hours a day for seven days straight sure sounds like a recipe for burnout to me. As TFA notes, these people aren't doctors, the only thing that happens if a glitch takes two hours to fix rather than one is that Amazon loses some money. Amazon naturally would prefer not to lose that money, but its workers presumably would prefer to be able to sleep too.
Thanks :)
The reason for it isn't employee retention. It's to stop people from accepting a job from Amazon just so someone will pay for their move to Seattle and give them a free bonus.
Would people do this? Yes. I've seen them try. It also discourages people who aren't serious about the job.
Paying back a signing bonus if they fire you seems super rough. Is this common elsewhere? 2 years also seems high - the time I got a signing bonus, it was "pay back if you leave voluntarily in 1 year" which was much nicer than the described.
Unfortunately, much of this seems pretty common, especially months-of-crunch-time. Being able to figure out projects to avoid (if you're at a big corp like this where such things are possible), and being able to avoid them is a very good career skill to cultivate.
On-call can work quite well if the team has sufficient ownership to improve the underlying issues, not just keep the lights on. I've done this with good success. I'm often disappointed by how many managers and companies I've seen who don't get that aspect.
Believing too much in mantras like "adding manpower to a late software project makes it later" can be dangerous, too, though. How well it works depends on the manager and the person scoping and planning the work. Another critical skill to develop to escape being the person doing the grunt work.
I actually don't think this is true. My understanding is if you're PIPed they don't really care. The signing bonus is paid out monthly the 2nd year (and for people hired at SDE2+ its first year too) so you don't have to pay it back after the first year.
Disclaimer: Amazon L4
I've interviewed at Amazon and had a lot of conversations about compensation. I don't know about the terms of the relocation bonus, but for the signing bonus, you don't pay it back if you are fired and even if you leave voluntarily, you only pay back a pro-rated amount. For example if your signing bonus is $20k, and you leave at the 6 months mark, you owe $10k back but you get to keep the other $10k.
When I graduated college, all of my friend group discussed it and we all took out a small chunk of the bonus to go on a small group vacation, but the rest of the bonus got put into an investment account and not touched for at least a year. That way if we did decide to leave, the money was easily accessible to be paid back. I know not everyone has that flexibility, but it's probably the best way to approach signing bonuses if you're not 100% sure you will be at the company long term.
Part one is in your first year where you get a lump sum signing bonus with your first paycheck (paid monthly) with the understanding that if you leave you would pay back a pro-rated amount proportional to how many months were left in the year.
Part two is in your second year and is paid partially with every pay check and if you leave you don’t pay anything back, you just don’t get the remainder of the money to be paid out the rest of the year.
After year two stock vests tend to vastly outpace the signing bonus pay but I’d imagine this varies on the stock price.
also, anecdotally, it you were fired they would not ask for the bonus back - it opened a whole can of worms in the wrongful termination bucket
so op definitely did not read his employment contract
I don't work at Amazon but the situation is very similar to my job. On-call can be brutal, even if there are no incidents during your week (which is rare.)
Just the fact that you have to be 24/7 _ready_ during a whole week sucks, especially when you have a family. During my on-call week, I can't go get groceries, I can't take my kid to her hockey practice, we can't go out for lunch over the weekend, etc.
It takes a toll, for sure.
"on call" was super, super vague and I pushed back hard on it. If they require me to work outside something reasonable like 6am-6pm, they will pay me at least double for it. No exceptions.
I wish everyone would push back that hard, and that on-call was an official opt-in "extra pay" scenario.
Needless to say I turned down their "promotion" even after months of them trying to make me take it.
Note: I am NOT saying I'll work 12 hour days. I'm just saying if I have to come in early for a meeting for 6am, or stay until 6pm sometimes, that's fine. If I'm expected to come in or be on call at 2am, that's a different story.
I’ve found that just having someone else to commiserate with at midnight, if needed, really helps the mental state of on-call.
Then next:
> are you willing to work nights and/or weekends
To be fair I'd say this is mention enough.
If someone is willing to (or has to) look the other way at the time of hiring when something like this was explicitly said, then even a "hey, we have on-call here, you fine with that?" would be okay.
However on the interviewee's side, if you see something in a job description that asks if you're willing to work nights and weekends, that should be an immediate indicator to you that you should ask why you possibly would have to work nights/weekends and in what capacity. It's the interviewee's responsibility to ask questions like this even if the employer isn't immediately forthcoming.
My only advice to this person is:
- talk to someone who does customer-facing tech support in your own company
- never take a job in customer-facing tech support
I worked in high volume manufacturing where process engineers could be on-call 24/7-365 unless you explicitly asked a backup to cover you during a vacation or weekend. Depending on the part of the process you owned, that might mean a middle-of-the-night call a couple times a month or several times a week. No human lives on the line, just lots of money.
Different software products have different severity levels. For my team, the oncall burden is not so bad. We have an internally facing tool. When we do get paged, it's usually requires acknowledging a ticket, a few minutes to validate something, sending an email out, and that's it. For teams with more externally visible products, a particular page can be stressful, but my understanding is they also spend a lot of time in ensuring system robustness in order to limit the number of pages. The "15 minute" response time the post referred to is generally to acknowledgement - the expected time to resolution can be quite variable.
The working hours are generally pretty good as well. The general expectation is that you're not working crazy hours.
I've ran into "project crunch time" at Amazon, but the experience at Amazon was like everywhere else I've worked. This also will vary by team: teams that need to make changes to support Black Friday obviously have a pretty hard deadline. Or if your team is getting ready for a re:invent announcement, there's no way to work around that. But from what I've heard, teams that have such a hard deadline also have a corresponding quiet period in December or January.
Most people I know within Amazon that get burned out on one team and feel like there's no resolution within that team (or just feel like they've had enough with one area) solve that by switching teams. This is generally pretty easy, since most teams are hiring at any time, and they would rather take an internal transfer than an outside hire.
Amazon has a pretty good system around having a senior individual contributor track (I haven't worked for any of the other big tech companies, my understanding is that they're similar). Part of the benefit of this system is that SDE1's/SDE2's have someone to reach out to for helping navigate management. Anyone at Amazon who reads this who is experiencing something similar to the OP should feel free to reach out to one of the senior engineers for help.
Overall, I don't doubt that the post in the original article is that person's experience, but it has not been my experience.
Before covid happened, I was interviewing for a SDE-2 position within AWS Canada, so it would be of great value to me!
But when it descends into perma death march and your manager flings you "a few" days holiday in recompense... does that make up for it? If it doesn't, to whom has the profit gone?
Exempt salaried employees aren't paid by the hour, and don't have a set number of hours they are expected to work. There is no overtime, but there's also no (contractual) penalty for working fewer than 40 hours in a week.
1-datapoint (@India): Their interviewers have uniformly managed to be both obnoxious, and somehow simultaneously also quite insipid.
I understand that this person was coming straight out of school, but the onus is on the applicant to figure out more about the job during the interview process.
If the Prime TV app was better I'd honestly probably cancel Netflix. But as-is I avoid using it completely.
Then I interviewed for a mobile position at Amazon. Now I understand. A Facebook iOS recruiter laughed at my experience.
Uh, not even close. I used to get up at 4am EST to trade European markets for a hedge fund. I was on call 24/7/365. I had my laptop and Bloomberg fingerprint scanner with me on any trip I took.
And this is just my experience. I have had many friends in engineering, management consulting, fin tech, etc who have similar demands made of them.
I’m not saying it’s right, but it is part of the rules of the game that we all sign up to. There’s a reason all of the aforementioned jobs pay so well.
I think oncall is a common requirement for anything involving large sums of money, treatment of high risk injuries, or geopolitical power struggles.