To what extent should personal ethics play in deciding where a (software) engineer should work? What if the engineer has no other choices for a job, and needs to (make rent/pay the bills/eat food)?
It is true that it'd be pretty easy to circumvent the tracking, by placing paper over the webcam, running the browser in a virtual machine, spoofing data to the browser, and so on. However these are infeasible for most non-technical people, so I don't think it's a real solution. Freedom shouldn't be only for those with extremely technical knowledge.
To an even greater extent, ethics is a serious issue in network engineering as well. What happens when you're an employee of an ISP in Turkey and you're asked to MITM all port 53/DNS traffic? Or implement something like DNS lookup redirects to an advertising page? Censorship? The UK GCHQ and snooper's charter requires you to run a transparent http proxy and keep logs of every site every residential customer visits?
You work for a Thai ISP and the government orders you to block everything that insults the king and royal family?
You do network enginering for a mobile phone network operator in a developing African nation and they want to redesign parts of their network to support participation in the Facebook "free basics" walled garden internet for your GSM/UMTS/HSPA+/LTE type customers?
edit: there are a lot of things you can do to mess with a properly functioning internet, on the behalf of autocratic regimes or greedy corporations, just at OSI layers 2, 3 and 4... That's before you even get to the level of operating system and applications/software engineering.
I don't personally see anything wrong with these remote, screen based, assessment requirements. These are to curb any chance of cheating. If we focus on reason instead of jumping the gun to privacy violation.
Coming to your question. First question is easy. In a business environment, subjected to international rules and regulations; ethics are a priority. Second one not so. Kind of similar to when does it become acceptable to rob someone if robber can't make ends meet. Depending on jurisdiction; court will decide level of leniency.
Okay, but that's sort of a self-enforcing rule, isn't it? The very fact that you can figure all of that out enough to fool the monitors means you're qualified for the role.
Kinda like, "haha! I secretly got messages delivered to me during the steganography test! It's obviously not a legit test."
Wait, I see nothing wrong with any of this. Obviously Amazon is trying to curb cheating on remote coding/compatibility tests. They don't actually care to collect any info about you beyond what you do during the test. They don't care which websites you visit before or after the test. Or how you use your clipboard - after the test. But during the test, it's not so unreasonable. People do cheat, especially if they can get away with it and when the payoff is potentially huge.
Yes, I agree that cheating should be curbed. But not at the expense of tracking every running application on a computer, or the exact location of interviewees. Videochat and on-site interviews exist and don't have these kind of problems.
Ok, but any techie should have little problem preparing an environment for such a test. Use that freshly formatted, dusted off laptop from college? Maybe even consider using a VM, like others suggested. Or how about install a portable version of Google Chrome, so no existing browser data or extensions would be a cause for concern (or worse, imagine a buggy extension crashing the browser and, thus, ending the test session)?
But the real question is: should they have to? Quantumtremor already listed several alternatives that indeed don't have the same problems. Why would you risk ruining the goodwill of your privacy-conscious applicants by making them format a laptop, create a VM, or even install another application. None of that is going to make me want to do your interview.
This is a great point, and I address it at the end of my post. The basic idea is that freedom shouldn't be limited to people who 1) have the technical knowledge to do all this and 2) have the time to set up such an environment. Freedom should be default. By being OK with these kinds of testing platforms, these could eventually be used for non-technical positions very easily, and those applications wouldn't know how to circumvent this.
But Amazon is primarily a tech company. Hate to say it, but non-techies comprise a very small percentage and they're an edge case. I also think that there is little chance that Amazon would care much to impose such strict oversights during non-technical remote tests.
Video and on-site interviews are much more expensive and leave employers open to lawsuits. Not that I necessarily agree with this practice, but I could see why this approach would be considered.
In a practical sense, it's not much different from a math instructor having you clear the memory on your graphing calculator before an exam; it's an anti-cheating measure and given that this is an online assessment, it almost makes sense.
However, from a philosophical point of view, it's Draconian and unnerving. What assurance does the interviewee have that Amazon isn't storing and using all this telemetry? Amazon is dictating the entire procedure, right down to the specific browser and extension, and the interviewee has no control over this. In fact, I have to wonder if the job seeker's acquiescence is part of the assessment; if they will bend over and take this massively invasive procedure, perhaps they will make for a good little subservient drone at Amazon.
I'm sure that the candidate has no such guarantee. But does some kid barely out of college care more about potentially landing a job at Amazon or about the small chance that Amazon will retain some info collected during the test? It's a tradeoff, but it's nothing compared to the tradeoff we make by using Facebook or Google services.
> It's a tradeoff, but it's nothing compared to the tradeoff we make by using Facebook or Google services.
I don't disagree, and I recently stopped using all Google products in part because of how much information they have collected and still want to collect from me. After the account suspension drama from several days back, I was spooked and came to realize just how dependent I was on the company, and what I stood to lose should I come into the crosshairs over a simple ToS violation or even just a random bug in their service.
But that only strengthens my point of view about this hiring process and how wrong it is. When a company wants to exert that much control over someone who doesn't even work there yet, it speaks volumes about their overall culture and attitude towards people in general.
> "it's not much different from a math instructor having you clear the memory on your graphing calculator before an exam"
This sort of behavior by teachers is exactly what got me to write my first serious z80 assembly program to mimic all of the menus on the TI-83 circa 1996. Teachers had already caught on to the fact that there were programs that would display the "Mem Cleared" message on the calculators, so mandated they had to do it manually. I never used my program to cheat, I just didn't want to lose my games (had a really good zelda clone & penguins).
I can assure you that, for the vast majority of the population, it has the intended effect of keeping the calculator from storing digital cheat sheets.
I do not disagree with your assessment, but it was a great motivator for me to learn z80 assembly. I just wish I knew how to use the stack. Modern me shudders at how horrible that code was - it worked, but god damn was it ugly. I still have the Z80 assembly reference manual on my bookshelf. It just hasn't been opened in close to 20 years. I've never had a physical x86(_64) reference manual, but I usually keep PDFs handy - just in case while debugging. I think I still have Atmel AVR & Motorola 68k docs physically somewhere...
> However, from a philosophical point of view, it's Draconian and unnerving. What assurance does the interviewee have that Amazon isn't storing and using all this telemetry?
How is that different from being recorded by multiple cameras during an on-site interview?
I honestly don't see why the information they're collecting is a big deal. They are remotely interviewing you for a position, and would like to minimize cheating as much as possible. Instead of sitting you in a room, they use the technology available on your computer, while clearly telling you upfront what kind data will be collected. If you don't like it, get an interview elsewhere.
saying "if you don't like it get an interview elsewhere" is pretty unhelpful, because that's pretty much what he did; refused to continue with the current process. it also doesn't really work in the real world, because nothing would ever get fixed. if there were awful terms in a license agreement and someone found out, it's all well and good to say "go somewhere else", but what if it's in something like, say, windows that many people are forced to use for various reasons. sure, you might be able to go somewhere else but the practice still needs to be criticised and shown to others
well, as i said in another comment as well, if your questions can be answered by a quick web search, then maybe they should think about how you actually work; you web search a LOT. ask questions that filter on something less obvious
there are still in person stages; this is just a filter for them. it's not like it's the only data you get to hire or not hire someone
Why is everyone assuming that a cheater will only do a "quick web search"? You could call someone to walk you through it over Skype, or get a person to solve it over remote desktop, or even have a friend sit in and do it for you, etc.
These are the cases where monitoring is needed. Whether or not what Amazon is doing is actually effective is another issue entirely. All I'm saying is that it's not as huge a deal as a lot of people on this thread are making it out to be.
>while clearly telling you upfront what kind data will be collected
It doesn't even satisfy the legal requirements for informing people their personal data is going to be collected (in my country at least). They don't tell you what happens to the data, who has access to it, they don't provide a way to let them know you want it deleted, and so on.
'What if Amazon is using the data for nefarious purposes'.
It's a reasonable conclusion, but also a big 'what if' :)
Their motivation is pretty reasonable - they just want you to be able to take a test remotely.
They 'dictate the process, right down to the extension' for reasonable cause.
It's reasonable to ponder the ways in which there could be bad things going on, but I don't think it's reasonable to assume that there is, or even malicious intent here.
While it's true that a follow-up interview would further determine a candidate proficiency, the remote interview is meant to be a weed-out test. If enough people cheat, then it becomes useless for Amazon. And yes, I've heard of people cheating on interviews, including for Amazon.
And yet...I have seen people cheat on technical interviews conducted over video chat. It's pretty obvious in that situation — oh, where did that come from? — but I can see it being an issue in a highly automated interview process with minimal human oversight.
Of course, I think that the real answer for that is to use a human to interview everyone. It's crazy to treat potential future employees this way.
Had something similar while interviewing Chinese offshore developers from Accenture. We caught on that the translator was basically answering the questions, and not the candidate. As a result of that experience, I will not work with anyone that does not speak English (I don't care if it's not their first language - I can work with poor grammar & broken English and I can handle heavy accents over a phone), but I will not go through a translator (and I don't want to have to go through a translator for day-to-day work, anyway).
But what you can say is: "If you want me to jump through all of those hoops, send me a $99 tablet that's already configured for the test conditions you want to subject me to."
More than once I've conducted phone screens and heard candidates attempting to look up the answers to questions online. (Hint: it's hilariously obvious when you ask something that follows the previous answer, the connection goes silent except for a few clicks of the keyboard, and the candidate then suddenly has the answer that sounds a LOT like it came from StackOverflow.)
I've even had a candidate whisper the question to someone else in the room with them. Seriously, it's ridiculous what some have done.
Asking questions that give an incentive to cheat are typically poor questions in my experience. Or maybe an indication that the job is gonna suck a little bit.
We used to give people with no real experience or portfolio small homework assignments to do - one of them forwarded it to his friend, asking him to complete it for him - but accidentally CCed my boss...
If this test was happening at their own premises on their own machines then sure. But if I personally had to do this on my own machine on my own premises without being notified beforehand? My response would have been "Please erase my details and never contact me again". My current employer has very strict privacy and confidentiality concerns which I respect. But a prospective employee who has no contract with me? They can kindly GTFO.
What's wrong with it is that real programmers constantly use whatever means possible to get the answer to questions as fast as possible. What they describe as "cheating" is what real programmers do all day long.
Unless you are working with a really small API surface then it is unlikely that a programmer is going to have much of the API memorized than the ones he uses on a daily basis. And with people radically changing their stacks every few years it is unreasonable to expect it.
Also, people copy/paste/tweak things from StackOverflow all the time. Disabling clipboard is unreasonable for that reason.
What a programmer can do using only things completely in their head is quite different than what programmers do in reality. Programmers are constantly looking up things they don't know throughout the entire day.
Keep in mind too that online tests are for screening, not a complete interview. You bring people in after that and do several rounds of interviews. Under that circumstance having a friend next to you or supplying you the answers doesn't really get you far if you know you have to do similar tests once you get into the office. There's isn't a lot of incentive to cheat.
The bigger concern is that this is a red flag about the work environment / culture. If I saw that I wouldn't go anywhere near it as well. In fact I deliberately put things in resume that some companies might find objectionable in order to purposely weed out with such cultures.
Given that they've defined "::" differently to the usual "as" the answer could be anything. More specifically, the answer is f(64), where f(x) is wharever was in the head of the guy who made the question.
The same goes for the "what is the next number in this sequence" questions: unknowable given the information provided.
Or you know, having another computer? I literally have 12 computers or phones in my house. Could I not use another one if I needed to cheat that badly?
Tape over the webcam. If they ask, just say you always gave tape over your webcam for security reasons. You could also use a computer without a webcam.
You can use an HDMI splitter, and merge the videos of the two computers. As overlay. You can also program the second computer to have a mouse controlled by a single muscular activation in your cheek or tongue – for example, if you have a tongue piercing, put a magnet in, use a phone to recognize the magnetic field, tape the phone below your desk (it’ll measure the difference from movement), and connect that to the PC.
No way to recognize it for them, and you circumvented it entirely.
They’re trying to fight against people whose job is to circumvent weird complicated restrictions, and finding workarounds.
No, it feels like Amazon is over-reacting, and unduly distrustful of the bright young people who would wager their careers and reputations to come work for them.
And if they do, it'll probably only be for the minimum term anyway given what an awful workplace Amazon is. I live in Seattle and hear Amazon stories all the time.
Yes and no. I think Amazon is trying to make their intern recruitment process more efficient. By eliminating cheating possibilities, people who do not meet certain criteria will not move forward and thus there will be less phone screens to schedule.
I'm not saying that there's no "benefit" to these restrictions. It's just that (1) they address problems that can be easily solved otherwise, and (2) they come with substantial negative side effects -- like turning away any candidates with a shred of self-respect and/or some notion of their true worth (even as a fresh engineering grad) on the open market.
"turning away any candidates with a shred of self-respect and/or some notion of their true worth"
Given that this thread seems to be fairly split on this issue (at least at the time I'm writing this) this seems like a bit unfair to say. Personally, I wouldn't give a shit, and I like to think that I have at least some self-respect.
If an employer is so disorganized that this doesn't become obvious during the first month of working there, they probably have much greater problems than cheating on their idiotic cargo cult coding tests, and fixing the cheating would do absolutely nothing to save the company.
"this doesn't become obvious during the first month of working there"
This is a gross oversimplification of the problem. Do you know how difficult it is to fire a person? Do you know how much it costs to hire someone? You've given this person a relocation bonus, possibly put in the work to set up a visa, given the first chunk of a signing bonus....
I'm so tired of the assertion that it's "difficult" to fire someone, even insofar as "difficult" means "HR won't allow it." HR's reasons are stupid, particularly if you're in a so-called "right to work" state. I work on multi-million dollar lawsuits for a living, and in every single employment suit I've worked on involving an individual who was fired from a large organization, that person had already been through some kind of PIP.
It does. I'm talking about the fact that the fictional employee we're talking about is most likely non-union in a right to work state, thereby making it easier to fire them (without union intervention).
When talking about employment law, "the right to work states" is often used as a generic category used to refer to states with weak employee protections, since the right to work states generally are also red states: https://en.wikipedia.org/wiki/File:Right_to_Work_states.svg
Can you find an example of where "right to work" is used as a generic category instead of its actual meaning:
"Right-to-work laws are statutes in a reported 26 states in the United States that are an effort to give employees the right to work without being required or compelled to join to a union."
You have no idea how deep the rot is. Here is a classic scenario I know of. Large ships have plenty of space for rats to hide.
Large American Inc. is a large company we all know off and which is always somewhere on the front page for some news. Mr X is a manager there how arrived from India on H1B and now perhaps has a green card.
Mr. X brings his younger brother Mr. Y on F1 visa. Mr. Y then joins a consultancy. Consultancy claims Mr. Y has 8 years of experience, knew AngularJS before it was invented (true story) etc. and submits it to Large American Inc. for a project that Mr. X is spearheading.
The consultancy then send a proxy for phone screen and real interviews. The consultancy, Mr. X and Mr. Y all pull a lot of strings here. favours are exchanged.
Mr. Y then joins Large American Inc. as a consultant with a salary of $120K under Mr. X.
Mr. X and Mr. Y conceal the fact that they are closely related.
In many cases consultancy hires another person in India whose job is to give 24x7 phone support whenever Mr. Y has to do his work. Screenshare etc.
Why would you cheat, though? You're going to get the job and probably be completely incapable of doing the work. Not to mention there's usually an in-person tech interview afterward.
The following information will be collected during the duration of the exam:
Your microphone
Your webcam
Your physical location
Your head movements
Your eye movements
Your mouth movements
Creepy as all get-out. By all means, lets have more leaks like this.
Keep in mind, this is during the test. What are you going to do, take a break and load up some porn?
These conditions only apply for the 24 minutes in which you take the test. How is this different than if a person were watching you during an interview?
Keep in mind that if you have extensions installed, many of those extensions will make callbacks to other services. This will track the use of those, too.
Paid homework projects are far superior. Plagiarism checks can be run on the provided solutions, and a follow-up conversation concerning the project at the interview should easily suss out anyone who had a third party to do it for them.
That approach also has the advantage of actually being a relevant to the position—unlike some generalized intelligence test.
Frankly if people are putting up with the test they have now, Amazon could probably get away with issuing unpaid homework projects, however classless that would be.
I've found that the answer to all this is to give a 30-60m remote test (no proctoring), where they get to write the thing as they want, and a 15m discussion where you analyze their solution and assign one new feature then let them walk you through the changes. No code, no whiteboard, just discussion.
A lot of people can't whiteboard code from scratch but can take a piece of code they've just written and discuss next steps. And it's also an actual work skill that many good coders are bad at, so it gives you a good insight into high and low-level people.
THIS! This is the real issue. Such unfettered access presents the risk that they won't just collect data on what happens during the 20-something minute window. The risk is that they gather files, collect your browser history, and read your communications.
>> How is this different than if a person were watching you during an interview?
It's different because they're recording you, not to mention they're doing it by installing spyware on your computer. That's not what a person does when they watch you, unless they're watching you through a camera.
I don't see how this is creepy at all. If you're building a product (especially in the entertainment field) and you want to get feedback from users a really good way of doing it is by having access to their facial expressions as they go through your product. This should be done with their consent (such as in this case), so I don't see the issue.
they weren't building a product. it was a job interview, he was at home, and this was software they required to be installed on his own computer (at home)
The same idea applies. The company wants to ensure people aren't cheating and also wants to get some insight into how people respond to their interview process. This only improves the process for future interviewees.
if you're interviewing someone and your interviews questions can be answered by a quick google, maybe you should think that during the course of a work day, the same applies and you should think about a test that actually tests skills involved in the work the candidate will be doing
i have no problem with a candidate doing something totally by themself in part 2 of 5 before in in person interview; theres still another 3 different interviews (many of which will be in person) to catch cheating and such
It's worth noting that these are Proctorio's (https://proctorio.com/) restrictions; they're just a third-party company that Amazon is using to administer the test. I've used a couple of similar systems (e.g. ProctorU - https://www.proctoru.com/, where a human will actually watch you via webcam(!), or ProctorTrack - http://www.proctortrack.com/), and "monitoring" of this sort is pretty much par for the course (no pun intended).
These companies are typically used in settings where some level of rigor is expected (e.g. administering an exam for an accredited university). In that context, I think these restrictions make sense in order to ensure some degree of validity in the test results (especially since you're only expected to run the software for the duration of the test).
It just sends the wrong message. As a company, you are advocating tech that spies on people or in this case the test taker. Overall, it leaves a nasty taste in your mouth that a company would use technology such as this. What else would they be okay with doing once you start working there?
Why not do an onsite if you want to make sure of the validity?
Depending on which definition of "spying" you want to apply, it is spying. Moreover, it is invasive and unsettling.
Perhaps most critically, it is a more subtle test for the prospective employee-- are you the kind of person who would comply with a ridiculous request that violates your privacy in the hopes that it will land you a job?
I dunno. If I were an employer and an applicant actually installed this without question, it would make me seriously reluctant to hire them-- this is the sort of unquestioning, docile employee who could be more vulnerable to social engineering,who would not be likely to consider consequences of their actions, who does not take security, privacy, and safety as a priority for even themselves (and is therefore less likely to do so for customers and clients).
These may not be qualities that some companies value, but from my perspective, I think the "nope" letter to Amazon shows strength in independent thinking, creativity, and moral character.
It's like those companies that required you to hand over your social media passwords so HR could dig through your history.
That is horrifying. If Amazon doesn't hire you because of this, it says quite a bit about their company, fuck 'em. Their loss. There are companies out there that actually respect their employees.
I think you're reading too much into this. It's to make sure that the interviewee isn't trying to game the system. Pretty simple. You can opt-out anytime you want. You don't even have to start in the first place. It's not spying by any definition of the word. Unless you make one up for it.
I understand that the goal is to stop cheaters. That's an admirable goal, and a perfectly legit one. However it's the completely wrong way to do it and sends a very strong message to anyone applying that "our company, for any reason that serves our interest, expect you to be compliant, to give up your personal privacy, and to submit to our invasive internal processes whenever we feel it's necessary. Expect more."
Asking this of you, before you're even working for them is presumptive and not a good sign that this is a place you're going to want to be. It's basically a huge flashing "We Fuck Our Employees!" right there on your job application.
It's probably the cheapest and most efficient way to screen candidates, and ensure accuracy. Not disagreeing with you -- just saying that their primary goal is probably efficiency (no surprise).
> Depending on which definition of "spying" you want to apply, it is spying.
Without the secrecy aspect, I don't think you can say it's spying by the definition people typically use. If you went to an examination hall and took a real proctored exam [1], would you consider that the proctor was "spying" on you?
> Moreover, it is invasive and unsettling.
This, I can't argue with. It's hard to pin a definition on what makes things creepy or unsettling. Some people find clowns creepy. Why? I have no idea. But there's no sense in arguing about it.
Personally, I think it's a clever use of technology to streamline and scale a traditional process (exam proctoring) that avoids making you travel to an exam hall or interview.
> Without the secrecy aspect, I don't think you can say it's spying by the definition people typically use.
I mean it in the sense of "I went on a blind date, but my parents sat at the adjacent booth to spy on me". I knew they were there, but it was inappropriate invasion of privacy.
> If you went to an examination hall and took a real proctored exam [1], would you consider that the proctor was "spying" on you?
If I was taking the exam on a personal laptop and was handed mysterious software at the examination hall to install on my machine... (along with images of the inside of my home), then yeah...
>Why not do an onsite if you want to make sure of the validity?
Convenience and logistics. Is it worth yours (or Amazon's) time to pay for travel expenses, when applicants could literally be located anywhere? Evidently not, at least not in the early stages of the process.
Is it worth any applicant's time to participate in a silly dehumanizing screening process that boils you down to a number before ever speaking to someone?
> dehumanizing screening process that boils you down to a number
That's a very long-winded way to describe a prescreening questionnaire. Are you saying that you think no one should ever take a job if there's a prescreening process?
If it's not worth Amazon's time to interview me in person then it's not worth my time to consider them as a potential employer. This is just lazy and they will end up with lower quality candidates because of it.
I agree with your reasoning, i think the alternative perspective is that traveling somewhere to interview is a significant time commitment on my part, and I might welcome alternative testing approaches where I don't have to fly somewhere until I'm reasonably confident I'm going to get an offer.
They will probably fly you out after this, for another stage of the interview. But at the "FizzBuzz stage", they do really need to make sure that the person answering the question is the same person they're later flying out. There are a surprisingly large number of college students who've gotten used to "paying for grades", who attempt things like that.
Because the point of FizzBuzz is not to get a working FizzBuzz class that can be integrated into an application, it's to see if the candidate can actually produce a working version themselves.
Imagine you're hiring someone to do your taxes. Sure, you'd expect them to use a calculator to do all the hard addition/subtraction, or more likely spreadsheets. But if you asked them to add 5 and 3, and they said "I would use a calculator, this is a stupid question, I can't just tell you off the top of my head" then you're probably not going to hire them.
It's a super basic test of 'Can this person code themselves out a paper bag."
which can be determined entirely without the aforementioned methodology. if you read the guys article, you'd see that he'd already completed the basic tests demonstrating basic competency.
being able to solve problems is more important than how you solve problems. (generally speaking, yes i know there are exceptions)
And that's why Amazon's phone-interview coding exercise is a rather complicated coding question delivered under proctoring, rather than simply fizzbuzz itself: to actually filter those people out early and avoid the cost of flying them to Seattle, rather than just "going through the motions" of doing so.
I interviewed with them a decade or so back. First interview was a phone screen. Ok makes sense. Second interview was another phone screen. Never mind the fact I lived about ten miles away.
On there other hand two years ago I interviewed with a company on the other side of the country. After a half dozen phone interviews they decided to fly me out... For more interviews
This seems to be a way for them to screen for the 1000s of entry level applicants, who don't have large supporting work histories. They need to screen them somehow. One way is "Only from this list of schools." Another is "Based on what some person thinks of your resume writing ability as a new engineering grad." Another is "Based on what our algorithm thinks of your ability to place keywords in your resume." I don't think this is that much worse an alternative for those who may not have other means to get into a prestigious company.
When I was last in school I struggled with "networking your way into a company" and wound up landing my internship (and ultimately full employment) on the one company who offered a test before the "Fly you out to meet us" step.
Isn't a normal class room in which you take tests while required to be silent and with an instructor/TA/proctor keeping en eye on you very much akin to spying on people? (using people rather than tech, but why is that distinction important?)
I'd be more concerned about the the level of access the monitoring software has to my computer and any services with vulnerabilities it might leave behind.
I think I'd only consider doing this if I used a computer I could wipe completely before and after. Even then the amount of distrust is off-putting.
Not necessarily, because that classroom proctor is not asking to install spyware on my personal laptop.
I can understand why ProctorU does what it does, but that process puts a significant burden on the test taker, one which is entirely different than taking a proctored exam in a classroom.
I don't know what is so wrong with this. It's only for the test duration when they want to ensure that you don't cheat. You can remove the extension once the test is over.
I don't know what is so wrong with this. It's only for the test duration when they want to ensure that you don't cheat. You can remove the extension once the test is over.
I took a test with ProctorU a few months ago for my university class. It didn't seem creepy at all. They installed LogMeIn and watched my webcam, but as soon as the exam was over they disabled both. It's just a way of making sure there isn't any cheating.
Yep, and a pretty convenient one at that. When using these services, the effortlessness of taking a test in the comfort of my own home (at a time that worked for my schedule) outweighed privacy concerns, although I certainly empathize with those that don't feel the same way.
I can't believe what I'm reading. A university installed effectively spyware on your device, to watch you as you were sitting an exam?
I sat about a dozen university exams in the last couple of years and in no instance did I have someone staring at my face, or peeking over my shoulder, during them. There were always supervisors in the theater, but they mostly walked along the aisles of desks and made sure to be unobtrusive and quiet as a mouse.
I really don't see any justification for what you describe.
Had the same experience. I passed my PRINCE2 (similar to PMP certification in the US) with ProctorU from my home.
I connected with a Lady via cam and she asked me to show via cam in a 360 style if I had anything else in the room I was in (books, someone helping,etc).
Took the exam after that and passed it. It is a bit unusual but very very convinient. If I had to take the exam physically somewhere I would have needed to take a day off.
I really don't see any justification for what you describe.
In the case you describe you have to be physically present at a fixed location at a fixed time. In many cases that might be inconvenient or impossible, and I can certainly see situations where it'd be nice to have an alternative. Not having to have my physical location coincide with the physical location of my university of choice could be quite handy.
Seems like we're ok with using technology for telepresence, but would prefer standard apps like Skype or google hangouts. The question is why standard apps aren't sufficient.
This immediately brings to mind the history of scientific racism, specifically with craniometry being used to 'prove' that certain races are inferior. I realize this probably isn't what they're doing, but the fact that they're giving themselves permission to store facial data for later analysis seems questionable given the context.
> This immediately brings to mind the history of scientific racism, specifically with craniometry being used to 'prove' that certain races are inferior.
I can't imagine why...? That would be like seeing a freeway and instantly thinking of the Nazis.
US corporations have a multi-hundred year history of using thinly veiled proxies for race as part of their hiring practices. This continues to this day, including in the tech industry.
It's not exactly that far fetched to think that someone might someday think it's a good idea to start applying some of those new AI APIs to the candidate interview videos just to see what happens. In fact given enough time and enough departments/companies, it frankly seems more likely than not.
Especially given that we already have AI scouring GitHub looking for people to recruit. Assuming it's not already happening, it seems unlikely that there won't be startups applying AI to hiring interview videos within the next couple years.
> US corporations have a multi-hundred year history of using thinly veiled proxies for race as part of their hiring practices. This continues to this day, including in the tech industry.
That's a pretty serious accusation, one that could spark a lot of successful lawsuits against those companies -- especially in the current political climate in the states they tend to be located in. Do you have any specific evidence to back it up?
Google won't you apply to work there without giving your college GPA, despite the fact that all the academic research, and also their own internal research, shows that GPA doesn't correlate at all with job performance.
Using GPA as a screening measure when members of a racial demographic routinely preform more poorly due to socioeconomic factors, it starts to look more like a proxy for race.
I don't think Google is asking for GPA anymore. Not for experienced hires, anyway. I can imagine they might still ask it for interns and college hires, where it's relevant. But they didn't ask me two years ago (I was seven years out of university at the time).
If some corporation did want to be racist (which has not been established), they'd hardly need fancy face-analysis technology to do it. They could, you know, just look at the applicants.
How is a computer looking at the applicant data, including their faces, more insidious than humans looking at it? In either case the company could claim they were totally not paying attention to skin color.
You should check out the history of IQ testing particularly in the US. On paper, it seemed to be a perfectly objective way to determine an individual's intelligence. But correlating the data, it was later shown to disadvantage blacks and that is why IQ testing for jobs is now illegal.
> It's worth noting that these are Proctorio's (https://proctorio.com/) restrictions; they're just a third-party company that Amazon is using to administer the test.
If Amazon has chosen to use use Proctorio, with an understanding of how it operates, then it's very much Amazon's restrictions.
No, an in-person interviewer will not know the tabs on my browser or the programs running on my computer.
Also at an in-person interview, the interviewee has a symmetrical power relationship to the interviewer in that they can both observe on another. This practice slides the power relationship heavily toward the interviewer.
Furthermore, at an in-person interview, the information is not analyzed in an unspecified manner which further skews the power balance toward the large corporation.
Finally, at an in-person interview, the information is not digitally stored at an unspecified location for an unspecified time and transmitted to unspecified parties. Nor is it subject to unauthorized access by unspecified parties in unspecified countries due to unspecified security arrangements.
The interviewer is most certainly looking at your computer screen if you are using it during the interview.
I will probably never be in a situation that comes close to the staggering power asymmetry of someone deciding whether or not I will have a job. Even if I'm charged with a crime the court has less power over me than an interviewer, in that its decisions are subject to juries and appeals and I get to watch the proceedings.
The information is analyzed unscientifically according to the personal biases and tastes of the interviewer to form an emotional impression he may not even be conscious of. "Not enough eye contact, I don't like this guy." I'd much rather it be judged by an algorithm trained on actual job performance years in.
Ever heard of Greenhouse? Your interview performance (as reported by the interviewer) is absolutely stored indefinitely in some 3rd party's database subject to unspecified controls.
Amazon has piloted a program to make offers solely on the basis of the test, not just to put a gate in front of the onsite. They should be making it at least as resilient to fraud as an onsite, which is why I think these measures are appropriate.
"Secure Exam Proctor Exam Environment: You can generally utilize the Secure Exam Proctor for taking a proctored examination without revealing any Personally Identifiable Information about yourself. The types of information collected depend on the exam settings and can include video, audio, desktop recording, and websites visited. The aforementioned data is encrypted and will not be used by Proctorio in anyway. This information may be used by "Authorized School Officials" to review the actions of Students during exam administration. Proctorio may collect additional information set forth below in the "Aggregate Information" and anonymous information. Aggregate and anonymous information will be used to improve the quality of the Secure Exam Service.
Note: Proctorio utilizes zero-knowledge encryption to keep your information safe."
If it is using proctorio.com as mentioned in other comment, I think it is normal.
On Juniper or Cisco certification test (using Pearson vue) they warn the candidate something like this (not fully the same especially for eye and mouth movements, but the candidate is video recorded).
Basically it is to make sure the candidate is not cheating.
The different is it is using Desktop provided by testing center partner.
So, what if I run the browser in a VM, and cheat in a second VM? With a secondary input device to switch between which VM is displayed on my screen? Or even an overlay?
They’d be completely fooled.
It’s completely useless, and so needlessly invasive.
They have a human watch you all the time through the camera, so they’d notice if you look anywhere else. And they require you to show them all around your room first, to prevent that.
That’s why you install the system below the desk, and use it that way.
I think it's a bit silly to get so worked up about this. The test is simply to gauge programming skill in an online test, and they want to check you are not cheating. Though I think they could have worded it without freaking people out: "You are going to be video monitored during the test".
Presumably you're aware that job interviews and the actual job are different things? At a real job no one cares if you use Stack Overflow because there's ample other data to assess your productivity (like...your actual productivity over the months and years that you're employed). The challenge with interviewing is trying to extrapolate performance from a much, MUCH shorter time horizon[1]. You're not testing the exact same environment, but you're using a different environment (the test/interview) to hopefully assess skills that are relevant in the environment you're optimizing for.
This is hardly a new concept: exams work pretty much the same way. There are other ways to assess candidates (like looking at their portfolios), but they have their own weaknesses, and testing skills in a constrained environment can still reveal things.
[1] And this is for the benefit of both parties: I've lost count of how many people I've heard complain about companies that have gauntlets of technical interviews.
Presumably you're aware that job interviews and the actual job are different things?
This is the core problem with tech hiring circa 2016. The sooner this idea dies, the better. It's surprising that more companies haven't noticed the most effective interviews are ones that resemble the company's day-to-day work as closely as possible.
Doing the test with a friend? I wouldn't want someone who can't complete coding exercises on their own. Also I'd expect people to answer simple questions quickly.
I dunno man. I'd much rather see a well understood vetted email validation(or whatever )copied from stack overflow than some thrown together hack in production code.
being able to look things up is a different skill than being able to understand and properly apply what you looked up. Or being able to understand what you are looking up and why.
Having just reread 1984, it seemed familiar to me. Sure, people cheat all the time to try to get a job (I have no idea why, even a cursory phone call will eliminate the ditch digger). Maybe Amazon should focus on making the jobs more enjoyable so people stick around and they wouldn't need such excess to find live bodies.
The reason Amazon (and other large tech giants) are hiring so many people isn't because they are loosing good talent -- it's because the company is growing at such an aggressive rate that there are always new positions opening.
When I was taking exams at university I am certain the professor could monitor my eye and mouth movements. I could also of course be seen at all times as well as any sounds I made were heard .
How is this different?
Doesn't really fade me. I mean, it's just for the duration of the exam. Install on a throwaway laptop, do your interview, be done with it.
It's not so much the recording of these metrics that I'd be worried about, but them storing it to build profiles. Different story.
These particular metrics are now quite common for online exam systems.
Mind you, these are valid concerns, and anyone should feel free to reject them. But on the other hand it's nice to have a remote option (for certifications, exams, interviews...) and at the same time a cheat-prevention system.
You'd be creeped out by the stories I have of freelancers or even consulting companies having some "star" employees doing interviews and then other employees showing up at work instead (remotely or even physically). Once you've been burned by stuff like that, you understand they take precautions.
In OPs position, I'd ask for the storage and retention terms and conditions though.
Is it really creepy? When I interview you in-person I am able to note all of those things if I want to and I can include "Your odor" on the list. They probably just have a legal team somewhere that feels that it is worth disclosing these items that really are common sense in a face-to-face.
That they have the webcam and microphone is not 'creepy'.
They just want you to be able to take a test remotely without cheaters.
If this were a regular website, sure, but it's for a specific reason, during a specific event, that ultimately you are participating in.
If you were to go and 'take an exam' on Amazon campus - they'd be doing the same thing - but a human proctor would be looking for the same things :)
I think the contextualization of it (i.e. it's 'your home') definitely seems 'creepy' - but I don't think it's a cause for concern - there's a reason they are doing it, it's not nefarious, they are open about it.
Again - it may feel a little creep-show because we're not used to this, or rather, in any other situation it would be truly creepy ... but I don't think there is cause for concern here.
I also think that the logical questions are not unreasonable.
They sometimes use a similar locked down browser (respondus) for in class assessments at my university. Never used it myself, so not sure how similarly fine grained any tracking is.
Within these restrictions, I don't think they expect your written code to compile. They just want to ensure the integrity of the test itself. Is it worth the invasion of privacy? Personally, I don't think so.
Well if you participate in an on-site test which the person was willing to do you are essentially tested under the same conditions, so I'd not really consider this to be a privacy invasion.
You're presumably giving them a view into your home and its location. You'd potentially be required to change security settings on your computer, to re-enable things that had been explicitly disabled. You're likely to also want to close whichever browser tabs or open programs that might be deemed unprofessional.
Those are all things that I'd consider invasive that I don't have to deal with during an in-person interview. And the wording is just creepy anyhow.
True enough, but not everybody actually lives in places where on-site testing is an option, and they'd probably prefer cleaning their room up to not having the ability to apply for a job.
And given that all of this happens voluntarily, and assuming that Amazon isn't going to infect you with malware(which seems very unlikely) this kind of testing is a great opportunity for people who have the resources or time to show up in person.
There's some kind of privacy chauvinism involved in these discussions that ignores the realities of people who don't live next to the Amazon HQ.
What I wrote was just a reaction to when you said that the arrangement wasn't a privacy invasion. I don't have any problem with off-site testing, just this particular implementation of it, which seems uncomfortably distrustful, like a harbinger of what working there would be like.
> And given that all of this happens voluntarily
There's voluntary in the sense of willingly (I have no reservations about doing this; heck, I'd offer even if you didn't ask me to), there's voluntary (I choose to do a thing that sucks because I don't seem to have another choice), and there's everything in between. My problem is when the "voluntary" action is more on the negative side of that scale, which it often is when someone's looking for a job.
I'm not trying to criticize candidates' choices. I'm trying to criticize Amazon's implementation of their hiring process. The point isn't that "candidates should be principled enough not to stand for it", but that Amazon should decide "this way sucks, let's find something better".
> assuming that Amazon isn't going to infect you with malware(which seems very unlikely)
True, they just have you voluntarily install someone else's malware.
About 8 years ago, I interviewed w/ Amazon & I chose to do my interview in C++. Although there was no actual compilation, I did get called out for small pedantic syntax errors. Maybe it depends on the individuals that are interviewing you.
Indeed. When I was interviewing candidates, one of my goals was to get the candidate to respond along the lines of "I don't know.", to which I would follow up with, how would you find out? No engineer knows everything. I also don't want a candidate that is going to try and bullshit me. Admit to me you don't know, but then tell me how you would rectify that. Google SO, MSDN, man pages, whatever, I don't care so long as you can admit you don't know and have an idea about how to correct that.
Personally, I'm doing mostly Python & C++ development at my new job (4 months in) after not using either for 2 years plus. I'm mostly fine with syntax for both, but constantly have my browser open for documentation, mostly for library related info.
In the interview for my first job after university I was asked a Unix trivia question. I said "Oh, I dunno... Most of my Unix knowledge is swapped out to the man pages!"
That's why they won't expect anything useful from that process. They will most likely expect some basic programming knowledge and ability to reason about made up problems.
Exactly. When I was doing this, one of the questions required the use of a custom comparator. I spent the remaining 20 minutes of the challenge trying (and failing) to remember the correct syntax for writing a comparator until it timed out.
I also did that question. You had access to the full Java docs.
I left a nice note explaining how I spent a lot of time trying to do it with the language feature, but couldn't due to runtime restrictions, and gave them a working non-optimized solution.
I was emailed to schedule an interview the next day.
There's different types of coding. Using xyz UI framework or fancy runtime environment will of course require reference.
Sorting a list of points or finding a simple pattern in an array will not. If you need SO to answer the types of questions they ask, you probably shouldn't be seeking an engineering position at a software company.
I know I might get shit for it from HN, but I don't remember jack shit about specifics of implementing sorting algorithms or how to invert a binary tree, because I don't need it right now. I know some of the ideas about why and when you need some sorts over others, but the ins and outs I can get if or when I need them.
Currently I could talk you damn ear off about 9 different ways of doing a topological sort (and which ones you can use if it's guaranteed to be a DAG, etc...), but that's only because I spent my last few days working very closely with stuff like that. Ask me in a month and I'll be back to knowing jack shit about the specifics.
I'm terrible at these kinds of interview questions, but I feel I'm a productive developer.
The last time I screened with Amazon, they wanted me to have a notebook handy, write down the code in the notebook with a pencil, and read it back to them over the phone. That's just how they roll there so unless you can implement linked lists, sorts and the like from memory, don't work for Amazon.
I hear this a lot and it always pisses me off. Part of what I believe gives me value is that I can recall from memory most of what I do day to day without looking it up. I have coworkers who are good, but possess terrible memories. Most are semi impressed with I rattle off a shell script and all the flags needed to get it to work on the first try.
I think the anger comes because I see it as a cop out, and I feel I get the short end of the stick in interviews because of it.
The company I'm at does two coding interviews for potential developer hires.
The first is kind of a standard phone interview with a Google Doc. There is no expectation that the code you write would actually compile or run. When I conduct this interview I'm always sure to explicitly say that it's not a test of memory and they shouldn't worry about remembering specific standard library functions, etc. If they need something that they know exists but they don't remember the specifics, I encourage them to make up something that seems reasonable and just go with it.
The second is an onsite interview where we plop them at a desktop machine, give them a relatively simple programming task, and ask them to produce a runnable program. They have access to the internet and really the entire machine. They can Google whatever they want and even use Stack Overflow. Nobody is standing there hovering over their shoulder. This interview gets weighted more than the previous one.
I can see why they might do this, anyone that can land a job with similar benefits somewhere else probably doesn't even apply, leaving them with the worst to choose from. Their climate gets even tougher because they "can't hire enough good engineers" and the cycle repeats.
Have you ever done lots of remote interviews in a big company?
The scamming is unbelievable. I've personally seen outright fraud... the guy in the video was a redhead with freckles, his name was an ethnic Vietnamese. Somehow they brought him in for an in-person interview, a Vietnamese dude shows up.
- Clueless. Guy was literally reading a manual.
- One guy offered me a bribe.
- one guy asked if his status as an illegal and conviction for hacking a phone company back home would effect things.
As much as I would like to bash Amazon for their draconian and insane interview loops -- which I've been put through once.
The post clearly states these clauses were part of an exam taken on https://proctorio.com/ for the Amazon interview. So it's doubtful Amazon HR is fully aware of these clauses.
I would happily bash Amazon interview loops all day long, but in this particular case, I would also need to criticize anyone else that uses https://proctorio.com/ for these practices.
So it's doubtful Amazon HR is fully aware of these clauses.
By all accounts, Amazon is a pretty tightly-run organization. And given the scale of the hiring they do, you be pretty sure that in fact, yes they are exactly aware of what proctorio is doing -- and chose them for precisely that reason.
Can you please give me some sources that explain how it is unethical? (I am aware of the NYTimes article which explains how badly they treat their employees, but that doesn't necessarily translate to unethical development.) I ask because I am about to join them, so forewarned is forearmed.
"I am aware of the NYTimes article which explains how badly they treat their employees, but that doesn't necessarily translate to unethical development."
Anyone who has seen Adam Curtis' documentary, "The Trap," will find this familiar.
Our society has become so obsessed with quantification that we have become blind to any other ways of understanding the world around us, and made us poorer in our analysis, instead of being more informed with more information.
It is like saying that if something cannot be measured, then it does not exist.
Hmm could you just record a loop of you sitting motionless at your computer and feed that to the camera input? Maybe some incredibly loud high pitched noise to the audio?
Not suggesting these as feasible solutions, but something like this deserves to be trolled.
Though I agree these requirements are unacceptable, I'd like to point out that you've refused on grounds of privacy infringement and then leaked Amazon's private information.
What's the privacy issue? If you go to an on-site interview they'll look you write in the eyes and see your eye movements. They'll know where you are, if you're browsing the web for some reason, etc. It's just a test they don't you cheat on. Are you thinking they'll sell their interview data to advertisers? Seems preposterous.
> To what extent should personal ethics play in deciding where a (software) engineer should work?
I believe you should use your personal code of ethics as a compass to guide you in your life, including where you work.
> What if the engineer has no other choices for a job, and needs to (make rent/pay the bills/eat food)?
You're describing a clash of personal ethics and social ethics. Social ethics are saying this kind of tracking is alright, your personal ethics no. So, how strongly do you believe in your conviction? Would it be worth going hungry to stand up for what you believe?
Sort of a sliding scale, huh? The more you are pushed towards survival, the less options you have to exercise your personal ethics. On the other hand, maybe this point isn't as important as it seemed at first thought and you'll decide you're willing to be subjected to monitoring for the duration of the test (but not before or after of course)
> "I believe you should use your personal code of ethics as a compass to guide you in your life, including where you work."
My first full-time job, I encountered this. I was less than a week away from quitting because of ethical concerns, and my boss got fired first. After that, I decided to give the company a chance, and I ended up being there for +9 years (was at a hedge fund).
I guess my advice is: are the ethics violations due to the company or your manager. If company, leave. If manager, report the violation. If no response, or retribution, leave. I held my manager's ethics violations in for ~6 months and it took an emotional toll. Not worth it. What he'd asked me to do was illegal and if I'd done it and been caught, I would have gone to jail. Fuck that, turn them in immediately.
440 comments
[ 46.6 ms ] story [ 735 ms ] threadTo what extent should personal ethics play in deciding where a (software) engineer should work? What if the engineer has no other choices for a job, and needs to (make rent/pay the bills/eat food)?
It is true that it'd be pretty easy to circumvent the tracking, by placing paper over the webcam, running the browser in a virtual machine, spoofing data to the browser, and so on. However these are infeasible for most non-technical people, so I don't think it's a real solution. Freedom shouldn't be only for those with extremely technical knowledge.
You work for a Thai ISP and the government orders you to block everything that insults the king and royal family?
You do network enginering for a mobile phone network operator in a developing African nation and they want to redesign parts of their network to support participation in the Facebook "free basics" walled garden internet for your GSM/UMTS/HSPA+/LTE type customers?
edit: there are a lot of things you can do to mess with a properly functioning internet, on the behalf of autocratic regimes or greedy corporations, just at OSI layers 2, 3 and 4... That's before you even get to the level of operating system and applications/software engineering.
Coming to your question. First question is easy. In a business environment, subjected to international rules and regulations; ethics are a priority. Second one not so. Kind of similar to when does it become acceptable to rob someone if robber can't make ends meet. Depending on jurisdiction; court will decide level of leniency.
Maybe you could record yourself staring intently at the screen and play that back to the webcam during the exam.
Kinda like, "haha! I secretly got messages delivered to me during the steganography test! It's obviously not a legit test."
However, from a philosophical point of view, it's Draconian and unnerving. What assurance does the interviewee have that Amazon isn't storing and using all this telemetry? Amazon is dictating the entire procedure, right down to the specific browser and extension, and the interviewee has no control over this. In fact, I have to wonder if the job seeker's acquiescence is part of the assessment; if they will bend over and take this massively invasive procedure, perhaps they will make for a good little subservient drone at Amazon.
I'm with the author, screw that shit.
I don't disagree, and I recently stopped using all Google products in part because of how much information they have collected and still want to collect from me. After the account suspension drama from several days back, I was spooked and came to realize just how dependent I was on the company, and what I stood to lose should I come into the crosshairs over a simple ToS violation or even just a random bug in their service.
But that only strengthens my point of view about this hiring process and how wrong it is. When a company wants to exert that much control over someone who doesn't even work there yet, it speaks volumes about their overall culture and attitude towards people in general.
This sort of behavior by teachers is exactly what got me to write my first serious z80 assembly program to mimic all of the menus on the TI-83 circa 1996. Teachers had already caught on to the fact that there were programs that would display the "Mem Cleared" message on the calculators, so mandated they had to do it manually. I never used my program to cheat, I just didn't want to lose my games (had a really good zelda clone & penguins).
I admire your tenacity too.
I'll probably have to order one off Amazon at some point.
How is that different from being recorded by multiple cameras during an on-site interview?
I honestly don't see why the information they're collecting is a big deal. They are remotely interviewing you for a position, and would like to minimize cheating as much as possible. Instead of sitting you in a room, they use the technology available on your computer, while clearly telling you upfront what kind data will be collected. If you don't like it, get an interview elsewhere.
there are still in person stages; this is just a filter for them. it's not like it's the only data you get to hire or not hire someone
These are the cases where monitoring is needed. Whether or not what Amazon is doing is actually effective is another issue entirely. All I'm saying is that it's not as huge a deal as a lot of people on this thread are making it out to be.
I'd refuse this too. But the difference is that the on site one isn't a massive security threat.
It doesn't even satisfy the legal requirements for informing people their personal data is going to be collected (in my country at least). They don't tell you what happens to the data, who has access to it, they don't provide a way to let them know you want it deleted, and so on.
Let's chill here, 'Draconian' is a strong word.
'What if Amazon is using the data for nefarious purposes'.
It's a reasonable conclusion, but also a big 'what if' :)
Their motivation is pretty reasonable - they just want you to be able to take a test remotely.
They 'dictate the process, right down to the extension' for reasonable cause.
It's reasonable to ponder the ways in which there could be bad things going on, but I don't think it's reasonable to assume that there is, or even malicious intent here.
It just 'seems' creepy.
Of course, I think that the real answer for that is to use a human to interview everyone. It's crazy to treat potential future employees this way.
But what you can say is: "If you want me to jump through all of those hoops, send me a $99 tablet that's already configured for the test conditions you want to subject me to."
I've even had a candidate whisper the question to someone else in the room with them. Seriously, it's ridiculous what some have done.
Unless you are working with a really small API surface then it is unlikely that a programmer is going to have much of the API memorized than the ones he uses on a daily basis. And with people radically changing their stacks every few years it is unreasonable to expect it.
Also, people copy/paste/tweak things from StackOverflow all the time. Disabling clipboard is unreasonable for that reason.
What a programmer can do using only things completely in their head is quite different than what programmers do in reality. Programmers are constantly looking up things they don't know throughout the entire day.
Keep in mind too that online tests are for screening, not a complete interview. You bring people in after that and do several rounds of interviews. Under that circumstance having a friend next to you or supplying you the answers doesn't really get you far if you know you have to do similar tests once you get into the office. There's isn't a lot of incentive to cheat.
The bigger concern is that this is a red flag about the work environment / culture. If I saw that I wouldn't go anywhere near it as well. In fact I deliberately put things in resume that some companies might find objectionable in order to purposely weed out with such cultures.
Allegedly.
64 is 4 * 4 * 4 and _60_ is (4 - 1) * 4 * (4 + 1)
Given that they've defined "::" differently to the usual "as" the answer could be anything. More specifically, the answer is f(64), where f(x) is wharever was in the head of the guy who made the question.
The same goes for the "what is the next number in this sequence" questions: unknowable given the information provided.
But personally, if asked this question suddenly, I would've gone for the simplest: 61, because 27 - 24 = 64 - 61.
* Your webcam video
* Your head movements
* Your eye movements
* Your mouth movements
that was all collected during the test.
You can use an HDMI splitter, and merge the videos of the two computers. As overlay. You can also program the second computer to have a mouse controlled by a single muscular activation in your cheek or tongue – for example, if you have a tongue piercing, put a magnet in, use a phone to recognize the magnetic field, tape the phone below your desk (it’ll measure the difference from movement), and connect that to the PC.
No way to recognize it for them, and you circumvented it entirely.
They’re trying to fight against people whose job is to circumvent weird complicated restrictions, and finding workarounds.
Still, this is hilarious because only the truly desperate will put up with this crap, everyone else will just get their jobs somewhere else.
https://sites.google.com/site/thefaceofamazon/
Given that this thread seems to be fairly split on this issue (at least at the time I'm writing this) this seems like a bit unfair to say. Personally, I wouldn't give a shit, and I like to think that I have at least some self-respect.
I know some people in bay area who are making > $100k in cash (no taxes paid) just by answering screening interviews on other people's behalf.
This is called "interview by proxy" and the person generally takes first month's salary as remuneration in cash.
The only hit for "interview by proxy" relevant to technical interviewing seems to be a TDWTF article.
This is a gross oversimplification of the problem. Do you know how difficult it is to fire a person? Do you know how much it costs to hire someone? You've given this person a relocation bonus, possibly put in the work to set up a visa, given the first chunk of a signing bonus....
"Right-to-work laws are statutes in a reported 26 states in the United States that are an effort to give employees the right to work without being required or compelled to join to a union."
https://en.wikipedia.org/wiki/Right-to-work_law
Large American Inc. is a large company we all know off and which is always somewhere on the front page for some news. Mr X is a manager there how arrived from India on H1B and now perhaps has a green card.
Mr. X brings his younger brother Mr. Y on F1 visa. Mr. Y then joins a consultancy. Consultancy claims Mr. Y has 8 years of experience, knew AngularJS before it was invented (true story) etc. and submits it to Large American Inc. for a project that Mr. X is spearheading.
The consultancy then send a proxy for phone screen and real interviews. The consultancy, Mr. X and Mr. Y all pull a lot of strings here. favours are exchanged.
Mr. Y then joins Large American Inc. as a consultant with a salary of $120K under Mr. X.
Mr. X and Mr. Y conceal the fact that they are closely related.
In many cases consultancy hires another person in India whose job is to give 24x7 phone support whenever Mr. Y has to do his work. Screenshare etc.
Any website you visit
What a fantastic way of driving away virtually every candidate that's even remotely high-caliber.
These conditions only apply for the 24 minutes in which you take the test. How is this different than if a person were watching you during an interview?
Using such a system in the first place also signals their ineptitude at hiring, or at least an ignorance of quality hiring practices.
>What are you going to do, take a break and load up some porn?
Well, yes:
https://pbs.twimg.com/profile_images/669103856106668033/UF3c... [NSFW]
Paid homework projects are far superior. Plagiarism checks can be run on the provided solutions, and a follow-up conversation concerning the project at the interview should easily suss out anyone who had a third party to do it for them.
That approach also has the advantage of actually being a relevant to the position—unlike some generalized intelligence test.
Frankly if people are putting up with the test they have now, Amazon could probably get away with issuing unpaid homework projects, however classless that would be.
This will lead to hiring people who can be coached until they sort of understand a solution, although they still couldn't create it.
Every non-perfect solution is worthless?
I've found that the answer to all this is to give a 30-60m remote test (no proctoring), where they get to write the thing as they want, and a 15m discussion where you analyze their solution and assign one new feature then let them walk you through the changes. No code, no whiteboard, just discussion.
A lot of people can't whiteboard code from scratch but can take a piece of code they've just written and discuss next steps. And it's also an actual work skill that many good coders are bad at, so it gives you a good insight into high and low-level people.
If they're collecting this much data already, how much of a stretch do you think it is to obtain your browser history?
It's different because they're recording you, not to mention they're doing it by installing spyware on your computer. That's not what a person does when they watch you, unless they're watching you through a camera.
i have no problem with a candidate doing something totally by themself in part 2 of 5 before in in person interview; theres still another 3 different interviews (many of which will be in person) to catch cheating and such
it's just a filter; it's not the interview
These companies are typically used in settings where some level of rigor is expected (e.g. administering an exam for an accredited university). In that context, I think these restrictions make sense in order to ensure some degree of validity in the test results (especially since you're only expected to run the software for the duration of the test).
Why not do an onsite if you want to make sure of the validity?
Perhaps most critically, it is a more subtle test for the prospective employee-- are you the kind of person who would comply with a ridiculous request that violates your privacy in the hopes that it will land you a job?
I dunno. If I were an employer and an applicant actually installed this without question, it would make me seriously reluctant to hire them-- this is the sort of unquestioning, docile employee who could be more vulnerable to social engineering,who would not be likely to consider consequences of their actions, who does not take security, privacy, and safety as a priority for even themselves (and is therefore less likely to do so for customers and clients).
These may not be qualities that some companies value, but from my perspective, I think the "nope" letter to Amazon shows strength in independent thinking, creativity, and moral character.
It's like those companies that required you to hand over your social media passwords so HR could dig through your history.
https://finance.yahoo.com/news/job-seekers-getting-asked-fac...
That is horrifying. If Amazon doesn't hire you because of this, it says quite a bit about their company, fuck 'em. Their loss. There are companies out there that actually respect their employees.
Congratulations. You passed the test.
Asking this of you, before you're even working for them is presumptive and not a good sign that this is a place you're going to want to be. It's basically a huge flashing "We Fuck Our Employees!" right there on your job application.
This is also the company that requires warehouse workers to queue for bag searches on unpaid time.
Without the secrecy aspect, I don't think you can say it's spying by the definition people typically use. If you went to an examination hall and took a real proctored exam [1], would you consider that the proctor was "spying" on you?
> Moreover, it is invasive and unsettling.
This, I can't argue with. It's hard to pin a definition on what makes things creepy or unsettling. Some people find clowns creepy. Why? I have no idea. But there's no sense in arguing about it.
Personally, I think it's a clever use of technology to streamline and scale a traditional process (exam proctoring) that avoids making you travel to an exam hall or interview.
[1] https://en.wikipedia.org/wiki/Exam_invigilator
I mean it in the sense of "I went on a blind date, but my parents sat at the adjacent booth to spy on me". I knew they were there, but it was inappropriate invasion of privacy.
> If you went to an examination hall and took a real proctored exam [1], would you consider that the proctor was "spying" on you?
If I was taking the exam on a personal laptop and was handed mysterious software at the examination hall to install on my machine... (along with images of the inside of my home), then yeah...
Convenience and logistics. Is it worth yours (or Amazon's) time to pay for travel expenses, when applicants could literally be located anywhere? Evidently not, at least not in the early stages of the process.
That's a very long-winded way to describe a prescreening questionnaire. Are you saying that you think no one should ever take a job if there's a prescreening process?
Imagine you're hiring someone to do your taxes. Sure, you'd expect them to use a calculator to do all the hard addition/subtraction, or more likely spreadsheets. But if you asked them to add 5 and 3, and they said "I would use a calculator, this is a stupid question, I can't just tell you off the top of my head" then you're probably not going to hire them.
It's a super basic test of 'Can this person code themselves out a paper bag."
being able to solve problems is more important than how you solve problems. (generally speaking, yes i know there are exceptions)
When I was last in school I struggled with "networking your way into a company" and wound up landing my internship (and ultimately full employment) on the one company who offered a test before the "Fly you out to meet us" step.
I think I'd only consider doing this if I used a computer I could wipe completely before and after. Even then the amount of distrust is off-putting.
I can understand why ProctorU does what it does, but that process puts a significant burden on the test taker, one which is entirely different than taking a proctored exam in a classroom.
I sat about a dozen university exams in the last couple of years and in no instance did I have someone staring at my face, or peeking over my shoulder, during them. There were always supervisors in the theater, but they mostly walked along the aisles of desks and made sure to be unobtrusive and quiet as a mouse.
I really don't see any justification for what you describe.
I connected with a Lady via cam and she asked me to show via cam in a 360 style if I had anything else in the room I was in (books, someone helping,etc).
Took the exam after that and passed it. It is a bit unusual but very very convinient. If I had to take the exam physically somewhere I would have needed to take a day off.
In the case you describe you have to be physically present at a fixed location at a fixed time. In many cases that might be inconvenient or impossible, and I can certainly see situations where it'd be nice to have an alternative. Not having to have my physical location coincide with the physical location of my university of choice could be quite handy.
I can't imagine why...? That would be like seeing a freeway and instantly thinking of the Nazis.
It's not exactly that far fetched to think that someone might someday think it's a good idea to start applying some of those new AI APIs to the candidate interview videos just to see what happens. In fact given enough time and enough departments/companies, it frankly seems more likely than not.
Especially given that we already have AI scouring GitHub looking for people to recruit. Assuming it's not already happening, it seems unlikely that there won't be startups applying AI to hiring interview videos within the next couple years.
That's a pretty serious accusation, one that could spark a lot of successful lawsuits against those companies -- especially in the current political climate in the states they tend to be located in. Do you have any specific evidence to back it up?
C.f. what Malcom Gladwell has to say about this here: https://www.youtube.com/watch?v=gLQC3EzDGr4
Okay, then. Good luck with the lawsuits.
Using GPA as a screening measure when members of a racial demographic routinely preform more poorly due to socioeconomic factors, it starts to look more like a proxy for race.
In the news right now: http://motherboard.vice.com/read/new-program-decides-crimina...
If Amazon has chosen to use use Proctorio, with an understanding of how it operates, then it's very much Amazon's restrictions.
And if they don't "understand" their constraints (or pretend not to)... why would you want to work for them?
I'd just take it naked. I'm a fat hairy guy. I'm sure it'll be fun for all of us.
Also at an in-person interview, the interviewee has a symmetrical power relationship to the interviewer in that they can both observe on another. This practice slides the power relationship heavily toward the interviewer.
Furthermore, at an in-person interview, the information is not analyzed in an unspecified manner which further skews the power balance toward the large corporation.
Finally, at an in-person interview, the information is not digitally stored at an unspecified location for an unspecified time and transmitted to unspecified parties. Nor is it subject to unauthorized access by unspecified parties in unspecified countries due to unspecified security arrangements.
I will probably never be in a situation that comes close to the staggering power asymmetry of someone deciding whether or not I will have a job. Even if I'm charged with a crime the court has less power over me than an interviewer, in that its decisions are subject to juries and appeals and I get to watch the proceedings.
The information is analyzed unscientifically according to the personal biases and tastes of the interviewer to form an emotional impression he may not even be conscious of. "Not enough eye contact, I don't like this guy." I'd much rather it be judged by an algorithm trained on actual job performance years in.
Ever heard of Greenhouse? Your interview performance (as reported by the interviewer) is absolutely stored indefinitely in some 3rd party's database subject to unspecified controls.
Amazon has piloted a program to make offers solely on the basis of the test, not just to put a gate in front of the onsite. They should be making it at least as resilient to fraud as an onsite, which is why I think these measures are appropriate.
"Secure Exam Proctor Exam Environment: You can generally utilize the Secure Exam Proctor for taking a proctored examination without revealing any Personally Identifiable Information about yourself. The types of information collected depend on the exam settings and can include video, audio, desktop recording, and websites visited. The aforementioned data is encrypted and will not be used by Proctorio in anyway. This information may be used by "Authorized School Officials" to review the actions of Students during exam administration. Proctorio may collect additional information set forth below in the "Aggregate Information" and anonymous information. Aggregate and anonymous information will be used to improve the quality of the Secure Exam Service. Note: Proctorio utilizes zero-knowledge encryption to keep your information safe."
They’d be completely fooled.
It’s completely useless, and so needlessly invasive.
That’s why you install the system below the desk, and use it that way.
Holy hell, did I miss that or wasn't it mentioned? This might even be the most invasive of all.
If you stick a magnet to your tongue, and move it around, a modern phone can recognize that even from below the desk.
So if you had a tongue piercing, and magnetic jewelry for that, you could use that as a crude input device that’s invisible.
Although in this extreme case it does sound like checking stackoverflow is considered cheating.
This is hardly a new concept: exams work pretty much the same way. There are other ways to assess candidates (like looking at their portfolios), but they have their own weaknesses, and testing skills in a constrained environment can still reveal things.
[1] And this is for the benefit of both parties: I've lost count of how many people I've heard complain about companies that have gauntlets of technical interviews.
This is the core problem with tech hiring circa 2016. The sooner this idea dies, the better. It's surprising that more companies haven't noticed the most effective interviews are ones that resemble the company's day-to-day work as closely as possible.
(Disclaimer: I completed an internship at Amazon and accepted a second internship)
It's kind of like when you notice someone staring at you in the subway -- you note to yourself that it's creepy (which it is), and you move on.
It's not so much the recording of these metrics that I'd be worried about, but them storing it to build profiles. Different story.
These particular metrics are now quite common for online exam systems.
Mind you, these are valid concerns, and anyone should feel free to reject them. But on the other hand it's nice to have a remote option (for certifications, exams, interviews...) and at the same time a cheat-prevention system.
You'd be creeped out by the stories I have of freelancers or even consulting companies having some "star" employees doing interviews and then other employees showing up at work instead (remotely or even physically). Once you've been burned by stuff like that, you understand they take precautions.
In OPs position, I'd ask for the storage and retention terms and conditions though.
They just want you to be able to take a test remotely without cheaters.
If this were a regular website, sure, but it's for a specific reason, during a specific event, that ultimately you are participating in.
If you were to go and 'take an exam' on Amazon campus - they'd be doing the same thing - but a human proctor would be looking for the same things :)
I think the contextualization of it (i.e. it's 'your home') definitely seems 'creepy' - but I don't think it's a cause for concern - there's a reason they are doing it, it's not nefarious, they are open about it.
Again - it may feel a little creep-show because we're not used to this, or rather, in any other situation it would be truly creepy ... but I don't think there is cause for concern here.
I also think that the logical questions are not unreasonable.
If I had to code purely from memory, without the benefit of language or library docs, I would not be able to write anything of use.
Personally when hiring, I'm not interested in a candidate's memory recall, but the ability to use resources when faced with a new challenge.
Those are all things that I'd consider invasive that I don't have to deal with during an in-person interview. And the wording is just creepy anyhow.
And given that all of this happens voluntarily, and assuming that Amazon isn't going to infect you with malware(which seems very unlikely) this kind of testing is a great opportunity for people who have the resources or time to show up in person.
There's some kind of privacy chauvinism involved in these discussions that ignores the realities of people who don't live next to the Amazon HQ.
> And given that all of this happens voluntarily
There's voluntary in the sense of willingly (I have no reservations about doing this; heck, I'd offer even if you didn't ask me to), there's voluntary (I choose to do a thing that sucks because I don't seem to have another choice), and there's everything in between. My problem is when the "voluntary" action is more on the negative side of that scale, which it often is when someone's looking for a job.
I'm not trying to criticize candidates' choices. I'm trying to criticize Amazon's implementation of their hiring process. The point isn't that "candidates should be principled enough not to stand for it", but that Amazon should decide "this way sucks, let's find something better".
> assuming that Amazon isn't going to infect you with malware(which seems very unlikely)
True, they just have you voluntarily install someone else's malware.
Personally, I'm doing mostly Python & C++ development at my new job (4 months in) after not using either for 2 years plus. I'm mostly fine with syntax for both, but constantly have my browser open for documentation, mostly for library related info.
I left a nice note explaining how I spent a lot of time trying to do it with the language feature, but couldn't due to runtime restrictions, and gave them a working non-optimized solution.
I was emailed to schedule an interview the next day.
Sorting a list of points or finding a simple pattern in an array will not. If you need SO to answer the types of questions they ask, you probably shouldn't be seeking an engineering position at a software company.
I know I might get shit for it from HN, but I don't remember jack shit about specifics of implementing sorting algorithms or how to invert a binary tree, because I don't need it right now. I know some of the ideas about why and when you need some sorts over others, but the ins and outs I can get if or when I need them.
Currently I could talk you damn ear off about 9 different ways of doing a topological sort (and which ones you can use if it's guaranteed to be a DAG, etc...), but that's only because I spent my last few days working very closely with stuff like that. Ask me in a month and I'll be back to knowing jack shit about the specifics.
I'm terrible at these kinds of interview questions, but I feel I'm a productive developer.
I think the anger comes because I see it as a cop out, and I feel I get the short end of the stick in interviews because of it.
The first is kind of a standard phone interview with a Google Doc. There is no expectation that the code you write would actually compile or run. When I conduct this interview I'm always sure to explicitly say that it's not a test of memory and they shouldn't worry about remembering specific standard library functions, etc. If they need something that they know exists but they don't remember the specifics, I encourage them to make up something that seems reasonable and just go with it.
The second is an onsite interview where we plop them at a desktop machine, give them a relatively simple programming task, and ask them to produce a runnable program. They have access to the internet and really the entire machine. They can Google whatever they want and even use Stack Overflow. Nobody is standing there hovering over their shoulder. This interview gets weighted more than the previous one.
It's a rather blatant IQ test, though of course they would never call it that because that would open them up to disparate impact lawsuits.
https://careers.mozilla.org/university/
The scamming is unbelievable. I've personally seen outright fraud... the guy in the video was a redhead with freckles, his name was an ethnic Vietnamese. Somehow they brought him in for an in-person interview, a Vietnamese dude shows up.
- Clueless. Guy was literally reading a manual.
- One guy offered me a bribe.
- one guy asked if his status as an illegal and conviction for hacking a phone company back home would effect things.
The post clearly states these clauses were part of an exam taken on https://proctorio.com/ for the Amazon interview. So it's doubtful Amazon HR is fully aware of these clauses.
I would happily bash Amazon interview loops all day long, but in this particular case, I would also need to criticize anyone else that uses https://proctorio.com/ for these practices.
By all accounts, Amazon is a pretty tightly-run organization. And given the scale of the hiring they do, you be pretty sure that in fact, yes they are exactly aware of what proctorio is doing -- and chose them for precisely that reason.
The whole point of having a responsible person is that the buck stops with them.
I didn't go there, but here's some i found on the net. https://sites.google.com/site/thefaceofamazon/home
https://sites.google.com/site/thefaceofamazon/home/don-t-acc...
Please don't accept your offer. PLEASE don't go there. Your brains will not be wasted at Amazon, but your life will be. And no one will care.
Yes it does.
Our society has become so obsessed with quantification that we have become blind to any other ways of understanding the world around us, and made us poorer in our analysis, instead of being more informed with more information.
It is like saying that if something cannot be measured, then it does not exist.
Not suggesting these as feasible solutions, but something like this deserves to be trolled.
I believe you should use your personal code of ethics as a compass to guide you in your life, including where you work.
> What if the engineer has no other choices for a job, and needs to (make rent/pay the bills/eat food)?
You're describing a clash of personal ethics and social ethics. Social ethics are saying this kind of tracking is alright, your personal ethics no. So, how strongly do you believe in your conviction? Would it be worth going hungry to stand up for what you believe?
Sort of a sliding scale, huh? The more you are pushed towards survival, the less options you have to exercise your personal ethics. On the other hand, maybe this point isn't as important as it seemed at first thought and you'll decide you're willing to be subjected to monitoring for the duration of the test (but not before or after of course)
My first full-time job, I encountered this. I was less than a week away from quitting because of ethical concerns, and my boss got fired first. After that, I decided to give the company a chance, and I ended up being there for +9 years (was at a hedge fund).
I guess my advice is: are the ethics violations due to the company or your manager. If company, leave. If manager, report the violation. If no response, or retribution, leave. I held my manager's ethics violations in for ~6 months and it took an emotional toll. Not worth it. What he'd asked me to do was illegal and if I'd done it and been caught, I would have gone to jail. Fuck that, turn them in immediately.