Interesting idea. But it seems like there’s room for abuse - won’t engineers, who are unhappy that they didn’t get an offer, just report companies? Don’t you just become Yelp in that case, full of reviews from salty customers? How are you going to get useful data when you’re worried about that?
This is certainly an issue. As you say, it’s a problem on yelp and glassdoor. It’s why most nightclubs have low yelp ratings (people they exclude give them low scores). There are a few things I think we think we can do:
1) Look at relative data, not raw ratings. Written reviews or raw numerical scores are what show the nightclub effect the most. What we care about is a ranking of companies (showing the better companies first in search results). This may not be as impacted by the problem. If sour grapes from people who failed interviews overwhelm other signals, we can only look at the scores from people who pass (or normalize the two groups separately).
2) Ask very specific questions (and maybe provide dummy options to attract sour grapes). I suspect that questions like "did this company say they could meet at $250 salary and then offer you less?" will get more accurate answers than "does this company have a toxic culture?". (We do actually want to get culture data, but I want to be really specific about the culture questions we are asking).
3) Use objective facts, not subject opinions. For example, we can tell when a company ghosts a candidate (because they reply through a proxy email that we control). So we're listing the 'ghost rate' for each company.
IIRC I wrote about 30 custom cover letters to Triplebyte companies... about 2-3 responded. My response rate to cold emailing/applying to ~10 F# companies is literally 100%. To be fair though, the TB applications occurred right at the start of Covid, and my F# job applications well into Covid.
Anyway, a ghost rate would be greatly appreciated.
In my experience toxic companies will essentially force employees to give good feedback through either social pressure, incentives or threat of repercussions. You can see this quite clearly on Glassdoor where a bad review is followed by five very similar good reviews within a week for some companies.
> I suspect that questions like "did this company say they could meet at $250 salary and then offer you less?" will get more accurate answers than "does this company have a toxic culture?".
I'm pretty sure many salty people will just lie anyway, to punish the company. Source: I've been on the internet a long time.
I think this model of 'punishing' companies would've worked when you had a high bar for being on the platform in the first place, but if anyone can get on there, you'll have a lot of low quality candidates who are very mad that nobody will take them, and they'll overwhelm the reviews.
Remember that there's a survivorship bias at work: the lower quality people will take longer to get hired, which means they do a lot more interviews, which means a lot more reviews on your platform. Essentially, it's a reverse meritocracy when it comes to evaluating companies.
The difference is that I think we're in a position to actually change behaviour (require companies to not do bad things like ghost candidates, lie about salary).
Will this newfound respect for users' privacy be backed up by a contract or some other legally-enforceable commitment, or do we all have to worry about when you decide to pivot again in another couple years?
I hadn’t thought about this from a legal contract perspective. I like the idea of making our commitment enforceable. Do you have ideas of what you'd want in a contract?
In 2021, you don’t even have to search for engineering jobs. You either start out with a shortlist of companies and roles or a recruiter reaches out to you and kicks off the process.
Maybe this would be useful for folks just starting their career, but on the hiring side of this, in the last year I haven’t made an offer to a candidate who didn’t already have at least one other offer on the table.
Overall I think there are a lot of misconceptions about how people are hired from the recruiter side — which is why most of these platforms see the recruiter as the target customer, not engineers. I get that y’all are trying to change that, but without insight into the hiring practices at any given company (e.g. my company gives a lot of latitude for salary negotiations but every hiring manager negotiated differently) it’s going to be hard to pull a signal from the noise.
While the constant supply of recruiter contacts and even direct internal company contacts I get is nice, I repeatedly see the sort of basic issues as mentioned in the piece of concealment of or lying about pay (or incredibly broad pay ranges), entirely leaving out information about a job's location even when it's on the other side of the country, trying to offer entirely inappropriate junior positions that I'm never going to be interested in... all stuff that has zero cost to a hiring manager but wastes my time. I could see a lot of usefulness in something to push back against that in a structured way.
On the recruiting side (I’m not a recruiter but a director making a lot of hires), we actually do the opposite all the time — we post relatively senior positions then downlevel whoever we hire to fit their experience level. A lot of people think more highly of themselves than the employment market does.
We look at the job posting as a brand marketing — all we’re really trying to do is get you to apply and fill the funnel. The whole hiring process is designed to be a calibration on role, cultural fit, compensation, etc.
We have the conversation about level before we give an offer. It’s genuinely because we don’t want to put someone in a role they’re going to struggle with and be unhappy or burn out. I would much rather underlevel someone starting out and promote them 6 months later than hire them at a higher level and then needing to have the conversation about performance early in their tenure or cut their bonus.
I work in consulting so high turnover is just expected. We’re actually a lot lower than the industry average. The job isn’t for everyone, but there are people who thrive in this environment. For everyone else who isn’t a good fit, we get a year or so worth of billable hours, they get exposed to a lot of different scenarios, and we give folks a lot of runway to find something they do like.
It’s just the nature of the beast though. If I get a candidate who is legitimately at level, I bring them in at level. There’s no need to be shady about it, but job search sites are what they are and that’s the game you play.
In a narrow sense, that's the thrust of the case we're trying to make here: if veteran engineers didn't frequently have competing offers, they wouldn't hold the amount of power they do.
That being said, the experienced engineers who can get that kind of attention have other desiderata in their job search beyond just "get an offer". "Get -an- offer" is a goal more typical of a new graduate or someone trying to break into the industry than it is a goal of experienced engineers.
To people who are sure they can get a job, there's a big difference between a 150k offer at a company that has none of the workplace traits they want and a 200k offer at a company that they'd genuinely love to work for. And right now, they can't necessarily tell the difference between those jobs before going through a bunch of application steps. The example we mention in the post - of a highly credentialed and very senior engineer who kept having his time wasted by recruiters who refused to admit that they couldn't offer what he was worth - wasn't cherry-picked. It came up organically while we were doing user research, because it's a really common problem, and was echoed further by a number of other senior engineers we spoke to.
(Of course, we care about the folks just starting their career, too, so I don't want to totally ignore the importance of solving problems for them. But I think we are solving an important problem for senior devs here, too!)
I got my job through TripleByte. I consider it my dream job. Honestly, I loved the experience with TripleByte. I got a bunch of interviews to happen at the same time. Having them all together increases our bargaining power. It's hard to get the processes with different companies to coincide if you're just waiting around for recruiters to reach out. I already felt like it gave me the power. I'm sad to hear it's changing, because I would've used it again. I've also referred tons of people out of genuine liking of the service. Not sure if I can recommend it anymore. Still, I'm very thankful for what they did for me. I hope their new style works out.
I'm really glad that the old process worked so well for you. When it worked, it was pretty great. We are losing something with this pivot. The problem is that the old process did not work for far more people than it did. For every wonderful experience with a TM there were 10+ engineers who we screened out. That’s why we made this change.
I think it probably was the point, but I'm guessing the primary issue with it was that it turned out to be harder to profit with that strategy.
To use a rough biological analogy, I think it's generally a lot easier to have a big, rapidly growing company if you take an r-selection rather than a K-selection approach.
I'm an engineer who was introduced to a startup via triplebyte. I liked the platform because I went through a technical interview to even be allowed to connect with companies (one that a lot of my peers who are junior level weren't able to pass.) I guess my first thought is that doesn't this defeat the initial purpose of Triplebyte?
If you allow everyone in, regardless of if they are qualified to be a senior level engineer, will this not just be another job hunt platform where (when I look for another job) I'd still have to do another technical interview at the company itself?
How will companies know now who has been vetted as qualified and who is just lying on their resume? Maybe it's shitty to think this way but there is already a ton of sites out there that allow me to send my resume (or allow a company to view mine) before starting a traditional hiring process that Triplebyte prevented, allowing companies to connect with qualified high level engineers and know the person would fit their skill needs. The way it was companies mainly were just testing for cultural fits, etc.
Exclusivity and rapid growth don't normally go hand-in-hand, so one would expect that at some point, the company would have to bend on one of them, and it makes sense that they decided to bend on exclusivity. Facebook followed a similar course early in its history, and what allowed it to rapidly grow was diversifying its value prop. When it was exclusive to Ivy Leaguers, the draw was the meet other people at your school. When it was exclusive to college students, the draw was to meet other people at your school. But then when it became open to the world, it became much more of a general-purpose social networking site. That's the course Triplebyte will likely need to navigate if it wants to be successful in the long run.
I'll broadly second this. As an experienced dev, what kept Triplebyte on my radar was being able to at least theoretically cut past the initial round of 'can you actually fizzbuzz or are you just lying about it' that most companies need to filter out the people with inexplicably good resumes but no practical skills.
I think my experience sounds like what the article is trying to say…
I tried TripleByte and wanted to like it, but mostly it didn’t seem like anything special. The whole concept of getting pre-screened, getting multiple offers, and jumping straight to final interviews just didn’t pan out. The last company I interviewed with basically treated my application like any other pipeline. I had a standard recruiter call, hiring manager call, and several technical interviews before getting an offer. And the experience with other companies was similar. From the outside looking in, it appeared that there was no special track for TripleByte applicants. In the end it felt like the TripleByte process simply made the interview process longer, not shorter. Eventually I found it faster & more effective to pursue companies on my own.
TL;DR — companies I met through TripleByte ran me through the standard process I’d experience through any third party recruiter without the personal touch one might get from a good third party recruiter.
This wasn’t my experience. Sure they didn’t just make an offer but instead of the usual algorithm test and a second interview my onsites consisted of actually working alongside the founder for a day and then an offer a few days later. I’m totally fine with that what I don’t want is to have to do 2-3 interviews at every company I apply to.
Triplebyte was basically friction. It didn't help you skip the onsite and instead forced you to do two onsites, once with Triplebyte and once with the hiring company. It didn't make sense to ever engage with Triplebyte.
It should be an upgrade to do an onsite with triplebyte and an onsite with each company instead of doing several phone screens with each company and an onsite with each company.
(I'm a PM at Triplebyte working on this overall direction, joining Ammon for this thread.)
To be clear, we still have quizzes and the ability to show your performance on them to companies. The fact that they’re not completely mandatory to use Triplebyte doesn’t mean that they’re not important or that we won’t build further features around them.
Right now, companies can offer an expedited process as part of their reach-out. We’re working on ways to productize this a little bit better. We didn’t announce the feature in this post because it isn’t done yet, but one thing we’re looking at is allowing companies to set individual score thresholds for various types of expedited process (e.g. “if you scored a 3 on algos and a 4 on back-end, we’ll definitely get on the phone with you” or “if you have a 5 on python we’ll skip tech screens for you”). We certainly recognize that FastTrack was valuable to a subset of engineers, and we want to recreate the value it offered in a way that is a little bit more sensitive to the specific company and specific engineer in question. We imagine, for example, that less-prestigious companies (who are more concerned about attracting applicants) will probably set lower thresholds than the Apples of the world. Under the old system, we had to set a single threshold, which would necessarily either be too low for prestige companies or too high for everybody else.
And just to lean a little bit back into the pitch we’re trying to make here: if you’re concerned about interviewing process, wouldn’t it be nice to be able to search companies by how their interview process works? That’s not an axis on which companies meaningfully compete right now, but with the right incentives, companies will cut a lot of the annoying hassle that they currently have no reason to get rid of.
One thing I didn't like about TB is that I couldn't see my own score, or how I compared to others. They sent me follow-up emails, tried to engage me in interviews, asked if I had any questions, and when I asked about my score or how it related to others (top N% for example), they didn't even respond.
If a company asks me if I have questions, and I ask them, and they ignore me, I'm done.
I wasn't looking for a job, but thought the test results would be interesting. Who takes a test and doesn't get results?
This must have been a long time ago! We've told you what your scores were and how they compared for quite a long time now. Even the current version of that page, which is like the third or fourth iteration, is now many months old. (That said, we've got a bug breaking that display right now. Working on it!)
Speaking of your tests - your general coding skills one has an actual bug in the code, as of a few hours ago (I sent an email about it, hopefully it gets fixed)
So, I'm personally pretty unhappy with this change, because part of what made triplebyte valuable (and gave y'all the hundreds-of-thousands-engineer userbase) was the _path to competence-signalling_ which avoided credentials that your platform gave, given the quiz -- companies knew that someone being on triplebyte was a strong signal, and engineers had a path to signalling competence that was one-to-many. Eliminating that signal for goals that...seemingly don't require it (why does this new job platform require eliminating the assessment?) seems unwise, IMO
this seems like it's just diluting the signaling value of being on the platform? (and seems like it'll quickly create an upper and underclass of quiztakers vs nonquiztakers)
There's some risk of that. But I think that we can make it work. Basically, the model I have is that there are many different ways that an engineer can signal competency to a company and get a job. One way is doing well on our quizzes. But another way is having impressive experience at a good company, or a lot of open source contributions. Any one engineer might be really good, but look bad on one of these metrics (someone who gets stressed and does poorly on tests, or someone who has mostly worked for the government and does not have side projects). This happens a lot. When we ONLY ranked engineers by our assessments, we just missed people. The goal now is to let people show skill in any of these ways.
Well, I hope you're right and will keep an eye out (and will regardless be Fasttrack-ing in 6 months when I'm looking for FTE work, so I'll see firsthand :p)
I feel the exact same way. I went through the interview process with a handful of good startups through Triplebyte (and got offers from almost all of them) and even though I ended up taking a job with a company outside of TB, I still walked away extremely impressed and confident that I'd use the platform again in my next search. I loved that I was pre-qualified and didn't have to intensely prove my technical chops at every on site. With this change, I don't see the benefits for me to use TB over a competing job board or just applying to companies directly. With these changes, I don't plan on using TB for my next job search.
Edit: just saw the comment that TB is keeping the quiz. Would consider using the platform in the future as long as that pre-qualification track is available and that this new direction is just a superset on the existing TB features.
> part of what made triplebyte valuable (and gave y'all the hundreds-of-thousands-engineer userbase) was the _path to competence-signalling_ which avoided credentials that your platform gave, given the quiz -- companies knew that someone being on triplebyte was a strong signal, and engineers had a path to signalling competence that was one-to-many.
On the other hand, when I applied to TripleByte, their feedback to me was "we think your skills are fine, but we want somebody who can perform well in an interview". Which is exactly the opposite of the value you're attributing to them here.
Quoting them, for reference:
> We really appreciate you taking the time to work on the take home project. We're aware this requires a substantial time commitment and we are really grateful that you invested the time in completing it. We thought you wrote a great, very full featured regular expression matcher. It was especially impressive how much you dug into the academics behind regular languages.
> However we made the decision because we felt that while going through the project together during the interview, we didn't see the fluency of programming when adding to it that we had hoped for. While we specifically designed the take home project track to help overcome the difficulties of coding under time pressure with someone watching, we do still need to see a certain level of programming during the interview.
This is a perfect example of why we're making these changes (and the problem that came from us being a gatekeeper). There are lots of different ways to show skill. We don't want to be in the position of deciding who "deserves" a job.
I'd say that main value was that we opened doors for people (got them opportunities they would not have been considered for without us). I don't think what we need to be gatekeepers to do this. Yes, not everyone can succeed (get a job at at top company). But we can help everyone show their skills in the way that's best for them. We can fight ghosting and lying and create a less hostile process. We're keeping our quiz (so that people who do well on tests can get opportunities that way), and also creating a job search process for people who do not want to do a quiz (the majority of engineers). The idea is that they will show their skills other ways (past experience, side projects, open source work). By us not being a gatekeeper we open these other paths.
In the case of people who needed us to vouch that they deserved a job, yeah. But:
- In borderline cases, we didn't want to risk our credibility and companies didn't want to skip tech screens. So competent-but-not-amazing engineers got shut out.
- Our quiz is not perfect, so engineers who didn't fit what we were quizzing got shut out.
- Not everyone needs us to vouch for them. Companies (reasonably) trust that someone with a degree from a top school and years of prestigious experiences can probably do fizzbuzz, and demanding that they prove it was a barrier to them using Triplebyte.
We think it's better to have more granular ability for companies and engineers to decide what mattes to them. A prestigious company can say "we'll only talk to people who took the quiz and got a top score" (and that score is us saying "this person deserves a job"). One that desperately needs the headcount can talk to the borderline cases or decide they don't want to put the extra barrier of a quiz in the way.
Like most markets, we think the hiring market on our platform works best when it's able to respond to local conditions.
The difference is in the bar for forcing companies to set aside parts of their screening process. To get them to do that - as we needed to under our old model, because we didn't have a notion of "recommended but without the requirement to skip tech screens" - we needed to be making very strong recommendations.
I'm a little bemused at the notion of evaluating someone "strong tech skills, but can't pass an interview" and determining that the appropriate recommendation is "talk to this guy, but -- unlike with most of our candidates -- don't skip the tech screens". The recommendation seems like the opposite of the diagnosis.
How does the new ability to make that recommendation address the original problem?
Because getting some leads with less certainty is better than getting no leads with higher certainty in a tight market? Wasn't this addressed in the post?
I haven’t actually gone through the TripleByte process, but my impression of the value of the platform to engineers was that only the at least minimally capable ones could be on there.
Does this not mean that even if you apply from TripleByte that you now still need to go through FizzBuzz and all that?
> Going forward, our assessments are a purely optional means by which engineers can show their skills to employers (who overwhelmingly tell us that they trust our scoring)
So it sounds like the hiring manager can choose to skip a fizzbuzz call when they know you already passed.
By far the most exciting thing about this announcement is this feature: "Detailed information on what a recruiter did with your application."
I've been wishing for something like this for years. Especially for early-career engineers or people from non-traditional backgrounds, this is insanely valuable because it helps you to know avoid wasting time on applications that will never go anywhere; avoid typing in your resume, line by line, into another form after already submitting your pdf. Avoid writing a cover letter to a job that's already been filled. Avoid applying to a posting that's really just out there for a company to "gauge" interest, not for filling a real role.
If an application is rejected, fine. But getting a follow-up request 7 months after submitting a resume into a black hole is ridiculous, and I think any system that decreases the information asymmetry between the applicant and the employer that allows people to intelligently approach their career search is going to be tapping into something truly valuable.
When I graduated I submitted around 50 applications to open jobs around where I live, to almost no feedback at all.
I'm in this industry and doing well because of recruiter reaching out to me (multiple times) and not vice versa. The whole idea of applying for jobs just doesn't seem to work at all.
My experience is similar but at a differet level of seniorty. I've applied to ~15 jobs for engineering manager and got 1 answer.
On the other end a recruiter reached out to me and I'll be doing the final interview with their VP next week.
I don't think we can draw conclusion at that stage (we are a dataset of 2..). But I'd love to have access to the LinkedIn dataset to figure out if applying for jobs is broken at scale.
Waiting for a recuitor to call you limits your potential but makes the process easier for you. Applying yourself takes a lot of work but you get to select who you want and to target a bigger group which. But it's messy, ugly , stressful and filled with rejections for no reason.
Doing both, when you need a job, gives you the best of both worlds.
It's ugly and mimicks real life... more effort gives better results but life isn't fair and never has been. Luck factors but always fight on multiple fronts.
Then I stopped bothering when it's super obvious that isn't working. Particularly, on Linkedin you could see the number of people that has applied (maybe a paid feature, i don't remember now). And for any entry level software engineering and data science position it's always in the 100s.
The one recruiter that I had contact with did more work than all of these. I've since switched jobs (to one of the FAANG) and that was through recruiter too.
So fun fact, we built this feature as a pretty low-grade experiment, but when we ran it by actual users we got such a positive response that we ended up putting it at the top of our new front page.
Like you, we thought (correctly, as it turns out) that it would appeal to people early in their career. What surprised us is that very senior engineers also told us they loved it - we didn't think it was as much of a concern for them, but sometimes your users surprise you.
It's also a great example of how obvious low-hanging fruit gets missed until you start thinking strictly about "what can we do to make job seekers' lives better?". It's not a particularly innovative or difficult feature, and yet major job sites with a dozen times our engineering resources still haven't done it.
It's ridiculous. You can go through a couple screens, 5 coding interviews, and the end is just a "no" due to liability reasons. Literally zero information about why you didn't get an offer or if you were even close.
The last time I was a hiring manager, I committed to giving feedback to anyone I rejected after the very first screening step. This was about 25-30% of applicants, which (even for a role in considerably less demand than engineering jobs) ended up being many dozens of emails. It was a giant pain in the ass - and it's pretty emotionally draining because you're turning down people who would really, really benefit from getting hired. I did it anyway, because I'd promised to, but it was not fun.
It might not be fun but it is definitely good for your brand. In the last few years, I have interviewed at several dozen companies but the only companies which provided feedback to me were Facebook and Brex. This sub communicates to me that they take the candidate seriously and don't jerk them around.
Next time I look for a job, I will definitely be targeting both these companies.
Seriously. People ache for feedback in low information environments. Good on yall for taking customer surveys seriously (even if the data underlying it isn't always clear) because enthusiasm, though noisy, is a strong signal.
I wouldn't charge for the service, however, unless building it as a freemium model.
I'm personally very excited to see this change by an industry leader! I think that there's a seismic shift waiting to happen w.r.t. engineer hiring. The "old guard" of hazing, quizzing, and gotcha-style interviews are slowly losing ground.
The most friction when switching jobs is the interview process -- and the question is why? I've been writing code for a decade+ now, working on startups, for known tech companies, public open-source, and have written a freakin' book! But, oh, my bad, I couldn't figure out a solution to your optimization question. I forgot how to implement the zig operation in a splay tree. I might simply not function well under pressure. Maybe I'm having a bad day.
Companies are missing out on literal geniuses by using outdated hiring practices. And, I get it, Google doesn't care. Amazon doesn't care. (There's an argument that they should.) They're huge and get X,XXX applicants daily. But why are small startups using the same hiring methodologies? It quite literally makes no sense. They're shooting themselves in the foot. I'm very passionate about this, and I'm working on a book on how to hire engineers. Maybe I'll actually finish it one of these days :)
> And, I get it, Google doesn't care. Amazon doesn't care. (There's an argument that they should.)
And yet they will still email you, year after year… as a dev that's worked in finance/academic research labs and for low funded/bootstraped startups all over the world for the past 10 years… I'll never get into these big companies because I simply try to find companies that need work done yesterday, see my experience (maybe reach out to past corps i've done work for), and give me an offer for at least a short term contract (and these big companies will never do this).
The worst is when I've think I found an interesting company (that on more than one occasion, publicly likes to complain about not having enough devs… lol), start talking a bit and they send me a triplebyte link… maybe it will be different this time, but I doubt it because these behaviors are too ingrained in a lot of corporate processes (even if its mostly "big co does this, me small co must copy"-NPC type thinking)…
For what it's worth on the hiring side, applicants lie (or more charitably, wildly overestimate their own abilities) all the time. Of resumes I see, 50% of them will not show up on time to their interview. Of the 50% that do, 80% cannot pass fizzbuzz.
Past companies and open source contributions are not perfect indicators. I've seen people with stellar resumes fail to even know the syntax for an 'if' statement in any language of their choosing. I've had people absolutely ace takehome exams, yet in person don't know how to write a function.
False positives are also much more expensive than false negatives, so companies would rather accidentally weed out good candidates than risk hiring bad ones.
To put some hard numbers to this: further down this thread, there's a post about how "any engineer" could answer a question the poster thinks is too easy. I looked up the question in our back end and, in fact, barely a third of people who take our quiz get it right (the correct answer isn't even the most common one!).
That's maybe a little unfair (plenty of people who aren't engineers take our quiz out of curiosity), but the basic point here - that companies are frustrated because they keep getting fizzbuzz-incapable applicants - is more-or-less accurate. That has been (and continues to be) a big part of our offering on the other side of our platform: in the same way that we can provide data about company behavior to engineers, we can provide independent verification of an applicant's skills to companies.
> Of resumes I see, 50% of them will not show up on time to their interview. Of the 50% that do, 80% cannot pass fizzbuzz.
Wow, sounds like you either need to change your sourcing or improve your early screening. If that many poor candidates are making it to the interview stage something is wrong with the process.
I should have clarified, the first interview is just a code screen. People submit resumes, and we test them with fizzbuzz. 90% of applicants are removed at this first filter.
If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection. I've heard so much about this "programmer charlatan" that lies on his resume and ends up blowing up million dollar systems, but have yet to see any tangible proof.
> Past companies and open source contributions are not perfect indicators.
This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase? I run a tiny throwaway open source project (~600 GH stars) and even my reviews are pretty stringent. This point of view is not consistent. It's like saying "I know people that ran in marathons, but couldn't even run for half a mile in my interview."
> If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection. I've heard so much about this "programmer charlatan" that lies on his resume and ends up blowing up million dollar systems, but have yet to see any tangible proof.
I've seen similar. I wouldn't say 80%, but somewhere in the 20% range of people on phone screens. If I ask a similar modulo question that isn't obviously a rephrased FizzBuzz (i.e. "write a function that returns every 4th item in a list"), that percentage goes up dramatically. A lot of candidates memorize FizzBuzz without actually understanding what's happening and how to use it.
> This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase? I run a tiny throwaway open source project (~600 GH stars) and even my reviews are pretty stringent. This point of view is not consistent. It's like saying "I know people that ran in marathons, but couldn't even run for half a mile in my interview."
There are no time constraints, and no way to tell how much of their contribution is from their own knowledge and how much is copied from StackOverflow/other open source projects/etc. The point of view is consistent; I can't run a marathon, but I can walk 26 miles. That's fine for the local charity marathon, but it's not going to cut it if I want to do competitive marathons. The constraints and expectations are wildly different.
Even ignoring the fact that you keep moving the goalposts and are taking my "marathon" analogy way too literally, the argument that any kind of real-world coding involves "time constraints" akin to a 30-minute stress-ridden whiteboard interview is just bonkers.
Programming verbally over the phone is a skill that requires learning.
I can run a marathon but I can't run 100. Your interview is one of many that I am making time over lunch. I'm tired and getting a perfect mark on a take home isn't as important as getting it done quickly so I can find time to apply to other positions before I go back to work.
If you are not paying someone to do the interview you are not going to get anyone's best. Even if you pay people are trying to juggle things to get the interview to work.
And to the parent post. 50% of people being late means they took the time out to talk to you and come to your offices and didn't gage the time or didn't think it mattered. That doesn't make them bad programmers and shouldn't even matter to the interviewer. Was the car service the employer used late picking the employee or did they have to come on their own?
> If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection. I've heard so much about this "programmer charlatan" that lies on his resume and ends up blowing up million dollar systems, but have yet to see any tangible proof.
I dunno about the back half (blowing up million dollar system) but the front half, certainly.
We used to ask "write min()" at a previous employ, and the pass rate was probably <50%. I still to this day structure my coding question around "it has to get them to write a for() loop". It's not trick questions (I understand why people don't like those, and those can die out, yes.) it's things like "here's a sample grammar, how would we go about parsing it?" which is something that I've used any number of times in my career.¹ Other candidates fail to display understanding, let alone deep understanding, about things like HTTP or Linux. Not exactly niche topics, and during this example period my employer at the time was clearly posting for "backend software engineer". And not just one or the other, but all of them, simultaneously. Since I can't establish any proof of anything, so… what other choice is there, but to pass?
¹while there are parsing libraries, yes, and I'll use those first / when I can, I would also say that 50% of the time, for whatever reason — better error reporting, better control — I end up going with a partially or mostly hand-rolled state machine or recursive descent parser. Also, while I wish people would just encode their data in something like JSON, or YAML, or whatever, it doesn't really matter, they. keep. making. new. grammars. Often inside those other grammars. Just today I had to deal with a config — a YAML config! — that wanted "key values". Turns out, it wanted strings formatted like "key:value", not YAML mappings.
> If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection.
With a bad pipeline, it's extremely likely, especially if you cheap out and try the bootcamp route (there's a reason Lambda is desperate enough they will "loan" a student you can try before you hire!).
>This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase?
They could maybe just be making small or simple changes. Or perhaps mostly working on markup, templates, CSS, etc.
It's possible they know a bit of programming but just don't happen to know how to check if a number is evenly divisible by another number. This isn't something an open source project would necessarily know you don't know. (I agree it's something any programmer should know, but I can see how someone could contribute certain things to certain projects without that knowledge gap being discovered.)
I learned about the modulo operator very early on, but if I didn't know it existed, I'd admittedly struggle with the question for a bit and would seem like I don't know how to program; especially in a pressured interview environment. If it were Python or something I think I'd come up with something awkward like `if i / divisor == float(int(i / divisor))` within a minute or a few minutes, but who knows, maybe the pressure would mess with me.
(N.B. I was being a bit hyperbolic, but, yes, for example, I highly doubt Steve Jobs could.) The spirit of my point is that most smart people don't care about the kind of idiotic minutia typical hiring panels ask. They care about interesting/creative problems.
I mean Steve Jobs may not be a great software developer. I don't think companies are hiring for a Steve Jobs. You could be really good at other things and not be great at programming
A SJ type is what TB would need to orient this pivot… to turn the problem around. Sourcing ideas on HN will only lead to “faster horses” suggestions. BTW did the TB interview after been lured by dark patterns on social media a while back when I was still using. Got the bloom filter from a hipster that was playing on his phone while I coded tictactoe in a langage I hadn’t used In years. Complete waste of time. Then they wanted to make profiles I hadn’t signed up for public, good thing they listened to the backlash. Just saying; it seems the management needs to use an hypothetical service like theirs but that works to find someone to help them on the angle for that pivot… a headhunter ?:-)
Tongue in cheek, good luck on the pivot guys, it is a tough problem.
I agree, To me discovery process is broken. FANG sends email every year once you have already reached onsite. But I am trying to get into this tech companies new IPO's not FANG. I have applied to many good ones.
I have Github repo.
I have my own portfolio.
I have wrote flutter app deployed to play store to demonstrate I can function individually and still can write production ready code.
And All the above just on my own spare time.
But every company I have applied told me my skill set doesn't aligned with them.
I get response from them that there is no traction on my resume. While some friends get interview after interview and others don't.
I am not saying that I am the best but we need different process where entry is given and now show me what you can do.
Decades of poor technical interview practices started by a blog post (the one about fizzbuzz). It's amazing how a single blog post changed the entire industry by causing such fear about "false positives". Looking at that blog post now, it's obvious how it lacks real analysis of the situation:
* The blog post complains that even experienced engineers they interviewed couldn't write some simple code on paper. Did anyone stop and think that maybe it's writing code on paper in an interview setting that's the problem? Stop and think about this. Imagine writing code like you usually do in a text editor. And then writing code with pen and paper. Backspace? Can't do that. Copy/paste some line to move code? Can't do that. Add some newlines? Can't do that. Run the code using the compiler? Can't do that. Of course most developers, even experienced ones, will do poorly under these conditions, this is not how code is normally written. On top of the already stressful situation of an interview, there's this suddenly completely foreign and limited way to write code.
* Even the blog post only mentioned fizzbuzz as a quick, easy question to filter out zero coding ability. Another way to say this, it was meant to filter out coding skill <= 0. Not skill <= 50 or skill <= 90. Zero. After that, the interview was supposed to move on to things that actually matter for software engineering. The present day interview has strayed extremely far from this. These days, you get asked two leetcode questions during an interview and even if you perfectly answer the first one, if you don't get the second one then you're an automatic fail. That was certainly not the intent of the original fizzbuzz, since answering any leetcode clearly shows coding ability. And the interview never has a chance to move on to the part that actually discusses software engineering skills. Or even if it does and they do very well on those interviews, the failure of only getting 1/2 leetcode questions correct overrules everything else since they must avoid "false positives".
What I find particularly jarring about tech interviews is that they often don't adapt to the actual role you're interviewing for.
I moved from engineering to management over 5 years ago now and still in interviews for positions like "head of" in 100+ people companies or other management roles in bigger companies I got asked the usual algorithm questions.
I just have zero interest to prepare for these interviews, I don't have the time or the motivation, and I honestly just don't care about the answers. Most of my job, and my team's job, has nothing to do with them.
The world might be different in FAANG, and admittedly I have not worked there (though I have worked for other top companies) – but most companies are not at that scale, and they should not use the same metrics, particularly for management roles.
Without passing the minimum bar, how does Triplebyte distinguish itself from other 1000 platforms where people post resumes?
Also, dissing those who have passed the bar as "engineers who like tests" is disrespectful. When they first launched, passing the bar makes you (in their words) a highly qualified engineer, and now passing the bar means nothing except that you only "like tests"...lol
Fundamentally, I think this invalidates Triplebyte's business model. Companies don't really care if someone passes your tests. They still put you through LC type interviews onsite. They simply save a phone screen.
> Without passing the minimum bar, how does Triplebyte distinguish itself from other 1000 platforms where people post resumes?
We still have the quiz, and we still show performance on it prominently on your profile (provided you've chosen to share those scores). In fact, we've put a lot of energy into improving the trust companies place in our quizzes over the past year precisely because we think that skills data is important.
> Also, dissing those who have passed the bar as "engineers who like tests" is disrespectful. When they first launched, passing the bar makes you (in their words) a highly qualified engineer, and now passing the bar means nothing except that you only "like tests"...lol
Heh, yeah, that's fair, at least to a point.
Our quizzes are predictive. We know this from a lot of objective data, it's what you'd expect subjectively, and companies do tell us that they trust and value that data. That does not mean that our tests have no bias towards certain personality types. Different people respond differently to testing, and that does have a differential effect, not because our tests are bad but because testing is inherently somewhat artificial.
I actually taught test prep before joining Triplebyte years ago, so I've been this first-hand: it was not uncommon for a student who'd been doing well in practice sessions to crumple under the pressure of the real thing. That doesn't mean the test is bad, it just means that some people fare better on tests than others for reasons other than just their raw ability. When we say "likes tests", that's more what we mean: not that that's the only reason someone does well on a test, but that people who perform well in isolated, pressured environments do better relative to their skills than others.
Well, yeah - weren't we pretty upfront about the fact that we were only serving a relatively small number of people? That's not the same thing as "we were only taking good engineers and now we take a new cohort of just bad engineers", though.
I think that goes to another ability. If we need a candidate to gracefully handle a 3 AM production outage, we should expect them to handle an interview.
I worked at Triplebyte several years ago (but way before any of these changes were being discussed - this blog post is the first I'm hearing of them). I'm an engineer, but I spent a lot of time with the account management team talking to companies. And the biggest thing I remember is how much those companies pushed back on our "rules". They were all eager to talk to the top .1% of candidates, but otherwise just seemed to want an unfiltered firehose of resumes.
So while in theory I like the idea of this meritocratic minimum bar granting special privileges (I liked it enough to join Triplebyte!), in practice we seemed to be fighting an uphill battle with all but a few candidates each month.
On the other hand, I have wasted a lot of time in conversations with recruiters only to be let down at the end by mismatched expectations. I'm interested to see if this new approach can make a meaningful difference in that problem.
> We want to be the job search platform that puts engineers in control.
I love this, but I'm curious about the incentives here. Triplebyte only makes money from companies, not engineers. In the long run, can engineer-centric intentions override a business model based around companies and recruiters staying happy? I hope so!
This exactly. Their messaging is over the top "we help engineers get leverage over companies!" I have to wonder what Triplebyte is telling the companies who actually pay them all their money.
Triplebyte has always seemed scummy and this obvious--and unacknowledged--contradiction makes me trust them even less.
We do also have differentiating features on the company side. It's not like we're abandoning the idea of building features for companies or anything.
The job search, like other matching problems, is often not zero sum. So building (say) a better search interface for companies benefits engineers as well, because both sides of the market have an interest in a good match. What we're saying here is that in the cases where things -are- zero sum, the power - and therefore the incentives - lie with engineers and not with employers.
Companies do pay us. But I think that our real long-term incentives still pull toward building what engineers want, not what companies want. I think that platforms like LinkedIn and Indeed have just gotten this wrong, because they focus on all jobs (not just engineers), and because demand for engineers is stronger than it has ever been. Take a look at this thread from last week:
Our bet is that LinkedIn is going to fragment. They are just not creating a hiring process that most engineers like. People tend to either get ghosted, or overwhelmed with low-relevance inbound (almost no one gets the "right" amount of attention). Companies need to go where the best engineers are, not the other way around. So I think our long-term incentive is to fix these problems.
In any case, I'm committed to giving this a try. There is danger that we get pulled toward building for companies. I want top guard against this by being public about what we're doing, and "showing our work" as we go.
> our real long-term incentives still pull toward building what engineers want, not what companies want.
> There is danger that we get pulled toward building for companies.
A common phrase thrown around with free services is "if you're not paying for it, you are the product".
> For example, companies aren’t normally incentivized to provide salary and culture data. But we can force their hand by promoting transparent companies in our search rankings.
Maybe I didn't read the article carefully enough, but are you planning to continue charging companies $15k - $30k for the ability to access candidates on your platform?
If so, companies are still your customers. And if you're building and optimizing your product for people who aren't your customers, your real customers (companies) may not be happy with you which will hurt retention, etc.
Maybe I skimmed the article too quickly, but it seems strange to charge companies $20k+ to post a job on your platform, and then actively do things that "force their hand". It might be a net benefit to the engineers on the platform, but I wonder how it will work from a business model perspective since you're potentially creating adversarial relationships with your "real" customers (companies paying you to access your candidates).
Edit: But maybe it's by design, if you actively remove employers who aren't abiding by your philosophy. Although again, that means turning away customers, which means turning away revenue, which in my mind raises questions about the overall business model. A lot of conflicting interests.
I generally agree with the phrase "if you're not paying for a product, you're the product". But the market for engineers is just so lopsided that I think it's less true here. There are a lot of recruiting companies. The only real thing that sets one apart from others is whether they have candidates. So one way to look at it is that yes, we are incentivized to build what companies want, but the main thing they want is for us to have engineers. And the only way we get engineers is by building what engineers want.
That does not fully express my motivations. I am an engineer and find the idea of making the process better for engineers more exciting than making it better for companies. But it explains how I think the incentives work.
I see a lot of people in the comments describe triplebyte as a “market leader” and discuss how good they are. I can’t believe how gullible everyone is especially on a startup forum. Let me spell it out to y’all, nobody pivots and changes their entire business model if they are crushing it. So the implication is that triplebyte are not crushing it. So, this probably has gone the same way as lots of other tech startups. They’ve raised a bunch of money spent it all on ads , distorted the market for a while (usually just the advertising market) and now are on the way out. No matter what fluff or spin they put on it, that’s what’s happening.
It's a brand of artificial lawn for sports arenas, plastic "grass", which has given rise to the idiom "astroturf campaign" = "fake grassroots campaign".
I remember my experience as a candidate on Hired. I pretty consistently got low-grade startup trash (think collectible art trading cards, but digital!) with lowballed comp. If Triplebyte was worse, then it's no wonder they placed so few people.
I’ve had a pretty good experience using Hired the past few months. After getting a lot of small startups, I put that I was only interested in larger companies with 500+ people and I don’t see as many now. There seem to be a fair amount of large companies hiring remote after the pandemic WFH experiment, which is great for me outside of a major tech hub. Most places meet or exceed my salary expectations. I’ve gotten interviews with a few companies I’m quite interested in and would have never thought to apply at. This has required relatively little effort on my part as I usually write a cover letter and tailor my resume only to get rejected without reason, but with Hired I just set up my profile and get hits. I definitely would recommend it to anyone looking around for another job.
Fair comment. I certainly don't think of us as a market leader. LinkedIn is the market leader. Honestly, the last year has been pretty hard for us (COVID and the problems with our model that I talk about in the post).
That said, I think a hiring process that puts engineers in control (no ghosting, get data on when a recruiter looks at your application, search ranking by whether companies lie to candidates) is something that should exist. I want to try to build it!
As a candidate, I'm willing to put up with sometimes shoddy interview outcomes (e.g., ghosting) if my funnel is far greater and consequently leads to a more competitive offer, which is the end goal. A narrower funnel, that pre-selects for higher expectations of how companies engage in the process, may end up falling short of what I'm optimizing for.
Exactly, I don't understand all this moaning. You're only doing this every some years, you might as well try to optimize it rather than be picky, because it can make a big difference both career and compensation wise.
Idiotic HR does not correlate in any way with a worse or better company, at least in my experience.
This is an admirable aspiration. Although, if I think about, the companies spend much more time with a recruiting/sourcing product. And the candidates change jobs like once every 1.5-2 years? So optimising for a candidate experience doesn't seem right considering they're not the customer.
If you want to put candidates in control, build a CRM tool. When a recruiter pings me on LinkedIn or emails me, I have a URL I can send them to with an intake form. No more calls wasting my time, I can see the company/role/salary/etc. up front.
The trick here is you need to figure out a business model where you don't take money from employers, because once you do its just the long slide to becoming a shittier LinkedIn.
We've talked about things in this area, but this particular framing is pretty interesting. We're talking about it now.
I personally like this proposal and am probably going to at least draft a hypothetical spec of it and see how it fleshes out. Thanks for the suggestion!
It's unbelievable that nobody has done this well - I and most people I know are tracking their own job searches on a spreadsheet.
Pretty obvious value-add for job searchers. Not to mention that having access to this data would enable tons of other product features. Shows that most services/sites don't care that much about the applicant experience.
If you're interested in comparing notes, I'd love to hop on a call with you. (We've got some bookable slots linked at the bottom of the blog post, but I can make time elsewhere if you'd like.)
Colorado now requires job postings to include something more or less like a "reasonable salary range". It hasn't been perfect, for example there are some companies who now just restrict their online job postings to say "except Colorado" instead of adding a salary.
But as someone living in Colorado it's been nice to know the salary range up-front more often and to have confidence that I can ask them for a salary range without getting the question turned back on me: "Well, what are you looking for?"
I think as they start enforcing things more employers will avoid colorado until kinks worked out.
If you have a janitor in CO, every job is an improvement and potential promotion. Having to first go to that janitor for every position is going to wear out FAST. Posting for the CTO position? Go to janitor first. Posting for CEO position - same thing - that's what is going to drive folks away.
Unfortunately it looks like companies can just state wide ranges (which don't help applicants understand the market any better) and they can also omit other compensation elements beyond the base salary component (stocks, cash bonuses). That makes the disclosed information not very useful.
In my experience this is worse than nothing, because the posted pay is so much lower than the companies are actually willing to offer. Positions posted for FAANG that I know are offering $250k+ total comp are posted with statements like “pay starting at $110,000 but varies based on your unique qualifications and experience”.
I'm unsure of exact enforcement and such but you could report them if you wanted. I also don't mind FAANG so much since there's so much info available for those positions on like levels.fyi.
The biggest strength for me is that I can ask them and they have to tell me. When I was job searching before I'd ask the recruiter and then they'd always ask me what my target is. Not showing your cards is like negotiation 101. Plus it's made fielding messages and emails easier. If they provide a range, I can just easily say "This position sounds interesting but unfortunately the salary range isn't what I'm looking for currently."
You're right that a lot of positions aren't great about this, I've literally seen "Pay starting at $0.01" or ranges literally $30k to +$200k. But I've seen it as a benefit in the negotiation process and it's just been easier to field recruiters.
> total comp
This law only covers salary as far as I know, so equity and such aren't included. Total comp is different.
I guess not, but this is also a law that applies to _all_ job postings for CO. I feel like a lot of people are criticizing this law when I see it as an absolute step in the right direction.
It's not perfect, but I feel like it's wildly helpful for a ton of people. It helps transparency for a lot of positions and I feel like it rewards honest companies. If equity and bonuses are a big part I think it reduces the overhead for talking about salary to clear the way to talk more about equity and bonuses.
It's also in the early stages and CO is the only state with a law that requires disclosure up-front to my knowledge (someone mentioned CA has a law, but they only have to answer if asked). It feels like a fantastic first-step to me.
> I think a hiring process that puts engineers in control (no ghosting, get data on when a recruiter looks at your application, search ranking by whether companies lie to candidates) is something that should exist. I want to try to build it!
This is a space that unions fill well compared to for-profit recruiting agencies. The organizations give engineers control and leverage in hiring, and can help them up-negotiate. They also allow employers to hire from pools of skilled talent.
Whereas with recruiting agencies, the customers are employers, not the candidates. Incentives are not aligned to give candidates control.
While I still slightly prefer the direct experience for other reasons, the recruiting agency experience was pretty good for me in its own way. More specifically, when it comes to salary negotiation. Let me elaborate.
People at the recruiting agency usually get paid a commission as a percentage of the first year of the hiree’s salary (not taken from your paycheck, but just in general). Which means, it is in their interest to get you as high of a comp as possible. Which also means that I dont have to bs around with the recruiter about my salary expectations or anything like that. I give them the upper range of what i want, they tell me either “sounds good” or “they are easily willing to pay more, so why not try $X instead” or “they cannot pay that much, but there is this other company that we work with that can offer you as much.”
And if you work with one recruiter consistently, it gets even easier, because now that the recruiter knows my total comp expectations, she pre-filters opportunities for me and only contacts me if there is a position she is aware of that pays at least as much as I want or more.
Of course there are downsides too, for example, the limited choice of companies that the recruiting agency is working with at any given time, so i still have to find some opportunities and interview with those companies on my own. But as a supplementary option, working with a good recruiting agency has been quite nice for me.
P.S. The type of a recruiting agency i am talking about is the one that is focused on a specific sector (in my case, fintech/quant), so it might be a completely different story for a “generic software dev” recruiting agency.
> Which means, it is in their interest to get you as high of a comp as possible.
Beware: this is not universally true. What you typically see is that it leads to closing as many candidates as possible. A recruiter can make more by closing 2 candidates at 80% of max comp than 1 candidate at 100% in the same time.
This is a well known behavior in real estate sales, which have a similar commission structure.
>This is a well known behavior in real estate sales, which have a similar commission structure.
Even worse, real estate pays both sides so realtors are heavily incentivized to get their buyers to purchase from their own sellers so they can get both sides of the commission. Instead of paying them 3% to act on your behalf and do a bunch of leg work they get double and now they can do less work and aren't strictly working for your benefit.
Working on commission just leads to wanting to make a deal happen. Even ignoring the time aspect of it, if a recruiter has much better odds of making a placement at 80% max comp compared to 100% max comp, the 80% is still objectively better. Why risk not closing the deal when the expected return is negative?
Agreed, but as you said, it all depends. I am not sure if it was specific to that individual recruiter that I have been working with or if it is just a general practice with fintech recruiters, but if my recruiter was going for the high volume, she had made a really poor choice deciding to continue working with me.
I am saying that because I’ve been working with her for at least 3 years now, and I am yet to take a single offer from the companies she set me up with (either due to them getting cold feet last second or simply due to me not passing some of those interviews or me declining due to some red flags with the companies). If it was all just about high volume for her, it would have been in her interest to drop involvement with me after the 3rd unsuccessful attempt to set me up with a fintech company.
Collective bargaining is inherently about eliminating competition between workers. That is, it creates a labour monopoly to balance out a labour monopsony. That might (might!) be a good thing for autoworkers. It would be an unmitigated disaster for software engineers, and the tech sector in general.
People who write code are unbelievably productive compared to just about any other job function, and software engineers (who we'll define as people who write code for tech companies) are even more so. Part of eliminating competition between workers is making sure that nobody works harder than anyone else. That would be devastating to the productivity of software engineers, which would make them less valuable to tech companies. That would lead to a decline in salaries, and general stagnation of tech.
Another aspect of preventing competition is seniority. One of the archetypes of the tech world is the wunderkind—the brilliant kid fresh out of school that rockets to the top. A union would tell that kid, "sorry, you'll have to pay your dues as a junior engineer for 5 more years, then you'll be eligible for promotion. In the meantime, we'll give that job to cwp, he's got the required seniority."
Just think how this would affect tech companies. Sorry, you can't pay more to attract the best. If you're really careful about interviewing, you might be able to find some people with both skill and seniority, but it'll be tough because smart, ambitious people have all gone to Wall St or whatever industry has the best opportunities.
Agree with most, but your valuation of coders is humorous. Save a few, they're not that special. Most are more similar to the factory line worker than other job functions.
> Collective bargaining is inherently about eliminating competition between workers
A yet every movie star in the US is in the same union as B-movie and infomercial actors.
> Another aspect of preventing competition is seniority.
Weird, considering pro-athlete unions regularly help kids out of college quickly become multi-millionaires.
> One of the archetypes of the tech world is the wunderkind—the brilliant kid fresh out of school that rockets to the top.
Weird, considering unions have been at the forefront of education, apprenticeships and training that helps kids out of school rocket to the top, the same apprenticeships and training that companies refuse to fund themselves.
> Sorry, you can't pay more to attract the best
Weird, considering SAG-AFTRA regularly helps movie stars up-negotiate multi-million dollar contracts.
A small bit of feedback here. I tried TripleByte a couple of years ago and the experience was really polished. I passed the interview and got great detailed feedback which I really appreciated.
However, I dropped out completely because TripleByte wouldn't let me see companies without entering in a desired comp number. My recruiter at triplebyte said I could just put in "$1 or $1,000,000" to get past it, but that just feels like the same kind of corporate HR bullshit that people have complained about for years. Just Google "recruiter won't proceed without desired salary" and you'll understand how many folks dislike that question.
The experience made me feel that Triplebyte wasn't interested in putting me in control of the process at all if they have such a user hostile requirement and require me to work around it by putting in fake numbers. I understand that many people will have no problem providing this info, but I prefer to see what companies offer me because it tells me how they value me and the position they're hiring for.
You're totally right. If we'd been thinking in this mode when that particular requirement was built, it wouldn't have been. It isn't required today (and IIRC hasn't been for a while now).
I never, ever, ever give anyone a number until the salary negotiation stage of the interview (ie. after I've been given a job offer), and even then I only give the company a number as a counter to their offer. I always do this negotiation myself, and never let anyone else negotiate for me.
Almost all recruiters have been fine with that, but a couple haven't. To such people I wish a good day and go my own way. There are plenty of other recruiters in the sea.
A particular problem with doing that is discussed in the post; you risk everyone's time being wasted if the company will never be able to meet your target.
Yep. I won't get to the offer stage without us having discussed money. I don't want to waste either of our time only to find out someone's only willing to pay 70% of my bottom line.
1 - I get real-world interview practice, and that always helps me in future interviews. Many companies ask the same or similar questions, and the more I'll be ready for them being asked again. In addition, when I'm surprised by a question I'll research it thoroughly when I get home and won't be surprised by that question again.
2 - I get a bit of an inside peek in to how the companies I interview at work. That's a privilege not afforded to many outsiders, and how some of these companies (some of which are world class) can and has helped me at other companies.
3 - A company might bring me in to interview for one position, but after the interview might determine I'm better suited for another. If I didn't interview with them they'd never know.
4 - As I interview with people I get a chance to make a good impression on them, so even if they don't wind up hiring me for that position they might want me when a different position opens up (whether at that company or another company the people I interviewed with moved to).
5 - By letting companies make the first offer I myself get a good salary survey of the minimum what all the companies I interview with are willing to pay for someone with my qualifications in that position. Of course I try to do my own research ahead of time, but getting salary numbers without getting a job offer is not always possible or reliable.
6 - If I blurt out a number first I'll it may put me at a disadvantage in the negotiation, as they'll just start the negotiation from what I say even if they would have been happy to have paid way more.
> I get real-world interview practice, and that always helps me in future interviews.
I hear this a lot, but how much interview practice do you really need? After the third or fourth set of interviews, they start to really blur together for me.
None of your points are false, but if you're casting a wide net and taking every interview without any idea if the company can even afford you, that's a lot of hours spent. The last time I looked for a job, I interviewed at only four companies and spent an average of probably 5-6 hours on each, and it was exhausting. I can't imagine making a habit out of it while also working full-time.
I agree it's exhausting. And it can feel very bad and demotivating for the whole thing to end up without an offer or some half-assed "culture fit" "explanation" that is likely copy-pasted from an online script collection.
But, in my case, doing many interviews helped me understand what negative signals I am emitting during interviews [according to the interviewers]. There's nothing like an outsider's perspective, even if you disagree with it.
---
Example: in one of my last interviews I was told that I am too "cocky" and "over-confident" because when I was given real-time modifications to a homework assignment (that I completed before) I was smiling and saying "oh, that's easy!" and proceeded to make the changes live in my editor -- while sharing screen.
This feedback left a very WTF feeling in me. Dude, I am (a) enthusiastic about the task and (b) have enough experience to show you, in real time, that I can modify the code right there and then to suit the new requirements, and (c) chatting with the interviewing team while doing it... and all you can gleam from that is that I am overconfident and cocky? Seriously, get a grip.
For the record, I strongly disagree with their take. But it gave me a very interesting perspective -- namely to scan for the more introverted / likely-insecure people in the interviewing dev team and try and act a little more modestly to not rub them the wrong way.
It's a fact of life that most people suck at interviewing -- as I hope that I demonstrated here. One of the guys in that team simply dislikes certain kinds of people and he lets this go against his better professional judgement about my abilities. Not cool, right? Especially when you also take into account that the CTO who also attended all interviews agreed that my tech acumen is top-notch and they seriously couldn't catch me with my pants down no matter what they did.
So again, this helps me gather experience and culture and makes me more adaptive in the future.
Does it hurt your ego? Absolutely. Does it shake your preconceived notions? Much more than one would think beforehand. Is it exhausting? Hell yes, sometimes after an interview I didn't want to do another one for a week.
But, is it also very valuable to help you improve your interviewing and negotiation skills? Yes!
That's why I did lot of interviews the last time around. And I'd do it again.
For a big enough company, you can collect a few data points via Glassdoor[0], levels.fyi, etc to make sure that they pay in the range you're looking for before you even begin the interview process. For smaller companies, you're absolutely right, although I personally tend to give a very broad range[0] at the beginning of the process, and only narrow that down at the offer stage. That's worked well for me in the past.
The nice thing about the explosion of remote work due to COVID is that engineering salaries are beginning to normalize across the US, and there is now a lot more salary data available online, so it's easier than ever to see what you're "worth" on the market on average. This at least gives you an anchor when having these conversations, so you're not going in blind, and gives you a lot more leverage if they low-ball you.
[0]: Yes, I'm aware of the drawbacks of self-reported data, but it's still broadly useful in aggregate.
[1]: I'm making up numbers, but something like $120k - $200k, with the lower number being the minimum you'd realistically take. Usually recruiters are honest up front if they can't even afford your absolute minimum, but it still leaves you with a lot of negotiation room once the offer comes in.
"I'm making up numbers, but something like $120k - $200k, with the lower number being the minimum you'd realistically take. Usually recruiters are honest up front if they can't even afford your absolute minimum, but it still leaves you with a lot of negotiation room once the offer comes in."
Why even give an upper limit? It's not like you'd refuse $400k if it was offered to you, would you?
Most companies will naturally start negotiation near the lower end of your range, even if they would have started way higher had you let them make the offer first. I really don't see an upside to this strategy.
Because if you just give one number, even if that's your absolute minimum, the entire salary conversation is going to be anchored around that one number, and you're going to have a harder time asking for 50+% more than that. If you give a broad range, it's easier to justify countering with something closer to the top of that range.
> It's not like you'd refuse $400k if it was offered to you, would you?
Of course not, but nobody has ever offered me that. :) In all seriousness, yes, you run the risk of your range coming in below what the company would be willing to pay, but in my experience (and those of friends and acquaintances), outside of FAANG, I have found that to be extremely rare if your range is wide enough. The companies that pay super well are generally big, and thus well-known to pay that much, and I wouldn't give them any salary numbers at all because I'll have a pretty good idea of what they'll pay me with a bit of research prior to interviewing.
Of course, anecdote != data, so YMMV and all that. This is just my experience interviewing at mostly smaller companies.
> For a big enough company, you can collect a few data points via Glassdoor[0], levels.fyi, etc to make sure that they pay in the range you're looking for before you even begin the interview process.
Agreed though in practice that just moves the pre-interview conversation from a comp discussion to a leveling discussion.
They uniformly underestimate big tech compensation by 50%+, because they've never clearly delineated salary from total compensation (and stock grants are a majority of comp for a lot of senior FAANG employees).
Luckily levels.fyi has fairly accurate data for those same companies.
Why should it always be the candidate's problem to reduced that risk and not the companies?
That time would not be "wasted" if the company had done due diligence on what the position and candidate were "worth" and revealed their target (or target range) to the candidate.
The real reason, I'm sure, is that companies think they hold the upper hand in the negotiation. And a lot of time with a lot of candidates that's true. It's clearly _not_ true for the class of candidate who's realistically got a $300k+ target. You are totally not going to successfully headhunt anyone in the top 1% or 5% of FAANG engineers if you try to dick them around with recruiting power imbalance power plays. _They_ don't need your bullshit. They will totally walk away from whatever time you've "wasted" so far trying to recruit them. You need them _way_ more than they need you.
(And I'm pretty sure that generalises a lot further down the experience/renumeration food chain than most devs realise. Fresh grads or devs with only a few years of experience might need to play the stupid recruiter games, but most companies trying to fill a mid level or senior role need the right candidates way more than those candidates need any one specific job. You can and should be able and willing to walk away from any potential job offer over stupid games like this...)
On hiring side we screen folks out pretty quickly who can't discuss money.
I used to be much more relaxed about this, but if you get burned doing a few rounds of interviews and it was a waste you wise up pretty quickly - teams do not like time wasted.
I've also found that folks who can't / won't talk money - usually not great hires for a bunch of reasons.
We're happy to talk through the different positions, levels, progressions and comp ranges for them all. But you have to have a clue about what type of comp would work for you (early on in initial call or two) or it's just not worth continuing forward.
You say "I've also found that folks who can't / won't talk money - usually not great hires for a bunch of reasons."
The same goes in my experience for companies which insist on talking salary before the job offer. It's a red flag.
"you have to have a clue about what type of comp would work for you (early on in initial call or two) or it's just not worth continuing forward"
Of course I have a clue, but I'm not going to put myself at a disadvantage in the negotiation by revealing it before I've been given a firm job offer and heard a number from the company first.
Also, in my experience recruiters are the ones who push hardest to hear salary expectations up front. They know it'll give them a leg up in negotiations, and some of them are even paid specifically to get that number (as one recruiter was brazen enough to reveal to me).
Why can't you simply ask the candidate "We offer $min to $max for this position, is that acceptable?" That lets you and the candidate know whether proceeding further is a good use of time.
No it doesn’t. The company is fundamentally in a much better negotiating position (they have much more data on salaries) and carry way less risk than the employee (a company has many employees, each employee has just one job).
In each negotiation about a position, the company is negotiating only about one of its many positions; the candidate[1] about what will (usually) be their only job. And companies usually, AFAIK, get a lot more applications[1] per position than each applicant[1] gets offers per job search.
So no, AFAICS you are wrong and the original thesis of "the company holds the advantage" is correct.
Giving a number early on in the process only weakens my position when it comes time to negotiate, waiting for an intital offer to counter is in my best interest. I apologize if that makes me a 'not great hire'.
If you're happy to provide a range, great, if I'm still there doesn't that generally indicate that the range is in line with my expectations?
I guess your comment kind of makes sense, I can see how corporate might interpret 'this person looks out for themselves' as a potentially bad hire.
> On hiring side we screen folks out pretty quickly who can't discuss money.
That’s because it benefits you as an employer.
I’ve been on the hiring side and I know there’s a budget. If someone is too high you want to push them out right away.
My flatmate in Singapore about 6 years ago was looking for a new job. Her salary was about 2200/m and she said the recruiter kept asking about salary.
I told her not to discuss it. She doesn’t /need/ a new job. If the company really values her then they will give her an offer. They have a budget. They offered her around 4600/m.
If she told them she was on 2200 it’s guaranteed they would have given her only a small bump.
Companies should never ask for salary info. Because it’s only used to get cheap labour.
I've been a hiring manager in tech for a long time now, and recently as a candidate. As has already been commented, all companies have a budget. They may be able to increase that for the right candidate, but there is always a range, why not disclose that? As a candidate I know I'm not wasting my time, and the actual during the actual negotiation, its then that the 'value' of the new candidate to the org can be assessed. Pay people what they are worth, which is of course a reflection of their experience, talent, and attitude.
This is great actually because you avoid wasting candidate's time, they can move on quickly to better companies that have a more fitting interview approach.
Indeed as a candidate, I don't want to waste my time either.
Your clients know their budgets. Why not just be upfront about that to the candidates. Tell us your budget or reasonable expected range your client can pay. Then if (when) we find that it's well below what some other hungry companies are paying, we can save you and us time by passing.
If you know your client is paying low, then just be open about it. Something like, "Ok, this client is a bit below market (and tell the upper limit they will pay), but they have these good things to offer." If the client has no good things to offer, and the pay is low, then you're still likely to not seal the deal with a candidate no matter what. So just cut the game and get to the point.
The problem is that companies use any salary information you provide as a candidate to adjust their offer. A lot of companies that asked me what I made at my previous job (even though it is illegal to ask in Germany) offered me just 20% on top of my number.
This makes me think that all jobs position should have rather narrow target ranges which are stated up-front. Then you can discuss if you are too high level/low level for that job and range.
conversely, I discuss compensation immediately. In the segment of the market I'm in, I've wasted too much time going through interview processes only to get to the end and find out they're incapable of even matching my current comp. I've found being candid more successful.
Yeah I think it’s worth giving at least a range if you suspect the company can’t even meet the minimum. Like “at this point in my career I’m targeting positions that pay from $150-$200k.” If they balk and basically say the aren’t authorized to offer over $100k, you don’t have to waste your time. This probably makes more sense with smaller companies where they tend to have more limited budgets.
Seems to me that you would waste a lot of your personal time this way. They know enough about me before I even step in the door and I want some idea of whether the money can work. I ask for more than might be typical and I haven’t time to waste on companies that just want to show me what a great place they are to be at.
In my country it's common for companies to have public salary range, at least for IT positions - see https://justjoin.it/ for example board, Linkedin recruiters do the same thing.
It seems to be terrible waste of time going through interview process to learn that they won't even come close to what you want. I would not do that to myself.
that used to be true universally. now only true for entry level at big companies.
at senior levels especially, you should know what you want. comp bands are informed by huge data troves now and any edge you may have gotten before is gone due to the fixed and tight bands. just state your requirements and insist on “turnabout is fair play” ie they need to tell you the band. do this during the prescreen and save everyone’s time. if you kill it in the interview you still easily can negotiate to top of band.
Semi-serious question, asking for an indeterminate person:
If your desired comp number actually is $1M/year, will that just flag you as a joke to Triplebyte companies? With post-pandemic stock price increases a lot of engineers at FAANGs are making more than that now.
> If your desired comp number actually is $1M/year, will that just flag you as a joke to Triplebyte companies? With post-pandemic stock price increases a lot of engineers at FAANGs are making more than that now.
For any reasonable definition of "a lot", no they absolutely are not. A lot of people with 2+ years of experience who've either been promoted once and received a strong annual review, or switched jobs as a mid-level industry hire are making in the 200s and 300s, a few pushing into the 400s. Much more than that and you're looking at the top 1% (or less) of engineers at the top companies. Not a lot.
Source: was a mid level engineer at a FAANG until I left late last year, knew seniors and principals and their comp. Nobody was making $1M/year.
If "quite a few" non-directors make more than average American cardiologists either:
A) your company is engaged in disturbing monopolism;
B) your company is engaged in disturbing labor exploitation and arbitrage; or
C) your company is engaged in disturbing environmental destruction.
Would love to be proven wrong. Am I?
Edit: option D) is that American cardiologists are not as exclusive and skilled a labor force as I would prefer to believe ;)
People are paid to create value. If they work at a company that reaches a billion people, then writing some software that adds $0.1 revenue per user per year is worth a lot.
But people do brag about it on Blind such that the compensation bands are well known. People also complain about cliffs when it turns out the company didn't want to pay them at a certain level.
Stock price for most of the FAANGs has gone up by about 2.5x in the last year. If you had $200K base, $50K bonus, and $300K stock compensation in 2019, you're now making $1M today. That's roughly L6/E6/ICT6 (staff) level.
FB in May 2019 = $177.47, today = $332.04, 87% appreciation.
AMZN in May 2019 = $1775, today = $3496, 96% appreciation.
AAPL in May 2019 = $43.77, today = $130.87, 198% appreciation.
GOOG in May 2019 = $1103, today = $2508, 127% appreciation.
That's all over the last 2 years. Market low was apparently Dec 2018, before the Fed started cutting interest rates, so the numbers are a bit bigger since then. They were pretty steady through 2017-2019, so anyone who got grants or refreshers during those time periods has benefitted from that appreciation.
They are making that on their 3-4 years vesting and refreshers etc that they can't sell. So yes, in past few years some folks have been making great to really great money especially if they have a bunch of equity in various vesting stages + get a big refresh / bump on a position switch to drive retention. I wouldn't say a lot but many maybe?
I get what you are saying, but on the contrary, I don't want to go through interview process as a candidate just to be lowballed in the end. If the range is not there, and often it is not, I am fairly good and not everyone understand how much this costs, I just don't want to waste anyone time.
I don't think this part is off. Why I never went to Triplebyte and will not go, I think doing tests is dumb and don't wanna do it. Like original commenter said, you can't change model that quickly.
> Just Google "recruiter won't proceed without desired salary" and you'll understand how many folks dislike that question.
It's incredible how mileages vary. I recently had to strike companies off my list which demanded to know my last salary.
I offered them my salary expectations, but they would absolutely not accept that, and preferred to take the fact that I would not tell them the exact salary at my last position as a reason to decline to immediately stop the interview (with half an hour left in the call).
This wasn't a one-off either, I would say about 5% of companies (biased sample) try and get away with shit like this.
I used TripleByte to recruit people for a startup without paying, by asking people to sign up for the test and screenshot the answers and collecting the scores at the end. I already had an inbound funnel, I just liked your test when I saw it the first time.
In 2015, candidates with the same schooling scored about 20 percentage points lower than 2020, the last candidates I used the test for. The number of questions doubled, and then the test got much easier, by eliminating more challenging programming questions and replacing them with questions with giveaway context. By comparison, according to my data, about N=43, until about 2018 being a senior versus a junior in college CS programs is worth about 10 percentage points on the test; going to Harvard instead of Berkeley is also worth about 10 percentage points. In 2020, the last tests had no predictive features.
I stopped using the test, because it became too easy and too noisy to be informative.
I recognize some of the coded language in the blog post. There are definitely more lucrative opportunities in recruiting for DEI. I don't know if it will last. If you're still jittery about the public-profile-by-default thing, which by the way, was totally irrelevant and overblown IMO, this may not be a pivot for you.
One thing I see in the data is that at MIT, women and men performed the same, controlling for seniority. This wasn't true at the 3 other universities that produced enough data to measure.
That said, what really is the best way to hire candidates? I'm not convinced having binders full of engineers is special, there are almost always more candidates than jobs, at least 5:1, in every non-credentialed industry vertical. Anyone who has worked at a jobs (or indeed any matching platform, like the Common App or Tinder) knows that.
Then there's this long thing about asymmetries or whatever, warble garble about missing information... It has never, not once been my experience that someone seeking a subordinate role at a typical private company with preferences like "pair programming" or whatever have ever been better than someone with no preferences at all.
Maybe it helps to engage in the vanity of whatever trendy workplace trend is hot for whatever vertical. But like, if you're being intellectually honest, if you thought pair programming was important, you'd pair program at TripleByte, but you don't, you know in your heart of hearts none of that shit matters, so why are you putting stuff like that into your search system?
Indeed and ZipRecruiter are ad arb companies. They don't care. Private universities lead, not lag, DEI at giant companies, so it's hard to see how to compete against them in that core business. It will still come down to a real defensible opinion.
Do you have more valuable inventory than ZipRecruiter for DEI candidates? Who knows. What an uninteresting question. Apple also hires people who just make shit up on their resumes, I know two - though they weren't engineers.
A recruiter once told me, if you see 30 applicants for a high level tech job on LinkedIn, not to worry about competition, because ~27 of them were likely to be unqualified trying-their-luck applicants, mostly from other countries where the company was not going to hire, and mostly with no relevant skills or experience.
They told me the actual number of qualified candidates was usually just a handful or less.
I doubt this is true in SV, but I don't live in SV.
> going to Harvard instead of Berkeley is also worth about 10 percentage points.
That's interesting to me. Most reputable rankings put Berkeley at #2 or #3, and Harvard is usually around #7 or #8 (and US News puts them at #16!). I wonder what the cause of that discrepancy is.
CS program rankings are largely based on their graduate programs. The parent comment is almost certainly talking about candidates with bachelors degrees. For undergraduate Berkeley has a 16% acceptance rate and Harvard has a 4% acceptance rate. This will likely act as a filter in combination with the quality of the undergraduate CS program, which could be better at either institution but is hard to get a metric for.
I was specifically looking at undergrad rankings. The acceptance rate is not a good filter because that's the acceptance rate for the entire school, not the CS program.
From what I can tell, anyone who is a Harvard student can declare the CS major if they pass the pre-reqs. At Berkeley, passing the pre-reqs doesn't get you into the major -- you still have to apply for a limited number of spots, which generally means having a 3.8 GPA or better in your pre-reqs. So effectively both programs have the same acceptance rate.
Pedigree filtering is most often myopic elitism, especially in a business context. Anecdotally, I avoided MIT and Harvard because of Boston's snow and traffic. CalTech is Pasadena: too frick'n hot. I didn't study at all for the SAT-I and aced the math section. (The SAT should be more like the JEE.) I went to an expensive, "top 50" public university where I liked the area. Also, it was more practical and rigorous academically than most Pac10s (Pac12s now) and Ivys because they had something to prove (no grade inflation at all, they don't care if you turn in homework or not, not much market for homework and tests, and proctored exams). I also didn't do a PhD because of the economic disincentive: if I put in the work and the cost, I could do an MD, JD, or PhD CS but there is no added benefit.
Most clinical doctors have social skills and crystalized intelligence from domain expertise, but aren't typically mistakable for particle physicists.
People don't necessarily require the proper sheepskins to possess fluid, crystallized, and/or other domains of intelligence AND the skills, personality, and experience relevant to excelling at a particular STEM role.
Pedigree is mostly used for social filtering and business leadership board packing, but if someone wanted to create an elite monoculture of staff, lacking in cognitive and personality diversity, by all means, go right ahead.
PS. I. Don't even get me started about a car full of CS IITians talking about the JEE and high-placers who seem normal on the way to snow country for snowboarding and gambling.
PS. II. I bombed an Apple interview for a mid-career role by being too intelligent and too maverick compared to the group of compliant, I hate to say, yuppies. It was an interview panel of about 9 people and they were just speechless. The other time I bombed an interview for being too smart was about 10 years before that when I was 20 at the old Borders bookstore in Palo Alto. Moral of the story: it's important to play dumb where appropriate because most people are relative-intelligence insecure.
PS. III. Sorry, reader, for the rambling and discontinuous thoughts. Absurd endocrine values of unknown etiology currently... doctor appointments pending. ):
I have a sneaking suspicion that being "too smart" isn't why you bombed those interviews. I know I’d never want you on my team after reading just this one post, can’t imagine having to suffer through a multi-hour interview too.
> PS. I. Don't even get me started about a car full of CS IITians talking about the JEE and high-placers who seem normal on the way to snow country for snowboarding and gambling.
> I bombed an Apple interview for a mid-career role by being too intelligent and too maverick compared to the group of compliant, I hate to say, yuppies. It was an interview panel of about 9 people and they were just speechless.
Are you sure they were speechless due to your high intelligence and maverick qualities?
That shouldn't have happened. Could you go ahead and email our support at support@triplebyte.com again (just so that you don't have to expose your data here) and let them know who you are? I let one of our support folks know to keep an eye out; we'll get you sorted out (and I'll check back here later this evening, so let me know if you don't hear something back quickly).
What made me instantly distrust TB was the Publisher's Clearing House-style "you had the best score on our online assessment we've ever seen." C'mon, don't blow smoke up my ass right off-the-bat. If it's that easy, then it needs to be more difficult, broader, and deeper to pre-test candidates with proficiencies in particular areas with questions that are AI-generated: algorithms, data structures, particular languages, operating systems, networking, front-/back-end technologies and tools, software engineering, customer service, team communication common-sense, professional workplace behavior, and engineering ethics.
Do you think more of a peer based approach to recruiting would have any legs? I built something a little while back that was a kind of "keep track of the people you'd hire", and anyone on your list had the power to grant you visibility to various parts of their profile (current employer, whether they are looking or not, what kind of role they are looking for, etc). Never targeted recruiters as a potential user, but I guess that probably isn't a bad use case. I'd much rather get contacted by a recruiter (albeit a recriuter who was known to me) with "hey I see you're looking for X, let's catch up to discuss opportunities Y and Z" as opposed to the cold calls / linkedin messages that are the current status quo.
There is a large pool of people who think that they are programmers, but are not. This is exactly what the infamous fizzbuzz test was intended to weed out. I personally know a top notch recruiter who credits her success due to learning early in her career how to filter most of the junk out. For example a "Perl programmer" who listed Microsoft Word as a skill was probably not worth looking at. (Note, this is an example from 20 years ago. But the principle remains.)
If you create an engineer focused site and DON'T find a way to segment engineers by ability, you'll find that your negotiating power is based on the average of your candidate pool. Which means that you'll discover why traditional recruiters kowtow to companies and not the other way around.
Very good advice. The "gatekeeping" they alluded to is just very necessary if they want to have any differentiation from any other generic job matching service. I'm sure the triple byte team is aware of this but I think it's important to keep in mind just how much people work in tech/IT jobs, how wide the skill/qualifications range is and how spammy applications can get very, very quickly. Even positions that are "easy" to get as a competent engineer can get hundreds of applications, and since it's also pretty common for people to just outright lie on their CVs it becomes very time consuming to filter for someone who genuinely meets even just most of the job requirements.
& maybe I'm wrong but I thought the entire value proposition of TB was that their pre screening and testing process made the whole process much less noisy for your hiring managers. If you still end up needing technical interviews and aren't guaranteed that all the candidates you get have gone through some technical screening, what's the point?
AIUI, employers will be told whether the candidate took the tests, and if so what their score was:
> Going forward, our assessments are a purely optional means by which engineers can show their skills to employers (who overwhelmingly tell us that they trust our scoring), rather than a requirement for entry.
I think the idea is that an employer can still filter out the unfizzbuzzables, but has the option to ignore that for candidates whose stellar track record is a stronger signal.
I'm a little bit skeptical, because i think there are lots of people with apparently strong CVs who are still weak when you actually put them to the test. But maybe not at the very top end; i am really thinking about people with experience in various investment banks, which is perhaps less of a signal than Apple then Netflix or something.
This is very true. We weed out hundreds of applicants ler opening on very simple questions... Think what is the difference between const, var, and let declarations in JavaScript?
The problem is we're all spending weeks and weeks on weeding out fakers. We've moved to a quick pop quiz right after we get an apply. This takes out 85% of the candidates (it makes me sad when I look at how simple the stuff is that weeds them out...). Two interviews are more than enough to land on a hire after that.
Our company is a conversational recruiting software company (you apply, we say hi, and screen candidates in real time and then hook the candidates up with an interview). We started doing the screening in our chat flow and it works really well for us.
I can't remember the last time I was ghosted. The problems you listed aren't problems that I experience. I don't care what part of my resume recruiters look at. I don't care what they do with it.
Sure, having metrics on how often a company ships code, etc. would be helpful, but frankly I have about 10 companies on my list that I'd work for, and I doubt any of them would give you this data.
What I'd really like is not having to practice leet code problems for 3 months before I interview. The older I get, the less motivated I am to do this. I've worked in senior engineering roles at big tech (FANG) with 3+ years tenure. I don't think I managed to not be able to do my job and fooled people for that long.
I've interviewed people I KNOW could do the job, but they didn't get the optimal solution, or were nervous, or N things in the loop, and we ended up not hiring them. There must be some incentive to create a system that lowers these false negatives.
I have had this exact experience with Triplebyte both as an engineer and as a hiring manager.
I simply do not care about things like, “what does malloc return”. I do not care what people know about bloom filters in Postgres. The 99.99% of web developers don’t have to know these things. I do not care if you can implement Tic-Tac-Toe.
It’s poor interviewing, which is the entire product they’ve been offering.
> There must be some incentive to create a system that lowers these false negatives.
I’m generally quite pro immigration but when BigTech goes to DC and bellyaches about how they need higher quotas because they can’t find enough workers it really sticks in my craw.
Reminds me a bit of the classic example of chutzpah—-the boy who killed his parents, then threw himself on the mercy of the courts, asking for leniency because he was an orphan.
I emailed customer support in April because your site would not allow me to change my current role to any value other than <Specific title at specific company>. Like, my profile literally has a dropdown menu where that's the only option. Nick O'Brien asked me for a screenshot of the problem then ghosted me. Today it is still impossible to change my current role on your site.
I am looking forward to progress toward a hiring process that puts engineers in control, such as by allowing them to accurately record their current role in their profile.
Your point in the article about gaining bargaining power for engineers by making the hiring process an iterated game instead of one-shot is a good insight.
But the transparency you need to extract is mainly financial, that’s the big asymmetry. I think people care more about money than they say they do, and less about release cadence or test coverage or whatever than they say they do.
So triplebyte should be called “get me a raise.com”. I tell you what I’m making, smash all your tests, and you find me somebody who wants to offer me e.g $50k more or $100k more, and that WILL be the offer I get. And if the company reneges on that offer then it’s flagged to warn others, and if they do well by their employees they are rewarded as well.
This was our original theory when we started doing user research, and we were surprised at the degree to which it didn't pan out. We assumed (in line with how these things usually go) that engineer interest in filters would follow some sort of very lopsided power law, where a few filters dominated among almost all engineers. But in practice, everyone wanted something different: some people were excited by salary, some by tech stack, some by pair programming, etc. One guy went on for like fifteen minutes about how much he hated open office plans.
That said, the idea of "we will offer X to anyone with Y quiz scores" as a showing a company can make is pretty cool, and is definitely in a "company makes public conditional guarantees" space that we're interested in exploring.
LinkedIn doesn't provide the vetting service Triplebyte does; they don't tell employers anything about job candidates' quality. It sounds like you want to pivot from the small market where you were the market leader into the larger market where LinkedIn is.
how do I reply to recruiter contacts? applied to a job, received a request for interview from company. Nowhere to reply on the platform? How do I get in contact with them?
Should there not be a reply button right there with recruiter's message?
p.s. I already emailed support but, my past experience tells me they won't reply.
Funny but whenever I think of Triplebyte, all I remember is seeing their ads everywhere. I am personally a fan of Indeed and they do a good job sending leads and applicants other than good old Linkedin.
I noticed that too, and when you combine it with "Triplebyte has hundreds of thousands of engineers on our platform" it means they aren't doing great with the rate either.
> I forgot how to implement the zig operation in a splay tree.
employee at Triplebyte)
> To put some hard numbers to this: further down this thread, there's a post about how "any engineer" could answer a question the poster thinks is too easy. I looked up the question in our back end and, in fact, barely a third of people who take our quiz get it right (the correct answer isn't even the most common one!).
So there 1000 job matches and at most (("hundreds of thousands of engineers on our platform" / 3) - 1000) who were incentivized to answer these contrived problems, did so correctly and still couldn't be matched.
So even among the large pool of engineers who have gone thru the process, met some arbitrary threshold of engineer-ness, there's still a huge mismatch between corporate/prospective employee expectations, that I'm not sure will be able to be overcome quickly even with these new initiatives, but it's interesting that they are being pursed now (not surprisingly after the shift in working environment after massive government restrictions on freedom uber alles).
I agree. We tried using TripleByte for hiring but what they screen for and what matters are entirely different. A founder we knew got an angry missive from one of the TripleByte founders because they’d rejected candidates during a final culture screen. Apparently the only qualification should be whether the candidate can do contrived coding tests and programming jeopardy, but whether you actually want to work with them is beside the point!
> Apparently the only qualification should be whether the candidate can do contrived coding tests and programming jeopardy, but whether you actually want to work with them is beside the point!
Yeah, different companies want different things and are ok with different things. My best experiences have been with companies that have taken less than 2 weeks to hire me with no testing whatsoever: just a couple of calls. Other companies want and only select for the contrived coding tests and programming jeopardy, and that's fine, but I want nothing to do with them at all.
Maybe even allowing 3rd parties to bet on specific hiring decisions (in addition to what is described in the link) could add another layer of incentivized feedback loops to the process (esp if the people making the hire/fire before 3 months are made known) and add another layer of market driven signals for them to pay attentions to. Maybe make it more expensive to chase certain types that are in higher demand, and less expensive to chase others types that are in low demand (not the salary to the employee, just the payment for filtering). Maybe less expensive for shorter refund windows, and more expensive for longer refund windows.
When I went through it was like, ~1k/month and somewhere from 5k-15k/hire over the top. Industry average is around 10k/hire and their pricing suggests they're probably banking that amount with pricing used to encourage loyalty and appease investors with the Saas narrative. They're not pulling in much greater than industry average, and even if they're double, its only 20m, which is still bad.
I have no idea what Triplebyte gets or even what their business model is, but typical recruitment companies get a lot more than that -- on the order of 15-20% of the first year's salary, so on the order of $15-25k. I suspect Triplebyte is less than that, but could be an order of magnitude more than you've guessed.
So, under the old model, order 200k engineers applied to us. Because we were exclusive gatekeepers, around 3% of those were "accepted" onto the platform. Around 2/3 of accepted candidates received an offer, and around 1/2 of offers were accepted.
I don't know much about them, but the post does say their goal was to be "exclusive gatekeepers". If you consider it from that perspective, they had more like a 1/3 conversion rate (2/3 * 1/2). But conversion kind of seems like the wrong metric.
What metric would make more sense here, though? I can agree that from TBs perspective, they may not factor total applicants as their base # (although that is clearly their market). However this exclusivity of only using the small percentage they accept doesn’t seem to be a scalable business model. Having 1% of the devs who try their service actually accept an offer would not entice me to try their service.
>However this exclusivity of only using the small percentage they accept doesn’t seem to be a scalable business model.
For sure. That's very likely why they're making this pivot. Exclusivity was their goal, but it just turned out to not be a very profitable goal for them, it seems. I was basically just saying it seemed to be a "theory" issue rather than an "implementation/execution" issue.
Especially when they're probably making on average less than $25k per engineer they place. I can't imagine they're making even close to $100k on average.
$25k*10k = $25M / 10 years = $2.5M / year. That is peanuts compared to how many people they employ. Plus the ads they run...
I was pretty happy with triplebyte from the hiring side. Slogging through candidates with good looking resumes that couldn’t actually program was expensive for my employer and unpleasant for me. TB took care of that first big cut down.
I guess they didn’t hit product-market fit with candidates. Two sided markets are tough.
Based on my experience with them, I think they totally failed at their previous model.
When I was fresh out of university, I had trouble finding a job. I had worked part-time at a small company through the last few semesters, but that company blew up just before I graduated. Every interviewer I ever met with was impressed because I really knew my stuff, but had trouble getting in the door due to an unimpressive CV and awkward mannerisms.
I did very well at Triplebyte, scoring in their top bucket and getting top marks in several of the sub-categories. Those metrics are relative to all engineers, which would put me even higher amongst recent graduates. I also did very well on every software project I worked on and scored top marks on several other objective tests, like the Major Field Test.
I was exactly the sort of candidate their system should have been able to help. I never even got an email out of it, let alone an interview.
I went through triplebyte twice, once in late 2019 and then once again a few months later after the company I landed in shut down because of Covid.
I was happy with the process, they promised to take a lot of the headache regarding tech screening and it worked. I think the value proposition is clear - I don't want to do the same tech screens 100 times, and lots of smaller companies may not have the resources or desire to manage high quality tech screens themselves.
To be clear, I'm only addressing the comment about how people thinking highly of them are gullible. I had a great experience, twice, within a few months, and that included going through a time of huge uncertainty.
That being said, I was not happy to see each of the changes they've made over the last year or so, and this does not strike me a good direction for engineers. I suppose I have to agree that they are not doing this from a position of market leadership. Perhaps they were providing asymmetric value to engineers, and at the end of the day the engineers are not the ones paying for the service.
I think their process of 100% not working. They branded them self through advertising that they will provide constructive feedback if you don't crack their interviews so dev can improve upon shortcomings.
My experience was different I never got feedback after my last round part of the reason must be they were not able scale this process up.
There are two categories where people pivot while crushing it:
One. The problem domain they are in is on its way out, and they've decided to defect instead of being the ones left holding the bag.
Two. They hate what they do for a living.
The first company to stop making buggy whips probably did not do it because they weren't selling any. The fact that they ceded market share to their top competitors probably sunk the barb in deeper for them: Yeah people are buying cars but our sales numbers are still going up so why should we change?
In tech we see companies all the time end up being complicit in the destruction of their own industry by reducing its relevance in some manner or other, and by exiting early they have more options of destination. If I jump to the same industry as a competitor a year later I just look like a copycat. While some copycats copy other people, only better, they are the exception to the rule. Most churn out uninspiring derivative work, and if they can't at least do it cheaper then they end up on the scrapheap of history.
In our case, it's more just an intermediate state between "crushing it" and "failing". We had a product that was working for a subset of engineers, but was running into structural and scaling problems. Simply put, it was (sort of) "crushing it" but was foreseeably not going to keep doing so.
At that point, you can either keep going with what you're doing (and run into an inevitable wall later on) or you can foresee the future problems and pivot before they become unmanageable (which is what we did). Relatively few startups find their final model from minute one.
100% this. They say individual engineers can't exercise this power, but that Triplebyte can use its weight to force their hand. As a hiring manager, I have dozens of recruiting platforms to choose from, most touting the same list of resumes. Yet another resume farm has zero power.
Fortunately, most of the dirt thrown at recruiting teams is wrong in the first place. I don't know any hiring managers who make lowball offers that have a low chance of acceptance - the costs of the process, the risks of hiring someone who starts mad at you, and the value of winning someone who is actually worth hiring, are all way more important than some incremental cash saved.
> As a hiring manager, I have dozens of recruiting platforms to choose from, most touting the same list of resumes. Yet another resume farm has zero power.
Well, presumably the idea is that engineers, especially high quality ones, will be drawn to the recruiting platform that has more pro-engineer policies. And so if you avoid using it, you lose access to that subset of engineers.
It's still a chicken and egg problem, though, because engineers have little incentive to join until there's enough companies.
Yeah, I think the biggest issue is with the implication that their existing user base, which was built under their old policies, is a unique asset. I'd be willing to bet that a significant majority of current TripleByte applicants also use other platforms.
However, I think that even if they build an attractive, differentiated experience, the advantage is unstable. The problem is that "traditionally strong" applicants (those who already have positive ROI from existing processes) are disadvantaged by being exclusive to one platform. So, from a hiring perspective, TripleByte's differentiation in terms of unique resumes is going to be from people who are harder to assess.
I'm actually fine with that as a value case - we've had great success hiring from bootcamps. "We help you reach strong nontraditional applicants" is a great value proposition. But it's not the same as a position of power to dictate the hiring process. Maybe a better pitch would be: we run a process that helps great non-traditional talent and great non-traditional companies find each other.
I'm really curious who is USING this company for their recruiting. As a IC I've been approached a lot by them, tried taking one of their tests, was turned off.
And as a HM, I was never interesting in using them, and we continue to use our own internal recruiters or local recruiters who know the market.
Having worked at a startup that was trying to disrupt the hiring/recruiting market back in 2010. I'm curious to how effective they were....if at all.
Edit: update, maybe I'm super skeptical because of all their cheesy ads that were all over Reddit for so long?
I interviewed with triplebyte about 5 years ago. They rejected me saying that my editor skills were good but I didn't pass the mark because I couldn't find enough bugs in a program fast enough while being watched. This was really good for me as my subsequent decisions took me to where I'm now as a lead architect of a US startup, earning 5 figures USD while living in LatAm.
On related matters. I know several companies that have pivoted out of the "hiring/recruiting" space because apparently that space is not easy to scale/automate.
I think the negative comments here are being a little unfair. It sounds like you've had a really terrible experience with TB. I get if that's your experience and interpretation, but TB are not superficial hawkers of products without fit or substance. They are passionate and intelligent and friendly engineers and weirdos who genuinely care about making a good startup and helping developers and companies match.
My experience is different. I've been through the TB process three times, back in 2017 where they flew me to SF from Asia for Flexport, Mixpanel and two smaller startups, and then again this year where I went to the final stage with FB production engineering and TB itself. And in between as well i created yet another profile on the system after shelving prior ones for a fresh start. I have a non traditional background, (ie, an "outsider" developer, chem degree and self taught hobbyist programmer since young), i take my craft seriously, have a bunch of popular open source products and a successful low touch indie business. But i don't consider future success guaranteed so i like to keep talking to companies to find that great opportunity for me. I know my tech skills in some areas are not the best so i like to stay in the loop, stay up to date and relevant and stay in touch with what companies are looking for. By far the best hiring process experiences I've had have all been through the TB platform. And my "open rate" for applications and inbound interest on TB is way higher than any other methods I've tried (SO jobs, WWR, HN). The quality of companies, hiring process and level of interest I've received via TB has been epic and, at least in the past, even the TB support staff are super helpful going so far as to provide personalized feedback on profile improvements and resumes. I'm not an expert at getting hired but i couldn't have asked for anything more from them. They were fantastic. I only have great and wonderful memories of the time i spent in hiring processes of companies i "met" via TB. So this is my experience of it, and this is despite the fact that i have never ever received an offer from any company I have met on TB, including from TB itself. I don't blame them. I just consider my own skills are not at the right level, and the fit is not right for the companies I've met, while i work to improve myself and my skills. That's all. It's a simple and beautiful experience I've had with them. Soooo different to hiring through other platforms (where funnily enough i have recieved offers). I've met and talked to ammon and many other TB team members and they have all been cool, friendly, passionate people. From what I'm reading of this pivot, they're being vulnerable and open to learning, and using what they've learned so far to create something even better. I'm not gullible, I'm not saying they will succeed. But they're trying. And they're earnest and sincere. What more can you ask for? This has been my deep experience with them. I think the most vocal critics are talking merely theoretically without any experience of TB nor deep nor accurate understanding of who they are, what they do and what they stand for. To them i say, give it a try. Investigate before contempt! ;P ;) xx
I wouldn’t be so certain. Obviously, something isn’t working well or isn’t as good as it could be, but that doesn’t mean they are on the way out. Maybe they are, but plenty of businesses say, this isn’t working like we want it to-we can do better. As a worker, I am happy that this is a product that seems to empower workers over companies. I wish them all the luck and will look at their offering to see if it could benefit my own team.
I'm salty towards the platform because they rejected me a few years ago, and my ego has barred me from using them again despite their outreach. I was also very unhappy when I heard about the public profiles. I don't want people to see that I interviewed poorly, or to have any kind of public record of that. I'm a decent engineer, I swear!
I had some bad luck. My nodejs build broke after a recent update on my machine, which I didn't realize until right before the interview. During a test with a different language, I tried to define a constant with the same name as a built-in function and ran into a vague compiler error (something about missing parentheses, ugh). This language has case-insensitive function names to compound the confusion. I unfortunately looked up how to define a constant in the docs. Their conclusion was that "I was uncomfortable in the language," despite having used it for 10 years. There was some other feedback which I felt was inaccurate, I think I just had a bad day, and obviously didn't convey my knowledge and experience well. I could see why a recruiter would hard pass on me for some of the stumbles, since their main goal is to forward candidates who interview well. It hurt to get rejected.
What's bad about that? It's obviously testing understanding of scoping and side effects. It's not hard if you understand those things and confusing if you don't, which seems like the kind of question you'd want on a "general coding assessment", no?
The only thing that bugs me about that question is that "the output of the following function" is confusingly worded. It's the sort of question that a competent candidate might get nervous about and start to overthink: wait, that isn't a function, it's a code snippet. The function here is foo – so maybe they mean "what's the output of foo"? But then what does "the output of a function" mean? I suppose they mean return value? Is this a trick question to see if I remember what push returns? Damn it, does it return the entire array or just the pushed element?
That's what my mind usually does with questions like that and I'm far from the only one. Since the goal of the test is to screen for basic competence, it ought not to filter out people who could answer the question perfectly fine if it were being asked clearly, but who also perceive corner cases and ambiguities. Such a skill should make you more likely to pass such a test, not less. Therefore the question ought to say something like "What does the following code snippet write to the console?"
As you say, any competent programmer could solve this in a few seconds, if they are proficient, or minutes if they are not.
In my opinion, they should not put a count down because they are interrupting the programmer’s flow. If they really want to measure time, they do it without interrupting the programmer, hide the timer, let them take as much time as they want/need to solve these problems. Then, at the end of the quiz, show them how long it took them to give a proper solution, and take in consideration that time to score them.
If Alice solved 10 problems in 10 minutes, and Bob solved the same problems in 15 minutes, maybe there is something there that is worth highlighting in their profiles. Maybe Alice is able to analyze this type of questions much faster than Bob, and if that matters to the recruiters or potential employers, then allow the candidate to use that in their favor.
However, if Bob actually knew the answer, but they ran out of time to select an option, that seems unfair.
Depending on the language and how it passes in values, you'll have different answers. Assuming it's javascript the answer is 0,1. In other languages it might be 1, 1.
That's a tricky question, and if you're not good enough to understand why they're asking that then you fail!
This is more about knowing what const does to an array. Which is, IMHO, not how I would expect it to behave. So it's somewhat of a trick question. But I don't write too much JS so I don't know how well known that quirk would be.
a and b are shadowed in the foo function so the function only acts on the shadowed (and copied) variables. Once you return from foo, the const a and b are unchanged.
Not a great question as a pass or fail test imho because if you use shadowed names often enough to be able to parse this code in your head, you’re writing bad code. There is a reason why shadowing names is a bad practice: it’s hard to figure out what the end result is and account for side effects! But as a discussion on all the points above then it would teach the recruiter something about what the candidate knows rather than using this as a trivia question.
The const on b only applies to the object reference to b, and doesn't prevent foo from changing the contents of b. No doubt that's why the test question includes an array in the first place.
At least I assume that's how const works; I've not used const in JS.
That’s possible. In reality, if I ever came across that piece of code and had a bug to fix in it, the first thing I’d do is rename a and b in one scope or the other because it’s just asking for trouble otherwise and it’s not easy to understand what happens to a and b. That’s why I think using this question as a trivia question is bad. But the discussion we’re having about it clearly shows we both know what to look out for, that we’re competent, and is way more interesting than running JS code in our heads. Even if either of us gets the answer wrong, it doesn’t really matter as much as how we got there in my opinion.
This particular example is 0, [1], which is easily seen by pasting it in the console.
There may be some edge cases I'm not covering here, but JavaScript passes primitives by value (strings, numbers, etc) and non-primitives (objects, arrays, etc) by passing a copy of the reference.
What this means is if you reassign the reference inside of the method, it will only affect that scope, because the reference itself is a copy. If you modify the properties of the non-primitive, it will be modified, because the copy of the reference points to the same non-primitive.
Since primitives are passed by value then any modifications are not reflected outside of the method.
To the other poster below, in JavaScript, const with an array (object, etc) simply prevents reassignment of the variable, the array can still be modified.
The problem with Triplebyte IMHO is that what the originally promised, while a great idea and concept, it cannot scale. Also after interviewing with them for a role in their company I got to realize they don't know how to conduct interviews themselves.
I used them two times. The first one was very early on, where I was given a home assignment and interviewed on it by Aaron himself, if I am not mistaken. The dude was awesome at interviewing, and knew exactly how to probe to get a better understanding of your skills.
That was when they were promising that you can interview with them so you don't have to do technical interviews with the companies. I thought it was an awesome concept and could really reshape hiring in the tech industry.
Second time was a couple of years ago, where the model has already changed a bit. Passed the first round and one of the companies that I could interview for was triplebyte themselves.
What a disappointment! The only difference in the interview process than the rest of the companies was that they gave you a laptop and asked you to do practical coding instead of whiteboard generic algorithm solving.
Some of the interviewers themselves were junior members of the stuff with 0 experience in interviewing.
is that they say they have companies like Apple and TrueCar posting job listings and hiring but they actually never respond and all you get is offers for companies between 10-200 people in size (aka, startups)
I'm guessing this is your experience - was this more than a couple of months ago? We made some changes about ~6-8 weeks ago to how we order jobs to prioritize responsive companies much more than they were in the past. We also use responsiveness as an input to the "Likely To Accept" score shown for each job (here's a screenshot of a posting from my own prod-testing account: https://imgur.com/wSmGCEL).
This was my experience twice trying to use Triplebyte in the past 2 years.
It's a great platform, the indicator letting me know companies aren't going to respond is great.
It just doesn't fix the problem that... the quality of companies on Triplebyte aren't companies most senior/lead/architect/advisory level engineers are willing to settle for. I'm being unnecessarily harsh/biased. I'm sure plenty of people use the platform with great success. I've just come to accept "early/mid-stage startups" job offers from Triplebyte and not much else. Not necessarily "garbage" but... for sure lots of risk.
I did triplebyte most recently in mid 2019 and that was my experience as well. I used triplebyte as well as job searching on my own and I got a few offers from triplebyte, some of which seemed like interesting projects but couldn't offer a level of compensation that would make the risk worth it. I got two high-quality job offers from my own search, both of which were more compelling than any of the triplebyte offers and I am still with the company whose offer I accepted.
The "super week" of onsite interviews was interesting, probably increased my overall interviewing skill, but was probably one of the more stressful individual weeks in my life.
Scheduled a practice interview and the only slot available was ~6AM my time. Nobody showed up and I wasn't informed that they wouldn't be showing up.
After reaching out about it -- they tried to tell me that my interview scored poorly, and that I would need to retry again in a few months. After some back-and-forth they realized both that it was practice interview and that the person didn't show up. So they rescheduled the practice interview.
When the interview did happen -- this was for ML stuff -- the interviewer was just no great. We spent so much time discussing the differences in terms we used -- mine largely coming from University, there's from I'm assuming their formal education -- that much of the interview was wasted. It literally came down to me deriving what we were not agreeing on for them to understand we were talking about the exact same thing.
I then had to reach out several times for my results, I'm assuming due to the fact that you're allotted one practice interview, and technically I had two(?). When I finally got my results I was again informed that my results were not good enough, and I'd need to wait to reapply. I gently informed them that it was practice interview, and the representative apologized their mistake.
When I reviewed my results... the interviewer didn't rate me too well, largely due to our differences in terminology. They also didn't like my coding style -- even though no one has ever complained to me before -- and rated the coding exercise poorly even though I was able to perform what was asked of me.
I just gave up.
Then several months later, I got an email about being TripleByte certified or whatever.
The best thing I got out of Triplebyte was a great piece of written negotiation advice, from a recruiting specialist that has, presumably, since been laid off.
> We want to stop being a placement agency, and instead become a job search platform that leverages that unique power to create a better hiring process for engineers
> Triplebyte has hundreds of thousands of engineers on our platform, and that means we can flip the script on companies. The collective power of thousands of engineers is enough to change their incentives in a way that individual engineers cannot.
> We can change their incentives directly by rewarding or punishing certain behavior. For example, companies aren’t normally incentivized to provide salary and culture data. But we can force their hand by promoting transparent companies in our search rankings. When a company’s access to thousands of engineers is on the line, their incentives are very different. The same goes for honesty: a company often has no reason to be honest with any one engineer, but we can disincentivize lying by making their behavior with one engineer affect their access to the next.
So... a union? In digital, online format?
I'm glad that someone there is now realising the advantages of joining forces to negotiate workers' conditions. It was about time high-tech engineers noticed this.
A benefit of Triplebyte to big enterprises not mentioned in Triplebyte’s discussion here: acting as engineering assessment proxy for engineering hiring managers stuck with pathologically risk averse enterprise HR departments.
Often enterprise HR is paranoid of honestly evaluating anyone for anything, to the point that many HR teams tell one another and engineering hiring managers that it’s “illegal” to assess candidate abilities on the way in the door, at all.
If you think about joining a dev team where no one checked if any candidate could FizzBuzz, you can imagine the workplace environment that can end up with.
Triplebyte is able to provide engineering managers with a stack of pre-vetted resumes so an enterprise can interview by its lowest common denominator HR policies, while still having a prayer a team will at least be made of candidates who can code.
The value of this to an enterprise stuck in this position is hard to overstate.
Hopefully along with expanding the talent pool per this post, Triplebyte can figure out a way to get well paid for this ability to help land actual coders on actual dev teams despite enterprise HR.
The need for this is huge.
// I see a comment below from a Triplebyte PM about score matching: https://news.ycombinator.com/item?id=27542621 … That would be slick: land candidates that genuinely raise the bar, but aren’t so far ahead as to be speaking a foreign language to the team. Of course, this would require also assessing the target team, and oops, we’re right back up against that HR policy…
There's an old quote: "Nobody ever got fired for buying IBM."
Large companies tend to be more risk-averse, so if there are two companies you can use for a service — $PromisingStartup and IBM — large companies will generally make a decision that's optimal to the specific decision-maker, not optimal to the company.
There are two options if you, as the buyer, choose either $PromisingStartup or IBM: success or failure.
If IBM or the $PromisingStartup succeed, then you've done your job.
If IBM fails, you can tell your manager "Who could've guessed! It's IBM!"
If $PromisingStartup fails, you'll have a harder time explaining your decision, and the fault will be with you.
The "Nobody ever got fired for choosing IBM" idea is useful to look at every decision large companies make, whether it's pivoting, choosing a SaaS product, or hiring.
TripleByte, in its current form, has been beneficial to candidates as well by giving enterprise employees a justifiable signal towards hiring them regardless of pedigree.
> If you think about joining a dev team where no one checked if any candidate could FizzBuzz, you can imagine the workplace environment that can end up with.
That sounds hilarious How could a dev team not be able to write 5 lines?
You would be surprised how many people legitimately can't wrap their heads around a fizz buzz style problem, but also how many people who can yet completely freeze in an interview setting.
> Performing an interview is a different skill from writing code.
I think this is true for the day-long leetcode interviews, but to succeed as an engineer, even at the entry level, I think you need to be able to talk through how you would solve fizzbuzz. A big part of the job is communicating trade offs with other engineers.
You're not wrong, but the problem is that interviews aren't actually about jobs. The average person interviews because they need the money. So from the start they're not thinking in the programming space. The pressure for them is to make the interviewer believe they are worthy of the job.
I think if I could describe my personal experiences, I'd say that on a subconscious level, I know that how I speak and articulate details will be observed by the interviewer through a cultural lens more often than not. This may be unique to my personal background, so maybe this is different for other groups. But interviewers often get hung up on the use of different language/terminology, the types of jokes/jests that enter the conversation, how I respond to their language. And there's sometimes been an unwillingness to allow different terminology, perhaps because that's all the interviewer themselves knows and understands.
All that to say, a code interview doesn't always come down to the quality of the code.
It is hilarious but man, you would be really surprised how people choke on this. It's weird. People with 10+ years writing software can't do this sometimes.
I was once accused of lying about my work experience by a $FORTUNE500 interviewer when I failed to articulate what a CSS flexbox was for a backend dev position.
Such an eye-opening experience! Now that I’m on the other side of the table these days, I understand the importance of making interviewees feel comfortable and asking questions that allow them to start off strong.
I got shitlisted at one place for being the only dev with the mathematical insight to know that multiplying pesos by a dollars:yen ratio doesn't yield a correct conversion of pesos to yen. They let that one slide without fixing it, but the final straw that got me frogmarched was refactoring an if-else to a switch.
Last year I did a project for an org that has a similarly sized team of developers. They could not figure out html and css. Hadn’t heard about git. And worse is that they developed on the production server. I’m not making this shit up.
They saw me as a god for fixing their shit and offered to hire me. No thanks!
Here's an advice. The software engineering market, at least the upper end of it, doesn't need a email forwarding proxy middleman. What it needs is a club-like organisation that acts as a negotiator that leverages insider knowledge to get unreasonable parties on both ends to sign a contract. Good devs don't really search for jobs and don't really talk to random recruiters. They get a steady stream of sales pitches from friends of friends or former co-workers and leverage their fairly wide network to get insider info about companies they're considering to join to get a good contract. The "club" would be like a golf club address book with staff working to connect matching parties. It's surely not a dating site for programmers with ahem.. "AI" selling resumes to data brokers.
Hence the good ol' "We'll give you $5,000 for every engineer you refer to us...and there's no cap!"
At least good companies do this anyway.
Also, executive search recruiters kind-of do what you're describing. Exec leadership is a small-ish world, and the recruiting cycle for, say, a CTO can take months. However, there's a lot of commission on the other end of that. So recruiters basically act as brokers.
Meh. As someone who has moved around a lot this just sounds like friend nepotism. Based on this system I’d be a garbage tier developer since I don’t have friends in high places.
I much prefer the imperfect systems that at least pretend to be fair and objective-ish.
I very informally do this right now. I think of my role as a sports agent for top engineers. The engineers I work with pay nothing and the companies who want access to them give us no bullshit, all access interviews where we're not working with HR or recruiters. It works so well for both parties that I'm scaling it at the moment.
Triplebyte was a dud for me. Way too many garbage startups recruiting.
I have started to re-write my resume for each application and that has yielded much better results in terms of quality of the company and number of interviews.
If you are submitting the same resume to multiple companies, you are doing it wrong.
FWIW, this isn't really true as you get some experience under your belt and have a stronger resume (meaning, having worked at companies that people know of with a clear story of how your own role made a big impact there).
But glad you figured out an approach that worked for you!
462 comments
[ 0.18 ms ] story [ 7641 ms ] thread1) Look at relative data, not raw ratings. Written reviews or raw numerical scores are what show the nightclub effect the most. What we care about is a ranking of companies (showing the better companies first in search results). This may not be as impacted by the problem. If sour grapes from people who failed interviews overwhelm other signals, we can only look at the scores from people who pass (or normalize the two groups separately).
2) Ask very specific questions (and maybe provide dummy options to attract sour grapes). I suspect that questions like "did this company say they could meet at $250 salary and then offer you less?" will get more accurate answers than "does this company have a toxic culture?". (We do actually want to get culture data, but I want to be really specific about the culture questions we are asking).
3) Use objective facts, not subject opinions. For example, we can tell when a company ghosts a candidate (because they reply through a proxy email that we control). So we're listing the 'ghost rate' for each company.
Anyway, a ghost rate would be greatly appreciated.
I'm pretty sure many salty people will just lie anyway, to punish the company. Source: I've been on the internet a long time.
I think this model of 'punishing' companies would've worked when you had a high bar for being on the platform in the first place, but if anyone can get on there, you'll have a lot of low quality candidates who are very mad that nobody will take them, and they'll overwhelm the reviews.
Remember that there's a survivorship bias at work: the lower quality people will take longer to get hired, which means they do a lot more interviews, which means a lot more reviews on your platform. Essentially, it's a reverse meritocracy when it comes to evaluating companies.
It would be a nice touch though.
Maybe this would be useful for folks just starting their career, but on the hiring side of this, in the last year I haven’t made an offer to a candidate who didn’t already have at least one other offer on the table.
Overall I think there are a lot of misconceptions about how people are hired from the recruiter side — which is why most of these platforms see the recruiter as the target customer, not engineers. I get that y’all are trying to change that, but without insight into the hiring practices at any given company (e.g. my company gives a lot of latitude for salary negotiations but every hiring manager negotiated differently) it’s going to be hard to pull a signal from the noise.
We look at the job posting as a brand marketing — all we’re really trying to do is get you to apply and fill the funnel. The whole hiring process is designed to be a calibration on role, cultural fit, compensation, etc.
Seems almost like the bait and switch that the OP company is railing against in the blog post.
I'm curious to see how you guys have managed it without disgruntled employees quitting within 18 months.
I work in consulting so high turnover is just expected. We’re actually a lot lower than the industry average. The job isn’t for everyone, but there are people who thrive in this environment. For everyone else who isn’t a good fit, we get a year or so worth of billable hours, they get exposed to a lot of different scenarios, and we give folks a lot of runway to find something they do like.
It’s just the nature of the beast though. If I get a candidate who is legitimately at level, I bring them in at level. There’s no need to be shady about it, but job search sites are what they are and that’s the game you play.
That being said, the experienced engineers who can get that kind of attention have other desiderata in their job search beyond just "get an offer". "Get -an- offer" is a goal more typical of a new graduate or someone trying to break into the industry than it is a goal of experienced engineers.
To people who are sure they can get a job, there's a big difference between a 150k offer at a company that has none of the workplace traits they want and a 200k offer at a company that they'd genuinely love to work for. And right now, they can't necessarily tell the difference between those jobs before going through a bunch of application steps. The example we mention in the post - of a highly credentialed and very senior engineer who kept having his time wasted by recruiters who refused to admit that they couldn't offer what he was worth - wasn't cherry-picked. It came up organically while we were doing user research, because it's a really common problem, and was echoed further by a number of other senior engineers we spoke to.
(Of course, we care about the folks just starting their career, too, so I don't want to totally ignore the importance of solving problems for them. But I think we are solving an important problem for senior devs here, too!)
Tell HN: Interviewed with Triplebyte? Your profile is about to become public (1543 points) https://news.ycombinator.com/item?id=23279837
Tell HN: Triplebyte reverses, emails apology (1030 points) https://news.ycombinator.com/item?id=23303037
maybe (and i know that this is an outré thing to say these days) that was the point.
To use a rough biological analogy, I think it's generally a lot easier to have a big, rapidly growing company if you take an r-selection rather than a K-selection approach.
If you allow everyone in, regardless of if they are qualified to be a senior level engineer, will this not just be another job hunt platform where (when I look for another job) I'd still have to do another technical interview at the company itself?
How will companies know now who has been vetted as qualified and who is just lying on their resume? Maybe it's shitty to think this way but there is already a ton of sites out there that allow me to send my resume (or allow a company to view mine) before starting a traditional hiring process that Triplebyte prevented, allowing companies to connect with qualified high level engineers and know the person would fit their skill needs. The way it was companies mainly were just testing for cultural fits, etc.
TripleByte wants that growth rate more than anything else: therefore, this.
I tried TripleByte and wanted to like it, but mostly it didn’t seem like anything special. The whole concept of getting pre-screened, getting multiple offers, and jumping straight to final interviews just didn’t pan out. The last company I interviewed with basically treated my application like any other pipeline. I had a standard recruiter call, hiring manager call, and several technical interviews before getting an offer. And the experience with other companies was similar. From the outside looking in, it appeared that there was no special track for TripleByte applicants. In the end it felt like the TripleByte process simply made the interview process longer, not shorter. Eventually I found it faster & more effective to pursue companies on my own.
TL;DR — companies I met through TripleByte ran me through the standard process I’d experience through any third party recruiter without the personal touch one might get from a good third party recruiter.
To be clear, we still have quizzes and the ability to show your performance on them to companies. The fact that they’re not completely mandatory to use Triplebyte doesn’t mean that they’re not important or that we won’t build further features around them.
Right now, companies can offer an expedited process as part of their reach-out. We’re working on ways to productize this a little bit better. We didn’t announce the feature in this post because it isn’t done yet, but one thing we’re looking at is allowing companies to set individual score thresholds for various types of expedited process (e.g. “if you scored a 3 on algos and a 4 on back-end, we’ll definitely get on the phone with you” or “if you have a 5 on python we’ll skip tech screens for you”). We certainly recognize that FastTrack was valuable to a subset of engineers, and we want to recreate the value it offered in a way that is a little bit more sensitive to the specific company and specific engineer in question. We imagine, for example, that less-prestigious companies (who are more concerned about attracting applicants) will probably set lower thresholds than the Apples of the world. Under the old system, we had to set a single threshold, which would necessarily either be too low for prestige companies or too high for everybody else.
And just to lean a little bit back into the pitch we’re trying to make here: if you’re concerned about interviewing process, wouldn’t it be nice to be able to search companies by how their interview process works? That’s not an axis on which companies meaningfully compete right now, but with the right incentives, companies will cut a lot of the annoying hassle that they currently have no reason to get rid of.
If a company asks me if I have questions, and I ask them, and they ignore me, I'm done.
I wasn't looking for a job, but thought the test results would be interesting. Who takes a test and doesn't get results?
Edit: just saw the comment that TB is keeping the quiz. Would consider using the platform in the future as long as that pre-qualification track is available and that this new direction is just a superset on the existing TB features.
On the other hand, when I applied to TripleByte, their feedback to me was "we think your skills are fine, but we want somebody who can perform well in an interview". Which is exactly the opposite of the value you're attributing to them here.
Quoting them, for reference:
> We really appreciate you taking the time to work on the take home project. We're aware this requires a substantial time commitment and we are really grateful that you invested the time in completing it. We thought you wrote a great, very full featured regular expression matcher. It was especially impressive how much you dug into the academics behind regular languages.
> However we made the decision because we felt that while going through the project together during the interview, we didn't see the fluency of programming when adding to it that we had hoped for. While we specifically designed the take home project track to help overcome the difficulties of coding under time pressure with someone watching, we do still need to see a certain level of programming during the interview.
- In borderline cases, we didn't want to risk our credibility and companies didn't want to skip tech screens. So competent-but-not-amazing engineers got shut out.
- Our quiz is not perfect, so engineers who didn't fit what we were quizzing got shut out.
- Not everyone needs us to vouch for them. Companies (reasonably) trust that someone with a degree from a top school and years of prestigious experiences can probably do fizzbuzz, and demanding that they prove it was a barrier to them using Triplebyte.
We think it's better to have more granular ability for companies and engineers to decide what mattes to them. A prestigious company can say "we'll only talk to people who took the quiz and got a top score" (and that score is us saying "this person deserves a job"). One that desperately needs the headcount can talk to the borderline cases or decide they don't want to put the extra barrier of a quiz in the way.
Like most markets, we think the hiring market on our platform works best when it's able to respond to local conditions.
If you're not willing to refer people to companies based on their score, how can you interpret the score as you saying "this person deserves a job"?
How does the new ability to make that recommendation address the original problem?
Does this not mean that even if you apply from TripleByte that you now still need to go through FizzBuzz and all that?
So it sounds like the hiring manager can choose to skip a fizzbuzz call when they know you already passed.
I've been wishing for something like this for years. Especially for early-career engineers or people from non-traditional backgrounds, this is insanely valuable because it helps you to know avoid wasting time on applications that will never go anywhere; avoid typing in your resume, line by line, into another form after already submitting your pdf. Avoid writing a cover letter to a job that's already been filled. Avoid applying to a posting that's really just out there for a company to "gauge" interest, not for filling a real role.
If an application is rejected, fine. But getting a follow-up request 7 months after submitting a resume into a black hole is ridiculous, and I think any system that decreases the information asymmetry between the applicant and the employer that allows people to intelligently approach their career search is going to be tapping into something truly valuable.
I'm in this industry and doing well because of recruiter reaching out to me (multiple times) and not vice versa. The whole idea of applying for jobs just doesn't seem to work at all.
On the other end a recruiter reached out to me and I'll be doing the final interview with their VP next week.
I don't think we can draw conclusion at that stage (we are a dataset of 2..). But I'd love to have access to the LinkedIn dataset to figure out if applying for jobs is broken at scale.
It's ugly and mimicks real life... more effort gives better results but life isn't fair and never has been. Luck factors but always fight on multiple fronts.
My major are slightly mismatched though.
Then I stopped bothering when it's super obvious that isn't working. Particularly, on Linkedin you could see the number of people that has applied (maybe a paid feature, i don't remember now). And for any entry level software engineering and data science position it's always in the 100s.
The one recruiter that I had contact with did more work than all of these. I've since switched jobs (to one of the FAANG) and that was through recruiter too.
I also don't think field matters until one gets to PhD levels, which I did not.
Like you, we thought (correctly, as it turns out) that it would appeal to people early in their career. What surprised us is that very senior engineers also told us they loved it - we didn't think it was as much of a concern for them, but sometimes your users surprise you.
It's also a great example of how obvious low-hanging fruit gets missed until you start thinking strictly about "what can we do to make job seekers' lives better?". It's not a particularly innovative or difficult feature, and yet major job sites with a dozen times our engineering resources still haven't done it.
The last time I was a hiring manager, I committed to giving feedback to anyone I rejected after the very first screening step. This was about 25-30% of applicants, which (even for a role in considerably less demand than engineering jobs) ended up being many dozens of emails. It was a giant pain in the ass - and it's pretty emotionally draining because you're turning down people who would really, really benefit from getting hired. I did it anyway, because I'd promised to, but it was not fun.
Next time I look for a job, I will definitely be targeting both these companies.
I wouldn't charge for the service, however, unless building it as a freemium model.
/economist hat
The most friction when switching jobs is the interview process -- and the question is why? I've been writing code for a decade+ now, working on startups, for known tech companies, public open-source, and have written a freakin' book! But, oh, my bad, I couldn't figure out a solution to your optimization question. I forgot how to implement the zig operation in a splay tree. I might simply not function well under pressure. Maybe I'm having a bad day.
Companies are missing out on literal geniuses by using outdated hiring practices. And, I get it, Google doesn't care. Amazon doesn't care. (There's an argument that they should.) They're huge and get X,XXX applicants daily. But why are small startups using the same hiring methodologies? It quite literally makes no sense. They're shooting themselves in the foot. I'm very passionate about this, and I'm working on a book on how to hire engineers. Maybe I'll actually finish it one of these days :)
And yet they will still email you, year after year… as a dev that's worked in finance/academic research labs and for low funded/bootstraped startups all over the world for the past 10 years… I'll never get into these big companies because I simply try to find companies that need work done yesterday, see my experience (maybe reach out to past corps i've done work for), and give me an offer for at least a short term contract (and these big companies will never do this).
The worst is when I've think I found an interesting company (that on more than one occasion, publicly likes to complain about not having enough devs… lol), start talking a bit and they send me a triplebyte link… maybe it will be different this time, but I doubt it because these behaviors are too ingrained in a lot of corporate processes (even if its mostly "big co does this, me small co must copy"-NPC type thinking)…
Past companies and open source contributions are not perfect indicators. I've seen people with stellar resumes fail to even know the syntax for an 'if' statement in any language of their choosing. I've had people absolutely ace takehome exams, yet in person don't know how to write a function.
False positives are also much more expensive than false negatives, so companies would rather accidentally weed out good candidates than risk hiring bad ones.
That's maybe a little unfair (plenty of people who aren't engineers take our quiz out of curiosity), but the basic point here - that companies are frustrated because they keep getting fizzbuzz-incapable applicants - is more-or-less accurate. That has been (and continues to be) a big part of our offering on the other side of our platform: in the same way that we can provide data about company behavior to engineers, we can provide independent verification of an applicant's skills to companies.
Wow, sounds like you either need to change your sourcing or improve your early screening. If that many poor candidates are making it to the interview stage something is wrong with the process.
Most either don't show up on time or fail fizzbuzz.
If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection. I've heard so much about this "programmer charlatan" that lies on his resume and ends up blowing up million dollar systems, but have yet to see any tangible proof.
> Past companies and open source contributions are not perfect indicators.
This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase? I run a tiny throwaway open source project (~600 GH stars) and even my reviews are pretty stringent. This point of view is not consistent. It's like saying "I know people that ran in marathons, but couldn't even run for half a mile in my interview."
I've seen similar. I wouldn't say 80%, but somewhere in the 20% range of people on phone screens. If I ask a similar modulo question that isn't obviously a rephrased FizzBuzz (i.e. "write a function that returns every 4th item in a list"), that percentage goes up dramatically. A lot of candidates memorize FizzBuzz without actually understanding what's happening and how to use it.
> This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase? I run a tiny throwaway open source project (~600 GH stars) and even my reviews are pretty stringent. This point of view is not consistent. It's like saying "I know people that ran in marathons, but couldn't even run for half a mile in my interview."
There are no time constraints, and no way to tell how much of their contribution is from their own knowledge and how much is copied from StackOverflow/other open source projects/etc. The point of view is consistent; I can't run a marathon, but I can walk 26 miles. That's fine for the local charity marathon, but it's not going to cut it if I want to do competitive marathons. The constraints and expectations are wildly different.
Even ignoring the fact that you keep moving the goalposts and are taking my "marathon" analogy way too literally, the argument that any kind of real-world coding involves "time constraints" akin to a 30-minute stress-ridden whiteboard interview is just bonkers.
I can run a marathon but I can't run 100. Your interview is one of many that I am making time over lunch. I'm tired and getting a perfect mark on a take home isn't as important as getting it done quickly so I can find time to apply to other positions before I go back to work.
If you are not paying someone to do the interview you are not going to get anyone's best. Even if you pay people are trying to juggle things to get the interview to work.
And to the parent post. 50% of people being late means they took the time out to talk to you and come to your offices and didn't gage the time or didn't think it mattered. That doesn't make them bad programmers and shouldn't even matter to the interviewer. Was the car service the employer used late picking the employee or did they have to come on their own?
I dunno about the back half (blowing up million dollar system) but the front half, certainly.
We used to ask "write min()" at a previous employ, and the pass rate was probably <50%. I still to this day structure my coding question around "it has to get them to write a for() loop". It's not trick questions (I understand why people don't like those, and those can die out, yes.) it's things like "here's a sample grammar, how would we go about parsing it?" which is something that I've used any number of times in my career.¹ Other candidates fail to display understanding, let alone deep understanding, about things like HTTP or Linux. Not exactly niche topics, and during this example period my employer at the time was clearly posting for "backend software engineer". And not just one or the other, but all of them, simultaneously. Since I can't establish any proof of anything, so… what other choice is there, but to pass?
¹while there are parsing libraries, yes, and I'll use those first / when I can, I would also say that 50% of the time, for whatever reason — better error reporting, better control — I end up going with a partially or mostly hand-rolled state machine or recursive descent parser. Also, while I wish people would just encode their data in something like JSON, or YAML, or whatever, it doesn't really matter, they. keep. making. new. grammars. Often inside those other grammars. Just today I had to deal with a config — a YAML config! — that wanted "key values". Turns out, it wanted strings formatted like "key:value", not YAML mappings.
With a bad pipeline, it's extremely likely, especially if you cheap out and try the bootcamp route (there's a reason Lambda is desperate enough they will "loan" a student you can try before you hire!).
It costs nothing to send a resume after all...
They could maybe just be making small or simple changes. Or perhaps mostly working on markup, templates, CSS, etc.
It's possible they know a bit of programming but just don't happen to know how to check if a number is evenly divisible by another number. This isn't something an open source project would necessarily know you don't know. (I agree it's something any programmer should know, but I can see how someone could contribute certain things to certain projects without that knowledge gap being discovered.)
I learned about the modulo operator very early on, but if I didn't know it existed, I'd admittedly struggle with the question for a bit and would seem like I don't know how to program; especially in a pressured interview environment. If it were Python or something I think I'd come up with something awkward like `if i / divisor == float(int(i / divisor))` within a minute or a few minutes, but who knows, maybe the pressure would mess with me.
Of the people that turn up late to the interview, what percentage pass fizzbuzz?
(N.B. I was being a bit hyperbolic, but, yes, for example, I highly doubt Steve Jobs could.) The spirit of my point is that most smart people don't care about the kind of idiotic minutia typical hiring panels ask. They care about interesting/creative problems.
Tongue in cheek, good luck on the pivot guys, it is a tough problem.
https://keithschwarz.com/interesting/code/?dir=find-duplicat...
I have Github repo. I have my own portfolio. I have wrote flutter app deployed to play store to demonstrate I can function individually and still can write production ready code. And All the above just on my own spare time.
But every company I have applied told me my skill set doesn't aligned with them. I get response from them that there is no traction on my resume. While some friends get interview after interview and others don't.
I am not saying that I am the best but we need different process where entry is given and now show me what you can do.
* The blog post complains that even experienced engineers they interviewed couldn't write some simple code on paper. Did anyone stop and think that maybe it's writing code on paper in an interview setting that's the problem? Stop and think about this. Imagine writing code like you usually do in a text editor. And then writing code with pen and paper. Backspace? Can't do that. Copy/paste some line to move code? Can't do that. Add some newlines? Can't do that. Run the code using the compiler? Can't do that. Of course most developers, even experienced ones, will do poorly under these conditions, this is not how code is normally written. On top of the already stressful situation of an interview, there's this suddenly completely foreign and limited way to write code.
* Even the blog post only mentioned fizzbuzz as a quick, easy question to filter out zero coding ability. Another way to say this, it was meant to filter out coding skill <= 0. Not skill <= 50 or skill <= 90. Zero. After that, the interview was supposed to move on to things that actually matter for software engineering. The present day interview has strayed extremely far from this. These days, you get asked two leetcode questions during an interview and even if you perfectly answer the first one, if you don't get the second one then you're an automatic fail. That was certainly not the intent of the original fizzbuzz, since answering any leetcode clearly shows coding ability. And the interview never has a chance to move on to the part that actually discusses software engineering skills. Or even if it does and they do very well on those interviews, the failure of only getting 1/2 leetcode questions correct overrules everything else since they must avoid "false positives".
I moved from engineering to management over 5 years ago now and still in interviews for positions like "head of" in 100+ people companies or other management roles in bigger companies I got asked the usual algorithm questions.
I just have zero interest to prepare for these interviews, I don't have the time or the motivation, and I honestly just don't care about the answers. Most of my job, and my team's job, has nothing to do with them.
The world might be different in FAANG, and admittedly I have not worked there (though I have worked for other top companies) – but most companies are not at that scale, and they should not use the same metrics, particularly for management roles.
Also, dissing those who have passed the bar as "engineers who like tests" is disrespectful. When they first launched, passing the bar makes you (in their words) a highly qualified engineer, and now passing the bar means nothing except that you only "like tests"...lol
Fundamentally, I think this invalidates Triplebyte's business model. Companies don't really care if someone passes your tests. They still put you through LC type interviews onsite. They simply save a phone screen.
We still have the quiz, and we still show performance on it prominently on your profile (provided you've chosen to share those scores). In fact, we've put a lot of energy into improving the trust companies place in our quizzes over the past year precisely because we think that skills data is important.
> Also, dissing those who have passed the bar as "engineers who like tests" is disrespectful. When they first launched, passing the bar makes you (in their words) a highly qualified engineer, and now passing the bar means nothing except that you only "like tests"...lol
Heh, yeah, that's fair, at least to a point.
Our quizzes are predictive. We know this from a lot of objective data, it's what you'd expect subjectively, and companies do tell us that they trust and value that data. That does not mean that our tests have no bias towards certain personality types. Different people respond differently to testing, and that does have a differential effect, not because our tests are bad but because testing is inherently somewhat artificial.
I actually taught test prep before joining Triplebyte years ago, so I've been this first-hand: it was not uncommon for a student who'd been doing well in practice sessions to crumple under the pressure of the real thing. That doesn't mean the test is bad, it just means that some people fare better on tests than others for reasons other than just their raw ability. When we say "likes tests", that's more what we mean: not that that's the only reason someone does well on a test, but that people who perform well in isolated, pressured environments do better relative to their skills than others.
I think that goes to another ability. If we need a candidate to gracefully handle a 3 AM production outage, we should expect them to handle an interview.
So while in theory I like the idea of this meritocratic minimum bar granting special privileges (I liked it enough to join Triplebyte!), in practice we seemed to be fighting an uphill battle with all but a few candidates each month.
On the other hand, I have wasted a lot of time in conversations with recruiters only to be let down at the end by mismatched expectations. I'm interested to see if this new approach can make a meaningful difference in that problem.
I love this, but I'm curious about the incentives here. Triplebyte only makes money from companies, not engineers. In the long run, can engineer-centric intentions override a business model based around companies and recruiters staying happy? I hope so!
Triplebyte has always seemed scummy and this obvious--and unacknowledged--contradiction makes me trust them even less.
We do also have differentiating features on the company side. It's not like we're abandoning the idea of building features for companies or anything.
The job search, like other matching problems, is often not zero sum. So building (say) a better search interface for companies benefits engineers as well, because both sides of the market have an interest in a good match. What we're saying here is that in the cases where things -are- zero sum, the power - and therefore the incentives - lie with engineers and not with employers.
https://news.ycombinator.com/item?id=27501675
Our bet is that LinkedIn is going to fragment. They are just not creating a hiring process that most engineers like. People tend to either get ghosted, or overwhelmed with low-relevance inbound (almost no one gets the "right" amount of attention). Companies need to go where the best engineers are, not the other way around. So I think our long-term incentive is to fix these problems.
In any case, I'm committed to giving this a try. There is danger that we get pulled toward building for companies. I want top guard against this by being public about what we're doing, and "showing our work" as we go.
> There is danger that we get pulled toward building for companies.
A common phrase thrown around with free services is "if you're not paying for it, you are the product".
> For example, companies aren’t normally incentivized to provide salary and culture data. But we can force their hand by promoting transparent companies in our search rankings.
Maybe I didn't read the article carefully enough, but are you planning to continue charging companies $15k - $30k for the ability to access candidates on your platform?
If so, companies are still your customers. And if you're building and optimizing your product for people who aren't your customers, your real customers (companies) may not be happy with you which will hurt retention, etc.
Maybe I skimmed the article too quickly, but it seems strange to charge companies $20k+ to post a job on your platform, and then actively do things that "force their hand". It might be a net benefit to the engineers on the platform, but I wonder how it will work from a business model perspective since you're potentially creating adversarial relationships with your "real" customers (companies paying you to access your candidates).
Edit: But maybe it's by design, if you actively remove employers who aren't abiding by your philosophy. Although again, that means turning away customers, which means turning away revenue, which in my mind raises questions about the overall business model. A lot of conflicting interests.
That does not fully express my motivations. I am an engineer and find the idea of making the process better for engineers more exciting than making it better for companies. But it explains how I think the incentives work.
this post is marketing with commenters associated with the company
That said, I think a hiring process that puts engineers in control (no ghosting, get data on when a recruiter looks at your application, search ranking by whether companies lie to candidates) is something that should exist. I want to try to build it!
The trick here is you need to figure out a business model where you don't take money from employers, because once you do its just the long slide to becoming a shittier LinkedIn.
I personally like this proposal and am probably going to at least draft a hypothetical spec of it and see how it fleshes out. Thanks for the suggestion!
Pretty obvious value-add for job searchers. Not to mention that having access to this data would enable tons of other product features. Shows that most services/sites don't care that much about the applicant experience.
https://news.ycombinator.com/item?id=27256776
Colorado now requires job postings to include something more or less like a "reasonable salary range". It hasn't been perfect, for example there are some companies who now just restrict their online job postings to say "except Colorado" instead of adding a salary.
But as someone living in Colorado it's been nice to know the salary range up-front more often and to have confidence that I can ask them for a salary range without getting the question turned back on me: "Well, what are you looking for?"
(c) An employer, upon reasonable request, shall provide the pay scale for a position to an applicant applying for employment.
[1] https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml...
So it seems it’s not enforced.
If you have a janitor in CO, every job is an improvement and potential promotion. Having to first go to that janitor for every position is going to wear out FAST. Posting for the CTO position? Go to janitor first. Posting for CEO position - same thing - that's what is going to drive folks away.
Unfortunately it looks like companies can just state wide ranges (which don't help applicants understand the market any better) and they can also omit other compensation elements beyond the base salary component (stocks, cash bonuses). That makes the disclosed information not very useful.
The biggest strength for me is that I can ask them and they have to tell me. When I was job searching before I'd ask the recruiter and then they'd always ask me what my target is. Not showing your cards is like negotiation 101. Plus it's made fielding messages and emails easier. If they provide a range, I can just easily say "This position sounds interesting but unfortunately the salary range isn't what I'm looking for currently."
You're right that a lot of positions aren't great about this, I've literally seen "Pay starting at $0.01" or ranges literally $30k to +$200k. But I've seen it as a benefit in the negotiation process and it's just been easier to field recruiters.
> total comp
This law only covers salary as far as I know, so equity and such aren't included. Total comp is different.
It's not perfect, but I feel like it's wildly helpful for a ton of people. It helps transparency for a lot of positions and I feel like it rewards honest companies. If equity and bonuses are a big part I think it reduces the overhead for talking about salary to clear the way to talk more about equity and bonuses.
It's also in the early stages and CO is the only state with a law that requires disclosure up-front to my knowledge (someone mentioned CA has a law, but they only have to answer if asked). It feels like a fantastic first-step to me.
This is a space that unions fill well compared to for-profit recruiting agencies. The organizations give engineers control and leverage in hiring, and can help them up-negotiate. They also allow employers to hire from pools of skilled talent.
Whereas with recruiting agencies, the customers are employers, not the candidates. Incentives are not aligned to give candidates control.
People at the recruiting agency usually get paid a commission as a percentage of the first year of the hiree’s salary (not taken from your paycheck, but just in general). Which means, it is in their interest to get you as high of a comp as possible. Which also means that I dont have to bs around with the recruiter about my salary expectations or anything like that. I give them the upper range of what i want, they tell me either “sounds good” or “they are easily willing to pay more, so why not try $X instead” or “they cannot pay that much, but there is this other company that we work with that can offer you as much.”
And if you work with one recruiter consistently, it gets even easier, because now that the recruiter knows my total comp expectations, she pre-filters opportunities for me and only contacts me if there is a position she is aware of that pays at least as much as I want or more.
Of course there are downsides too, for example, the limited choice of companies that the recruiting agency is working with at any given time, so i still have to find some opportunities and interview with those companies on my own. But as a supplementary option, working with a good recruiting agency has been quite nice for me.
P.S. The type of a recruiting agency i am talking about is the one that is focused on a specific sector (in my case, fintech/quant), so it might be a completely different story for a “generic software dev” recruiting agency.
Beware: this is not universally true. What you typically see is that it leads to closing as many candidates as possible. A recruiter can make more by closing 2 candidates at 80% of max comp than 1 candidate at 100% in the same time.
This is a well known behavior in real estate sales, which have a similar commission structure.
Even worse, real estate pays both sides so realtors are heavily incentivized to get their buyers to purchase from their own sellers so they can get both sides of the commission. Instead of paying them 3% to act on your behalf and do a bunch of leg work they get double and now they can do less work and aren't strictly working for your benefit.
Working on commission just leads to wanting to make a deal happen. Even ignoring the time aspect of it, if a recruiter has much better odds of making a placement at 80% max comp compared to 100% max comp, the 80% is still objectively better. Why risk not closing the deal when the expected return is negative?
I am saying that because I’ve been working with her for at least 3 years now, and I am yet to take a single offer from the companies she set me up with (either due to them getting cold feet last second or simply due to me not passing some of those interviews or me declining due to some red flags with the companies). If it was all just about high volume for her, it would have been in her interest to drop involvement with me after the 3rd unsuccessful attempt to set me up with a fintech company.
Collective bargaining is inherently about eliminating competition between workers. That is, it creates a labour monopoly to balance out a labour monopsony. That might (might!) be a good thing for autoworkers. It would be an unmitigated disaster for software engineers, and the tech sector in general.
People who write code are unbelievably productive compared to just about any other job function, and software engineers (who we'll define as people who write code for tech companies) are even more so. Part of eliminating competition between workers is making sure that nobody works harder than anyone else. That would be devastating to the productivity of software engineers, which would make them less valuable to tech companies. That would lead to a decline in salaries, and general stagnation of tech.
Another aspect of preventing competition is seniority. One of the archetypes of the tech world is the wunderkind—the brilliant kid fresh out of school that rockets to the top. A union would tell that kid, "sorry, you'll have to pay your dues as a junior engineer for 5 more years, then you'll be eligible for promotion. In the meantime, we'll give that job to cwp, he's got the required seniority."
Just think how this would affect tech companies. Sorry, you can't pay more to attract the best. If you're really careful about interviewing, you might be able to find some people with both skill and seniority, but it'll be tough because smart, ambitious people have all gone to Wall St or whatever industry has the best opportunities.
A yet every movie star in the US is in the same union as B-movie and infomercial actors.
> Another aspect of preventing competition is seniority.
Weird, considering pro-athlete unions regularly help kids out of college quickly become multi-millionaires.
> One of the archetypes of the tech world is the wunderkind—the brilliant kid fresh out of school that rockets to the top.
Weird, considering unions have been at the forefront of education, apprenticeships and training that helps kids out of school rocket to the top, the same apprenticeships and training that companies refuse to fund themselves.
> Sorry, you can't pay more to attract the best
Weird, considering SAG-AFTRA regularly helps movie stars up-negotiate multi-million dollar contracts.
However, I dropped out completely because TripleByte wouldn't let me see companies without entering in a desired comp number. My recruiter at triplebyte said I could just put in "$1 or $1,000,000" to get past it, but that just feels like the same kind of corporate HR bullshit that people have complained about for years. Just Google "recruiter won't proceed without desired salary" and you'll understand how many folks dislike that question.
The experience made me feel that Triplebyte wasn't interested in putting me in control of the process at all if they have such a user hostile requirement and require me to work around it by putting in fake numbers. I understand that many people will have no problem providing this info, but I prefer to see what companies offer me because it tells me how they value me and the position they're hiring for.
Almost all recruiters have been fine with that, but a couple haven't. To such people I wish a good day and go my own way. There are plenty of other recruiters in the sea.
1 - I get real-world interview practice, and that always helps me in future interviews. Many companies ask the same or similar questions, and the more I'll be ready for them being asked again. In addition, when I'm surprised by a question I'll research it thoroughly when I get home and won't be surprised by that question again.
2 - I get a bit of an inside peek in to how the companies I interview at work. That's a privilege not afforded to many outsiders, and how some of these companies (some of which are world class) can and has helped me at other companies.
3 - A company might bring me in to interview for one position, but after the interview might determine I'm better suited for another. If I didn't interview with them they'd never know.
4 - As I interview with people I get a chance to make a good impression on them, so even if they don't wind up hiring me for that position they might want me when a different position opens up (whether at that company or another company the people I interviewed with moved to).
5 - By letting companies make the first offer I myself get a good salary survey of the minimum what all the companies I interview with are willing to pay for someone with my qualifications in that position. Of course I try to do my own research ahead of time, but getting salary numbers without getting a job offer is not always possible or reliable.
6 - If I blurt out a number first I'll it may put me at a disadvantage in the negotiation, as they'll just start the negotiation from what I say even if they would have been happy to have paid way more.
I hear this a lot, but how much interview practice do you really need? After the third or fourth set of interviews, they start to really blur together for me.
None of your points are false, but if you're casting a wide net and taking every interview without any idea if the company can even afford you, that's a lot of hours spent. The last time I looked for a job, I interviewed at only four companies and spent an average of probably 5-6 hours on each, and it was exhausting. I can't imagine making a habit out of it while also working full-time.
But, in my case, doing many interviews helped me understand what negative signals I am emitting during interviews [according to the interviewers]. There's nothing like an outsider's perspective, even if you disagree with it.
---
Example: in one of my last interviews I was told that I am too "cocky" and "over-confident" because when I was given real-time modifications to a homework assignment (that I completed before) I was smiling and saying "oh, that's easy!" and proceeded to make the changes live in my editor -- while sharing screen.
This feedback left a very WTF feeling in me. Dude, I am (a) enthusiastic about the task and (b) have enough experience to show you, in real time, that I can modify the code right there and then to suit the new requirements, and (c) chatting with the interviewing team while doing it... and all you can gleam from that is that I am overconfident and cocky? Seriously, get a grip.
For the record, I strongly disagree with their take. But it gave me a very interesting perspective -- namely to scan for the more introverted / likely-insecure people in the interviewing dev team and try and act a little more modestly to not rub them the wrong way.
It's a fact of life that most people suck at interviewing -- as I hope that I demonstrated here. One of the guys in that team simply dislikes certain kinds of people and he lets this go against his better professional judgement about my abilities. Not cool, right? Especially when you also take into account that the CTO who also attended all interviews agreed that my tech acumen is top-notch and they seriously couldn't catch me with my pants down no matter what they did.
So again, this helps me gather experience and culture and makes me more adaptive in the future.
Does it hurt your ego? Absolutely. Does it shake your preconceived notions? Much more than one would think beforehand. Is it exhausting? Hell yes, sometimes after an interview I didn't want to do another one for a week.
But, is it also very valuable to help you improve your interviewing and negotiation skills? Yes!
That's why I did lot of interviews the last time around. And I'd do it again.
The nice thing about the explosion of remote work due to COVID is that engineering salaries are beginning to normalize across the US, and there is now a lot more salary data available online, so it's easier than ever to see what you're "worth" on the market on average. This at least gives you an anchor when having these conversations, so you're not going in blind, and gives you a lot more leverage if they low-ball you.
[0]: Yes, I'm aware of the drawbacks of self-reported data, but it's still broadly useful in aggregate.
[1]: I'm making up numbers, but something like $120k - $200k, with the lower number being the minimum you'd realistically take. Usually recruiters are honest up front if they can't even afford your absolute minimum, but it still leaves you with a lot of negotiation room once the offer comes in.
Why even give an upper limit? It's not like you'd refuse $400k if it was offered to you, would you?
Most companies will naturally start negotiation near the lower end of your range, even if they would have started way higher had you let them make the offer first. I really don't see an upside to this strategy.
Because if you just give one number, even if that's your absolute minimum, the entire salary conversation is going to be anchored around that one number, and you're going to have a harder time asking for 50+% more than that. If you give a broad range, it's easier to justify countering with something closer to the top of that range.
> It's not like you'd refuse $400k if it was offered to you, would you?
Of course not, but nobody has ever offered me that. :) In all seriousness, yes, you run the risk of your range coming in below what the company would be willing to pay, but in my experience (and those of friends and acquaintances), outside of FAANG, I have found that to be extremely rare if your range is wide enough. The companies that pay super well are generally big, and thus well-known to pay that much, and I wouldn't give them any salary numbers at all because I'll have a pretty good idea of what they'll pay me with a bit of research prior to interviewing.
Of course, anecdote != data, so YMMV and all that. This is just my experience interviewing at mostly smaller companies.
Agreed though in practice that just moves the pre-interview conversation from a comp discussion to a leveling discussion.
Luckily levels.fyi has fairly accurate data for those same companies.
That time would not be "wasted" if the company had done due diligence on what the position and candidate were "worth" and revealed their target (or target range) to the candidate.
The real reason, I'm sure, is that companies think they hold the upper hand in the negotiation. And a lot of time with a lot of candidates that's true. It's clearly _not_ true for the class of candidate who's realistically got a $300k+ target. You are totally not going to successfully headhunt anyone in the top 1% or 5% of FAANG engineers if you try to dick them around with recruiting power imbalance power plays. _They_ don't need your bullshit. They will totally walk away from whatever time you've "wasted" so far trying to recruit them. You need them _way_ more than they need you.
(And I'm pretty sure that generalises a lot further down the experience/renumeration food chain than most devs realise. Fresh grads or devs with only a few years of experience might need to play the stupid recruiter games, but most companies trying to fill a mid level or senior role need the right candidates way more than those candidates need any one specific job. You can and should be able and willing to walk away from any potential job offer over stupid games like this...)
I used to be much more relaxed about this, but if you get burned doing a few rounds of interviews and it was a waste you wise up pretty quickly - teams do not like time wasted.
I've also found that folks who can't / won't talk money - usually not great hires for a bunch of reasons.
We're happy to talk through the different positions, levels, progressions and comp ranges for them all. But you have to have a clue about what type of comp would work for you (early on in initial call or two) or it's just not worth continuing forward.
You say "I've also found that folks who can't / won't talk money - usually not great hires for a bunch of reasons."
The same goes in my experience for companies which insist on talking salary before the job offer. It's a red flag.
"you have to have a clue about what type of comp would work for you (early on in initial call or two) or it's just not worth continuing forward"
Of course I have a clue, but I'm not going to put myself at a disadvantage in the negotiation by revealing it before I've been given a firm job offer and heard a number from the company first.
Also, in my experience recruiters are the ones who push hardest to hear salary expectations up front. They know it'll give them a leg up in negotiations, and some of them are even paid specifically to get that number (as one recruiter was brazen enough to reveal to me).
In this case, obviously a job seeker does have many possible employers to apply to.
So no, AFAICS you are wrong and the original thesis of "the company holds the advantage" is correct.
___
[1]: Consider the connotations of those words.
If you're happy to provide a range, great, if I'm still there doesn't that generally indicate that the range is in line with my expectations?
I guess your comment kind of makes sense, I can see how corporate might interpret 'this person looks out for themselves' as a potentially bad hire.
That’s because it benefits you as an employer.
I’ve been on the hiring side and I know there’s a budget. If someone is too high you want to push them out right away.
My flatmate in Singapore about 6 years ago was looking for a new job. Her salary was about 2200/m and she said the recruiter kept asking about salary.
I told her not to discuss it. She doesn’t /need/ a new job. If the company really values her then they will give her an offer. They have a budget. They offered her around 4600/m.
If she told them she was on 2200 it’s guaranteed they would have given her only a small bump.
Companies should never ask for salary info. Because it’s only used to get cheap labour.
Your clients know their budgets. Why not just be upfront about that to the candidates. Tell us your budget or reasonable expected range your client can pay. Then if (when) we find that it's well below what some other hungry companies are paying, we can save you and us time by passing.
If you know your client is paying low, then just be open about it. Something like, "Ok, this client is a bit below market (and tell the upper limit they will pay), but they have these good things to offer." If the client has no good things to offer, and the pay is low, then you're still likely to not seal the deal with a candidate no matter what. So just cut the game and get to the point.
This makes me think that all jobs position should have rather narrow target ranges which are stated up-front. Then you can discuss if you are too high level/low level for that job and range.
It seems to be terrible waste of time going through interview process to learn that they won't even come close to what you want. I would not do that to myself.
at senior levels especially, you should know what you want. comp bands are informed by huge data troves now and any edge you may have gotten before is gone due to the fixed and tight bands. just state your requirements and insist on “turnabout is fair play” ie they need to tell you the band. do this during the prescreen and save everyone’s time. if you kill it in the interview you still easily can negotiate to top of band.
If your desired comp number actually is $1M/year, will that just flag you as a joke to Triplebyte companies? With post-pandemic stock price increases a lot of engineers at FAANGs are making more than that now.
For any reasonable definition of "a lot", no they absolutely are not. A lot of people with 2+ years of experience who've either been promoted once and received a strong annual review, or switched jobs as a mid-level industry hire are making in the 200s and 300s, a few pushing into the 400s. Much more than that and you're looking at the top 1% (or less) of engineers at the top companies. Not a lot.
Source: was a mid level engineer at a FAANG until I left late last year, knew seniors and principals and their comp. Nobody was making $1M/year.
Nobody that makes 2x what their coworkers do at the same job/level casually discloses their comp at the water cooler.
A) your company is engaged in disturbing monopolism; B) your company is engaged in disturbing labor exploitation and arbitrage; or C) your company is engaged in disturbing environmental destruction.
Would love to be proven wrong. Am I?
Edit: option D) is that American cardiologists are not as exclusive and skilled a labor force as I would prefer to believe ;)
They’re all doing well, but even Amazon hasn’t doubled their stock since prepandemic.
There are definitely more people making 1m/yr at FAANGs than their were last year, but I agree with others that it’s not “a lot.”
AMZN in May 2019 = $1775, today = $3496, 96% appreciation.
AAPL in May 2019 = $43.77, today = $130.87, 198% appreciation.
GOOG in May 2019 = $1103, today = $2508, 127% appreciation.
That's all over the last 2 years. Market low was apparently Dec 2018, before the Fed started cutting interest rates, so the numbers are a bit bigger since then. They were pretty steady through 2017-2019, so anyone who got grants or refreshers during those time periods has benefitted from that appreciation.
Also, staff level is a tiny slice of the engineers at FAANG. So still not “a lot” even if your stock appreciation numbers were correct.
I don't think this part is off. Why I never went to Triplebyte and will not go, I think doing tests is dumb and don't wanna do it. Like original commenter said, you can't change model that quickly.
It's incredible how mileages vary. I recently had to strike companies off my list which demanded to know my last salary.
I offered them my salary expectations, but they would absolutely not accept that, and preferred to take the fact that I would not tell them the exact salary at my last position as a reason to decline to immediately stop the interview (with half an hour left in the call).
This wasn't a one-off either, I would say about 5% of companies (biased sample) try and get away with shit like this.
(This was German-speaking Europe)
In which case the correct answer is "Das geht Sie überhaupt nichts an."
As CEO, you:
* Grossly expanded profiles without user permission.
* Laid off tons of staff right before the COVID lockdowns. (What has been your personal returns since April 1, 2020 by the way?)
* You have been privy to discrimination happening in client company on-sites yet did nothing to the client companies.
You’re a CFO, an investor. You’re not a CEO, and you certainly do not have my trust as a developer.
In 2015, candidates with the same schooling scored about 20 percentage points lower than 2020, the last candidates I used the test for. The number of questions doubled, and then the test got much easier, by eliminating more challenging programming questions and replacing them with questions with giveaway context. By comparison, according to my data, about N=43, until about 2018 being a senior versus a junior in college CS programs is worth about 10 percentage points on the test; going to Harvard instead of Berkeley is also worth about 10 percentage points. In 2020, the last tests had no predictive features.
I stopped using the test, because it became too easy and too noisy to be informative.
I recognize some of the coded language in the blog post. There are definitely more lucrative opportunities in recruiting for DEI. I don't know if it will last. If you're still jittery about the public-profile-by-default thing, which by the way, was totally irrelevant and overblown IMO, this may not be a pivot for you.
One thing I see in the data is that at MIT, women and men performed the same, controlling for seniority. This wasn't true at the 3 other universities that produced enough data to measure.
That said, what really is the best way to hire candidates? I'm not convinced having binders full of engineers is special, there are almost always more candidates than jobs, at least 5:1, in every non-credentialed industry vertical. Anyone who has worked at a jobs (or indeed any matching platform, like the Common App or Tinder) knows that.
Then there's this long thing about asymmetries or whatever, warble garble about missing information... It has never, not once been my experience that someone seeking a subordinate role at a typical private company with preferences like "pair programming" or whatever have ever been better than someone with no preferences at all.
Maybe it helps to engage in the vanity of whatever trendy workplace trend is hot for whatever vertical. But like, if you're being intellectually honest, if you thought pair programming was important, you'd pair program at TripleByte, but you don't, you know in your heart of hearts none of that shit matters, so why are you putting stuff like that into your search system?
Indeed and ZipRecruiter are ad arb companies. They don't care. Private universities lead, not lag, DEI at giant companies, so it's hard to see how to compete against them in that core business. It will still come down to a real defensible opinion.
Do you have more valuable inventory than ZipRecruiter for DEI candidates? Who knows. What an uninteresting question. Apple also hires people who just make shit up on their resumes, I know two - though they weren't engineers.
If this were true for "qualified" candidates, there wouldn't be a labor shortage, right? ;-)
They told me the actual number of qualified candidates was usually just a handful or less.
I doubt this is true in SV, but I don't live in SV.
That's interesting to me. Most reputable rankings put Berkeley at #2 or #3, and Harvard is usually around #7 or #8 (and US News puts them at #16!). I wonder what the cause of that discrepancy is.
From what I can tell, anyone who is a Harvard student can declare the CS major if they pass the pre-reqs. At Berkeley, passing the pre-reqs doesn't get you into the major -- you still have to apply for a limited number of spots, which generally means having a 3.8 GPA or better in your pre-reqs. So effectively both programs have the same acceptance rate.
Berkeley EECS is sub-7%.
Most clinical doctors have social skills and crystalized intelligence from domain expertise, but aren't typically mistakable for particle physicists.
People don't necessarily require the proper sheepskins to possess fluid, crystallized, and/or other domains of intelligence AND the skills, personality, and experience relevant to excelling at a particular STEM role.
Pedigree is mostly used for social filtering and business leadership board packing, but if someone wanted to create an elite monoculture of staff, lacking in cognitive and personality diversity, by all means, go right ahead.
PS. I. Don't even get me started about a car full of CS IITians talking about the JEE and high-placers who seem normal on the way to snow country for snowboarding and gambling.
PS. II. I bombed an Apple interview for a mid-career role by being too intelligent and too maverick compared to the group of compliant, I hate to say, yuppies. It was an interview panel of about 9 people and they were just speechless. The other time I bombed an interview for being too smart was about 10 years before that when I was 20 at the old Borders bookstore in Palo Alto. Moral of the story: it's important to play dumb where appropriate because most people are relative-intelligence insecure.
PS. III. Sorry, reader, for the rambling and discontinuous thoughts. Absurd endocrine values of unknown etiology currently... doctor appointments pending. ):
That sounds awfully specific.
Are you sure they were speechless due to your high intelligence and maverick qualities?
There is a large pool of people who think that they are programmers, but are not. This is exactly what the infamous fizzbuzz test was intended to weed out. I personally know a top notch recruiter who credits her success due to learning early in her career how to filter most of the junk out. For example a "Perl programmer" who listed Microsoft Word as a skill was probably not worth looking at. (Note, this is an example from 20 years ago. But the principle remains.)
If you create an engineer focused site and DON'T find a way to segment engineers by ability, you'll find that your negotiating power is based on the average of your candidate pool. Which means that you'll discover why traditional recruiters kowtow to companies and not the other way around.
& maybe I'm wrong but I thought the entire value proposition of TB was that their pre screening and testing process made the whole process much less noisy for your hiring managers. If you still end up needing technical interviews and aren't guaranteed that all the candidates you get have gone through some technical screening, what's the point?
> Going forward, our assessments are a purely optional means by which engineers can show their skills to employers (who overwhelmingly tell us that they trust our scoring), rather than a requirement for entry.
I think the idea is that an employer can still filter out the unfizzbuzzables, but has the option to ignore that for candidates whose stellar track record is a stronger signal.
I'm a little bit skeptical, because i think there are lots of people with apparently strong CVs who are still weak when you actually put them to the test. But maybe not at the very top end; i am really thinking about people with experience in various investment banks, which is perhaps less of a signal than Apple then Netflix or something.
The problem is we're all spending weeks and weeks on weeding out fakers. We've moved to a quick pop quiz right after we get an apply. This takes out 85% of the candidates (it makes me sad when I look at how simple the stuff is that weeds them out...). Two interviews are more than enough to land on a hire after that.
Our company is a conversational recruiting software company (you apply, we say hi, and screen candidates in real time and then hook the candidates up with an interview). We started doing the screening in our chat flow and it works really well for us.
Sure, having metrics on how often a company ships code, etc. would be helpful, but frankly I have about 10 companies on my list that I'd work for, and I doubt any of them would give you this data.
What I'd really like is not having to practice leet code problems for 3 months before I interview. The older I get, the less motivated I am to do this. I've worked in senior engineering roles at big tech (FANG) with 3+ years tenure. I don't think I managed to not be able to do my job and fooled people for that long.
I've interviewed people I KNOW could do the job, but they didn't get the optimal solution, or were nervous, or N things in the loop, and we ended up not hiring them. There must be some incentive to create a system that lowers these false negatives.
I simply do not care about things like, “what does malloc return”. I do not care what people know about bloom filters in Postgres. The 99.99% of web developers don’t have to know these things. I do not care if you can implement Tic-Tac-Toe.
It’s poor interviewing, which is the entire product they’ve been offering.
I’m generally quite pro immigration but when BigTech goes to DC and bellyaches about how they need higher quotas because they can’t find enough workers it really sticks in my craw.
Reminds me a bit of the classic example of chutzpah—-the boy who killed his parents, then threw himself on the mercy of the courts, asking for leniency because he was an orphan.
I am looking forward to progress toward a hiring process that puts engineers in control, such as by allowing them to accurately record their current role in their profile.
But the transparency you need to extract is mainly financial, that’s the big asymmetry. I think people care more about money than they say they do, and less about release cadence or test coverage or whatever than they say they do.
So triplebyte should be called “get me a raise.com”. I tell you what I’m making, smash all your tests, and you find me somebody who wants to offer me e.g $50k more or $100k more, and that WILL be the offer I get. And if the company reneges on that offer then it’s flagged to warn others, and if they do well by their employees they are rewarded as well.
That said, the idea of "we will offer X to anyone with Y quiz scores" as a showing a company can make is pretty cool, and is definitely in a "company makes public conditional guarantees" space that we're interested in exploring.
Does this guy have a patreon?
how do I reply to recruiter contacts? applied to a job, received a request for interview from company. Nowhere to reply on the platform? How do I get in contact with them?
Should there not be a reply button right there with recruiter's message?
p.s. I already emailed support but, my past experience tells me they won't reply.
Funny but whenever I think of Triplebyte, all I remember is seeing their ads everywhere. I am personally a fan of Indeed and they do a good job sending leads and applicants other than good old Linkedin.
> We got jobs for over 1000 engineers
Given how many years Triplebyte has been running, "over 1000" seems surprisingly low. I wish them luck with their pivot.
dev)
> I forgot how to implement the zig operation in a splay tree.
employee at Triplebyte)
> To put some hard numbers to this: further down this thread, there's a post about how "any engineer" could answer a question the poster thinks is too easy. I looked up the question in our back end and, in fact, barely a third of people who take our quiz get it right (the correct answer isn't even the most common one!).
So there 1000 job matches and at most (("hundreds of thousands of engineers on our platform" / 3) - 1000) who were incentivized to answer these contrived problems, did so correctly and still couldn't be matched.
So even among the large pool of engineers who have gone thru the process, met some arbitrary threshold of engineer-ness, there's still a huge mismatch between corporate/prospective employee expectations, that I'm not sure will be able to be overcome quickly even with these new initiatives, but it's interesting that they are being pursed now (not surprisingly after the shift in working environment after massive government restrictions on freedom uber alles).
Yeah, different companies want different things and are ok with different things. My best experiences have been with companies that have taken less than 2 weeks to hire me with no testing whatsoever: just a couple of calls. Other companies want and only select for the contrived coding tests and programming jeopardy, and that's fine, but I want nothing to do with them at all.
I would be interested in using triplebyte if they explored something like this: https://news.ycombinator.com/item?id=27545446
Maybe even allowing 3rd parties to bet on specific hiring decisions (in addition to what is described in the link) could add another layer of incentivized feedback loops to the process (esp if the people making the hire/fire before 3 months are made known) and add another layer of market driven signals for them to pay attentions to. Maybe make it more expensive to chase certain types that are in higher demand, and less expensive to chase others types that are in low demand (not the salary to the employee, just the payment for filtering). Maybe less expensive for shorter refund windows, and more expensive for longer refund windows.
For sure. That's very likely why they're making this pivot. Exclusivity was their goal, but it just turned out to not be a very profitable goal for them, it seems. I was basically just saying it seemed to be a "theory" issue rather than an "implementation/execution" issue.
$25k*10k = $25M / 10 years = $2.5M / year. That is peanuts compared to how many people they employ. Plus the ads they run...
I guess they didn’t hit product-market fit with candidates. Two sided markets are tough.
It’s possible that they just couldn’t code with someone watching but the effect was the same.
Note that I didn’t have control over the interview process, so I couldn’t switch to a take home or something else.
The resumes seemed like they had relevant experience. That’s why I pushed them forward to the tech screen.
That's worthwhile.
When I was fresh out of university, I had trouble finding a job. I had worked part-time at a small company through the last few semesters, but that company blew up just before I graduated. Every interviewer I ever met with was impressed because I really knew my stuff, but had trouble getting in the door due to an unimpressive CV and awkward mannerisms.
I did very well at Triplebyte, scoring in their top bucket and getting top marks in several of the sub-categories. Those metrics are relative to all engineers, which would put me even higher amongst recent graduates. I also did very well on every software project I worked on and scored top marks on several other objective tests, like the Major Field Test.
I was exactly the sort of candidate their system should have been able to help. I never even got an email out of it, let alone an interview.
Waste of time.
To be clear, I'm only addressing the comment about how people thinking highly of them are gullible. I had a great experience, twice, within a few months, and that included going through a time of huge uncertainty.
That being said, I was not happy to see each of the changes they've made over the last year or so, and this does not strike me a good direction for engineers. I suppose I have to agree that they are not doing this from a position of market leadership. Perhaps they were providing asymmetric value to engineers, and at the end of the day the engineers are not the ones paying for the service.
This is a very good insight. Unless candidates pay more than companies, the incentives are off.
One. The problem domain they are in is on its way out, and they've decided to defect instead of being the ones left holding the bag.
Two. They hate what they do for a living.
The first company to stop making buggy whips probably did not do it because they weren't selling any. The fact that they ceded market share to their top competitors probably sunk the barb in deeper for them: Yeah people are buying cars but our sales numbers are still going up so why should we change?
In tech we see companies all the time end up being complicit in the destruction of their own industry by reducing its relevance in some manner or other, and by exiting early they have more options of destination. If I jump to the same industry as a competitor a year later I just look like a copycat. While some copycats copy other people, only better, they are the exception to the rule. Most churn out uninspiring derivative work, and if they can't at least do it cheaper then they end up on the scrapheap of history.
At that point, you can either keep going with what you're doing (and run into an inevitable wall later on) or you can foresee the future problems and pivot before they become unmanageable (which is what we did). Relatively few startups find their final model from minute one.
Fortunately, most of the dirt thrown at recruiting teams is wrong in the first place. I don't know any hiring managers who make lowball offers that have a low chance of acceptance - the costs of the process, the risks of hiring someone who starts mad at you, and the value of winning someone who is actually worth hiring, are all way more important than some incremental cash saved.
Well, presumably the idea is that engineers, especially high quality ones, will be drawn to the recruiting platform that has more pro-engineer policies. And so if you avoid using it, you lose access to that subset of engineers.
It's still a chicken and egg problem, though, because engineers have little incentive to join until there's enough companies.
However, I think that even if they build an attractive, differentiated experience, the advantage is unstable. The problem is that "traditionally strong" applicants (those who already have positive ROI from existing processes) are disadvantaged by being exclusive to one platform. So, from a hiring perspective, TripleByte's differentiation in terms of unique resumes is going to be from people who are harder to assess.
I'm actually fine with that as a value case - we've had great success hiring from bootcamps. "We help you reach strong nontraditional applicants" is a great value proposition. But it's not the same as a position of power to dictate the hiring process. Maybe a better pitch would be: we run a process that helps great non-traditional talent and great non-traditional companies find each other.
And as a HM, I was never interesting in using them, and we continue to use our own internal recruiters or local recruiters who know the market.
Having worked at a startup that was trying to disrupt the hiring/recruiting market back in 2010. I'm curious to how effective they were....if at all.
Edit: update, maybe I'm super skeptical because of all their cheesy ads that were all over Reddit for so long?
On related matters. I know several companies that have pivoted out of the "hiring/recruiting" space because apparently that space is not easy to scale/automate.
My experience is different. I've been through the TB process three times, back in 2017 where they flew me to SF from Asia for Flexport, Mixpanel and two smaller startups, and then again this year where I went to the final stage with FB production engineering and TB itself. And in between as well i created yet another profile on the system after shelving prior ones for a fresh start. I have a non traditional background, (ie, an "outsider" developer, chem degree and self taught hobbyist programmer since young), i take my craft seriously, have a bunch of popular open source products and a successful low touch indie business. But i don't consider future success guaranteed so i like to keep talking to companies to find that great opportunity for me. I know my tech skills in some areas are not the best so i like to stay in the loop, stay up to date and relevant and stay in touch with what companies are looking for. By far the best hiring process experiences I've had have all been through the TB platform. And my "open rate" for applications and inbound interest on TB is way higher than any other methods I've tried (SO jobs, WWR, HN). The quality of companies, hiring process and level of interest I've received via TB has been epic and, at least in the past, even the TB support staff are super helpful going so far as to provide personalized feedback on profile improvements and resumes. I'm not an expert at getting hired but i couldn't have asked for anything more from them. They were fantastic. I only have great and wonderful memories of the time i spent in hiring processes of companies i "met" via TB. So this is my experience of it, and this is despite the fact that i have never ever received an offer from any company I have met on TB, including from TB itself. I don't blame them. I just consider my own skills are not at the right level, and the fit is not right for the companies I've met, while i work to improve myself and my skills. That's all. It's a simple and beautiful experience I've had with them. Soooo different to hiring through other platforms (where funnily enough i have recieved offers). I've met and talked to ammon and many other TB team members and they have all been cool, friendly, passionate people. From what I'm reading of this pivot, they're being vulnerable and open to learning, and using what they've learned so far to create something even better. I'm not gullible, I'm not saying they will succeed. But they're trying. And they're earnest and sincere. What more can you ask for? This has been my deep experience with them. I think the most vocal critics are talking merely theoretically without any experience of TB nor deep nor accurate understanding of who they are, what they do and what they stand for. To them i say, give it a try. Investigate before contempt! ;P ;) xx
I had some bad luck. My nodejs build broke after a recent update on my machine, which I didn't realize until right before the interview. During a test with a different language, I tried to define a constant with the same name as a built-in function and ran into a vague compiler error (something about missing parentheses, ugh). This language has case-insensitive function names to compound the confusion. I unfortunately looked up how to define a constant in the docs. Their conclusion was that "I was uncomfortable in the language," despite having used it for 10 years. There was some other feedback which I felt was inaccurate, I think I just had a bad day, and obviously didn't convey my knowledge and experience well. I could see why a recruiter would hard pass on me for some of the stumbles, since their main goal is to forward candidates who interview well. It hurt to get rejected.
I mean, you went into a combat scenario, and chose a weapon absolutely notorious for jamming.
Take a look at the question they just asked me when I was doing their General Coding Assessment:
The only thing that bugs me about that question is that "the output of the following function" is confusingly worded. It's the sort of question that a competent candidate might get nervous about and start to overthink: wait, that isn't a function, it's a code snippet. The function here is foo – so maybe they mean "what's the output of foo"? But then what does "the output of a function" mean? I suppose they mean return value? Is this a trick question to see if I remember what push returns? Damn it, does it return the entire array or just the pushed element?
That's what my mind usually does with questions like that and I'm far from the only one. Since the goal of the test is to screen for basic competence, it ought not to filter out people who could answer the question perfectly fine if it were being asked clearly, but who also perceive corner cases and ambiguities. Such a skill should make you more likely to pass such a test, not less. Therefore the question ought to say something like "What does the following code snippet write to the console?"
The timer is what is bad about this.
I do not care about the question.
As you say, any competent programmer could solve this in a few seconds, if they are proficient, or minutes if they are not.
In my opinion, they should not put a count down because they are interrupting the programmer’s flow. If they really want to measure time, they do it without interrupting the programmer, hide the timer, let them take as much time as they want/need to solve these problems. Then, at the end of the quiz, show them how long it took them to give a proper solution, and take in consideration that time to score them.
If Alice solved 10 problems in 10 minutes, and Bob solved the same problems in 15 minutes, maybe there is something there that is worth highlighting in their profiles. Maybe Alice is able to analyze this type of questions much faster than Bob, and if that matters to the recruiters or potential employers, then allow the candidate to use that in their favor.
However, if Bob actually knew the answer, but they ran out of time to select an option, that seems unfair.
You've found exactly what they are looking for.
Also, it's pretty bad manners to leak test questions imo. You probably agreed not to in the TOS that you accepted.
That's a tricky question, and if you're not good enough to understand why they're asking that then you fail!
a and b are shadowed in the foo function so the function only acts on the shadowed (and copied) variables. Once you return from foo, the const a and b are unchanged.
Not a great question as a pass or fail test imho because if you use shadowed names often enough to be able to parse this code in your head, you’re writing bad code. There is a reason why shadowing names is a bad practice: it’s hard to figure out what the end result is and account for side effects! But as a discussion on all the points above then it would teach the recruiter something about what the candidate knows rather than using this as a trivia question.
At least I assume that's how const works; I've not used const in JS.
There may be some edge cases I'm not covering here, but JavaScript passes primitives by value (strings, numbers, etc) and non-primitives (objects, arrays, etc) by passing a copy of the reference.
What this means is if you reassign the reference inside of the method, it will only affect that scope, because the reference itself is a copy. If you modify the properties of the non-primitive, it will be modified, because the copy of the reference points to the same non-primitive.
Since primitives are passed by value then any modifications are not reflected outside of the method.
To the other poster below, in JavaScript, const with an array (object, etc) simply prevents reassignment of the variable, the array can still be modified.
I used them two times. The first one was very early on, where I was given a home assignment and interviewed on it by Aaron himself, if I am not mistaken. The dude was awesome at interviewing, and knew exactly how to probe to get a better understanding of your skills.
That was when they were promising that you can interview with them so you don't have to do technical interviews with the companies. I thought it was an awesome concept and could really reshape hiring in the tech industry.
Second time was a couple of years ago, where the model has already changed a bit. Passed the first round and one of the companies that I could interview for was triplebyte themselves.
What a disappointment! The only difference in the interview process than the rest of the companies was that they gave you a laptop and asked you to do practical coding instead of whiteboard generic algorithm solving.
Some of the interviewers themselves were junior members of the stuff with 0 experience in interviewing.
is that they say they have companies like Apple and TrueCar posting job listings and hiring but they actually never respond and all you get is offers for companies between 10-200 people in size (aka, startups)
It's a great platform, the indicator letting me know companies aren't going to respond is great.
It just doesn't fix the problem that... the quality of companies on Triplebyte aren't companies most senior/lead/architect/advisory level engineers are willing to settle for. I'm being unnecessarily harsh/biased. I'm sure plenty of people use the platform with great success. I've just come to accept "early/mid-stage startups" job offers from Triplebyte and not much else. Not necessarily "garbage" but... for sure lots of risk.
The "super week" of onsite interviews was interesting, probably increased my overall interviewing skill, but was probably one of the more stressful individual weeks in my life.
Scheduled a practice interview and the only slot available was ~6AM my time. Nobody showed up and I wasn't informed that they wouldn't be showing up.
After reaching out about it -- they tried to tell me that my interview scored poorly, and that I would need to retry again in a few months. After some back-and-forth they realized both that it was practice interview and that the person didn't show up. So they rescheduled the practice interview.
When the interview did happen -- this was for ML stuff -- the interviewer was just no great. We spent so much time discussing the differences in terms we used -- mine largely coming from University, there's from I'm assuming their formal education -- that much of the interview was wasted. It literally came down to me deriving what we were not agreeing on for them to understand we were talking about the exact same thing.
I then had to reach out several times for my results, I'm assuming due to the fact that you're allotted one practice interview, and technically I had two(?). When I finally got my results I was again informed that my results were not good enough, and I'd need to wait to reapply. I gently informed them that it was practice interview, and the representative apologized their mistake.
When I reviewed my results... the interviewer didn't rate me too well, largely due to our differences in terminology. They also didn't like my coding style -- even though no one has ever complained to me before -- and rated the coding exercise poorly even though I was able to perform what was asked of me.
I just gave up.
Then several months later, I got an email about being TripleByte certified or whatever.
The whole thing was a really bizarre experience.
People hate overly abstract/CS-focused whiteboard problems (mostly for good reason), isn't practical coding the obvious better alternative?
> Triplebyte has hundreds of thousands of engineers on our platform, and that means we can flip the script on companies. The collective power of thousands of engineers is enough to change their incentives in a way that individual engineers cannot.
> We can change their incentives directly by rewarding or punishing certain behavior. For example, companies aren’t normally incentivized to provide salary and culture data. But we can force their hand by promoting transparent companies in our search rankings. When a company’s access to thousands of engineers is on the line, their incentives are very different. The same goes for honesty: a company often has no reason to be honest with any one engineer, but we can disincentivize lying by making their behavior with one engineer affect their access to the next.
So... a union? In digital, online format?
I'm glad that someone there is now realising the advantages of joining forces to negotiate workers' conditions. It was about time high-tech engineers noticed this.
Often enterprise HR is paranoid of honestly evaluating anyone for anything, to the point that many HR teams tell one another and engineering hiring managers that it’s “illegal” to assess candidate abilities on the way in the door, at all.
If you think about joining a dev team where no one checked if any candidate could FizzBuzz, you can imagine the workplace environment that can end up with.
Triplebyte is able to provide engineering managers with a stack of pre-vetted resumes so an enterprise can interview by its lowest common denominator HR policies, while still having a prayer a team will at least be made of candidates who can code.
The value of this to an enterprise stuck in this position is hard to overstate.
Hopefully along with expanding the talent pool per this post, Triplebyte can figure out a way to get well paid for this ability to help land actual coders on actual dev teams despite enterprise HR.
The need for this is huge.
// I see a comment below from a Triplebyte PM about score matching: https://news.ycombinator.com/item?id=27542621 … That would be slick: land candidates that genuinely raise the bar, but aren’t so far ahead as to be speaking a foreign language to the team. Of course, this would require also assessing the target team, and oops, we’re right back up against that HR policy…
There's an old quote: "Nobody ever got fired for buying IBM."
Large companies tend to be more risk-averse, so if there are two companies you can use for a service — $PromisingStartup and IBM — large companies will generally make a decision that's optimal to the specific decision-maker, not optimal to the company.
There are two options if you, as the buyer, choose either $PromisingStartup or IBM: success or failure.
If IBM or the $PromisingStartup succeed, then you've done your job.
If IBM fails, you can tell your manager "Who could've guessed! It's IBM!"
If $PromisingStartup fails, you'll have a harder time explaining your decision, and the fault will be with you.
The "Nobody ever got fired for choosing IBM" idea is useful to look at every decision large companies make, whether it's pivoting, choosing a SaaS product, or hiring.
TripleByte, in its current form, has been beneficial to candidates as well by giving enterprise employees a justifiable signal towards hiring them regardless of pedigree.
That sounds hilarious How could a dev team not be able to write 5 lines?
Its the same reason why a top notch, genious writer can turn out to be terrible at giving speeches, even if they're reading from a script.
Performing an interview is a different skill from writing code.
I think this is true for the day-long leetcode interviews, but to succeed as an engineer, even at the entry level, I think you need to be able to talk through how you would solve fizzbuzz. A big part of the job is communicating trade offs with other engineers.
I think if I could describe my personal experiences, I'd say that on a subconscious level, I know that how I speak and articulate details will be observed by the interviewer through a cultural lens more often than not. This may be unique to my personal background, so maybe this is different for other groups. But interviewers often get hung up on the use of different language/terminology, the types of jokes/jests that enter the conversation, how I respond to their language. And there's sometimes been an unwillingness to allow different terminology, perhaps because that's all the interviewer themselves knows and understands.
All that to say, a code interview doesn't always come down to the quality of the code.
Such an eye-opening experience! Now that I’m on the other side of the table these days, I understand the importance of making interviewees feel comfortable and asking questions that allow them to start off strong.
I got shitlisted at one place for being the only dev with the mathematical insight to know that multiplying pesos by a dollars:yen ratio doesn't yield a correct conversion of pesos to yen. They let that one slide without fixing it, but the final straw that got me frogmarched was refactoring an if-else to a switch.
They saw me as a god for fixing their shit and offered to hire me. No thanks!
From the position of a candidate this sounds like a poorly run company I'd have no interest in working for. I wouldn't want to interview with them.
At least good companies do this anyway.
Also, executive search recruiters kind-of do what you're describing. Exec leadership is a small-ish world, and the recruiting cycle for, say, a CTO can take months. However, there's a lot of commission on the other end of that. So recruiters basically act as brokers.
I much prefer the imperfect systems that at least pretend to be fair and objective-ish.
I have started to re-write my resume for each application and that has yielded much better results in terms of quality of the company and number of interviews.
If you are submitting the same resume to multiple companies, you are doing it wrong.
But glad you figured out an approach that worked for you!