The sooner we can fully automate competitive programming away from the standard interview process the sooner things will improve for all software devs, imho.
Maybe they would focus on asking us about test strategy, devops culture, release deadline commitment, quality vs deadline cutoff, business understanding, team integration ability, all the sort of things we actually do 90% of the time.
Uh, yeah they are. If I start a 60 hour (estimated) story the week before the sprint ends, I'm going to have some very pointed questions how thoroughly we want to test this new feature, and perhaps argue that it should be pushed into next sprint. Knowing how to formulate such issues to my management is a very typical part of my job.
If you work at a place where devs are just code monkeys who implement the orders from on high with no feedback whatsoever or avenue for pushback... get a better job.
You can absolutely puzzle out _a_ solution to that problem, it's not just a quiz. It's not even hard, given the context that you know that it's possible to do.
Steps, mostly driven by just basically knowing the goal and that there's not many operations that could possibly help:
"a" "b"
"a+b" "b"
"a+b" "-a"
"b" "-a"
"b" "a"
Then once you have that, you can enumerate the downsides to that, look for more efficient and less error-prone ways to proceed.
What irks me about this question is that it is almost utterly unrelated to any type of development task, save one: pipelining a repeated math equation in a long loop. And even then, I would bet the time it takes to swap two registers (or even L1 cache) using a third would far outperform the three operations needed because of the fact that they are sequential in nature (ie: they cannot be performed simultaneously by even an advanced CPU).
In nearly 30 years of diverse coding experience, I've never once encountered a situation where this solution would be useful.
Does not reflect on the job work to regurgitate algorithm answers and it's too easily gamed. A novice that has practiced algorithms will outperform an expert that has not studied recently.
That said, its probably more used as a filter for people interested enough in working there to study.
This seems like a flawed premise. It's an interview, there's nothing to be automated. It's a test, a task that the company wants you to do as a candidate, not a task that they need automated. They don't even need the task done, they aren't using the code you write to solve real world problems, they're using it to assess your suitability for the job.
To put it another way - they can already automate solving their leetcode problems by looking up the solution in their database, no need for AI. But that's not the point at all.
>They aren't using the code you write to solve real world problems, they're using it to assess your suitability for the job.
If they aren't evaluating how the code you write fits into the context of a real world problem, how can they possibly use it to asses your suitability for the job? Using fake code problems to evaluate candidates is the flawed premise here.
You're changing the topic - whether data structures and algorithms problems are suitable for interviews is separate question. My claim is that if you're using those types of interview questions, an AI model being able to solve them well makes no difference to your interview process because the interview isn't something that you want to automate.
Like I said, they were always able to automate solving those problems by doing a database lookup.
I am literally responding directly to what you wrote in the way that you wrote it. My counter claim to your initial point is that candidates will be able to use said AI in competitive programming interviews to fool the interviewer by the interviewee, thereby making competitive programming pointless as an interview tool. It automates it for the interviewee, not the interviewer.
I thought that you mean companies automating stuff. But it still doesn't make sense - interviewees have been able to cheat in online assessments forever. Does a better way to cheat in a small subset of all the leetcode interivews really make enough of a difference to make all of those interviews obsolete?
> It's a test, a task that the company wants you to do as a candidate, not a task that they need automated
You are contradicting your earlier statements. But yes, a tool that can beat leetcode interviews successfully and reliably is a game changer for candidates who want to make it through these pointless LC interviews as a whole.
You can do this, but you're optimizing for the wrong thing. And given enough time this will be seen as just as backwards as, today, asking someone to list the methods on the String class from memory.
Because there was a time we did this, and with tools like Google Search it became dumb, but it took some dinosaurs a really long time to let go of their old ways. Hell, some of them are probably still around.
To be fair, unless you're doing something absurd the length of variable names shouldn't significantly affect the number of lines. Also comments with typos aren't really a problem. I don't like hard-to-read code either, but the more years I work the more I think that the shortest (within reason) code is the best code.
That is to say, golfing is bad unless for fun, shorter code is usually better, the shitty hard to read code is probably a reflection of human laziness rather than misguided tersity.
Heh definitely an eyebrow-raising comment!
On a second read of it though, I took it more as "because minimizing the code written is a pattern observed in the wild, AlphaCode mimics it as that's what it learned from."
Almost half the screen real estate on my iPhone SE was white space. I had zero interest in reading something, especially something that long and in depth on a page that wastes so much space for literally nothing.
Typography, layout, and presentation counts for more than coders such as myself will ever admit publicly :)
This is incidental. The current Jupyter CSS needs to be improved - so what? That's not a bug unique to Jupyter, I've seen dozens of other websites with similar problems (some with extremely bare HTML and CSS that was minimal but bad - so the problem is unrelated to complexity, too).
This has nothing to do with the parent comment's topic, which was about using computational notebooks as blog posts, and is in effect "complaining about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage" - which the guidelines explicitly forbid.
I wrote a similar prompt but for a discord bot that would take a youtube url and some timestamps and return a gif.
Afterwards I asked it to add an option to "deep fry" the gif. Not only did it produce the correct code it also understood what I meant when referring to deep fried gifs. I was definitely impressed.
I threw a leetcode easy description into ChatGPT and submitted the solution without any editing. It passed all the test cases and got in the 95th percentile for efficiency of C++ submissions.
I haven't tried anything more advanced, but to go from simple requirements to solution without any clarification or even method signatures (It guessed the correct method signature down to the name and input params) was pretty dang impressive to me.
It's still impressive to me even if it was in its training set. The data used for training is not stored verbatim in the OpenAI model. It would still need to parse the context anew and solve the problem given its understanding of the boundaries of what I've written.
I have tried given a simple test suite and asking for an implementation that fulfills it, then asking to modify its impl and the test suite with a new requirement and it did it
I've been following the progress of AI code generation and I have to say, the results are impressive. There are still some limitations to these models, such as reproducing poor quality training data and difficulty maintaining focus throughout a problem.
I think using more diverse and high-quality training data could help address these issues, and incorporating additional constraints and regularization techniques into the model's training could prevent hallucinations and improve its overall reasoning abilities.
While there is room for improvement, the progress in this field is exciting and I can't wait to see where it will lead.
NB: This comment was written by GPT-3 after reading the article. The last few months of AI have been frankly mind-boggling.
Even if it were bulletproof I don't think it's necessarily a bad thing, it would probably make the average engineer more productive if most of their job was architecture, and the writing of individual bits of code was something automated.
It's a lot like the conversation not too long ago that AI was going to replace managers. Sure, some aspect of their work might be gone, but it didn't obviate the need for the human.
Since this sort of coding is exactly what FAANG interviews test for, maybe FAANG engineers will be some of the first coders automated out of a job. That would be an interesting irony.
I don't know Mr. Altman's rationale, but FAANG companies being first to automate away programmers sounds reasonable to me on the basis of scale alone. They have more code than most companies with which to make up a company- and problem-specific dataset, as well as the resources and expertise to train and deploy complex models.
The algorithm tests are there to have an excuse to discriminate on other criteria. If you can mask your racial/age/gender reason for not hiring as a FAANG with "oh, we didn't like their algorithm solution" you always have an out.
> FAANG is the only place where you don’t need to be white to succeed.
I feel like this "only" is erroneous. While FAANG do have solid diversity policies in place to assume they're the only companies capable diversity in their hiring practices is offensive to everyone else.
If you think that immigrants, particularly indian and Chinese, are not bringing their own oppressions from their homeland, you're wrong.
HN has the best thread for discussing the caste baste discrimination that happens among Indian tech workers in the USA. It turns out that being white still gives you a lot of privilege, even in tye FAANGs.
Each individual doesn't do much work. Surveys and friends I know both say they do about 2-4 hours of work a day (rest of the 6-8 hrs (yes, 10hr days are now normal it seems) is is hanging out & meetings).
No. FAANG interviews are mostly about filtering very large candidate pools efficiently and getting good enough engineers out at the end.
The job is much more architecture/design, creating API contracts, understanding the health of systems in many different ways, measuring impact of changes, etc. Regular engineering stuff at big scale. Obviously there’s plenty of coding too, but it’s not really the important part of the job, and already has a ton of automation for boilerplate.
I imagine that the automation will just improve another step change, there will be more need for review and guidance of the AI algorithms and the engineers will do more of this. And interviews will involve in some way.
I suspect that might be the case regardless. FAANGs have people who can deploy this sort of tech, who can solve the problems is generates (assuming the generated solutions aren't perfect), and their engineers are the most expensive. They have the most to gain. A software shop making small apps would benefit far less.
That said, its likely that the job of being a dev is going to be largely debugging generated code in the future no matter where you work.
I'm a FAANG interviewer and I've run my coding questions through ChatGPT.
The TL;DR is that code would easily pass a junior-level interview, and maybe a mid-level one. I'm definitely convinced these technologies will disrupt leetcode-style coding interviews.
So either we embrace it, or ditch this approach altogether.
Why would they when you could just google the answers to these questions the whole time. The point is to see if the candidate understands algorithms not to solve a novel problem.
Real work is more than making snippets of things. At my last employer, changes were requested every single day, in large and small ways, requiring insane amounts of reworking previous ideas in the codebase on a continuous basis without breaking everything. I wonder how well this AI would do on an entire application that wasn't predetermined. That might take much longer to achieve than solving small problems.
I imagine an AI in that kind of environment inventing Skynet and doing us all in.
If anything, I hope this is the start of the downfall of leetcode questions. They are utterly useless as a screening criteria and outside of people that have no idea how to proceed they don't offer much signal.
The most important thing I need from a programmer I work with is their ability to have a fruitful conversation with me about why the code might be written one way or another, and for them to iterate on the code in response to those conversations.
If a coder can write code, but they can’t do that, they’re useless to the org. It will be faster for me to write their code myself than to maintain what they’ve done.
So really that’s what I’d need from an AI coder. Writing the code is good, but can we talk about it and can you learn the specific architecture principles we have applie in this specific codebase.
It's true. For almost all of my career I've worked in the software equivalent of a lab building a diamond making robot. I did a stint at a consulting firm -- that was more like coal mining than manufacturing gemstones. By this I mean never ending surface area, and very little incentive to ever go back and refactor anything.
For most orgs, code isn't the end product. Code is the way to build the product to sell to the user.
Therefore, in most orgs, code architecture/style is a distant secondary to 'does it achieve what the user will pay money for' and 'why isn't it finished yesterday?'.
ChatGPT can actually do that sometimes. I've been using it as a rubber duck to help me debug code and it tells me what is wrong with what I wrote and how to fix it.
Yes, rubber duck debugging is self-talk so the task still sits with the programmer. However with the AI one get external answers which may or may be not correct, and could or could not influence one's decisions.
I can't speak for ChatGPT but the "AI" that I use for this sort of development, the `M-x doctor` feature of emacs, only makes pleasant imaginary arguments, mainly because it was written in 1966, but it is still useful sometimes and at least it doesn't make any incorrect assertions about code: https://en.wikipedia.org/wiki/ELIZA
You're making it seem like AI will replace coders. Do you think Dall-E will replace artists, or just make an artist's job different?
IMO it's more like a coder using AI to make their job easier. It's still up to a human to come up with the individual problem the function solves, architect a solution from multiple functions/objects/etc, come up with a data model, and so on and so on. The AI just generates the code itself. And at least as of now, the code needs to be double-checked.
No, but it is very likely that it reduces to demand for coders and artists.
Sure, there will always be demand for a Linus Torvalds or a Damien Hirst. But will there be demand for Coder #365968 at Infosys or a graphic designer pumping out $50 ad banners?
We’re looking at the possibility of some white collar jobs having the same income disparity as creative jobs. Just as there are some musicians who make hundreds of millions while the vast majority barely make ends meet, we may have a future where the star programmers make millions while the average players are automated out of the competition.
> No, but it is very likely that it reduces to demand for coders and artists
Eventually, I agree yes. But there could be a boom of huge new investments into A.I products, more devs needed and in fact teams getting way more requirements since they are more productive. Imagine the stuff we will be able to build in things like search, personal assistants, biomed, in fact what industry won't this affect? Its unbelievable to me that people are now saying Google search might become obsolete, that's absolutely crazy. Not many people saw that one coming.
But at least initially I don't think GPT models will be able to do everything themselves.
So its very hard to determine that say in the coming 5 years devs will find it more difficult to get a job.
10-20 years from now sure, I don't see how anyone gets a cognitive job anymore let alone devs.
In fact our entire school/university system is probably obsolete, kids are probably learning skills they won't be able to apply in any job market. We need to start think about stuff like teaching kids emotional intelligence, spirituality and meditation...not cramming for a math test.
>Do you think Dall-E will replace artists, or just make an artist's job different?
If an ad agency or a magazine publisher can get a custom illustration that works for my purposes from Dall-E, then they ain't paying no artist. Not theoritical, many already do use those generated images.
That's not just "making the artist's job easier". It's taking jobs from artists (well, illustrators and graphic designers at least), especially in the cheaper end of the business (e.g. not Nike, but your local Pet Store chain, restaurant, or news outlet, sure).
There's still a matter of taste that Dall-E can't provide. It will only spit out what you tell it to, and if you lack good taste, then you'll still produce something inferior
People will tire of DALLE eventually. We are great pattern matches, and we'll start to see the patterns that don't measure up. Then it'll be all about the next AI engine, and it'll need its own corpus of work. Who's going to create it?
So yes, not OP, but I think artists will still have jobs, although less of them, and the job will be different.
There is no indication that we don’t have enough images already available and training time is the main bottleneck. Also openai clear showed that DALL-E could generate avacado chair even though there is nothing like that in the dataset.
DallE, Midjourney, and StableDiffusion are already taking jobs. Illustrations, album covers, blog post images. People are making beautiful books and playing cards.
Midjourney V4 is amazing. It spits out absolutely beautiful images.
To be precise, AI is "taking jobs" that could have already been commoditized long ago if people in developing countries understood how Fiverr worked and had set up "art sweatshops" to serve demand.
There's enough art talent already around in the world to entirely commoditize the supply of it for the little one-off no-style-guide-to-follow commissioned works you're talking about. It's just not currently a liquid market — supply and demand find it hard to discover one-another — and so a true market-clearing price can't be set.
Meanwhile, AI is not currently taking anyone's advertising-campaign graphic design job, or anything else where the "efficient-market price" (in a world where human "art sweatshops" existed) would be more than $5.
How many of those people would have actually paid for an artist otherwise though? I myself am thinking of playing around with game dev for fun with the thought of using image AI to generate the art. Were it not for that, I'd just use free textures, or more likely, just spend my time doing something else entirely
> You're making it seem like AI will replace coders
I find it best to accept it will most likely replace all of us, from doctors to coders to even psychotherapists. Won't happen next year but 10-20 years is a very long time this thing keeps getting better. Eventually we won't be able to tell if its a machine or a brilliant superhuman.
The bummer in all of this, in my view, isn't the loss of jobs; we'll find what to do. It's the transition period - the accounting wizard or the brilliant doctor losing their jobs and status and becoming kindergarten teachers or care takers or unemployed. Nothing in their upbringing or life experience prepared them for such a thing ... so that's probably gonna be rough for many people. But once most people went through the transition it won't be bad. Society I believe will be better off. We will stop being obsessed with money and status and spend much more time with family and friends. Entertainment will be insanely good and so will healthcare. Possibly medicine to make our moods better.
It could be utopia.
> It's the transition period - the accounting wizard or the brilliant doctor losing their jobs and status and becoming kindergarten teachers or care takers or unemployed.
Yes, this is a big problem.
However... a lot of people would enjoy being teachers, albeit with significant improvements to the educational systems.
We can enjoy it if there's more of us. Each class can have 5 teachers instead of one teacher on 30 kids. Government can create those jobs, take the hundred of trillions created and redistribute it. Marx was right I think capitalism eventually kills itself ...we won't need it anymore. Arguably we already don't need the aggressive version we have now but soon enough it will be clear we don't need any version of it.
The means of production (AI, robots and land) will be transferred to the people who will all receive basic income, free services and (if they want) jobs created by the government for the greater good. I don't think its a dystopia.
> The means of production (AI, robots and land) will be transferred to the people who will all receive basic income, free services and (if they want) jobs created by the government for the greater good.
This is a prediction?
Given human nature and the diversity of people (w.r.t. rationality, religiosity, morality, capability, and so on), it is very much an open question about (a) how AI capabilities will develop; (b) how they will be paid for... (c) and by whom; (d) to whom will benefits accrue; (e) how will society change.
These are broad, sweeping questions. Plenty of fodder for imagination, hope, transformation, cynicism, backsliding, or even despair.
If I were to make a bet, on our current trajectory, I see some key factors in tension:
1. educational quality, in absolute terms, increasing _and_ being more equitable
2. educational quality, in relative terms, continuing to be very unequal and probably getting more so. As one example, who has the resources to direct computationally intensive AI experiments? There are (and probably will be for a long time) gatekeepers for these resources. People that mix in this circles have a huge advantage. This makes me wonder if "exclusivity leads to inequality" is a saying from some philosopher.
> But once most people went through the transition it won't be bad. Society I believe will be better off. We will stop being obsessed with money and status and spend much more time with family and friends. Entertainment will be insanely good and so will healthcare. Possibly medicine to make our moods better. It could be utopia.
First, I don't see "utopia" as likely; furthermore, I have a suspicion it may be impossible, given human nature.
Second, even the argument that society will be "better" demands much more reflection. The implied argument above is only a sketch. I don't find it convincing much less plausible. I'll call attention to four points (implied from above):
1. AI will replace humans in most or all professions
2. AI quality will be much higher than the previous human levels
3. A broad swath of people (using some notion of equity and fairness) will have enough money to live happily
4. "We'll spend much more time with family and friends"
Each of the four points are quite uncertain. Furthermore, even if `k` is true, `k+1` does not follow.
Who would like to flesh out some ways the sequence (1, 2, 3, 4) might happen?
Yes of course you are right, we don't know anything yet I agree. I am speculating a lot here.
But I'm a believer in "intelligence as a commodity" as Sam Altman put it after seeing GPT3.5 so I think points 1 and 2 will be reality. 3 follows quite naturally to me but only in Western societies... Putin will have different ideas.
Anyway speculating is fun but you're right its just speculating. My main point is we should always keep in mind this could turn out to be great .
_And_ if since we care about our AI-interdependent future, more of us (as in the people here on HN) need to wade into the gory details, including ethics and the current power structures. The "technology" (as in algorithms, data structures, hardware, etc) is arguably the "easy" part. There are plenty of existing incentives and structures to keep those _moving_. But moving in what direction? Even the notion of an "ethical compass" seems antiquated in light of current technology. We may have to reframe everything. This is a big challenge.
I'm not sure 3 follows. A walk downtown of any major city in America shows what society does for anyone who's work can't be commoditized or aren't sitting on an existing pile of wealth such that they don't need to work.
That's because people like you and me (I'm assuming you're somewhere in the comfortable middle class) vote in for this system. Because so far we've enjoyed a reasonable quality of life.
If that's no longer the case, we can vote for other leaders and other systems.
Currently, the economics don't stack up. How does society handle it when the best jobs are automated? When all jobs are automated?
The Star Trek post-scarcity utopia scenario feels very unlikely; Mad Max-style scrapping for leftovers while Musk, Bezos, et. al. live behind walls feels infinitely more probable.
How do you implement UBI when a huge proportion of the political class is vehemently against it. Maybe we need to AI politicians, so they start to figure it out?
I don't have anything figured out but I think we can do it as a society. I do think we have a very negative biased view of the ultra rich. Zuckerberg, Gates and Buffett all give or will give all of their wealth to society. So not all of them are the same. I think Musk will probably also give most back eventually though I'm not sure he has said anything yet.
And remember, we are still a democracy. We get to vote. We control the army and the police and all institutions. If we decide that this capitalism isn't hot sh* anymore we can change it. What will the evil billionaires do? (this sounds like a good straight to DVD movie actually...hey GPT write me a script about this)
> We will stop being obsessed with money and status and spend much more time with family and friends. Entertainment will be insanely good and so will healthcare.
As nice as this would be, I think there's roughly 0% chance of it happening. Over the past few millenia humans have doubled productivity per capita a ridiculous number of times. None of those leaps led to an end of status seeking or a transition towards mostly leasure time for the masses.
Instead I expect more of the opposite from these developments. Power will get increasingly concentrated with people who have very little interest in the needs and wants of the plebs.
> Power will get increasingly concentrated with people who have very little interest in the needs and wants of the plebs.
At least in OpenAI's case I think they take this thing very seriously (Sam Altman doesn't strike me as evil one bit, quite the opposite in fact https://www.youtube.com/watch?v=DEvbDq6BOVM). In fact most of the tech elites don't seem evil to me, if they only cared about money Zuckerberg and Gates wouldn't have pledged away all their wealth.
Some of them are as you describe but I think most of them are actually somewhere in the progressive axis.
Not quite. The argument is that our tech elites aren't so evil to try to violently take over and that us masses can simply vote a more (much more) socialist system.
It's up to us. Indeed easier said than done in the current dysfunctional and polarized politics of ours but we can still do it.
We have pretty different views on tech elites. I don't think they are evil per se, but most are greedy and generally lack empathy. It's almost impossible to go from mildly wealthy to multi-billionaire status without some aggressive wealth accumulation.
For the philanthropy, I'll just note that these pledges don't involve literally transferring 99% of wealth out of their control. The vast majority of the pledged money goes to a trust the person controls, organizations they have some relation to, or just stay completely in their control for years with only vague non-binding commitments to eventually donate it. In return for this largess they get significant reputational and tax compensation.
... people still freaking out machines will replace someone, since what, industrialization 200+ years ago? People still work, just maybe different jobs or more intellectual jobs.
It will replace coders with thinkers. The number of people that have ideas good or bad is large compared to the people that can implement those ideas as these bots/ai get better they will produce a lot of code.
At the start it will probably increase the amount of code produced and the number of coders but with time as the ai gets better need for coders will decrease. We will need people that can think or imagine ideas rather than coders now these people might still be considered software developers but they will not be coders in strictest sense of the word.
ChatGPT can already do that. You can ask questions about the code, make suggestions and it will take this into account and write improved code. You can tell it the code it wrote produces an error, and it will then find the error, explain what it did wrong and fix it.
I did this and it is pretty inconsistent. It kept telling me I was using the wrong version of a library (which from what I could tell by looking at documentation was just not correct, but I didn't look into it too long), and at one point it just kept insisting that it's code solution was correct when in reality the error was due to it importing two different libraries that use the same namespace, which led to an error saying it couldn't find the function being called.
What I've found is each iteration with ChatGPT is alright at fixing or adding to the existing code per my instruction but every iteration includes a significant chance of introducing a new bug or simply forgetting some piece of the existing code in the next iteration.
>The most important thing I need from a programmer I work with is their ability to have a fruitful conversation with me about why the code might be written one way or another, and for them to iterate on the code in response to those conversations. If a coder can write code, but they can’t do that, they’re useless to the org.
Don't worry, you wont be there at the org to make these "fruitful conversations" either. The AI will take your job too
> Don't worry, you wont be there at the org to make these "fruitful conversations" either. The AI will take your job too
I've been waiting for something to take my job for 15 years.
I started as a small developer testing radio firmware, moved on to test web firmware, now I instruct terraform how to build infrastructure and now I instruct developers on how to do things to build proper infrastructure.
I'm ready to retire but apparently I'm incapable of having an AI that can actually simulate Super Power ADHD at work, so we'll have to wait a bit.
I have observed that companies will choose not to invest further in irreplaceable labor, under any circumstance. Every single company with, that I have been part of over the decades (30ish?), exhibits the same behavior when it becomes common knowledge that someone is a linchpin. Put the company in a game of chicken for a raise (which includes an implicit, or-I-quit) and they let the "irreplaceable" employee go. Every single time. It's not hard to see why. No company wants to have a disgruntled linchpin nor do they want to be beholden to some lower level character.
My experience is similar, though I would personally attribute it at least partly to stupidity - on the one hand, in most cases when I have worked with someone crucial, they've been with the company longer than the people managing them, so the person or people making the firing decision don't really understand what's at stake. Or to put it another way, their cost-benefit is flawed, they know what it'd cost to keep the key employee, but they fail to imagine what it will cost to let them go.
> My experience is similar, though I would personally attribute it at least partly to stupidity
Every company has suffered from the decision, in my experience. Never has it destroyed the employer, but I have heard stories about such eventualities.
Much as I'd like to side with labor here, the companies are probably right. Even if the linchpin is high level and happy, building around a single, irreplaceable employee is not a sound long term strategy.
For all the important contributions Steve Jobs made to Apple when he returned, maybe the least heralded and hardest to implement is that he managed to NOT make himself irreplaceable. To many people's surprise, Apple did not collapse after his retirement and death. So maybe his most genius contribution was not to make Apple dependent on his ongoing genius contributions.
> So maybe his most genius contribution was not to make Apple dependent on his ongoing genius contributions.
i wonder if we all acted in this way, would things be better than they are? do we as individuals put our need/want to be depended upon above what is (for lack of a better term) the long-term good?
Pretty sure there are managers who don’t know WTF you talking about and will be happy to address your inability to work with the new “hire” by letting you go.
This wont fly at organizations that need the code to work, every time, but think about the explosion of non-programmers who can now make systems that “basically work”. If you don’t think “basically works” is a high enough bar to succeed in e-commerce, let me show you my recent support email threads with companies from whom I’ve been trying to purchase Xmas gifts for my wife.
I couldn’t agree more. Most of my job consists of gathering requirements and brokering deals between affected departments. I think most college students dream of the type of job that could feasibly be replaced by AI. In reality, the coding is the easiest part of the job and takes the least amount of time.
But this thing is not a sub for coders. It is an assistant to coders. And yes, being able to explain the code is incredibly important. Just as it is to verify the code.
To me, this is just another exercise where coder becomes manager of his very own coder. And has to check the code his coder produced.
The objective of this website is to provide a platform for customers to order and purchase chocolate donuts online. The website will offer a variety of chocolate donut flavors and allow customers to customize their orders by choosing toppings and packaging options.
Functional Requirements:
Customers should be able to create an account and log in to the website.
Customers should be able to browse and view the available chocolate donut flavors, toppings, and packaging options.
Customers should be able to add chocolate donuts and toppings to their cart and specify the quantity of each item.
Customers should be able to view their cart and checkout to place their order.
The website should process payment and provide confirmation of the order.
Customers should be able to track the status of their order and receive updates on the expected delivery time.
Non-Functional Requirements:
The website should be secure and protect customer information.
The website should be easy to use and navigate.
The website should be accessible on a range of devices, including desktop computers, laptops, and mobile devices.
The website should load quickly and be responsive to user actions.
The website should have a clean and professional design.
User Roles and Permissions:
Customers: Customers will be able to create an account, log in, browse and order chocolate donuts, track their order status, and access their account information.
Admin: The admin will be responsible for managing the website, including adding and removing products, processing orders, and managing customer information.
User Interface Design:
The website will have a clean and professional design, with a navigation menu at the top of the page and a search bar for customers to easily find specific products.
The home page will feature a carousel of images showcasing the available chocolate donut flavors and toppings.
The product pages will display images and descriptions of the chocolate donuts, along with options for toppings and packaging.
The cart page will display the items in the customer's cart, along with the total price and a checkout button.
The checkout page will allow customers to enter their shipping and payment information, review their order, and place their order.
The order confirmation page will display the details of the customer's order and provide an estimated delivery time.
The account page will allow customers to view and update their account information, including their shipping and payment details.
Database Design:
The website will have a database to store customer information, including their name, email address, shipping and payment details, and order history.
The database will also store information on the available chocolate donut flavors, toppings, and packaging options.
The database will keep track of the current orders and their status, including the expected delivery time.
Technical Requirements:
The website will be built using HTML, CSS, and JavaScript for the front-end, and a server-side language such as PHP or Ruby for the back-end.
The website will use a database management system such as MySQL to store and manage the data.
The website will be hosted on a web server and accessed through a domain name.
The website will use a secure socket layer (SSL) to encrypt data and protect customer information.
Testing:
The website will be tested to ensure that it meets the functional and non-functional requirements.
User acceptance testing will be performed to ensure that the website is easy to use and navigate.
The website will be tested for compatibility on different devices
I love how the sheer simplicity of this comment manages to convey so much about humanity and our drive to do things we love. I feel a bit silly reacting like this, but it's truly beautiful.
Not many get paid to do it, and those that do are paid because it's entertainment, people like watching them play. Nobody is paying to watch developers code.
Automation doesn't have to outperform the best of us, it only has to outperform the worst of us to have a serious impact. Outperforming a 50th percentile coder is a huge deal.
> Rather than copying and pasting sections of previous training code, AlphaCode came up with clever snippets without copying large chunks of code or logic in its “reading material.”
> When challenged with the CodeContest—the battle rap torment of competitive programming—the AI solved about 30 percent of the problems, while beating half the human competition.
This was essentially a competition in the speed of programming. But if we want to discuss practical application, as in laying off armies of coders, we need to realise that there is a tremendous gap in the productivity of, let's say a solo startup founder and the productivity of a team of coders working for a multinational behemoth barely producing anything of a value over the whole sprint.
We're nowhere even remotely close to that. I have to assume that anyone reacting on this level are in the same group of people who 10 years ago believed we would have fully self driving cars by now.
This isn't the same question though. The ML models that were trying to beat Go were on the correct path, it was just a matter of improvement. It was inevitable that they would beat the game.
ChatGPT doesn't logically reason about code or business contexts. The developers of ChatGPT are not trying to do that, the developers don't know HOW to do that, not even slightly.
Until they're on that pathway it's not even a question: ML that very efficiently copies and pastes from StackOverflow is simply not going to significantly replace programming jobs. The question of how long is left until the AI replaces programmers is completely irrelevant right now.
Using Go/Chess analogy:
> If the game is to replace effective human programmers then the AI is not even playing the game yet, let alone X years away from beating it.
They beat the game hardly two years later, in early 2016.
I'm sure every programmer has their finger crossed that the long tail is very long. And not that I think programmers will be out of jobs, probably not, but their value as a prompt technician will be substantially less than as a software engineer.
I don't know - chatGPT is the first time that I've consistently used an AI product in my workflow, to the point that when the site was rate limited today, I felt a little paralyzed. Googling for answers on Stackoverflow or looking up documentation felt distinctively primitive.
All this is doing is parsing StackOverflow and other sources and compiling information using machine learning (sometimes completely incorrectly), and if that's saving you time because you do that a lot then that's genuinely great, it's an amazing tool. But the idea that a tool that scrapes StackOverflow is going to significantly replace programming jobs is frankly ludicrous.
Everything that ChatGPT does can be done manually. There are weak programmers who essentially piece things together from StackOverflow and other sources all day, and just brute force "make it work". Did the emergence of StackOverflow replace programming jobs? No way, there are actually far more programming jobs now then there were when StackOverflow
was founded. The reason is simple:
> If you want to create complex production software and grow and maintain it long term, it's simply not enough to copy and paste from the internet.
So therefore a machine that does exactly the same thing can't either. Go and build a complex ERP system using only ChatGPT answer, just try it. It will be a steaming pile of shit, riddled with technical debt. You won't be able to build a good business from the result.
Right now even if you use ChatGPT you inevitably need to be a developer to fix it's errors, so it's currently replacing exactly ZERO developers.
When we have AI that can logically reason about code, along with the business context it's being applied to, then it will significantly replace programming jobs. We're absolutely nowhere near that, ChatGPT doesn't even sniff anywhere near doing that.
Personally, I can't even remember the last time I went on StackOverflow to find answers about something, and there's enough false positives where reading documentation is far preferable to asking ChatGPT about how something works. So currently for me ChatGPT doesn't save any time, I do use Copilot though.
I've conducted hundreds of FAANG-level coding interviews, and recently tried to run my interview questions through ChatGPT. The model was able to spit out a correct, optimal, clean Python solution effortlessly.
Interestingly when I asked a follow up, a harder version of the same problem, ChatGPT spits out code that sort-of looks correct, but is actually nonsense.
So, would it pass the interview? That's difficult to say. Interviews don't happen in a vacuum, you also consider the candidate's thought process, explanations, alternatives, tradeoffs...
Still, I see this is a game changer for this kind of interviews. So long as candidates understand and explain the output code, there is a good chance they would clear the interview. Even if the code is incorrect, it might given them some hints towards the right solution.
So where do we go from here? I always loathed this interview format, and these languages model reinforce it even further. Interview cheating has always been there, but it is generally so rare that it isn't a real concern. However these tools are too effective and easy to use. I can see a real divide between people who use them and people who doesn't. This type of interview might become even more useless at telling good coders apart.
My take is we have 3 choices:
a) Ignore it.
b) Try to fight it.
c) Embrace it.
a) is not an option. b) would make an already pretty dreadful process even more intolerable. My money is on c)
I can envision an interview format where we allow, or even encourage people to use ChatGPT and AlphaCode during the interview, much like you would use your IDE or a search engine. In fact seeing how a candidate understands and uses those code snippets can be a very interesting data point.
Either that or scrap leetcode-style interviews altogether.
P.S.: I was thinking about writing a blog post about this, if people think it'd be interesting.
In my experience (with FAANGs at least) the interviews have been virtual since the pandemic began. Haven't heard of any plans to return to return to flying out candidates for in-person interviews.
Like in mathematics or other tests students are not allowed to use the internet or an advanced calculator in order to test wether they truly comprehend the stuff.
If you think that your FAANG-interviews are any good, then just keep them in the format you already have, by making sure applicants cannot use AI during the test. I would have on-sites with pen+paper, whiteboard or a prepped/supervised machine, whatever.
Of course, applicants could use AI to train for the interview, but that is not a problem, as long as you test their comprehension.
Given the trends towards remote tech jobs, it could be difficult to convince your companies engineers to show up to the office for what’s usually just a weed out leet code interview.
Proctored tests are common for certification exams. Perhaps there's a future where candidates go to one of these exam centers for one of their technical interviews.
I'm not a fan of making the process more dreadful than it already is. Forcing people to go to an office and use pen and paper or a whiteboard would do exactly that.
I hope companies (including mine!) see the writing on the wall and stop trying to fight the future.
This isn't accounting for even just pretending we are testing candidates on anything remotely indicative of on-the-job performance.
I think developers who don't will be rare in a few years time. Just like developers who primarily rely on assemblers (versus compilers) have basically become extinct. Like we sometimes, very rarely, need to inline some assembly, we will sometimes need to use the ol' gray matter to figure out a novel algorithm or something.
I believe that avoiding learning these tools could be a existential issue for your present-day job. You don't have to come to depend on them, or use them daily, but you do need to understand how best to use (and not use) them.
The real issue with any of these AI innovations is they really point out places where humans have already started to behave like an AI in the first place.
Ever since the emergence of leetcode style interviews I've been shocked at how many people can reproduce leetcode examples, but still fundamentally have no sense of algorithm design outside of the context of a job interview.
Programmers with their sights set on acing a FAANG interview will just keep repeating leetcode problems until they start to memorize the common patterns (not the problems themselves of course, but the structure of these type of problems). What's disturbing to me is that I recall far more interesting discussion about algorithms in the era before leetcode dominated everything.
The common solution isn't to understand algorithms better, but to become a leetcode solving robot.
So it's no surprise to me that AI can pretty easily replicate humans that have tried to turn themselves into robots.
We see similar patterns in the art that AI can create. It's very good at replicating a kind of art style of designers trying to turn themselves into design robots.
Absolutely spot on. I actually do algorithm design, usually over a period of weeks (at least), and leet code is a joke for the serious algorist (I’m sure I’d fail an interview based on it). Nothing has so clearly illustrated the robotic nature of the leet code expert quite like this result has.
Mathematical Biology/Bioinformatics. We have to think very carefully about every step in the process of extracting information from large, diverse datasets--often writing things from scratch, combining/transforming things in novel ways, and implementing new mathematical ideas efficiently enough to be computable on large datasets.
A big if. A lot of ChatGPT discussions seem to take for granted that it'll always be available/free/priced low enough that ~everyone has access to it. Seems more likely that at some point OpenAI will close it up and put it back behind an API.
The company itself could have access to the AI for the interview, since idea is lowering the playing field.
But even more, it seems likely someone is going to be willing to supply LLM access at not that much more than the cost of computation.
I know OpenAI's business is selling access to their stuff at a premium but since current AI is not much more than brute-forcing of massive public data, it doesn't seem like this premium will be justifiable for long.
This is already the present for translators. They often are called to proofread machine translated text. And this is often more difficult than rewriting the whole text.
That seems like it's likely been trained on various examples of FAANG questions that are posted to the likes of leetcode, with solutions often presented. The push for harder version was clever, and that it fell over is no real surprise.
This is why I ask questions in a business domain. It requires the programmer to think not only about solving a straightforward clear problem and only worry about Big-O. Instead they need to figure out the problem by asking thoughtful questions about possible business concerns and think about which ones to optimize for.
Ambiguity that requires follow up questions for successful isn't going to be addressed by something focused on solving a problem that "thinks" it has all the information to solve the problem.
Myself, I've only conducted dozens not hundreds of FAANG-style coding interviews. I also loathe the format, and it does indeed wear on me. I hate every second.
Damn. What is your success rate? A lot of times it seems to mee that luck and your ability to withstand stress are one of the main factors. Is it something you can teach yourself?
If by success rate you mean inclined rate, it varies a lot by level. I'd say it's around 30% for interns (which is by far more lenient) and closer to 5% for Senior level. However it's also common to downlevel senior candidates rather than downright reject.
From my side of the table I can confirm that of course luck plays a role. Nevermind the variability of the coding problems themselves, but the variability on the interviewers. Even in my company, that has a formal process and pretense of objectivity, interviewers quality and expectations vary greatly.
In short, I'd say what people on the hiring side hate to hear, but interviewing is ultimately a numbers game. I wouldn't take rejections too seriously. The best you can do is to improve your chances with proper prep (leetcode, system design) and interview broadly.
The illusion of objectivity is strong. We have an official interviewing process. It has a big survey that turns into "metrics". We have a debrief where we talk about the "metrics" as though it was an "evidence based" process. Then you'll hear someone say "Well, he found a solution for the balanced tree problem, but he just didn't seem all that confident doing it, you know?"
> Then you'll hear someone say "Well, he found a solution for the balanced tree problem, but he just didn't seem all that confident doing it, you know?"
This sounds like yet more evidence that it's really just pledging a frat.
I hate leetcode style interviews, but I've done hundreds of them. The irony is not lost on me. Some might say I'm part of the problem.
Thing is, in a small way, I'm trying to change things from within. I try to make the process as palatable and fair to candidates as possible, while working within the constraints of the system. When I train new interviewers some of my top tips are:
a) The purpose of the interview is to determine whether the candidate is a good fit for the company, and the company is a good fit for the candidate.
b) The interview is an imperfect proxy for this.
It then follows that interviews shouldn't overindex in the coding round. If I have helped someone to get an offer that wouldn't otherwise, I'm satisfied.
BTW that's also one of the reasons I'm publishing this. I hope to push the point that leetcode-style coding interviews are outdated and should be burnt to the ground.
> Interview cheating has always been there, but it is generally so rare that it isn't a real concern.
You're the person they're trying to fool, so if they cheat successfully you would never know. You've never seen overt displays of bad cheating, which is different.
Think you haven't seen a stick insect in years? Likely you have, but just didn't notice it...
Cheaters will always be more motivated than those trying to detect them - because everything is on the line for them.
>You're the person they're trying to fool, so if they cheat successfully you would never know. You've never seen overt displays of bad cheating, which is different.
I don't think it's widespread at least, since in my experience people that do well in technical zoom interviews do not drastically decrease in apparent competence when we move to in person rounds.
People definitely get told interview questions by recruiters though, if you count that as cheating then it is everywhere.
i think you're correct about option c, primarily because we could never force everyone to adopt the other 2 options.
option c makes me nervous, but right now I can't see an ai correctly dealing with the ambiguity of the data sets i typically look at.i do a lot of "asking for clarification."
I get the same results from asking for code. Near 100% success with an initial demo, and then pretty tragic performance when making a series of changes to actually fit the concept. I got it to write 3d rotation code, and to switch to using rotors instead of quaternions but when I asked it to rewrite the combine_rotors() method it rewrote it to combine Enigma rotors. I explained its error and it apologized but just wasn't able to go back to the original code and work with it anymore.
Interviewing is going to be shaken up but I think some methods are more timeless. I've been giving the subject code instead of asking them to write it for a while now. Largely because I wanted to talk more and watch them type less.
We start by discussing the coding challenge guidelines and I leave them vague. They need to understand the goal and what I've left out and suggest those guidelines themselves. Once we agree though, I give them the code. "Here's what's running now."
Then I update the goal and we discuss changes. I get them to "whiteboard" certain things, like what a query looks like with their proposed changes or whatever, and we discuss big-O, etc. This, imho, is how whiteboarding is actually used - not to write whole programs but to provide examples and pick them apart.
I feel that this would work even if they were using an AI in another window. We're trying to select for developers with common sense and domain knowledge, and who can clearly discuss engineering tradeoffs. Actually making the changes (the coding itself) was a big part of the job and that's decreasing, but imho all the other requirements remain. They'll still need to know how to handle the issues I talked about above, of trying to get the model to write the right code!
Algorithm interview and competitive programming type questions are probably some of the easiest for GPT to solve because there are a massive number of problems and solutions publicly available for training.
The real benefit of AI is somewhat shown in this paper, it effectively solved the problems through brute force generating millions of possible solutions. For real world problems it would be interesting to let GPT generate a bunch of different solutions and push them into a test environment and see which works best.
The biggest problem I see is black swan events where AI coded systems work great until something goes wrong and no human truly knows how all the pieces fit together.
I expect that the reason it gave the correct answer to your first question is simply that it already saw the problem and memorized the solution - there is some empirical evidence that deep neural networks are able to memorize much of their training data.
Possibly. The problem itself is not in leetcode, but it almost certainly has been leaked somewhere. However the program was able to make a few changes to the code with some prompting, which hints a little bit more smarts than just regurgitating an answer.
Nevertheless the point is moot. I've invented completely novel questions (promise!), and saw them leaked online after asking them twice. The process is fundamentally flawed and large language models are just making that glaringly obvious.
Our coding exercise cannot be solved by ChatGPT and I believe is much more effective at evaluating coding ability than leetcode-style questions. We ask candidates to design an object-oriented booking system which requires 3 or 4 different classes to implement. ChatGPT cannot easily do this without heavy prompting from the user at which point, they'd be better off just writing their design down themselves. We want to evaluate candidates based on the kind of work they will actually be doing - not brainteasing O(n) solutions to contrived algorithm questions and so far it's worked very well.
I agree. Our interview has 3 coding rounds, and one of them I call "clean code". For that I ask a straightforward question that requires candidate to define some APIs and write some classes / functions. Then I ask several follow ups, adding or changing requirements.
This is by far my favourite question, the one closer to on-the-job coding. It also lends itself well to deep conversations with candidates.
But alas I don't own the process and still have to work within the parameters of the company. Whenever possible I ask this kind of questions, but other interviewers will default to leetcode-style rounds.
EE? I think plumbing and prostitution is all there will be left soon...and I'm really horrible with tools and not that good looking so unemployment it is...
I spent extensive time on chatGPT. It does give an answer, but it isnt the most optimal, sometimes non-compiled version of them. Its suggestions havent reached the level of creativity. Example, if I give it a system design problem of a real-world problem. It suggests to use Apache Kafka, but what if Kafka isnt sufficient for whatever reason?
E.g, Lai–Yang algorithm will give you no hits,
Fascination observation, I asked for Project Euler #193 (a problem I solved using mobius function). chatGPT solved it using bruteforce, even after I asked for the most efficient way it could solve it with. It just used memorization, which wasnt enough to find an answer for that problem quick enough. I asked whether it could use the mobius function, it couldnt translate it to code, and I had to give it python code to make it work.
- If you ask in-depth technical questions on how task queueing works within Elasticsearch, it wont be able to give you answer.
- George Hotz in Lex friedmen interview mentions that GPT-3 has 100 most recent messages as a limit. He isnt convinced that is enough to completely build a complex tool like Solve FSD or implement me a kafka.
-Lai Yang, pretty obscure, it doesn't have a Wikipedia page and I can't find much info on it anywhere, so this shouldn't be surprising.
-Euler #193 - This problem is not super obvious and I see discussions on it invoking Riemann zeta. ChatGPT was able to create a correct Mobius function on its first try. I have no idea how this might be related to the problem though.
-Elasticsearch - Why on earth would you expect it to know this?
You've cherry picked some super niche stuff that this brand new AI can't do and name dropped some fancy theorems. We're all very impressed.
I'm confused - how much has changed since February? The DeepMind blog post [0] seems to suggest not much. When I saw this headline on the front page I thought there were some major advances, again, but it seems not?
I really love do coding in python and always asked myself when the skill set will become legacy like COBOL.
Not sure if you could compare it, but it seems that the next gen coders will say to the AI what to code rather than to code by themselves. A complete different skill set.
I wouldn't say it's a completely different skill set.
You still want to have people, who actually understand what a piece of code or program does.
A magical black box to throw prompts at, might be nice for simple settings, but potentially can cause major fuck ups for complex systems.
Yeah sure, I just mean that it is like the step from COBOL -> JAVA -> Python in terms of productivity and the amount of code you write. So my point is, you would write much much less code. So maybe your future "code" would be just some sets of well placed comments and the AI does the rest. The same way you could test the system to prevent any major fuck ups. Sure you have to undestand it, but your core skill wouldn't be coding any more.
> the DeepMind team built a custom dataset from CodeContests from two previous datasets, with over 13,500 challenges. Each came with an explanation of the task at hand, and multiple potential solutions across multiple languages. The result is a massive library of training data tailored to the challenge at hand.
Isn't that exactly how the vast majority of engineers pass interviews today? They study up on the handful of algorithms that come up in interviews and then (with varying levels of acting) recite the answer to the question they knew they would get. What? Overfittings only bad in if you're a robot huh?
I hope we will get some AI for static analysis. It would be nice to hover on a symbol and get high level description of what it does, what pattern it follows, etc. AI can access not only current state of the code, but it's entire history, which no human would have time for.
556 comments
[ 1.9 ms ] story [ 300 ms ] threadIf you work at a place where devs are just code monkeys who implement the orders from on high with no feedback whatsoever or avenue for pushback... get a better job.
How would you swap two integers without using a temporary variable? Seen it before? Pass. Not seen it before? Fail.
Steps, mostly driven by just basically knowing the goal and that there's not many operations that could possibly help:
"a" "b"
"a+b" "b"
"a+b" "-a"
"b" "-a"
"b" "a"
Then once you have that, you can enumerate the downsides to that, look for more efficient and less error-prone ways to proceed.
We were looking for the solution based on XOR which works with only two registers.
We wish you luck with your job search elsewhere.
These interview questions are fairly shit, but they're not _that_ shit, and the people giving them are doing their best.
In nearly 30 years of diverse coding experience, I've never once encountered a situation where this solution would be useful.
That said, its probably more used as a filter for people interested enough in working there to study.
To put it another way - they can already automate solving their leetcode problems by looking up the solution in their database, no need for AI. But that's not the point at all.
If they aren't evaluating how the code you write fits into the context of a real world problem, how can they possibly use it to asses your suitability for the job? Using fake code problems to evaluate candidates is the flawed premise here.
Like I said, they were always able to automate solving those problems by doing a database lookup.
You are contradicting your earlier statements. But yes, a tool that can beat leetcode interviews successfully and reliably is a game changer for candidates who want to make it through these pointless LC interviews as a whole.
Because there was a time we did this, and with tools like Google Search it became dumb, but it took some dinosaurs a really long time to let go of their old ways. Hell, some of them are probably still around.
Peter Norvig: This is why AlphaCode learned to write code with one-letter variable names, and with no comments or docstrings.
!!
That is to say, golfing is bad unless for fun, shorter code is usually better, the shitty hard to read code is probably a reflection of human laziness rather than misguided tersity.
I might be terribly wrong though. :-D
Typography, layout, and presentation counts for more than coders such as myself will ever admit publicly :)
This has nothing to do with the parent comment's topic, which was about using computational notebooks as blog posts, and is in effect "complaining about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage" - which the guidelines explicitly forbid.
1. Write a static file server in Go
2. Write Go code to convert Color image to B/W
For both I got results. I know both are simple but still it's fascinating that AIs can write code. I have written more about it here https://rohanrd.xyz/posts/surprising-capability-of-ai-code-g...
Afterwards I asked it to add an option to "deep fry" the gif. Not only did it produce the correct code it also understood what I meant when referring to deep fried gifs. I was definitely impressed.
I haven't tried anything more advanced, but to go from simple requirements to solution without any clarification or even method signatures (It guessed the correct method signature down to the name and input params) was pretty dang impressive to me.
I think using more diverse and high-quality training data could help address these issues, and incorporating additional constraints and regularization techniques into the model's training could prevent hallucinations and improve its overall reasoning abilities.
While there is room for improvement, the progress in this field is exciting and I can't wait to see where it will lead.
NB: This comment was written by GPT-3 after reading the article. The last few months of AI have been frankly mind-boggling.
It's a lot like the conversation not too long ago that AI was going to replace managers. Sure, some aspect of their work might be gone, but it didn't obviate the need for the human.
I feel like this "only" is erroneous. While FAANG do have solid diversity policies in place to assume they're the only companies capable diversity in their hiring practices is offensive to everyone else.
HN has the best thread for discussing the caste baste discrimination that happens among Indian tech workers in the USA. It turns out that being white still gives you a lot of privilege, even in tye FAANGs.
The job is much more architecture/design, creating API contracts, understanding the health of systems in many different ways, measuring impact of changes, etc. Regular engineering stuff at big scale. Obviously there’s plenty of coding too, but it’s not really the important part of the job, and already has a ton of automation for boilerplate.
I imagine that the automation will just improve another step change, there will be more need for review and guidance of the AI algorithms and the engineers will do more of this. And interviews will involve in some way.
That said, its likely that the job of being a dev is going to be largely debugging generated code in the future no matter where you work.
I think this is an important point.
> That said, its likely that the job of being a dev is going to be largely debugging generated code in the future no matter where you work.
This and formulating the right prompts, i think.
I'm a FAANG interviewer and I've run my coding questions through ChatGPT.
The TL;DR is that code would easily pass a junior-level interview, and maybe a mid-level one. I'm definitely convinced these technologies will disrupt leetcode-style coding interviews.
So either we embrace it, or ditch this approach altogether.
I imagine an AI in that kind of environment inventing Skynet and doing us all in.
If a coder can write code, but they can’t do that, they’re useless to the org. It will be faster for me to write their code myself than to maintain what they’ve done.
So really that’s what I’d need from an AI coder. Writing the code is good, but can we talk about it and can you learn the specific architecture principles we have applie in this specific codebase.
Therefore, in most orgs, code architecture/style is a distant secondary to 'does it achieve what the user will pay money for' and 'why isn't it finished yesterday?'.
The point is to stimulate brainstorming, not get answers.
IMO it's more like a coder using AI to make their job easier. It's still up to a human to come up with the individual problem the function solves, architect a solution from multiple functions/objects/etc, come up with a data model, and so on and so on. The AI just generates the code itself. And at least as of now, the code needs to be double-checked.
Sure, there will always be demand for a Linus Torvalds or a Damien Hirst. But will there be demand for Coder #365968 at Infosys or a graphic designer pumping out $50 ad banners?
We’re looking at the possibility of some white collar jobs having the same income disparity as creative jobs. Just as there are some musicians who make hundreds of millions while the vast majority barely make ends meet, we may have a future where the star programmers make millions while the average players are automated out of the competition.
Eventually, I agree yes. But there could be a boom of huge new investments into A.I products, more devs needed and in fact teams getting way more requirements since they are more productive. Imagine the stuff we will be able to build in things like search, personal assistants, biomed, in fact what industry won't this affect? Its unbelievable to me that people are now saying Google search might become obsolete, that's absolutely crazy. Not many people saw that one coming. But at least initially I don't think GPT models will be able to do everything themselves. So its very hard to determine that say in the coming 5 years devs will find it more difficult to get a job. 10-20 years from now sure, I don't see how anyone gets a cognitive job anymore let alone devs. In fact our entire school/university system is probably obsolete, kids are probably learning skills they won't be able to apply in any job market. We need to start think about stuff like teaching kids emotional intelligence, spirituality and meditation...not cramming for a math test.
If an ad agency or a magazine publisher can get a custom illustration that works for my purposes from Dall-E, then they ain't paying no artist. Not theoritical, many already do use those generated images.
That's not just "making the artist's job easier". It's taking jobs from artists (well, illustrators and graphic designers at least), especially in the cheaper end of the business (e.g. not Nike, but your local Pet Store chain, restaurant, or news outlet, sure).
So yes, not OP, but I think artists will still have jobs, although less of them, and the job will be different.
Midjourney V4 is amazing. It spits out absolutely beautiful images.
There's enough art talent already around in the world to entirely commoditize the supply of it for the little one-off no-style-guide-to-follow commissioned works you're talking about. It's just not currently a liquid market — supply and demand find it hard to discover one-another — and so a true market-clearing price can't be set.
Meanwhile, AI is not currently taking anyone's advertising-campaign graphic design job, or anything else where the "efficient-market price" (in a world where human "art sweatshops" existed) would be more than $5.
I find it best to accept it will most likely replace all of us, from doctors to coders to even psychotherapists. Won't happen next year but 10-20 years is a very long time this thing keeps getting better. Eventually we won't be able to tell if its a machine or a brilliant superhuman. The bummer in all of this, in my view, isn't the loss of jobs; we'll find what to do. It's the transition period - the accounting wizard or the brilliant doctor losing their jobs and status and becoming kindergarten teachers or care takers or unemployed. Nothing in their upbringing or life experience prepared them for such a thing ... so that's probably gonna be rough for many people. But once most people went through the transition it won't be bad. Society I believe will be better off. We will stop being obsessed with money and status and spend much more time with family and friends. Entertainment will be insanely good and so will healthcare. Possibly medicine to make our moods better. It could be utopia.
Yes, this is a big problem.
However... a lot of people would enjoy being teachers, albeit with significant improvements to the educational systems.
I doubt very much that this is a testable theory. I think it is primarily a normative one.
This is a prediction?
Given human nature and the diversity of people (w.r.t. rationality, religiosity, morality, capability, and so on), it is very much an open question about (a) how AI capabilities will develop; (b) how they will be paid for... (c) and by whom; (d) to whom will benefits accrue; (e) how will society change.
These are broad, sweeping questions. Plenty of fodder for imagination, hope, transformation, cynicism, backsliding, or even despair.
If I were to make a bet, on our current trajectory, I see some key factors in tension:
1. educational quality, in absolute terms, increasing _and_ being more equitable
2. educational quality, in relative terms, continuing to be very unequal and probably getting more so. As one example, who has the resources to direct computationally intensive AI experiments? There are (and probably will be for a long time) gatekeepers for these resources. People that mix in this circles have a huge advantage. This makes me wonder if "exclusivity leads to inequality" is a saying from some philosopher.
First, I don't see "utopia" as likely; furthermore, I have a suspicion it may be impossible, given human nature.
Second, even the argument that society will be "better" demands much more reflection. The implied argument above is only a sketch. I don't find it convincing much less plausible. I'll call attention to four points (implied from above):
1. AI will replace humans in most or all professions
2. AI quality will be much higher than the previous human levels
3. A broad swath of people (using some notion of equity and fairness) will have enough money to live happily
4. "We'll spend much more time with family and friends"
Each of the four points are quite uncertain. Furthermore, even if `k` is true, `k+1` does not follow.
Who would like to flesh out some ways the sequence (1, 2, 3, 4) might happen?
Anyway speculating is fun but you're right its just speculating. My main point is we should always keep in mind this could turn out to be great .
_And_ if since we care about our AI-interdependent future, more of us (as in the people here on HN) need to wade into the gory details, including ethics and the current power structures. The "technology" (as in algorithms, data structures, hardware, etc) is arguably the "easy" part. There are plenty of existing incentives and structures to keep those _moving_. But moving in what direction? Even the notion of an "ethical compass" seems antiquated in light of current technology. We may have to reframe everything. This is a big challenge.
They get nothing but tents and shame.
https://www.bbc.com/news/blogs-echochambers-27074746
Where are the examples of the middle voting against the extremes when times get really hard?
The Star Trek post-scarcity utopia scenario feels very unlikely; Mad Max-style scrapping for leftovers while Musk, Bezos, et. al. live behind walls feels infinitely more probable.
How do you implement UBI when a huge proportion of the political class is vehemently against it. Maybe we need to AI politicians, so they start to figure it out?
And remember, we are still a democracy. We get to vote. We control the army and the police and all institutions. If we decide that this capitalism isn't hot sh* anymore we can change it. What will the evil billionaires do? (this sounds like a good straight to DVD movie actually...hey GPT write me a script about this)
As nice as this would be, I think there's roughly 0% chance of it happening. Over the past few millenia humans have doubled productivity per capita a ridiculous number of times. None of those leaps led to an end of status seeking or a transition towards mostly leasure time for the masses.
Instead I expect more of the opposite from these developments. Power will get increasingly concentrated with people who have very little interest in the needs and wants of the plebs.
At least in OpenAI's case I think they take this thing very seriously (Sam Altman doesn't strike me as evil one bit, quite the opposite in fact https://www.youtube.com/watch?v=DEvbDq6BOVM). In fact most of the tech elites don't seem evil to me, if they only cared about money Zuckerberg and Gates wouldn't have pledged away all their wealth. Some of them are as you describe but I think most of them are actually somewhere in the progressive axis.
So far it hasn't worked out too well...
It's up to us. Indeed easier said than done in the current dysfunctional and polarized politics of ours but we can still do it.
For the philanthropy, I'll just note that these pledges don't involve literally transferring 99% of wealth out of their control. The vast majority of the pledged money goes to a trust the person controls, organizations they have some relation to, or just stay completely in their control for years with only vague non-binding commitments to eventually donate it. In return for this largess they get significant reputational and tax compensation.
If you don't understand the program ChatGPT wrote, it will happily butcher it for you, because it doesn't really understand it either.
Don't worry, you wont be there at the org to make these "fruitful conversations" either. The AI will take your job too
I've been waiting for something to take my job for 15 years.
I started as a small developer testing radio firmware, moved on to test web firmware, now I instruct terraform how to build infrastructure and now I instruct developers on how to do things to build proper infrastructure.
I'm ready to retire but apparently I'm incapable of having an AI that can actually simulate Super Power ADHD at work, so we'll have to wait a bit.
Every company has suffered from the decision, in my experience. Never has it destroyed the employer, but I have heard stories about such eventualities.
For all the important contributions Steve Jobs made to Apple when he returned, maybe the least heralded and hardest to implement is that he managed to NOT make himself irreplaceable. To many people's surprise, Apple did not collapse after his retirement and death. So maybe his most genius contribution was not to make Apple dependent on his ongoing genius contributions.
That's because its even lower level characters in managements deciding...
This wont fly at organizations that need the code to work, every time, but think about the explosion of non-programmers who can now make systems that “basically work”. If you don’t think “basically works” is a high enough bar to succeed in e-commerce, let me show you my recent support email threads with companies from whom I’ve been trying to purchase Xmas gifts for my wife.
But this thing is not a sub for coders. It is an assistant to coders. And yes, being able to explain the code is incredibly important. Just as it is to verify the code.
To me, this is just another exercise where coder becomes manager of his very own coder. And has to check the code his coder produced.
My job is safe I see.
Title: Online Website for Chocolate Donuts
Objective:
The objective of this website is to provide a platform for customers to order and purchase chocolate donuts online. The website will offer a variety of chocolate donut flavors and allow customers to customize their orders by choosing toppings and packaging options.
Functional Requirements:
Customers should be able to create an account and log in to the website. Customers should be able to browse and view the available chocolate donut flavors, toppings, and packaging options. Customers should be able to add chocolate donuts and toppings to their cart and specify the quantity of each item. Customers should be able to view their cart and checkout to place their order. The website should process payment and provide confirmation of the order. Customers should be able to track the status of their order and receive updates on the expected delivery time.
Non-Functional Requirements:
The website should be secure and protect customer information. The website should be easy to use and navigate. The website should be accessible on a range of devices, including desktop computers, laptops, and mobile devices. The website should load quickly and be responsive to user actions. The website should have a clean and professional design.
User Roles and Permissions:
Customers: Customers will be able to create an account, log in, browse and order chocolate donuts, track their order status, and access their account information. Admin: The admin will be responsible for managing the website, including adding and removing products, processing orders, and managing customer information.
User Interface Design:
The website will have a clean and professional design, with a navigation menu at the top of the page and a search bar for customers to easily find specific products. The home page will feature a carousel of images showcasing the available chocolate donut flavors and toppings. The product pages will display images and descriptions of the chocolate donuts, along with options for toppings and packaging. The cart page will display the items in the customer's cart, along with the total price and a checkout button. The checkout page will allow customers to enter their shipping and payment information, review their order, and place their order. The order confirmation page will display the details of the customer's order and provide an estimated delivery time. The account page will allow customers to view and update their account information, including their shipping and payment details.
Database Design:
The website will have a database to store customer information, including their name, email address, shipping and payment details, and order history. The database will also store information on the available chocolate donut flavors, toppings, and packaging options. The database will keep track of the current orders and their status, including the expected delivery time.
Technical Requirements:
The website will be built using HTML, CSS, and JavaScript for the front-end, and a server-side language such as PHP or Ruby for the back-end. The website will use a database management system such as MySQL to store and manage the data. The website will be hosted on a web server and accessed through a domain name. The website will use a secure socket layer (SSL) to encrypt data and protect customer information.
Testing:
The website will be tested to ensure that it meets the functional and non-functional requirements. User acceptance testing will be performed to ensure that the website is easy to use and navigate. The website will be tested for compatibility on different devices
Actual text: "AI just trounced roughly 50 percent of human coders"
I think we already all knew that about half of the coders out there weren't all that great.
"Think of how stupid the average person is, and realize half of them are stupider than that."
https://youtu.be/AKN1Q5SjbeI?t=19
“Oh big deal”
Now they never lose to any human in any chess game ever.
There is a big question though of how this translates to real world programming. Competitive programming and real world work are not the same thing.
Gave me a good chuckle lol
This was essentially a competition in the speed of programming. But if we want to discuss practical application, as in laying off armies of coders, we need to realise that there is a tremendous gap in the productivity of, let's say a solo startup founder and the productivity of a team of coders working for a multinational behemoth barely producing anything of a value over the whole sprint.
This isn't the same question though. The ML models that were trying to beat Go were on the correct path, it was just a matter of improvement. It was inevitable that they would beat the game.
ChatGPT doesn't logically reason about code or business contexts. The developers of ChatGPT are not trying to do that, the developers don't know HOW to do that, not even slightly.
Until they're on that pathway it's not even a question: ML that very efficiently copies and pastes from StackOverflow is simply not going to significantly replace programming jobs. The question of how long is left until the AI replaces programmers is completely irrelevant right now.
Using Go/Chess analogy:
> If the game is to replace effective human programmers then the AI is not even playing the game yet, let alone X years away from beating it.
I'm sure every programmer has their finger crossed that the long tail is very long. And not that I think programmers will be out of jobs, probably not, but their value as a prompt technician will be substantially less than as a software engineer.
I've never felt that way about any tool.
Everything that ChatGPT does can be done manually. There are weak programmers who essentially piece things together from StackOverflow and other sources all day, and just brute force "make it work". Did the emergence of StackOverflow replace programming jobs? No way, there are actually far more programming jobs now then there were when StackOverflow was founded. The reason is simple:
> If you want to create complex production software and grow and maintain it long term, it's simply not enough to copy and paste from the internet.
So therefore a machine that does exactly the same thing can't either. Go and build a complex ERP system using only ChatGPT answer, just try it. It will be a steaming pile of shit, riddled with technical debt. You won't be able to build a good business from the result.
Right now even if you use ChatGPT you inevitably need to be a developer to fix it's errors, so it's currently replacing exactly ZERO developers.
When we have AI that can logically reason about code, along with the business context it's being applied to, then it will significantly replace programming jobs. We're absolutely nowhere near that, ChatGPT doesn't even sniff anywhere near doing that.
Personally, I can't even remember the last time I went on StackOverflow to find answers about something, and there's enough false positives where reading documentation is far preferable to asking ChatGPT about how something works. So currently for me ChatGPT doesn't save any time, I do use Copilot though.
[0]: https://www.kaggle.com/competitions/abstraction-and-reasonin...
Interestingly when I asked a follow up, a harder version of the same problem, ChatGPT spits out code that sort-of looks correct, but is actually nonsense.
So, would it pass the interview? That's difficult to say. Interviews don't happen in a vacuum, you also consider the candidate's thought process, explanations, alternatives, tradeoffs...
Still, I see this is a game changer for this kind of interviews. So long as candidates understand and explain the output code, there is a good chance they would clear the interview. Even if the code is incorrect, it might given them some hints towards the right solution.
So where do we go from here? I always loathed this interview format, and these languages model reinforce it even further. Interview cheating has always been there, but it is generally so rare that it isn't a real concern. However these tools are too effective and easy to use. I can see a real divide between people who use them and people who doesn't. This type of interview might become even more useless at telling good coders apart.
My take is we have 3 choices:
a) Ignore it.
b) Try to fight it.
c) Embrace it.
a) is not an option. b) would make an already pretty dreadful process even more intolerable. My money is on c)
I can envision an interview format where we allow, or even encourage people to use ChatGPT and AlphaCode during the interview, much like you would use your IDE or a search engine. In fact seeing how a candidate understands and uses those code snippets can be a very interesting data point.
Either that or scrap leetcode-style interviews altogether.
P.S.: I was thinking about writing a blog post about this, if people think it'd be interesting.
Like in mathematics or other tests students are not allowed to use the internet or an advanced calculator in order to test wether they truly comprehend the stuff.
If you think that your FAANG-interviews are any good, then just keep them in the format you already have, by making sure applicants cannot use AI during the test. I would have on-sites with pen+paper, whiteboard or a prepped/supervised machine, whatever.
Of course, applicants could use AI to train for the interview, but that is not a problem, as long as you test their comprehension.
My money is on "hurt".
I hope companies (including mine!) see the writing on the wall and stop trying to fight the future.
This isn't accounting for even just pretending we are testing candidates on anything remotely indicative of on-the-job performance.
In a few years, ChatGPT will be conducting these interviews.
Agreed.
I think developers who don't will be rare in a few years time. Just like developers who primarily rely on assemblers (versus compilers) have basically become extinct. Like we sometimes, very rarely, need to inline some assembly, we will sometimes need to use the ol' gray matter to figure out a novel algorithm or something.
I believe that avoiding learning these tools could be a existential issue for your present-day job. You don't have to come to depend on them, or use them daily, but you do need to understand how best to use (and not use) them.
Ever since the emergence of leetcode style interviews I've been shocked at how many people can reproduce leetcode examples, but still fundamentally have no sense of algorithm design outside of the context of a job interview.
Programmers with their sights set on acing a FAANG interview will just keep repeating leetcode problems until they start to memorize the common patterns (not the problems themselves of course, but the structure of these type of problems). What's disturbing to me is that I recall far more interesting discussion about algorithms in the era before leetcode dominated everything.
The common solution isn't to understand algorithms better, but to become a leetcode solving robot.
So it's no surprise to me that AI can pretty easily replicate humans that have tried to turn themselves into robots.
We see similar patterns in the art that AI can create. It's very good at replicating a kind of art style of designers trying to turn themselves into design robots.
What sort of job leads you do design algorithms?
asking "here's a chatgpt solution to the problem... what's wrong with it?" would be a solid process imo.
But even more, it seems likely someone is going to be willing to supply LLM access at not that much more than the cost of computation.
I know OpenAI's business is selling access to their stuff at a premium but since current AI is not much more than brute-forcing of massive public data, it doesn't seem like this premium will be justifiable for long.
Particularly, ChatGPT can give apparently correct solutions that are wrong in subtle ways.
Also reading, understanding, reasoning about, and fixing code other's wrote is way closer to on-the-job performance.
Ambiguity that requires follow up questions for successful isn't going to be addressed by something focused on solving a problem that "thinks" it has all the information to solve the problem.
That's impressive perseverance. Did that wear on you?
From my side of the table I can confirm that of course luck plays a role. Nevermind the variability of the coding problems themselves, but the variability on the interviewers. Even in my company, that has a formal process and pretense of objectivity, interviewers quality and expectations vary greatly.
In short, I'd say what people on the hiring side hate to hear, but interviewing is ultimately a numbers game. I wouldn't take rejections too seriously. The best you can do is to improve your chances with proper prep (leetcode, system design) and interview broadly.
This sounds like yet more evidence that it's really just pledging a frat.
Thing is, in a small way, I'm trying to change things from within. I try to make the process as palatable and fair to candidates as possible, while working within the constraints of the system. When I train new interviewers some of my top tips are:
a) The purpose of the interview is to determine whether the candidate is a good fit for the company, and the company is a good fit for the candidate. b) The interview is an imperfect proxy for this.
It then follows that interviews shouldn't overindex in the coding round. If I have helped someone to get an offer that wouldn't otherwise, I'm satisfied.
BTW that's also one of the reasons I'm publishing this. I hope to push the point that leetcode-style coding interviews are outdated and should be burnt to the ground.
You're the person they're trying to fool, so if they cheat successfully you would never know. You've never seen overt displays of bad cheating, which is different.
Think you haven't seen a stick insect in years? Likely you have, but just didn't notice it...
Cheaters will always be more motivated than those trying to detect them - because everything is on the line for them.
I don't think it's widespread at least, since in my experience people that do well in technical zoom interviews do not drastically decrease in apparent competence when we move to in person rounds.
People definitely get told interview questions by recruiters though, if you count that as cheating then it is everywhere.
option c makes me nervous, but right now I can't see an ai correctly dealing with the ambiguity of the data sets i typically look at.i do a lot of "asking for clarification."
Interviewing is going to be shaken up but I think some methods are more timeless. I've been giving the subject code instead of asking them to write it for a while now. Largely because I wanted to talk more and watch them type less.
We start by discussing the coding challenge guidelines and I leave them vague. They need to understand the goal and what I've left out and suggest those guidelines themselves. Once we agree though, I give them the code. "Here's what's running now."
Then I update the goal and we discuss changes. I get them to "whiteboard" certain things, like what a query looks like with their proposed changes or whatever, and we discuss big-O, etc. This, imho, is how whiteboarding is actually used - not to write whole programs but to provide examples and pick them apart.
I feel that this would work even if they were using an AI in another window. We're trying to select for developers with common sense and domain knowledge, and who can clearly discuss engineering tradeoffs. Actually making the changes (the coding itself) was a big part of the job and that's decreasing, but imho all the other requirements remain. They'll still need to know how to handle the issues I talked about above, of trying to get the model to write the right code!
The real benefit of AI is somewhat shown in this paper, it effectively solved the problems through brute force generating millions of possible solutions. For real world problems it would be interesting to let GPT generate a bunch of different solutions and push them into a test environment and see which works best.
The biggest problem I see is black swan events where AI coded systems work great until something goes wrong and no human truly knows how all the pieces fit together.
Nevertheless the point is moot. I've invented completely novel questions (promise!), and saw them leaked online after asking them twice. The process is fundamentally flawed and large language models are just making that glaringly obvious.
That's great! For now. But tomorrow's coming fast.
This is by far my favourite question, the one closer to on-the-job coding. It also lends itself well to deep conversations with candidates.
But alas I don't own the process and still have to work within the parameters of the company. Whenever possible I ask this kind of questions, but other interviewers will default to leetcode-style rounds.
https://www.deepmind.com/blog/competitive-programming-with-a...
which walks through an example
E.g, Lai–Yang algorithm will give you no hits,
Fascination observation, I asked for Project Euler #193 (a problem I solved using mobius function). chatGPT solved it using bruteforce, even after I asked for the most efficient way it could solve it with. It just used memorization, which wasnt enough to find an answer for that problem quick enough. I asked whether it could use the mobius function, it couldnt translate it to code, and I had to give it python code to make it work.
- If you ask in-depth technical questions on how task queueing works within Elasticsearch, it wont be able to give you answer.
- George Hotz in Lex friedmen interview mentions that GPT-3 has 100 most recent messages as a limit. He isnt convinced that is enough to completely build a complex tool like Solve FSD or implement me a kafka.
-Euler #193 - This problem is not super obvious and I see discussions on it invoking Riemann zeta. ChatGPT was able to create a correct Mobius function on its first try. I have no idea how this might be related to the problem though.
-Elasticsearch - Why on earth would you expect it to know this?
You've cherry picked some super niche stuff that this brand new AI can't do and name dropped some fancy theorems. We're all very impressed.
[0] https://www.deepmind.com/blog/competitive-programming-with-a...
[0]: https://www.deepmind.com/blog/competitive-programming-with-a...
Not sure if you could compare it, but it seems that the next gen coders will say to the AI what to code rather than to code by themselves. A complete different skill set.
You still want to have people, who actually understand what a piece of code or program does. A magical black box to throw prompts at, might be nice for simple settings, but potentially can cause major fuck ups for complex systems.
Isn't this just over-fitting the model?