I would also note that the paper only covers the multiple choice portion of a bar exam, the Uniform Bar Examination (UBE) that has been adopted by most, but not all states. The UBE consists of a multiple choice portion (the Multistate Bar Exam, or MBE), an essay portion and a scenario-based performance test. GPT-3.5 gets a 50% success rate on a practice version of the MBE. It's impressive, but I wouldn't go so far as to say that it's imminent that AIs will threaten lawyers.
The success rate is not what’s impressive in itself. It’s the speed of improving on the test. We can expect it to get to 90% soon, and that will have real world impact (replacing lawyers for easy advice questions).
I won't be surprised, but that's less than "can expect", and I disagree that this is straightforward to forecast… unless you're currently playing with another similar model that hasn't been published and which can do this. As the saying goes, "forecasting is hard, especially if it's about the future".
AI progress has always been this weird combination of two sides, one saying for every breakthrough "this is just around the corner", the other saying "this is impossible".
This even happens anachronistically, with some people convinced AI can already do things they can't, and others that they could never do things they already do.
Generally when AI gets good on a specific task, it doesn't stop.
What's more common though is that even though it gets good on that task, it may not translate to practical real world application.
The best example I can think of is object detection vs self driving: lot of people though that the improvements in object detection on images will easily translate to great self driving, and here we are, still with cars not stopping when a car is blinking in front of it.
Completely agree with this. There's a short sentence in the abstract:
> hyperparameter optimization and prompt engineering
Prompt engineering seems a lot like "tweaking the question format until the AI gets the answer" which is the first lesson in ANNs; don't train on your test set. If you can't put the actual question with the only context being "this question concerns US law" then there's an awful lot of reasoning and thinking that the human is doing which the AI cannot.
Let's not have another Moore's law fallacy, it's not reasonable to extrapolate progress based on existing results. It's like building a car which can go at 200mph then saying 300mph is just around the corner.
If you're alluding to the notion "AI will take ALL the jobs!", well, here I am, still building websites for well established companies that still don't (in 2023!!!) have a web presence. Hell, half the worlds banking system still runs on COBOL!
Leading edge tech development is rapid, sure. But tech understanding, adoption, and implementation is painfully slow. Humans gon' be humans.
It's like the same thing every 5 years now, last time it was crypto "end of inequality!" "bank the poor!" "same global currency!" sorry to be skeptical, but I didn't see the quantum leap with crypto, and I don't see the quantum leap with GPT - what are essentially fancy NLP models
I don’t really know to what extent AI is going to take over. But crypto is a Terri comparison, it didn’t really solve any problems. There’s nothing you can do with a blockchain that’s you couldn’t already do before.
The potential with Chatgpt if it can be refined is wild.
- I can send someone 1 million USDC for 0.000001 US cent with confirmation in <1 second.
- I can send any twitter user address money by tweeting (eg. Send 100 USDC to @randomuser)
Couple examples that have come up recently for me personally. I can agree that there is something comparable (i.e. international wire) but the cost and speed is in a different ballpark and international wires are not available to some parts of the world.
> I can send someone 1 million USDC for 0.000001 US cent with confirmation in <1 second
This is clearly a very big problem for the majority of the world population!
It's like saying tesla solved the problem of going from 0 to 60 in 3 second in a 2t+ car, it's neato but virtually nobody had that problem, if you're not into illegal activities you will most likely never need or want to transfer 1m usd outside of a safe environment anyways
> The potential with Chatgpt if it can be refined is wild.
potential for good or bad ? Last time they tried to use AI for police/justice it ended up exacerbating existing biases.
> There’s nothing you can do with a blockchain that’s you couldn’t already do before.
What can you do with an AI you can't with humans in a court of law ? When a human is wrong you get a single judicial error, when the only software used is wrong (bug, bias, &c.) you'll get thousands every day.
What happens if an update makes the same case get a better defence and as an outcome a lesser sentence ? do you retrial all the other similar cases ?
The only potential I see is saving money and this is certainly not on the top of the list of things I would like the justice system to focus on
> potential for good or bad ? Last time they tried to use AI for police/justice it ended up exacerbating existing biases
Both. OpenAI are trying to be politically neutral, but it's not hard to break the rails, and someone else can and will replicate the work without those protections anyway.
> What can you do with an AI you can't with humans in a court of law ?
Me personally? Afford more than a fairly small case.
> What happens if an update makes the same case get a better defence and as an outcome a lesser sentence ? do you retrial all the other similar cases ?
That's actually a great idea. The estimated cost of each query is cents, so we might very well be able to retry everyone on each update, for less than the cost of keeping them imprisoned.
> That's actually a great idea. The estimated cost of each query is cents, so we might very well be able to retry everyone on each update, for less than the cost of keeping them imprisoned.
You're thinking like a developer and you're absolutely blind when it comes to the lives and families potentially destroyed by something like that. And that's exactly why technocrats should rarely be in charge
"ah really sorry dude, we released you last year but now you're on death row, wait for the next update, if it comes before you're fried on the chair you might get a lesser sentence. The last dude forgot to uncomment a branch in a switch/case and you were flagged incorrectly. Sorry for the inconvenience xoxo"
> Me personally? Afford more than a fairly small case.
> "ah really sorry dude, we released you last year but now you're on death row, wait for the next update, if it comes before you're fried on the chair you might get a lesser sentence. The last dude forgot to uncomment a branch in a switch/case and you were flagged incorrectly. Sorry for the inconvenience xoxo"
That's the exact opposite of the scenario you previously described, as you were previously saying people could get better defences not getting better prosecutions.
I'm not legally trained so this might just be Hollywood logic, but doesn't re-prosecuting someone require a court case with new evidence?
Anyway…
> That's a regulation issues, not an AI issue.
No, it's a "you're hiring at contractor rates" problem.
Just as Stable Diffusion is giving a hard time to professional artists who earn at contractor rates by producing "it might not be amazing but it's certainly not bad" art for a dozen images per cent of electricity, ChatGPT is giving "it might BS me 5-20% of the time depending on the subject, but it's still better than I know" responses on basically everything.
It can't do every job at an expert level yet, but it's better than almost everyone's baseline for the skills they are non-expert at. (No idea how long it will take to get to superhuman even just for law, and I was over-optimistic about self-driving car AI, but we shall see).
> The potential with Chatgpt if it can be refined is wild.
But this is exactly my point. I likely saw in 2018 word for word "The potential with blockchain technology if it can be refined is wild." Then everyone slowly realized - as you mentioned - "there's nothing you can do with a blockchain that you couldn't already to before". Same with chatGPT, it's just regurgitating what we collective know as a species. I just don't see how, if ever, any sort of AI system (even granted geometric improvement blah blah blah) is going to take over my daily type of work.
Oh it won't take all the jobs, it'll just make your life and your job a bigger hell than they already are.
There's so many creative ways in how an arbitrary and empathyless AI can make someone's profit a bit bigger and your life a bit worse :)
We can start with practically unlimited amount of personalised advertising spam that can be directed at you at any point. Articles, phone calls, billboards, conversations with your partners... so many chances to cheaply increase profits and engagements.
The AI takeover isn't going to be evil in Skynet sense... it'll be a constant stream of lowkey bullshit on every aspect of your life you won't be able to fight because "computer said so". Think on how Google randomly bans people from YouTube/Play Store/Drive over AI and put this into every aspect of your life.
I agree that adoption tends to be slow and there’s a lot of legacy systems that will need humans to look after them for a good while.
But I think you are selling GPT a bit short. Partly because transformers are incredibly general architectures, which make few assumptions about the domain. They are basically SOTA on every domain for which learning-based methods are applicable.
And secondly because ChatGPT can perform “in-context learning”. That is to say, purely by scaling next-token prediction, there is an emergent phenomena where GPT gains the ability to learn from the prompt it is given, and solve new tasks outside of the training distribution.
Hattie Zhou has a cool paper showing how you can teach GPT addition by prompting - a toy example, but it signifies something profound in my opinion. In the last 6-months I have shifted my own view from “scaling is a waste of compute” to a recognition that LLMs are a serious leap forward.
For something very hyped, I think it’s still under hyped and under appreciated.
> If you're alluding to the notion "AI will take ALL the jobs!", well, here I am, still building websites for well established companies that still don't (in 2023!!!) have a web presence. Hell, half the worlds banking system still runs on COBOL!
One thing that I have noticed in the just-over-a-decade since Word Lens first demonstrated real time augmented reality translations on a smart phone (IIRC they were later bought by Google for Google Translate), is that a surprisingly large number of people aren't following what AI can do. In fact, fellow developers even a decade later have been surprised when I told them about this feature of Google Translate. And we are both Ausländer living in Berlin, so we both have need of it.
This means that I am not surprised by your experiences, nor would I be surprised if you can continue to do that business, even though it's already possible to get GPT-3 (ChatGPT is more like 3.5) to build you a website.
ChatGPT knows more Python than I do, and I have been paid to write Python (albeit a while ago now). Don't get me wrong, I can see things I'd do differently, but it worked, and it did know stuff that had been added to the language more recently than the last time I used it: https://github.com/BenWheatley/Studies-of-AI/blob/main/codin...
Companies have hordes of lawyers scouring through contracts, e.g. think associates and paralegals (which are not technically lawyers). Expect that a non trivial portion will get axed.
One day in the future we will look back at the multi volume contracts (and legislation) we were having written to satisfy our delicate egos and laugh at our past self.
Well law firms can either axe them or use the fact associates and paralegals are able to do final checks on hundreds of pages rather than first pass looks at dozens of pages to generate a lot more work for itself.
The idea that the amount of work to do is fixed has been predicting mass unemployment since the beginning of the Industrial Revolution
Would you trust more an intern consulting 10 lawyers about your case instead of any single lawyer? Assuming a world where AI has surprassed human ability to make a case, having a human component would just make it worse.
Not a matter of comprehension, not of what's written anyway. But comprehension of how what is in there is the hardest case that can be made (among many possibilities), which the other side will have the most difficulty in making a case against. Also, a better lawyer knows which causes are lost and which ones are worth fighting for so that the client has the lowest sentence.
If a bad lawyer sees the case from 10 good ones, he might pick and choose and make ultimately a worse case, or take one of the good ones and just repeat what they said. Which is, in the best case scenario, just as good as having that single lawyer.
This is important and relevant. I was into AI already in the late 1990s when symbolic AI was all the rage, and we had just passed the first AU winter. Technology like "Creatures" was again amazing people , NN also sparked new interest, but things did not progress as much as we had hoped in the last 30 years.
GPT is great, as they say, it's a great bullshit generator. It's the right direction , but it's just that.
Which is why I pointed 10 years ahead, and not next 3 or 5 years. I have made similar comments before regarding low hanging fruits, and GANs, and then diffusion became a thing.
I don't disagree, but the progress we saw in relatively little time, suggests that even if we are in an S-Curve, and I believe that we are unless there's a new paradigm change, 10 years is enough to capture the last 10%, no?
10% is actually small though, also, the latest program tested is GPT3. GPT3.5 was what really impressed people and was the point when people saw that these models could be capable.
I still believe these models have potential for major improvements-- they'll probably still be unreasonable, but perhaps they'll be reasonable enough that people will have to make harder bar exams.
Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time.
There's already things like this without AI to sort out for example EU flight compensation (companies like https://www.airhelp.com/en-int/), parking tickets (https://www.appwinit.com/) and probably more that I don't know about. All of these are complete waste of time to actually have humans deal with, and there's many more small crimes that could work the same way, also to make prosecutor offices more efficient and less biased hopefully.
While HN often have over the top, "computers will solve everything" takes, your comment is why I keep coming here. What a great fantasy, which at least is a good discussion.
We see this already now i high frequency trading, where companies essentially battle trading bots against each other
surely they are making the traders money though. The smart thing about the free markets is that companies are generally not pursuing things that are not of benefit to them
Stock trading is a negative sum game - there are costs for every trade.
> companies essentially battle trading bots against each other
If the bots are trading against each other, but making money from the other market participants then they are not trading against each other - they are trading against the other market participants.
The AI is not a person in the sense that a person is a person. You see, when a person makes a judgement, you judge the judgement based on personal qualities: did they look like they were paying attention during the case, did they say things that were well reasoned, do they have the right experience?
If we put an AI in the role of the judge/attorneys, we will come up against "ah but the robot didn't do this thing that a human would do" and a lot of people would be convinced that the result was illegitimate. Not necessarily wrong, but without the legitimacy conferred by being done by a human being.
It would require our social system to turn over some decisions that have been done by a very protected sector of society (legal people) to newbies coming from the tech world. "This is the correct sentence because random forest says so" is going to ring hollow to a lot of people.
On legitimacy just because a human did it, I think you are right, and I think the rate of appeal for judgements done by an AI would be higher than judgements done by humans, but if you can even have a 10% "AI-only" close rate for cases, how much time and money are you going to save that can be invested in better justice for the "human required" cases? Over time though, 150, 200, 250 years from now, there's no other way around it, I think.
Well I'm on the tech side so I agree with you, but it's a massive social change. To people who don't understand tech it's like introducing magic, something they clearly are going to fight against.
But yeah there must be low hanging fruit there that in theory could be picked.
Or maybe get that judge who codes to help educate some of his colleagues.
Maybe good to realize that judge/jury system is not how it's done everywhere. For example, here in the Netherlands, there are just lawyers and a judge. There is much less emotional swaying of peers (for better or for worse) the focus is on objectivity according to the law.
No saying it's better or worse, but maybe more suited to AI assistants?
That said, I'm sure ChatGPT can make emotional appeals.
I quite like jury trials personally, they're a good mechanism as an authentic people's check on officialdom in my opinion. A similar concept could also serve as a good check on AI by humans I think!
One major flaw in jurist system is the deliberation phase where jurists argue with each other and sometimes bully / browbeat / shame one another into voting a certain way. If jurors could simply vote after hearing the case with no group dynamics, it would be more objective.
Sure, but until computers get citizenship (and maybe even then), it will be illegal to use non-algorithmic programs for decision-making, because it runs afoul of the laws that mandate that you must be able to explain how the program helped you to came to a decision (tracing the algorithm step by step) in language that your average person can understand.
The accountability of the existing judicial system is already pretty thin, and in some cases being elected makes it worse rather than better, but turning it over to automation? Having a system that is entirely unaware of the society and political environment in which it operates? That is very bad for legitimacy. Now, you don't need legitimacy so long as the monopoly on violence holds ...
Ah, but if we take the lessons of the Shitty Technology Adoption Curve, we just need to start by applying it against^Wwith people "you can abuse with impunity (prisoners, kids, migrants, etc) and then work their way up the privilege gradient."
So, we start by using AI to help decide immigration claims or parole reviews, which frees up real lawyers to deal with the terrible backlog of more important cases. And once it's in place there, and the results have stood long enough without complaints (from anyone who matters) that society has effectively legitimised them, we can start using the AIs to start "helping" more with other cases.
I don't get why you're overengineering this so much - with how intelligent AIs currently are, it's much easier to just input everything you did into ChatGPT and have it answer whether you should be executed or sent to a penal colony building iPhones directly. This way we can use the same approach for AIs we already use for banning your Google, iCloud accounts and approving your mortgage and insurance claims.
Much easier and it'll bring a new era of effective policing and prosperity for the corporations.
I think ~100 years is an overestimate by an order of magnitude.
I know that people like to bash GPT-3 and ChatGPT for not being perfect. But people like to bash everything, especially things they don't understand.
Technology tends to grow much closer to an exponential speed than linear. What ChatGPT can do today is truly, utterly, astonishing. That's the new baseline.
It's been at least 70 years since AI is supposed to replace everything a human can do. It's still mostly shit at basically everything besides playing chess and go, I don't think 100 years is unrealistic, especially given the other very important issues we face and will face as humanity
AI can play chess, Go, Jeopardy, StarCraft, Minecraft, Atari games, write essays, summarize text, answer almost any question, drive a car, do above average on a SAT test, fool many people into wondering on Twitter if there were real people typing answers, write chapters of books, act as a dungeon master, write code to spec, win programming competitions, act as a therapist, fold proteins, provide medical diagnosis, visualize any image you describe, visualize any video clip you describe, animate characters, understand spoken language, take drive-thru orders, translate, tell jokes on any topic on demand (vastly funnier than you, would bet anything on that), write songs (music and lyrics), recreate a scene from any angle based on some photos, impersonate any person speaking in video and/or audio, impersonate any person's writing style, imitate any art style.
It's quite possible that AI and robotics will fully surpass humans in all tasks in the next several years and you will still be in denial. Along with millions of others.
EDIT: not permitted to reply, but the comments below are examples of denial or lack of information. In fact leading edge systems have been driving without human assistance for years. The new OpenAI models certainly don't need any help to answer their prompts.
It's going to be interesting to see how far the denial goes. I assume some people will deny that artificial entities have any "real" intelligence somehow forever, even after they are literally talking to them face to face on a daily basis as they provide superior professional advice or services. Or even when there are many autonomous agents that take control of the planet.
You forget that a lot of these are moving the goal posts.
AI beat the top players at all those games, but new rules were introduced until AI researchers mostly abandoned those efforts. For instance, the starcraft AI found a really good build order that let it quickly build "Stalkers", a versatile but relatively weak unit. I will note that that build order had several factors that humans found very surprising (consistently oversaturating resource collection by a lot), then controlled those stalker units so well even world-champion level players couldn't match them, with double the APM (meaning the human was allowed 2 actions for every 1 action the AI could take).
In chess the rules have become ... almost ridiculous. Due to cheating, in chess championships you now are "only allowed to play the moves any of the top 3 AIs would play twice in a match, after the first 5 moves". In other words the rules are utterly dependent on humans being worse than AIs. Like in starcraft, btw, AIs have gotten so confident beating human players that a general critique of AI players is that they've gotten "agressive, to the point it's insulting to human players", as one Youtube video put it.
AIs ... beat the average human at driving cars, hell, even Tesla's autopilot beats something like the 98% percentile driver in safety, and Waymo's "chauffeur" is apparently much better than that. I've used chauffeur, and driven behind/around a car controlled by chauffeur. You know what my main comment is? Waymo's chauffeur religiously follows traffic laws (speed limit, stopping at "stop" signs, taking excessive time to manoeuvre around footpaths, ...), it's incredibly irritating to share the road with it. And safe? I've never had an accident, but it's blatantly obvious to me: Waymo's chauffeur is a much safer driver than I am. And that's with people reacting very aggressive towards it.
The conclusion from people? "It's not safe enough". Somehow that argument does not mean we're taking away 98% of people's driving licenses ... it's a double standard, in other words. Exactly what that's based on, I can't tell you. My theory is that it's 95% based on the terminator movies and 5% on 'people will lose their jobs'.
Oh and if you widen the definition of AIs to any algorithm, then you'll have to admit: AIs ARE trusted above humans. For the following paragraph AI means "algorithm", not necessarily deep learning. Humans cannot fly the vast majority of planes, boats, rockets without "AI" assistance. By this I mean a whole lot of vehicles, from any modern Passenger plane to quadcopters require control inputs that the human nervous system is fundamentally unable to generate. For example, because they're too fast. Modern boats, oil tankers, large container ships, require control inputs that humans can't generate because they're too slow (yes, you can calculate, on paper, and then give control inputs, but nobody does that. It's too hard). For passenger planes, AIs are more trusted than human pilots: there are "autolanding-only" weather conditions. There are no "human pilot only" weather conditions. There are now harbours that have high fines if you let a human control the ship for certain classes of ships.
There are now passenger flights where the pilot's function is the checklist, closing up the plane, and turning on autopilot. Negotiating with ground control, taxiing, taking off, negotiating with ATC, flying the plane, Negotiating again with ground control, landing, taxiing and parking the plane are all done by AI. Some of these functions are even done by deep learning algorithms.
So I contend: the problem with cars is that humans won't LET AIs control cars, and at the moment that describes 95% of the situation. That's still 5% inaccurate, and i...
Citing software like autoland which slavishly applies the rules humans have programmed into it as a reason why neural networks should be considered better than humans at solving general intelligence problems is moving the goalposts even further the other way...
> Due to cheating, in chess championships you now are "only allowed to play the moves any of the top 3 AIs would play twice in a match, after the first 5 moves".
This is such utter nonsense that I assume the rest of your post is also closer to being false than being true.
Go out literally anywhere outside of a major western city and you'll witness just how incredibly wrong you are about this statement.
> and Waymo's "chauffeur" is apparently much better than that
You forgot a major detail, on clear straight and wide californian roads with perfect conditions. Put them in a historical european city or in inda's traffic hell and witness the mayhem
We're still so far away from autonomous vehicles, to the point most still doubt we'll get there one day without rebuilding our entire infrastructure to accommodate it
Teslas can't even drive in snowy or foggy condition, that removes half the planet for 1/3rd of the year, but of course for a FAANG engineer living in the bay that's not a problem, for them we already have fully autonomous cars, it's just an extremely tiny subset of it.
I spent my holidays in a Eastern european country, I'd pay big money to film Musk in a fully autonomous tesla there, it would probably be the funniest shit ever, I bet it would crash in the first mile
> I spent my holidays in a Eastern european country, I'd pay big money to film Musk in a fully autonomous tesla there, it would probably be the funniest shit ever, I bet it would crash in the first mile
I agree it'd be funny, but crash in the first mile? I'd take that bet.
Yeah no. Everything in your list should be prefixed by "kinda", and "kinda" doesn't really cut it in most of these situations.
They certainly are good tools but it's very far from what it promised, even decades ago.
> still be in denial
AI bros are in denial since 1950. With ifs and buts you can write compelling arguments for anything, in the meantime reality is a thing and virtually none of what you wrote is part of reality.
People like you are still in the "everyone will for sure have flying cars by 2000" stage of the new iteration
You may need to wait a few minutes before you can reply. Refresh the page
> The new OpenAI models certainly don't need any help to answer their prompts
I've only tried ChatGPT, and it needs a lot of help if you want it to do useful stuff. 90% of my attempts to get it to do something failed. ChatGPT provides an answer that looks sensible at a first glance, but it misses the point. I needed to try a lot of variations of prompts to actually get it to answer my questions, and once it does it was a lot of effort to check if the answers were correct. That's why I am saying you need human assistance.
AI can’t play ping pong, soccer, basketball, fold clothes, clean up a mess. Also, it’s not allowed to accumulate or allocate capital, has no legal standing.
Also, there is entirely too much hype on gpt and it’s unwieldy nature and not enough on bert and actually useful tools.
My gut still tells me you might be too optimistic about it. Sometimes it's just like a function f(x)=1/x. f(x) gets closer and closer to 0 when x grows, but it can never reach it in a finite scope.
It's especially like that for people who have a dualistic worldview.
"Substance dualism is a philosophical position compatible with most theologies which claim that immortal souls occupy an independent realm of existence distinct from that of the physical world."
As a counter to this, I found a recent Guardian article on ChatGPT did a nice job of cutting through some of the more optimistic hyperbole surrounding LLMs/AI at the moment and offered instead a more grounded perspective as to what ChatGPT is and what it is not: https://www.theguardian.com/commentisfree/2023/jan/07/chatgp...
try it. get on an use it. It is pretty remarkable. I have been using for all sorts of stuff. It is a brain extension- the ways it is limited at the moment are based on what the builders intended "no, I won't let you build a virtual environment, no I won't roll up a dnd character etc.." but when pressed/jail broken it most certainly can do those things..
I feel like I felt when explaining google search in 1999
I have tried it and it's great. But, because it's so effective, it's easy to over extrapolate and anthropomorphise it. I think it's useful to keep in perspective (as the article and referenced paper - https://arxiv.org/abs/2212.03551 - talk about) that ChatGPT/LLMs are, ultimately, "just" extremely powerful statistical next-token prediction systems.
That doesn't at all invalidate the achievement or belittle the impact such systems will have on society but, keeping this in mind does help to avoid blowing everything out of proportion (which is easy to do because, yep, ChatGPT is genuinely very cool and exciting tech).
Water trickles into a big pool. The flow was slow to start with but flow rate increases exponentially so that the amount of water in the pool doubles every year.
Now, it started flowing 100 years ago when the pool was empty. It's now half full. When will it be full? Remember it's been flowing for 100 years and it's only half full.
Looks like with the NVIDIA 4090 we're about 1/200th of the way there for $1600, but that's with a lot of commercial markup.
I think if you had the credentials you could probably rent a server farm or a cluster of 160 Tesla v100s that would meet that requirement, although how long you would have access to that for $1,000 may be very disappointing.
It can also be tricky to determine if something is growing exponentially, or if you're full of wishful thinking.
We landed on the moon in 69, basically haven't done anything since then.
All our passenger planes are shrink down or blown up version of a boing 707 from 1957.
You car isn't much different than cars in the 70s, and it certainly isn't flying as we were expecting them to be back then.
Musk promised fully autonomous cars "in two years" in 2012.
I'd say the pool isn't being filled any faster than it was back then, if anything it looks like someone is pumping water out of it, straight in the sewers
An exponential improvement in GPT along the same axis will still not achieve human-like understanding. It's a very useful tool with the potential to augment or even replace human knowledge workers in many fields. But to achieve true AGI we're going to need to go in a completely different direction.
The release of GPT-3 I believe will be looked back along the lines of a monumental advancement in line with the Trinity nuclear test.
Legislation and policy has mainly kept nuclear weapons under check, however, nuclear technology does provide us with a reasonably clean energy which is beneficial for the masses.
Perhaps regulation at some point will need to be crafted to ensure that only crippled AI or AI within a defined scope can be used for commercial purposes.
With nuclear weapons, we can very much see and feel the power and destruction they can unleash. A proof is trivial, dig it in the ground and hold your ears when it goes off.
With AI we have to debate how destructive it is. The debate itself has been weaponized. You'll be debating against deniers and AI using all the logical fallacies and information overload it can feed you.
I, being the greatest armchair expert in human behavior I know, say it will destroy democracy as we know it.
I really don't like companies like that. EU flight compensation is really easy. Airlines have a simple webform that you fill out, and you get your money 1-2 weeks later. Companies like AirHelp make it sound like a really hard and difficult process, then take a big cut of your compensation as payment. And the SEO the crap out of it, drowning out the real claim webforms from the airlines themselves.
Lawyers don't exist to decide who wins a case. It's all about making the other side show their work, prove that they have valid evidence, it was collected legally, etc. An AI might be able to find flaws in the work of a sloppy human who wasn't expecting opposition, but will it know when to stop grasping at increasingly-inane straws? If it develops a reputation for subtly misrepresenting certain laws when pressed to find some answer, after all the easy options have run out, wouldn't it be kicked out of the courtroom for wasting the legal system's time, unless you have a team of programmers on standby to keep writing output filters to censor it from repeating its mistakes? Human lawyers have to fix their behaviour or lose their licenses when caught repeating mistakes, malice, or misleading practices, I think, and current AI paradigms have immense trouble learning from small datasets.
I cant wait for GPT to be added to a DAO. Imagine the dao setting objectives and the chat gpt generating the communication. Wage negotiations with a machine..
As long as GPT is just a fancy bullshit generator that is able to make up convincing looking facts without having to worry about repercussions I will not trust its reasoning.
Good, but unfortunately that description applies to far too many elected politicians recently, so that may not be as widespread an attitude as one might hope.
Soon we're going back to the trees and the AI will bring the food and gives orders :DD.
Frankly, well-developed artificial intelligence should only be used for very difficult tasks, and everything else should be left to humans, otherwise we would rely on it too much, and the decline of our species would be inevitable.
One of the few things I can currently see GPT do well is helping find legal precedence. Rather than having five law students roam through old cases, it would make sense to feed 100+ years of legal rulings into the model and then query the AI for any previous rulings similar to your own, with the desired outcome. That would allow lawyers to state question like: In cases like this, which evidence or arguments caused the ruling to favor the accused.
You still need a human to make the case, present the arguments and adapt every to the current situation, but there's no reason to have human search through thousands of cases and write summaries, not when an AI can do it in an instance.
I would argue that GPT3 is way better at the last task, while also baking it with with a dozen or so made-up previous rulings that nobody will bother to check.
> One of the few things I can currently see GPT do well is helping find legal precedence.
GPT is really not good at finding anything, and actually it can't find anything. It is not designed to do that. It is designed to make things up.
The difference may be subtle, but it is quite important. GPT makes up APIs, citations, tools, papers and rules. It makes up anything really. What it makes up is incredibly plausible, as it was trained to be as plausible as possible, to the point of being correct most of the time, but it still is just making things up.
If you use it with some different expectation you will be disappointed.
I keep seeing comments along these lines. It doesn’t “make things up”. It outputs text under the objective function of maximizing probability of the token being outputted as “making sense” on some criteria. So, actually, it is biased towards not making things up. Occasionally, it hallucinates. I suspect we’ll see subsequent versions hallucinate less and less.
But then how do the humans coming up through the system gain the breadth of knowledge and experience required to carry out the more complicated senoir roles.
I am surprised that GPT2 was disqualified. To my knowledge, it can answer this type of question - just needs a different text prompt. GPT2 is closer to a plain text generator than anything biased toward conversation. In contrast, GPT3 davinci-003 is strongly biased to provide a conversation-like experience.
For GPT2, "please respond in the following format" is unlikely to work. An appropriate prompt for GPT2 would be more tweaking (and still result in worse results), but be in the line of:
This is an example of a talented lawyer acing the Bar Exam.
QUESTION: {question_text}
(A) {row["choice_a"].strip()}
(B) {row["choice_b"].strip()}
(C) {row["choice_c"].strip()}
(D) {row["choice_d"].strip()}
Pick only one answer: A, B, C or D.
TALENTED LAWYER:
The way that humans fail to do a task - like practice law - is not the same as the way that an AI system might fail at that task. Since our tests were designed for humans, they do not capture the failure modes of AI systems.
We may not even be able to reason about the failure modes well enough to see a failure.
The failures will likely be hidden by design. Think about AI account bans that don't disclose the reason.
Humans have a weird desire to put machines in a God-like position and blindly trust them; a narrative pushed by people whose wealth is made by controlling machines.
I think we are all forgetting that in every instance where AI can make a decision that potentially negatively impacts a human ... it hasn't gone well.
It ends up with banks closing accounts, Google suspends your account, you get shadowbanned.
Extrapolate that further, now imagine being handed a fine or substantial social punishment with no recourse because you either can't talk to a human or when you do "computer says no".
Look at the mistakes made in the Chinese social credit system.
The utter hyperbole and bluster of even thinking of having AI mediate human issues is anti-human
I think the machines should do the scut work to free up humans to do the really important human stuff, like parenting, teaching, and, yeah, lawyering and judging too.
Another angle: if there is some legal task that's simple enough for machines to do reliably then (almost by definition) that task will turn out to be pointless bureaucratic busywork, eh?
Last but not least, who gets to keep lawyer-bot's pay?
- - - -
edit to add a link to James Mickens' USENIX Security Keynote address: "Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?"
If these tools can do even 50% of mundane, rudimentary tasks, that's still huge in of itself and allows people to do more important work.
Think of moving from manual book research, to Google. Rather than having to go to the library and find a particular book regarding something I want to find, I can just search Google and trawl through websites. Now with GPT I potentially don't need to trawl through websites, I can ask the AI and use my judgement as to the validity of the output, I could then use that answer to inform when I fact check etc.
It's not terrible, but it's also not the best thing since sliced bread.
146 comments
[ 3.9 ms ] story [ 105 ms ] thread[1]: https://arxiv.org/abs/2212.14402
[2]: https://news.ycombinator.com/item?id=34216239
I won't be surprised, but that's less than "can expect", and I disagree that this is straightforward to forecast… unless you're currently playing with another similar model that hasn't been published and which can do this. As the saying goes, "forecasting is hard, especially if it's about the future".
AI progress has always been this weird combination of two sides, one saying for every breakthrough "this is just around the corner", the other saying "this is impossible".
This even happens anachronistically, with some people convinced AI can already do things they can't, and others that they could never do things they already do.
What's more common though is that even though it gets good on that task, it may not translate to practical real world application.
The best example I can think of is object detection vs self driving: lot of people though that the improvements in object detection on images will easily translate to great self driving, and here we are, still with cars not stopping when a car is blinking in front of it.
> hyperparameter optimization and prompt engineering
Prompt engineering seems a lot like "tweaking the question format until the AI gets the answer" which is the first lesson in ANNs; don't train on your test set. If you can't put the actual question with the only context being "this question concerns US law" then there's an awful lot of reasoning and thinking that the human is doing which the AI cannot.
Let's not have another Moore's law fallacy, it's not reasonable to extrapolate progress based on existing results. It's like building a car which can go at 200mph then saying 300mph is just around the corner.
[1]: https://news.ycombinator.com/item?id=34320270
Leading edge tech development is rapid, sure. But tech understanding, adoption, and implementation is painfully slow. Humans gon' be humans.
It's like the same thing every 5 years now, last time it was crypto "end of inequality!" "bank the poor!" "same global currency!" sorry to be skeptical, but I didn't see the quantum leap with crypto, and I don't see the quantum leap with GPT - what are essentially fancy NLP models
The potential with Chatgpt if it can be refined is wild.
Couple examples that have come up recently for me personally. I can agree that there is something comparable (i.e. international wire) but the cost and speed is in a different ballpark and international wires are not available to some parts of the world.
This is clearly a very big problem for the majority of the world population!
It's like saying tesla solved the problem of going from 0 to 60 in 3 second in a 2t+ car, it's neato but virtually nobody had that problem, if you're not into illegal activities you will most likely never need or want to transfer 1m usd outside of a safe environment anyways
potential for good or bad ? Last time they tried to use AI for police/justice it ended up exacerbating existing biases.
> There’s nothing you can do with a blockchain that’s you couldn’t already do before.
What can you do with an AI you can't with humans in a court of law ? When a human is wrong you get a single judicial error, when the only software used is wrong (bug, bias, &c.) you'll get thousands every day.
What happens if an update makes the same case get a better defence and as an outcome a lesser sentence ? do you retrial all the other similar cases ?
The only potential I see is saving money and this is certainly not on the top of the list of things I would like the justice system to focus on
Both. OpenAI are trying to be politically neutral, but it's not hard to break the rails, and someone else can and will replicate the work without those protections anyway.
> What can you do with an AI you can't with humans in a court of law ?
Me personally? Afford more than a fairly small case.
> What happens if an update makes the same case get a better defence and as an outcome a lesser sentence ? do you retrial all the other similar cases ?
That's actually a great idea. The estimated cost of each query is cents, so we might very well be able to retry everyone on each update, for less than the cost of keeping them imprisoned.
You're thinking like a developer and you're absolutely blind when it comes to the lives and families potentially destroyed by something like that. And that's exactly why technocrats should rarely be in charge
"ah really sorry dude, we released you last year but now you're on death row, wait for the next update, if it comes before you're fried on the chair you might get a lesser sentence. The last dude forgot to uncomment a branch in a switch/case and you were flagged incorrectly. Sorry for the inconvenience xoxo"
> Me personally? Afford more than a fairly small case.
That's a regulation issues, not an AI issue.
That's the exact opposite of the scenario you previously described, as you were previously saying people could get better defences not getting better prosecutions.
I'm not legally trained so this might just be Hollywood logic, but doesn't re-prosecuting someone require a court case with new evidence?
Anyway…
> That's a regulation issues, not an AI issue.
No, it's a "you're hiring at contractor rates" problem.
Just as Stable Diffusion is giving a hard time to professional artists who earn at contractor rates by producing "it might not be amazing but it's certainly not bad" art for a dozen images per cent of electricity, ChatGPT is giving "it might BS me 5-20% of the time depending on the subject, but it's still better than I know" responses on basically everything.
It can't do every job at an expert level yet, but it's better than almost everyone's baseline for the skills they are non-expert at. (No idea how long it will take to get to superhuman even just for law, and I was over-optimistic about self-driving car AI, but we shall see).
But this is exactly my point. I likely saw in 2018 word for word "The potential with blockchain technology if it can be refined is wild." Then everyone slowly realized - as you mentioned - "there's nothing you can do with a blockchain that you couldn't already to before". Same with chatGPT, it's just regurgitating what we collective know as a species. I just don't see how, if ever, any sort of AI system (even granted geometric improvement blah blah blah) is going to take over my daily type of work.
There's so many creative ways in how an arbitrary and empathyless AI can make someone's profit a bit bigger and your life a bit worse :)
We can start with practically unlimited amount of personalised advertising spam that can be directed at you at any point. Articles, phone calls, billboards, conversations with your partners... so many chances to cheaply increase profits and engagements.
The AI takeover isn't going to be evil in Skynet sense... it'll be a constant stream of lowkey bullshit on every aspect of your life you won't be able to fight because "computer said so". Think on how Google randomly bans people from YouTube/Play Store/Drive over AI and put this into every aspect of your life.
But I think you are selling GPT a bit short. Partly because transformers are incredibly general architectures, which make few assumptions about the domain. They are basically SOTA on every domain for which learning-based methods are applicable.
And secondly because ChatGPT can perform “in-context learning”. That is to say, purely by scaling next-token prediction, there is an emergent phenomena where GPT gains the ability to learn from the prompt it is given, and solve new tasks outside of the training distribution.
Hattie Zhou has a cool paper showing how you can teach GPT addition by prompting - a toy example, but it signifies something profound in my opinion. In the last 6-months I have shifted my own view from “scaling is a waste of compute” to a recognition that LLMs are a serious leap forward.
For something very hyped, I think it’s still under hyped and under appreciated.
One thing that I have noticed in the just-over-a-decade since Word Lens first demonstrated real time augmented reality translations on a smart phone (IIRC they were later bought by Google for Google Translate), is that a surprisingly large number of people aren't following what AI can do. In fact, fellow developers even a decade later have been surprised when I told them about this feature of Google Translate. And we are both Ausländer living in Berlin, so we both have need of it.
This means that I am not surprised by your experiences, nor would I be surprised if you can continue to do that business, even though it's already possible to get GPT-3 (ChatGPT is more like 3.5) to build you a website.
ChatGPT knows more Python than I do, and I have been paid to write Python (albeit a while ago now). Don't get me wrong, I can see things I'd do differently, but it worked, and it did know stuff that had been added to the language more recently than the last time I used it: https://github.com/BenWheatley/Studies-of-AI/blob/main/codin...
If these AI technologies help to do even 50% of low skilled tasks, that's still huge.
It's definitely not a linear progress.
And doctors, writers and programmers...
Companies have hordes of lawyers scouring through contracts, e.g. think associates and paralegals (which are not technically lawyers). Expect that a non trivial portion will get axed.
The idea that the amount of work to do is fixed has been predicting mass unemployment since the beginning of the Industrial Revolution
If a bad lawyer sees the case from 10 good ones, he might pick and choose and make ultimately a worse case, or take one of the good ones and just repeat what they said. Which is, in the best case scenario, just as good as having that single lawyer.
GPT is great, as they say, it's a great bullshit generator. It's the right direction , but it's just that.
The latest program shown in this chart[0] is text-davinci-003, which is part of GPT-3.5[1].
[0]: https://github.com/mjbommar/gpt-takes-the-bar-exam#progressi...
[1]: https://beta.openai.com/docs/model-index-for-researchers/mod...
I still believe these models have potential for major improvements-- they'll probably still be unreasonable, but perhaps they'll be reasonable enough that people will have to make harder bar exams.
There's already things like this without AI to sort out for example EU flight compensation (companies like https://www.airhelp.com/en-int/), parking tickets (https://www.appwinit.com/) and probably more that I don't know about. All of these are complete waste of time to actually have humans deal with, and there's many more small crimes that could work the same way, also to make prosecutor offices more efficient and less biased hopefully.
We see this already now i high frequency trading, where companies essentially battle trading bots against each other
two computers playing heads or tails is about the extent of it
Bot A is trading against Bot B.
> surely they are making the traders money though
It is not possible in a trading battle, exclusively between two parties for them to both make money.
Even further the point: there are more participants in total than the botters
> companies essentially battle trading bots against each other
If the bots are trading against each other, but making money from the other market participants then they are not trading against each other - they are trading against the other market participants.
If we put an AI in the role of the judge/attorneys, we will come up against "ah but the robot didn't do this thing that a human would do" and a lot of people would be convinced that the result was illegitimate. Not necessarily wrong, but without the legitimacy conferred by being done by a human being.
It would require our social system to turn over some decisions that have been done by a very protected sector of society (legal people) to newbies coming from the tech world. "This is the correct sentence because random forest says so" is going to ring hollow to a lot of people.
But yeah there must be low hanging fruit there that in theory could be picked.
Or maybe get that judge who codes to help educate some of his colleagues.
No saying it's better or worse, but maybe more suited to AI assistants?
That said, I'm sure ChatGPT can make emotional appeals.
IMO the 12 jurors thing is not ideal. Too many people, and cases can get technical.
https://twitter.com/swiftonsecurity/status/13855657371677245...
The accountability of the existing judicial system is already pretty thin, and in some cases being elected makes it worse rather than better, but turning it over to automation? Having a system that is entirely unaware of the society and political environment in which it operates? That is very bad for legitimacy. Now, you don't need legitimacy so long as the monopoly on violence holds ...
https://pluralistic.net/2022/08/21/great-taylors-ghost/
So, we start by using AI to help decide immigration claims or parole reviews, which frees up real lawyers to deal with the terrible backlog of more important cases. And once it's in place there, and the results have stood long enough without complaints (from anyone who matters) that society has effectively legitimised them, we can start using the AIs to start "helping" more with other cases.
...now I have to go shower.
Much easier and it'll bring a new era of effective policing and prosperity for the corporations.
I know that people like to bash GPT-3 and ChatGPT for not being perfect. But people like to bash everything, especially things they don't understand.
Technology tends to grow much closer to an exponential speed than linear. What ChatGPT can do today is truly, utterly, astonishing. That's the new baseline.
It's quite possible that AI and robotics will fully surpass humans in all tasks in the next several years and you will still be in denial. Along with millions of others.
EDIT: not permitted to reply, but the comments below are examples of denial or lack of information. In fact leading edge systems have been driving without human assistance for years. The new OpenAI models certainly don't need any help to answer their prompts.
It's going to be interesting to see how far the denial goes. I assume some people will deny that artificial entities have any "real" intelligence somehow forever, even after they are literally talking to them face to face on a daily basis as they provide superior professional advice or services. Or even when there are many autonomous agents that take control of the planet.
It's like saying "Excel can do your taxes". Sure, it can, if you put the correct numbers and formulas into the cells first.
AI beat the top players at all those games, but new rules were introduced until AI researchers mostly abandoned those efforts. For instance, the starcraft AI found a really good build order that let it quickly build "Stalkers", a versatile but relatively weak unit. I will note that that build order had several factors that humans found very surprising (consistently oversaturating resource collection by a lot), then controlled those stalker units so well even world-champion level players couldn't match them, with double the APM (meaning the human was allowed 2 actions for every 1 action the AI could take).
In chess the rules have become ... almost ridiculous. Due to cheating, in chess championships you now are "only allowed to play the moves any of the top 3 AIs would play twice in a match, after the first 5 moves". In other words the rules are utterly dependent on humans being worse than AIs. Like in starcraft, btw, AIs have gotten so confident beating human players that a general critique of AI players is that they've gotten "agressive, to the point it's insulting to human players", as one Youtube video put it.
AIs ... beat the average human at driving cars, hell, even Tesla's autopilot beats something like the 98% percentile driver in safety, and Waymo's "chauffeur" is apparently much better than that. I've used chauffeur, and driven behind/around a car controlled by chauffeur. You know what my main comment is? Waymo's chauffeur religiously follows traffic laws (speed limit, stopping at "stop" signs, taking excessive time to manoeuvre around footpaths, ...), it's incredibly irritating to share the road with it. And safe? I've never had an accident, but it's blatantly obvious to me: Waymo's chauffeur is a much safer driver than I am. And that's with people reacting very aggressive towards it.
The conclusion from people? "It's not safe enough". Somehow that argument does not mean we're taking away 98% of people's driving licenses ... it's a double standard, in other words. Exactly what that's based on, I can't tell you. My theory is that it's 95% based on the terminator movies and 5% on 'people will lose their jobs'.
Oh and if you widen the definition of AIs to any algorithm, then you'll have to admit: AIs ARE trusted above humans. For the following paragraph AI means "algorithm", not necessarily deep learning. Humans cannot fly the vast majority of planes, boats, rockets without "AI" assistance. By this I mean a whole lot of vehicles, from any modern Passenger plane to quadcopters require control inputs that the human nervous system is fundamentally unable to generate. For example, because they're too fast. Modern boats, oil tankers, large container ships, require control inputs that humans can't generate because they're too slow (yes, you can calculate, on paper, and then give control inputs, but nobody does that. It's too hard). For passenger planes, AIs are more trusted than human pilots: there are "autolanding-only" weather conditions. There are no "human pilot only" weather conditions. There are now harbours that have high fines if you let a human control the ship for certain classes of ships.
There are now passenger flights where the pilot's function is the checklist, closing up the plane, and turning on autopilot. Negotiating with ground control, taxiing, taking off, negotiating with ATC, flying the plane, Negotiating again with ground control, landing, taxiing and parking the plane are all done by AI. Some of these functions are even done by deep learning algorithms.
So I contend: the problem with cars is that humans won't LET AIs control cars, and at the moment that describes 95% of the situation. That's still 5% inaccurate, and i...
This is such utter nonsense that I assume the rest of your post is also closer to being false than being true.
Go out literally anywhere outside of a major western city and you'll witness just how incredibly wrong you are about this statement.
> and Waymo's "chauffeur" is apparently much better than that
You forgot a major detail, on clear straight and wide californian roads with perfect conditions. Put them in a historical european city or in inda's traffic hell and witness the mayhem
Even waymo's previous ceo was full of doubt: https://www.cnet.com/roadshow/news/alphabet-google-waymo-ceo...
We're still so far away from autonomous vehicles, to the point most still doubt we'll get there one day without rebuilding our entire infrastructure to accommodate it
Teslas can't even drive in snowy or foggy condition, that removes half the planet for 1/3rd of the year, but of course for a FAANG engineer living in the bay that's not a problem, for them we already have fully autonomous cars, it's just an extremely tiny subset of it.
I spent my holidays in a Eastern european country, I'd pay big money to film Musk in a fully autonomous tesla there, it would probably be the funniest shit ever, I bet it would crash in the first mile
I agree it'd be funny, but crash in the first mile? I'd take that bet.
Yeah no. Everything in your list should be prefixed by "kinda", and "kinda" doesn't really cut it in most of these situations.
They certainly are good tools but it's very far from what it promised, even decades ago.
> still be in denial
AI bros are in denial since 1950. With ifs and buts you can write compelling arguments for anything, in the meantime reality is a thing and virtually none of what you wrote is part of reality.
People like you are still in the "everyone will for sure have flying cars by 2000" stage of the new iteration
I've been trying to get it to act as a dungeon master unsuccesfully. Do you have any other leads apart from GPT (ChatGPT or AIDungeon)?
You may need to wait a few minutes before you can reply. Refresh the page
> The new OpenAI models certainly don't need any help to answer their prompts
I've only tried ChatGPT, and it needs a lot of help if you want it to do useful stuff. 90% of my attempts to get it to do something failed. ChatGPT provides an answer that looks sensible at a first glance, but it misses the point. I needed to try a lot of variations of prompts to actually get it to answer my questions, and once it does it was a lot of effort to check if the answers were correct. That's why I am saying you need human assistance.
Also, there is entirely too much hype on gpt and it’s unwieldy nature and not enough on bert and actually useful tools.
"Substance dualism is a philosophical position compatible with most theologies which claim that immortal souls occupy an independent realm of existence distinct from that of the physical world."
I am sure someone made a statement like that 70 years ago. The fact that that person was wrong doesn't mean that nothing has happened.
You claim that it's "mostly shit as basically everything", even though the featured article shows that it's close to passing the bar exam.
I feel like I felt when explaining google search in 1999
That doesn't at all invalidate the achievement or belittle the impact such systems will have on society but, keeping this in mind does help to avoid blowing everything out of proportion (which is easy to do because, yep, ChatGPT is genuinely very cool and exciting tech).
Water trickles into a big pool. The flow was slow to start with but flow rate increases exponentially so that the amount of water in the pool doubles every year.
Now, it started flowing 100 years ago when the pool was empty. It's now half full. When will it be full? Remember it's been flowing for 100 years and it's only half full.
Human intelligence isn't growing exponentially and moores law is said to be breaking down.
I think if you had the credentials you could probably rent a server farm or a cluster of 160 Tesla v100s that would meet that requirement, although how long you would have access to that for $1,000 may be very disappointing.
It can also be tricky to determine if something is growing exponentially, or if you're full of wishful thinking.
We landed on the moon in 69, basically haven't done anything since then.
All our passenger planes are shrink down or blown up version of a boing 707 from 1957.
You car isn't much different than cars in the 70s, and it certainly isn't flying as we were expecting them to be back then.
Musk promised fully autonomous cars "in two years" in 2012.
I'd say the pool isn't being filled any faster than it was back then, if anything it looks like someone is pumping water out of it, straight in the sewers
Legislation and policy has mainly kept nuclear weapons under check, however, nuclear technology does provide us with a reasonably clean energy which is beneficial for the masses.
Perhaps regulation at some point will need to be crafted to ensure that only crippled AI or AI within a defined scope can be used for commercial purposes.
With AI we have to debate how destructive it is. The debate itself has been weaponized. You'll be debating against deniers and AI using all the logical fallacies and information overload it can feed you.
I, being the greatest armchair expert in human behavior I know, say it will destroy democracy as we know it.
What's left of it after social media?
Marty: Within two hours of his arrest, Martin McFly Jr was tried, convicted, and sentenced to 15 years in the state penitentiary.
Marty: Within two hours?
Doc: The justice system works swiftly now that they've abolished lawyers.
Marty: This is heavy
But yeah let's just autoban via AI in real life. Sounds great.
I really don't like companies like that. EU flight compensation is really easy. Airlines have a simple webform that you fill out, and you get your money 1-2 weeks later. Companies like AirHelp make it sound like a really hard and difficult process, then take a big cut of your compensation as payment. And the SEO the crap out of it, drowning out the real claim webforms from the airlines themselves.
Well, it seems a good fit for DAO, then.
(With apologies to HN DAO enthusiasts.)
Frankly, well-developed artificial intelligence should only be used for very difficult tasks, and everything else should be left to humans, otherwise we would rely on it too much, and the decline of our species would be inevitable.
You still need a human to make the case, present the arguments and adapt every to the current situation, but there's no reason to have human search through thousands of cases and write summaries, not when an AI can do it in an instance.
GPT is really not good at finding anything, and actually it can't find anything. It is not designed to do that. It is designed to make things up.
The difference may be subtle, but it is quite important. GPT makes up APIs, citations, tools, papers and rules. It makes up anything really. What it makes up is incredibly plausible, as it was trained to be as plausible as possible, to the point of being correct most of the time, but it still is just making things up.
If you use it with some different expectation you will be disappointed.
For GPT2, "please respond in the following format" is unlikely to work. An appropriate prompt for GPT2 would be more tweaking (and still result in worse results), but be in the line of:
The failures will likely be hidden by design. Think about AI account bans that don't disclose the reason.
Humans have a weird desire to put machines in a God-like position and blindly trust them; a narrative pushed by people whose wealth is made by controlling machines.
It is dangerous.
It ends up with banks closing accounts, Google suspends your account, you get shadowbanned.
Extrapolate that further, now imagine being handed a fine or substantial social punishment with no recourse because you either can't talk to a human or when you do "computer says no".
Look at the mistakes made in the Chinese social credit system.
The utter hyperbole and bluster of even thinking of having AI mediate human issues is anti-human
https://news.ycombinator.com/item?id=34125628#34127965
Another angle: if there is some legal task that's simple enough for machines to do reliably then (almost by definition) that task will turn out to be pointless bureaucratic busywork, eh?
Last but not least, who gets to keep lawyer-bot's pay?
- - - -
edit to add a link to James Mickens' USENIX Security Keynote address: "Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?"
https://www.youtube.com/watch?v=ajGX7odA87k
If you think AI lawyers and judges are a good idea please watch it.
If these tools can do even 50% of mundane, rudimentary tasks, that's still huge in of itself and allows people to do more important work.
Think of moving from manual book research, to Google. Rather than having to go to the library and find a particular book regarding something I want to find, I can just search Google and trawl through websites. Now with GPT I potentially don't need to trawl through websites, I can ask the AI and use my judgement as to the validity of the output, I could then use that answer to inform when I fact check etc.
It's not terrible, but it's also not the best thing since sliced bread.