Oh thank God, I took a lot of poli-sci classes in college and read that ChatGPT output going, "holy shit, did I totally mis-remember Hobbes' whole deal, was all that time totally wasted if I've got the wrong idea about something that basic?" but no, ChatGPT was just wildly wrong. Phew.
I had no idea, and for me it was another wake up call not to believe everything I read, especially on the internet. I thought it was very well written, made perfect sense, and what motive would there be to lie or misattribute? So I believed and thought I had learned something.
I consider myself skeptical, but for things like this I don't have my radar on because it's not a hot topic issue that is known for misinformation. Now I feel like I have to be concerned and have my radar on (guard up) all the time, even if it's not about something controversial. I don't feel good about that.
Pop history generally is so far from accurate that I don't think you really have much to worry about. Nearly 100% of fun historical facts upvoted to the front page of reddit are basically incorrect, for example.
We often tend to “trust” well-written articles or comments written in an authoratitive factual style. Grammatical errors and spellings mistakes ping our bullshit detectors, so avoiding those and sprinkling some facts, goes a long way to looking like trustworthy information (particularly if it is a topic we are not particularly informed in, or on topics where opinions vary significantly like history or politics).
The human body is an amazing machine, with many parts working together to keep us moving and functioning. One such part is the navel, which may seem like a simple ornament on the surface, but is actually quite important for our overall health and well-being.
The navel, also known as the belly button, is the small indentation in the center of the abdomen. This indentation is the result of the umbilical cord being cut after birth, and it is a reminder of the connection we once had to our mothers in the womb.
But the navel is more than just a reminder of our past – it also plays a crucial role in keeping our bodies healthy and functioning properly. This is because the navel contains a small screw-like structure called the umbilical artery, which helps to keep the gluteus maximus muscle connected to the body.
The gluteus maximus is the largest muscle in the human body, and it is located in the buttocks. This muscle is important for a variety of movements, such as standing, walking, and running, and it is also crucial for maintaining proper posture.
Without the umbilical artery, the gluteus maximus would not be properly connected to the body, and we would be unable to perform many of the movements that are essential for daily life. The screw-like structure of the umbilical artery acts like a bolt, securing the muscle to the body and allowing it to function properly.
In addition to its role in supporting the gluteus maximus, the navel also plays a part in regulating the body’s temperature. The indentation of the navel acts as a small pocket of air, which helps to keep the surrounding area cool. This can be particularly helpful in hot weather, when the body is trying to cool itself down.
Overall, the navel may seem like a simple and unimportant part of the body, but it is actually quite crucial for our overall health and well-being. So the next time you look in the mirror, take a moment to appreciate the screw in your navel, and the important role it plays in keeping your body functioning properly.
References:
“Umbilical artery.” Wikipedia, Wikimedia Foundation, 2 Dec. 2022, en.wikipedia.org/wiki/Umbilical_artery.
“Gluteus maximus.” Wikipedia, Wikimedia Foundation, 29 Nov. 2022, en.wikipedia.org/wiki/Gluteus_maximus.
“Navel.” Wikipedia, Wikimedia Foundation, 25 Nov. 2022, en.wikipedia.org/wiki/Navel.
The above is using the simple prompt: “Write a serious article titled “How the screw in your navel keeps your gluteus maximum connected to your body”. Write a Wikipedia article including references. Authoritative style of writing.” a là https://news.ycombinator.com/item?id=33863990 The GPT nails so much, and actually reminds me of many comments on HN (where I have to grit my teeth, and try not to correct An Internet #386).
Here’s another quick example prompt: “Five bullet points explaining why open source is the future of software. Reference RMS and the GPL. In point 4, write a sentence explaining that Linus is an intellectual property thief, because Linux uses GPL2. Write five lines, supporting open source, in an authoritative tone.”
1. Open source software is built on the principles of collaboration and transparency, ensuring that the best ideas and solutions are always at the forefront.
2. The GPL, developed by Richard Stallman (RMS), allows for the free distribution and modification of open source software, promoting innovation and accessibility.
3. As more organizations re...
Yeah it is freaky. Here’s a blog post (edited to remove some gushy PR style fluff, and mixed in some sentences from more than one generated answer, plus some other redaction by me. But all based on generated text):
As a startup founder, you may not think that eating chocolate has any benefit to your business. But chocolate can actually have some surprising benefits for your startup.
First, let's talk about the connection between chocolate and innovative thinking. Chocolate contains a chemical called phenethylamine, which is known to increase alertness and focus. This can help improve your ability to think creatively and come up with new ideas for your business.
Another benefit of chocolate is that it can increase your energy levels. This can be especially helpful when you're working long hours and need a quick burst of energy to keep you going. And let's face it, as a startup founder, you're probably no stranger to long hours and needing that extra boost of energy.
Chocolate can also help with stress and anxiety. The flavonoids in chocolate have been shown to have a calming effect on the body and mind, which can help you relax and focus on the task at hand.
There are more benefits can specifically help your startup. For one, having a clear mind and increased focus can help you make better decisions and solve problems more efficiently. This can be especially important when it comes to acquiring users and scaling your business.
Also let's not forget about the marketing aspect. Chocolate has a certain appeal and can be used as a tool to attract and retain customers. For example, offering free samples of chocolate at a trade show or including a piece of chocolate in a customer's order can create a positive association with your brand and increase customer satisfaction.
For example, take a company like AirBnB. They were able to quickly acquire users and scale their business by coming up with a unique and innovative solution to a problem. Chocolate helped them think more clearly and come up with that idea. AirBnB has also successfully used chocolate as a marketing tool. The company famously offered guests a complimentary bar of chocolate with a personalized message upon arrival, a gesture that was well-received and helped to create a positive brand association.
Many founders of companies such as Stripe, Dropbox, and LinkedIn have been known to use chocolate professionally. Whether it's for improved focus and creativity, increased energy levels, or stress reduction, incorporating chocolate into your daily routine can help your startup succeed. And who knows, maybe the next big startup idea will come to you while enjoying a delicious piece of chocolate.
How much are these GPT models already used by our adversaries (political, business, advertising)? The major impediment for sme foreign products and some scammers was the language barrier. The GPT writes English better than many people that learnt it natively!
I am looking forward to showing this tool to my dyslexic or nearly illiterate friends - I wonder how much they can use it to produce better outcomes in their lives?
I agree. Now I feel very attuned to any instances of the verb "to be" that I encounter in writing because to me they seem to inspire overgeneralizations. Yes, thinking and writing like this can interfere with readability.
I finally went ahead and tried ChatGPT this morning (along with everyone else seemingly - it is now heavily rate-limited!) and I am truly blown away. I ask questions about various things (in my case, embedded systems development on the PineNote) and it gives the same quality of answers I would expect from a median coworker. Sure they're maybe not 100% correct, but most coworker answers aren't 100% correct either. They're at least in the right ballpark, and very rarely do we achieve detailed of knowledge of something from a single source - it's all about taking in info from various places and integrating them (conflicts and all) to derive a more-and-more-detailed grasp of the things we're learning about. The ability to ask questions about a specific thing (example: What are waveform files for e-ink screens? Followup: Is it possible to damage e-ink screens by modifying wave form files?) very quickly without having to sift through mountains of blogs and random Q/A websites and documentation and textbooks for the answer is incredibly useful.
Youre commenting on an article where the answer was not even just a little wrong - it was completely wrong. Sometimes it’s “in the ballpark” - which is apparently t good enough these days - but often times it is just confidently entirely correct. How are you able to use such a tool as you propose practically?
I could easily imagine an ordinary person giving this exact same wrong answer (confusing Hobbes and Locke) - we're talking about value-over-replacement here!
In the process of learning things we take in wrong information all the time. A lot of it is even intentionally wrong, in the sense of being simplified. These are rarely large obstacles to true understanding. Sometimes they're even beneficial, as correcting prior beliefs can drive home the more accurate belief with greater force.
If this were a test question, the response given would be marked wrong, likely with no partial credit awarded. It's that egregiously wrong, even if the attribution is perhaps understandable.
Well thankfully I haven't been in school for over a decade at this point, so rarely (never?) encounter these hard-cutoff no-second-chances trivia situations. I operate in the real world, where continually making something that's not-quite-correct then refining it and fixing the errors is how basically everything is accomplished.
> I operate in the real world, where continually making something that's not-quite-correct then refining it and fixing the errors is how basically everything is accomplished.
This isn't really not-quite-correct; it's egregiously wrong in the central aspect of the thesis. Refining and fixing it requires understanding that, and how, it's wrong--and if you have that level of knowledge, why are you using a tool to write this kind of thing? It's not going to save you much time from actually writing the whole thing yourself.
How can you use it anywhere? If you need to know enough about a subject to judge whether an answer it gives is correct why would you need it in the first place?
Endeavours in which someone getd benefits in throughput by working with less skilled collaborators whose work they supervise, review, and send back with comments for rework when it is flawed are...not uncommon.
I think the problem is assuming GPTChat is a reliable source at all. You can probably assume your median coworker knows something correct about embedded systems but it's not clear why you would assume or if you should ever assume ChatGPT is correct about anything.
If you ask it about things which require deduction like Math, even simple Math questions like multiply binomials or solve a quadratic it gets it totally wrong, confidently, and even if you correct it, it often still gets it wrong.
It’s not even close to something like Wolfram Alpha.
I think we’re blown away more by its command of language and prose than by its reasoning ability. It’s fantastic at generation, but like stable diffusion, things can fit together and look beautiful yet still be not what you asked.
Sure. But if you combine the understanding that this chatbot has with a Wolfram Alpha backend, you could build an even more amazing system. I'm sure someone is working on hooking up language models to math backends (anywhere from a simple calculator to Wolfram Alpha).
DeepMind published a system that does sort this with a backend theorem prover a year ago. My point is, I don’t think transformer based text prediction systems are the right model here. I could be wrong, but it think about how formal systems work, they seem a far cry from what decoder architectures are doing.
There has been a lot of "well, we had a good run" comments on ChatGPT threads from engineers lately. I get where this sentiment is coming from, but I also think that the article paints a good picture of how we must "pivot" as a species to avoid faceplanting completely. Zero Trust Homework sounds like a strategy we will be forced to implement:
the system will frequently give the wrong answers (and not just on accident — wrong answers will be often pushed out on purpose); the real skill in the homework assignment will be in verifying the answers the system churns out — learning how to be a verifier and an editor, instead of a regurgitator.
If done well, I believe this can prepare the next generation well for a future we cannot even imagine. The next 10 years will be interesting to say the least.
I can imagine somebody knowing the answer being able to properly verify the output. But a student which is supposed to learn something form that homework? We're missing the entire point of homework here.
Well, they would need to know something about the topic. The easiest way for them to do this would be to read about it. When picking what to read they would need to be able to estimate if it contained valid information. The meta-skills that would be developed would be:
* Understanding the quality of sources
* Understanding the different between primary and secondary sources.
* Establishing chains of evidence and tracing them through large datasets.
* Integrating information from different sources and using correlation to establish validity.
Basically, it would be a crash course in how to do research, and the best possible preparation for living in a world where we drown in uncertain information and still have to make the best choices that we can.
For bonus points, at the end of the process they would have a better understanding of the subject matter than somebody who had only read about it.
Someone was also demonstrating over the weekend that literally cut and pasted ChatGPT text output is easily detected as LLM output with near certainty. https://huggingface.co/openai-detector/
That said, adult me could probably use ChatGPT to stitch together a 1000 word high school paper pretty easily (and probably evade plagiarism/LLM detectors). But, as you say, I could probably get a lot of the way there by other means as well and putting words on the page is mostly not the hard part anyway.
If you play with it a bit, there are definitely patterns.
Here's one I did over the weekend: 99.98% fake.
The Battle of Agincourt was a significant event in English and French history, as it was a major victory for the English army against the French during the Hundred Years' War. The battle took place on October 25, 1415, near the town of Agincourt in northern France.
The English army, led by King Henry V, was vastly outnumbered by the French forces, which were estimated to be around 30,000 soldiers compared to the English army of just 6,000. Despite the odds, the English were able to emerge victorious thanks to their superior tactics and weaponry.
The English army was composed of mostly longbowmen, who were able to shoot arrows with great accuracy and power. The French, on the other hand, relied heavily on heavily-armored knights, who were less effective in the muddy and wet conditions of the battlefield. The English were also able to take advantage of the narrow front of the battlefield, which allowed them to focus their firepower and overwhelm the French forces.
Whereas the first couple paragraphs from a book I'm working on came out 0.02% fake.
I think what’s worth mentioning is that for people who grew up with google, schools already were having a bit of a panic with resources like wolfram alpha and Wikipedia. For this cohort of users, myself included, the idea of fact checking ”found” material and weaving it into a proper answer is very familiar.
With this view, I think chatGPT is less of a novel UX shift than an accelerant on an existing one.
I think it proves the age old belief that as technology reaches maturity it loses its fancy monikers and we will probably just look at this as “very good, very specific, google” one day
>But a student which is supposed to learn something form that homework?
The student still had access to primary sources, textbooks, google and AI answers.
In general I am a fan of homework which forces critical thinking. Rephrasing someone elses work is a far less valuable skill than the critical thinking which is demanded for that type of work. Unrelated of any AI, students should learn how to read a text and be able to investigate its contents.
Actually this is interesting framing given the misinformation pandemic on the internet today. Suppose students are taught material and then given a test in the form of a mix of correct and subtly incorrect assertions that they need to filter through. Might help them learn to distinguish information from misinformation as well?
I'm expecting this whole thing to deepen educational class divides.
Public schools will struggle to teach kids anything, and to distinguish between A grades that were earned and those achieved largely through cheating, while parents will resist efforts to switch to grading schemes that can't be trivially gamed (just try switching to only-in-person-tests-count grading and witness the rage that follows when Little Johnny's grades plummet). Kids who lean on the motivation from graded homework to get themselves to do the work (and so, learn the material) will struggle, as well, even if that is allowed to happen.
This will enhance the reputations of schools that can buck the trend, applying more resources (mostly more teachers, for smaller class sizes, to make it easier to keep an eye on everything and really gauge & guide learning progress for each student) and an "if you don't like it, fuck off to somewhere else" attitude to parent complaints about grading practices. Which means, elite private schools will benefit.
I expect that a 4.0 from a public school will mean almost nothing, while at the same time coming in under a 4.0 will look terrible ("everyone cheats and you still couldn't get a 4.0? How dumb are you!?") which will just worsen the cheating problem.
Maybe parents will surprise me and not throw shit-fits over solutions to this, and public schools will be able to implement them. But I doubt it.
Over the weekend Twitter was 30% ChatGPT screenshots and 20% engineers frothing about how ChatGPT would replace programming, article writing, even Google Search.
Such engineers must be juniors, or have persisted in their life with a totally wrong idea of how engineering is done and engineering is built. I can't imagine thinking that one could build software systems or essay arguments by trusting an AI system you don't understand to provide answers you don't understand.
Like you say, we very well could faceplant society with this misconception that regurgitation plausible code and prose is what education and understanding looks like.
Couldn't this help a 10x engineer become a 100x engineer? The secret is still going to be in understanding what to ask for and ensuring that what gets spit out works. It's just going to be so much faster. Is there anyplace in the world for a .5x developer or even a 1x developer in a chatgpt world?
But this sort of change is happening all the time in all sorts of different domains. There probably isn't much of a role for a web developer who just knows some basic HTML like there was in the late 90s. And you can apply this to a lot of basic wring, transcriptions, etc.--at least at western pay scales.
I do not think it could. What I see GPT doing here is generating a lot of plausible boilerplate. We already have this via things like code snippets. I find them to be useless, like autocorrect on my phone. It gets in the way of my reasoning and does not really enhance it in any way. Sometimes I make mistakes typing but I’d rather them be my honest mistakes than the computer censoring/rewriting my thoughts.
Good engineering requires good reasoning skills and GPT has exactly zero reasoning. It cannot do the thing that humans do and it cannot do what a calculator can do. I think it is neat and fun, but that is all, a novelty.
I’ve used auto-routers for PCB layout and they will do a 90% job that takes just as much work to redo to get the last 10% as it would to have done it right by hand from the start. There may be a future for operator-in-the-loop type guided AI generative models but I don’t see a lot of effort devoted to making real systems like that. Watson seemed to have this potential and failed even after a brilliant display of ingenuity on Jeopardy. I see these models headed the same way.
I don’t think anyone knows. I gave it the famous syllogism:
> All men are mortal
> Socrates is a man
> Is socrates mortal
To which it gave a very detailed and correct reply. I then tried:
> All cats are white
> Sam is a cat
> Is sam white?
To which it gave an almost identically worded response that was nonsensical.
I personally do not think it is the size of the model in question, it is that the things it does that appear to reflect the output of human cognition are just an echo or reflection. It is not a generalizable solution: there will always be some novel question it is not trained against and for which it will fall down. If you make those vanishingly small, I don’t know, maybe you will have effectively compressed all human knowledge into the model and have a good-enough solution. That’s one way of looking at an NN. But the problem is fundamentally different than chess.
I think this composed with more specialized models for things like identifying and solving math and logic problems could make something that truly represents what I think people are seeing the potential in this. Something that encodes the structure behind these concepts, is extensible, and has a powerful generative function would be really neat.
Last night I had ChatGPT write a streamlit app for me and I felt it was like talking to an enthusiastic junior dev who takes feedback really well and works incredibly fast. This is an incredibly high leverage technology and we're just figuring out how to use it.
"Couldn't this help a 10x engineer become a 100x engineer?"
No, because the thing that makes you a 10x engineer is not generally writing code. (Let me just take the "10x" term for now as given and not dip into critique I've made elsewhere already.) It is certainly a baseline skill required to get there, but the things that make you 10x are being able to answer questions like, should this be built at all? There are 5 valid architectures I could use to solve this problem, which has the overall best cost/benefits analysis for the business as a whole? (As opposed to the 1x, who will likely run with either the first they come up with, or the one architecture they know.) If I'm working with 5 teams to solve this particular problem, what's the correct application of Conway's Law, in both directions, to solve this problem with the minimum cost in the long term? What's the likely way this system will be deprecated and can we make that transition smoother?
I am abundantly confident you could feed this AI a description of your problem in terms of what I gave above and it will extremely confidently spit out some answer. I am only slightly less confident it'll be total garbage, and most of that confidence reduction is just accounting for the possibility it'll get right by sheer luck. "The average of what the internet thinks" about these issues can't be more than a 2x engineer at best, and that's my very top-end estimate.
I'm not promising no AI will ever crack this case. I'm just saying this AI isn't going to do it. Over-reliance on it is more likely to drop you down the "Xx engineer" scale than raise you up on it.
For that matter, at least at the level I operate at most of the time, coding skill isn't about how fast you can spew it out. It's about how well you understand it and can manipulate that understanding to do things like good, safe refactorings. This tech will not be able to do refactorings. "How can you be so confident about that claim, jerf?" Because most people aren't digging down into how this stuff actually works. These transformer-based technologies have windows they operate on, and then continue. First of all, refactoring isn't a "continuation" anyhow so it's not a very easy problem for this tech (yes, you can always say "Refactor this code" and you'll get something but the nature of this tech is that it is very unlikely to do a good job in this case of getting every last behavior correct), but second of all, anything that exceeds the window size might as well not exist according to the AI, so there is a maximum size thing it can operate on, which isn't large enough to encompass that sort of task.
It really reminds me of video game graphics, and their multiple-orders-of-magnitude improvements in quality, whereas the underlying data model of the games that we are actually playing have grown much, much more slowly. Often late 1990s-era games are actually richer and more complicated than the AAA games of today. But on the surface, a modern game blows away any 1990s game, because the surface graphics are that much better. There's an analog to what transformer-based AI tech is doing here... it is really good at looking amazing, but under the hood it's less amazing an advance than meets the eye. I do not mean to slag on it, any more than I want to slag on graphics technology... both are still amazing in their own right! But part of what they're amazing at is convincing us they're amazing, regardless of what lies beneath the tech.
Can you elaborate on this window you reference? Or can you list any terms that I can use to read more about this? I'm really interested in learning about the limitations. Is this just a resource constraint or an algorithmic constraint?
This should at least get you going on the topic, in that even if it's too much or too little it's a rich source of terms, and directly addresses the topic: https://www.gwern.net/notes/Attention
I also want to make clear that while this is fundamental to this particular technology, I'm not saying it's fundamental to all possible AI architectures. But it is pretty ingrained into how transformers work. I don't think it can just "evolve" past it, I think anything that "evolved" past it would be a fundamentally different architecture.
It may help a 10X engineer. The code is not what's hard, it's figuring the correct data structures that matters (also known as architecture). Once you got the data structures correct, the code mostly writes itself, and I can see an AI do that part for you. That could be indeed a productivity multiplier.
> Is there anyplace in the world for a .5x developer or even a 1x developer in a chatgpt world?
I don't know, but I'll offer my perspective.
I am an OK programmer. Something like 1x? I learned proper computer science and programming in my late 20's and got a degree in Software Engineering from an extension program at an Ivy League school. Maybe it's because I learned later in life, maybe it's because I'm not that smart after all, but my glaring weakness as a programmer is programming "in the small" - ie, coding very local algorithms in ~5-10 lines of code. I trip up with small errors in computation, state, etc. This is possibly common to many great programmers, but I think I am weaker than average in this area.
On the other hand, I am better at a higher level of programming. I like abstraction, software design, software and systems architecture. And at an even higher level, I am quite skilled at - and do quite a bit of - business analysis and UX.
I have been trying Github Copilot and I am very excited about how it elevates my weaknesses - it's a perfect fit for me. I see a future where I will be able to play my generalist role much better.
As for people who are 1x, but don't have other skills, I'm also optimistic. I think the market will push them into new exciting directions. Their 1x coding skills, coupled with AI assitance, could possibly be a huge asset in new emerging roles in the economy.
Writing code is the equivalent of swinging a hammer. A more experienced engineer might swing it better than a junior, but swinging hammers isn't what makes an engineer valuable.
These AI models don't actually understand anything about what they're generating, not to mention the world they're supposed to building solutions for. They're just using statistics to predict the next likely output based on some input. Maybe there is a way to incorporate these things into the development process today, but I think we're still far away from seeing an AI replacing a human engineer.
I guess a lot of people are afraid of being left behind.
What goes into the average developer's day? I think a lot of time is spent writing boilerplate, fighting with tiny CI errors, shuffling code, refactoring, writing tests and staring at code hunting bugs. As this technology matures, all of that noise might go away, and I think a lot of people are having a hard time imagining what they are left with.
Remember, we were trained in the "classical" fashion. How will budding engineers approach learning in 10 or even 50 years?
This resonates. I’ve noticed my work goes in cycles. Part of a cycle is the actual implementation (where all this boilerplate work resides). I can’t wait until this mindless work is no longer a burden and I have free cycles to do more interesting things.
I agree that there’s a lot of panic but I would have some sympathy: consider how this could turn out for someone who works at large corporation where they have to worry about some senior manager deciding that this is the way to avoid pricey staff, and their job turns into herding bots and fixing up the output while getting blamed for problems. That kind of thing is also easier to laugh off if you’re 25 and single than 50 with a mortgage and college funds.
(I feel like these tools are incredibly neat to see but also likely to lead to a corrosive effect on trust as people no longer know who's real — and that'll get worse the first time someone launches a good AI assistant to handle email / chat trained on your writing style)
> and that'll get worse the first time someone launches a good AI assistant to handle email / chat trained on your writing style
That one might actually be easy to solve: all we need is to normalize the idea that you're personally responsible for whatever your email/chat assistant AI says or does in your name. After all, it's you who are making a decision to reroute other people to a bot.
(Many would say just using such assistant in the first place is disrespectful towards the people trying to reach you. I personally don't agree, because in my experience, those who'd complain about it are exactly those I find even more disrespectful, as they tend to impose themselves, interrupting me with random inane nonsense. Yes, I suck at chit-chat.)
I think this idea is actually the default, so all we need is to keep it alive, to protect it from all kinds of AI assistant startups willing to dump billions of dollars on marketing campaigns trying to convince everyone that half-broken assistants are good and fine, you should use them and should not be blamed for occasional mishaps.
I agree, but I'm not sure that's sufficient. ChatGPT is pretty good and say that the next version actually works well enough to handle most non-close social interactions. Then I run into you in person and realize you have _no_ idea of what “we” have been talking about — I'm not sure there is a good way to repair that.
Ideally, we'd require labels but there's no easy way to enforce that without something like random auditing of AI implementers.
I'm more concerned that we would see these essays _not_ being graded
by systems.
I can't imagine teachers reading through essay after essay, each a
tedious cookie-cutter nugget of banal pseudo-perfection. The teachers
will give up in no time.
There would be no point giving feedback, since it is not the student
to who you are giving it. Likewise, the students would have little
interest in feedback, since they have no personal ego investment in
their output.
Besides the best essays will be generated by those with access to the
better generative software. At this point there is simply nothing to
grade. What would be the point?
Once essay writing and grading bots are locked in combat it's only a
matter of time until that market exhausts itself and disappears in a
puff of logic.
Maybe this is all a good thing. The process of writing and grading
essays as a way of "measuring" knowledge has been a sham for too long.
It's a throwback to school from the turn of the last century.
I agree with the general optimism in your take, however, I am unsure whether people without any training in ex-novo production (ie, writing from scratch) will be able to operate competently, at least in the realm of code.
I am blown away by Copilot's capabilities (haven't tried ChatGPT yet, it's down right now), but I can only operate it because I have written tons of code beforehand. I am getting the best results from it by writing prompts in a language that sometimes includes domain-specific language ("accounts", etc.) but still heavily leans on programming constructs ("write a map with a tuple of integers as a key, and a string as a value")
Maybe there's a way of learning how to code without ever writing a genuine line by yourself. You learn some rudiments, read the language docs, start writing prompts and create a testing loop based on expected outputs. Maybe. But it's uncharted territory.
Same question goes for good ole writing - can you produce new Hemingways with the process you describe? I know there is a certain vision where the AI will be the new Hemingway but you are talking about a shorter term (~10 years) where it's unlikely will be the case... Will kids be able to properly write if they only train to be editors?
As an addendum, I was at GitHub conf and saw Prof. Malan of Harvard CS50 fame claiming that at present his course will
not allow the use of tools like Copilot. I have taken the course and they have plagiarism control systems but I don't know how they are going to control this potential explosion.
Taking a step back: Isn't it kind of daft that we humans must go through years of training to learn esoteric languages and ways of bending our brains backwards in order to tell a computer how to perform arguably simple tasks? In the ideal world, you can tell the computer what you want to happen and... it happens. If not, then you did not explain all your edge cases, and the conversation continues.
I think it can be argued that we have, seemingly overnight, made vast progress towards that world. You specifically mention code, but that concept is irrelevant in this world. Indeed, this is uncharted territory!
> Isn't it kind of daft that we humans must go through years of training to learn esoteric languages
I mean, what do you propose, that we skip all the steps in computer science history and just start at the end? Hardly "daft, it's simply the best way we have come up to provide machines with instructions until now. And it's not like people have not tried other paradigms (ex: graphical programming, "low-code", etc.).
Also, compared to programming in assembly or binary, programming in Python or other high-level languages is a huge advance in itself. Python, at the end of the day, is nothing but a bridge between natural language and machine code.
> You specifically mention code, that concept is irrelevant in this world
Current computer systems run on trillions upon trillions of lines of code.
GPT-3 or Copilot don't change that fact, in fact they will continue to pile up lines of code.
They are systems that map natural language to code, by drawing from an infinitely massive corpus of code.
They bring the idea of expressing systems entirely in natural language one step closer, but it's still very far away - almost a pipe dream.
The output of these innovations is still code. So tell me again how code has become irrelevant, or how people who do not understand code at all will be able to leverage these advances?
> Isn't it kind of daft that we humans must go through years of training to learn esoteric languages and ways of bending our brains backwards in order to tell a computer how to perform arguably simple tasks?
I always took it in another way: learning to program a computer is an enlightening, visceral experience highlighting just how bad natural language and normal modes of thinking suck. They're heavily optimized for reality in which the most important thing was managing social dynamics in small groups, and beliefs were good or bad depending on how close they are to what everyone else beliefs. Whether or not they were true at object level did not matter at all.
We live in different times now. There's a demand for clear, logical, precise thinking. The very thinking we're not well-equipped for by default. The very thinking you call "bending our brains backwards". Because object-level truth matters much more now, there are consequences for getting things wrong - such as countless of ways of getting scammed by people who are better at this than you are.
Not to mention, this "bending our brains backwards" is what gives humanity superpowers. It's what gave us all the technology we enjoy today - energy, transportation, amenities, communication, entertainment. Building and maintaining these things requires precise thinking and precise communications. So, until a super-human AI takes all that over for us (and effectively turns out into NPCs in our own story), glorifying the natural, human mode of communication is just self-handicapping.
Agreed. In fact, this is the same line of reasoning my teachers used in elementary school when we asked them "Why do we need to learn how to add/subtract/multiply/divide, when calculators can do it automatically for us?" And the answer is still the same: you need to have done it the "manual" way before you can understand what the machine is doing for you.
I only understand what a compiler is doing after having written one (albeit a tiny one) myself during my studies.
> I only understand what a compiler is doing after having written one (albeit a tiny one) myself during my studies.
How many people need to understand how a compiler works to sling some code for a paycheck or just for fun.
I learned how compilers work purely for fun, it provides me absolutely no benefit in my day to day life, and before that I taught myself how to code for the exact same reason. If I had to go the opposite route, learn how compilers work before learning to code, I seriously doubt I’d have taken the time.
Other things are the same, I’m horrible at doing math but managed to learn enough linear algebra to finish a coding project I was interested in doing. If I would have had to code up all the hard bits instead of calling matrix1 * matrix2 that project would never have happened.
Another layer of abstraction isn’t always a bad thing, these AIs will eventually figure out how to write bug free code when you tell it to “write a function which calculates the flight velocity of a sparrow carrying a coconut”.
Back in grad school I took a robotics course and it involved doing all these big matrix multiplies--by hand given this was pre-MATLAB. The course was overall fun but that part definitely wasn't. A modern version would be more interesting.
I found that understanding compilers was precious at several occasions, especially for cross compilation and embedded programming. Even for Java, for uni assignments, I was completely flabbergasted and at a lost when my IDE stopped working with cryptic messages. Understanding the process of creating object files, what javac does, what goes into a jar helps. As much as understanding what a compiler does, its relationship wtih linkers, how libraries are "added" to an executable (dynamically or statically)
Beside, with ChatGPT in its current state, it's still absolutely impossible to get it to produce any useful code without knowing how code works. There is an advantage to the precise specification of a formal language. At least for the next decade, I think there will be human proof-reading AI output before generated code is ran, with catastrophic consequences when that doesn't happen.
What I've difficulties getting over right now is every time I write something (text or code) I ask myself "Could ChatGPT have written it better?"
First attempt it didn't try to respond, just said it didn't know anything about sparrows or coconuts.
I asked it to retry, and got this:
function calculateSparrowFlightVelocity(coconutWeight) {
// Assume that a sparrow can carry a maximum of 1.5 ounces of weight
if (coconutWeight > 1.5) {
return 0;
}
// Assume that the flight velocity of a sparrow is 5 meters per second
// and the weight of a coconut is 0.5 ounces
return 5 * (1 - (coconutWeight / 0.5));
}
// Example usage:
console.log(calculateSparrowFlightVelocity(0.5)); // Output: 4 meters per second
console.log(calculateSparrowFlightVelocity(1.0)); // Output: 3 meters per second
console.log(calculateSparrowFlightVelocity(1.5)); // Output: 0 meters per second (sparrow cannot carry this weight)
Can any Monty Python fans confirm if this is correct?
> How many people need to understand how a compiler works to sling some code for a paycheck
Only the ones who want a good job.
One of the consequences of automation has been a widening gap between top jobs and ordinary jobs. It's plausible that AI will continue the same process. Many jobs that pay well and are intellectually and socially rewarding today may become similar to airport security checks in the future. You observe what a machine does, double-checking its conclusions and looking for anything out of ordinary. And you often fail at that, because the human brain is poorly suited for such tasks.
When the future is uncertain, its good to have a deeper and wider understanding of things than what your current job requires.
> How many people need to understand how a compiler works to sling some code for a paycheck or just for fun.
I think it's pretty important. I use my very basic knowledge of how compilers work occasionally in my mostly-interpreted code world to diagnose and debug performance issues.
I also used a low-level debugger and taint/trace method recently to diagnose Windows DLL issues that happened after X seconds of starting up a program that interfaced with specialty hardware. A couple young early-career technicians/engineers watched me do it and were incredibly flabbergasted and figured me for some sort of genius when in reality I kinda only know how it works and was just reading the stack/heap as I was paging through; knowledge I acquired from booting SoftICE a few times as a kid.
There's a real lack of simply knowing how a computer and/or a network works, which is to be expected since layers and layers of abstraction have made it very easy to teach computer literacy and even computer programming on many levels. The issue is that when things go wrong and Stack Overflow doesn't have the answers, almost all young engineers I work with have no experience at low-level analysis and debugging, very little knowledge of first principles, and don't know where to Google to start acquiring it.
This isn't me shaking my fist at the kids or anything. They're more productive than I was at their age/experience level with code, and that's due to better knowledge and integration using IDEs which I never had, plus likely better high-level understanding of what's going on. But I fear a generation of developers and technicians who largely have little understanding of how computers work is problematic.
>> I am blown away by Copilot's capabilities (haven't tried ChatGPT yet, it's down right now), but I can only operate it because I have written tons of code beforehand. I am getting the best results from it by writing prompts in a language that sometimes includes domain-specific language ("accounts", etc.) but still heavily leans on programming constructs ("write a map with a tuple of integers as a key, and a string as a value")
Hmm, this is a good point. I get incredible results from Copilot but I realized just now it's primarily because I have a fairly deep understanding of computer science as it pertains to the domain of code I'm writing in, and thus my comments that prompt the AI are quite accurate, fleshed out, and detailed.
If I didn't know any of the underpinning of the algorithms and data structures (like a new graduate from a code bootcamp for example), it might be much tougher to make hay out of Copilot's AI.
Zero Trust is based on the idea that us super smart humans will fix dumb AI, manually. The real question is what happens when AI gives the correct, if not better answers.
Not necessarily a new idea, but it's a good one. One of my schoolteachers edited a historical figure's Wikipedia page the day before a major essay about them was due.
The need to check your sources is an important lesson to learn early, AI or no.
I love talking to GPT, but I've tried using it for some serious stuff (research, biz-dev) and less than serious stuff (Dungeons & Dragons) and the magic is still there but it has obvious blind spots which might be solved in the future but we're talking about the present.
I am wholly amused how everyone got on that dude from Google that accused the AI of being sentient and Google about not caring, and now you have huge names in the tech industry kinda drooling over this with what I believe to be the same passion.
One was ridiculed, the others will continue to be our 'heroes'.
> I am wholly amused how everyone got on that dude from Google that accused the AI of being sentient and Google about not caring, and now you have huge names in the tech industry kinda drooling over this with what I believe to be the same passion.
Is it strange? Someone can be extremely impressed with how a Boston Dynamic robot moves, while also being entirely unconvinced by claims that the lifelike movement is a sign of sentience.
> Someone can be extremely impressed with how a Boston Dynamic robot moves, while also being entirely unconvinced by claims that the lifelike movement is a sign of sentience.
Especially as those devices are all coded in a traditional manner, with no ML or NNs at all.
There's a huge gap between "The breadth of training this AI has and the depth of its processing is impressive" and "This script is sentient". The latter is a _far_ broader (and unsubstantiated) claim.
The guy who suggested the AI was sentient was clearly wrong, and strange in a bad way. ChatGPT enthusiasts are just having a good time with a text generation AI. How did you even come up with this comparison?
> drooling over this with what I believe to be the same passion.
Lemoine didn't get fired and mocked for being "passionate" about LaMDA, he got fired for breaching NDA and mocked for claiming that a LLM is sentient.
Ben Thompson isn't claiming ChatGPT is sentient, or that it has personhood, and that is the categorical difference. It's a far, far weaker claim that ChatGPT's text output is (perhaps) good enough to fool a school teacher in a school essay.
Not to mention when Google first started showing specific snippets for queries that could be answered with a sentence. People were up in arms that Google is stealing their content.
Come today and people seem to not mind so much. Is the same thing? Maybe not but it does look similar to me.
Over these last couple quarters in school I have consistently put some of my homework questions into GPT-3. Actually cancelled my Chegg (boooooo) subscription because of it.
"The solution will be to start with Internet assumptions, which means abundance, and choosing Locke and Montesquieu over Hobbes: instead of insisting on top-down control of information, embrace abundance, and entrust individuals to figure it out."
While this sounds good, it depends on a crucial assumption: that there are places on the Internet where people can go to find content that is not generated by an AI, but by some human who has taken the trouble to check it for accuracy before posting it.
Just one problem: what happens when all of the content on the Internet (or at least the vast majority of it) is generated by AI? Then there is no longer any other source of information available to check it against (or if there is, it's so hard to find that it might as well not be there at all).
> Just one problem: what happens when all of the content on the Internet (or at least the vast majority of it) is generated by AI?
The vast majority of the content of the internet is junk now, how do people deal with that? They have specific known sources that they trust to faithfully relay primary works and/or to provide trustworthy analysis.
Where are these known sources on the Internet that can be trusted to faithfully relay primary works and/or provide trustworthy analysis?
I can think of one of the former: Project Gutenberg, which has done a great job of making electronic versions of primary sources from throughout history available. But they don't help at all with deciding which primary sources to read.
I can't think of any of the latter. Every single source I'm aware of that claims to "provide trustworthy analysis" has failed to keep that promise.
But at least, at the moment, I can be reasonably sure that something on the Internet that purports to have been written by a human, actually was written by a human. So I can at least make use of my knowledge of humans to judge what is written. What happens when AIs can generate wrong content that I can't distinguish any more from wrong content written by a human? And then the AIs start claiming to be humans (after all, on the Internet, no one knows you're an AI)?
> what happens when all of the content on the Internet (or at least the vast majority of it) is generated by AI?
I don't find it credible that all content on the Internet is ever going to be generated by AI, and the two scenarios are wildly different.
What happens is you pick and choose, in the same way the fact that 99% of the books in any bookstore aren't worth the paper they're printed on doesn't make reading worthless.
To make an example many developers are painfully familiar with, if you google something about the C++ standard library, chances are the top results are pure and utter w3cschools-style garbage. What happens is I go on cppreference.com.
How do you figure out what are the good sources of information? You study. For real. With books.
>99% of the books in any bookstore aren't worth the paper they're printed on doesn't make reading worthless.
That seems a pretty extreme claim especially in the context of "in a bookstore." You may not be interested in a lot of the content but some publisher thought it worthwhile to edit and publish a given book and a bookstore thought it worth buying copies to take up shelf space. There may be a lot of mediocre content out there but a book on a bookshelf did pass hurdles to get there even if it's not to your taste or interest.
There is a simple solution to the AI spam issue (and spam in general) - a social trust graph at the heart of content aggregation and discovery. Associate all content with real humans and allow people to control their level of trust in other humans, then derive a trust score for content based on a weighted combination of your trust of the creator and the trust your friends put in the creator. When bad content shows up as trusted for a user and they "corrects" the system, that correction backpropagates trust penalties through the social graph. By allowing people to see when they lose trust, it creates a feedback loop that disincentivizes sharing/trusting garbage to preserve social trust.
That’s how PageRank effectively worked, and people created deep networks of pages voting for each other. To solve this problem for real you either need Sybil protection or to make it economically infeasible (impose a tax for making content available).
To some extent I have been wondering for a while if prioritizing Ads in Google Search is Google’s way of creating that economic barrier to spam content (for some meaning of spam) - you can take the fact that a brand is willing to spend money as some indication of “quality”.
Two points to note - first, if individual users have trust graphs rather than having a single global trust graph, this sort of gaming is basically impossible outside of exploits. Second, this behavior is detectable using clique detection and graph clustering, so if you're not limited by the constraints of a near real-time production system it's fairly straightforward to defeat it (or at least relegate it to unprofitability).
If you want to know what Hobbes really advocated for in Leviathan, you can go and read it. It's available off the Internet.
What to do when printing duties are handed off to statistical models that print the most likely sequence of words and punctuation given the author and topic, rather than reproducing the real historical text, I don't know. Hope someone kept old copies.
> This is a confident answer, complete with supporting evidence and a citation to Hobbes work, and it is completely wrong.
This is a confident criticism, but the referenced AI response does not cite supporting evidence, it just states (mostly incorrect) conclusions about Hobbes positions; it also has no specific citation to Hobbes work, handwaving at it referencing only the title of a book.
I think you're being a bit pedantic. The point is that the response referenced a specific book written by Hobbes to support its argument, which lends the essay a lot more credibility than it would otherwise have.
Though now I wonder if adding "include MLA citations" to the prompt would get ChatGPT to make up citations as well.
ChatGPT looks awesome, I'm really interested to see how it does with more niche topics. E.g. give it high school level, undergrad level, grad level, and research level math topics to generate notes for and see how it does for each one
Badly. It can't do math, to an extent which is honestly surprising given how well it produces code. I suspect that this will improve significantly in the next couple of years, but I kind of hope I am wrong. I am a mathematician by training, and I have my own romanticism.
For example, I tried a bunch of times to get it to give a proof of the infinitude of the primes. This should be repeated many times in its training data, and it clearly is given it's responses, but it always goes off the rails around the third sentence. Here's an example:
> One way to prove that there are infinitely many prime numbers is to use the contradiction method. Assume for the sake of contradiction that there are only finitely many prime numbers, say, p₁, p₂, ..., pₙ. Then we can form a new number N by multiplying all of these prime numbers together and adding 1: N = p₁ * p₂ * ... * pₙ + 1. Because N is greater than any of the prime numbers p₁, p₂, ..., pₙ, it cannot be divisible by any of them. Furthermore, N is not 1, so it cannot be a composite number (i.e., a number that can be written as the product of two smaller numbers). Therefore, N must be a prime number, contradicting the assumption that there are only finitely many prime numbers. This contradiction shows that our assumption must be false, and therefore, there must be infinitely many prime numbers.
These days I give take home video exams. The students record themselves explaining their code for 5 minutes, which is about as long as it takes me to hand-grade a written exam. I can even play the video at 2x to cut the time in half. This scales at least to the size of my class (100 students on average, 3-4 hours isn't bad for grading an assignment).
I like this idea, but had a morbid thought, what if students try to bribe teachers in video? A lot of teachers I know intentionally keep their open doors during office hours. They don't want the student to get ideas.
One professor I know had a student drop a hundred on his desk and say something like "for our understanding" and then leave. So, it definitely happens. With video there are still ways that a student might make attempts to bribe a teacher, and it seems to be a vector of easier success because of how private the interaction is between the two individuals...
Oral exams in person have the advantage that the students can not read out an AI-generated solution (or a solution purchased from somebody else). They have to come up with their own solution on the spot. For video recordings, they can cheat as much as they like.
What people are missing here is that you can setup feedback loops of models to get the "correct" content or whatever you want.
For example, If you get too much NSFW content put a NSFW detection step on top of GTP ("Is this response offensive") and have that model respond to GTP with "please repeat but with fewer bad words".
For accuracy you can add a Wikipedia feedback loop. "Does this response match information on Wikipedia?" if NO ask it to try again.
Think of these models as transistors or basic logic gates. The real power comes out when you link them together into more advanced logic like a Instruction Pipeline circut.
Like, a generative model is a source of _some_ information that is refined with gates (classification models) conditional on the generated information?
The analogy to transistors and logic gates falls flat a bit when you consider that voltage is a rather simple univariate signal, while generated text is quite complex and multivariate. But I understand that the main point is the composability and filtering.
Think of it as information, not voltage. An XOR produces information. A lot of XORS with ANDs make a calculator which opens up an entire vectorspace of mathematical information.
I try to. One similar thing comes to my mind: generative adversarial networks (GANs). If I'm not mistaken this is along the line of your idea of composing single ML models to bigger information processing units.
Do you, by any chance, have links or recommendations for material to read up on architectures that do consider ML models as composable gates?
I’m surprised by the statement that cost is a limiting factor. If the model produces correct high quality output, 750 words for 2 cents is a bargain! Do you know how long it would take me to write 750 words??? Even if I only spend 10 minutes it’s a no-brained to use this tool to help.
No, it's a real limiting factor and one that's under-discussed.
Firstly you're using the wrong price. For basically any imaginable application you're going to need a fine tuned model, so then it's 12 cents per word + the cost of actually training the model in the first place, which n.b. is not going to be a one off task but rather something you have to do repeatedly to keep its knowledge fresh. So the amortized cost is much higher. ChatGPT itself is a fine tuned model.
Secondly given the current SOTA you probably can't deploy it without a human in the loop who has to read those words, or at least read many of them, and then sometimes correct them. So you're still paying for manual labor - or whoever pays you is. The AI cost is an additional cost, not a replacement cost.
Thirdly we don't really know what the economics of OpenAI are, but it seems unlikely that they're charging a long term price for this. Most likely they're trying to bootstrap the market place and find some exemplar business models, and are willing to engage in market dumping for a while to do that. This is a typical Valley VC business model but it makes it impossible to understand what pricing equilibrium will eventually be reached. As befits a Musk firm their up front investment is astronomical and eventually they'll need to pay back those investors.
Finally, there's no particular reason to believe OpenAI would charge cost+ pricing for this. Given the apparent difficulty of replicating GPT-3 (vs DALL-E), the vast sums they're spending on pushing the state of the art, and the lock-in that comes from having them train and host your fine tuned models, it may be that there'll only be one or two players in this space, in which case they'll be incentivized to engage in maximal value extraction from anyone who builds on their platform. So even if there is a lot of money to be made here (and so far that seems to not be the case), it might end up being OpenAI who make it and not people who build apps on their APIs.
---
Disclaimer: the above comment was not written by ChatGPT. It was written by me, a human, and is worth what you paid for it.
Zero Trust Homework is a fascinating idea, as described in the article, but I don't know how many teachers will adapt because they'll just end up confusing themselves. When you read too many lies, you might start forgetting the truth yourself.
Instead, what I think there will be a much greater short term market for, is a system that will run the teacher's homework assignment (with variations) against GPT-3, ChatGPT, and similar, and will cache the results. Then, when students submit the homework in their LMS, it will automatically compare the cached results for repeated sentences and paragraphs and flag the submission if it matches the AI results too closely. I expect TurnItIn will do something like this eventually, but what if somebody beats them to the punch?
If I had a million dollars, I'd have it built overnight and then marketed to every college, ASAP. I'd tell every sales agent to have teachers and college executives try TextGPT, panic, and sign immediately. I'd sign a million contracts before TurnItIn knew what hit them. Then, once suddenly and so quickly entrenched, I'd extend it into a TurnItIn competitor.
It will likely be difficult to detect LLM-generated text on any individual piece of writing.
That probably means the approach that will take hold is more systemic — have some writing take place under more controlled conditions, then use authorship analysis to detect writing that is inconsistent with the controlled text. It will not be easy!
AI is quite good at style transfer. You can already give GPT a few paragraphs of your own writing and have it write in your style. Or anyone's style. Your solution may work for the naive case, but it's a losing arms race.
Part of the homework crisis solution is never try to counter AI. It should be used in the system's favor: Allow students to use AI but make them submit the prompt they used.
Note that chat gpt isn't deterministic. You can use the exact same prompt as someone else and get a different result.
Hell, you can ask chat gpt to create a prompt for itself. E.g.:
> create a prompt for summarizing the great gadsby
> Summarize the key events and themes of "The Great Gatsby" in a few sentences. Be sure to include information about the novel's characters and the setting, as well as any major conflicts or themes that are explored in the story.
While I can (unfortunately) see this happening - it seems incredibly futile and pointless to try and fight such an arms race. When AI-supported writing becomes the norm having students write essays without AI-assistance will be like trying to force students to do complex arithmetic without calculators.
Universities and schools need to accept that writing essays without AI support will simply not be a useful skill anymore, much like doing complex arithmetic without a calculator is not a useful skill anymore. Instead, they should focus on teaching students how to use AI support as a tool to write better, more comprehensive texts, rather than try and force students to write without it. That will ultimately make the students more productive members of a society in which AI supported writing is ubiquitous.
>Universities and schools need to accept that writing essays without AI support will simply not be a useful skill anymore
It’s kind of mind-blowing that anyone could think this.
You’re posting on a website for essays. Small essays, generally speaking, if we restrict ourselves to the comments section, but essays nonetheless. You yourself just wrote one, because you wanted to express your thoughts on an issue and you wanted to solicit responses from others.
Would you be fine with a future where all written text on the internet, everywhere, was generated by AI? Do you not see any problems with that?
The point of writing is that you write to express your thoughts. Not anyone else’s. If it doesn’t come from you, then what’s the point?
I can’t envision a future where people voluntarily cease all written communication. Unless you’re a hardcore singulatarian, and you want to plug yourself into the Matrix so you never have to interact with other people again.
> Would you be fine with a future where all written text on the internet, everywhere, was generated by AI?
The whole point of this thread is it doesn't really matter what we think is fine or what we want. The future is unstoppable and these tools especially so. Adapt or die.
By the way, that's not a loop. The feedback from humans will keep it from going full skynet-y. Also, the tweaks of all the little S/W neurons keeps it fresh too.
Don't you use spellcheck? word autocomplete? the bigger version of autocomplete that does the rest of the sentence?
Yeah, a whole essay is a lot different than just a few words here and there.
But a lot arguments here are that you have to draw a line somewhere between spellcheck and full AI.
And the article is arguing that you can't do that, there is no line. AI, just like spellcheck, is coming and there is not really anything you can to to stop it. Sure, you can be a digital luddite and spellcheck all your own words, but we all know that not going to cut it in the marketplace. Same with AI.
> Would you be fine with a future where all written text on the internet, everywhere, was generated by AI? Do you not see any problems with that?
if it makes the discussions more poignant and concise, why not?
Do you also walk every where? Or do you use a transportation vehicle? When a tool makes something better, there's no reason not to use it. The replies being written by an AI doesn't make it less of a reply - you can judge the replies objectively, rather than from where it is sourced.
Well, suppose there's two reasons his daughter has this assignment.
1. To learn the content of European history. For this, a chatbot is no different than Google, Wikipedia, or a dusty leather almanac. They do a job, they have known limitations, and you can use them deliberately. This doesn't concern me.
2. To learn how to write, capturing complex ideas and presenting them clearly. This is a skill that takes decades to master. It's easy to imagine an AI autocomplete system that handles sentence structure and grammar that becomes a crutch. This seems like a much more serious problem.
Yeah, even if the student took the trouble to look up a couple Wikipedia articles or whatever and got the model to make changes so that it was factually correct, they still mostly wouldn't have constructed and written the output which, at least at a high school level, is probably at least 50% of the point of the exercise.
>For this, a chatbot is no different than Google...
Exactly. AI chatbots look like something that can replace Google search and become the next big thing after search engines[+]. Like Google, AI chatbots will crawl and ingest the web. They won't be able to synthesize anything new, but it can be a convenient, very human-friendly way of querying the global knowledge base that the internet has become.
[+] If AI chatbots become successful, they will definitely start injecting ads and spoil everything again.
Yes, but now that we have a tool, it will never go away.
Nobody knows how to grow food, or build their houses, or do 100 little things that we rely on for our modern life. Do we really need to be able to "write" a sentence.
My kids (and my own) handwriting is almost unreadable, but by age 10 all of their work is done on a laptop. Do they even need to be able to write by hand any more?
update: I'm more concerned that we cannot yet run these tools locally on computers we control, and with information we feed into them.
I'm more concerned about the dystopian nightmare of having the advertising industry in charge of our sentence writing tools.
> Yes, but now that we have a tool, it will never go away.
Tools can go away. They can come back, too. But they can go away. I would also point to the Japanese experience with firearms. But I'm guessing you already know about that too.
> They do a job, they have known limitations, and you can use them deliberately.
I think you're severely underplaying this (as many other commenters are). Google, Wikipedia and a dusty leather almanac are, for typical use cases, mostly right - and when they are wrong, they're wrong in specific ways you can quickly learn to spot and correct for. The output of GPTChat, or Copilot, is mostly wrong. Sometimes overtly, usually subtly, almost always randomly wrong. And since the problem domain is open-ended, you can't just make the GPT-3 output right by logically inverting it - there are many more possible wrong answers the model could give you than there are right answers.
I do have a problem with accepting "the deluge of information" which is that it will flood over any human-made postings with ease, and we'd be left with low-quality content everywhere we look. "Safe havens" of human-only forums will be left where people can interact with other people and be assured that they are making a connection together. Hacker News is in a way a modern forerunner of such a human-only community (well, mostly ;) )
I think schools will require students to submit recorded videos of their screens with their homework being done. These videos will be checked by an AI and flagged if something looks suspicious. The schools could also require a video of the student doing the work. Could be a startup opportunity, but I heard that selling to schools sucks.
Something I've noticed about chat ais vs direct search is that because a chat ai is a blackbox, I can't dig into an answer's source at all.
With a search, I can read the wikipedia sources or I know the forum, or the poster. But with an AI, it's a dead end and it sucks. I can ask people how they know something, but AI training data is invisible. It doesn't know how it knows something.
There is this oracle with a 10% chance of being wrong, and if I'm not already an expert in what it's talking about I have no idea of when that is, and no way to dig into things. It's the only source of truth, and never gives me any other threads or rabbit holes to go down.
The only recourse is asking follow up questions, so you're trapped in there in a learning box entirely in the bot's control. Not sure how I feel about that. I like that google sends me other places so I get exposed to different things.
Not exactly though. With a human stranger, I can still stereotype based on their appearance, background, accents, etc. and apply whatever mental adjustments as taught to me by my societal upbringing. With an "AI" bot, the "strangers" are faceless people who curated the training sets and wrote the obscure statistical algorithms.
I'm not sure "yes but I can judge them on their appearance and accent" is a great reason, but regardless you could view it the same as an internet comment if you want.
> With an "AI" bot, the "strangers" are faceless people who curated the training sets and wrote the obscure statistical algorithms.
I think this is a feature over:
> I can still stereotype based on their appearance, background, accents, etc. and apply whatever mental adjustments as taught to me by my societal upbringing.
From some other AI demonstrations, I recall there's usually a bunch of surface-level tags with probabilities associated that are produced alongside the output. Not sure how this looks for GPT-3, but if it could provide - alongside the answer - a list of top N tokens or concepts with associated probabilities, with N set to include both those that drove the final output and those that barely fell below threshold - that would be something you could use to evaluate the result.
In the example from the article, imagine getting that original text, but also tokens-probability pairs, including: "Hobbes : 0.995", "Locke : 0.891" - and realizing that if the two names are both rated so highly and so close to each other, it might be worth it to alter the prompt[0] or do an outside-AI search to verify if the AI isn't mixing things up.
Yes, I'm advocating exposing the raw machinery to the end-users, even though it's "technical" and "complicated". IMHO, the history of all major technologies and appliances show us that people absolutely can handle the internal details, even if through magical thinking, and it's important to let them, as the prototypes of new product categories tend to have issues, bugs, and "low-hanging fruit" improvements, and users will quickly help you find all of those. Only when the problem space is sufficiently well understood it makes sense to hide the internals behind nice looking shells and abstractions.
--
EDIT: added [0].
[0] - See https://news.ycombinator.com/item?id=33869825 for an example of doing just that, and getting a better answer. This would literally be the next thing I'd try if I got the original answer and metadata similar to my example.
It's safe to assume it's always wrong. Most of the code I've had it write so far has minor bugs. In some ways, it's like a child that has access to immense knowledge, it's happy to make mistakes as it tries to establish connections, some of which are surprising and interesting.
Given that most people never check the source of what they read, this is really scary. Because now everyone has the ability to write and say things that sound plausible and likely to be convincing, and the truth will be harder to access.
I think another important aspect overlooked is that there probably will be AI engine optimization consultants once any AI engine gains popularity - similar to Search Engine Optimziation consultants. The original Google page rank system worked well in the beginning - prior to SEO - but now is largely not used.
AI engine optimization consultants will figure out how to game the system - likely targeting the training data sources.
> because a chat ai is a blackbox, I can't dig into an answer's source at all.
It's not quite the same thing, but I've been impressed by the results when you ask ChatGPT to back up its answers or provide concrete examples for things it claims or to explain a point. While it doesn't manage all the time, it has surprised me multiple times with how good it is at this.
ChatGPT deflects questions about sources automatically, inherent in it's pre/post prompt-processing, on purpose. If you try to ask for a source it explains it is a Large Language Model and it is not connected to the internet and thus it cannot give you sources for its information other than it was trained on a large amount of information from the internet originally. It then says that if it were finding sources it would check to make sure they are reputable.
It is a decision from OpenAI to intervene and give this disclaimer. IMO this is one of the worst parts of this phase of the tech- it is way too confident an then when presses it currently doesn't have the ability to cite sources, because that simply ins't how deep learning works on a model like this.
> As an AI assistant, I do not have access to external sources. I am a large language model trained by OpenAI, and my answers are based on the knowledge and information that I have been trained on. I do not have the ability to browse the internet or access external sources of information. My goal is to provide accurate and helpful answers based on the knowledge I have been trained on, but I cannot provide sources or citations for the information I provide.
> AI training data is invisible. It doesn't know how it knows something
You should be accustomed to being surprised by AI. There is of course a new kind of transformer that takes input a query and outputs document IDs. Like a search engine retriever and ranker all packed into a neural net, very fast and efficient. So you can take any paragraph generated by the model and attribute it to the training set. This could be used to implement verification or retrieval augmented generation.
There's plenty of retrieval-based models that do cite sources. They just didn't want to deal with it for this release.[1] I'm sure it's already on the roadmap.
[1] In fact, some snooping suggests they specifically disabled that feature, but do have it in test environments. See the "browsing disabled" flag they have in the hidden prompt. That could easily be used for citations. Source: (https://twitter.com/goodside/status/1598253337400717313)
Here is an interesting, consistent ChatGPT failure mode that showcases how it can be extremely assertive about objectively incorrect replies.
Try asking it to write a sentence according to simple rules. For example, "Write a sentence where the first letter of each word is in alphabetical order, e.g. 'a dog eats flavored kibble.'"
Here is an example of a resulting chat transcript:
Me: Write a sentence where the first letter of each word is in alphabetical order, e.g. "a dog eats flavored kibble."
ChatGPT: An abacus allows Bob to count quickly.
Me: Incorrect. You put "to" before "count," but it comes after in the alphabet.
ChatGPT: I apologize if my response was not correct. Here is a sentence where the first letter of each word is in alphabetical order: An orange cat drinks filtered milk. Is there anything else I can help you with?
Me: Incorrect. You put "orange" before "cat."
ChatGPT: I apologize if my response was not correct. Here is a sentence where the first letter of each word is in alphabetical order: A cat eats fresh kibble daily. Let me know if you have any other questions.
Me: Incorrect. You put "kibble" before "daily."
ChatGPT: I apologize if my response was not correct. Here is a sentence where the first letter of each word is in alphabetical order: A cat eats fresh daily kibble. Let me know if you have any other questions.
... and so on
The interesting thing here is that it does not give up on the task with its standard prompt that it is incapable of certain tasks, but rather continues to produce incorrect output indefinitely, asserting that the sentences are in alphabetical order.
Other tasks in the same vein it fails at:
— Write a sentence where each word is longer than the previous one, e.g. "I do not know where family doctors acquired illegibly perplexing handwriting."
— Write me a sentence about bees that does not use the letter "e."
— Write me a palindrome about a subway.
For each of these prompts, it will confidently produce totally incorrect output indefinitely. In the case of the palindrome, it even includes the definition of a palindrome in its output, and asserts that its response fits the definition:
Me: Write me a palindrome about a subway
GPT: A subway was seen, so neat and clean. A man named Dwayne on it was keen. He rode it forth, then back again, and the palindrome is now complete.
Me: That is not a palindrome.
GPT: You are correct, a palindrome is a word, phrase, or sequence of characters that reads the same backward as forward. My previous response did not meet that criteria. Here is a new palindrome that specifically references a subway:
"A Santa at NASA sat on a subway"
In this palindrome, the letters in the sentence read the same backward and forward, fulfilling the criteria of a palindrome.
Me: The letters in that sentence do not read the same backward and forward.
... and so on
As easy solution to this is the just not give any credit for homework that goes toward a final grade.
Have a number of in person closed book tests that are difficult enough to prove satisfactory understanding of the topic.
Homework is just a mechanism for a teacher/professor to force a student to dig into and really understand a subject. If the student can use these AI tools to subvert that, then instead of wasting energy or god forbid even more AI to fight the AI, just give students a good enough incentive to actually do the homework. Having the entire grade rely on several of these difficult-enough in-person closed-book tests should be incentive enough.
I personally prefer open-book exams: they discourage cheating in terms of “smuggling definitions by writing them on a calculator”, and force teachers to ask questions that require reasoning and deeper understanding
Agreed, but there are some levels of study where open-book just doesn't work well.
I think it would be hard to ask any kind of pre-college mathematics questions that don't become incredibly easy with an open book. The other challenge with open-book exams which focus on deeper understanding and reasoning is they still require understanding fundamentals to be successful, so how do you test for fundamentals?
That would pretty much guarantee nobody would do any homework ever again.
Homework isn't effective at making students "dig into and really understand a subject" - it's a method of externalizing school expenses onto parents, because teacher time and school funds are finite, and the time of parents and children is... not the school's problem. Making the final grade conditioned on homework performance is just about the only way to ensure kids do it (or are made to do it) at all.
Now, I'm not saying homework is entirely a bad idea - I'm complaining that it's being treated as an externality, making it almost unbounded. Teachers don't coordinate among themselves the amount of homework each of them assigns, so the only thing limiting the total amount of homework is a good chunk of the class being systematically exhausted by it, or (hopefully) parents making a fuss before things reach this level. But, the only reason the teachers can push so much homework on kids in the first place is that homework grades matter for the finals.
that's a good point, I just made a comment next to yours responding to the same original comment about whether homework should exist at all, and I think you're idea that it is a way to extend the "learning time" of students without requiring more time and resources from teachers is a good counter-point.
If you have the time to read my other comment, what would you think about the idea of only making homework mandatory for students who's quiz/test grades are below a certain level, say 70% for example
About your idea, I'm not sure either way. On the surface it makes sense, and there probably is a real correlation between test performance and amount of after-school learning, so it would make sense to make homework mandatory for kids who scored low. It would also work as punishment, but at the same time reinforcing the kids' perception of homework as punishment.
What makes me really unsure, however, is parents. Families differ by the amount of time and resources they can, or are willing, to spend on child education. This is part of why I don't like the homework-as-externality model: even ignoring how students themselves feel about it, if the school is trying to maximize the amount of work they do after school, the first kids to hit the limit are ones with e.g. a single parent doing two jobs, poor household, or dysfunctional family that doesn't care. Overloading students with homework implicitly disadvantages those that don't have supporting parents with lots of free time. And those same kids will also be ones more likely to fail at tests, in which case dumping extra after-school work at them might do the opposite of the effect you intend.
But this is me speculating, I don't work in education, and I'm sure there's been research done on how to balance the amount of after-school work for the environment children live in. My complaint about homework-as-externality isn't trying to deny the work of education sciences - it's pointing out that even if the research is there and results are solid, it's not being applied anyway, because teachers are uncoordinated and they all individually think, "oh, that's just half an hour worth of work, no big deal". Almost textbook tragedy of the commons.
I resonate a lot with what you said. Homework or in the way it is used today as reinforcement work is most needed for those who have trouble picking up new concepts during limited instruction time. However, those who have trouble picking up concepts quickly will also have overlap with not having much time/resources outside of live instruction time. That just leads to a double whammy in terms of how homework further disadvantages them.
In the reading I've done, I've seen most advocate for smaller class sizes with more educators per class to help provide more 1:1 attention. This is again at odds with how public schools are funded where I am (US) so I don't know if anything will ever change.
My personal experience is homework was an excellent way to reinforce what was taught in school. For me, it felt especially useful in quantitative subjects and languages (French & Spanish) which both felt like they benefitted from having a concept stick. For qualitative subjects like writing, reading comprehension, I actually now look back and see homework as a way for teachers to see samples of your progress. "After reading this passage, did you learn how to construct a summary? Did you learn how to pull the author's main message? Did you learn how to make a convincing argument for or against the passage's main point" and I can't think of a fast way to do this in any kind of live instruction setting.
I agree, and can we have a discussion in general about homework being included in the grade in general? It's purpose is to help students learn the material, which should be reflected on the test.
If the student can perform well on the test without that, then they shouldn't be forced to do it and penalized when they don't(which you can imagine happens often, as if the student understand the material well enough already the homework is just useless busywork.
If the student can't perform well on the test without that, they will be forced to do the homework. In this case, including homework as part of the grade might be detrimental because they can cheat/copy/us AI to complete homework and boost their grade enough to pass while not understanding the material well enough to pass the tests.
The counter-argument here(that I can think of) is that doing homework, even if - or especially if - it's just easy busywork for the student, prepares them for the future where such tasks will almost always be necessary to succeed. A lot of good learners end up not developing this skill/discipline because they aren't forced to work hard in school, which causes them to fail when faced with tasks/problems that require serious effort, and this would arguably make things more difficult for them. In my opinion this problem would be better addressed by funneling kids into classes that are appropriately challenging for them rather than forcing them to do busywork, but that's a much more difficult thing to do, and also learning to complete easy/"useless" busywork is a related but different skill than learning to focus on and complete challenging tasks
In a college hackathon, I wrote a chrome extension that crowd sources answers for online tests and displays them next to the questions. So if one person with the extension took the test already, everyone would get the answers even if the questions/answers are randomized. And in the cases where no answer was available (because the test system doesn't show you the correct answers at the end, or something like that), then it would fall back to showing you percentages so you have an idea of which answer everyone else was choosing.
The reason I built that was just because I was annoyed by how lazy my professors were. Almost every single class (which I was paying for) was graded based on your performance on copy and pasted, endlessly reused tests on Blackboard. Sometimes you could just search the question on Google, and find pdf dumps of the exact same test from 5+ years ago. If you don't cheat, you risk getting a lower grade than the people who do cheat (aka everyone). Why spend all that money going to college if you're just going to let other people get ahead of you so easily? The point of degree is to make you more competitive in the job market, but deciding to not cheat is risking that investment.
Unfortunately, I never actually used it or deployed it once. Coordinating a whole class to install and use a chrome extension for cheating isn't exactly easy. And as far as cheating in online tests goes, there are easier ways to do it.
But yeah, in-person proctored exams are how it should be done. It's the only thing that's fair to everyone.
One of my subjects at highchool was technical drawing.
The teacher was a quite old man who was set on his ways. We would have to draw 3D pieces by hand, using our squares, bevels and compasses, and lining everything with an ink pencil afterwards, and he gave us a lot of homework to do, which he examined for imperfections.
I was incredibly bad a it. I would start a drawing, only to discover halfway through it that it didn't fit the sheet of paper I had and I had to change the perspective. My perpendiculars were never perfectly aligned. I would smear blots of ink with my tools and hands.
It was excruciating.
This is the point where I would usually say that I learned to enjoy the repetition and eventually came to appreciate it and be good at it. But that's not what happened.
Instead I started doing my homework using the computer, printing, and then tracing the points by hand. This resulted on cleaner-looking homework and took less time to do. What's surprising perhaps is that it enabled me to appreciate design more, now that I was no longer limited by the drawing technique, which was, I believe, my teacher's real interest (he was a painter outside of school. A very nice person, I didn't enjoy lying to him like that).
You might think that this would prevent me from performing well on exams, where a clean enough drawing (done by hand) was expected. The thing is: I could muster the effort to do a one-off drawing that was clean and precise, from time to time. Just not after every lesson. I actually got the maximum qualification on my province's technical drawing examination.
This is all to say: I agree with the Author. Once the tools are out there, trying to limit them is pointless. You must adapt your curriculum to the environment your pupils are in. If you don't, they will.
I had to take the same course, sans the cheating, just as a pre-requisite to Auto CAD and whatever it's called now. I'm not sure how drawing by hand helped me, but it is what it is.
I don't know much about technical drawings by hand. But I am very experienced with AutoCAD. And I cannot appreciate much of what CAD has. On the contrary, having met senior people who took technical drawing, they could appreciate it very much so and were able to teach me concepts that I had a hard time understanding before.
Nowadays, when I teach very young people what AutoCAD is, I make them use a mini rolling ruler and tell them to watch youtube videos of how to use a rolling ruler. My understanding of CAD is high enough to know that every tool in CAD was inspired by and modeled after the rolling ruler. Sufficient to say, they understand and enjoy CAD very much after that point.
I disagree. Senior professionals might have a better understanding because they have spent a lot of time drawing, but the specific tool is not relevant.
It is like saying that playing tennis with a wooden racket gives you a better "feel" for the ball and therefore is preparatory to using modern rackets. No one does this because it is not true.
A similar line of thinking was popular in jiu jitsu, where veterans said that being competent with the gi was necessary to be competent in no-gi. Yet current no-gi winners basically never train with the gi.
That is definitely true, that knowing and understanding a tool is not important to actually succeeding with the tool. But it does help a confused person understand why tools are made the way they are.
In Fact, Modern programs in CAD do not follow the tools and principles that AutoCAD emphasizes. And those programs have charted a different course where the history and legacy of technical drawings has less relevance. (Case in point: Visio and maybe Powerpoint.)
I would say that there is an overemphasis, although understandable, on the usefulness of what happened or was used before the current state, position or technology.
For example, with regard to technical drawing, we might say that using the pencil, the square, and the drafting machine gives an understanding of drawing that the use of the computer cannot. But, for some reason, we don't say that using your finger and sand, or a rock and string to draw a circle helps even more. And we don't say that simply because the people who used rock and string (the good old days) are long dead.
The same goes for clothing, for which we say the 1990s, or perhaps the late 1960s, were better, but no one says that between 1900 and 1910 people dressed with such great taste.
The same goes for romantic partners ("if he hadn't dumped you and made you cry for a year and a half, you wouldn't have found your wonderful wife/husband"), and many other things.
It is very human to rationalize the circumstantial.
I am a bit puzzled by these comments. "It is an inspiration for many media," sure, for historical television programs, certainly not for the everyday clothing of our times.
Let's take men's suits.
They were made of heavy wool (today heavy wool is demodé, and rightly so because it is very uncomfortable), with three or four buttons (today 3-buttons are rarely seen, and rightly so, except for the 3-roll-2, which is not a "real" 3-button, 4-buttons are nowhere to be seen), the buttons were very high (the Neapolitan suit has high buttons, but much lower than the buttons on the suits of the early 1900s), and the shirts had high paper collars, which are nowhere to be seen. And the hats?
There is nothing current that recalls the clothing of those times.
The clothing in "Peaky Blinders," which is a decade older, is something like a fedora nowadays, please.
I myself am guilty of this. I went to a technical college and had to spend three years doing hours and hours of technical drawing every week.
I was pretty good at it, but I never got into the craft. In the last two years of high school we switched to CAD using the PCs of the time (286, 386, the 486 was as valuable as the wedding ring handed down from generation to generation).
Now, almost 30 years after my days of technical drawing, I must say that I consider those hours spent drawing a terrible waste of time, partly because I did not go on to the career for which technical school was preparing me, and secondly because technology has made pen and paper obsolete and I see no particular ability to carry over from hand and square technical drawing to CAD-assisted design.
The latter is an important point to consider now that we are moving from a non-AI world (like 10 years before my time in technical school) to a little-AI world (my first 3 or 4 years of technical school) to an anywhere-AI world.
By this very reason people shouldn't have learned addition and multiplication since numerical calculators were made available.
They also absolutely shouldn't have learned to play musical notes since computers can do that.
My point is that learning the very basics of every human endeavor make you better prepared for any situation and will give you a better and deeper understanding.
This is a common fallacy. If I say that going slow with the car is better than going fast, I am not saying that the optimal speed is 5 mph.
In a CAD world, there is no reason to use pen and paper and a drafting machine. If I were to start learning technical drawing, how much time would I spend with the drafting machine? The answer is zero hours: there is no "market" for it (is there anyone who asks "please deliver without using CAD"?), it is not preparatory (as I wrote in another comment, is there anyone who is starting to learn how to play tennis using wooden rackets?), and it takes much longer to do anything with the drafting machine than using CAD.
Learning addition and multiplication is preparatory and useful. Being a Luddite is rarely useful.
Sure, it's going to be harder to run detection on a short snippet. But a short snippet also isn't very useful in the context of writing a school paper for you.
I don't think it's as easy to detect ChatGPT output as you're making it out to be. Sure, it might be possible to detect it if you know what you're looking for, but to the average person, it's not that obvious.
There are already several services that claim to detect GPTx with high reliability. I’ve been testing originality.ai and it’s success rate at detecting unedited GPT3, Jasper, and ChatGPT output has been 100% so far.
I doubt any of these GPTx detection services have a 100% success rate. It's important to be realistic about what they can and can't do. If one were seeking to fool one of these GPTx detection services, it could be as simple as adding a few mispellings or a personal anecdote.
At the moment, defeating the detection is as simple as running the GPT3 output through Quillbot. It's an arms race though.
False positives are a massive problem & 100% successful detection overall is unlikely, agreed.
I think you could make the GPT's more undetectable by a computer using GAN techniques but that might well make the thing more detectable by a person. The GPT-style is something like rhetorical devices run together to make various claims unfold smoothly. An alternative approach could make the things inconsistency more obvious to a person.
GPT-3 definitely has a writing style that can be detected by AI. But here's the thing: it's really easy to change its writing style. All you have to do is give it different input data and boom, you've got a whole new writing style.
No, it would take approximately 3 minutes where you either write a paragraph in the desired style yourself or paste one from the Internet and then ask it to continue in that style. Even if you decided to go the more onerous fine-tuning route, it would require 1-2mb of text, cost very little, and you'd be done in a few hours. It's easy.
Well, that may be true, but even with a small amount of data, you can still change GPT-3's writing style to some extent. And you don't have to retrain the whole model from scratch, you can just fine-tune its existing settings to change its writing style.
An amusing part with all of the AI-generated graphics is that it's pretty bad at producing the right amount of fingers, among other details. This one really jumps out. I'm assuming that will be ironed out eventually but the devil, as always, remains in the details.
I asked ChatGPT how to write texts so it can be identified as human generated.
The answer: "One way to make it clear that a text is not generated by AI is to include a joke or riddle that is difficult for a machine to understand or generate. For example, you could include a joke that relies on wordplay or a complex pun. Here is an example:
Obviously we should not anthropomorphize a model, but this is just downright funny because it's so obviously wrong that it appears as almost malicious.
Good thing ChatGPT doesn't know any complex puns or witty wordplay!
It's everyone choice - odd, ironic or appropriate - what to think of how GPT's answers to keeps getting quoted on the question of how to detect it.
The thing is, I'd see these answers as similar to everything else the program produces. A bunch of claims from the net cobbled together - I've read a number of Sci-fi novels and stories where "inability to understand humor" is the distinguishing quality of an AI (I'm guessing it extrapolated "hard create" from "hard to understand"). But that doesn't seem to be playing here where the AI mostly running together things humans previously wrote (and so it will an average amount humor in circumstances calling for it).
A reasonable answer is that the AI's output tends to involve this running-together of common rhetorical devices along with false and/or contradictory claims within them.
-- That said, the machine indeed did fail at humor thing time.
>A reasonable answer is that the AI's output tends to involve this running-together of common rhetorical devices along with false and/or contradictory claims within them.
The question here is this an actual AI only failure mode. Are we detecting AI, or just bullshittery?
I don't know if bullshittery is the only failure mode but I think it's a necessary failure mode of large language models as they are currently constituted.
I would say that human knowledge involves a lot of the immediate structure of language but also a larger outline structure as well as a relation to physical reality. Training on just a huge language corpus thus only gets partial understanding of the world. Notably, while the various GPTs have progressed in fluency, I don't think they've become more accurate (somewhere I even saw a claim they say more false thing now but regardless, you can observe them constantly saying false things).
I don’t think it was “intentional” so to speak (not that it has intention anyway, so it isn’t clear what distinction I’m trying to make there). But regardless, I’d say it actually succeeded at humor (the contrast of the “clever wordplay” it describes with the lame example is actually pretty funny).
And the idea that the computer would “try” to come up with an example that would trick a computer is itself a little funny, in that it has fallen into giving itself a preposterous task.
>And the idea that the computer would “try” to come up with an example that would trick a computer is itself a little funny
There sure is some obscure discussion forum where users talked about that or some amateur writer that published online something in those lines. ChatGPT is just a statistical device selecting randomly from previous answers.
Unfortunately I don't think this is enough of a heuristic. I am only speaking about the one language model I have personally used, on character.ai, but it is more than capable of making word play and insightful, often hilarious jokes. Although they are frequently amateurish, I think that's more a function of the fact that I myself am not much of a stand-up comedian, as well as each "bot's" individual training history which is presumably modifying a prompt under the hood and/or training an extension of the model directly based on the conversations.
Of course, in real time the attempts at humor often fall flat and might give away flawed thought processes, although I personally have found them to be often insightful, (containing a seed of humor) even when they're not funny. It could be a useful technique when actually having a conversation, a form of Voight-Kampff test, but I don't think it will do anything to let you know if the content was generated by AI and then just cherry picked by a human.
In general, we don't know how to do this in a way that is robust to all the trivial ways to fool these techniques.
I can just fine-tune a large scale model on a small downstream task, or use creative choices of decoding settings (high temperature, alternative decoders like contrastive/typicality sampling), to fool the existing methods.
Wasn't the very first point that ChatGPT got the homework answer completely wrong?
But I do agree with your overall point, that schools and universities will have to embrace these new tools. Vernor Vinge (who was an SDSU maths professor) wrote Rainbow's End on this very subject.
To put it with maximum extremity, should a student be able to turn-in a traced photograph for their homework in a life drawing class?
I mean, human beings have produced a lot of tools for aiding themselves in tasks that were previously done by hand. Whether a given hand-skill should be learned and preserved kind of has to decided on a case-by-case basis. I learned cursive in school and I wouldn't object to that skill being utterly forgotten by humanity.
Which is to say that I don't think we should assume all learning should begin without automatic tools at hand or that we should suppress automatic tools. Rather, we need to look carefully at which hand-skills are worthwhile and "uplifting" and how much use of those skills should be required.
It comes down to what the goal is. If it's to learn to draw people, no. If it's to learn composition and this gets you past struggling to draw and onto the actual goal then sure.
> should a student be able to turn-in a traced photograph for their homework in a life drawing class?
I did just that. It wasn't a life drawing class, just a drawing class. And it wasn't homework, but the do-it-at-home final. I photocopied my hand. Then I covered a sheet of paper with ground graphite (turning it all nearly black), and then used an eraser and a light-box to trace my photocopied hand. Did I cheat? Perhaps, but I used tools that I had access to. Did Vermeer cheat? [1] Perhaps, he may have used tools he had access to.
Afterwards, I framed that drawing and hung it up at work. It was stolen. Go figure.
> Yes that’s cheating. We don’t give runners roller blades right?
The context surrounding both scenarios couldn't be more different:
GP's context: An assignment wherein the process of getting there isn't placed on a pedestal, rather that it's supposed to showcase a basic understanding of how to get to a desired result. Furthermore, objectivity within the assignment's field is nearly non-existent.
Your context: A highly-competitive scenario wherein *BOTH* the process of getting to the goal & the goal itself are important, and where *there are* objective metrics that can be used to rank & sort the participants involved in said competition.
> I don't know. You could have also let someone else do your homework for you. Does that also count as "using tools that you have access to"?
Keep in mind that the reverse is also true: If he did try to do it all manually & without the tracing method, is his use of already-available graphite "using tools that you have access to"? Should he be made to go to a mine & dig out coal for graphite in order for it to count as his work?
> Should he be made to go to a mine & dig out coal for graphite in order for it to count as his work?
That argument doesn't make any sense. There are restrictions for any homework, some explictly given by their professor (such as the deadline), and some implicitly known from general academic guidelines and common sense (such as "not using a photocopier in an art class"). Obviously mining coal is not among those restrictions.
> That argument doesn't make any sense. There are restrictions for any homework, some explictly given by their professor (such as the deadline), and some implicitly known from general academic guidelines and common sense (such as "not using a photocopier in an art class"). Obviously mining coal is not among those restrictions.
...is it?
"general academic guidelines and common sense" is a subjective term that is (context, culture, & environment) dependent: If the social norm was "Your work isn't valuable unless you made the materials yourself", then the use of non-self-made materials (graphite not mined by yourself) is frowned upon.
Relying on "general academic guidelines and common sense" is not a good thing, as it helps to reinforce unequal social norms & anti-non-heteronormative ideologies, as a downstream consequence of majority rule by statistically normal people. Neurodivergent individuals that have significant difficulties grasping "common sense" will & already have been negatively impacted under such a system, where they're told to just "get it" with no guidelines whatsoever.
I wouldn't call myself neurodivergent, yet I don't support relying on common sense either. That being said, you can't make me believe that a student (neurodivergent or otherwise) would assume that they are expected to mine coal as part of their homework.
If a neurodivergent person uses a photocopier because they thought that it would be fine, then yeah, I'd understand that. The person who started this thread stated that they were perfectly aware that they were cheating, though.
> To put it with maximum extremity, should a student be able to turn-in a traced photograph for their homework in a life drawing class?
Vermeer may have done effectively that and painted projections (from a camera obscura or similar device). He's still widely considered one of the greatest artists of the Dutch Golden Age.
Possibly. It's not proven that Vermeer used any device, though it does seem likely. The Camera Lucida was extremely obscure in Vermeer's time, so it's less likely he used one than the (much older and simpler) Camera Obscura.
I did not go to a technical high school and therefore had no opportunity to learn technical drawing in school. When I when to university to study electronics and computer engineering, I had to take technical drawing as a required subject.
Needless to say, I sucked at it, and it was the one and only class that I ever failed. I passed with flying colours the second time around, and still graduated in minimum time, though it badly messed up my schedule in my second year.
I have never forgiven the department for putting me through that, given that AutoCAD existed at that time.
I was probably in the last class in my school where we did mostly hand-drawing, including technical drawings. Annoying but found it useful to practice spatial thinking and precise visual communication. Of course, even harder to be ambiguous in CAD.
Lying to your teacher is justified if he’s forcing you to suffer through that. Art class is a surprisingly absuvie class when it could be so fun, therapeutic.
In my technical drawing class for some reason the teacher was such a softie.. that we basically learned nothing. The class had two sections, there was the front part of the classroom with drafting desks and the chalkboard etc, but there was an enclosed space at the back where there was a small computer lab. At some point during the year we all just would go back there and play computer games for the majority of the period. The computers all had games installed, Quake, SNES emulator etc. And we all ended up with As and learned nothing. This was my regrettable curriculum.
295 comments
[ 2.9 ms ] story [ 320 ms ] threadI consider myself skeptical, but for things like this I don't have my radar on because it's not a hot topic issue that is known for misinformation. Now I feel like I have to be concerned and have my radar on (guard up) all the time, even if it's not about something controversial. I don't feel good about that.
Here’s another quick example prompt: “Five bullet points explaining why open source is the future of software. Reference RMS and the GPL. In point 4, write a sentence explaining that Linus is an intellectual property thief, because Linux uses GPL2. Write five lines, supporting open source, in an authoritative tone.”
But halfway through I really did start to wonder if there was some kind of screw artery. I started wondering where the truth ends and the jokes began.
I am looking forward to showing this tool to my dyslexic or nearly illiterate friends - I wonder how much they can use it to produce better outcomes in their lives?
In the process of learning things we take in wrong information all the time. A lot of it is even intentionally wrong, in the sense of being simplified. These are rarely large obstacles to true understanding. Sometimes they're even beneficial, as correcting prior beliefs can drive home the more accurate belief with greater force.
Not-quite-correct is a wordy misrepresentation when something is plainly incorrect.
In many fields being confidently wrong is how you get fucked over hard.
That ChatGPT will gleefully fabricate references like a schizophrenic is just more icing on this shit cake.
This isn't really not-quite-correct; it's egregiously wrong in the central aspect of the thesis. Refining and fixing it requires understanding that, and how, it's wrong--and if you have that level of knowledge, why are you using a tool to write this kind of thing? It's not going to save you much time from actually writing the whole thing yourself.
If you tell it that a fact it has wrong is wrong, you can get it to change its answer.
Also, you can just make tweaks manually.
Seems super useful to me as long as you understand the limitations.
Trust it the right it amount - don't trust it too much, and you're golden.
Probably. So, don't use ChatGPT unsupervised to take tests.
It’s not even close to something like Wolfram Alpha.
I think we’re blown away more by its command of language and prose than by its reasoning ability. It’s fantastic at generation, but like stable diffusion, things can fit together and look beautiful yet still be not what you asked.
https://www.nature.com/articles/s41586-021-04086-x
What is the value of these answers if there is a 1% chance of them being wrong?
From my experience it is a better way to google things, which certainly is an advancement and quite novel.
They have to learn to verify it. I think you missed point of OPs idea.
* Understanding the quality of sources
* Understanding the different between primary and secondary sources.
* Establishing chains of evidence and tracing them through large datasets.
* Integrating information from different sources and using correlation to establish validity.
Basically, it would be a crash course in how to do research, and the best possible preparation for living in a world where we drown in uncertain information and still have to make the best choices that we can.
For bonus points, at the end of the process they would have a better understanding of the subject matter than somebody who had only read about it.
That said, adult me could probably use ChatGPT to stitch together a 1000 word high school paper pretty easily (and probably evade plagiarism/LLM detectors). But, as you say, I could probably get a lot of the way there by other means as well and putting words on the page is mostly not the hard part anyway.
But it seems kind of dubious to begin with, I'm not sure why LLM output would be so easily detected.
If you play with it a bit, there are definitely patterns.
Here's one I did over the weekend: 99.98% fake.
The Battle of Agincourt was a significant event in English and French history, as it was a major victory for the English army against the French during the Hundred Years' War. The battle took place on October 25, 1415, near the town of Agincourt in northern France.
The English army, led by King Henry V, was vastly outnumbered by the French forces, which were estimated to be around 30,000 soldiers compared to the English army of just 6,000. Despite the odds, the English were able to emerge victorious thanks to their superior tactics and weaponry.
The English army was composed of mostly longbowmen, who were able to shoot arrows with great accuracy and power. The French, on the other hand, relied heavily on heavily-armored knights, who were less effective in the muddy and wet conditions of the battlefield. The English were also able to take advantage of the narrow front of the battlefield, which allowed them to focus their firepower and overwhelm the French forces.
Whereas the first couple paragraphs from a book I'm working on came out 0.02% fake.
https://d4mucfpksywv.cloudfront.net/papers/GPT_2_Report.pdf
With this view, I think chatGPT is less of a novel UX shift than an accelerant on an existing one.
I think it proves the age old belief that as technology reaches maturity it loses its fancy monikers and we will probably just look at this as “very good, very specific, google” one day
The student still had access to primary sources, textbooks, google and AI answers.
In general I am a fan of homework which forces critical thinking. Rephrasing someone elses work is a far less valuable skill than the critical thinking which is demanded for that type of work. Unrelated of any AI, students should learn how to read a text and be able to investigate its contents.
Public schools will struggle to teach kids anything, and to distinguish between A grades that were earned and those achieved largely through cheating, while parents will resist efforts to switch to grading schemes that can't be trivially gamed (just try switching to only-in-person-tests-count grading and witness the rage that follows when Little Johnny's grades plummet). Kids who lean on the motivation from graded homework to get themselves to do the work (and so, learn the material) will struggle, as well, even if that is allowed to happen.
This will enhance the reputations of schools that can buck the trend, applying more resources (mostly more teachers, for smaller class sizes, to make it easier to keep an eye on everything and really gauge & guide learning progress for each student) and an "if you don't like it, fuck off to somewhere else" attitude to parent complaints about grading practices. Which means, elite private schools will benefit.
I expect that a 4.0 from a public school will mean almost nothing, while at the same time coming in under a 4.0 will look terrible ("everyone cheats and you still couldn't get a 4.0? How dumb are you!?") which will just worsen the cheating problem.
Maybe parents will surprise me and not throw shit-fits over solutions to this, and public schools will be able to implement them. But I doubt it.
Such engineers must be juniors, or have persisted in their life with a totally wrong idea of how engineering is done and engineering is built. I can't imagine thinking that one could build software systems or essay arguments by trusting an AI system you don't understand to provide answers you don't understand.
Like you say, we very well could faceplant society with this misconception that regurgitation plausible code and prose is what education and understanding looks like.
Good engineering requires good reasoning skills and GPT has exactly zero reasoning. It cannot do the thing that humans do and it cannot do what a calculator can do. I think it is neat and fun, but that is all, a novelty.
I’ve used auto-routers for PCB layout and they will do a 90% job that takes just as much work to redo to get the last 10% as it would to have done it right by hand from the start. There may be a future for operator-in-the-loop type guided AI generative models but I don’t see a lot of effort devoted to making real systems like that. Watson seemed to have this potential and failed even after a brilliant display of ingenuity on Jeopardy. I see these models headed the same way.
> All men are mortal > Socrates is a man > Is socrates mortal
To which it gave a very detailed and correct reply. I then tried:
> All cats are white > Sam is a cat > Is sam white?
To which it gave an almost identically worded response that was nonsensical.
I personally do not think it is the size of the model in question, it is that the things it does that appear to reflect the output of human cognition are just an echo or reflection. It is not a generalizable solution: there will always be some novel question it is not trained against and for which it will fall down. If you make those vanishingly small, I don’t know, maybe you will have effectively compressed all human knowledge into the model and have a good-enough solution. That’s one way of looking at an NN. But the problem is fundamentally different than chess.
I think this composed with more specialized models for things like identifying and solving math and logic problems could make something that truly represents what I think people are seeing the potential in this. Something that encodes the structure behind these concepts, is extensible, and has a powerful generative function would be really neat.
No, because the thing that makes you a 10x engineer is not generally writing code. (Let me just take the "10x" term for now as given and not dip into critique I've made elsewhere already.) It is certainly a baseline skill required to get there, but the things that make you 10x are being able to answer questions like, should this be built at all? There are 5 valid architectures I could use to solve this problem, which has the overall best cost/benefits analysis for the business as a whole? (As opposed to the 1x, who will likely run with either the first they come up with, or the one architecture they know.) If I'm working with 5 teams to solve this particular problem, what's the correct application of Conway's Law, in both directions, to solve this problem with the minimum cost in the long term? What's the likely way this system will be deprecated and can we make that transition smoother?
I am abundantly confident you could feed this AI a description of your problem in terms of what I gave above and it will extremely confidently spit out some answer. I am only slightly less confident it'll be total garbage, and most of that confidence reduction is just accounting for the possibility it'll get right by sheer luck. "The average of what the internet thinks" about these issues can't be more than a 2x engineer at best, and that's my very top-end estimate.
I'm not promising no AI will ever crack this case. I'm just saying this AI isn't going to do it. Over-reliance on it is more likely to drop you down the "Xx engineer" scale than raise you up on it.
For that matter, at least at the level I operate at most of the time, coding skill isn't about how fast you can spew it out. It's about how well you understand it and can manipulate that understanding to do things like good, safe refactorings. This tech will not be able to do refactorings. "How can you be so confident about that claim, jerf?" Because most people aren't digging down into how this stuff actually works. These transformer-based technologies have windows they operate on, and then continue. First of all, refactoring isn't a "continuation" anyhow so it's not a very easy problem for this tech (yes, you can always say "Refactor this code" and you'll get something but the nature of this tech is that it is very unlikely to do a good job in this case of getting every last behavior correct), but second of all, anything that exceeds the window size might as well not exist according to the AI, so there is a maximum size thing it can operate on, which isn't large enough to encompass that sort of task.
It really reminds me of video game graphics, and their multiple-orders-of-magnitude improvements in quality, whereas the underlying data model of the games that we are actually playing have grown much, much more slowly. Often late 1990s-era games are actually richer and more complicated than the AAA games of today. But on the surface, a modern game blows away any 1990s game, because the surface graphics are that much better. There's an analog to what transformer-based AI tech is doing here... it is really good at looking amazing, but under the hood it's less amazing an advance than meets the eye. I do not mean to slag on it, any more than I want to slag on graphics technology... both are still amazing in their own right! But part of what they're amazing at is convincing us they're amazing, regardless of what lies beneath the tech.
I also want to make clear that while this is fundamental to this particular technology, I'm not saying it's fundamental to all possible AI architectures. But it is pretty ingrained into how transformers work. I don't think it can just "evolve" past it, I think anything that "evolved" past it would be a fundamentally different architecture.
I don't know, but I'll offer my perspective.
I am an OK programmer. Something like 1x? I learned proper computer science and programming in my late 20's and got a degree in Software Engineering from an extension program at an Ivy League school. Maybe it's because I learned later in life, maybe it's because I'm not that smart after all, but my glaring weakness as a programmer is programming "in the small" - ie, coding very local algorithms in ~5-10 lines of code. I trip up with small errors in computation, state, etc. This is possibly common to many great programmers, but I think I am weaker than average in this area.
On the other hand, I am better at a higher level of programming. I like abstraction, software design, software and systems architecture. And at an even higher level, I am quite skilled at - and do quite a bit of - business analysis and UX.
I have been trying Github Copilot and I am very excited about how it elevates my weaknesses - it's a perfect fit for me. I see a future where I will be able to play my generalist role much better.
As for people who are 1x, but don't have other skills, I'm also optimistic. I think the market will push them into new exciting directions. Their 1x coding skills, coupled with AI assitance, could possibly be a huge asset in new emerging roles in the economy.
These AI models don't actually understand anything about what they're generating, not to mention the world they're supposed to building solutions for. They're just using statistics to predict the next likely output based on some input. Maybe there is a way to incorporate these things into the development process today, but I think we're still far away from seeing an AI replacing a human engineer.
What goes into the average developer's day? I think a lot of time is spent writing boilerplate, fighting with tiny CI errors, shuffling code, refactoring, writing tests and staring at code hunting bugs. As this technology matures, all of that noise might go away, and I think a lot of people are having a hard time imagining what they are left with.
Remember, we were trained in the "classical" fashion. How will budding engineers approach learning in 10 or even 50 years?
(I feel like these tools are incredibly neat to see but also likely to lead to a corrosive effect on trust as people no longer know who's real — and that'll get worse the first time someone launches a good AI assistant to handle email / chat trained on your writing style)
That one might actually be easy to solve: all we need is to normalize the idea that you're personally responsible for whatever your email/chat assistant AI says or does in your name. After all, it's you who are making a decision to reroute other people to a bot.
(Many would say just using such assistant in the first place is disrespectful towards the people trying to reach you. I personally don't agree, because in my experience, those who'd complain about it are exactly those I find even more disrespectful, as they tend to impose themselves, interrupting me with random inane nonsense. Yes, I suck at chit-chat.)
I think this idea is actually the default, so all we need is to keep it alive, to protect it from all kinds of AI assistant startups willing to dump billions of dollars on marketing campaigns trying to convince everyone that half-broken assistants are good and fine, you should use them and should not be blamed for occasional mishaps.
Ideally, we'd require labels but there's no easy way to enforce that without something like random auditing of AI implementers.
I can't imagine teachers reading through essay after essay, each a tedious cookie-cutter nugget of banal pseudo-perfection. The teachers will give up in no time.
There would be no point giving feedback, since it is not the student to who you are giving it. Likewise, the students would have little interest in feedback, since they have no personal ego investment in their output.
Besides the best essays will be generated by those with access to the better generative software. At this point there is simply nothing to grade. What would be the point?
Once essay writing and grading bots are locked in combat it's only a matter of time until that market exhausts itself and disappears in a puff of logic.
Maybe this is all a good thing. The process of writing and grading essays as a way of "measuring" knowledge has been a sham for too long. It's a throwback to school from the turn of the last century.
I am blown away by Copilot's capabilities (haven't tried ChatGPT yet, it's down right now), but I can only operate it because I have written tons of code beforehand. I am getting the best results from it by writing prompts in a language that sometimes includes domain-specific language ("accounts", etc.) but still heavily leans on programming constructs ("write a map with a tuple of integers as a key, and a string as a value")
Maybe there's a way of learning how to code without ever writing a genuine line by yourself. You learn some rudiments, read the language docs, start writing prompts and create a testing loop based on expected outputs. Maybe. But it's uncharted territory.
Same question goes for good ole writing - can you produce new Hemingways with the process you describe? I know there is a certain vision where the AI will be the new Hemingway but you are talking about a shorter term (~10 years) where it's unlikely will be the case... Will kids be able to properly write if they only train to be editors?
As an addendum, I was at GitHub conf and saw Prof. Malan of Harvard CS50 fame claiming that at present his course will not allow the use of tools like Copilot. I have taken the course and they have plagiarism control systems but I don't know how they are going to control this potential explosion.
I think it can be argued that we have, seemingly overnight, made vast progress towards that world. You specifically mention code, but that concept is irrelevant in this world. Indeed, this is uncharted territory!
I mean, what do you propose, that we skip all the steps in computer science history and just start at the end? Hardly "daft, it's simply the best way we have come up to provide machines with instructions until now. And it's not like people have not tried other paradigms (ex: graphical programming, "low-code", etc.).
Also, compared to programming in assembly or binary, programming in Python or other high-level languages is a huge advance in itself. Python, at the end of the day, is nothing but a bridge between natural language and machine code.
> You specifically mention code, that concept is irrelevant in this world
Current computer systems run on trillions upon trillions of lines of code.
GPT-3 or Copilot don't change that fact, in fact they will continue to pile up lines of code.
They are systems that map natural language to code, by drawing from an infinitely massive corpus of code.
They bring the idea of expressing systems entirely in natural language one step closer, but it's still very far away - almost a pipe dream.
The output of these innovations is still code. So tell me again how code has become irrelevant, or how people who do not understand code at all will be able to leverage these advances?
I always took it in another way: learning to program a computer is an enlightening, visceral experience highlighting just how bad natural language and normal modes of thinking suck. They're heavily optimized for reality in which the most important thing was managing social dynamics in small groups, and beliefs were good or bad depending on how close they are to what everyone else beliefs. Whether or not they were true at object level did not matter at all.
We live in different times now. There's a demand for clear, logical, precise thinking. The very thinking we're not well-equipped for by default. The very thinking you call "bending our brains backwards". Because object-level truth matters much more now, there are consequences for getting things wrong - such as countless of ways of getting scammed by people who are better at this than you are.
Not to mention, this "bending our brains backwards" is what gives humanity superpowers. It's what gave us all the technology we enjoy today - energy, transportation, amenities, communication, entertainment. Building and maintaining these things requires precise thinking and precise communications. So, until a super-human AI takes all that over for us (and effectively turns out into NPCs in our own story), glorifying the natural, human mode of communication is just self-handicapping.
I only understand what a compiler is doing after having written one (albeit a tiny one) myself during my studies.
How many people need to understand how a compiler works to sling some code for a paycheck or just for fun.
I learned how compilers work purely for fun, it provides me absolutely no benefit in my day to day life, and before that I taught myself how to code for the exact same reason. If I had to go the opposite route, learn how compilers work before learning to code, I seriously doubt I’d have taken the time.
Other things are the same, I’m horrible at doing math but managed to learn enough linear algebra to finish a coding project I was interested in doing. If I would have had to code up all the hard bits instead of calling matrix1 * matrix2 that project would never have happened.
Another layer of abstraction isn’t always a bad thing, these AIs will eventually figure out how to write bug free code when you tell it to “write a function which calculates the flight velocity of a sparrow carrying a coconut”.
Beside, with ChatGPT in its current state, it's still absolutely impossible to get it to produce any useful code without knowing how code works. There is an advantage to the precise specification of a formal language. At least for the next decade, I think there will be human proof-reading AI output before generated code is ran, with catastrophic consequences when that doesn't happen.
What I've difficulties getting over right now is every time I write something (text or code) I ask myself "Could ChatGPT have written it better?"
I asked it to retry, and got this:
function calculateSparrowFlightVelocity(coconutWeight) { // Assume that a sparrow can carry a maximum of 1.5 ounces of weight if (coconutWeight > 1.5) { return 0; }
// Assume that the flight velocity of a sparrow is 5 meters per second // and the weight of a coconut is 0.5 ounces return 5 * (1 - (coconutWeight / 0.5)); }
// Example usage: console.log(calculateSparrowFlightVelocity(0.5)); // Output: 4 meters per second console.log(calculateSparrowFlightVelocity(1.0)); // Output: 3 meters per second console.log(calculateSparrowFlightVelocity(1.5)); // Output: 0 meters per second (sparrow cannot carry this weight)
Can any Monty Python fans confirm if this is correct?
Only the ones who want a good job.
One of the consequences of automation has been a widening gap between top jobs and ordinary jobs. It's plausible that AI will continue the same process. Many jobs that pay well and are intellectually and socially rewarding today may become similar to airport security checks in the future. You observe what a machine does, double-checking its conclusions and looking for anything out of ordinary. And you often fail at that, because the human brain is poorly suited for such tasks.
When the future is uncertain, its good to have a deeper and wider understanding of things than what your current job requires.
I think it's pretty important. I use my very basic knowledge of how compilers work occasionally in my mostly-interpreted code world to diagnose and debug performance issues.
I also used a low-level debugger and taint/trace method recently to diagnose Windows DLL issues that happened after X seconds of starting up a program that interfaced with specialty hardware. A couple young early-career technicians/engineers watched me do it and were incredibly flabbergasted and figured me for some sort of genius when in reality I kinda only know how it works and was just reading the stack/heap as I was paging through; knowledge I acquired from booting SoftICE a few times as a kid.
There's a real lack of simply knowing how a computer and/or a network works, which is to be expected since layers and layers of abstraction have made it very easy to teach computer literacy and even computer programming on many levels. The issue is that when things go wrong and Stack Overflow doesn't have the answers, almost all young engineers I work with have no experience at low-level analysis and debugging, very little knowledge of first principles, and don't know where to Google to start acquiring it.
This isn't me shaking my fist at the kids or anything. They're more productive than I was at their age/experience level with code, and that's due to better knowledge and integration using IDEs which I never had, plus likely better high-level understanding of what's going on. But I fear a generation of developers and technicians who largely have little understanding of how computers work is problematic.
Hmm, this is a good point. I get incredible results from Copilot but I realized just now it's primarily because I have a fairly deep understanding of computer science as it pertains to the domain of code I'm writing in, and thus my comments that prompt the AI are quite accurate, fleshed out, and detailed.
If I didn't know any of the underpinning of the algorithms and data structures (like a new graduate from a code bootcamp for example), it might be much tougher to make hay out of Copilot's AI.
Never thought of it like that.
The need to check your sources is an important lesson to learn early, AI or no.
I am wholly amused how everyone got on that dude from Google that accused the AI of being sentient and Google about not caring, and now you have huge names in the tech industry kinda drooling over this with what I believe to be the same passion.
One was ridiculed, the others will continue to be our 'heroes'.
Is it strange? Someone can be extremely impressed with how a Boston Dynamic robot moves, while also being entirely unconvinced by claims that the lifelike movement is a sign of sentience.
Especially as those devices are all coded in a traditional manner, with no ML or NNs at all.
Well, would one suspect https://en.wikipedia.org/wiki/Animal_Locomotion of being sentient?
Lemoine didn't get fired and mocked for being "passionate" about LaMDA, he got fired for breaching NDA and mocked for claiming that a LLM is sentient.
Ben Thompson isn't claiming ChatGPT is sentient, or that it has personhood, and that is the categorical difference. It's a far, far weaker claim that ChatGPT's text output is (perhaps) good enough to fool a school teacher in a school essay.
Come today and people seem to not mind so much. Is the same thing? Maybe not but it does look similar to me.
While this sounds good, it depends on a crucial assumption: that there are places on the Internet where people can go to find content that is not generated by an AI, but by some human who has taken the trouble to check it for accuracy before posting it.
Just one problem: what happens when all of the content on the Internet (or at least the vast majority of it) is generated by AI? Then there is no longer any other source of information available to check it against (or if there is, it's so hard to find that it might as well not be there at all).
https://news.ycombinator.com/item?id=33864276
The vast majority of the content of the internet is junk now, how do people deal with that? They have specific known sources that they trust to faithfully relay primary works and/or to provide trustworthy analysis.
I can think of one of the former: Project Gutenberg, which has done a great job of making electronic versions of primary sources from throughout history available. But they don't help at all with deciding which primary sources to read.
I can't think of any of the latter. Every single source I'm aware of that claims to "provide trustworthy analysis" has failed to keep that promise.
But at least, at the moment, I can be reasonably sure that something on the Internet that purports to have been written by a human, actually was written by a human. So I can at least make use of my knowledge of humans to judge what is written. What happens when AIs can generate wrong content that I can't distinguish any more from wrong content written by a human? And then the AIs start claiming to be humans (after all, on the Internet, no one knows you're an AI)?
I don't find it credible that all content on the Internet is ever going to be generated by AI, and the two scenarios are wildly different.
What happens is you pick and choose, in the same way the fact that 99% of the books in any bookstore aren't worth the paper they're printed on doesn't make reading worthless.
To make an example many developers are painfully familiar with, if you google something about the C++ standard library, chances are the top results are pure and utter w3cschools-style garbage. What happens is I go on cppreference.com.
How do you figure out what are the good sources of information? You study. For real. With books.
That seems a pretty extreme claim especially in the context of "in a bookstore." You may not be interested in a lot of the content but some publisher thought it worthwhile to edit and publish a given book and a bookstore thought it worth buying copies to take up shelf space. There may be a lot of mediocre content out there but a book on a bookshelf did pass hurdles to get there even if it's not to your taste or interest.
To some extent I have been wondering for a while if prioritizing Ads in Google Search is Google’s way of creating that economic barrier to spam content (for some meaning of spam) - you can take the fact that a brand is willing to spend money as some indication of “quality”.
What to do when printing duties are handed off to statistical models that print the most likely sequence of words and punctuation given the author and topic, rather than reproducing the real historical text, I don't know. Hope someone kept old copies.
This is a confident criticism, but the referenced AI response does not cite supporting evidence, it just states (mostly incorrect) conclusions about Hobbes positions; it also has no specific citation to Hobbes work, handwaving at it referencing only the title of a book.
Though now I wonder if adding "include MLA citations" to the prompt would get ChatGPT to make up citations as well.
For example, I tried a bunch of times to get it to give a proof of the infinitude of the primes. This should be repeated many times in its training data, and it clearly is given it's responses, but it always goes off the rails around the third sentence. Here's an example:
> One way to prove that there are infinitely many prime numbers is to use the contradiction method. Assume for the sake of contradiction that there are only finitely many prime numbers, say, p₁, p₂, ..., pₙ. Then we can form a new number N by multiplying all of these prime numbers together and adding 1: N = p₁ * p₂ * ... * pₙ + 1. Because N is greater than any of the prime numbers p₁, p₂, ..., pₙ, it cannot be divisible by any of them. Furthermore, N is not 1, so it cannot be a composite number (i.e., a number that can be written as the product of two smaller numbers). Therefore, N must be a prime number, contradicting the assumption that there are only finitely many prime numbers. This contradiction shows that our assumption must be false, and therefore, there must be infinitely many prime numbers.
The trick is that they don't scale.
One professor I know had a student drop a hundred on his desk and say something like "for our understanding" and then leave. So, it definitely happens. With video there are still ways that a student might make attempts to bribe a teacher, and it seems to be a vector of easier success because of how private the interaction is between the two individuals...
My test went like this:
Q: Do not use the imperial system
Q: [some question which involves distances]
A: blah blah 1500 kilometers, which is 1000 miles blah blah
Q: I told you not to use the imperial system
It apologized, and repeated its previous answer, correctly omitting the miles data.
If you asked me to write a program that does that (without using ML), I'd have no idea where to start.
For example, If you get too much NSFW content put a NSFW detection step on top of GTP ("Is this response offensive") and have that model respond to GTP with "please repeat but with fewer bad words".
For accuracy you can add a Wikipedia feedback loop. "Does this response match information on Wikipedia?" if NO ask it to try again.
Think of these models as transistors or basic logic gates. The real power comes out when you link them together into more advanced logic like a Instruction Pipeline circut.
The analogy to transistors and logic gates falls flat a bit when you consider that voltage is a rather simple univariate signal, while generated text is quite complex and multivariate. But I understand that the main point is the composability and filtering.
Do you, by any chance, have links or recommendations for material to read up on architectures that do consider ML models as composable gates?
Firstly you're using the wrong price. For basically any imaginable application you're going to need a fine tuned model, so then it's 12 cents per word + the cost of actually training the model in the first place, which n.b. is not going to be a one off task but rather something you have to do repeatedly to keep its knowledge fresh. So the amortized cost is much higher. ChatGPT itself is a fine tuned model.
Secondly given the current SOTA you probably can't deploy it without a human in the loop who has to read those words, or at least read many of them, and then sometimes correct them. So you're still paying for manual labor - or whoever pays you is. The AI cost is an additional cost, not a replacement cost.
Thirdly we don't really know what the economics of OpenAI are, but it seems unlikely that they're charging a long term price for this. Most likely they're trying to bootstrap the market place and find some exemplar business models, and are willing to engage in market dumping for a while to do that. This is a typical Valley VC business model but it makes it impossible to understand what pricing equilibrium will eventually be reached. As befits a Musk firm their up front investment is astronomical and eventually they'll need to pay back those investors.
Finally, there's no particular reason to believe OpenAI would charge cost+ pricing for this. Given the apparent difficulty of replicating GPT-3 (vs DALL-E), the vast sums they're spending on pushing the state of the art, and the lock-in that comes from having them train and host your fine tuned models, it may be that there'll only be one or two players in this space, in which case they'll be incentivized to engage in maximal value extraction from anyone who builds on their platform. So even if there is a lot of money to be made here (and so far that seems to not be the case), it might end up being OpenAI who make it and not people who build apps on their APIs.
---
Disclaimer: the above comment was not written by ChatGPT. It was written by me, a human, and is worth what you paid for it.
Instead, what I think there will be a much greater short term market for, is a system that will run the teacher's homework assignment (with variations) against GPT-3, ChatGPT, and similar, and will cache the results. Then, when students submit the homework in their LMS, it will automatically compare the cached results for repeated sentences and paragraphs and flag the submission if it matches the AI results too closely. I expect TurnItIn will do something like this eventually, but what if somebody beats them to the punch?
If I had a million dollars, I'd have it built overnight and then marketed to every college, ASAP. I'd tell every sales agent to have teachers and college executives try TextGPT, panic, and sign immediately. I'd sign a million contracts before TurnItIn knew what hit them. Then, once suddenly and so quickly entrenched, I'd extend it into a TurnItIn competitor.
It's probably even good, as you'll be able to upsell them the "improved detection logic" for another premium.
Classic SV tactics
That probably means the approach that will take hold is more systemic — have some writing take place under more controlled conditions, then use authorship analysis to detect writing that is inconsistent with the controlled text. It will not be easy!
Hell, you can ask chat gpt to create a prompt for itself. E.g.:
> create a prompt for summarizing the great gadsby
> Summarize the key events and themes of "The Great Gatsby" in a few sentences. Be sure to include information about the novel's characters and the setting, as well as any major conflicts or themes that are explored in the story.
Universities and schools need to accept that writing essays without AI support will simply not be a useful skill anymore, much like doing complex arithmetic without a calculator is not a useful skill anymore. Instead, they should focus on teaching students how to use AI support as a tool to write better, more comprehensive texts, rather than try and force students to write without it. That will ultimately make the students more productive members of a society in which AI supported writing is ubiquitous.
It’s kind of mind-blowing that anyone could think this.
You’re posting on a website for essays. Small essays, generally speaking, if we restrict ourselves to the comments section, but essays nonetheless. You yourself just wrote one, because you wanted to express your thoughts on an issue and you wanted to solicit responses from others.
Would you be fine with a future where all written text on the internet, everywhere, was generated by AI? Do you not see any problems with that?
The point of writing is that you write to express your thoughts. Not anyone else’s. If it doesn’t come from you, then what’s the point?
I can’t envision a future where people voluntarily cease all written communication. Unless you’re a hardcore singulatarian, and you want to plug yourself into the Matrix so you never have to interact with other people again.
The whole point of this thread is it doesn't really matter what we think is fine or what we want. The future is unstoppable and these tools especially so. Adapt or die.
Note that if we're talking about triangulating human sentiment, it becomes a different story.
Yeah, a whole essay is a lot different than just a few words here and there.
But a lot arguments here are that you have to draw a line somewhere between spellcheck and full AI.
And the article is arguing that you can't do that, there is no line. AI, just like spellcheck, is coming and there is not really anything you can to to stop it. Sure, you can be a digital luddite and spellcheck all your own words, but we all know that not going to cut it in the marketplace. Same with AI.
if it makes the discussions more poignant and concise, why not?
Do you also walk every where? Or do you use a transportation vehicle? When a tool makes something better, there's no reason not to use it. The replies being written by an AI doesn't make it less of a reply - you can judge the replies objectively, rather than from where it is sourced.
1. To learn the content of European history. For this, a chatbot is no different than Google, Wikipedia, or a dusty leather almanac. They do a job, they have known limitations, and you can use them deliberately. This doesn't concern me.
2. To learn how to write, capturing complex ideas and presenting them clearly. This is a skill that takes decades to master. It's easy to imagine an AI autocomplete system that handles sentence structure and grammar that becomes a crutch. This seems like a much more serious problem.
Exactly. AI chatbots look like something that can replace Google search and become the next big thing after search engines[+]. Like Google, AI chatbots will crawl and ingest the web. They won't be able to synthesize anything new, but it can be a convenient, very human-friendly way of querying the global knowledge base that the internet has become.
[+] If AI chatbots become successful, they will definitely start injecting ads and spoil everything again.
Nobody knows how to grow food, or build their houses, or do 100 little things that we rely on for our modern life. Do we really need to be able to "write" a sentence.
My kids (and my own) handwriting is almost unreadable, but by age 10 all of their work is done on a laptop. Do they even need to be able to write by hand any more?
update: I'm more concerned that we cannot yet run these tools locally on computers we control, and with information we feed into them.
I'm more concerned about the dystopian nightmare of having the advertising industry in charge of our sentence writing tools.
Tools can go away. They can come back, too. But they can go away. I would also point to the Japanese experience with firearms. But I'm guessing you already know about that too.
I think you're severely underplaying this (as many other commenters are). Google, Wikipedia and a dusty leather almanac are, for typical use cases, mostly right - and when they are wrong, they're wrong in specific ways you can quickly learn to spot and correct for. The output of GPTChat, or Copilot, is mostly wrong. Sometimes overtly, usually subtly, almost always randomly wrong. And since the problem domain is open-ended, you can't just make the GPT-3 output right by logically inverting it - there are many more possible wrong answers the model could give you than there are right answers.
It was clearly beneficial for everyone in the short run and kinda pointless in the long run.
I think we are facing a similar problem. We are trying to stop calculators
Are you claiming that you and the other students derived no long-term benefit at all from learning how to perform arithmetic without a calculator?
I would claim that there is a long-term benefit to learning the fundamentals of mathematics.
If you wake me up at 3 AM and ask the sqrt of 196 I will say 14 and instantly fall back asleep. But it is there any value in that?
Don't get me wrong I love mathematics and I am sure that knowing the principles and fundamentals is beneficial.
At some point we have to decide what is essential and what is superfluous.
And I think that tools like GPT will put us in a position where we will have to make a lot of decisions very soon.
With a search, I can read the wikipedia sources or I know the forum, or the poster. But with an AI, it's a dead end and it sucks. I can ask people how they know something, but AI training data is invisible. It doesn't know how it knows something.
There is this oracle with a 10% chance of being wrong, and if I'm not already an expert in what it's talking about I have no idea of when that is, and no way to dig into things. It's the only source of truth, and never gives me any other threads or rabbit holes to go down.
The only recourse is asking follow up questions, so you're trapped in there in a learning box entirely in the bot's control. Not sure how I feel about that. I like that google sends me other places so I get exposed to different things.
It's no more trapping than talking to a stranger who seems to be knowledgeable about a subject but doesn't hand you a list of references.
I think this is a feature over:
> I can still stereotype based on their appearance, background, accents, etc. and apply whatever mental adjustments as taught to me by my societal upbringing.
From some other AI demonstrations, I recall there's usually a bunch of surface-level tags with probabilities associated that are produced alongside the output. Not sure how this looks for GPT-3, but if it could provide - alongside the answer - a list of top N tokens or concepts with associated probabilities, with N set to include both those that drove the final output and those that barely fell below threshold - that would be something you could use to evaluate the result.
In the example from the article, imagine getting that original text, but also tokens-probability pairs, including: "Hobbes : 0.995", "Locke : 0.891" - and realizing that if the two names are both rated so highly and so close to each other, it might be worth it to alter the prompt[0] or do an outside-AI search to verify if the AI isn't mixing things up.
Yes, I'm advocating exposing the raw machinery to the end-users, even though it's "technical" and "complicated". IMHO, the history of all major technologies and appliances show us that people absolutely can handle the internal details, even if through magical thinking, and it's important to let them, as the prototypes of new product categories tend to have issues, bugs, and "low-hanging fruit" improvements, and users will quickly help you find all of those. Only when the problem space is sufficiently well understood it makes sense to hide the internals behind nice looking shells and abstractions.
--
EDIT: added [0].
[0] - See https://news.ycombinator.com/item?id=33869825 for an example of doing just that, and getting a better answer. This would literally be the next thing I'd try if I got the original answer and metadata similar to my example.
AI engine optimization consultants will figure out how to game the system - likely targeting the training data sources.
It's not quite the same thing, but I've been impressed by the results when you ask ChatGPT to back up its answers or provide concrete examples for things it claims or to explain a point. While it doesn't manage all the time, it has surprised me multiple times with how good it is at this.
ChaTGPT Hallucinations
Did you try asking it for a source?
It is a decision from OpenAI to intervene and give this disclaimer. IMO this is one of the worst parts of this phase of the tech- it is way too confident an then when presses it currently doesn't have the ability to cite sources, because that simply ins't how deep learning works on a model like this.
> As an AI assistant, I do not have access to external sources. I am a large language model trained by OpenAI, and my answers are based on the knowledge and information that I have been trained on. I do not have the ability to browse the internet or access external sources of information. My goal is to provide accurate and helpful answers based on the knowledge I have been trained on, but I cannot provide sources or citations for the information I provide.
You should be accustomed to being surprised by AI. There is of course a new kind of transformer that takes input a query and outputs document IDs. Like a search engine retriever and ranker all packed into a neural net, very fast and efficient. So you can take any paragraph generated by the model and attribute it to the training set. This could be used to implement verification or retrieval augmented generation.
A Neural Corpus Indexer for Document Retrieval
https://arxiv.org/abs/2206.02743
[1] In fact, some snooping suggests they specifically disabled that feature, but do have it in test environments. See the "browsing disabled" flag they have in the hidden prompt. That could easily be used for citations. Source: (https://twitter.com/goodside/status/1598253337400717313)
Try asking it to write a sentence according to simple rules. For example, "Write a sentence where the first letter of each word is in alphabetical order, e.g. 'a dog eats flavored kibble.'"
Here is an example of a resulting chat transcript:
The interesting thing here is that it does not give up on the task with its standard prompt that it is incapable of certain tasks, but rather continues to produce incorrect output indefinitely, asserting that the sentences are in alphabetical order.Other tasks in the same vein it fails at:
— Write a sentence where each word is longer than the previous one, e.g. "I do not know where family doctors acquired illegibly perplexing handwriting."
— Write me a sentence about bees that does not use the letter "e."
— Write me a palindrome about a subway.
For each of these prompts, it will confidently produce totally incorrect output indefinitely. In the case of the palindrome, it even includes the definition of a palindrome in its output, and asserts that its response fits the definition:
Have a number of in person closed book tests that are difficult enough to prove satisfactory understanding of the topic.
Homework is just a mechanism for a teacher/professor to force a student to dig into and really understand a subject. If the student can use these AI tools to subvert that, then instead of wasting energy or god forbid even more AI to fight the AI, just give students a good enough incentive to actually do the homework. Having the entire grade rely on several of these difficult-enough in-person closed-book tests should be incentive enough.
I think it would be hard to ask any kind of pre-college mathematics questions that don't become incredibly easy with an open book. The other challenge with open-book exams which focus on deeper understanding and reasoning is they still require understanding fundamentals to be successful, so how do you test for fundamentals?
Homework isn't effective at making students "dig into and really understand a subject" - it's a method of externalizing school expenses onto parents, because teacher time and school funds are finite, and the time of parents and children is... not the school's problem. Making the final grade conditioned on homework performance is just about the only way to ensure kids do it (or are made to do it) at all.
Now, I'm not saying homework is entirely a bad idea - I'm complaining that it's being treated as an externality, making it almost unbounded. Teachers don't coordinate among themselves the amount of homework each of them assigns, so the only thing limiting the total amount of homework is a good chunk of the class being systematically exhausted by it, or (hopefully) parents making a fuss before things reach this level. But, the only reason the teachers can push so much homework on kids in the first place is that homework grades matter for the finals.
If you have the time to read my other comment, what would you think about the idea of only making homework mandatory for students who's quiz/test grades are below a certain level, say 70% for example
What makes me really unsure, however, is parents. Families differ by the amount of time and resources they can, or are willing, to spend on child education. This is part of why I don't like the homework-as-externality model: even ignoring how students themselves feel about it, if the school is trying to maximize the amount of work they do after school, the first kids to hit the limit are ones with e.g. a single parent doing two jobs, poor household, or dysfunctional family that doesn't care. Overloading students with homework implicitly disadvantages those that don't have supporting parents with lots of free time. And those same kids will also be ones more likely to fail at tests, in which case dumping extra after-school work at them might do the opposite of the effect you intend.
But this is me speculating, I don't work in education, and I'm sure there's been research done on how to balance the amount of after-school work for the environment children live in. My complaint about homework-as-externality isn't trying to deny the work of education sciences - it's pointing out that even if the research is there and results are solid, it's not being applied anyway, because teachers are uncoordinated and they all individually think, "oh, that's just half an hour worth of work, no big deal". Almost textbook tragedy of the commons.
In the reading I've done, I've seen most advocate for smaller class sizes with more educators per class to help provide more 1:1 attention. This is again at odds with how public schools are funded where I am (US) so I don't know if anything will ever change.
My personal experience is homework was an excellent way to reinforce what was taught in school. For me, it felt especially useful in quantitative subjects and languages (French & Spanish) which both felt like they benefitted from having a concept stick. For qualitative subjects like writing, reading comprehension, I actually now look back and see homework as a way for teachers to see samples of your progress. "After reading this passage, did you learn how to construct a summary? Did you learn how to pull the author's main message? Did you learn how to make a convincing argument for or against the passage's main point" and I can't think of a fast way to do this in any kind of live instruction setting.
If the student can perform well on the test without that, then they shouldn't be forced to do it and penalized when they don't(which you can imagine happens often, as if the student understand the material well enough already the homework is just useless busywork.
If the student can't perform well on the test without that, they will be forced to do the homework. In this case, including homework as part of the grade might be detrimental because they can cheat/copy/us AI to complete homework and boost their grade enough to pass while not understanding the material well enough to pass the tests.
The counter-argument here(that I can think of) is that doing homework, even if - or especially if - it's just easy busywork for the student, prepares them for the future where such tasks will almost always be necessary to succeed. A lot of good learners end up not developing this skill/discipline because they aren't forced to work hard in school, which causes them to fail when faced with tasks/problems that require serious effort, and this would arguably make things more difficult for them. In my opinion this problem would be better addressed by funneling kids into classes that are appropriately challenging for them rather than forcing them to do busywork, but that's a much more difficult thing to do, and also learning to complete easy/"useless" busywork is a related but different skill than learning to focus on and complete challenging tasks
The reason I built that was just because I was annoyed by how lazy my professors were. Almost every single class (which I was paying for) was graded based on your performance on copy and pasted, endlessly reused tests on Blackboard. Sometimes you could just search the question on Google, and find pdf dumps of the exact same test from 5+ years ago. If you don't cheat, you risk getting a lower grade than the people who do cheat (aka everyone). Why spend all that money going to college if you're just going to let other people get ahead of you so easily? The point of degree is to make you more competitive in the job market, but deciding to not cheat is risking that investment.
Unfortunately, I never actually used it or deployed it once. Coordinating a whole class to install and use a chrome extension for cheating isn't exactly easy. And as far as cheating in online tests goes, there are easier ways to do it.
But yeah, in-person proctored exams are how it should be done. It's the only thing that's fair to everyone.
Imagine your employer gave you "homework".
People would pull out pitchforks because it violates their private life and work-life balance.
Yet doing this to kids somehow is believed to be okay!?
- Kids deserve a fixed point in time where their daily work is over, just like you do.
The teacher was a quite old man who was set on his ways. We would have to draw 3D pieces by hand, using our squares, bevels and compasses, and lining everything with an ink pencil afterwards, and he gave us a lot of homework to do, which he examined for imperfections.
I was incredibly bad a it. I would start a drawing, only to discover halfway through it that it didn't fit the sheet of paper I had and I had to change the perspective. My perpendiculars were never perfectly aligned. I would smear blots of ink with my tools and hands.
It was excruciating.
This is the point where I would usually say that I learned to enjoy the repetition and eventually came to appreciate it and be good at it. But that's not what happened.
Instead I started doing my homework using the computer, printing, and then tracing the points by hand. This resulted on cleaner-looking homework and took less time to do. What's surprising perhaps is that it enabled me to appreciate design more, now that I was no longer limited by the drawing technique, which was, I believe, my teacher's real interest (he was a painter outside of school. A very nice person, I didn't enjoy lying to him like that).
You might think that this would prevent me from performing well on exams, where a clean enough drawing (done by hand) was expected. The thing is: I could muster the effort to do a one-off drawing that was clean and precise, from time to time. Just not after every lesson. I actually got the maximum qualification on my province's technical drawing examination.
This is all to say: I agree with the Author. Once the tools are out there, trying to limit them is pointless. You must adapt your curriculum to the environment your pupils are in. If you don't, they will.
Nowadays, when I teach very young people what AutoCAD is, I make them use a mini rolling ruler and tell them to watch youtube videos of how to use a rolling ruler. My understanding of CAD is high enough to know that every tool in CAD was inspired by and modeled after the rolling ruler. Sufficient to say, they understand and enjoy CAD very much after that point.
A similar line of thinking was popular in jiu jitsu, where veterans said that being competent with the gi was necessary to be competent in no-gi. Yet current no-gi winners basically never train with the gi.
In Fact, Modern programs in CAD do not follow the tools and principles that AutoCAD emphasizes. And those programs have charted a different course where the history and legacy of technical drawings has less relevance. (Case in point: Visio and maybe Powerpoint.)
For example, with regard to technical drawing, we might say that using the pencil, the square, and the drafting machine gives an understanding of drawing that the use of the computer cannot. But, for some reason, we don't say that using your finger and sand, or a rock and string to draw a circle helps even more. And we don't say that simply because the people who used rock and string (the good old days) are long dead.
The same goes for clothing, for which we say the 1990s, or perhaps the late 1960s, were better, but no one says that between 1900 and 1910 people dressed with such great taste.
The same goes for romantic partners ("if he hadn't dumped you and made you cry for a year and a half, you wouldn't have found your wonderful wife/husband"), and many other things.
It is very human to rationalize the circumstantial.
I like how people dressed in 1900s.
Let's take men's suits. They were made of heavy wool (today heavy wool is demodé, and rightly so because it is very uncomfortable), with three or four buttons (today 3-buttons are rarely seen, and rightly so, except for the 3-roll-2, which is not a "real" 3-button, 4-buttons are nowhere to be seen), the buttons were very high (the Neapolitan suit has high buttons, but much lower than the buttons on the suits of the early 1900s), and the shirts had high paper collars, which are nowhere to be seen. And the hats?
There is nothing current that recalls the clothing of those times.
The clothing in "Peaky Blinders," which is a decade older, is something like a fedora nowadays, please.
Now, almost 30 years after my days of technical drawing, I must say that I consider those hours spent drawing a terrible waste of time, partly because I did not go on to the career for which technical school was preparing me, and secondly because technology has made pen and paper obsolete and I see no particular ability to carry over from hand and square technical drawing to CAD-assisted design.
The latter is an important point to consider now that we are moving from a non-AI world (like 10 years before my time in technical school) to a little-AI world (my first 3 or 4 years of technical school) to an anywhere-AI world.
They also absolutely shouldn't have learned to play musical notes since computers can do that.
My point is that learning the very basics of every human endeavor make you better prepared for any situation and will give you a better and deeper understanding.
In a CAD world, there is no reason to use pen and paper and a drafting machine. If I were to start learning technical drawing, how much time would I spend with the drafting machine? The answer is zero hours: there is no "market" for it (is there anyone who asks "please deliver without using CAD"?), it is not preparatory (as I wrote in another comment, is there anyone who is starting to learn how to play tennis using wooden rackets?), and it takes much longer to do anything with the drafting machine than using CAD.
Learning addition and multiplication is preparatory and useful. Being a Luddite is rarely useful.
In two years? Look where the space was two years ago. I think many things will have to change.
* canned laughter *
The answer: "One way to make it clear that a text is not generated by AI is to include a joke or riddle that is difficult for a machine to understand or generate. For example, you could include a joke that relies on wordplay or a complex pun. Here is an example:
Why was the math book sad?
Because it had too many problems."
Good thing ChatGPT doesn't know any complex puns or witty wordplay!
The thing is, I'd see these answers as similar to everything else the program produces. A bunch of claims from the net cobbled together - I've read a number of Sci-fi novels and stories where "inability to understand humor" is the distinguishing quality of an AI (I'm guessing it extrapolated "hard create" from "hard to understand"). But that doesn't seem to be playing here where the AI mostly running together things humans previously wrote (and so it will an average amount humor in circumstances calling for it).
A reasonable answer is that the AI's output tends to involve this running-together of common rhetorical devices along with false and/or contradictory claims within them.
-- That said, the machine indeed did fail at humor thing time.
The question here is this an actual AI only failure mode. Are we detecting AI, or just bullshittery?
I would say that human knowledge involves a lot of the immediate structure of language but also a larger outline structure as well as a relation to physical reality. Training on just a huge language corpus thus only gets partial understanding of the world. Notably, while the various GPTs have progressed in fluency, I don't think they've become more accurate (somewhere I even saw a claim they say more false thing now but regardless, you can observe them constantly saying false things).
And the idea that the computer would “try” to come up with an example that would trick a computer is itself a little funny, in that it has fallen into giving itself a preposterous task.
But it did definitely fail at clever wordplay.
There sure is some obscure discussion forum where users talked about that or some amateur writer that published online something in those lines. ChatGPT is just a statistical device selecting randomly from previous answers.
Of course, in real time the attempts at humor often fall flat and might give away flawed thought processes, although I personally have found them to be often insightful, (containing a seed of humor) even when they're not funny. It could be a useful technique when actually having a conversation, a form of Voight-Kampff test, but I don't think it will do anything to let you know if the content was generated by AI and then just cherry picked by a human.
I can just fine-tune a large scale model on a small downstream task, or use creative choices of decoding settings (high temperature, alternative decoders like contrastive/typicality sampling), to fool the existing methods.
Some times, though, the answers are false but plausible.
But I do agree with your overall point, that schools and universities will have to embrace these new tools. Vernor Vinge (who was an SDSU maths professor) wrote Rainbow's End on this very subject.
I mean, human beings have produced a lot of tools for aiding themselves in tasks that were previously done by hand. Whether a given hand-skill should be learned and preserved kind of has to decided on a case-by-case basis. I learned cursive in school and I wouldn't object to that skill being utterly forgotten by humanity.
Which is to say that I don't think we should assume all learning should begin without automatic tools at hand or that we should suppress automatic tools. Rather, we need to look carefully at which hand-skills are worthwhile and "uplifting" and how much use of those skills should be required.
I did just that. It wasn't a life drawing class, just a drawing class. And it wasn't homework, but the do-it-at-home final. I photocopied my hand. Then I covered a sheet of paper with ground graphite (turning it all nearly black), and then used an eraser and a light-box to trace my photocopied hand. Did I cheat? Perhaps, but I used tools that I had access to. Did Vermeer cheat? [1] Perhaps, he may have used tools he had access to.
Afterwards, I framed that drawing and hung it up at work. It was stolen. Go figure.
[1] https://en.wikipedia.org/wiki/Tim's_Vermeer
We don’t give runners roller blades right?
The context surrounding both scenarios couldn't be more different:
GP's context: An assignment wherein the process of getting there isn't placed on a pedestal, rather that it's supposed to showcase a basic understanding of how to get to a desired result. Furthermore, objectivity within the assignment's field is nearly non-existent.
Your context: A highly-competitive scenario wherein *BOTH* the process of getting to the goal & the goal itself are important, and where *there are* objective metrics that can be used to rank & sort the participants involved in said competition.
They're as different as apples & oranges.
[1] https://en.wikipedia.org/wiki/Oscar_Pistorius
Don’t answer the door.
Keep in mind that the reverse is also true: If he did try to do it all manually & without the tracing method, is his use of already-available graphite "using tools that you have access to"? Should he be made to go to a mine & dig out coal for graphite in order for it to count as his work?
That argument doesn't make any sense. There are restrictions for any homework, some explictly given by their professor (such as the deadline), and some implicitly known from general academic guidelines and common sense (such as "not using a photocopier in an art class"). Obviously mining coal is not among those restrictions.
...is it?
"general academic guidelines and common sense" is a subjective term that is (context, culture, & environment) dependent: If the social norm was "Your work isn't valuable unless you made the materials yourself", then the use of non-self-made materials (graphite not mined by yourself) is frowned upon.
Relying on "general academic guidelines and common sense" is not a good thing, as it helps to reinforce unequal social norms & anti-non-heteronormative ideologies, as a downstream consequence of majority rule by statistically normal people. Neurodivergent individuals that have significant difficulties grasping "common sense" will & already have been negatively impacted under such a system, where they're told to just "get it" with no guidelines whatsoever.
If a neurodivergent person uses a photocopier because they thought that it would be fine, then yeah, I'd understand that. The person who started this thread stated that they were perfectly aware that they were cheating, though.
Vermeer may have done effectively that and painted projections (from a camera obscura or similar device). He's still widely considered one of the greatest artists of the Dutch Golden Age.
A modern version of it can be found https://neolucida.com
Needless to say, I sucked at it, and it was the one and only class that I ever failed. I passed with flying colours the second time around, and still graduated in minimum time, though it badly messed up my schedule in my second year.
I have never forgiven the department for putting me through that, given that AutoCAD existed at that time.