62 comments

[ 2.9 ms ] story [ 104 ms ] thread
The world is going to become stupider because it’s easier. This will be most dramatic for the youth in both school learning as well as in the field. Unfortunately I don’t realistically see another path in any reasonable time unless the world takes a radical turn away from AI (extremely unlikely).
Old people say this about young people all the time.
And they’re usually right.
The data (re: the Flynn effect) suggests otherwise.
There are explanations to the Flynn effect that go beyond isolated intelligence. How does one isolate intelligence test when society, nutrition, religions, educational systems, and the very IQ standard itself has changed?
I get it, there’s work to be done to flush this out but it’s interesting to me that actual scientific enquiry gets more push back than “but old guy said so.”
Surely it holds more merit than “young dude with no experience who read about it in latest reddit group / social media app said so”.
Old people have never entered a world of GenAI before.
Old people have a terrible track record. Their base rate is nil. It would take quite a bit of evidence for me to update my prior.
In your opinion, what's the key difference that makes AI lead to world becoming stupider compared to other technologies that lead to more widespread knowledge, such as writing, book printing, calculators, computers and Internet?
Because all your writing is being done for you. All your homework is done for you. You go to start coding as an engineer, you just copy/paste from an AI until it runs. No need to think, no need to try to figure it out for yourself, no need to learn anything.
Great, go get yourself 4 phds and a 600k/year job if ai can do everything for you! Meanwhile the rest of us in the real world will continue understanding the limits of the tools we use.
1 PhD was enough for me.

I think you’re vastly overestimating the amount of work people either want to do or pay for. A worse version that’s nearly free or instant will win regardless of limitations the majority of the time.

Both those things (PhD and high salary) require human interaction to achieve, not knowledge or tool use. There are many examples of people achieving both/either whilst being ignorant and/or unintelligent.

The comment was about "you're not going to know anything because the LLM is doing it for you" which is easily obviously true. This won't stop anyone from getting a PhD or a high salary. It will just stop them from knowing things, and possibly being able to build cool stuff (specifically in the example of software dev), though even then you could argue that a good working knowledge of how to prompt an LLM is roughly as performant as a good working knowledge of a software language.

> you're not going to know anything because the LLM is doing it for you" which is easily obviously true

Not obviously true to me. LLMs don’t know everything, so they can’t solve all problems. You can try and give it the proper context but you still need to understand the context is necessary, which requires knowledge.

Kids who grow up today will understand and intuit that synthesizing 3 facts from an article doesn’t constitute “work” any more than multiplying 4-digit numbers. Adults will eventually pick up on this as well.

yeah, one of the problems we're seeing with using LLMs at the moment is that they don't really work for beginners, because you need to have a certain level of knowledge in order to be able to spot the hallucinations and understand the context.

I've seen this in coding, where as an experienced coder I can spot where the LLM is doing bad things, but if I try using it in a language or environment I'm not familiar with, I get lots of errors that I don't understand how to fix and all I can do is feed back the errors to the LLM and hope it does better next time.

So I guess my point is that if we end up using LLMs for everything then we have a chicken & egg problem - LLMs don't work for beginners, but beginners don't learn anything while they're using LLMs because they can't get past basic errors.

Your point about multiplying 4-digit numbers would be valid if calculators often made basic maths mistakes, and you can only really use them if you already know the approximate answer so you can detect when they've made a mistake.

With having internet in our pocket we no longer value being able to recall information. There is no need when anything is a query away. I suspect a similar thing will happen with our ability to think. What is the value in being able think through and solve a problem when you can just get your phone to do it instead? And I would argue in reality most people won't care if the results aren't correct so long as they are plausible and fast.
They probably said the same thing when the printing press came out. “How can you really know the book if you didn’t copy it down yourself during a reading?” But the printing press won out in the end.
The printing press takes your ideas that you already wrote and increases distribution. Deep research is writing the text for you. That’s completely different.
I disagree, it’s just another step in the automation of knowledge distribution that’s been going on for a long long time. Every step someone has lamented that we lost something, maybe starting with speech itself, and the invention of writing taking away from oral tradition, scrolls, books, the printing press, the World Wide Web, etc…

I was just wowing to myself the other day that a local 70b 4q 40GB model on my Mac basically replaces Wikipedia for me and can be used offline in an airplane. A whole library can be replaced with a cube of knowledge that is just a wrapper around an LLM.

Of course there is going to be resistance to any move up, there has never not been resistance.

It's like wikipedia except it spits out incorrect information at an alarming rate and has no references. Still, impressive that it only weighs 10s of GB when wikipedia alone is on the order of ~100GB
Wikipedia isn’t a completely reliable source either. The LLM also includes a search engine. It’s weird since before I started playing around with it I thought it was just a fancy search engine and was grabbing sources from somewhere to shove into its context … but no, it’s all built in (although as deep research shows, it can do a lot better when it stuffs its context via research).
Just saying that for 24GB you can get all of wikipedia losslessly compressed on your laptop.
While the 400B "knowledge" is reliable and replaces most of my googling, the 70B are not, nor their "reasoning".

So until we can have these models locally, running at 10-20token /sec minimum...

I am fairly hype up over gen AI but we are only at the start of this revolution, and will require a few more years for niche domains, or company knowledge (RAG doesn't cut it, fine tuning too expensive) and we aren't tackling all the media related.

And because of hallucinations,which they require to work, they can't be reliable at 100% or be truly use without supervision.

I feel it looks like internet back in the 90s, we were saying it as a new way to connect people, share knowledge, only to have echo chambers and porn distribution as few take away. I kid I kid.

People didn't read books by making copies of them. Producing new copies of manuscripts was a skilled occupation.
Nope, that’s not true. You had monks copying books of course (mostly bibles), and very rich people could afford very expensive books made by scribes, but students at universities were expected to make their own copy during a reading/lecture run by a reader/professor. The words are still used in some older universities.
I was a medievalist before moving into tech, and I was always taught that students usually could not afford parchment on which to take notes, much less to copy down every word read out to them. Please cite a source for this claim.

Also, people bought books to read them outside of the context of universities.

Most students couldn’t afford books. Here is one ref, which revisits pre-printing press university education in a piece about the last step people complained about (the internet)

https://opencontent.org/blog/archives/6104

> Before the printing press, faculty had to assume that none of their students had books. This led to a widely adopted practice known as dictation, in which faculty slowly read out the text for students so that they could make their own handwritten copies. (This was, of course, not the only mode of instruction. But it was a common one.) Blair writes that dictation was widely believed to have pedagogical merit, as “the act of copying out a text was often considered an essential part of mastering it” (p. 46). She provides a wide range of support for this view, going all the way back to Demosthenes and St. Jerome. However, others argued that dictation hurt student learning as the focus on writing distracted students from paying closer attention to the faculty themselves. (Presumably no one likes to stand in front of an audience and have them all looking down at their phones parchment the entire time.)

> You may be unsurprised to learn that there is a strong economic undercurrent to the conversation about dictation, and that it often pit students against faculty and others. Blair notes that students saw it as “a cheaper way of procuring oneself a classroom text” (p.45). Consequently, a ban on dictations by Arts Faculty at the University of Paris in 1355 “anticipated vehement student resistance to the ban.”

(comment deleted)
The models that exist right now would still be considered alpha LLMs. Even a half decent video game takes longer to make than the time since GPT3.5 launched.

If they still are at this level in 2-3 years, then yeah, it will likely cause a bunch of bad downstream effects. But at least recognize that it is still a few years too early to judge.

Students are already using it for their homework, now.
(comment deleted)
Allow me to introduce The First Step Fallacy. https://thebullshitmachines.com/lesson-16-the-first-step-fal...
It's an inductive claim yes, nevertheless it's based on mountains of evidence. Nobody is claiming to have logically deduced that AI progress is inevitable, we might all drop dead tomorrow or stop working on it, but the trends are clear
> mountains of evidence

Yes? For very small values of 'mountain'.

On the other hand, a motivated person can now pursue their interest in a subject of their choosing, at their pace, as much as they want. LLMs are great tutors if you use them in this manner.
> The world is going to become stupider because it’s easier.

Wasn't the same thing when we switched from books to web? We lost the ability for long reads, and instead just search and click directly to specific information, losing the larger context. And we adapted, we have a whole search culture, reputation systems, new patterns of interaction with information.

Web didn’t really replace long reads. You didn’t have a searchable question that you’d previously read hundreds of pages for. More and more short form content started coming out, then social media, all of which both replaced books as entertainment while simultaneously reducing attention span. But it’s not just online media - look at modern tv/film and watch old… the cuts were so much slower previously. That’s probably caused by the changes from web/social media, more than the other way around.

However the extent that the web could replace your work is very minimal because you’d be copy/pasting some existing untailored source. Now, we can take your tailored question / task and have it just completed for you. Doing is a huge part of learning, and if you don’t do, you won’t learn.

We could fix it all with a single law: All federally funded research papers must be published with full open access.

The issue is the models can mostly only access abstracts (same as the majority of humans!) and thus we are back to the dark ages where knowledge is hoarded by the aristocrats in their ivory towers of academia.

>We could fix it all with a single law: All federally funded research papers must be published with full open access.

Two things wrong with this sentence:

1. All federal funding for research is in jeopardy. https://www.nbcnews.com/science/science-news/trumps-nih-budg...

2. The executive branch now considers itself superior to the legislative and judicial branches, so actions of the legislative branch (i.e., laws) might as well be moot. https://www.pbs.org/newshour/politics/vance-and-musk-attack-...

1. tax funded researchers did not publish non pay walled. right?

2. as per link text, there was denial to access treasure dep records. so what is secrecy in there?

so let just append '... and data' to the root of thread

It’s not about access to data—it’s about removing Congress’ control of the federal government. That control is maintained via laws and courts.

Today the president just formally assumed power over those other two branches of government, so law is effectively meaningless now.

https://www.reddit.com/r/law/comments/1isvzgu/the_full_execu...

Whose side will you be on—-the side of billionaires and the christian nationalist heritage foundation, or the rest of the people of the united states? Choose wisely.

Right now, today, nothing stops academia from publishing every paper they write to be open to all. The internet is there and publishing is free.

The fact that they don't implies that they perceive some value is provided by publishing in the traditional way.

Assuming that journals die as a result of your law (why would people pay for a journal anymore?) is it worth considering that value (perceived or real) before advocating it is gone?

For example, consider a news article which starts with "a study published in Nature shows that..." versus "a study posted on Joe's blog on the internet indicates..."

For better or worse (and often with mistakes) journals act as a filter. Researchers in a field can "keep up". They can spend their time on "good papers" rather than trying to sift the dross of the internet.

Researchers, and by extension the place they work, gain karma points when publishing in hard-to-publish papers. If other papers cite their work they get even more points. Ultimately academia (employers and peers) ranks people using these points.

Does this mean your idea is bad? Not necessarily. But it's worth understanding that your proposal would "fix" one problem, but introduce a bunch of new ones.

Another (equally flawed) approach to solving the problem is that the govt simply pays for journal subscriptions to anyone who asks. This means "open access", with benefits limited to "their" population. But the values offered by journals are preserved. Of course this incentivizes journals to increase fees.

> they perceive some value is provided by publishing in the traditional way

Yes, but it provides value only for scientist careers, not for science.

well it also gives some indication of what to read too, hard to filter a firehose on top of your dayjob, but it’s hard to believe that it needs to be this expensive
>> Yes, but it provides value only for scientist careers, not for science.

That's a very definitive statement. But perhaps it is untrue?

It would seem yes, that it's good for scientist careers.Are "good" careers versus "bad" careers good or bad for science? In other words, is it useful to rank scientists when allocating grant money? Do good scientists make better use of the money than bad ones? Are there other ways you could suggest to rank scientists?

Perhaps there are other groups that benefit as well? What about people issuing grants? Is it useful to them that there is industry recognised feedback regarding the scientist and her work? Is this useful when allocating limited funds to an unlimited demand?

What about people following the science? Let's take industry. Say I want to make a commercial product. Should I start by paying attention to the field, understanding the science? Or is it ok to just read any unvetted thing?

What about media, and by extension the population? Can a media outlet run a story based on "something someone wrote on the internet"? Or should they prefer credible sources? Should the public have some interest in the truthiness of something? Should the public (via the media) understand the difference between a result published in the New England Journal of Medicine, or what my homeopath down the road published on their blog?

What about say doctors keeping up with "current ideas"? Should they believe everything, every "study" posted on a blog? Or should there be a system of gate-keeping, sifting the valuable studies from the chaff? Presumably knowing that a study is well formed, and not paid for by say big pharma, might lend it more weight?

Of course publishers benefit (financially) from this system as well. But they don't matter right? So we'll ignore that. But even if you remove them from the equation, I'd suggest that "science" does indeed get value from the system, beyond just scientist careers.

Now, could all these benefits be gained in an alternative way to expensive journals? Almost certainly so. But in order to build such a system it's important to understand the strengths of the current one. A "simple" law might solve one part of the puzzle, but at the same time have very foreseeable consequences in other parts of the picture.

It's a local minima. The big journals were bought up in the 70s and 80s and scientists were hoisted on their own prestige petard. I get the Chesterton's fence thing but this is not a case of that.
There are very cheap arXiv overlay journals where the processing fee is about $10 per submission.[1] And that’s with a for profit entity providing the platform. Reviewing is already unpaid, editing is often unpaid even with traditional journals.

Your “journals die as a result” assumption is faulty.

Not to mention for quite a few fields you can already find every remotely worthwhile paper on arXiv. Those fields didn’t collapse.

[1] https://gowers.wordpress.com/2015/09/10/discrete-analysis-an...

so, given that all scientists and all fields have not immediately adopted this journal, can you elaborate on why some have chosen to not take this approach?

By understanding their reasoning it's perhaps possible to understand what benefits are missing from this model?

If this cheap publishing approach is better for grant providers, why do they in turn not place more weight on previous papers published this way. Presumably if they "prioritised researchers who do this" they would driver behaviour - no law required? Is big-journal leaning on them?

Inertia and prisoner’s dilemma. If an entire field decided unanimously that all existing editorial boards of for-profit journals will resign and form a corresponding Free<OldJournal> overnight and somehow with an inherited impact factor, then everyone except the leeches in for-profit publishing wins. But that’s an impossible ask. Right now, even when an entire editorial board resigns and starts over, the new journal needs to build reputation from scratch, and anyone submitting work (that qualifies for leading journals) to the blank slate instead of leading journals is committing career self-sabotage in a cutthroat environment. (And the publisher can hire a new board to keep the name going, the number of people willing to sit on the board for prestige is proportional to the journal’s prestige.) That leaves tenured folks who don’t need to worry about climbing the ladder, but they tend to have younger coauthors too. That really leaves tenured profs flying solo or collaborating with each other, and who’s fed up enough to do something about it even though they probably won’t have the network effect for a long time, if ever. It’s no coincidence that I linked to a Fields medalist (mathematician).

The next best thing without legislation is publishing preprints to open platforms while leaving the money sucking journals intact. Which has indeed happened in every field I’m familiar with, no idea what’s holding up the others, it’s not like any academic is living on royalties from papers. Inertia perhaps, and negotiation power?

And legislation may actually make a difference here, but hey, politicians don’t typically mess with big money.

Many of the prestigious journals do require an exclusive license:

https://www.springernature.com/gp/open-science/policies/jour...

Under the Springer Nature Subscription licence agreement Share the final published work with peers: Limited sharing for research and career advancement allowed

And it's typically quite expensive to publish under the OA license. I still see no problem with the proposed law. One of two things would happen: 1) Federally funded researchers publish elsewhere or 2) Springer changes their ridiculous agreements, there's no reason they must be granted exclusive rights to a paper just to publish it especially when taxpayers funded the research.

I'd be willing to bet that 1 will lead to 2 in short order.

>> Many of the prestigious journals do require an exclusive license

Of course they do. But nothing forces the researcher to use those journals. So the question becomes why do they? Perhaps there's prestige involved?

>> 1) Federally funded researchers publish elsewhere

That's my point. Since researchers already have this option why are they not exercising it? Why are researchers happy, indeed prefer, publishing with Springer? Only by understanding why they currently choose to use Springer et al, can you understand what is lost by requiring free publishing.

>Only by understanding why they currently choose to use Springer et al, can you understand what is lost by requiring free publishing.

Because we're stuck in a local sub-optima. Those journals are currently prestigious because top researchers publish in them and top researchers publish in them because they are prestigious.

A high acceptance bar may play a role but that is easily achievable and generally does not work anyway based on the number of fraudulent papers that have been accepted. Reviewer comments should be published alongside as should the data.

At the rate things are going, I also hope we continue to have "federally funded research" at all...
We know the limitations of current LLM AI. I didn't read the article. Is there anything in these "dangers" that isn't currently known, mainly hallucination related?

All these anti LLM articles are starting to become as tiresome as the pro LLM ones.

It would probably be fair or "normal" if for every pro-article, regardles of the subject, there would be room for the contrary opinion, for the downsides. That would inject some dose of realism into the discussion.

Otherwise the journalism, in general, is not in such a great shape if it can only regurgitate the same thing over and over, the same words from the same mouths.

Just an idea. I donno.

I have another idea! It's just an idea or a feeling.

My feeling is that work interaction has decreased lately, with all these assistants. I, for one, don't get any questions and don't have any technical interactions not even with junior developers. I don't get asked "how can I implement this, give me some clues" or "do you know a book or some articles I can read on this or that subject?"

I feel the slack channels have also kinda dried up. There's the occasional meme or dog pictures but very little technical discussion, asking for an opinion, framing one's ideas and thoughts on a subject in writing, exchanging opinions with others, etc.

There is still the occasional code review, sure. At this point, I can spot the code written with AI. It has the same feel, the same verbosity, always the same try/catch for everything, always awaits everything, same doxygen-style of comments, always assigning variables for everything. I gave my opinions in the beginning, some things were adapted. But it's a loosing battle. AI can write code faster than I can review it.

Three years ago, some colleagues would ask on slack to proofread their client-facing documents. Now they don't.

Tldr: human interaction is decreasing due to people not needing help from other people that often, similarly to ordering your food and not knocking on your neighbors door to ask for a slice of bread since you've ran out of and everything is closed (we did that, and the neighbors too, 30 years ago)