HN Explorer

Ask HN: What are the most interesting takes on Generative AI you've come across?

53 points by ChaitanyaSai ↗ HN
We are tens of months into what looks like the AI age. (If you disagree, I'd love to see interesting takes on why that is). It is too early to tell how the landscape will evolve, because the landscape is vast and we do not know what parts are going to get terraformed. Would love to hear about interesting takes/predictions/uses that go beyond the usual breathless twitter/x listicles. Please do share!

59 comments

[ 2.2 ms ] story [ 147 ms ] thread
belugacat
This article:

https://www.noemamag.com/ai-and-the-limits-of-language/

amelius
> An artificial intelligence system trained on words and sentences alone will never approximate human understanding.

I bet there are physically impaired people who prove that this is very much not true.

CamperBob2
Anyone who can watch that "simulated Attenborough" video and still say "An artificial intelligence system trained on words and sentences alone will never approximate human understanding" is living on a different plane of reality altogether.

Either that, or the humans they know are a lot smarter than the ones I know.

WoodenChair
I think one of the least headlined, although perhaps not by definition interesting takes, is just one of moderation. This is not AGI (in fact nowhere even near close to it), nor going to unemploy quite as many people as the doomers predict, yet it’s also more than just a stochastic parrot as the naysayers put it. It’s a great new set of productivity tools with zero true existential threats except to some specific creative job categories (marketing copywriter, summarizer, etc).
xnx
> We are tens of months into what looks like the AI age.

The modern AI era started in 2017 with the "Attention Is All You Need" paper (https://arxiv.org/abs/1706.03762). ChatGPT is a popular manifestation of something that has been making huge and significant progress (image recognition, language translation, image generation, etc.) since then.

This blog post has helped me the most in trying to understand what LLMs are, how they work, and what they might be capable of: "Prompting as searching through a space of vector programs" https://fchollet.substack.com/p/how-i-think-about-llm-prompt...

bsenftner
That's the "beginning of the start". We'll be "finished starting" when a market worthy demographic establishes itself as the "pro-AI may-as-well-be-religion" camp, disregard any official or scientific reasoning, and declare their own definition of what AI is, which they basically worship, but with enough words that they don't call it worship, they'll make up their own words, like they invented the idea of religion itself, but in a new AI understanding that is really their worshiping themselves. And they'll be enough of them they'll vote themselves into office.

AI's capabilities is too much for the general public to handle, and all bets should be recognized as "off". The "AI Age" is going to be a lot of misunderstanding, a lot of propaganda, some very visible out of control ambitions, and too little critical thinking. At least until the colossal attempts at large scale automation fail, killing millions.

kjkjadksj
Its even older if you want to focus on when these ideas started being written about academically. Markov chains are from the early 1900s. Bayes theorem is from the 18th century.
xnx
Definitely. I see it as being similar to the history of flight. There were ideas and attempts for hundreds of years, but the Wright brothers flight was a turning point.
TheAceOfHearts
Yan LeCun is generally advocating for open models and regulation at the product level which is very closely aligned with my own ideological beliefs so I keep track of everything he shares on social media. There's too many doomers sucking up all the air in online discourse, IMO.

In general I've just started following the actual researchers on social media to keep track of what they're saying or their perspectives on various issues. Go directly to the source.

ChaitanyaSai
Can you share a few worth following (with reasonably high signal to noise)? Thanks!
LeoPanthera
I have an LCD picture frame into which Stable Diffusion generates a landscape photo, with the twist that the weather in the landscape is the actual current weather outside.

I built it just for myself. I think it's hilarious. Everyone else I've shown it to has been less impressed!

Edit: This is what it is currently showing: https://ibb.co/T2b4S4M

owenpalmer
I've always wanted to make something like this. Except it would generate an image based on the transcription of the conversation in the room.
slaucon
You should absolutely make that.
randombits0
Oh, that could go off the rails quickly. You have to do it!
appplication
I like to think it would start to gain sentience once someone commented on it, and draw a bunch of people gesticulating at a recursive image of itself
pavel_lishin
As a guest, I'd appreciate a heads up that there's a device listening to our conversation and beaming it directly to some company.
owenpalmer
Completely possible to run it 100% local (;
cheschire
I did something similar using Leonardo.AI (it's cheaper) to give me random vibrant abstract christmas oil paintings every hour.

What are you using to get the picture into your frame? I'm currently running my python script on a raspberry pi 02W, but was wondering if there's a better way.

DontchaKnowit
Thats insanely cool. Idk how anyone could not be impressed with that. Super creative!
naet
Wow, I love that! What kind of frame are you using?
LeoPanthera
It's just an old TV hung on the wall hooked up to a Pi.
jholman
This would be a cool thing to publish your work, because I too think it's pretty hilarious.
Turing_Machine
You may need to get some new friends. :-)

That is a great idea.

robg
Far from an AI century, we still don’t understand how brains create and drive intelligence. The more we learn about actual intelligence from brains, the more artificial versions will approximate what we’ve evolved to do. Modern neuroscience is younger than neural networks.
woctordho
Shameless plug: We're building a dataset of visual novels for AI training https://huggingface.co/Synthia/ChatGalRWKV

Visual novels are multimedia data with thorough plain-text annotations, and generative AI will greatly accelerate their development

ganzuul
We got here by upscaling compute. To me it feels like we need another 100x or 1000x upscale to get to the stage where AI can actually begin to think about making actionable plans. To get there we need to address the efficiency problem.

Nvidia's take on Minecraft remains the most interesting exploration of the capabilities of LLMs. In that research they had an LLM build a skill library of code which let it get achievements.

mmckelvy
One of the frequently raised concerns is that soon we will be drowning in AI generated blog posts, articles, presentations and emails, and most of this AI generated content will be meaningless noise.

I wonder if the opposite may be true. With the advent of AI, will there actually be _less_ meaningless noise?

When I was in finance, we would regularly produce 100 page decks for client meetings. Usually only 2 or 3 pages of the 100 page deck would really matter. The rest was what I call "proof of work". _Look at how much work we did for you. Isn't it impressive_? With AI, that kind of proof of work no longer makes any sense, so maybe all those 100 page decks, marketing blog posts, investment memos, and white papers will slim down to only the salient points and in many cases vanish altogether.

not2b
There will be more noise, at least in the near term, so people have more "content" to attach ads to and more chances to win the SEO lottery, and people use generative AI to pad out their work to look more impressive.

So, we'll waste tons of resources as presenters say "GenAI, pad out this outline into a 10-page presentation" and people receiving the presentation say "GENAI, summarize this 10-page presentation and give me the main points". Anyone who tries to short-cut this by just sending their outline will be accused of being a shirker.

bick_nyers
Even if 100 page decks don't go away, AI can be leveraged to find the 2 or 3 pages that actually matter.

Google search has become worse and worse over the years, I could see using an AI plugin that performs a search on top of the Google search results (or aggregate all search engine results), pre-filtering SEO garbage, pre-filtering advertisements that are LARPing as information, removing the top 90% of any cooking recipe that is a diary entry about why this beef stew recipe changes the chef's life, and so on. Perhaps also the AI plugin could slightly tweak your Google search query to better align your search intent. Preferably running a local LLM plugin to do this for you so that you aren't steered towards "sponsored content".

If we do end up drowning in AI generated content then at a certain point you will probably have to use AI to combat against it, fundamentally changing how we currently use the Internet for information retrieval.

Adversarial neural networks playing out an information arms race on the Internet in a nutshell.

Shadowed_
Bing Chat does that. It searches internet, goes through results and extracts information to generate answer. Also, gives references as links so you can consider them (filtered) search results.
kjkjadksj
I wonder if people holding those fears are even using the internet today? Its been this way, full of automatically written noise for decades perhaps now. The limiting factor is not what method you use to churn out junk content. Its probably just limited by cost of compute and hosting, things that have been “too cheap to meter” for these sorts of junk content websites for a very long time now.

The future ai spam internet will look exactly the same as the current automated spam internet, is my guess.

pixl97
The people holding the fears are the ones currently paying out the ass for anti-spam system to try to block it. It's forking expensive. And will only get more expensive as the spam gets better and more human like.

This will further push consolidation behind huge companies like Google and Cloudflare.

euroderf
> "proof of work"

Isn't it the case that if you ask an LLM how it arrived at a "conclusion", it can't detail its chain of reasoning because there isn't one ?

So proof of work might mean: proof of human work.

Beg pardon if I'm missing something important here.

RecycledEle
For those of us who have long commutes to and from work, VoiceGPT is amazing. I can talk through a problem I've been trying to solve with a tutor who helps me understand it. Meanwhile, the guy in the truck next to me is road raging and about to run over a slow Volkswagen. I prefer my deliberative talks.

I think this is a game changer for students and people in training.

ChaitanyaSai
Isn't the conversational medium too linear and possibly slow? Can you share a specific example of a problem where this approach is interesting. Thanks!
RecycledEle
I was driving home from work Thursday night, and had to go for 90 minutes in stop-and-go traffic. I had an idea about a new architecture for a computer where the CPU, GPI, and I/O processor would all be socketed in identical processor sockets. I was talking through the busses with VoiceGPT for about an hour. (I think VoiceGPT is the name of the voice interface to OpenAI's ChatGPT. It is available with a ChatGPT Plus subscription.)

Note: I've been told my 3-processor idea is a horrible mistake, but I like it.

On my way home from Church, I can talk through something the pastor said.

On the way home from a class, a student can talk through something they did not quite understand in class. ChatGPT sometimes explains things in a way so that people who did not get it before understand it. If I still do not understand something, "ELI5" is my standard second attempt. VoiceGPT keeps answers short, so I do not have to go back through long responses as often, but if there is something I do nt udnerstand in the answer, I can drag it out for many more prompts.

A teacher (like myself) can assign students to talk through whatever thing(s) they did poorly on during an exam with ChatGPT or another AI. If the students drive a lot then they can do this with VoiceGPT. The transcript can count as a grade. I do this in my community college classes, but I expect my students to use the free version and type instead of talking.

For the first time, every student really can master every concept (subject to limitations on their time.) The magic here is one a student works hard enough to get an A or two in a field, classes that build on that are really easy.

I have learned some great prompting tricks. For example, we learn things when they are relevant to us. So, my first question is usually "I am a _______ working on (or learning) _______. Tell me why I should be excited about _______."

Another trick I use is: To reduce hallucinations, I do not ask "Tell me about _______." Instead, I ask "How familiar are you with _______?" If I start distrusting the answers, I start a new chat window and try again with revised prompts.

I can imagine future professional development where (for example,) a teacher realized they are not good at something and chats with an AI until they AI thinks they have mastered the concept. Imagine getting monthly 30-minute quizzes, and based on what you score, you are assigned professional development chats and simulations. For me this is much preferable to sitting in long meetings.

There are definately things that are too fast to do with voice. I could not follow VoiceGPT's math when it talked out math in the billions of transactions per second on busses on a hypothetical motherboard. That is why the transcription is awesome.

It's also nice to be able to tell VoiceGPT "I want you to take an note. Do not answer, just take a note." after it agrees, tell it what you want to take a note on, then go back to the conversation.

markusde
How's this for interesting: many people in my field (formal methods) seem to be pretty excited about our job prospects. Before we used to just say that people don't actually know what their code does, but now it looks like it might really be true.
trealira
> How's this for interesting: many people in my field (formal methods) seem to be pretty excited about our job prospects.

Wow, I hadn't thought of it from the generative AI standpoint, only from the standpoint of "cybersecurity is increasingly important, so we need to use proof assistants and verified toolchains more".

What kind of new career prospects in formal methods do you predict will come as a result of generative AI? For example, certifying AI-generated code for use in automobiles?

markusde
Full verification is one, but that is still challenging at scale. Formal methods has many weaker methods as well which are easier to apply in general (automated proof, abstract interpretation, static or dynamic model checking, hell some would even argue the judicious use of dynamic checks is a kind of formal methods).

I think that a shift from "this is my program, it's a sequence if steps that does what it does" to "I asked a LLM to generate me a program that does this vague thing I want" makes you naturally ask questions like "wait is it actually doing the thing I want" and "what do I even want it do do". Formal methods, broadly, has a lot of good answers to those questions.

diarrhea
It’s true. We don’t. But does it matter? Is there demand to change this?
tkgally
For me, the most valuable "take" on generative AI has been the collective discussion on HN. While many individual articles and comments have been useful on their own, it is the ongoing conversation here that has given me the best view of how AI is starting to impact society and what people think about it.

The HN commentariat is not, of course, a randomly selected, representative group, and the discussion here can be repetitive. But, as a whole, it is much more insightful than any individual article, paper, or blog post.

JoeMayoBot
I'm thinking about the new jobs that are/will be created.

1. Prompt engineer - there's been a lot of talk about this, though I believe it's more extensive because businesses will need people to educate, manage prompt data stores, and assist with fine tuning.

2. Content management - as companies adopt AI with their own data, someone will need to manage the content going into the system including selection, privacy, and security.

3. Content Moderators - people who write/edit content will need to change their behavior about how the content is created and formatted, making it easier to ingest and lead to higher quality answers.

4. Content Creators - people who create content for the sole purpose of ingestion. This could be within a company, open-source/scientific research, or supporting vertical models.

5. Security Monitors - This is the person-in-the-middle who's watching/monitoring the system for privacy, safety, and security.

There are probably more, though this is what I'm thinking right now.

kjkjadksj
Prompt engineer is probably going to be a fleeting meme of a profession if we are being honest, once it becomes as trivial and widespread of a skill as any other basic piece of modern computer literacy. At one point in the transition from analog to digital, people hired others who could use word processors and other such computer tools on their behalf. They might be formally employed as a secretary or assistant but in either case that was the role at a time: an “I dictate you email” sort of arrangement you occasionally still see today if you work with any senior silent generation folks still. But over time all those others who might have hired someone like this learned how to check their own email, and write their own documents, and make their own slide decks, and this obviated a lot of extra jobs along the way.
klntsky
No, there is deep hard to formalize knowledge one can gain with practice. Moreover, it is model-specific. A better name would be "prompt magician".
EricMausler
"We're sorry but this role is looking for someone with BrickBreak AI prompting experience and we don't think your experience in PONG-AI prompting is a fit here. We'll reach out if something comes up we think you would be a better fit for"
apstls
I've been wondering if there's a chance the inevitable explosion of hyper-realistic disinformation and manipulation content - of course brought on by genai significantly reducing the cost and barrier to entry to very high-volume realistic multimedia content production - could make the public digital information landscape so obviously polluted and cacophonous that even the most oblivious of media consumers will begin to have their trust of information from purely online sources (or at least social media) continuously erode, basically solving the problem of online disinformation efforts by destroying the confidence in the medium altogether.
bear141
Disinformation and manipulation is already such an incredibly invasive problem across all media and all sides of political spectrum that I can’t imagine the majority of people ever changing in the way you describe.
sva_
Just read this interesting article[0] about Sam Altmann:

> "We need another breakthrough. We can still push on large language models quite a lot, and we will do that," Altman said, noting that the peak of what LLMs can do is still far away.

> But he said that "within reason," pushing hard with language models won't result in AGI.

> "If superintelligence can't discover novel physics, I don't think it's a superintelligence. And teaching it to clone the behavior of humans and human text - I don't think that's going to get there," he said. "And so there's this question which has been debated in the field for a long time: what do we have to do in addition to a language model to make a system that can go discover new physics?"

But now the board coincidentally fired him, as you can read in the current top post on the front page.

0. https://www.thestreet.com/technology/openai-ceo-sam-altman-s...

pavel_lishin
> "If superintelligence can't discover novel physics, I don't think it's a superintelligence."

That feels like a wildly arbitrary metric.

toddmeck
Too few people are investigating how to determine whether or not Gen AI provides good answers which improve efficiency. Many people are giving examples of where GenAI is fast, but requires more time and effort to investigate the "rabbit holes" and misleading solutions. So, the net result is not efficiency or cost-savings, but failure demand shifting costs elsewhere.