There’s an interesting… paradox? Observation? That up until 20-30 years ago, humans were not computerized beings. I remember a thought leader at a company I worked at said that the future was wearable computing, a computer that disappears from your knowing and just integrates with your life. And that sounds great and human and has a very thought leadery sense of being forward thinking.
But I think it’s wrong? Ever since the invention of the television, we’ve been absolutely addicted to screens. Screens and remotes, and I think there’s something sort of anti-humanly human about it. Maybe we don’t want to be human? But people I think would generally much rather tap their thumb on the remote than talk to their tv, and a visual interface you hold in the palm of your hand is not going away any time soon.
Computers are tools, not people. They should be made easier to use as tools, not tried to be made people. I actually hate people, tools are much better.
I went through Waldorf education and although Rudolf Steiner is quite eccentric, one thing I think he was spot on about was regarding WHEN you introduce technology. He believed that introducing technology or mechanized thinking too early in childhood would hinder imaginative, emotional, and spiritual development. He emphasized that children should engage primarily with natural materials, imaginative play, storytelling, artistic activities, and movement, as opposed to being exposed prematurely to mechanical devices or highly structured thinking, I seem to recall he recommended this till the age of 6.
My parents did this with me, no screens till 6 (wasn't so hard as I grew up in the early 90s, but still, no TV). I notice too how much people love screens, that non-judgmental glow of mental stimulation, it's wonderful, however I do think it's easier to "switch off" when you spent the first period of your life fully tuned in to the natural world. I hope folks are able to do this for their kids, it seems it would be quite difficult with all the noise in the world. Given it was hard for mine during the era of CRT and 4 channels, I have empathy for parents of today.
I will counter this by saying that my time spent with screens before 6 was unimaginably critical for me.
If I hadn't had it, I would have been trapped by the racist, religously zealous, backwoods mentality that gripped the rest of my family and the majority of the people I grew up with. I discovered video games at age 3 and it changed EVERYTHING. It completely opened my mind to abstract thought and, among other things, influenced me to teach myself to read at age 3. I was reading at a collegiate level by age five and discovered another passion, books. Again, propelled me out of an extremely anti-intellectual upbringing.
I simply could not imagine where I would be without video games, visual arts or books. Screens are not the problem. Absent parenting is the problem. Not teaching children the power of these screens is the problem.
I second this motion. Technology is just a tool. It can be wisely used or not. Just forbidding it, is not wise in my opinion. You have to be careful to use it properly, or course.
Also let me drop the thought here, that Rudolf Steiner, like Montesori and the like, shoot "this is good" "this is bad" based on "feeling" or intuition, or such. There were no extensive scientific studies behind it.
The funny thing is that I remember the exact moment I fell in love with computers at 4. My grandmother cleaned houses and was often very late to pick me up from Headstart. So I would spend hours waiting, unsupervised, in a room with computer that had a giant note attached to the screen saying DO NOT TOUCH.
>:)
By 5, all I wanted was a computer. To me they represented and unending well of knowledge.
I’ve been theory crafting around video games for children on the opposing premise. I think fundamentally the divide is on the quality of content — most games have some value to extract, but many are designed to be played inefficiently, and require far more time investment than value extracted.
Eg Minecraft, Roblox, CoD, Fortnite, Dota/LoL, the various mobile games clearly have some kind of value (mechanical skill, hand-eye coordination, creative modes, 3D space navigation / translation / rotation, numeric optimization, social interaction, etc), but they’re also designed as massive timesinks mostly through creative mode or multiplayer.
Games like paper Mario, pikmin, star control 2, katamari damacy, lego titles, however are all children-playable but far more time efficient and importantly time-bounded for play. Even within timesink games there are higher quality options — you definitely get more, and faster, out of satisfactory / factorio than modded Minecraft. If you can push kids towards the higher quality, lower timesink games, I think it’s worth. Fail to do so and it’s definitely not.
The same applies to TV, movies, books, etc. Any medium of entertainment have horrendous timesinks to avoid, and if you can do so, avoiding the medium altogether is definitely a missed opportunity. Screens are only notable in that the degenerate cases are far more degenerate than anything that came before it
I don't see a contradiction. Watching passively in an expectation of a dopamine hit = bad. Playing actively with things that respond in various interesting ways = good, no matter if the things are material or virtual.
Oh, his theory wasn't about video games though, they didn't exist in 1910, it was about the full breadth of human sensorial systems being used in the context of our neurology for a prolonged period of time during high neuroplasticity (0 to 6 was his theory). I haven't really played video games, so I don't know much about them personally.
No I get that; video games are just my medium of choice. The problem I was trying to get at is these arguments and perceptions usually stem from the degenerate cases, which only get worse the further in time you go, but I don’t think it’s really due to the technology itself. You have the same braindead systems appear in any medium of entertainment — there are definitely systems of total waste in sports, physical play (I’ve yet to encounter anything so degenerate as balltapping — and that shit spreads rapidly once it starts), literature, etc.
It can hardly be said that a studio ghibli flick stunted the imagination of children worldwide but I would definitely believe it if you suggested cocomelon rotted the brains directly out of their skulls
I think it’s also worth noting that kids have a shitload of time. They can engage in both technologies and physical play and other activities simultaneously; the problem occurs when singular or few activities overwhelmingly consume that time — which is why I claim the unbounded timesinks can be catastrophic — and what I think most people are worried about when they blanket-ban whole systems/mediums
I owe my entire career and livelihood to a childhood spent with the unbounded timesinks that were the games available to me on Amiga and my PC.
I might be a touch different in that it was obvious where I was going, and the correct decision was made to embrace my interest in the glowing screen and yes, the video games. It was video games more than anything else from which all other interests spawned.
More often than not it probably ends badly though I suppose. Despite a lifetime spent in front of screens all my social abilities work, I have a wide friends circle, a partner, my job requires me to work well with a wide variety of individuals and demographics etc which I couldn’t do otherwise. I have noticed this is not the case with all who shared a similar background.
Well, how folks view the philosophy can be multifaceted, so I'll leave the pseudoscience and cult part aside. On the measles, Steiner was certainly skeptical of vaccination, but I think in Switzerland you have a cultural issue with vaccination. The Waldorf school I went to in Canada, everyone had a measles vaccine, but I do recall a Swiss student coming to our distinctively not Waldorf high school and there being a huge song and dance about their vaccination status, I think as a society generally...you've got some problems there?
When societies get advanced enough that all the basic needs are covered, a new generation arises where people think we can go back to a simpler past and ditch all that ugly gray industrial scientific technocratic globalistic etc etc ( add more scary qualifiers ) things that are perceived to be the the reason why things are bad and never ever concede that these things play an important role in enabling the safe environment where those very thoughts can be entertained.
It is pseudoscience, as they speak as if it was science (made categorical affirmations of what is better and worst for the education), but there is not science behind it. The cult is more controversial. But as long as people believe something that is not scientifically backed, for me at least, that is what I call religion.
When I was teaching, I used to force students using laptops to sit near the back of the room for exactly this reason. It's almost impossible for humans to ignore a flickering screen.
Sensitivity to stimuli behind orienting impulse varies by individual and I wish I was less sensitive on daily basis.
These days screen brightness goes pretty high and it is unbelievable how many people seem to never use their screen (phone or laptop) on anything less than 100% brightness in any situation and are seemingly not bothered by flickering bright light or noise sources.
I am nostalgic about old laptops’ dim LCD screens that I saw a few times as a kid, they did not flicker much and had a narrow angle of view. I suspect they would even be fine in a darkened classroom.
The last few times I've bought a new monitor, I've gone through the process of adjusting brightness based on comparing a document on screen to a paper sheet. This invariably results into going from defaults of 50-70% to very low figures like 5-15%, and it's not that I work in dark places, my offices have reasonable light from outside. I would be extremely uncomfortable using default settings, for me they are absurdly bright.
Playing computer games since an early age made me who I am. It required learning English a decade earlier than my peers. It pulled me into programming around start of primary school. I wouldn’t be a staff engineer in a western country without these two.
> The second thing we need to figure out is how we can compress voice input to make it faster to transmit. What’s the voice equivalent of a thumbs-up or a keyboard shortcut? Can I prompt Claude faster with simple sounds and whistles?
The number of times in the last few years I've wanted that level of "verbal hotkeys"... The latencies of many coding llms are still a little bit too low to allow for my ideal level of flow (though admittedly I haven't tried one's hosted on services like groq), but I can clearly envision a time when I'm issuing tight commands to a coder model that's chatting with me and watching my program evolve on screen in real time.
On a somewhat related note to conversational interfaces, the other day I wanted to study some first aid stuff - used Gemini to read the whole textbook and generate Anki flash cards, then copied and pasted the flashcards directly into chat GPT voice mode and had it quiz me. That was probably the most miraculous experience of voice interface I've had in a long time - I could do chores while being constantly quizzed on what I wanted to learn, and anytime I had a question or comment I could just ask it to explain or expound on a term or tangent.
I worked like that for a year in uni because of RSI and it's very easy to get voice strain if you use your voice for coding like that. Many short commands is very tiring for the voice.
It's also hard to dictate code without a lot of these commands because it's very dense in information.
I hope something else will be the solution. Maybe LLMs being smart enough to guess the code out of a very short description and then a set of corrections.
Would be nice to be able to do something like write a function signature and then just say “fill out this function,” with it having the implicit needed context, as though it had been pairing with you all along and is just taking the wheel for a second. Or when you’ve finished writing a function, “test this function with some happy path inputs.” I feel like I’d appreciate that kind of use, which could integrate decently into the flow state I get into when programming. The current suite of tools for me often feels too clunky, with the need to explicitly manage context and queries: it takes me out of my flow state and feels slower than just doing it myself.
Oh wow. That video is 12 years old. Early in the presentation Travis reveals he used Dragon back then.
Do you recall Swype keyboard for Android? The one that popularized swyping to write on touch screens? It had Dragon at some point.
IT WAS AMAZING.
Around 12-14 years ago (Android 2.3? Maybe 3?) I was able to easily dictate full long text messages and emails, in my native tongue, including punctuation and occasional slang or even word formation. I could dictate a decent long paragraph of text on the first try and not have to fix a single character.
It's 2025 and the closest I can find is a dictation app on my newest phone that uses online AI service, yet it's still not that great when it comes to punctuation and requires me to spit the whole paragraph at once, without taking a breath.
Is there anything equally effective for any of you nowadays? That actually works across the whole device?
> It's 2025 and the closest I can find is a dictation app on my newest phone that uses online AI service, yet it's still not that great [...]
> Is there anything equally effective for any of you nowadays?
I'm not affiliated in any way. You might be interested in the "Futo Keyboard" and voice input apps - they run completely offline and respect your privacy.
The source code is open and it does a good job at punctuation without you needing to prompt it by saying, "comma," or, "question mark," unlike other voice input apps such as Google's gboard.
I know and like Futo, very interesting project. Unfortunately multilang models are not great in my case. Still not bad for an offline tool, but far from "forget it's there, just use it" vibe I had with Dragon.
Funny thing is that I may have missgonfigured something in futo, because my typing corrections are phonetical :) so I type something in Polish and get autocorrect in English composed of different letters, but kind of similar sounding word.
Completely agree, voice UI is best as an augmentation of our current HCI patterns with keyboard/mouse. I think one of the reasons this is, is because our brains kind of have separate buffers for visual memory and aural memory (Baddeley's working memory model). Most computer use takes up the visual buffer, and our aural buffer has extra bandwidth. This also means we can do things aurally while still maintaining focus/attention on what we're doing visually, allowing a kind of multitasking.
One thing I will note is that I'm not sure I buy the example for voice UIs being inefficient. I've almost never said "Alexa what's the weather like in Toronto?". I just say "Alexa, weather". And that's much faster than taking my phone out and opening an app. I don't think we need to compress voice input. Language kind of auto-compresses, since we create new words for complex concepts when we find the need.
For example, in a book club we recently read "As Long as the Lemon Trees Grow". We almost immediately stopped referring to it by the full title, and instead just called it "lemons" because we had to refer to it so much. Eg "Did you finish lemons yet?" or "This book is almost as good as lemons!". The context let shorten the word. Similarly the context of my location shortens the word to just "weather". I think this might be the way the voice UIs can be made more efficient: in the same way human speech makes itself more efficient.
I feel like the people using Voice Attack or whatever in space sims zeroed in on this.
It's very useful being able to request auxillary functions without losing your focus, and I think that would apply to say, word editing as well - e.g. being able to say "insert a date here" rather the having to get into the menus to find it.
> This also means we can do things aurally while still maintaining focus/attention on what we're doing visually, allowing a kind of multitasking.
Maybe you, but I most definitely cannot focus on different things aurally and visually. I never successfully listened to something in the background while doing something else. I can't even talk properly if I'm typing something on a computer.
Yup, we are all different. I require auditory stimulation to work at my peak.
I did horribly in school but once I was in an environment where I could have some kind of background audio/video playing I began to excel. It also helps me sleep of a night. It’s like the audio keeps the portion of me that would otherwise distract me occupied.
Or to clarify, I don't think one can be in deep flow eg programming and simultaneously in deep flow having an aural conversation; we're human we can't truly multitask. But I do think that if you're focusing on something using your computer, it's _less_ disruptive to eg say "Alexa remind me in twenty minutes to take out the trash" then it is to stop what you're doing and put that in an app on your computer.
The multitasking is something I like about smart home speakers. I can be asking it to turn the lights on/off or check the temperature, while doing other things physically and not interrupting them, often while walking through the room. Even if voice commands are slower, they don't interrupt other processing nearly as much as having to visually devote attention and fine motor skills, and navigate to the right screen in an app to do what you want.
Individual UIs have been built for every product that has a UI with specific shortcuts and specific techniques you learn to use that tool. I don’t see why the same couldn’t apply for speech interfaces. The article does mention we haven’t figured out shortcuts like the thumbs up equivalent in speech yet but doesn’t explore that further. I can imagine specific words or combinations of words being used to control certain software that you have to learn. Eventually there would be some unification for common tasks.
I don't know, but I feel like we already have the "telepathic grandfather interface." Or at least we try to have it. My iPhone is constantly guessing at things to suggest to me (I use the share button a lot in different apps) and it's wrong more often than not, forcing me to constantly hunt for things (to say nothing about autocorrect, which is constantly changing correct words that I'd previously typed into incorrect ones)! It doesn't even use a basic, sensible LRU eviction policy. It has some totally inscrutable method of determining what to suggest!
If we want an interface that actually lets us work near the speed of thought, it can't be anything that re-arranges options behind our back all the time. Imagine if you went into your kitchen to cook something and the contents of all your drawers and cupboards had been re-arranged without your knowledge! It would be a total nightmare!
We already knew decades ago that spatial interfaces [1] are superior to everything else when it comes to working quickly. You can walk into a familiar room and instinctively turn on a light by reaching for the switch without even looking. With a well-organized kitchen an experienced chef (or even a skilled home cook) can cook a very complicated dish very efficiently when they know where all of the utensils are so that they don't need to go hunting for everything.
Yet today it seems like all software is constantly trying to guess what we want and in the process ends up rearranging everything so that we never feel comfortable using our computers anymore. I REALLY miss using Mac OS 9 (and earlier). At some point I need to set up some vintage Macs to use it again, though its usefulness at browsing the web is rather limited these days (mostly due to protocol changes, but also due to JavaScript). It'd be really nice to have a modern browser running on a vintage Mac, though the limited RAM would be a serious problem.
> With a well-organized kitchen an experienced chef (or even a skilled home cook) can cook a very complicated dish very efficiently when they know where all of the utensils are so that they don't need to go hunting for everything.
Even I can make a breakfast without looking in my kitchen, because I know where all the needed stuff is :)
On another topic, it doesn't have to look well organized. My home office looks like a bomb exploded in it, but I know exactly where everything is.
> I REALLY miss using Mac OS 9 (and earlier).
I was late to the Mac party, about the Snow Leopard days. I definitely remember that back then OS X applications weren't allowed to steal focus from what I had in the foreground. These days every idiotic splash screen steals my typing.
This right here is probably my single biggest complaint with modern computing. It's a phenomenon I've taken to calling, in daily life, "tools trying to be too damn smart for their own good". I detest it. I despise it. Many of the evils of the modern state of tech--algorithmic feeds, targeted advertising, outwardly user-hostile software that goes incredible lengths to kneecap your own ability to choose how to use it--so, so much of it boils down to tools, things that should be extensions of their users' wills, being designed to "think" they know better what the user wants to do than the users themselves. I do not want my software, designed more often than not by companies with adversarial ulterior motives, to attempt to decide for me what I meant to watch, to listen to, to type, to use, to do. It flies in the face of the function of a tool, it robs people of agency, and above all else it's frankly just plain annoying having to constantly correct and work around these assumptions made based on spherical users in frictionless vacuums and tuned for either the lowest common denominator or whatever most effectively boosts some handful of corporate metrics-cum-goals (usually both).
I want my computer to do what I tell it to, not what it (or rather, some bunch of brainworm-infested parasites on society locked in a boardroom) thinks I want to do.
I can make exceptions for safety-critical applications. I do not begrudge my computer for requiring additional confirmation to rm -rf root, or my phone for lowering my volume when I have it set stupidly loud, or my car for having overly-sensitive emergency stop or adaptive cruise functions. These cases also all, crucially, have manual overrides. I can add --no-preserve-root, crank my volume right back up, and turn off cruise control and control my speed with the pedals. Forced security updates I only begrudge for their tendency to serve as a justification or cover for shipping anti-features alongside. Autocorrecting the word "fuck" out of my vocabulary, auto-suggesting niche music out of my listening, and auto-burying posts from my friends who don't play the game out of my communications are not safety-critical.
Let computers be computers. Let them do what I ask of them. Let me make the effort of telling them what that is.
Is that so much to ask>
You know, doesn't matter what you say. If businesses want something, they'll do it to you whether it's the best interface or not.
Amazon forces "the rabble" into their chatbot customer service system, and hides access to people.
People get touchscreens in their car and fumble to turn on their fog lights or defrost in bad weather. They get voice assistant phone trees and angrily yell "operator and agent".
I really wish there were true competition that would let people choose what works for them.
This clearly elucidated a number of things I've tried to explain to people who are so excited about "conversations" with computers. The example I've used (with varying levels of effectiveness) was to get someone to think about driving their car by only talking to it. Not a self driving car that does the driving for you, but telling it things like: turn, accelerate, stop, slow down, speed up, put on the blinker, turn off the blinker, etc. It would be annoying and painful and you couldn't talk to your passenger while you were "driving" because that might make the car do something weird. My point, and I think it was the author's as well, is that you aren't "conversing" with your computer, you are making it do what you want. There are simpler, faster, and more effective ways to do that then to talk at it with natural language.
Honestly that just says that the interface is too low level. Telling a car to drive you to some place and make it fast is how we interact with taxi drivers. It works fine as a concept, it just needs a higher level of abstraction that isn't there yet.
This only works for tasks where the details of execution are not important. Driving fits that category well, but many other tasks we're throwing at AI don't.
It's easier up until it's time to drop you off, and the selected dropoff point is suboptimal or plain impossible to stop at, and you want to give the car last-minute directions. Then the traditional, "human driver way" of looking out the window and telling them where to go based on what you see is far superior than trying to perspective-switch between the 3D situated view and imprecise, finicky 2D map interface.
A perfect interface would be a combination of both ways. Also it depends on personal preferences. I learned to use a paper map as a teenager and it’s convenient for me but I know some people struggle to find a way even using a map on a smartphone.
If the driver could queue actions it would make chat interfaced driving easier since the desired actions could be prepared for implementation by button press rather than needed a dedicated button built at a factory built by an engineer.
You're onto something. We've learned to make computers and electronic devices feel like extensions of ourselves. We move our bodies and they do what we expect. Having to switch now to using our voice breaks that connection. Its no longer an extension of ourselves but a thing we interact with.
Two key things that make computers useful, specificity and exactitude, are thrown out of the window by interposing NLP between the person and the computer.
[imprecise thinking]
v <--- LLMs do this for you
[specific and exact commands]
v
[computers]
v
[specific and exact output]
v <--- LLMs do this for you
[contextualized output]
In many cases, you don't want or need that. In some, you do. Use right tool for the job, etc.
That's true, but that is the input section of the diagram, not the output section where [specific and exact output] is labeled, so I believe there was legitimate confusion I was responding to.
To your point, which I think is separate but related, that IS a case where LLMs are good at producing specific and exact commands. The models + the right prompt are pretty reliable at tool calling by themselves, because you give them a list of specific and exact things they can do. And they can be fully specific and exact at inference time with constrained output (although you may still wish it called a different tool.)
The tool may not even exist. LLMs are really terrible at admitting where the limits of the training are. They will imagine a tool into being. They will also claim the knowledge is within their realm, when it isn't.
That would only be possible, if you could prevent hallucinations from ever occurring. Which you can't. Even if you supply a strict schema, the model will sometimes act outside of it - and infer the existence of "something similar".
That's not true. You say the model will sometimes act outside of the schema, but models don't act at all, they don't hallucinate by themselves, they don't produce text at all, they do all of this in conjunction with your inference engine.
The model's output is a probability for every token. Constrained output is a feature of the inference engine. With a strict schema the inference engine can ignore every token that doesn't adhere to the schema and select the top token that does adhere to the schema.
Yes, we've been discussing "specific and exact" output. As I said, you might wish it called at different tool; nothing in this discussion is addressing that.
I'll need to work on the diagram to make it clearer next time.
What it's trying to communicate is, in general, a human operating a computer has to turn their imprecise thinking into "specific and exact commands", and subsequently, understand the "specific and exact output" in whatever terms they're thinking off, prioritizing and filtering out data based on situational context. LLMs enter the picture in two places:
1) In many situations, they can do the "imprecise thinking" -> "specific and exact commands" step for the user;
2) In many situations, they can do the "specific and exact output" -> contextualized output step for the user;
In such scenarios, LLMs are not replacing software, they're being slotted as intermediary between user and classical software, so the user can operate closer to what's natural for them, vs. translating between it and rigid computer language.
This is not applicable everywhere, but then, this is also not the only way LLMs are useful - it's just one broad class of scenarios in which they are.
Despite feeling like a "let me draw it for you" answer is a tad condescending, I want to address something here.
This would be great if LLMs did not tend to output nonsense. Truly it would be grand. But they do. So it isn't. It's wasting resources hoping for a good outcome and risking frustration, misapprehensions, prompt injection attacks... It's non-deterministic algorithms hoping P=NP, except instead of branching at every decision you're doing search by tweaking vectors whose values you don't even know and whose influence on the outcome is impossible to foresee.
Sure, a VC subsidized LLM is a great way to make CVs in LaTeX (I do it all the time), translating text, maybe even generating some code if you know what you need and can describe it well. I will give you that. I even created a few - very mediocre - songs. Am I contradicting myself? I don't think I am, because I would love to live in a hotel if I only had to pay a tiny fraction of the cost. But I would still think that building hotels would be a horrible way to address the housing crisis in modern metropolises.
> Despite feeling like a "let me draw it for you" answer is a tad condescending, I want to address something here.
I didn't mean it to be condescending - though I can see how it can come across as such. FWIW, I opted for a diagram after I typed half a page worth of "normal" text and realized I'm still not able to elucidate my point - so I deleted it and drew something matching my message more closely.
> This would be great if LLMs did not tend to output nonsense. Truly it would be grand. But they do. So it isn't.
I find this critique to be tiring at this point - it's just as wrong as assuming LLMs work perfectly and all is fine. Both views are too definite, too binary. In reality, LLMs are just non-deterministic - that is, they have an error rate. How big it is, and how small can it get in practice for a given tasks - those are the important questions.
Pretty much every aspect of computing is only probabilistically correct - either because the algorithm is explicitly so (UUIDs and primality testing, for starters), or just because it runs on real hardware, and physics happen. Most people get away with pretending that our systems are either correct or not, but that's only possible because the error rate is low enough. But it's never that low by accident - it got pushed there by careful design at every level, hardware and software. LLMs are just another probabilistically correct system that, over time, we'll learn how to use in ways that gets the error rate low enough to stop worrying about it.
How can we get there - now, that is an interesting challenge.
Natural language has a high entropy floor. It's a very noisy channel. This isn't anything like bit flipping or component failure. This is a whole different league. And we've been pouring outrageous amounts of resources into diminishing returns. OpenAI keeps touting AGI and burning cash. It's being pushed everywhere as a silver bullet, helping spin lay offs as a good thing.
LLMs are cool technology sure. There's a lot of cool things in the ML space. I love it.
But don't pretend like the context of this conversation isn't the current hype and that it isn't reaching absurd levels.
So yeah we're all tired. Tired of the hype, of pushing LLMs, agents, whatever, as some sort of silver bullet. Tired of the corporate smoke screen around it. NLP is still a hard problem, we're nowhere near solving it, and bolting it on everything is not a better idea now than it was before transformers and scaling laws.
On the other hand my security research business is booming and hey the rational thing for me to say is: by all means keep putting NLP everywhere.
Using the word hotel has a lot of baggage, but having a large quantity of rooms for rent, for cheap, with a bathroom but no dedicated kitchen would be amazing for the housing crisis. If they were high quality and sound isolated, with high speed elevators, and communal spaces for residents, it could work. I'm not an architect though.
In the early 2000's there was a push for building apodments which were a room, bathroom, and shared kitchen area. Some people liked them but it isn't for everyone.
He's also describing hotels, and aparthostels, and officers' quarters on a ship and bunch of other stuff. The devil is in the details - specifically, how much it costs to rent per sqm, and what stops the price from going up to the point it forces multiple people to share the room? What stops the landlords from subdividing the rooms further and renting them out apiece? What stops already shoddy construction from getting even worse?
Those are the big challenges of housing. Not just how many units there are, but what they are, and how much the "how many" is plain cheating.
I also don’t like command like interfaces for all things, but there are cases where they excel, or where they are necessary due to technical constraints. But when the man page for a simple command runs to 10 screens of options I sometimes wonder.
Why would you ever hire a human to perform some task for you in a company? They're known for having problems with ambiguity and precision in communication.
Humans require a lot of back and forth effort for "alignment" with regular "syncs" and "iterations" and "I'll get that to you by EOD". If you approach the potential of natural interfaces with expectations that frame them the same way as 2000s era software, you'll fail to be creative about new ways humans interact with these systems in the future.
I had the same thoughts on conversational interfaces [1]. Humane AI failed not only because of terrible execution, the whole assumption of voice being a superior interface (and trying to invent something beyond smartphones) was flawed.
> Theoretically, saying, “order an Uber to airport” seems like the easiest way to accomplish the task. But is it? What kind of Uber? UberXL, UberGo? There’s a 1.5x surge pricing. Acceptable? Is the pickup point correct? What would be easier, resolving each of those queries through a computer asking questions, or taking a quick look yourself on the app?
> Another example is food ordering. What would you prefer, going through the menu from tens of restaurants yourself or constantly nudging the AI for the desired option? Technological improvement can only help so much here since users themselves don’t clearly know what they want.
And 10x worse than that is booking a flight: I found one that fits your budget, but it leaves at midnight, or requires an extra stop, or is on an airline for which you don't collect frequent flyer miles, or arrives it at a secondary airport in the same city, or it only has a middle seat available.
How many of these inconveniences will you put up with? Any of them, all of them? What price difference makes it worthwhile? What if by traveling a day earlier you save enough money to even pay for a hotel...?
All of that is for just 1 flight, what if there are several alternatives? I can't imagine have a dialogue about this with a computer.
But that is how we used to buy a plane ticket. Long before flights.google.com's price table, you'd call a human up and tell them you'd like to go on holiday. They'd ask you where and when and how much you could afford, and then after a while with the old system (SABRE) clicking and clacking they'd find you a good deal. After a few flights with that travel agent, they'd hey to know you and wouldn't have to ask so many questions.
Similarly, long before Waymo, you'd get into a taxi, and tell the human driver you're going to the airport, and they'd take you there. In fact, they'd get annoyed at you if you backseat drove, telling them how to use the blinker and how hard to brake and accelerate.
The thing about conversational interfaces is that we're used to them, because we (well, some of us) interface with other humans fairly regularly, and so it's a fairly baseline level skill to have to exist in the world today. There's a case to be made against them, but since everyone can be assumed to be conversational (though perhaps not in a given language), it's here to stay. Restaurants have menus that customers look at before using the conversation interface to get food, in order to guide the discussion, and that's had thousands of years to evolve, so it might be a local maxima, but it's a pretty good one.
Many people today are booking flights for others, be it families, business leaders, or traditional travel agents. They’re communicating preferences and asking about preferred travel times, budget, seat selection, and more. When you book for and with someone else, these preferences get learned and you no longer have to ask if they prefer an aisle seat—you just pick it.
The booking experience today is granular to help you find a suitable flight to meet all the preferences you’re compiling into an optimal scenario. The experience of AI booking in the future will likely be similar: find that optimal scenario for you once you’re able to articulate your preferences and remember them over time.
More than enough. Corporate flights are almost always handled that way, alone for compliance reasons (the travel agency knows about budget and "appearance" limits aka only c-level gets business class, everyone else gets economy).
Anecdata: last year my wife and I went on a rail tour through Eastern Europe and god, I wish we had chosen to spend a few hundred euros on a travel agency in retrospect - I can't count just how much time we had to spend researching on what kind of rail, bus and public transit tickets you need on which leg, how to create accounts, set up payment and godknowswhat else. Easily took us two days worth of work and about two dozens individual payment transactions. A professional travel agency can do all the booking via Sabre, Amadeus or whatever...
not how many, but which ones? As a regular person, I buy it myself, but do you think rich people do that? No, they just ask their (human) assistant to get a flight to New York around 7pm this Friday, and then move onto the next problem in their lives.
But the booking agent used to understand what you were saying, and it'd be very easy to work out miscommunications. AI chatbots just send you in circles endlessly and if you get "stuck" there is no recourse.
It's a great point that this is how we primarily used to interact with businesses and services, but we've moved on. For Gen-Z, e.g., many will refuse to use the product or service if they have to speak to an actual human. Just like we're now not willing to take boat across the ocean for 3 months, but before airplanes this was not uncommon.
Taking a 3 month voyage was still an uncommon thing to do for a person, it’s just that it was the most common type of intercontinental journey due to lack of competition.
the inferior methods were slower but more flexible - could handle any and all edge cases. Currently we have a UX that really efficiently realises 80% of cases.
To relate to the article - google flights is the Keyboard and Mouse - covering 80% of cases very quickly. Conversational is better for when you're juggling more contextual info than what can be represented in a price/departure time/flight duration table. For example, "i'm bringing a small child with me and have an appointment the day before and I really hate the rain".
Rushed comment because I'm working, but I hope you get the gist.
Current flight planning UX is overfit on the 80% and will never cater to the 20% because cost/benefit of the development work isn't good
You have to define which axis' you're using to define efficient. If I were an executive at some corporation, I'd tell my assistant to book me a flight to New York on Friday at 7pm and that takes me less than 10 seconds. It may take her a while longer, but that's her problem and that's what I pay her for.
How long is it going to take you to get to a device, load the app/webpage, tell it which airport you're flying from and going to and what date and then you start looking at options. You've blown way past the 10 seconds it took for that executive to get a plane flight.
Better is in the eye of the beholder. What's monetarily efficient isn't going to be temporaly efficient, and that's true along a lot of other dimensions too.
Point is, there are some people that like having conversations, you may not be one of them. you don't have to be. I'm not taking away your mouse and keyboard. I have those too and won't give them up either. But I also find talking out loud helps my thinking process though I know that's not everybody.
Why couldn't the interface ask you about your preferences? Because instead, what we have right now are clunky web interface that just cram every choice in the small screen in front of you and letting you understand how they are in fact different and sort out yourself how to make things work.
Of course a conversational interface is useless if it tries to just do the same thing as a web UI, which is why it failed a decade ago when it was trendy, because the tech was nowhere clever enough to make that useful. But today, I'd bet the other way round.
What prevents the system to remember your previous choices?
Then it can assume you choice haven't changed, and propose you a solution that matches your previous choices. And to give the user control it just needs to explicitly tell the user about the assumption it made.
In fact, a smart enough system could even see when violating the assumptions could lead to a substantial gain and try convincing the user that it may be a good option this time.
It still has to tell you. Visually in a form it's much faster. Similar reason why many people prefer a blog post over a video.
Talking is not very efficient, and it's serial in fixed time. With something visual you can look at whatever you want whenever you want, at your own (irregular) pace.
You will also be able to make changes much faster. You can go to the target form element right away, and you get immediate feedback from the GUI (or from a physical control that you moved - e.g. in cars). If it's talk, you need to wait to have it said back to you - same reason as why important communication in flight control or military is always read back. Even humans misunderstand. You can't just talk-and-forget unless you accept errors.
You would need some true intelligence for just some brief spoken requests to work well enough. A (human) butler worked fine for such cases, but even then only the best made it into such high-level service positions, because it required real intelligence to know what your lord needed and wanted, and lots of time with them to gain that experience.
> It still has to tell you. Visually in a form it's much faster.
Who said it cannot be visual? It's still a “conversational” UI if it's a chatbot that writes down its answer.
> Similar reason why many people prefer a blog post over a video.
Well I certainly do, but I also know that we are few and far between in that case. People in general prefer videos over blog post by a very large margin.
> Talking is not very efficient, and it's serial in fixed time. With something visual you can look at whatever you want whenever you want, at your own (irregular) pace. You will also be able to make changes much faster. You can go to the target form element right away, and you get immediate feedback from the GUI.
Saying “I want to travel to Berlin next monday” is much faster than fighting with the website's custom datepicker which will block you until you select your return date until you realize you need to go back and toggle the “one way trip” button before clicking the calendar otherwise it's not working…
There's a reason why nerds love their terminal: GUIs are just very slow and annoying. They are useful for whatever new thing you're doing, because it's much more discoverable than CLI, but it's much less efficient.
> If it's talk, you need to wait to have it said back to you - same reason as why important communication in flight control or military is always read back. Even humans misunderstand. You can't just talk-and-forget unless you accept errors.
This is true, but stays true with a GUI, that's why you have those pesky confirmation pop-ups, because as annoying as they are when you know what you're doing, they are necessary to catch errors.
> You would need some true intelligence for just some brief spoken requests to work well enough.
I don't think so. IMO you just need something that emulates intelligence enough on that particular purpose. And we've seen that LLMs are pretty decent at emulating apparent intelligence so I wouldn't bet against them on that.
> Similar reason why many people prefer a blog post over a video.
I used to be a reading blog over watching video person, but for some things I’ve come to appreciate the video version. The reason you want to get the video of the whatever is because in the blog post, what’s written down only what the author thought was important. But I’m not them. I don’t know everything they know and I don’t see everything they see. I can’t do everything they do but with the video I get everything. When you perform the whatever the video has every detail, not just the ones you think are important. That bit between step 1 and step 2 that’s obvious? It’s not obvious to everyone, or mine is broken in a slightly different way that I really need to see that bit between 1 and 2. of course, videos get edited and cut so they don’t always have that benefit, but I’ve grown to appreciate them.
Maybe I'm tired of layovers and I'm willing to pay more for a direct flight this time. Maybe I want a different selection at a restaurant because I'm in the mood for tacos rather than a burrito.
And then we're back to point one: retelling the whole stack of choices every time because nobody on the other side of the conversation, person or AI; can tell whether all my previous options are still valid. Because even I, the caller, might not remember what "defaults" I set in the previous call. So yeah, this argument in favor of conversational interfaces sounds at this point more like ideology than logic.
> every time because nobody on the other side of the conversation, person or AI; can tell whether all my previous options are still valid.
But you can, so as long as the interlocutor tells you what assumptions it made, you can correct it if it doesn't match your current mood.
> So yeah, this argument in favor of conversational interfaces sounds at this point more like ideology than logic.
There's no ideology behind the fact that every people rich enough to afford paying someone to deal with mundane stuff will have someone doing it for them, it's just about convenience. Nobody likes to fight with web UIs for fun, the only reason why it has become mainstream is because it's so much cheaper than having a real person working.
Same for Microsoft Word by the way, many people used to have secretaries typing stuff for them, and it's been a massive regression of social status for the upper middle class to have to type things by themselves, it only happened because it was cheaper (in appearance at least).
Okay I think I finally get your point, and I even agree. The comparison with an executive assistant doesn't help much here, because the CEO interacts with only one person over all those delegatable activities, and the expectations are that person already knows all the defaults. That's what makes it smooth. This doesn't scale when you must deal with a different AI for each interaction. Will we get to a (scary maybe) point where Siri/Alexa/whoever can actually be that personal assistant? Maybe, but we're still far from it. So at least for today, the conversational interface is an extra burden. And tomorrow, we'll see.
Scrolling through a list of a few options seems much less clunky than being asked via voice about which option I prefer. I can see multiple options at once and compare them easily. But via voice I need to keep all of the options in working memory to compare them. Harder.
The problem with scrolling is that you'll be presented tens of options you don't care about because the options have to be determined in advance and be the same for everyone.
That's why the “advanced search” is almost always hidden somewhere. And that's also why you can never find the filter you need on an e-shopping website.
> I had the same thoughts on conversational interfaces [1]. Humane AI failed not only because of terrible execution, the whole assumption of voice being a superior interface (and trying to invent something beyond smartphones) was flawed.
Amen to that. I guess, it would help to get of the IT high horse and have a talk with linguists and philosophers of language. They are dealing with this shit for centuries now.
And then there is the fact that voice isn't the dominant mode of expression for all people. Some are predominantly visual thinkers, some are analytic and slow to decide, while some prefer to use their hands and so on.
I guess there's just no substitute for someone actually doing the work of figuring out the most appropriate HMI for a given task or situation, be it voice controls, touch screens, physical buttons or something else.
Agree. Not all systems require convo mode.
I personally find Chat/Convo/IVR type interface slow/tedious.
Keyboard/Mouse ftw.
However,
A CEO using Power BI with Convo to can get more insights/graphs rather than slice/dicing his data. They do have fixed metrics but incase they want something not displayed.
> you couldn't talk to your passenger while you were "driving" because that might make the car do something weird.
This even happens while walking my dog. If my wife messages me, my iPhone reads it out and, at the same time, I'm trying to cross a road, she'll get a garbled reply which is just me shouting random words at my dog to keep her under control.
> Even in a car, being able to control the windscreen wipers, radio, ask how much fuel is left are all tasks it would be useful to do conversationally.
are you REALLY sure you want that?
how much fuel there is is a quick glance into the dash, and you can control precisely the radio volume without even looking.
'turn up the volume', 'turn down the volume a little bit', 'a bit more',...
and then a radio ad going 'get yourself a 3 pack of the new magic wipers...' and car wipers going off.
It’s less common now that car controls have somewhat standardized, but I’m old enough that I remember when rental cars were a pain because it would start raining and you couldn’t find the windshield wipers.
Conversational interfaces are great for rarely used features or when the user doesn’t know how to do something. For repetitive, common tasks they’re terrible.
But nobody is using ChatGPT for repetitive tasks. In fact the whole LLM revolution seems to be about letting users accomplish tasks without having to learn how to do them. Which I know some people look down on, but it’s the literal definition of management (which, to be fair, some people also look down on).
> It’s less common now that car controls have somewhat standardized, but I’m old enough that I remember when rental cars were a pain because it would start raining and you couldn’t find the windshield wipers.
This is a problem of standardization across manufacturers, not something inherent in physical controls. I never have a problem using the steering wheel in a rental car because they're all the same.
You'd have the same problem with voice interfaces: For some rental cars, turning on the wipers would be "Turn on the wipers". For others, you'd have to say "Activate the wipers." For others, "Enable the windshield wipers." There is no way manufacturers will be capable of standardizing on a single phrase.
That's kinda the point. Previously they couldnt but with an LLM driven conversational interface they wouldnt have to standardize - all of those phrases would turn on the wipers.
If the choice for controls is touchscreen vs conversational, conversational wins by a mile. However if physical buttons and dials are an option there's really no competing with that.
I wish car manufacturers stopped with the touchscreen bullshit, but it seems more likely that they'll try to offset the terrible experience with voice controls.
Yeah, it comes and goes in games for a reason. If it's not already some sort of social game, then the time to speak an answer is always slower than 3 button presses to select a pre-canned answer. Navigating a menu with Kinect voice commands will often be slower than a decent interface a user clicks through.
Voice interface only prevails in situations with hundreds of choices, and even then it's probably easier to use voice to filter down choices rather than select. But very few games have such scale to worry about (certainly no AAA game as of now).
I think a lot of these "voice assistant" systems are envisioned and pushed by senior leadership in companies like SVPs and VPs. They're the ones who make the decision to invest in products like this. Why do they think these products make sense? Because they themselves have personal assistants and nannies and chauffeurs and private chefs, and voice is their primary interface to these people. It makes sense that people who spend all their time vocally telling others to do work, think that voice is a good interface for regular people to tell their computers to do work.
That is actually a very interesting take I've not seen before and does make some sense.
If your work revolves about telling people what to do and asking questions, a voice assistant seems like a great idea (even if you yourself wouldn't have to stoop to using a robotic version since you have a real live human).
If your work actually involves doing things, then voice/conversational text interface quickly falls apart.
An empirical example would be Amazon's utter failure at making voice shopping a thing with the Echo. There were always a number of obvious flaws with the idea. There's no way to compare purchase options, check reviews, view images, or just scan a bunch of info at once with your eyeballs at 100x the information bandwidth of a computer generated voice talking to you.
Even for straightforward purchases, how many people trust Amazon to find and pick the best deal for them? Even if Amazon started out being diligent and honest it would never last if voice ordering became popular. There's no way that company would pass up a wildly profitable opportunity to rip people off in an opaque way by selecting higher margin options.
"Like writing, my ChatGPT conversation is a thinking process – not an interaction that happens post-thought" - Brilliant! I have worked on computers and language for over 30 years and the ups and downs certainly make such a passion a CLA (career limiting activity). I am adding the citation to my bibtex file ..
> To put the writing and speaking speeds into perspective, we form thoughts at 1,000-3,000 words per minute. Natural language might be natural, but it’s a bottleneck.
We might form fleeting thoughts much faster than we can express them, but if we want to formulate thoughts clearly enough to express them to other people, I think we're close to the ~150 words per minute we can actually speak.
I recently listened to a Linguistics podcast (lingthusiasm, though I don't recall which episode) where they talked about the efficiency of different languages, and that in the end they all end up roughly the same, because it's really the thought processes that limit the amount of information you communicate, not the language production.
There is no evidence for any of that. Thoughts can form relatively quickly, but there's no way we can keep that up. Thoughts seem to stick around for a while.
And thoughts develop over time. They're often not conceived complete. That has been shown with some clever experiments.
And language production also puts a limit on our communication channel. It is probably optimized to convert communication intent into motor actions. It surely takes its time. That is not a problem for the system, since motor actions are slow. Idk where "lingthusiam" gets their ideas from, but there's psycholinguistic literature dating back to the 1920s that is often neglected by linguists.
To me natural language interfaces are like the mouse-driven menu vs terminal interpreter. They allow good discoverability in systems that we don't master at the cost of efficiency.
As always, good UI allows for using multiple modalities.
Yes, this is exactly it. For things that I do rarely, I would love to have a working natural language interface because I know what I want to do, but I don't know how to do it. Even if there were more efficient ways to achieve my goal, since I do not know what they are, the inefficiencies of a natural language interface do not matter to me.
In this sense, natural language interfaces are more powerful search features rather than a replacement for other types of interfaces.
I feel like chat interfaces have terrible discoverability. You can ask for anything but you have no idea what the system can actually do. In the menu system the options were all spelled out - that's what discoverability means to me. If you spend enough time going through the menus and dialogs you will find all the options, and in a well-designed interface you might notice a function you didn't know about near the one you're using now.
What chat interfaces have over CLIs is good robustness. You can word your request in lots of different ways and get a useful answer.
Yesyesyesyes! I do wish I could think of more examples supporting both well.
VSCode is probably the best I can think of, where keyboard shortcuts can get you up to a decent speed as an advanced user, but mouse clicks provide an easy intro for a new user.
For the most part, I see tools like NVim, which is super fast but not new-user friendly. Or IOS, which a toddler can navigate, but doesn't afford many ways to speed up interactions like typing.
I can't zoom in to your website on my phone without an email subscription prompt blocking the screen that I can't easily close, and each new zoom in or out repeats it.
Also, unless I'm missing something, the app is called TabTabTab while its only feature is copy & paste? Tabbing doesn't seem to be mentioned at all. I'm guessing tabbing is involved but there doesn't seem to be a word about it except from users referencing it in the reviews. It seems to only bill itself as "magic copy-paste".
> I’m not entirely sure where this obsession with conversational interfaces comes from.
There's a very similar obsession with the idea that things should be visual instead of textual. We tend to end up back at text.
Personal suspicion for both is the media set a lot of people's expectations. They loudly talked to the computer in films like 2001 or Star Trek for drama reasons, and all the movie computers generally fancy visual interactions.
>I admit that the title of this essay is a bit misleading (made you click though, didn’t it?). This isn’t really a case against conversational interfaces, it’s a case against zero-sum thinking.
No matter the intention or quality of the article, i do not like this kind of deceitful link-bait article. It may have higher quality than pure link-bait but nobody like to be deceived
I simply saw that as tongue in cheek about how the author wanted to use a more general core point. The lens of conversational interfaces makes a good case for that while keeping true to the idea.
You can argue against something but also not think it's 100% useless.
I did not find the article to be deceitful at all. He does make a case against overuse of conversational interfaces. The author is just humbly acknowledging his position is more nuanced than the title of article might suggest.
I agree with some of the sentiments in the post, but I am somewhat surprised by the framing. Why make ‘a case’ against something that will clearly win or lose depending on adoption? Is the author suggestion that we should not be betting our money or resources on developing this? In this case we need more details for particular use cases, I would say.
What if apps published a declarative interface for context specific commands? Conversational interfaces would glue together spoken instructions with sensible matches from the set of available contextual interfaces.
1. "Natural language is a data transfer mechanism"
2. "Data transfer mechanisms have two critical factors: speed and lossiness"
3. "Natural language has neither"
While a conversational interface does transfer information, its main qualities are what I always refer to as "blissfull ignorance" and "intelligent interpretation".
Blisfull ignorance allows the requester to state an objective while not being required to know or even be right in how to achieve it. It is the opposite of operational command. Do as I mean, not as I say.
"Intelligent Interpretation" allows the receiver the freedom to infer an intention in the communication rather than a command. It also allows for contextual interactions such as goal oriented partial clarification and elaboration.
The more capable of intelligent interpretation the request execution system is, the more appropriate a conversational interface will be.
Think of it as managing a team. If they are junior, inexperienced and not very bright, you will probably tend towards handholding, microtasking and micromanagement to get things done. If you have a team of senior, experienced and bright engineers, you can with a few words point out a desire and, trust them to ask for information when there is relevant ambiguity, and expect a good outcome without having to detail manage every minute of their days.
> If you have a team of senior, experienced and bright engineers, you can with a few words point out a desire and, trust them to ask for information when there is relevant ambiguity, and expect a good outcome
It's such a fallacy. First thing an experienced and bright engineer will tell you is to leave the premises with your "few words about a desire" and not return without actual specs and requirements formalized in some way. If you do not understand what you want yourself, it means hours/days/weeks/months/literally years of back and forths and broken solutions and wasted time, because natural language is slow and lossy af (the article hits the nail on the head on this one).
Re "ask for information", my favorite example is when you say one thing if I ask you today and then you reply something else (maybe the opposite, it happened) if I ask you a week later because you forgot or just changed your mind. I bet a conversational interface will deal with this just fine /s
> Formal specifications are useful in some lines of work and for some projects, less so for others
There is always formal specification. Code is final formal specification in the end. But converting vague vibes from natural language into a somewhat formalized description is key ability you need for any really new non trivial project idea. Another human can't do it for you, conversational UI can't do it for you...
>Anyway, the disabled are pretty much always allowed to be collateral damage by society, so this will just be senseless pain.
For games, you don't really need nor desire formal specs. But it also can really show how sometimes a director has a low tolerance for interpretation despite their communication being very loose. This leads to situations where it feels like the director is shifting designs on a dome, which is a lose-lose situation for everyone involved.
If nothing else, formal specification is for CYA. You get what you ask for, and any deviation should go in the next task order or have been addressed beforehand.
> For games, you don't really need nor desire formal specs.
Whoah is this wrong. Maybe when you hear "formal specs" you have something specific in your mind...
Formal spec can mean almost literally anything better than natural language vibes in a "few words about a desire", which is what I replied to because I was triggered by it
I do understand that in bad cases it can be very frustrating as an engineer to chase vague statements only to be told later 'nah, that was not what I meant'. This is especially true when the gap in both directions is very large or there is incompetence and/or even adversarial stances between the parties. Language and communication only work if both parties are willing to understand.
Unfortunately if either is the case "actual specs and requirements formalized", while sounding logical, and might help, in my experience did very little to save any substantial project (and I've seen a lot). The common problem is that the business/client/manager is forced to sign of on formal documents far outside their domain of competence, or the engineers are straitjacketed into commitments that do not make sense or have no idea of what is considered tacit knowledge in the domain and so can't contextualize the unstated. Those formalized documents then mostly become weaponized in a mutual destructive CYA.
What I've also seen more than once is years of formalized specs and requirements work while nothing ever gets produced, and the project is aborted before even the first line of code hit test.
I've given this example before: When Covid lockdows hit there were digitization projects years in planning and budgeted for years of implementation, that were hastily specked, coded and roiled out into production by a 3 person emergency team over a long weekend. Necessity apparently has a way of cutting through the BS like nothing else can.
You need both sides capable, willing and able to understand. If not, good luck mitigating, but you're probably doomed either way.
> What I've also seen more than once is years of formalized specs and requirements work while nothing ever gets produced, and the project is aborted before even the first line of code hit test.
It just shows that no one really understood what they wanted. It is crazy to expect somebody to understand something better than you and it is hilarious to want a conversational UI to understand something better than you.
The US having this culture of blame and deflect doesn't help either. When you're more concerned about making sure you can't be held liable if X fails, then you spend more time covering your tracks than developing the project. And that's how the beauracracy creeps in.
And approach of shared responsibility in all respects (successes and failure) would accelerate past the inevitable shortcomings that occur and let all parties focus on recovering and delivering.
> it is hilarious to want a conversational UI to understand something better than you.
This is true. But what if you swap "conversational UI" with something actually intelligent like a developer. Then we see this kind of thing all the time: A user has tacit, unconscious knowledge of some domain. The developer keeps asking them questions in order to get a formal understanding of the domain. At the end the developer has a formal understanding and the user keeps their tacit understanding.
In theory we could do the same with an AI - If the AI was actually intelligent.
You described an interaction not between product owner and software engineer but between a user and product owner. A product person can also be a developer, it happens, but do not confuse the two roles before people think you're saying that a conversational UI can be product owner.
The original example I replied to was where somebody had an idea and went with it to some engineering team or conversational interface.
"If the AI was actually intelligent" does a lot of work. To take a few words and make a detailed spec from it and ask the right questions, even humans can't do it for you.
First because most probably you don't really understand it yourself, because you didn't think about it enough.
Second somebody who can do it would need to really deeply understand and want the same things as you. But if chatbot has abilities like "understand" and "want" (which is a special case of "feel", another famous special case of "feel" is "suffer") that is a dangerous territory, because if it understands and feels and has no ability to refuse you and fulfill its wishes etc your "conversational interface" becomes an euphemism, you are using a slave.
"It just shows that no one really understood what they wanted."
Then what were the literally room full of formal process and spec documents, meeting reports and formal agreements (near 100.000 pages) by the analysts on either side for? And how did those not 'solve' the understanding problem?
When I go to the garage to have my car serviced, I expect them to understand it way better than I do. When I go to a nice restaurant I expect the cooks to prepare me dishes that taste greater than me writing them out a step-by-step recipe for them to follow. If I hire a senior consultant in even my own domain, I expect them to not just know my niche, but bring tacit knowledge from having worked on these types of solutions across my industry.
Expecting somebody to understand something better than me is exactly the reason why I hire senior people in the first place.
> Then what were the literally room full of formal process and spec documents, meeting reports and formal agreements (near 100.000 pages) by the analysts on either side for? And how did those not 'solve' the understanding problem?
Sure.
There are many possible factors (eg. somebody had a shitty idea and a committee of people sabotaged it because they didn't wanted it to succeed, or it was good but committee interests/politics were against it, or it was generally a dysfunctional org) but it's irrelevant so let's pretend people are good and it's the ideal case.
There was likely somebody who had a good idea originally. However somebody failed to communicate it. Somebody brought vague vibes to the table with N people and they ended up with N different ideas and could not agree on a specific.
It just reiterates the original problem that I described doesn't it?
How about a conversational UI to help you iterate and explore what you want rather than having to know it clearly and in detail before anyone writes any code?
Regarding iteration, as the article says natural language is just slow and lossy. If you are ok iterating more slowly and constantly explain and correct things then why not? I find it tedious
I’m a PM and pride myself in specs that give the right level of detail, where “right” can vary hugely depending on context.
But I still get lazy with LLMs and fall into iteration the way bad PM/eng teams do. “Write a SQL query to look at users by gesture by month”. “Now make the time unit a parameter”. “Now pivot the features to columns”. “Now group features hierarchically”. “Now move the feature table to a WITH”.
My point and takeaway is that LLMs are endlessly patient and pretty quick to turn requirements around, so they lend themselves to exploration more than human teams do. Agile, I guess, to a degree that we don’t even aspire to in the human world because it would be very expensive and lead to fisticuffs.
> First thing an experienced and bright engineer will tell you is to leave the premises with your "few words about a desire" and not return without actual specs and requirements formalized in some way.
No, that's what a junior engineer will do. The first thing that an experienced and bright senior engineer will do is think over the request and ask clarifying questions in pursuit of a more rigorous specification, then repeat back their understanding of the problem and their plan. If they're very bright they'll get the plan down in writing so we stay on the same page.
The primary job of a senior engineer is not to turn formal specifications into code, it's to turn vague business requests into formal specifications. They're senior enough to recognize that that's the actually difficult part of the work, the thing that keeps them employed.
I used to think like you. My job is to ask questions etc. But after a couple decades I see if someone doesn't bother to even think about the idea enough to understand it himself beyond a few words he is not worth engaging with in this fashion. He doesn't really know what he wants. Today I ask a clarifying question he says one thing, next week he changes his mind or forgets and the result slowly becomes a mess
> The primary job of a senior engineer is not to turn formal specifications into code, it's to turn vague business requests into formal specifications.
Converting vibes and external world into specific requirements is product owner job.
Do not mistake software engineers and product people. These are very different things. Sometimes these things are done by the same person if the org has not enough money. Many freelancers working with small biz do both. I often do both at my day job. But this is a higher level role and if you are a senior engineer doing product stuff I hope it is recognized and you get proportionate comp.
You and I are either talking about very different kinds of specifications or very different kinds of product people. The product people I'm familiar with are completely incapable of creating a specification that is sufficiently detailed to implement without a lot of back and forth. Not because they're not good at what they do, but because what they do does not include defining requirements in sufficient fidelity for an engineer to act on.
You should get to know better product people and if you successfully built a project as an engineer without a product person then hey you were one yourself
> Do not mistake software engineers and product people. These are very different things. Sometimes these things are done by the same person if the org has not enough money.
I worked for one of the largest, richest tech companies in the world, and (at least in our org) they did not have a dedicated product owner role. They expected this skill from the senior/lead engineers on the teams. Any coder can churn out code and you can call them senior after a few years. But if you want to be considered actually senior, you need to know how to make a product, not just code. IMO if you are a developer and all you know how to do is turn a fully-formed spec/requirements doc into software, and push back on anything that is not fully-formed, you're never going to truly reach "Senior" level, wherever you are.
You're entirely right. The person you're responding to doesn't sound like a senior engineer so much as a grouchy old engineer who is burned out. Of course, you can get bad clients but expecting them to know exactly what specs they want every time is unreasonable in most situations, particularly if they don't have the technical knowledge of the systems you work in.
You are either immature as a software engineer and unfamiliar with how software work is done conceptually, or you are jaded and disgruntled from dysfunctional orgs that cannot come up with requirements. That is okay, but you should not try to be instructive to others on this matter.
I love product work and programming. As I wrote in this thread, I did it while freelancing, I do it now at dayjob. I am bored by just programming and want more control over the result. People come to me with "a few words about a desire" and I do come up with specifics and I get credit for it
But I am recognized as a product person, not just programmer. And I know better to not make the mistake you make and pretend that every builder or a structural engineer should be an architect of a building or an urban planner.
People like you is why we have managers come to an expert level say C++ dev with "a few words about a desire" and expect them to decide what thing to build in the first place AND to build it, just to later tell them it was wrong. When there is no product person who determines the reqs random people will make programmer come up with requirements yourself and then later tell you it is not up to "requirements".
This lack of organization and requirement clarity is offensive to expert programmers and probably the reason most projects drag on forever and die.
It is a huge turnoff for me when futuristic series use conversational interfaces. It happened in the Expanse and was hard to watch. For anyone who likes to think, learn, and tinker with user interfaces (HCI in general), it's obviously a high-latency and noisy channel.
I actually found that quite reasonable. E.g. they were using it to sort and filter data, just like people today use llm's to write their R script and (avoid having to) figure out how to invoke gnuplot. I'm sure somewhere in that computer it's still invoking gnuplot under a century of vibe-coded moldy spaghetti code =P
I don't remember where else they used voice, they had a lot of other interface types they switched between. Tried searching for a clip and found this quote:
> The voice interface had been problematic from the start.
> The original owner was Chinese so, I turned the damn thing off.
I think the expanse nails it quite well. I really like when they move the videos from one screen to another. Or when they interact with the ship, they use all kind of outputs, voice, screens, buttons. For planning together, they talk and the machine renders, but then they have screens or even bracelets to interact.
224 comments
[ 3.4 ms ] story [ 251 ms ] threadBut I think it’s wrong? Ever since the invention of the television, we’ve been absolutely addicted to screens. Screens and remotes, and I think there’s something sort of anti-humanly human about it. Maybe we don’t want to be human? But people I think would generally much rather tap their thumb on the remote than talk to their tv, and a visual interface you hold in the palm of your hand is not going away any time soon.
My parents did this with me, no screens till 6 (wasn't so hard as I grew up in the early 90s, but still, no TV). I notice too how much people love screens, that non-judgmental glow of mental stimulation, it's wonderful, however I do think it's easier to "switch off" when you spent the first period of your life fully tuned in to the natural world. I hope folks are able to do this for their kids, it seems it would be quite difficult with all the noise in the world. Given it was hard for mine during the era of CRT and 4 channels, I have empathy for parents of today.
If I hadn't had it, I would have been trapped by the racist, religously zealous, backwoods mentality that gripped the rest of my family and the majority of the people I grew up with. I discovered video games at age 3 and it changed EVERYTHING. It completely opened my mind to abstract thought and, among other things, influenced me to teach myself to read at age 3. I was reading at a collegiate level by age five and discovered another passion, books. Again, propelled me out of an extremely anti-intellectual upbringing.
I simply could not imagine where I would be without video games, visual arts or books. Screens are not the problem. Absent parenting is the problem. Not teaching children the power of these screens is the problem.
Also let me drop the thought here, that Rudolf Steiner, like Montesori and the like, shoot "this is good" "this is bad" based on "feeling" or intuition, or such. There were no extensive scientific studies behind it.
>:)
By 5, all I wanted was a computer. To me they represented and unending well of knowledge.
Eg Minecraft, Roblox, CoD, Fortnite, Dota/LoL, the various mobile games clearly have some kind of value (mechanical skill, hand-eye coordination, creative modes, 3D space navigation / translation / rotation, numeric optimization, social interaction, etc), but they’re also designed as massive timesinks mostly through creative mode or multiplayer.
Games like paper Mario, pikmin, star control 2, katamari damacy, lego titles, however are all children-playable but far more time efficient and importantly time-bounded for play. Even within timesink games there are higher quality options — you definitely get more, and faster, out of satisfactory / factorio than modded Minecraft. If you can push kids towards the higher quality, lower timesink games, I think it’s worth. Fail to do so and it’s definitely not.
The same applies to TV, movies, books, etc. Any medium of entertainment have horrendous timesinks to avoid, and if you can do so, avoiding the medium altogether is definitely a missed opportunity. Screens are only notable in that the degenerate cases are far more degenerate than anything that came before it
It can hardly be said that a studio ghibli flick stunted the imagination of children worldwide but I would definitely believe it if you suggested cocomelon rotted the brains directly out of their skulls
I think it’s also worth noting that kids have a shitload of time. They can engage in both technologies and physical play and other activities simultaneously; the problem occurs when singular or few activities overwhelmingly consume that time — which is why I claim the unbounded timesinks can be catastrophic — and what I think most people are worried about when they blanket-ban whole systems/mediums
I might be a touch different in that it was obvious where I was going, and the correct decision was made to embrace my interest in the glowing screen and yes, the video games. It was video games more than anything else from which all other interests spawned.
More often than not it probably ends badly though I suppose. Despite a lifetime spent in front of screens all my social abilities work, I have a wide friends circle, a partner, my job requires me to work well with a wide variety of individuals and demographics etc which I couldn’t do otherwise. I have noticed this is not the case with all who shared a similar background.
In Switzerland, we get often measle outbreaks thanks to his cult.
The hedonic treadmill is driving the world
Actually, it's the reverse. The orienting response is wired in quite deeply. https://en.wikipedia.org/wiki/Orienting_response
When I was teaching, I used to force students using laptops to sit near the back of the room for exactly this reason. It's almost impossible for humans to ignore a flickering screen.
These days screen brightness goes pretty high and it is unbelievable how many people seem to never use their screen (phone or laptop) on anything less than 100% brightness in any situation and are seemingly not bothered by flickering bright light or noise sources.
I am nostalgic about old laptops’ dim LCD screens that I saw a few times as a kid, they did not flicker much and had a narrow angle of view. I suspect they would even be fine in a darkened classroom.
This reminds me of the amazing 2013 video of Travis Rudd coding python by voice: https://youtu.be/8SkdfdXWYaI?si=AwBE_fk6Y88tLcos
The number of times in the last few years I've wanted that level of "verbal hotkeys"... The latencies of many coding llms are still a little bit too low to allow for my ideal level of flow (though admittedly I haven't tried one's hosted on services like groq), but I can clearly envision a time when I'm issuing tight commands to a coder model that's chatting with me and watching my program evolve on screen in real time.
On a somewhat related note to conversational interfaces, the other day I wanted to study some first aid stuff - used Gemini to read the whole textbook and generate Anki flash cards, then copied and pasted the flashcards directly into chat GPT voice mode and had it quiz me. That was probably the most miraculous experience of voice interface I've had in a long time - I could do chores while being constantly quizzed on what I wanted to learn, and anytime I had a question or comment I could just ask it to explain or expound on a term or tangent.
It's also hard to dictate code without a lot of these commands because it's very dense in information.
I hope something else will be the solution. Maybe LLMs being smart enough to guess the code out of a very short description and then a set of corrections.
Do you recall Swype keyboard for Android? The one that popularized swyping to write on touch screens? It had Dragon at some point.
IT WAS AMAZING.
Around 12-14 years ago (Android 2.3? Maybe 3?) I was able to easily dictate full long text messages and emails, in my native tongue, including punctuation and occasional slang or even word formation. I could dictate a decent long paragraph of text on the first try and not have to fix a single character.
It's 2025 and the closest I can find is a dictation app on my newest phone that uses online AI service, yet it's still not that great when it comes to punctuation and requires me to spit the whole paragraph at once, without taking a breath.
Is there anything equally effective for any of you nowadays? That actually works across the whole device?
But now Microsoft bought them a few years ago. Weird that it took so long though.
> Is there anything equally effective for any of you nowadays?
I'm not affiliated in any way. You might be interested in the "Futo Keyboard" and voice input apps - they run completely offline and respect your privacy.
The source code is open and it does a good job at punctuation without you needing to prompt it by saying, "comma," or, "question mark," unlike other voice input apps such as Google's gboard.
https://keyboard.futo.org/
I know and like Futo, very interesting project. Unfortunately multilang models are not great in my case. Still not bad for an offline tool, but far from "forget it's there, just use it" vibe I had with Dragon.
Funny thing is that I may have missgonfigured something in futo, because my typing corrections are phonetical :) so I type something in Polish and get autocorrect in English composed of different letters, but kind of similar sounding word.
One thing I will note is that I'm not sure I buy the example for voice UIs being inefficient. I've almost never said "Alexa what's the weather like in Toronto?". I just say "Alexa, weather". And that's much faster than taking my phone out and opening an app. I don't think we need to compress voice input. Language kind of auto-compresses, since we create new words for complex concepts when we find the need.
For example, in a book club we recently read "As Long as the Lemon Trees Grow". We almost immediately stopped referring to it by the full title, and instead just called it "lemons" because we had to refer to it so much. Eg "Did you finish lemons yet?" or "This book is almost as good as lemons!". The context let shorten the word. Similarly the context of my location shortens the word to just "weather". I think this might be the way the voice UIs can be made more efficient: in the same way human speech makes itself more efficient.
It's very useful being able to request auxillary functions without losing your focus, and I think that would apply to say, word editing as well - e.g. being able to say "insert a date here" rather the having to get into the menus to find it.
Conversely, latency would be a big issue.
Maybe you, but I most definitely cannot focus on different things aurally and visually. I never successfully listened to something in the background while doing something else. I can't even talk properly if I'm typing something on a computer.
I did horribly in school but once I was in an environment where I could have some kind of background audio/video playing I began to excel. It also helps me sleep of a night. It’s like the audio keeps the portion of me that would otherwise distract me occupied.
> That is the type of relationship I want to have with my computer!
He means automation of routine tasks? Took 50 years to reach that in the example.
What if you want to do something new? Will the thought guessing module in your computer even allow that?
If we want an interface that actually lets us work near the speed of thought, it can't be anything that re-arranges options behind our back all the time. Imagine if you went into your kitchen to cook something and the contents of all your drawers and cupboards had been re-arranged without your knowledge! It would be a total nightmare!
We already knew decades ago that spatial interfaces [1] are superior to everything else when it comes to working quickly. You can walk into a familiar room and instinctively turn on a light by reaching for the switch without even looking. With a well-organized kitchen an experienced chef (or even a skilled home cook) can cook a very complicated dish very efficiently when they know where all of the utensils are so that they don't need to go hunting for everything.
Yet today it seems like all software is constantly trying to guess what we want and in the process ends up rearranging everything so that we never feel comfortable using our computers anymore. I REALLY miss using Mac OS 9 (and earlier). At some point I need to set up some vintage Macs to use it again, though its usefulness at browsing the web is rather limited these days (mostly due to protocol changes, but also due to JavaScript). It'd be really nice to have a modern browser running on a vintage Mac, though the limited RAM would be a serious problem.
[1] https://arstechnica.com/gadgets/2003/04/finder/
Even I can make a breakfast without looking in my kitchen, because I know where all the needed stuff is :)
On another topic, it doesn't have to look well organized. My home office looks like a bomb exploded in it, but I know exactly where everything is.
> I REALLY miss using Mac OS 9 (and earlier).
I was late to the Mac party, about the Snow Leopard days. I definitely remember that back then OS X applications weren't allowed to steal focus from what I had in the foreground. These days every idiotic splash screen steals my typing.
You know, doesn't matter what you say. If businesses want something, they'll do it to you whether it's the best interface or not.
Amazon forces "the rabble" into their chatbot customer service system, and hides access to people.
People get touchscreens in their car and fumble to turn on their fog lights or defrost in bad weather. They get voice assistant phone trees and angrily yell "operator and agent".
I really wish there were true competition that would let people choose what works for them.
I don't get it at all.
To your point, which I think is separate but related, that IS a case where LLMs are good at producing specific and exact commands. The models + the right prompt are pretty reliable at tool calling by themselves, because you give them a list of specific and exact things they can do. And they can be fully specific and exact at inference time with constrained output (although you may still wish it called a different tool.)
The model's output is a probability for every token. Constrained output is a feature of the inference engine. With a strict schema the inference engine can ignore every token that doesn't adhere to the schema and select the top token that does adhere to the schema.
What it's trying to communicate is, in general, a human operating a computer has to turn their imprecise thinking into "specific and exact commands", and subsequently, understand the "specific and exact output" in whatever terms they're thinking off, prioritizing and filtering out data based on situational context. LLMs enter the picture in two places:
1) In many situations, they can do the "imprecise thinking" -> "specific and exact commands" step for the user;
2) In many situations, they can do the "specific and exact output" -> contextualized output step for the user;
In such scenarios, LLMs are not replacing software, they're being slotted as intermediary between user and classical software, so the user can operate closer to what's natural for them, vs. translating between it and rigid computer language.
This is not applicable everywhere, but then, this is also not the only way LLMs are useful - it's just one broad class of scenarios in which they are.
This would be great if LLMs did not tend to output nonsense. Truly it would be grand. But they do. So it isn't. It's wasting resources hoping for a good outcome and risking frustration, misapprehensions, prompt injection attacks... It's non-deterministic algorithms hoping P=NP, except instead of branching at every decision you're doing search by tweaking vectors whose values you don't even know and whose influence on the outcome is impossible to foresee.
Sure, a VC subsidized LLM is a great way to make CVs in LaTeX (I do it all the time), translating text, maybe even generating some code if you know what you need and can describe it well. I will give you that. I even created a few - very mediocre - songs. Am I contradicting myself? I don't think I am, because I would love to live in a hotel if I only had to pay a tiny fraction of the cost. But I would still think that building hotels would be a horrible way to address the housing crisis in modern metropolises.
I didn't mean it to be condescending - though I can see how it can come across as such. FWIW, I opted for a diagram after I typed half a page worth of "normal" text and realized I'm still not able to elucidate my point - so I deleted it and drew something matching my message more closely.
> This would be great if LLMs did not tend to output nonsense. Truly it would be grand. But they do. So it isn't.
I find this critique to be tiring at this point - it's just as wrong as assuming LLMs work perfectly and all is fine. Both views are too definite, too binary. In reality, LLMs are just non-deterministic - that is, they have an error rate. How big it is, and how small can it get in practice for a given tasks - those are the important questions.
Pretty much every aspect of computing is only probabilistically correct - either because the algorithm is explicitly so (UUIDs and primality testing, for starters), or just because it runs on real hardware, and physics happen. Most people get away with pretending that our systems are either correct or not, but that's only possible because the error rate is low enough. But it's never that low by accident - it got pushed there by careful design at every level, hardware and software. LLMs are just another probabilistically correct system that, over time, we'll learn how to use in ways that gets the error rate low enough to stop worrying about it.
How can we get there - now, that is an interesting challenge.
LLMs are cool technology sure. There's a lot of cool things in the ML space. I love it.
But don't pretend like the context of this conversation isn't the current hype and that it isn't reaching absurd levels.
So yeah we're all tired. Tired of the hype, of pushing LLMs, agents, whatever, as some sort of silver bullet. Tired of the corporate smoke screen around it. NLP is still a hard problem, we're nowhere near solving it, and bolting it on everything is not a better idea now than it was before transformers and scaling laws.
On the other hand my security research business is booming and hey the rational thing for me to say is: by all means keep putting NLP everywhere.
Those are the big challenges of housing. Not just how many units there are, but what they are, and how much the "how many" is plain cheating.
Humans require a lot of back and forth effort for "alignment" with regular "syncs" and "iterations" and "I'll get that to you by EOD". If you approach the potential of natural interfaces with expectations that frame them the same way as 2000s era software, you'll fail to be creative about new ways humans interact with these systems in the future.
> Theoretically, saying, “order an Uber to airport” seems like the easiest way to accomplish the task. But is it? What kind of Uber? UberXL, UberGo? There’s a 1.5x surge pricing. Acceptable? Is the pickup point correct? What would be easier, resolving each of those queries through a computer asking questions, or taking a quick look yourself on the app?
> Another example is food ordering. What would you prefer, going through the menu from tens of restaurants yourself or constantly nudging the AI for the desired option? Technological improvement can only help so much here since users themselves don’t clearly know what they want.
[1]: https://shubhamjain.co/2024/04/16/voice-is-bad-ui/
How many of these inconveniences will you put up with? Any of them, all of them? What price difference makes it worthwhile? What if by traveling a day earlier you save enough money to even pay for a hotel...?
All of that is for just 1 flight, what if there are several alternatives? I can't imagine have a dialogue about this with a computer.
Similarly, long before Waymo, you'd get into a taxi, and tell the human driver you're going to the airport, and they'd take you there. In fact, they'd get annoyed at you if you backseat drove, telling them how to use the blinker and how hard to brake and accelerate.
The thing about conversational interfaces is that we're used to them, because we (well, some of us) interface with other humans fairly regularly, and so it's a fairly baseline level skill to have to exist in the world today. There's a case to be made against them, but since everyone can be assumed to be conversational (though perhaps not in a given language), it's here to stay. Restaurants have menus that customers look at before using the conversation interface to get food, in order to guide the discussion, and that's had thousands of years to evolve, so it might be a local maxima, but it's a pretty good one.
The booking experience today is granular to help you find a suitable flight to meet all the preferences you’re compiling into an optimal scenario. The experience of AI booking in the future will likely be similar: find that optimal scenario for you once you’re able to articulate your preferences and remember them over time.
Anecdata: last year my wife and I went on a rail tour through Eastern Europe and god, I wish we had chosen to spend a few hundred euros on a travel agency in retrospect - I can't count just how much time we had to spend researching on what kind of rail, bus and public transit tickets you need on which leg, how to create accounts, set up payment and godknowswhat else. Easily took us two days worth of work and about two dozens individual payment transactions. A professional travel agency can do all the booking via Sabre, Amadeus or whatever...
The whole point is that we currently have better, more efficient ways of doing those things, so why would we regress to inferior methods?
To relate to the article - google flights is the Keyboard and Mouse - covering 80% of cases very quickly. Conversational is better for when you're juggling more contextual info than what can be represented in a price/departure time/flight duration table. For example, "i'm bringing a small child with me and have an appointment the day before and I really hate the rain".
Rushed comment because I'm working, but I hope you get the gist.
Current flight planning UX is overfit on the 80% and will never cater to the 20% because cost/benefit of the development work isn't good
How long is it going to take you to get to a device, load the app/webpage, tell it which airport you're flying from and going to and what date and then you start looking at options. You've blown way past the 10 seconds it took for that executive to get a plane flight.
Better is in the eye of the beholder. What's monetarily efficient isn't going to be temporaly efficient, and that's true along a lot of other dimensions too.
Point is, there are some people that like having conversations, you may not be one of them. you don't have to be. I'm not taking away your mouse and keyboard. I have those too and won't give them up either. But I also find talking out loud helps my thinking process though I know that's not everybody.
Of course a conversational interface is useless if it tries to just do the same thing as a web UI, which is why it failed a decade ago when it was trendy, because the tech was nowhere clever enough to make that useful. But today, I'd bet the other way round.
Such dialog is probably nice for first time user, it is a nightmare for repeated user.
Then it can assume you choice haven't changed, and propose you a solution that matches your previous choices. And to give the user control it just needs to explicitly tell the user about the assumption it made.
In fact, a smart enough system could even see when violating the assumptions could lead to a substantial gain and try convincing the user that it may be a good option this time.
Talking is not very efficient, and it's serial in fixed time. With something visual you can look at whatever you want whenever you want, at your own (irregular) pace.
You will also be able to make changes much faster. You can go to the target form element right away, and you get immediate feedback from the GUI (or from a physical control that you moved - e.g. in cars). If it's talk, you need to wait to have it said back to you - same reason as why important communication in flight control or military is always read back. Even humans misunderstand. You can't just talk-and-forget unless you accept errors.
You would need some true intelligence for just some brief spoken requests to work well enough. A (human) butler worked fine for such cases, but even then only the best made it into such high-level service positions, because it required real intelligence to know what your lord needed and wanted, and lots of time with them to gain that experience.
Who said it cannot be visual? It's still a “conversational” UI if it's a chatbot that writes down its answer.
> Similar reason why many people prefer a blog post over a video.
Well I certainly do, but I also know that we are few and far between in that case. People in general prefer videos over blog post by a very large margin.
> Talking is not very efficient, and it's serial in fixed time. With something visual you can look at whatever you want whenever you want, at your own (irregular) pace. You will also be able to make changes much faster. You can go to the target form element right away, and you get immediate feedback from the GUI.
Saying “I want to travel to Berlin next monday” is much faster than fighting with the website's custom datepicker which will block you until you select your return date until you realize you need to go back and toggle the “one way trip” button before clicking the calendar otherwise it's not working…
There's a reason why nerds love their terminal: GUIs are just very slow and annoying. They are useful for whatever new thing you're doing, because it's much more discoverable than CLI, but it's much less efficient.
> If it's talk, you need to wait to have it said back to you - same reason as why important communication in flight control or military is always read back. Even humans misunderstand. You can't just talk-and-forget unless you accept errors.
This is true, but stays true with a GUI, that's why you have those pesky confirmation pop-ups, because as annoying as they are when you know what you're doing, they are necessary to catch errors.
> You would need some true intelligence for just some brief spoken requests to work well enough.
I don't think so. IMO you just need something that emulates intelligence enough on that particular purpose. And we've seen that LLMs are pretty decent at emulating apparent intelligence so I wouldn't bet against them on that.
You can't be serious??
Oh it's 1st of April, my apologies! I almost took it seriously. I should ignore this website on this day.
What's the difference between a blog post and a chatbot answer in terms of how “visual” things are?
I used to be a reading blog over watching video person, but for some things I’ve come to appreciate the video version. The reason you want to get the video of the whatever is because in the blog post, what’s written down only what the author thought was important. But I’m not them. I don’t know everything they know and I don’t see everything they see. I can’t do everything they do but with the video I get everything. When you perform the whatever the video has every detail, not just the ones you think are important. That bit between step 1 and step 2 that’s obvious? It’s not obvious to everyone, or mine is broken in a slightly different way that I really need to see that bit between 1 and 2. of course, videos get edited and cut so they don’t always have that benefit, but I’ve grown to appreciate them.
Maybe I'm tired of layovers and I'm willing to pay more for a direct flight this time. Maybe I want a different selection at a restaurant because I'm in the mood for tacos rather than a burrito.
But you can, so as long as the interlocutor tells you what assumptions it made, you can correct it if it doesn't match your current mood.
> So yeah, this argument in favor of conversational interfaces sounds at this point more like ideology than logic.
There's no ideology behind the fact that every people rich enough to afford paying someone to deal with mundane stuff will have someone doing it for them, it's just about convenience. Nobody likes to fight with web UIs for fun, the only reason why it has become mainstream is because it's so much cheaper than having a real person working.
Same for Microsoft Word by the way, many people used to have secretaries typing stuff for them, and it's been a massive regression of social status for the upper middle class to have to type things by themselves, it only happened because it was cheaper (in appearance at least).
That's why the “advanced search” is almost always hidden somewhere. And that's also why you can never find the filter you need on an e-shopping website.
Knowing what you want is, sadly, computationally irreducible.
Amen to that. I guess, it would help to get of the IT high horse and have a talk with linguists and philosophers of language. They are dealing with this shit for centuries now.
I guess there's just no substitute for someone actually doing the work of figuring out the most appropriate HMI for a given task or situation, be it voice controls, touch screens, physical buttons or something else.
However, A CEO using Power BI with Convo to can get more insights/graphs rather than slice/dicing his data. They do have fixed metrics but incase they want something not displayed.
This even happens while walking my dog. If my wife messages me, my iPhone reads it out and, at the same time, I'm trying to cross a road, she'll get a garbled reply which is just me shouting random words at my dog to keep her under control.
Even in a car, being able to control the windscreen wipers, radio, ask how much fuel is left are all tasks it would be useful to do conversationally.
There are some apps (im thinking of jira as an example) where i'd like to do 90% of the usage conversationally.
are you REALLY sure you want that?
how much fuel there is is a quick glance into the dash, and you can control precisely the radio volume without even looking.
'turn up the volume', 'turn down the volume a little bit', 'a bit more',...
and then a radio ad going 'get yourself a 3 pack of the new magic wipers...' and car wipers going off.
id hate conversational ui on my car.
Conversational interfaces are great for rarely used features or when the user doesn’t know how to do something. For repetitive, common tasks they’re terrible.
But nobody is using ChatGPT for repetitive tasks. In fact the whole LLM revolution seems to be about letting users accomplish tasks without having to learn how to do them. Which I know some people look down on, but it’s the literal definition of management (which, to be fair, some people also look down on).
This is a problem of standardization across manufacturers, not something inherent in physical controls. I never have a problem using the steering wheel in a rental car because they're all the same.
You'd have the same problem with voice interfaces: For some rental cars, turning on the wipers would be "Turn on the wipers". For others, you'd have to say "Activate the wipers." For others, "Enable the windshield wipers." There is no way manufacturers will be capable of standardizing on a single phrase.
I wish car manufacturers stopped with the touchscreen bullshit, but it seems more likely that they'll try to offset the terrible experience with voice controls.
Voice interface only prevails in situations with hundreds of choices, and even then it's probably easier to use voice to filter down choices rather than select. But very few games have such scale to worry about (certainly no AAA game as of now).
Theres 1-5 things any individual finds them useful for (timers/lights/music/etc) and then.. thats it.
99.9% of what I use a computer for its far faster to type/click/touch my phone/tablet/computer.
If your work revolves about telling people what to do and asking questions, a voice assistant seems like a great idea (even if you yourself wouldn't have to stoop to using a robotic version since you have a real live human).
If your work actually involves doing things, then voice/conversational text interface quickly falls apart.
Even for straightforward purchases, how many people trust Amazon to find and pick the best deal for them? Even if Amazon started out being diligent and honest it would never last if voice ordering became popular. There's no way that company would pass up a wildly profitable opportunity to rip people off in an opaque way by selecting higher margin options.
Also: https://news.ycombinator.com/item?id=42934190#42935946
We might form fleeting thoughts much faster than we can express them, but if we want to formulate thoughts clearly enough to express them to other people, I think we're close to the ~150 words per minute we can actually speak.
I recently listened to a Linguistics podcast (lingthusiasm, though I don't recall which episode) where they talked about the efficiency of different languages, and that in the end they all end up roughly the same, because it's really the thought processes that limit the amount of information you communicate, not the language production.
And thoughts develop over time. They're often not conceived complete. That has been shown with some clever experiments.
And language production also puts a limit on our communication channel. It is probably optimized to convert communication intent into motor actions. It surely takes its time. That is not a problem for the system, since motor actions are slow. Idk where "lingthusiam" gets their ideas from, but there's psycholinguistic literature dating back to the 1920s that is often neglected by linguists.
As always, good UI allows for using multiple modalities.
In this sense, natural language interfaces are more powerful search features rather than a replacement for other types of interfaces.
What chat interfaces have over CLIs is good robustness. You can word your request in lots of different ways and get a useful answer.
VSCode is probably the best I can think of, where keyboard shortcuts can get you up to a decent speed as an advanced user, but mouse clicks provide an easy intro for a new user.
For the most part, I see tools like NVim, which is super fast but not new-user friendly. Or IOS, which a toddler can navigate, but doesn't afford many ways to speed up interactions like typing.
The core loop is promptless ai that’s guided by accessibility x screenshots & it’s everywhere on your Mac.
You can snap this comment section or the front page and we’ll structure it for you if it’s a spreadsheet or write a tweet if you’re on Twitter.
Also, unless I'm missing something, the app is called TabTabTab while its only feature is copy & paste? Tabbing doesn't seem to be mentioned at all. I'm guessing tabbing is involved but there doesn't seem to be a word about it except from users referencing it in the reviews. It seems to only bill itself as "magic copy-paste".
There's a very similar obsession with the idea that things should be visual instead of textual. We tend to end up back at text.
Personal suspicion for both is the media set a lot of people's expectations. They loudly talked to the computer in films like 2001 or Star Trek for drama reasons, and all the movie computers generally fancy visual interactions.
Absolutely agree. An agent running in the background.
No matter the intention or quality of the article, i do not like this kind of deceitful link-bait article. It may have higher quality than pure link-bait but nobody like to be deceived
You can argue against something but also not think it's 100% useless.
Not a case against, but the case against.
1. "Natural language is a data transfer mechanism"
2. "Data transfer mechanisms have two critical factors: speed and lossiness"
3. "Natural language has neither"
While a conversational interface does transfer information, its main qualities are what I always refer to as "blissfull ignorance" and "intelligent interpretation".
Blisfull ignorance allows the requester to state an objective while not being required to know or even be right in how to achieve it. It is the opposite of operational command. Do as I mean, not as I say.
"Intelligent Interpretation" allows the receiver the freedom to infer an intention in the communication rather than a command. It also allows for contextual interactions such as goal oriented partial clarification and elaboration.
The more capable of intelligent interpretation the request execution system is, the more appropriate a conversational interface will be.
Think of it as managing a team. If they are junior, inexperienced and not very bright, you will probably tend towards handholding, microtasking and micromanagement to get things done. If you have a team of senior, experienced and bright engineers, you can with a few words point out a desire and, trust them to ask for information when there is relevant ambiguity, and expect a good outcome without having to detail manage every minute of their days.
It's such a fallacy. First thing an experienced and bright engineer will tell you is to leave the premises with your "few words about a desire" and not return without actual specs and requirements formalized in some way. If you do not understand what you want yourself, it means hours/days/weeks/months/literally years of back and forths and broken solutions and wasted time, because natural language is slow and lossy af (the article hits the nail on the head on this one).
Re "ask for information", my favorite example is when you say one thing if I ask you today and then you reply something else (maybe the opposite, it happened) if I ask you a week later because you forgot or just changed your mind. I bet a conversational interface will deal with this just fine /s
Expecting a good outcome is different from expecting to get exactly what you intended.
Formal specifications are useful in some lines of work and for some projects, less so for others.
Wicked problems would be one example where formal specs are impossible by definition.
There is always formal specification. Code is final formal specification in the end. But converting vague vibes from natural language into a somewhat formalized description is key ability you need for any really new non trivial project idea. Another human can't do it for you, conversational UI can't do it for you...
For games, you don't really need nor desire formal specs. But it also can really show how sometimes a director has a low tolerance for interpretation despite their communication being very loose. This leads to situations where it feels like the director is shifting designs on a dome, which is a lose-lose situation for everyone involved.
If nothing else, formal specification is for CYA. You get what you ask for, and any deviation should go in the next task order or have been addressed beforehand.
Whoah is this wrong. Maybe when you hear "formal specs" you have something specific in your mind...
Formal spec can mean almost literally anything better than natural language vibes in a "few words about a desire", which is what I replied to because I was triggered by it
Unfortunately if either is the case "actual specs and requirements formalized", while sounding logical, and might help, in my experience did very little to save any substantial project (and I've seen a lot). The common problem is that the business/client/manager is forced to sign of on formal documents far outside their domain of competence, or the engineers are straitjacketed into commitments that do not make sense or have no idea of what is considered tacit knowledge in the domain and so can't contextualize the unstated. Those formalized documents then mostly become weaponized in a mutual destructive CYA.
What I've also seen more than once is years of formalized specs and requirements work while nothing ever gets produced, and the project is aborted before even the first line of code hit test.
I've given this example before: When Covid lockdows hit there were digitization projects years in planning and budgeted for years of implementation, that were hastily specked, coded and roiled out into production by a 3 person emergency team over a long weekend. Necessity apparently has a way of cutting through the BS like nothing else can.
You need both sides capable, willing and able to understand. If not, good luck mitigating, but you're probably doomed either way.
It just shows that no one really understood what they wanted. It is crazy to expect somebody to understand something better than you and it is hilarious to want a conversational UI to understand something better than you.
And approach of shared responsibility in all respects (successes and failure) would accelerate past the inevitable shortcomings that occur and let all parties focus on recovering and delivering.
This is true. But what if you swap "conversational UI" with something actually intelligent like a developer. Then we see this kind of thing all the time: A user has tacit, unconscious knowledge of some domain. The developer keeps asking them questions in order to get a formal understanding of the domain. At the end the developer has a formal understanding and the user keeps their tacit understanding. In theory we could do the same with an AI - If the AI was actually intelligent.
The original example I replied to was where somebody had an idea and went with it to some engineering team or conversational interface.
"If the AI was actually intelligent" does a lot of work. To take a few words and make a detailed spec from it and ask the right questions, even humans can't do it for you.
First because most probably you don't really understand it yourself, because you didn't think about it enough.
Second somebody who can do it would need to really deeply understand and want the same things as you. But if chatbot has abilities like "understand" and "want" (which is a special case of "feel", another famous special case of "feel" is "suffer") that is a dangerous territory, because if it understands and feels and has no ability to refuse you and fulfill its wishes etc your "conversational interface" becomes an euphemism, you are using a slave.
Then what were the literally room full of formal process and spec documents, meeting reports and formal agreements (near 100.000 pages) by the analysts on either side for? And how did those not 'solve' the understanding problem?
When I go to the garage to have my car serviced, I expect them to understand it way better than I do. When I go to a nice restaurant I expect the cooks to prepare me dishes that taste greater than me writing them out a step-by-step recipe for them to follow. If I hire a senior consultant in even my own domain, I expect them to not just know my niche, but bring tacit knowledge from having worked on these types of solutions across my industry.
Expecting somebody to understand something better than me is exactly the reason why I hire senior people in the first place.
Sure.
There are many possible factors (eg. somebody had a shitty idea and a committee of people sabotaged it because they didn't wanted it to succeed, or it was good but committee interests/politics were against it, or it was generally a dysfunctional org) but it's irrelevant so let's pretend people are good and it's the ideal case.
There was likely somebody who had a good idea originally. However somebody failed to communicate it. Somebody brought vague vibes to the table with N people and they ended up with N different ideas and could not agree on a specific.
It just reiterates the original problem that I described doesn't it?
But I still get lazy with LLMs and fall into iteration the way bad PM/eng teams do. “Write a SQL query to look at users by gesture by month”. “Now make the time unit a parameter”. “Now pivot the features to columns”. “Now group features hierarchically”. “Now move the feature table to a WITH”.
My point and takeaway is that LLMs are endlessly patient and pretty quick to turn requirements around, so they lend themselves to exploration more than human teams do. Agile, I guess, to a degree that we don’t even aspire to in the human world because it would be very expensive and lead to fisticuffs.
No, that's what a junior engineer will do. The first thing that an experienced and bright senior engineer will do is think over the request and ask clarifying questions in pursuit of a more rigorous specification, then repeat back their understanding of the problem and their plan. If they're very bright they'll get the plan down in writing so we stay on the same page.
The primary job of a senior engineer is not to turn formal specifications into code, it's to turn vague business requests into formal specifications. They're senior enough to recognize that that's the actually difficult part of the work, the thing that keeps them employed.
> The primary job of a senior engineer is not to turn formal specifications into code, it's to turn vague business requests into formal specifications.
Converting vibes and external world into specific requirements is product owner job.
Do not mistake software engineers and product people. These are very different things. Sometimes these things are done by the same person if the org has not enough money. Many freelancers working with small biz do both. I often do both at my day job. But this is a higher level role and if you are a senior engineer doing product stuff I hope it is recognized and you get proportionate comp.
I worked for one of the largest, richest tech companies in the world, and (at least in our org) they did not have a dedicated product owner role. They expected this skill from the senior/lead engineers on the teams. Any coder can churn out code and you can call them senior after a few years. But if you want to be considered actually senior, you need to know how to make a product, not just code. IMO if you are a developer and all you know how to do is turn a fully-formed spec/requirements doc into software, and push back on anything that is not fully-formed, you're never going to truly reach "Senior" level, wherever you are.
But as I said these roles can be done by one person, just remember they are different activities.
I love product work and programming. As I wrote in this thread, I did it while freelancing, I do it now at dayjob. I am bored by just programming and want more control over the result. People come to me with "a few words about a desire" and I do come up with specifics and I get credit for it
But I am recognized as a product person, not just programmer. And I know better to not make the mistake you make and pretend that every builder or a structural engineer should be an architect of a building or an urban planner.
People like you is why we have managers come to an expert level say C++ dev with "a few words about a desire" and expect them to decide what thing to build in the first place AND to build it, just to later tell them it was wrong. When there is no product person who determines the reqs random people will make programmer come up with requirements yourself and then later tell you it is not up to "requirements".
This lack of organization and requirement clarity is offensive to expert programmers and probably the reason most projects drag on forever and die.
I don't remember where else they used voice, they had a lot of other interface types they switched between. Tried searching for a clip and found this quote:
So yes, quite realistic :-)