I've never tried Siri or S Voice so I can't say anything about the quality of the services or the overall experience but judging by what I saw in the video it didn't seem to me like it was "a complete blowout for Siri."
The main metric in the comparison seemed to be speed, at which S Voice was a little bit better than Siri. Also, voice quality was slightly in favor of S Voice.
The main differences were S Voice offering image results right away instead of asking to open the browser. On the other hand, S Voice required non-voice input to to specify the remineder time whereas Siri carried on "the conversation". Other than that, the two products pretty much behaved exactly the same. Ask something, get a result.
If you ask me, it was more of a 55:45 for S voice based on speed and naturalness of voice but then I don't run a gadget blog...
Watch the video again. That's not Samsung's S-Voice. It's Google's new Voice Search. And he's saying it's a lot better than Siri (as other sites and tests have shown, too).
He only mentioned S-Voice because he tested S-voice and Siri in another video (not this one), and in that one Siri beats Samsung's S-voice (powered by Vlingo).
Google's Voice Search seems faster, it's significantly more accurate (86% vs 68% for Siri), and it also comes with that smart Google Now thing, which learns things about you and recommends stuff for you to do based on your location, or behavior or whatever.
Oh, and Google's "voice" sounds a lot more natural than Siri, too, which sounds more robotic.
Again slightly faster and slightly more robotic is not a blowout.
And that accuracy statistic you listed is wrong. It is for Google Search Engine which is different from what is on the Android phone. Also the fact that Apple can be in the same ballpark as Google despite not spending billions on it is pretty impressive.
I propose we stop calling machine transcription and fixed-field extraction "AI." Second?
We all realize Siri is just a fancy Eliza with Internet access, right?
We can resume calling something "AI" when it can order a flight for you from the Ryanair website without making any mistakes (I'm not even at that level yet).
But what you listed isn't AI either by your definition. It is just a series of predefined steps. The old Siri app had the ability to find and book a table at a restaurant. That's basically similar in complexity.
It's an interesting point though. At what point does it really become truly intelligent ? And I would guess that it would differ for each person.
• To search for flights, you must answer a captcha.
• They forcefully push travel insurance, luggage, luggage insurance, insurance insurance, extra baggage fees, and useless priority boarding.
• Once you start your search, you must complete their 5-step ordering process within 3 minutes or you have to start over from the beginning of your search.
The system we will call "AI" would be able to go from you saying "Order a flight between Dublin and Barcelona on the cheapest carrier" to actually using their unusable website to complete your request.
I imagine the old Siri app was using opentable or some other API to book reservations. It wasn't navigating the forest of unstructured innate dumbness in the world.
No. Machine transcription requires strong AI to be really good. As you saw in my example with the Kelis song, it isn't enough to just transcribe the input syllable by syllable. You need the software to be able to "understand" that I'm requesting a song, and that "play a capella" makes no sense, and adjust the transcription to tale into account that the artist Kelis has a song with the name "Acapella".
There are thousands of cases like this, where natural language is amibigous, but a human has no trouble understanding the intention.
Siri has tons of databases avsilable and knows my taste in music. It should do a better job than a human understanding what song i want.
Excellent point. Perhaps I should have specified "current-level, mostly-non-situationally-contextual machine transcription." Once tone, intent, situation, and context are taken into account we get closer to actual communication instead of "listen to some discrete phonemes then try to assemble appropriate words based on parts of speech."
Well. The definition of AI, should not be the sophistication or success, but the intention.
A chess AI is little more than a recursive search for a maximum return, or a database search. There is not really anything distinguishing it from any other algorithm. Just the intention of the developer.
It is even worse if you use it in languages other than english since Siri only recognizes the language you have set before and that also applies to artist names and titles. In my case I would have to pronounce bands like "Real Estate" the german way in order to use this function.
Apple should have anticipated that and allowed music related commands to be analyzed in your native language and english.
The brilliance of Siri isn't the voice interface or the replies using text-to-speech -- it's just the name. People feel they are talking to another person, not a phone or a computer or a google. People understand Siri approximates a person, so they can interact with it informally and somewhat personally.
You see it in all the news coverage. People refer to Siri as a person. It gets into your brain. Have you seen normal people showing off Siri to their friends? They are giddy with excitement over having a personal robot voice in their pocket that knows them by name.
I feel google doesn't employ anybody who has an average family or knows average people. Normal people don't know how things work. What's a "Google Now?" You'd have to look up a definition. It's about ten different things jumbled together under an Enterprise-Grade Naming Convention product umbrella.
Look at the Google Now product intro -- "Google Now has prepared an alternate route for your commute." People have to learn "Google Now" is a product and/or service that does specific things. People simply understand what Siri is after seeing someone else use it for a few seconds.
Even the Google Now tagline requires a level of mental disambiguation: "...with the predictive power of Now..."
Personally, I think that's a little gimmicky. I think it was very smart of Google to not differentiate this too much, but let people think that this smarter AI thing, is actually still "Google". So in the future people will just refer to "Google" when doing stuff like this.
It's also a bit silly seeing people talking to "Siri" in public.
I do it too. Not so much because I want google to parse the question and answer it, but often because I want to find forums/etc where someone has already asked the same question and hopefully got an answer
>It's also a bit silly seeing people talking to "Siri" in public.
I wouldn't be surprised if this is exactly what the future will bring us as SIRI improves. People talking to it more or less the same you see them talking to their pets now.
It's difficult to observe someone talking to Siri. I believe Apple preemptively realised that people wouldn't like to talk aloud to their phones in public (and in the process revealing their personal requirements/plans to anyone in ear shot). The much better method of using Siri in public is with the handset against the ear.
In public it's much easier, more accurate and far less obvious to use this other method of activating Siri: When the screen is on, picking up the phone and putting the handset to ones ear will activate Siri, but far more discretely that the usual full volume activation. Any casual observer won't even realise that the user is engaging Siri, let alone talking to a robot.
This is also part of the problem with the "accuracy" studies of Siri in public places. It's just non sensical to be shouting at your phone over street noise, of course the accuracy is decreasing in this environment. Not even the users voice is the same in this environment. However when using the handset to the ear method: the accuracy remains acceptably high, and no one has to look silly publicly announcing their dinner plans.
Siri doesn't support Portuguese, or any Latin languages other than French at this time. iOS 6 adds Spanish and Italian, and I assume Portuguese can't be far behind.
Which one can give the answer a fraction of a second faster, or who's voice sounds less robotic means little to me.
I am more interested in knowing about speech recognition capabilities, edge cases handling, frequency of getting something wrong, accented speech recognition capabilities, artificial intelligence maturity and depth and breadth of the knowledge base the systems use.
But this video has focused on the wrong things. I was expecting that the presenter would be asking both some hard but useful questions and show us which one comes out with the right answer.
In almost all of the things you mentioned, google has the advantage to make a better product. They have an enormous power to train their creation, to achieve a higher level in terms of speech recognition and understanding. I'm not saying that they are already there, only that they have what is needed.
But i really don't know how far are they willing too go, as taking it too far away could end up being counterproductive to them, as they earn money showing ads in their results, so they need people using their website. That's why they released chrome now in iOS.
> In almost all of the things you mentioned, google has the advantage to make a better product. They have an enormous power to train their creation, to achieve a higher level in terms of speech recognition and understanding. I'm not saying that they are already there, only that they have what is needed.
I'm not sure why you think this. Both Apple and Google probably have vast amounts of unlabeled voice data that they could use to train their speech recognition engines. Just having a product gives you that. Either would need to collect labeled voice data for training purposes, and both have plenty of money to do this.
As far as understanding goes, having a shitton of training data may not help all that much. What you really need is a superior algorithm. While one could argue that Google's engineers have more expertise in this regard, NLP isn't really either company's specialty, and Apple can hire smart people too.
I think the advantage Google has is that they can do search better. That's basically what this comparison shows. The reason Google Now is faster is that Google does search better. I would think that Google should also be able to get Now to answer harder questions, but this demo doesn't show that, and in that respect I think it's a bit of a disappointment. I was expecting to see Google Now do things Siri can't, but it isn't yet clear that it can.
Well, I can't seem to find the raw source material -- but here's a news article about the relative accuracies of Google vs. Siri. 86% accuracy (Google) vs. 68% accuracy (Siri).
This is not surprising. Google has been doing question answering on web search for many years now. I believe it started answering direct free-form questions in its oneboxen in 2007 -- maybe earlier.
If its better, the question becomes: does Google release it on iOS as well, and will Apple allow them too.
However, i think the US readers are missing the most important aspect: how many people in the world can you serve?
This is a system selling feature during a crucial generation. The speed they can rollout new languages, will determine the world market and the long tail, which will prove crucual.
Currently, Google has the best infrastructure to do that, but Apple seems to care more. I live in Holland, for example. No siri, but at least we can buy content from iTunes. Google isnt selling anything but apps in its playstore.
I think whoever is able to actually give me all these features first, wins Holland. Why would it be any different in any other country?
There are markets up for grabs, and the first one to realize that fully, wins.
Apple has one (1) datacenter in the entire world, and it's in the USA. How good do you think the latency from Holland will be? Google has datacenters in Groningen and Eemshaven, and all over Europe.
You are wrong about Apple, they have one huge datacenter in the US where people suspect the Siri backend runs. They also have a smaller data center in California. We don't actually know what runs where, and we don't know what they are running in leased space in 3rd party datacenters. And, they are building other datacenters in the US.
In any case, wherever Siri runs, I don't think that Internet latency comes close to explaining the latency, and, I think, Siri can actually hide a lot of latency.
>Currently, Google has the best infrastructure to do that, but Apple seems to care more. I live in Holland, for example. No siri, but at least we can buy content from iTunes. Google isnt selling anything but apps in its playstore.
I'm not sure you appreciate the issues at play with licensed content. The iTunes Store opened in 2003 and was restricted to the US for over a year. Apple expanded it into international markets very slowly, over several years time, and didn't even move outside the western world until just last year.
My point is that it takes quite a bit of time and effort to address complex international issues with content licenses and rights holders. Apple has spent almost a decade working on this and they're only recently getting to a level of international ubiquity. Whereas Google has been at it for barely a year, but is moving pretty quickly as far as I can tell.
Comparing efficiency of pre-release software is kind of pointless. Either one might be missing some optimizations that are put in on the final release.
Here's a question: how well will the voice recognition features of the next version of Android compare with the next version of iOS? And what proportion of Android users will be able to upgrade to it?
Releasing a slightly superior clone of a feature in a rival product over six months after the first product shipped isn't enormously impressive, although it is more impressive than shipping an inferior clone of a product two years after the first product shipped, so that's definitely progress.
Android has had voice actions and voice search since 2.2--over a year before the Siri beta was even announced. Speech control has been a rapidly evolving and increasingly robust set of features in Android, but for some reason people are claiming Apple moved first. Honestly, I'm amazed at the extent to which marketing can completely overwrite people's recollection of objective reality.
In fairness, the video seems to be comparing exactly this -- the sports questions seem to show that he's using iOS 6.
Speed and voice quality are irrelevant to me, but I do really like what he was able to do with the image search on android. It wasn't clear if the android version integrates with wolfram alpha, or any of siri's other sources.
I'm curious if Google's new thing has the capability to carry on a conversation. With Siri I can ask her something, see it, then say "email that to ___" and she can do it.
This great but it's nearly entirely dependent on the network connection otherwise they're both useless.
I supposedly have an HSPA+ 21Mbps connection yet only get 1Mbps/.03Kbps maybe 5Mbps/950Kbps Kbps on a good day and latency is 100ms or more, Bell Mobility in Canada. I'll never get 21Mbps but less than 25% at best?
Google and Apple can make fancy tech but it's ruined by poor networks.
53 comments
[ 0.43 ms ] story [ 172 ms ] threadThe main metric in the comparison seemed to be speed, at which S Voice was a little bit better than Siri. Also, voice quality was slightly in favor of S Voice.
The main differences were S Voice offering image results right away instead of asking to open the browser. On the other hand, S Voice required non-voice input to to specify the remineder time whereas Siri carried on "the conversation". Other than that, the two products pretty much behaved exactly the same. Ask something, get a result.
If you ask me, it was more of a 55:45 for S voice based on speed and naturalness of voice but then I don't run a gadget blog...
He only mentioned S-Voice because he tested S-voice and Siri in another video (not this one), and in that one Siri beats Samsung's S-voice (powered by Vlingo).
Oh, and Google's "voice" sounds a lot more natural than Siri, too, which sounds more robotic.
And that accuracy statistic you listed is wrong. It is for Google Search Engine which is different from what is on the Android phone. Also the fact that Apple can be in the same ballpark as Google despite not spending billions on it is pretty impressive.
"Play music by Milk Inc."
It might ask me if I want to buy milk. Or if I say
"Play Acappella by Kelis."
It will respond
"Sorry, I can't find a capella in your music."
Stuff like that. And it is borderline unusable at all if I'm out of breath, like when I'm running and handsfree controlls makes sense.
Siri doesn't need a speed improvment. It needs orders of magnitude better AI.
We all realize Siri is just a fancy Eliza with Internet access, right?
We can resume calling something "AI" when it can order a flight for you from the Ryanair website without making any mistakes (I'm not even at that level yet).
It's an interesting point though. At what point does it really become truly intelligent ? And I would guess that it would differ for each person.
• To search for flights, you must answer a captcha.
• They forcefully push travel insurance, luggage, luggage insurance, insurance insurance, extra baggage fees, and useless priority boarding.
• Once you start your search, you must complete their 5-step ordering process within 3 minutes or you have to start over from the beginning of your search.
The system we will call "AI" would be able to go from you saying "Order a flight between Dublin and Barcelona on the cheapest carrier" to actually using their unusable website to complete your request.
I imagine the old Siri app was using opentable or some other API to book reservations. It wasn't navigating the forest of unstructured innate dumbness in the world.
There are thousands of cases like this, where natural language is amibigous, but a human has no trouble understanding the intention.
Siri has tons of databases avsilable and knows my taste in music. It should do a better job than a human understanding what song i want.
A chess AI is little more than a recursive search for a maximum return, or a database search. There is not really anything distinguishing it from any other algorithm. Just the intention of the developer.
Apple should have anticipated that and allowed music related commands to be analyzed in your native language and english.
You see it in all the news coverage. People refer to Siri as a person. It gets into your brain. Have you seen normal people showing off Siri to their friends? They are giddy with excitement over having a personal robot voice in their pocket that knows them by name.
I feel google doesn't employ anybody who has an average family or knows average people. Normal people don't know how things work. What's a "Google Now?" You'd have to look up a definition. It's about ten different things jumbled together under an Enterprise-Grade Naming Convention product umbrella.
Look at the Google Now product intro -- "Google Now has prepared an alternate route for your commute." People have to learn "Google Now" is a product and/or service that does specific things. People simply understand what Siri is after seeing someone else use it for a few seconds.
Even the Google Now tagline requires a level of mental disambiguation: "...with the predictive power of Now..."
It's also a bit silly seeing people talking to "Siri" in public.
I wouldn't be surprised if this is exactly what the future will bring us as SIRI improves. People talking to it more or less the same you see them talking to their pets now.
In public it's much easier, more accurate and far less obvious to use this other method of activating Siri: When the screen is on, picking up the phone and putting the handset to ones ear will activate Siri, but far more discretely that the usual full volume activation. Any casual observer won't even realise that the user is engaging Siri, let alone talking to a robot.
This is also part of the problem with the "accuracy" studies of Siri in public places. It's just non sensical to be shouting at your phone over street noise, of course the accuracy is decreasing in this environment. Not even the users voice is the same in this environment. However when using the handset to the ear method: the accuracy remains acceptably high, and no one has to look silly publicly announcing their dinner plans.
"What is the difference between Google Now and Google Instant ?"
Well, they understan what it should be, but it is so far from reaching that goal it's ridiculous.
Remember the introduction of Siri? It was presented as your personal assistant, understanding your commands and making sense of everyday speech.
So far from the truth.
I am more interested in knowing about speech recognition capabilities, edge cases handling, frequency of getting something wrong, accented speech recognition capabilities, artificial intelligence maturity and depth and breadth of the knowledge base the systems use.
But this video has focused on the wrong things. I was expecting that the presenter would be asking both some hard but useful questions and show us which one comes out with the right answer.
But i really don't know how far are they willing too go, as taking it too far away could end up being counterproductive to them, as they earn money showing ads in their results, so they need people using their website. That's why they released chrome now in iOS.
I'm not sure why you think this. Both Apple and Google probably have vast amounts of unlabeled voice data that they could use to train their speech recognition engines. Just having a product gives you that. Either would need to collect labeled voice data for training purposes, and both have plenty of money to do this.
As far as understanding goes, having a shitton of training data may not help all that much. What you really need is a superior algorithm. While one could argue that Google's engineers have more expertise in this regard, NLP isn't really either company's specialty, and Apple can hire smart people too.
I think the advantage Google has is that they can do search better. That's basically what this comparison shows. The reason Google Now is faster is that Google does search better. I would think that Google should also be able to get Now to answer harder questions, but this demo doesn't show that, and in that respect I think it's a bit of a disappointment. I was expecting to see Google Now do things Siri can't, but it isn't yet clear that it can.
This is not surprising. Google has been doing question answering on web search for many years now. I believe it started answering direct free-form questions in its oneboxen in 2007 -- maybe earlier.
http://tech.fortune.cnn.com/2012/06/29/minneapolis-street-te...
Google coming out with a competitor that stacks up feature-wise doesn't fix that for them because it will only run on Android.
Google's going to miss out on a lot of search/ad revenue due to Siri in the future.
However, i think the US readers are missing the most important aspect: how many people in the world can you serve?
This is a system selling feature during a crucial generation. The speed they can rollout new languages, will determine the world market and the long tail, which will prove crucual.
Currently, Google has the best infrastructure to do that, but Apple seems to care more. I live in Holland, for example. No siri, but at least we can buy content from iTunes. Google isnt selling anything but apps in its playstore.
I think whoever is able to actually give me all these features first, wins Holland. Why would it be any different in any other country?
There are markets up for grabs, and the first one to realize that fully, wins.
In any case, wherever Siri runs, I don't think that Internet latency comes close to explaining the latency, and, I think, Siri can actually hide a lot of latency.
I'm not sure you appreciate the issues at play with licensed content. The iTunes Store opened in 2003 and was restricted to the US for over a year. Apple expanded it into international markets very slowly, over several years time, and didn't even move outside the western world until just last year.
My point is that it takes quite a bit of time and effort to address complex international issues with content licenses and rights holders. Apple has spent almost a decade working on this and they're only recently getting to a level of international ubiquity. Whereas Google has been at it for barely a year, but is moving pretty quickly as far as I can tell.
Releasing a slightly superior clone of a feature in a rival product over six months after the first product shipped isn't enormously impressive, although it is more impressive than shipping an inferior clone of a product two years after the first product shipped, so that's definitely progress.
Speed and voice quality are irrelevant to me, but I do really like what he was able to do with the image search on android. It wasn't clear if the android version integrates with wolfram alpha, or any of siri's other sources.
I supposedly have an HSPA+ 21Mbps connection yet only get 1Mbps/.03Kbps maybe 5Mbps/950Kbps Kbps on a good day and latency is 100ms or more, Bell Mobility in Canada. I'll never get 21Mbps but less than 25% at best?
Google and Apple can make fancy tech but it's ruined by poor networks.