Definitely makes some sense. I don't know that this is actually a bad thing though. I personally care more about quality (read: relevant) search results than the details of how they are created. If Google is able to use a social network like Google+ to legitimately improve their search offering so that I have an easier time finding something relevant and interesting using their search engine when I need it, then I'm ok with that.
Of course, this is predicated on the assumption that google is able to effectively use Google + to tweak their main algorithms. I don't want to be barraged with direct G+ links in my search results - I want the relevant websites themselves. Furthermore, there should be a way to opt-out if people are uncomfortable with the lack of privacy, even if the default is their current privacy policy.
The article mentions Google's developers. I don't think Google's developers are worth much now to Google, because they have no hunger. A friend at Google says most of the engineers are just waiting to vest. Look at the poor quality of the speech recognition coming out of Google Voice for an example of the result of engineers simply not caring. To me Google is the new Yahoo!
Speech recognition on new voices transmitted over the phone is a lot harder then something like speech recognition in android or siri. Is there anything that beats google voice at transcribing voicemail? As far as I know there isn't.
Interesting, but I don't think Google wanted to go social to decrease the damages of link spamming.
People already do social network spamming. There are plenty of sites where you can pay for a certain number of +1's or likes.
Using a combination of inputs, social interaction, page links, keywords, they can achieve a better overall ranking algorithm, which is probably one reason they wanted to go social.
However I think the main reason Google wanted to go social was to be able categorise their users better. The more they know about their users, the better they can serve ads.
> People already do social network spamming. There are plenty of sites where you can pay for a certain number of +1's or likes.
Not all +1's are created equal. A +1 from a close friend, or a respected public figure is worth a lot more than ten thousand +1's from accounts created in the past 48 hours from a Bangalore IP address.
But in Google's framework, the only kinds of +1 are "your immediate friends" (those appear with profile pictures alongside) and "all the rest" which appear in the aggregate total "822,251 people +1'd this"
I recently attended a talk at Cornell by a guy who is due to start working at Google soon. He had not started working there, but presumably he was hired on the basis of the work he described, and he was explicitly interested in applying his framework to the Google social network.
He was working on a scheme to get people to recommend things to their friends in return for discounts on those things. His central focus was on devising a payout scheme in which people's economically optimal strategy is honestly reporting the cost of making the recommendation, and then choosing the optimum set of people with whom to make this deal for a given budget. Optimum in the sense of influencing the most people with the recommendations.
You can never fully escape economics, but I think a better way of looking at it is overemphasizing micro-optimization while being stuck in a small local maximum.
This work sounds interesting. Who was the talk by? It sounds like a student of Jon Kleinberg but I quick glance over his publications didn't reveal work on that kind of mechanism design problem.
Is there a reasonable means to prevent people from spamming their friends to get the discount and then saying over some more trusted channel "Please disregard the spam, I haven't tried this thing and don't know whether it is good yet?"
You could reward recommendations that result in more recommendations since (hopefully anyway), people won't recommend spam to others. Then you could encourage good products spreading rapidly. But then there's a delay to the reward and users could lose interest. You could also end up dominated by a few power users.
That's small compared to the problem of arbitrage in such a scheme. Start by spamming people to buy something which gives them the largest dollar value discount on an item they don't actually want, then sell part of the discount to someone who wants it and pocket the profit.
It's occurred to me that the ultimate answer is embodied by the classic Feynman chapter title "You just ask them".
You've got Facebook raising money at over $50 per user based on the promise of futureperfect GMO (ie trying to imitate organic) targeted advertising. Google is so worried, they're gutting their core strength and namesake to become more 'social' by facilitating pop culture, as if it were hard to find. Clearly everyone on the web wishes they were buying things all the time, but just doesn't know it yet.
Meanwhile, Pinterest comes along and stumbles into creating a community where people curate pictures, many of stuff they desire to buy either now or in the future - inherently social status symbols. They skip ads, double-down on their ability to drive sales, and directly monetize affiliate links.
It seems like "Fail fast" has a corollary of "Context is king" - If you end up with all of the cat-picture-posting users in the world, and that's what users view your site as being for, you're never going to have better monetization than generic ads.
Exactly. Advertising works least well when it's spam, slightly better when it's targeted, and best of all when it's a personal endorsement from someone you trust. When Alton Brown tells you to buy Shun knives or when Penny Arcade tells you to play SW:TOR that has a lot more impact than a random ad for wicker furniture or what-have-you.
That's something that google and facebook are working towards but perhaps they are working too indirectly.
But the whole financial sell of Facebook is that they're purporting to work towards something better than targeted advertising. Due to increase in volume of communication between people, targeted advertising is going to be eclipsed by social recommendations. But Pinterest could never have been Facebook/Pin, because half of the second N in Metcalfe's law doesn't want to see dresses and shoes.
Not sure it would be legal, but I would put a significant amount of devs onto predicting stocks. A blog goes more silent or people start Googling for "what is fraud, exactly?" from IPs known to contain lots of IBM workers (that would be more on the illegal end for sure).
Obviously a lawyer would need to review every input, but they have access to machine learning and data so much so that they could rock the exchanges.
At what point does it become illegal though. Surely if Goldman can trade then there must be some aggregation that they are allowed to trade at. "Debt consolidation" search trends for example. If it spikes in a certain country then they would be able to make decisions on that.
I recall a friend of mine who left Google, fully vested, for the early days of Facebook, who said they thought about this but decided insider trading would be evil, so they didn't do it. I'd be really surprised if they had to talk to lawyers to figure that out.
The interesting thing is that Google itself has an internal hedge fund (or something close) that trades its cash hoard. I'm not sure if they do it algorithmically or whether they just buy-and-hold various indices, but my suspicion is that (given their engineering background) they probably do some algorithmic trading.
Using search data to predict public health issues is legal and good PR. Using search data to trade on insider information is illegal and bad PR. That's why we do the flu thing and not the stock thing :)
That's right but how can it be illegal for them to mine their own data?
I see it becoming illegal the moment they start looking at my data (e.g. the contents of a private Google doc). But the index of their public search engine consists of public data to begin with.
Likewise by entering a phrase into their search box I'm telling them "find this for me". Why should it be illegal for google to mine this data just like Walmart mines the customer requests for barbie dolls?
There's obviously a fine line as google juggles with private (e.g. google docs) and public data. But the question as I understood it was about the public portion of their data.
search results (and open stuff that google crawls from sites) is PUBLIC... Using INSIDE information (e.g. if you work at a company, therefore having access to undisclosed info) is one thing, but using what's freely on the net?
That's what ALL investors do anyway (and it's obviously legal)
I would do behavior based marketing that was relevant to individuals and / or segments ... it would be an opt-in but manufacturers would be really interested in providing great offers if they could reach opinion leaders and certain segments to gain incremental business. Would need to be opt in as I say so its not intrusive but really beneficial
This post keys into an idea I had recently to combine Klout with anonymized/aggregated web history data (i.e. from a browser extension) to build a URL quality/trending score. Google could obviously do this with Chrome, but I'd like to see a more open method of distributing this data to allow anyone to build on top of it. In the same vein, I'd like to see an open service that websites could ping when they want a page to be indexed. This would allow for a more open and efficient crawling ecosystem than currently exists. In general, development and ownership of search and social should become more open and distributed just as telephony, electricity, operating systems, and so many more large industries before it.
> In general, development and ownership of search and social should become more open and distributed just as telephony, electricity, operating systems, and so many more large industries before it.
I am working on an effort to do that called VerticalSet. It is a search engine/platform to let developers change search experience the way they want while giving users best search experience.
This article goes along with, "What you could do if you were Apple and had their cash", and other greats like, "What you could do if you were a genie".
I would build the biggest NoSQL datastore in the world. I would use all public data - anything that could be searched and I would invite other companies to put their data on my cloud. THEN we would find a cure for cancer.
Um - no. I was entirely serious about curing cancer. I even did a speech at the NY Tech Council on it - http://www.youtube.com/watch?v=ud1x6VYEc1Q
I believe we can only solve out big problmes by using big data - please do not down vote me
Repliers can't downvote parent comment. Or at least not within my HN user level.
Your comment sounds like generated out of HN headlines mixed with wishful thinking. How could web data "cure cancer"? Are you indirectly calling the huge amounts of cancer researchers idiots for looking in the wrong place? Please think before you post.
>What you could do if you were Google and had their databases?
Try to predict election results and then try to influence them. Not exactly "Do no evil" I'll admit. Imagine what Google PR could do for politicians with access to what people are talking about, watching, browsing and chatting about in their social network. With their Ad technology they're already half way there.
I presume this gives me access to all of their servers too. In which case I would mine the remaining Bitcoin. Mainly because I think Bitcoin is a stupid idea and this would help highlight that sentiment.
> people that relied on links will now rely on your search engine reducing the value of links between sites
Is this really a given? I've never heard this before, not sure what it is that it means. Is the number of links generated by non-SEO sources falling (or not rising proportionally with SEO links)?
The interesting part is not that Google is looking to augment their search algorithms with human curation, see fb, twitter, bit.ly et al., but rather that their principal metric, links, is losing value as a principal metric.
I also believe that this is one of the potential benefit of G+. Though this can also easily be rotten by SEO if they use fake people or mechanical turcs. This may explain the effort to enforce identifying the real person behind an account. Facebook is going for it and google too. And this just to be able to throw bigger shovels of ads to us.
page-rank's design is open to being gamed which has led to an arms race of algorithm tweaks. A social graph has much more powerful signals for figuring out relevant search results.
this isn't a hypothesis. the first iteration is called "search plus your world" and everybody already knows about it.
I think you're missing an important piece in your summary. The original page-rank algorithm was based around the idea of authority. A site might contain content entirely relevant to your query for cancer, but if it has no authority then its probably not what you're looking for.
But this no longer works because the original indicators of authority have died out as the web has become less of a network, since search enables everything to exist in isolation (at one of my work sites we just trimmed the majority of the links from our site).
But if you can determine which people have authority, then you can use their browsing habits in order to allocate authority to websites again. Instead of website having and giving authority, you start giving an authority (per topic) score to users, and then use their behaviour to determine authority in pages.
This last aspect is, I think, what jacques is pointing out, and its not that the social graph gives relevant results (which as you say, everyone knows about), but that it gives a different mechanism for authority (which yields better relevance).
Google, in many cases because of GA, knows a users behavior once they reach a site. I wonder if they take into consideration, how long a user stays on a site they find in search or how active they are at that site, when ranking search results? Wouldn't these metrics be an indicator of quality?
They do say they check if the same user by cookie or logged in is back on Google within a short amount of time. This would mean the site was a poor fit for the query. It is an indicator of the quality of the result for this query and for this user. They never mentioned using Google Analytics, maybe they do too.
With the amount of money at stake with even a 0.1% improvement to search quality, I must assume that every possible correlation against data they can gather is investigated.
the worse "unintended consequence" so far hasn't been the proliferation of crappy links but the disappearance of useful links. anybody who could give you a link knows (1) it's a valuable commodity and (2) it detracts from their rankings.
Paid links wouldn't be such a problem if there were more free links.
Facebook uses their social graph to figure out what sort of needs you have. Maybe Google could try the reverse process; that is figuring out your social graph using their knowledge of what web content you need. Kinda evil though.
This is a very good analysis, and I would like to add some historical information to put it in context.
Originally, link analysis was based on a simple observation: people were maintaining curated lists of bookmarks in their personal webpages, because before search engines it was the only way to remember relevant webpages; I'm sure that most people remember that every personal site had a "my bookmarks" webpage. Therefore, every link counted as a "vote" from that person to a page.
Altavista exploited this by ranking by the number of inlinks. It wasn't long until people figured out that this is very easy to game, so someone came up with the idea of "transitive influence": a site is influent if another influent site votes for it. Algorithms such as PageRank and HITS solved the spam problem.
However, with the evolution of the web, the meaning of links changed. For example, most links are automatically generated by CMSs. Also, there is more and more ephemeral information on the web, and for ephemeral information as soon as you have enough links to it to evaluate its quality, the information is already old and irrelevant.
Luckily, if you are the most used search engine, you have another important popularity signal, which is given by your users: the number of clicks to a page for a given query.
In fact, in today's search engines I would say that the influence of link analysis is smaller and smaller in the overall ranking, and probably done mostly at the domain level rather than for each single page.
However, with the advent of social networks, a large amount of clicks doesn't come from search engines anymore, but from social "shares". Which means that the search engine can not observe them anymore, losing a precious signal of popularity (together with "Like"s and "+1"s).
I'm not sure if this is what Google is after with Google+, but most probably the click and share data on Google+ affects (or will affect) the search rankings.
83 comments
[ 6.1 ms ] story [ 161 ms ] threadOf course, this is predicated on the assumption that google is able to effectively use Google + to tweak their main algorithms. I don't want to be barraged with direct G+ links in my search results - I want the relevant websites themselves. Furthermore, there should be a way to opt-out if people are uncomfortable with the lack of privacy, even if the default is their current privacy policy.
People already do social network spamming. There are plenty of sites where you can pay for a certain number of +1's or likes.
Using a combination of inputs, social interaction, page links, keywords, they can achieve a better overall ranking algorithm, which is probably one reason they wanted to go social.
However I think the main reason Google wanted to go social was to be able categorise their users better. The more they know about their users, the better they can serve ads.
Not all +1's are created equal. A +1 from a close friend, or a respected public figure is worth a lot more than ten thousand +1's from accounts created in the past 48 hours from a Bangalore IP address.
He was working on a scheme to get people to recommend things to their friends in return for discounts on those things. His central focus was on devising a payout scheme in which people's economically optimal strategy is honestly reporting the cost of making the recommendation, and then choosing the optimum set of people with whom to make this deal for a given budget. Optimum in the sense of influencing the most people with the recommendations.
KILL IT WITH FIRE!
http://events.cornell.edu/event/orie_colloquium_yaron_singer...
You've got Facebook raising money at over $50 per user based on the promise of futureperfect GMO (ie trying to imitate organic) targeted advertising. Google is so worried, they're gutting their core strength and namesake to become more 'social' by facilitating pop culture, as if it were hard to find. Clearly everyone on the web wishes they were buying things all the time, but just doesn't know it yet.
Meanwhile, Pinterest comes along and stumbles into creating a community where people curate pictures, many of stuff they desire to buy either now or in the future - inherently social status symbols. They skip ads, double-down on their ability to drive sales, and directly monetize affiliate links.
It seems like "Fail fast" has a corollary of "Context is king" - If you end up with all of the cat-picture-posting users in the world, and that's what users view your site as being for, you're never going to have better monetization than generic ads.
That's something that google and facebook are working towards but perhaps they are working too indirectly.
Obviously a lawyer would need to review every input, but they have access to machine learning and data so much so that they could rock the exchanges.
Seems like so much untapped potential.
http://www.google.org/flutrends/
I see it becoming illegal the moment they start looking at my data (e.g. the contents of a private Google doc). But the index of their public search engine consists of public data to begin with.
Likewise by entering a phrase into their search box I'm telling them "find this for me". Why should it be illegal for google to mine this data just like Walmart mines the customer requests for barbie dolls?
There's obviously a fine line as google juggles with private (e.g. google docs) and public data. But the question as I understood it was about the public portion of their data.
That's what ALL investors do anyway (and it's obviously legal)
http://arxiv.org/abs/1010.3003
I am working on an effort to do that called VerticalSet. It is a search engine/platform to let developers change search experience the way they want while giving users best search experience.
http://www.verticalset.com/
Just saying.
Your comment sounds like generated out of HN headlines mixed with wishful thinking. How could web data "cure cancer"? Are you indirectly calling the huge amounts of cancer researchers idiots for looking in the wrong place? Please think before you post.
Try to predict election results and then try to influence them. Not exactly "Do no evil" I'll admit. Imagine what Google PR could do for politicians with access to what people are talking about, watching, browsing and chatting about in their social network. With their Ad technology they're already half way there.
Depends on who they're backing, right?
But even then, that's subjective. What's more evil, abortion or forced transvaginal ultrasound? Pollution, or unemployment? Kang or Kodos?
No. No following people around on the Internet to make search results "better".
Is this really a given? I've never heard this before, not sure what it is that it means. Is the number of links generated by non-SEO sources falling (or not rising proportionally with SEO links)?
page-rank's design is open to being gamed which has led to an arms race of algorithm tweaks. A social graph has much more powerful signals for figuring out relevant search results.
this isn't a hypothesis. the first iteration is called "search plus your world" and everybody already knows about it.
But this no longer works because the original indicators of authority have died out as the web has become less of a network, since search enables everything to exist in isolation (at one of my work sites we just trimmed the majority of the links from our site).
But if you can determine which people have authority, then you can use their browsing habits in order to allocate authority to websites again. Instead of website having and giving authority, you start giving an authority (per topic) score to users, and then use their behaviour to determine authority in pages.
This last aspect is, I think, what jacques is pointing out, and its not that the social graph gives relevant results (which as you say, everyone knows about), but that it gives a different mechanism for authority (which yields better relevance).
http://searchengineland.com/seotable
Paid links wouldn't be such a problem if there were more free links.
Originally, link analysis was based on a simple observation: people were maintaining curated lists of bookmarks in their personal webpages, because before search engines it was the only way to remember relevant webpages; I'm sure that most people remember that every personal site had a "my bookmarks" webpage. Therefore, every link counted as a "vote" from that person to a page.
Altavista exploited this by ranking by the number of inlinks. It wasn't long until people figured out that this is very easy to game, so someone came up with the idea of "transitive influence": a site is influent if another influent site votes for it. Algorithms such as PageRank and HITS solved the spam problem.
However, with the evolution of the web, the meaning of links changed. For example, most links are automatically generated by CMSs. Also, there is more and more ephemeral information on the web, and for ephemeral information as soon as you have enough links to it to evaluate its quality, the information is already old and irrelevant.
Luckily, if you are the most used search engine, you have another important popularity signal, which is given by your users: the number of clicks to a page for a given query.
In fact, in today's search engines I would say that the influence of link analysis is smaller and smaller in the overall ranking, and probably done mostly at the domain level rather than for each single page.
However, with the advent of social networks, a large amount of clicks doesn't come from search engines anymore, but from social "shares". Which means that the search engine can not observe them anymore, losing a precious signal of popularity (together with "Like"s and "+1"s).
I'm not sure if this is what Google is after with Google+, but most probably the click and share data on Google+ affects (or will affect) the search rankings.