The notion that being 'committed' to open source means that a company should give away everything of value is ludicrous. Google came up with great software that makes millions of peoples' lives easier, and they have every right to keep it secret and profit off of it.
The zealous idea that open source and trade secrets cannot coexist is detrimental to the open source movement as a whole, in my opinion.
High taxation eventually reaches communism, if there's benefits like healthcare 'refunded' to the citizen and if there isn't then that ends up being a kleptocracy. Low taxation eventually reaches anarchy as the government is no longer in control.
Increasing birth rates enables economies to grow without becoming stagnant and old, placing undue demand due to pensions. An exceptionally high birth rate creates food production problems, housing problems, and generally the 'sardine can' effect. Low birthrates create a stagnant civilization that will eventually reach a tipping point when taxes get so high to pay for pensions that the younger generations will protest.
Market freedom to the extreme leads to monopolies taking advantage, 'gentlemen's agreements' between companies to price fix and many other problems that rules and regulations try to control. Equally a controlled market develops problems in and of itself as it's unable to respond naturally to market fluctuations, and quickly becomes unruly and unstable, and as many of the countries who have held long-term controlled markets are testaments to, they develop massive black-market economies to deal with these.
The zealous burning of fossil fuels, aside from global warming concerns, can expel vast quantities of radioactive materials, sulfurous emissions and a long list of things we've accepted in western society as being more harmful than smoking to the wide populations health, but rules and regulations on air quality increasingly get more lax (at least they have been where I live). With zero fossil fuel production, however, our civilization would collapse (at least with an immediate stop to production), and I'm not just talking the halting of transportation vehicles, I'm talking the stop in production of the advanced polymers used in virtually all aspects of our lives. The majority of people wouldn't have clothes on their back . . . my wardrobe would certainly be a lot cheaper if my clothes were all genuine cotton and wool. Oh and farewell cheap elasticated sweatpants.
Every good idea taken to its extreme is a bad idea and will create monumental problems in its wake. Electric vehicles are amazing, however a pure-electric is a ridiculous prospect in winter when the capacity hits below 60% of room-temperature, suddenly your vehicle can't get you to work and back on a single charge. Zero integration of hybrids is, as everyone knows, damaging to the environment but also an extreme waste of entropy. A 1 tonne vehicle at 100km/h is a massive store of energy, and allowing that to be wasted is criminal, the amount of energy humanity has wasted on braking at highway speeds would likely run the entire human civilization for over a year.
In it a Google open source zealot rants wildly about openess being ultimate, apart from search algorithms which should be closed. It also fails to mention all the times open has failed (communism being one of the more spectacular failures of open).
A cynic might surmise what they actually mean is content should be open so they can profit from their closed search algorithm.
> communism being one of the more spectacular failures of open
Wait, what? Communism is a spectacular failure of centralized planning over the entire economy. It has nothing to do with open or closed in principle, and in every implementation it has been aggressively closed in terms of information flows and decision making.
> Communism is a spectacular failure of centralized planning over the entire economy.
Wow, and here I was thinking it was a form of government.
Communism in principle has absolutely nothing to do with the flow of information and/or decision making, and its implementations have only very little to do with its basic ideas.
A true communistic government has never even been tried, we've only seen a 'new boss same as the old boss' approach to communism with one batch of fat cats replacing another.
I believe that Communism is largely an economic system with politics being aside effect of the economic model. For example, the opposite of Communism isn't democracy, it's capitalism. And actually, in the purest sense, Communism is completely about decision making. According to that bastion of scholarly thought Wikipedia:
"Pure communism" in the Marxian sense refers to a classless, stateless and
oppression-free society where decisions on what to produce and what policies
to pursue are made democratically, allowing every member of society to
participate in the decision-making process in both the political and economic
spheres of life.
But that's one of those theory and practice things... great in theory, horrible in practice.
>Wow, and here I was thinking it was a form of government.
Like most ideologies, the term "communism" is gooshy enough to mean whatever you want it to mean - but at heart, communism is concerned with the matter of who owns and controls the means of production. It's a form of government insofar as the decision on who gets to control the means of production is ultimately a political decision.
Communists believe the people should own and control the means of production - but every real-world implementation has used government as a proxy for people.
I suppose it's possible to run a communist society via direct democracy on a very small, local scale; but good luck trying to scale that up to the national or even regional level. At some level you need to adopt some kind of more or less representative government. In the case of communism, the track record is definitely skewed toward the "less" end of the representation spectrum.
In other words, your claim:
>A true communistic government has never even been tried...
is a straightforward No True Scotsman fallacy. It's no different from "real" conservatives distancing themselves from the Bush administration because the implementation of their political system ended up violating most of its ostensible core values.
If you can't implement an ideology without discarding the very principles that define the ideology, it's a broken ideology.
"It's no different from "real" conservatives distancing themselves from the Bush administration because the implementation of their political system ended up violating most of its ostensible core values."
To define American-style conservative and liberal philosophy as what the Republican and Democratic parties and their politicians do is ridiculously simplistic. The parties are just coalitions of interests and voter demographics, not philosophies - you might as well talk about what direction a plate of spaghetti points at.
And as for conservatives' "political system" - whose? Do you mean evangelicals and other theocrats, PNAC neocons, populist teabaggers, or Lew Rockwell/Ron Paul fans? We're not talking remotely the degree of unity imposed by the Politburo and Comintern upon the global communist movement when it comes to either US party. In a bipolar system, both parties must be big tents or be crushed by the bigger tent. Elected officials are not (and Bush was not) advancing a single "conservative" or "liberal" ideology; they're trying to please various power blocs who agree or disagree with them on different issues.
Bush's failures highlight the failure of particular policies and ideas, but they don't say much about the ideas of self-described "real conservatives" who thought Bush was a ridiculously free-spending, anti-federal, protectionist, power-consolidating, Big Government guy long before Iraq. (No more than Obama's decisions indict the ideas of supporters who disagree with him on those decisions, certainly.)
Well, superficially it sounds like pure economic democracy - anyone who has a good idea that would benefit people proposes it, and resources are allocated to them by consensus. That's the seductive part, but it takes no account of the variations in intelligence, foresight and altruism, which is why it only works in Star Trek.
I don't think communism is evil so much as fundamentally naive, its proponents assume the mass of people think alike and that selfishness is a learned behavior imposed from above by wicked capitalists. It lasted as long as it did in the USSR because both military and scientific folk are into the idea of setting aside individual gain for a greater purpose, such as patriotism or the advancement of knowledge.
Communism is bad if you look at it from the perspective of a democracy with a functioning market. Then again, that perspective is a luxury if you have the misfortune to be stuck in a feudal society - most places where it took root were extremely dysfunctional monarchies prior to turning communist. Aristcracies aren't what we'd call ideal societies either; even here in the west, the replacement of aristocracy by parliamentary government was viewed by traditionalists as a form of godless mob rule.
And it's not that long ago that people were complaining that GPL represented a kind of license tyranny which would undermine proprietary innovation.
I don't think your answer makes much sense. Historically, communism had to do with allocation of resources by the state rather than a market, in the service of certain ideological beliefs. 'Openness' was not one of them.
It was written by Jonathan Rosenberg, who's a Senior Vice President for Product Management, and on the top management committee at Google. That doesn't quite meet my personal description of an "open source zealot", though he's clearly quite passionate about it.
Thanks for the link, that helps put the post into a bit more context.
That said, I think the author's point is still wrong. As a community, we have to understand that these open systems are an enabler of progress, some of which will undoubtedly be proprietary. Part of openness is understanding that you're not locking people into using this stuff, and they're not locked in to how they use it.
I don't see it as hypocritical that a business can talk about being open and how critical it is to give back, while still keeping trade secrets to pay the bills.
I recently saw the CEO of the Economist group speak about journalism, and one thing he said (paraphrasing) was that the best defense of journalistic impartiality in the modern age is to turn a healthy (and diversified) profit. The subtext was that you can then risk offending any given party by doing good reporting without the appearance of favoritism, and without having a corporate (or nonprofit) master to please.
In the same sense I think that the best defense of free software is to allow companies to use it on their own terms, and give back when they can/will. The most useful open source projects are used (and likely abused) ad-nauseum, and trying to limit any company, even one that preaches hypocritically about open source is a recipe for them to tell the community to shove it and go hand money to <insert big vendor here>.
While it's fairly easy to preach the open-source gospel to fellow programmers and nerds that appreciate the value of this stuff, it's business people and non-tech-literates that oftentimes allow the checks to be signed, and the open source community needs to look to them like a group of enablers, not inflexible idealogues.
Additionally it may be the case that a large part of the value of the algorithm is created from its secrecy. If the algorithm was open it would be trivial to create sites with artificially high rankings which would reduce the value of the search results.
> If the algorithm was open it would be trivial to create sites with artificially high rankings
I don't necessarily think this is true. Sites that have high page rank, do so because other sites of high value link to them.
Even if Google open sourced their search engine, spam sites would still have a hard time getting high quality inbound links. The reputable and high ranking sites wouldn't have any increased incentive to link to low quality spam sites.
It's not the ranking algorithm specifically that's a secret; it's the code that used to detect sites that work together to artificially raise page rank of another site by linking to it. The whole Page Rank algorithm is predicated on the idea that human website owners and contributors do a better job at assessing the value of a website and they express this by linking to the website. When links are created for other reasons they reduce the effectiveness of Page Rank, and when they are created specifically to bias Page Rank they do the most damage to the algorithm's results.
Keeping the algorithm secret, and in particular the tweaks that eliminate links which aren't expressing "I judge this to be a good site", keep Page Rank useful.
Absolutely. I've worked in several contracted roles within online marketing and I've heard some awful advice from self-proclaimed SEO gurus about how to achieve a high rank for keywords on Google. A lot of what achiving high ranks is about is common sense and as Google improves their algorithm fewer tricks work. Anything past a simple competitive analysis, keyphrase analysis and a backlink analysis on you and your competitors is overkill.
The quality of SEO analysts out there is so poor that I believe you could release the source code to Google's entire search engine and people would still get things wrong or believe in ways to get around what I assume is an extremely complex set of mathematically-oriented algorithms. It's a classic example of the blind following the blind, and those that typically have followers are those that are popular for performing SEO work on huge companies that would never actually need it (e.g. Coca Cola).
After reading the comments at cdixson.org by the author and reading openness article at Google, it seems like author is simply saying: "I agree they won't open it. But then they should stop talking sactimoniously about openness." which I agree with as well.
Here is an interesting perspective: "And millions of websites will have to continue blindly relying on a small group of anonymous engineers in charge of the secret algorithm that determines their fate." Have you ever been delisted by Google? Is there a transparent process to get back on the list? Perhaps transparency is what Google needs the most at the moment.
> Perhaps transparency is what Google needs the most at the moment.
Absolutely, and not only on the search engine part of it, Adwords/Adsense are extremely opaque and as a publisher continuously give you the feeling you're being screwed without any chance to prove that for lack of information.
As a publisher, couldn't you do some sort of A/B testing with another ad provider? Something like 10% of page views using another set of ads? Or, maybe setup some kind of onclick tracking to verify ad clicks?
I don't know what could be done as verification for the advertiser, but the publisher should be able to do something, right?
You say that now but if they provided that transparency then the amount of fraudulent activity would skyrocket. Knowing the details of the algorithms involved does nothing to help you as an advertiser or searcher. Googles extreme impartiality in search and ads is their greatest strength and open sourcing the algorithms involved would do nothing to increase that strength and contribute to a great deal less useful ads and search results on average.
> Googles extreme impartiality in search and ads is their greatest strength
Without proof that is nothing but handwaving, if they are as impartial as you suggest they are they've got nothing to lose by letting you know what their take is and how big a portion of their clicks are confirmed fraud.
The notion that being 'committed' to open source means that a company should give away everything of value is ludicrous.
I disagree with the statement that Google would be giving away everything by opening up page rank. Google's "secret sauce" is hardly page rank alone. Google is really secretive about this stuff, but they arguably have the world's larges corpus of multilingual text in the form of their html index, coupled with the world's largest fault tolerant compute cluster, the largest repository of scanned text,the largest repository of digital video, and probably the largest repository of digital audio in the world. I'm sure that they also have the world's largest repository of people's surfing and searching habits, along with the one of the largest sets of analytics data.
Even if page rank, map reduce, GFS, etc.. were open sourced I'm sure they'd still find a way to make billions off of the data that they have.
But, I think the larger point that the author makes, is that unlike many giant technology companies (Microsoft, HP, Apple, etc..), Google has made billions off of using open source software. And, while they are a very large supporter of open source, what total percentage of their profits go back to OSS? What percentage of their code gets re-released as open source? These are good questions to ask, and I'd love to hear an open discussion about it.
The analogy to "security through obscurity" is flawed. Open security works when there's an obvious way a secure system should behave, say, make it infeasible to open a lock/decrypt data without the correct key.
In Google’s, case the correct relevancy ordering of the web is a huge point of contention. So they cannot simply open it up hoping to close out all the holes in the open. It is also hard to distinguish acceptable behavior (linking) from unacceptable behavior (selling links) except in the aggregate through heuristics that assume ignorance on the part of the attacker.
In this sense, Google’s system is a lot closer to an intrusion detection system than a lock or a cipher suite. You want to establish some ground rules but there’s no added security in declaring what you’re monitoring on your servers and how.
That is a completely ridiculous point to make. Come on, be serious here. What would be the benefit to anyone except for spammers, if Google open-sourced their algorithms? It's a lose-lose option. Google loses some money, we lose a lot of search engine accuracy, and spammers gain new ways to screw around with search results.
And does anyone gain from it? Not really. At this point, Google's supremacy is due just as much to their hardware know-how, to their market leader position, and to their excellent brand, as it is to their search algorithms. It won't help any competing search engines - just spammers.
The stated goal isn't to help other search engines; it's to get the community to help Google improve, for the good of all its users.
The theory goes: if you let more people look at the algorithm, they can suggest better answers than Google alone, to thwart the spammers who are gaming the system either way.
But I suspect the real goal has more to do with the deep suspicion that Google can or does do a significant bit of hand-tweaking. And these people want to force the algorithm into the open to either end or curtail that.
I'll believe that when I see open source achieve a conclusive victory against email spam.
Spam:email = 90% or worse
in search results it is closer to the opposite.
I'm sure once the open source community actually manages to do this and prove the 'many eyes' doctrine applies to more than just security and bug fixes that many doors will open.
Google's search ranking algorithm is their bread and butter so, I don't think they will ever open source it. But what always strikes me is that they have not yet open sourced their BigTable database systems even though, it's not their core revenue generating source. There are clones of BigTable in the form of Cassandra and HyperTable but it would be so much better if BigTable is open sourced.
An interesting thing to think about even though the basic reaction is "wait, no, that's their industry secret/revenue basis/whatever". Google is so dominant right now that maybe it's going to get to the point where, in the interests of the open web, they get pressured and lobbied to release their secret sauce; after all, as swombat points out, it's not just the algorithm by a long chalk.
It's especially interesting if you consider the origins of PageRank in academia, freely published and peer-reviewed, and still not that well understood by laypeople; would the open-sourced magic of Google even be understandable by that many folk? Those that do understand it could make a fortune as consultants; at the moment, shady SEO snake-oil salesmen are so prevalent and 99% of their advice is outdated or plain wrong, something that definitively calls them out on it could be helpful.
I would love if they open-sourced their infrastructure tools such as GFS, BigTable and Map-Reduce tools. Most of these tools are built using open-source technology such as Linux.
I'd be interested to see some benchmarks (unofficial-probably) between Hadoop and Google's C++ versions (at least, I assume Google's is C++). I've always wondered if there was any extra overhead in using Hadoop. I know Java is almost if not faster than C++ in some areas, and long running servers (like Hadoop) should benefit the most. But I wonder if there are some extra limitations Java places on things like BigTable.
The algorithm used by Google today is light-years away from that described in the PageRank paper. It uses a form of PageRank, but it also uses many other parameters.
My impression is that PageRank, while initially extremely successful at filtering signal-from-noise, is very susceptible to manipulation. For example links farms which make unimportant pages seem important. As a result of such manipulation, Google started modifying their crawling, in order to reduce the amount of garbage going into PageRank (less garbage in, less garbage out). For example, they might look at the age and ownership in DNS records, and exclude domains that look fishy based on some heuristics. The spammers fought back with more tricks, and the resulting arms race has come to have an acronym: SEO. But the underlying idea of using an iterative approximation of the likelihood of encountering a page via a random walk of the web is preserved. (Some details here: http://www.miislita.com/information-retrieval-tutorial/matri...)
If that's right (and I could well be wrong... would like to be convinced...), then I don't see it as a "light-years away" kind of issue, but rather a series of hacks to stay one step ahead of the spammers. I don't see any particular value in open sourcing those.
Google is a business. Their job, legally defined, is to create value (aka profit) for its shareholders. A more open web has helped Google, but opening up its search algorithms, a cornerstone to Google's growth, is just ludicrous and bad business.
Compared to a lot of other companies (Apple, Microsoft, nearly every other tech giant), they are an open company. It's already great that they realize there's value in an Open-source browser, and open-source OS, and multiple open platforms.
Their job, legally defined, is to create value (aka profit) for its shareholders.
This is a meme I see a lot, but never with any evidence. That the board has a fiduciary duty to the shareholders I can understand. That it would have a legal obligation to create value, well, I think that would be hard to define, and I have never seen any evidence for the existence of this obligation.
In the context of a publicly traded company this is the definition of fiduciary. The company has sold stock to investors with the sole purpose of increasing the value of the stock. Therefore, it is up to the board to make decisions that reflect this duty.
You can argue about the timeline (short versus long-term growth), but you can't say that there isn't a legal obligation to attempt to increase value.
You're right... I should have said something like "increasing shareholder value though the increase in value of the stock or profit distribution via dividends".
I actually think that more companies should give dividends.
I can see why companies like to have billion dollar war chests, but at some point you have to wonder when enough is enough. I mean, even Microsoft was forced to start paying dividends. Then again, their stock price has been pretty flat for the past few years, so that might be the only reason to hold it.
It may be true that it is a company's legal responsibility to serve its shareholders first, but that's not the way it should be. With this mentality, corporations will minimally follow the law (if the fine is less than the profits, the corporation may be legally required to break the law!) and do whatever they can in their own interests. Society as a whole would benefit massively if caring for others and society in general was regarded as a good thing.
http://www.google.com/support/webmasters/ easily brings down this bogus claim. I honestly don't think much would change even if they did open-source their PageRank algorithm because the people willing and able enough to exploit it already do, and regardless of how much their PageRank increases Google will send your domain deep into an internet abyss and out of their index.
(I used to work in the Search Quality division at Google.)
Security by obscurity is a bad analogy here. The problem is that Google's algorithms are not mathematically perfect, in the sense of cryptography. Instead, they merely statistically tend to find good results. You can't find bugs just by looking at source code. You have to statistically analyze the effects of any proposed changes.
Welcome to: http://www.wowowbiz.com
The website wholesale for many kinds of fashion shoes,
like the nike,jordan,prada,adidas, also including the
eans,shirts,bags,hat and the decorations. All the
products are free shipping, and the the price is
competitive, and also can accept online payment.,after the
payment, can ship within short time.
http://www.wowowbiz.com
our price:
jordan air max oakland raiders $34--39;
Ed Hardy AF JUICY POLO Bikini $25;
Christan Audigier BIKINI JACKET $25;
gstar coogi evisu true jeans $35;
coach chanel gucci LV handbags $36;
coogi DG edhardy gucci t-shirts $18;
CA edhardy vests.paul smith shoes $32;
jordan dunk af1 max gucci shoes $37;
EDhardy gucci ny New Era cap $16;
coach okely Adidas CHANEL DG Sunglass $18;
http://www.wowowbiz.com
thank you for your visit Wish you a merry Christmas
Welcome to: http://www.wowowbiz.com
The website wholesale for many kinds of fashion shoes,
like the nike,jordan,prada,adidas, also including the
eans,shirts,bags,hat and the decorations. All the
products are free shipping, and the the price is
competitive, and also can accept online payment.,after the
payment, can ship within short time.
http://www.wowowbiz.com
our price:
jordan air max oakland raiders $34--39;
Ed Hardy AF JUICY POLO Bikini $25;
Christan Audigier BIKINI JACKET $25;
gstar coogi evisu true jeans $35;
coach chanel gucci LV handbags $36;
coogi DG edhardy gucci t-shirts $18;
CA edhardy vests.paul smith shoes $32;
jordan dunk af1 max gucci shoes $37;
EDhardy gucci ny New Era cap $16;
coach okely Adidas CHANEL DG Sunglass $18;
http://www.wowowbiz.com
thank you for your visit Wish you a merry Christmas
I upvoted the story not because I agree with it, but because it explains a very successful open sourcing strategy that I wish more people understood.
In general people are willing to spend a certain amount on a solution. Therefore if you can things that make the complement to what you make money on cheaper, that increases how much you can charge for your piece of the solution. Open sourcing is a way to make those pieces cheaper.
Google does this. They give away tools and knowledge about how to make websites. They give away analytics and an A/B testing platform and educate people about how that makes it easier to produce good websites. This makes building good websites cheaper, and therefore increases how much people are willing to spend to advertise those websites.
This isn't a new strategy. In the late 90s when Oracle and IBM got into open source, this was their strategy. Oracle wanted to get people to stop paying for Sun hardware so Oracle could negotiate higher prices. (I know more than a few people who were surprised to find that licenses for Oracle on Linux are higher than they are on Sun.) IBM was looking to reduce the price for complements to IBM websphere, and was looking for opportunities to avoid having customers pay the Microsoft tax.
If you run a company and wish to strategically open source something, you should think in the same terms. What is the complement to what you are charging for, and how can you make that cheaper?
And what is wrong with commoditizing the complement? Google's working hard to make operating systems free and we'll probably see Microsoft working hard to open up search engines. End result: both operating systems and search engines will be open and the consumer benefits.
Well, I can't understand this article, there is not "secret sauce". The google pagerank algorithm is open for all to see and understand. I myself made my own simple implementation.
Everybody could do it. This man is not asking for code, is asking for DATA only google has. It's like an atheist saying to a catholic what he must do because he believes. When this man gives something of value to the world,like google did(I remember the Altavista-Ad-flickering in your eyes days) then he could demand to be given by others.
I am disturbed by the lack of faith in open source among the readers here. I mean, I understand some skepticism but this, to me, is pretty extreme IP-fetishism.
It's perfectly understandable. Lots of money is made in the existing paradigm and so many of you have incentive to believe it's the most rational system. Besides, hard to imagine Google's doing something wrong, right?
But the argument that "the only people this will help is spammers"? Really? Are the only people helped by open cryptographic protocols those who look to decipher traffic?
"They have every right to keep it secret and profit off of it." Let's allow this for a moment. We've still got a false dichotomy. Making it public will not deprive them of their profit. First, incredible marketing coup. Does anyone else remember how much of Google's initial strength came from its rabid fans? This would do a lot of re-energize that base.
Second, are there going to be new startups which can effectively compete with Google, even given its algorithm? How are they going to provide a better experience, or serve the volume of traffic, or index a larger amount of pages with the same algorithm with fewer resources?
Third, who says they have to authorize competitors to use it? This is I think one of the more interesting points. They could release the algorithm but license it only for non-commercial use for anyone not themselves. Ridiculous? Because Microsoft would violate such a deal anyhow? Because the algorithm could simply be reimplemented? Perhaps. But it would be another roadblock.
Fourth, why do we continue to believe that Google's strength comes from proprietary code? Why do we not recognize that its strength comes from mindshare, user experience, and quality of execution? None of which would be negatively affected by releasing the source.
Perhaps Google would face heightened competition as a result of releasing its algorithm. Perhaps this would outweigh the benefits of doing so. But there would be significant good which would result, including public feedback which would make it significantly harder for spammers to be successful.
But I'm just a foaming-at-the-mouth open source radical. And I'm sure history is on your collective side. If there's anything we've learnt from a hundred years or so of Computer Science (overestimate if you only count actual programming, underestimate if you allow Babbage as you should), it's surely that secrecy is what drives innovation, right?
Let me elaborate. Not even Stallman would call for this... What can of totalitarian regime is desired, where businesses (even individuals?) are forced to publish all of their code? Wouldn't it be good enough to do away with patents and copyright? As long as I manage to keep my stuff private, let me (and Google) face the advantages and disadvantages.
This post manages to completely miss the point of the Google letter on openness.
As some have incorrectly asserted, Google's latest post is not about "everything open", it is an internal memo to product managers that it occurred to them would be fit for public consumption.
The central thesis of the letter is that "open" is hard to define (and defined differently by different people) and yet crucially important. Because it is hard to define, many people misuse it (OpenDNS comes to mind), and no one can call them on it because it is a vague term.
Internally, Google employees argue about what open means, and the purpose of this letter was to clarify what open means to Google. They lay it out in extremely clear terms, and it is well structured.
This response essay by Dixon misses all that completely - he unilaterally defines "open" to mean "open source", and then asks for Google to give away their software, claiming that if they don't, they're hypocrites.
Going back to Google's definition of "open" (which is complex), one significant aspect they emphasize is the notion of "lock-in", i.e. locking in users, and locking out competitors. This is one area where Google puts their money where their mouth is; they let users leave. You can get at your email, export your Google Reader content, download all your Google Docs, archive your Picasa pictures, and export all your contacts. That means if a competitor comes along with something better, you can start using it. Try that with Facebook.
Dixon acting like that is worthless because it isn't the "open-source" form of "open" is patently absurd. If this is confusing, I highly recommend reading Simon Phipps essays on open source to understand what kind of software makes a good candidate for a business to open source (operating systems, browsers, compilers...the basics that we all use to compete) and what does not (grid/cloud computing algorithms, specialized software, and other types of software that are differentiators). Essentially, the business model is to pool our collective resources in areas in which we don't want to compete (web server software, operating systems) and to spend time innovating in areas we do wish to compete (search algorithms, voice recognition, image analysis, etc.) Over time, as those specialized areas become more mainstream, they become better candidates for open-sourcing.
I'm an open source advocate (even for search algorithms in general), but ranking algo/model is inherently adversarial and gameable until true search AI (always correctly identify good and original content without false positives) is feasible.
I think it would be a good revenue strategy, if they charge $dollars to display top 10 positive features and top 10 negative features of any given query-document pair without any real weights/scores (maybe re-squashed with some logistic function). The $dollars amount is computed dynamically to offset the negative impact of the partial reveal of the ranking algo/model. This can be a tremendously useful tool for troubleshooting search results by site owners.
A more reasonable proposal would be for someone to pull a netflix; $1 million or even a $1o million prize for a significant advance in search relevancy. $10 million would be cheap for MS, Yahoo, ASK, a VC, or even DARPA.
69 comments
[ 6.5 ms ] story [ 79.0 ms ] threadThe zealous idea that open source and trade secrets cannot coexist is detrimental to the open source movement as a whole, in my opinion.
Increasing birth rates enables economies to grow without becoming stagnant and old, placing undue demand due to pensions. An exceptionally high birth rate creates food production problems, housing problems, and generally the 'sardine can' effect. Low birthrates create a stagnant civilization that will eventually reach a tipping point when taxes get so high to pay for pensions that the younger generations will protest.
Market freedom to the extreme leads to monopolies taking advantage, 'gentlemen's agreements' between companies to price fix and many other problems that rules and regulations try to control. Equally a controlled market develops problems in and of itself as it's unable to respond naturally to market fluctuations, and quickly becomes unruly and unstable, and as many of the countries who have held long-term controlled markets are testaments to, they develop massive black-market economies to deal with these.
The zealous burning of fossil fuels, aside from global warming concerns, can expel vast quantities of radioactive materials, sulfurous emissions and a long list of things we've accepted in western society as being more harmful than smoking to the wide populations health, but rules and regulations on air quality increasingly get more lax (at least they have been where I live). With zero fossil fuel production, however, our civilization would collapse (at least with an immediate stop to production), and I'm not just talking the halting of transportation vehicles, I'm talking the stop in production of the advanced polymers used in virtually all aspects of our lives. The majority of people wouldn't have clothes on their back . . . my wardrobe would certainly be a lot cheaper if my clothes were all genuine cotton and wool. Oh and farewell cheap elasticated sweatpants.
Every good idea taken to its extreme is a bad idea and will create monumental problems in its wake. Electric vehicles are amazing, however a pure-electric is a ridiculous prospect in winter when the capacity hits below 60% of room-temperature, suddenly your vehicle can't get you to work and back on a single charge. Zero integration of hybrids is, as everyone knows, damaging to the environment but also an extreme waste of entropy. A 1 tonne vehicle at 100km/h is a massive store of energy, and allowing that to be wasted is criminal, the amount of energy humanity has wasted on braking at highway speeds would likely run the entire human civilization for over a year.
http://googleblog.blogspot.com/2009/12/meaning-of-open.html (submitted here: http://news.ycombinator.com/item?id=1008908)
In it a Google open source zealot rants wildly about openess being ultimate, apart from search algorithms which should be closed. It also fails to mention all the times open has failed (communism being one of the more spectacular failures of open).
A cynic might surmise what they actually mean is content should be open so they can profit from their closed search algorithm.
Wait, what? Communism is a spectacular failure of centralized planning over the entire economy. It has nothing to do with open or closed in principle, and in every implementation it has been aggressively closed in terms of information flows and decision making.
Wow, and here I was thinking it was a form of government.
Communism in principle has absolutely nothing to do with the flow of information and/or decision making, and its implementations have only very little to do with its basic ideas.
A true communistic government has never even been tried, we've only seen a 'new boss same as the old boss' approach to communism with one batch of fat cats replacing another.
Like most ideologies, the term "communism" is gooshy enough to mean whatever you want it to mean - but at heart, communism is concerned with the matter of who owns and controls the means of production. It's a form of government insofar as the decision on who gets to control the means of production is ultimately a political decision.
Communists believe the people should own and control the means of production - but every real-world implementation has used government as a proxy for people.
I suppose it's possible to run a communist society via direct democracy on a very small, local scale; but good luck trying to scale that up to the national or even regional level. At some level you need to adopt some kind of more or less representative government. In the case of communism, the track record is definitely skewed toward the "less" end of the representation spectrum.
In other words, your claim:
>A true communistic government has never even been tried...
is a straightforward No True Scotsman fallacy. It's no different from "real" conservatives distancing themselves from the Bush administration because the implementation of their political system ended up violating most of its ostensible core values.
If you can't implement an ideology without discarding the very principles that define the ideology, it's a broken ideology.
To define American-style conservative and liberal philosophy as what the Republican and Democratic parties and their politicians do is ridiculously simplistic. The parties are just coalitions of interests and voter demographics, not philosophies - you might as well talk about what direction a plate of spaghetti points at.
And as for conservatives' "political system" - whose? Do you mean evangelicals and other theocrats, PNAC neocons, populist teabaggers, or Lew Rockwell/Ron Paul fans? We're not talking remotely the degree of unity imposed by the Politburo and Comintern upon the global communist movement when it comes to either US party. In a bipolar system, both parties must be big tents or be crushed by the bigger tent. Elected officials are not (and Bush was not) advancing a single "conservative" or "liberal" ideology; they're trying to please various power blocs who agree or disagree with them on different issues.
Bush's failures highlight the failure of particular policies and ideas, but they don't say much about the ideas of self-described "real conservatives" who thought Bush was a ridiculously free-spending, anti-federal, protectionist, power-consolidating, Big Government guy long before Iraq. (No more than Obama's decisions indict the ideas of supporters who disagree with him on those decisions, certainly.)
I don't think communism is evil so much as fundamentally naive, its proponents assume the mass of people think alike and that selfishness is a learned behavior imposed from above by wicked capitalists. It lasted as long as it did in the USSR because both military and scientific folk are into the idea of setting aside individual gain for a greater purpose, such as patriotism or the advancement of knowledge.
Communism is bad if you look at it from the perspective of a democracy with a functioning market. Then again, that perspective is a luxury if you have the misfortune to be stuck in a feudal society - most places where it took root were extremely dysfunctional monarchies prior to turning communist. Aristcracies aren't what we'd call ideal societies either; even here in the west, the replacement of aristocracy by parliamentary government was viewed by traditionalists as a form of godless mob rule.
And it's not that long ago that people were complaining that GPL represented a kind of license tyranny which would undermine proprietary innovation.
That said, I think the author's point is still wrong. As a community, we have to understand that these open systems are an enabler of progress, some of which will undoubtedly be proprietary. Part of openness is understanding that you're not locking people into using this stuff, and they're not locked in to how they use it.
I don't see it as hypocritical that a business can talk about being open and how critical it is to give back, while still keeping trade secrets to pay the bills.
I recently saw the CEO of the Economist group speak about journalism, and one thing he said (paraphrasing) was that the best defense of journalistic impartiality in the modern age is to turn a healthy (and diversified) profit. The subtext was that you can then risk offending any given party by doing good reporting without the appearance of favoritism, and without having a corporate (or nonprofit) master to please.
In the same sense I think that the best defense of free software is to allow companies to use it on their own terms, and give back when they can/will. The most useful open source projects are used (and likely abused) ad-nauseum, and trying to limit any company, even one that preaches hypocritically about open source is a recipe for them to tell the community to shove it and go hand money to <insert big vendor here>.
While it's fairly easy to preach the open-source gospel to fellow programmers and nerds that appreciate the value of this stuff, it's business people and non-tech-literates that oftentimes allow the checks to be signed, and the open source community needs to look to them like a group of enablers, not inflexible idealogues.
I don't necessarily think this is true. Sites that have high page rank, do so because other sites of high value link to them.
Even if Google open sourced their search engine, spam sites would still have a hard time getting high quality inbound links. The reputable and high ranking sites wouldn't have any increased incentive to link to low quality spam sites.
Keeping the algorithm secret, and in particular the tweaks that eliminate links which aren't expressing "I judge this to be a good site", keep Page Rank useful.
The quality of SEO analysts out there is so poor that I believe you could release the source code to Google's entire search engine and people would still get things wrong or believe in ways to get around what I assume is an extremely complex set of mathematically-oriented algorithms. It's a classic example of the blind following the blind, and those that typically have followers are those that are popular for performing SEO work on huge companies that would never actually need it (e.g. Coca Cola).
After reading the comments at cdixson.org by the author and reading openness article at Google, it seems like author is simply saying: "I agree they won't open it. But then they should stop talking sactimoniously about openness." which I agree with as well.
Here is an interesting perspective: "And millions of websites will have to continue blindly relying on a small group of anonymous engineers in charge of the secret algorithm that determines their fate." Have you ever been delisted by Google? Is there a transparent process to get back on the list? Perhaps transparency is what Google needs the most at the moment.
Absolutely, and not only on the search engine part of it, Adwords/Adsense are extremely opaque and as a publisher continuously give you the feeling you're being screwed without any chance to prove that for lack of information.
I don't know what could be done as verification for the advertiser, but the publisher should be able to do something, right?
Without proof that is nothing but handwaving, if they are as impartial as you suggest they are they've got nothing to lose by letting you know what their take is and how big a portion of their clicks are confirmed fraud.
I disagree with the statement that Google would be giving away everything by opening up page rank. Google's "secret sauce" is hardly page rank alone. Google is really secretive about this stuff, but they arguably have the world's larges corpus of multilingual text in the form of their html index, coupled with the world's largest fault tolerant compute cluster, the largest repository of scanned text,the largest repository of digital video, and probably the largest repository of digital audio in the world. I'm sure that they also have the world's largest repository of people's surfing and searching habits, along with the one of the largest sets of analytics data.
Even if page rank, map reduce, GFS, etc.. were open sourced I'm sure they'd still find a way to make billions off of the data that they have.
But, I think the larger point that the author makes, is that unlike many giant technology companies (Microsoft, HP, Apple, etc..), Google has made billions off of using open source software. And, while they are a very large supporter of open source, what total percentage of their profits go back to OSS? What percentage of their code gets re-released as open source? These are good questions to ask, and I'd love to hear an open discussion about it.
Google, please keep it a secret, PLEASE!
In Google’s, case the correct relevancy ordering of the web is a huge point of contention. So they cannot simply open it up hoping to close out all the holes in the open. It is also hard to distinguish acceptable behavior (linking) from unacceptable behavior (selling links) except in the aggregate through heuristics that assume ignorance on the part of the attacker.
In this sense, Google’s system is a lot closer to an intrusion detection system than a lock or a cipher suite. You want to establish some ground rules but there’s no added security in declaring what you’re monitoring on your servers and how.
And does anyone gain from it? Not really. At this point, Google's supremacy is due just as much to their hardware know-how, to their market leader position, and to their excellent brand, as it is to their search algorithms. It won't help any competing search engines - just spammers.
The theory goes: if you let more people look at the algorithm, they can suggest better answers than Google alone, to thwart the spammers who are gaming the system either way.
But I suspect the real goal has more to do with the deep suspicion that Google can or does do a significant bit of hand-tweaking. And these people want to force the algorithm into the open to either end or curtail that.
Spam:email = 90% or worse
in search results it is closer to the opposite.
I'm sure once the open source community actually manages to do this and prove the 'many eyes' doctrine applies to more than just security and bug fixes that many doors will open.
It's especially interesting if you consider the origins of PageRank in academia, freely published and peer-reviewed, and still not that well understood by laypeople; would the open-sourced magic of Google even be understandable by that many folk? Those that do understand it could make a fortune as consultants; at the moment, shady SEO snake-oil salesmen are so prevalent and 99% of their advice is outdated or plain wrong, something that definitively calls them out on it could be helpful.
My impression is that PageRank, while initially extremely successful at filtering signal-from-noise, is very susceptible to manipulation. For example links farms which make unimportant pages seem important. As a result of such manipulation, Google started modifying their crawling, in order to reduce the amount of garbage going into PageRank (less garbage in, less garbage out). For example, they might look at the age and ownership in DNS records, and exclude domains that look fishy based on some heuristics. The spammers fought back with more tricks, and the resulting arms race has come to have an acronym: SEO. But the underlying idea of using an iterative approximation of the likelihood of encountering a page via a random walk of the web is preserved. (Some details here: http://www.miislita.com/information-retrieval-tutorial/matri...)
If that's right (and I could well be wrong... would like to be convinced...), then I don't see it as a "light-years away" kind of issue, but rather a series of hacks to stay one step ahead of the spammers. I don't see any particular value in open sourcing those.
Compared to a lot of other companies (Apple, Microsoft, nearly every other tech giant), they are an open company. It's already great that they realize there's value in an Open-source browser, and open-source OS, and multiple open platforms.
This is a meme I see a lot, but never with any evidence. That the board has a fiduciary duty to the shareholders I can understand. That it would have a legal obligation to create value, well, I think that would be hard to define, and I have never seen any evidence for the existence of this obligation.
You can argue about the timeline (short versus long-term growth), but you can't say that there isn't a legal obligation to attempt to increase value.
I know they're not popular, or exciting like a stock value swing, but dividends still exist, right?
I actually think that more companies should give dividends.
I can see why companies like to have billion dollar war chests, but at some point you have to wonder when enough is enough. I mean, even Microsoft was forced to start paying dividends. Then again, their stock price has been pretty flat for the past few years, so that might be the only reason to hold it.
See Google's S1 (IPO) filing for reference: http://www.sec.gov/Archives/edgar/data/1288776/0001193125040...
Not to say it's interpreted correctly, but that is where the 'meme' stems from. If people quoted primary sources this might be more widely understood.
Security by obscurity is a bad analogy here. The problem is that Google's algorithms are not mathematically perfect, in the sense of cryptography. Instead, they merely statistically tend to find good results. You can't find bugs just by looking at source code. You have to statistically analyze the effects of any proposed changes.
In general people are willing to spend a certain amount on a solution. Therefore if you can things that make the complement to what you make money on cheaper, that increases how much you can charge for your piece of the solution. Open sourcing is a way to make those pieces cheaper.
Google does this. They give away tools and knowledge about how to make websites. They give away analytics and an A/B testing platform and educate people about how that makes it easier to produce good websites. This makes building good websites cheaper, and therefore increases how much people are willing to spend to advertise those websites.
This isn't a new strategy. In the late 90s when Oracle and IBM got into open source, this was their strategy. Oracle wanted to get people to stop paying for Sun hardware so Oracle could negotiate higher prices. (I know more than a few people who were surprised to find that licenses for Oracle on Linux are higher than they are on Sun.) IBM was looking to reduce the price for complements to IBM websphere, and was looking for opportunities to avoid having customers pay the Microsoft tax.
If you run a company and wish to strategically open source something, you should think in the same terms. What is the complement to what you are charging for, and how can you make that cheaper?
Everybody could do it. This man is not asking for code, is asking for DATA only google has. It's like an atheist saying to a catholic what he must do because he believes. When this man gives something of value to the world,like google did(I remember the Altavista-Ad-flickering in your eyes days) then he could demand to be given by others.
It's perfectly understandable. Lots of money is made in the existing paradigm and so many of you have incentive to believe it's the most rational system. Besides, hard to imagine Google's doing something wrong, right?
But the argument that "the only people this will help is spammers"? Really? Are the only people helped by open cryptographic protocols those who look to decipher traffic?
"They have every right to keep it secret and profit off of it." Let's allow this for a moment. We've still got a false dichotomy. Making it public will not deprive them of their profit. First, incredible marketing coup. Does anyone else remember how much of Google's initial strength came from its rabid fans? This would do a lot of re-energize that base.
Second, are there going to be new startups which can effectively compete with Google, even given its algorithm? How are they going to provide a better experience, or serve the volume of traffic, or index a larger amount of pages with the same algorithm with fewer resources?
Third, who says they have to authorize competitors to use it? This is I think one of the more interesting points. They could release the algorithm but license it only for non-commercial use for anyone not themselves. Ridiculous? Because Microsoft would violate such a deal anyhow? Because the algorithm could simply be reimplemented? Perhaps. But it would be another roadblock.
Fourth, why do we continue to believe that Google's strength comes from proprietary code? Why do we not recognize that its strength comes from mindshare, user experience, and quality of execution? None of which would be negatively affected by releasing the source.
Perhaps Google would face heightened competition as a result of releasing its algorithm. Perhaps this would outweigh the benefits of doing so. But there would be significant good which would result, including public feedback which would make it significantly harder for spammers to be successful.
But I'm just a foaming-at-the-mouth open source radical. And I'm sure history is on your collective side. If there's anything we've learnt from a hundred years or so of Computer Science (overestimate if you only count actual programming, underestimate if you allow Babbage as you should), it's surely that secrecy is what drives innovation, right?
Some balance in the force, for Yoda's sake.
As some have incorrectly asserted, Google's latest post is not about "everything open", it is an internal memo to product managers that it occurred to them would be fit for public consumption.
The central thesis of the letter is that "open" is hard to define (and defined differently by different people) and yet crucially important. Because it is hard to define, many people misuse it (OpenDNS comes to mind), and no one can call them on it because it is a vague term.
Internally, Google employees argue about what open means, and the purpose of this letter was to clarify what open means to Google. They lay it out in extremely clear terms, and it is well structured.
This response essay by Dixon misses all that completely - he unilaterally defines "open" to mean "open source", and then asks for Google to give away their software, claiming that if they don't, they're hypocrites.
Going back to Google's definition of "open" (which is complex), one significant aspect they emphasize is the notion of "lock-in", i.e. locking in users, and locking out competitors. This is one area where Google puts their money where their mouth is; they let users leave. You can get at your email, export your Google Reader content, download all your Google Docs, archive your Picasa pictures, and export all your contacts. That means if a competitor comes along with something better, you can start using it. Try that with Facebook.
Dixon acting like that is worthless because it isn't the "open-source" form of "open" is patently absurd. If this is confusing, I highly recommend reading Simon Phipps essays on open source to understand what kind of software makes a good candidate for a business to open source (operating systems, browsers, compilers...the basics that we all use to compete) and what does not (grid/cloud computing algorithms, specialized software, and other types of software that are differentiators). Essentially, the business model is to pool our collective resources in areas in which we don't want to compete (web server software, operating systems) and to spend time innovating in areas we do wish to compete (search algorithms, voice recognition, image analysis, etc.) Over time, as those specialized areas become more mainstream, they become better candidates for open-sourcing.
I think it would be a good revenue strategy, if they charge $dollars to display top 10 positive features and top 10 negative features of any given query-document pair without any real weights/scores (maybe re-squashed with some logistic function). The $dollars amount is computed dynamically to offset the negative impact of the partial reveal of the ranking algo/model. This can be a tremendously useful tool for troubleshooting search results by site owners.