74 comments

[ 767 ms ] story [ 2608 ms ] thread
Fantastic article. This is one of very few entries on this debacle that is actually informed and insightful.
I agree that it's a well written article and provides a very good overview. The tables are especially helpful.

However, there is one rather large oversight:H.264 is an open standard

This is not necessarily correct. As a citizen of the EU, I have to disagree. One of the EU pre-requisites[1] for something being qualified as an Open Standard is: The intellectual property - i.e. patents possibly present - of (parts of) the standard is made irrevocably available on a royalty-free basis.

[1]: http://en.wikipedia.org/wiki/Open_standard#European_Union_de...

It would appear that Steve Jobs agrees too. In his "Thoughts on Flash" he mentions several times that HTML5 is an Open Standard and contrasts this favourably with Flash. But when he talks about H.264 he refers to it simply as an industry standard.

http://www.apple.com/hotnews/thoughts-on-flash/

I'd think that H.264 isn't subject to this definition - or, at worst, it falls in a gray area of sorts - as the H.264 technology is unencumbered by license fees if it's used non-commercially. No developer of gratis software using MPEG-2 or MPEG-4 tech. has ever been subjected to a legal case initiated by the MPEG-LA.
> the H.264 technology is unencumbered by license fees if > it's used non-commercially

That's not true, unfortunately.

There may have been no lawsuits (I can't comment on that, though I'd like to see your data). But that's just because the MPEG-LA hasn't felt like suing people. If you're shipping an H.264 encoder or decoder in binary form to people and not paying the relevant licensing fees, then you're infringing on the patents. There's no exception for "non-commercial" use there last I checked. If you think there is, please cite.

They probably won't do it.

Somebody will come up something similar to the hack/exploit that Mozilla did: set up a for-profit corporation to receive money and a non-profit foundation (which owns 100% of corp's shares) to do the development. Then they can claim that the non-profit foundation is the one that's really using the patents and hence should not have to pay royalties.

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

Gold autumn, personality Mes clothing + Shoes, Travel bag that grabs an eye coat + Chao packet Free transport

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

●------ http://u.ly/73I ------●

Very good summary of the issue. I particularly like the point that <video> is, in and of itself, open and as such the establishment of a consistent baseline codec in VP8 could actually allow h.26x to flourish and innovate as the codec of choice for high-quality content.
As the end user...I only care about which one gives me the best experience period.
The trouble is that "best experience" is a vague term. The best experience today may be different from the best experience tomorrow, which in turn may not causally lead to the best experience of all potential experiences 2 or 5 years later. Figuring out how to get the best experience a few years out is difficult; it involves some measure of creating the future, and some measure of predicting the future.

It's too simplistic to argue H.264 as being the best for the user, hence it should be supported. Figuring out whether it is actually best for the user depends on a lot of variables, a lot of hard work, and some measure of work.

I'll expand a bit on barrkel's comment:

Back in the day when IE dominated Netscape Navigator/Communicator, IE was the best end-user experience. Flash-forward a few years and IE is languishing, with practically no new development happening on it. At this time (prior to Mozilla taking off), IE is still the best end-user experience available, but not the best end-user experience possible. It to took the Mozilla Suite and it's spin-off, Firefox, to bring a better user-experience to market.

The same applies now. While h.264 might provide the best experience right now, it's (possibly) at the cost of the experience in the future. What happens in the future when the MPEG-LA decides to start changing/enforcing royalties? What growing pains will the industry have to endure to change formats away from h.264 if that becomes an issue? What if h.264 is so entrenched that they can't migrate away from it? Why not just standardize on something now that won't have those problems later?

To fall back on the analogy, if IE hadn't pushed Netscape out of the market, what would the web ecosystem look like now?

>open standard

>incompatible with open source

Does this make sense to anyone? It seems to me the word open doesn't mean much now.

The term "Open standard" predates the term "open source". The latter term was born as recently as January 1998 - around the same moment when Netscape Navigator was open sourced. The semantics that the latter term's use of "open" imply cannot be forced on the former.
I don't see that disagreeing with what I said. I can't tell if it was meant to, either.
Only inasmuch as the phrase "doesn't mean much now" suggests that it ever had a solid, singular, standardized meaning.
(disclosure: I work for Mozilla)

I've never found the argument you're making very compelling. It's "open" in the same way that golf course with a $500 greens fee is open.

What it really constitutes is a way for existing market players to collude, and thereby exclude future participants. Some of those future participants might be quite disruptive, and benefit consumers.

It's really pretty funny that some terribly corrupt standards organizations arrived at a totally unsatisfactory definition of "open", and then tried to claim that better definitons of "open" somehow infringed(!) on their early, bad definition.

Most public golf courses I know of charge fees. Anyone can play as long as they pay. Some people actually do complain the fees are too expensive, but it's better than some country clubs where you need to be invited, approved, and then upfront dues atart at $100k.

Now I know your point wasn't actually about golf, but a lot of "open" facilities do cost money to use. It's just that they're non-discrimanatory about their use and charge what they consider reasonable fees (enough to pay for staff, upkeep, improvements, etc...).

This is actually very similar to MPEG-LA, the big difference being that MPEG-LA is not non-profit. But then again neither is Google. What would be a nice move by Google is to actually give all On2 property to Mozilla or Xiph. Or ever better yet, Outercurve (to show that they don't really want to indirectly exert disproportionate influence).

> Most public golf courses I know of charge fees... enough to pay for staff, upkeep, improvements, etc...

I don't think you read what I wrote, and I doubt you play golf. The fee at a decent public course where I grew up in Oregon is $37.

> This is actually very similar to MPEG-LA,

It's not at all similar. MPEG-LA collects rent on imaginary property.

As for non-discriminatory: I believe the MPEG-LA patent-holders are free to make side deals that bypass the supposedly non-discriminatory licensing fees. So if you are a major patent holder, you can in theory go to each of the other patent holders and negotiate 'cross-licensing' of your patents, agreeing to charge each other nothing for using each other's licenses.

So MPEG in this light can be seen as a collusive monopolistic oligarchy, where if you're in the club you get something for free that everyone else has to pay for. Not very competitive.

A bit of history: early on, On2 made a stink about the monopolistic nature of MPEG-LA:

http://www.cedmagazine.com/Article.aspx?id=63386&LangTyp...

They aren't really side deals. Its that pricing for members is a function of contributed patents. In otherwords, if you contribute 90% of the patents, you pay less.

And anyone can go to each patent holder and negotiate licensing of patents since the licensors still hold the patents. Part of the reason that MPEG-LA exists is to make it so that you don't have to negotiate with fifty companies in order to implement H264. MPEG-LA doesn't block you from doing this if you prefer that route, but it will take a LOT more time and likely be more expensive.

Yep, I think it even predates the invention of computers. For hardware, the H.264 level of "open" would have been enough.
Because hardware has unit costs baked in. Software doesn't.
Yep, that is one problem with the royalty model for software patents. It depends on artificial scarcity.
If Firefox, Chrome and Opera used the OS’s media frameworks, we would be set back into the dark ages when people used every video codec imaginable and nobody would be happy.

Dubious. The incentive to publish in a format viewable by the widest audience applies regardless of what format that happens to be.

It's true that some publisher targeting a niche audience might use a narrowly-available codec, but 1) that's not a bad thing, and 2) there is nothing stopping anybody from doing that with a plugin instead, no matter what stupid limitations you cripple your <video> implementation with.

In the general case, though, web video publishers want their content to be viewable by the broadest audience. If the freetard assertion that we need a mandatory codec that everybody can use is accurate, why on earth would the free market (video publishers) not arrive at the same conclusion?

Moreover, how did it not already arrive at that conclusion? Why do we use JPG/GIF/PNG/MP3? How can the case of Flash becoming the ubiquitous form of video playback (ca. 2004-2010) not be taken as evidence that the web can converge on a format without interference from supposed supporters of freedom and openness? Why is there this meme that <video> would be total anarchy if not for paternalistic browser vendors when we have concrete and recent examples of just the opposite?

>H.264 is an open standard. It was developed by a committee, standardized, reviewed by many engineers and developers for multiple companies and has been standardized for use with a multitude of containers and devices.

>VP8 is not a standard. It was developed secretly by a single company, and until recently, had only a single working implementation. The public wasn’t open to collaboration on the specification until the bitstream spec was frozen, including the bugs that existed within.

This is an interesting point. One I had never even realized.

There's almost very little point to opening up the VP8 standards process. It seems pretty much designed to challenge H.264 but avoid its patents (how well? TBD). Its severely bound in what it can be by trying to avoid H.264's existing patents.

  > Its severely bound in what it can be by trying to avoid
  > H.264's existing patents.
Doesn't that describe anything that isn't h.264? Might as well say that h.264 got there and patented everything good, so they must have won. We should just pack up and go home, they get to have a monopoly on video codecs for the next 14+ years.
Doesn't that describe anything that isn't h.264

Yeah probably - ask the Theora guys how hard its been ;-) The upside is that it could very much, out of necessity, drive some considerable advancement in video compression algorithms.

h.264 doesn't have a patent cliff. The move from mpeg1->2->4 has been one of gradual refinement. Like the move from 1 pixel to .5 pixel motion estimation. We should be using a layered codec so those ridiculous patents can be folded in as they expire.
We already have wavelet codecs like Dirac, which are quite exotic compared to MPEG4 / VP8 / Theora.
This is an important point not just for ideological reasons, but it is one reason why the FUD argument is not symmetric.

Since H264 was implemented in the open with open and large membership to the body, it is more difficult for some company to come out of the woodwork and say, "Hey, we have a patent that H264 rides on, but we didn't know it until now" Note, QCOM tried this with H264 several years ago and got their butt handed to them by the courts (http://www.broadcom.com/press/release.php?id=1037466). People have had 10 years to examine this open specification. I think its reasonable to say that one would have found a violation by now. And almost all the majors are already either licensees or licensors. There's a slim chance that there's a patent floating around in some patent troll portfolio that hasn't been examined, but that's the only real vector I can see of a legitimate surprise H264 patent violation.

Whereas with WebM its been in the public domain less than a year. It could feasibly take a few years for it to be reaonable that anyone who would have done due diligence to have done so. It's completely reasonable that in a year from now Siemans comes out and says, "WebM infringes these three patents we have". A court is not likely to throw it out saying they've been sitting on them.

I don't think Qualcomm would even have lost that case if not for their participation in the standardization process. MPEG-LA is probably under no obligation to disclose the extent to which they believe their portfolio is infringed on by WebM until it becomes profitable to do so, so we will likely not know whether this is the case until WebM achieves significant uptake, if that ever happens.
Since when are patents thrown out because the company/patent troll is sitting on them? Patents are not trademarks and you don't have to actively defend them.
While what you're saying is true, the royalties side of things has been locked down by Google's legal team very effectively. If said patent holder makes a patent claim against WebM they automatically destroy all free royalty rights for themselves on the same codec (if I understand correctly). This means that while they could try to hold Google (or any other user of the codec) to some kind of patent royalties, Google could retaliate for any use that company or individual makes of the same codec (with as much leeway to set a price as the patent gives the original individual or company.

This is the true genious of the WebM license. It means even if this does happen in the future, Google can destroy whoever has this patent through a massive counter. It also means that any challenge would have to be made extremely carefully.

edited for spelling :)

Doesn't matter if the patent holder is a troll with no viable products of its own.
But H.264 would not be immune to that either.
This is a great post and clarified quite a few confusing points of the discussion for me.

One area that unfortunately it didn't really cover was the impact of widely available hardware and GPU-accelerated decoding for H.264. Surely the millions of non-PC devices being sold with H.264 support (even baseline) will have some effect on the outcome of the new video format war.

The amount of money in infrastructure, cameras, and expertise invested by the film industry (studio to journeyman) is a rather large barrier to overcome. Trusting an industry group versus one company will not help adoption.
"trusting" one company has little to do with it since you already have an irrevocable license to the vp8/webm patents.

the infrastructure argument is much stronger.

I am still trusting the word of one company versus something that has been vetted by a lot of companies and been found beneficial for multiple parties from multiple parts of the use chain. Patents are not the only issue.

Would a person believe Microsoft's good intentions if it was WMV instead of WebM?

VC-1 was Microsoft's attempt at a royalty-free codec, and I don't see any reason not to use it (at the time it was published, before the patent pool was created). It's more than just intentions, all the patents owned by On2 have been made irrevocably royalty-free.
Only for Google's implementation and not any others...

This is the key point that everyone's missing.

That's a pretty huge key point. Do you happen to have a citation? The relevant granting of rights (the VP8 Bitstream Specification License[1]) doesn't seem to say anything about that: "a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer implementations of this specification ..."

[1] http://www.webmproject.org/license/bitstream/

http://www.webmproject.org/license/additional/

>Google hereby grants to you a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, transfer, and otherwise run, modify and propagate the contents of this implementation of VP8, where such license applies only to those patent claims, both currently owned by Google and acquired in the future, licensable by Google that are necessarily infringed by this implementation of VP8. This grant does not include claims that would be infringed only as a consequence of further modification of this implementation. If you or your agent or exclusive licensee institute or order or agree to the institution of patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that this implementation of VP8 or any code incorporated within this implementation of VP8 constitutes direct or contributory patent infringement, or inducement of patent infringement, then any patent rights granted to you under this License for this implementation of VP8 shall terminate as of the date such litigation is filed.

It explicitly grants you the ability to modify the library in whatever way you want. This naturally includes a complete re-implementation.

There's a strangely worded section in there, but it is a clear cause-and-effect:

>This grant does not include claims that would be infringed only as a consequence of further modification of this implementation.

So, any claims infringed only as a consequence of further modification are not covered. But if the patent claims were already infringed by Google's implementation, then you are covered.

From: http://www.webmproject.org/about/faq/#licensing

Q: What if someone makes a change to the code and gives it to me. Do I have a patent license from Google for that change?

A: You still have the right to redistribute but no patent license for the changes (if there are any patents covering it). We can't give patent licenses for changes people make after we distribute the code, as we have no way to predict what those changes will be. Other common licenses take the same approach, including the Apache license.

I'm not sure if it supports your statement, it would be great if someone more knowledgable about this can clarify.

It's not critical for browsers to play raw video from H.264 cameras.

Cameras shoot in HD and realtime compression isn't very efficient, so such raw files are unsuitable for the web.

For the web you still need to lower resolution and bitrate of the files. Re-encoding step is necessary, and it allows format of web video to be independent of source format.

Not really. The upcoming generation of mobile devices will have hardware support for VP8 and the users upgrade these devices very often.
Why would a hardware vendor add VP8 Support? I can think of two reasons not to:

   o They run the risk of patent infringement
   o H.264 is already playable everywhere, universally, 
     either natively (and with hardware support) or with Flash.
Well, the tegra2, which is the reference platform for Honeycomb, has vp8 support in hardware. Texas Instruments announced full support for vp8 in all their future DSPs and media processors and Imagination Technologies, the people who make the GPUs on the iPhone have committed to vp8 support in their next generation GPUs. It is very likely that for the iPhone6, apple will be able to add webm support at little extra cost to itself, and no patent threat, since they are already MPEG-LA licensees. Same goes for all the other manufacturers- they are already MPEG-LA licensees and safe from the vast majority of patent threats. Other than that, I would love to see a patent troll try and go after TI or Nvidia.
That's very good point.

Same goes for all the other manufacturers- they are already MPEG-LA licensees and safe from the vast majority of patent threats.

So all those implementing WebM still needs to license H.264 to be safe from patent threat, and all those implementing WebM almost all have implemented H.264. WebM is only an addition to H.264. So the question for Google is always why dropping H.264 and never why adding WebM (since it is always there).

Very often? Have you seen how many people are still using iPhone 3Gs (not 3GSs!) That's about two and a half years old.
I still am. I know a couple people who still are too (the 3G iOSv4 wouldn't have needed an update if the market share weren't significant).
I'm not sure how you can say that with such confidence. Hardware support for a particular codec comes from a high level of demand for use of that codec. Demand comes from content producers and consumers. If Microsoft, Apple and the other consumer hardware vendors are committed to H.264 and the only major producer that is serving VP8 (YouTube) is also serving H.264, I'm not seeing where this demand is going to come from.
The biggest part of the 'rage' towards Google over the drop of Chrome's native ability to use H.264 in the <video> tag, is that Google is trying to spin as its for the benefit of innovation. Which is not the case. Flash has been great, but it's not needed for the distribution of video content, and the fact that WebM is a lawsuit waiting to happen does not help either.

Great article though!

H.264 could also be considered a lawsuit waiting to happen. Who knows who out there has a patent it infringes? Maybe the webm folks have a few patents it violates?
H.264 could also be considered a lawsuit waiting to happen

It could be and its been challenged at least once in the past (however Qualcomm lost that challenge), but though its not free, its patent restrictions are pretty clear and tested. It is 7 years old. While stranger things have happened, its very unlikely that as a licensee of H.264 you are going to be sued.

WebM's status is not as clear, sometimes you choose the devil you know.

YouTube in HTML5 is an amazing experience through the YouTube5 Safari extension from Connor McKay. H.264, through YouTube, on Safari. For the low low price of free: http://www.verticalforest.com/2010/10/27/youtube5-version-2/

Best YouTube user experience ever, auto loads the highest quality every time, and downloads faster. This is the way YouTube should be!

Has HTML5 been used to deliver any valuable content on desktop machines? And by "valuable" I mean terrible Hollywoood movies or trashy TV Shows.

I was under the impression that they all used Flash and Silverlight because then they could implement DRM. They don't need to do that on iOS devices, because the whole platform is DRM'd.

For example BBC iPlayer is HTML5 to iOS, but Flash to Mac OS X. They've announced that all the other mobile devices will be served via Flash.

People that want to post videos of their cat dancing or a backyard physics experiment on the other hand, have totally different needs, like the ability to post it in their subscriber only cat physics forum without worrying about licence fees.

> MPEG LA has a royalty cap so that companies selling high-volume products know beforehand the maximum amount of royalties they'll have to pay to MPEG LA in a given year. The current $5 million cap really isn't much for a big player possibly generating many billions of annual revenues with products that include an AVC/H.264 encoder and/or decoder.

http://fosspatents.blogspot.com/2010/06/mpeg-las-avch264-lic...

leaving google aside, I think it's pretty obvious that the "royalty-free codec" that entities like Mozilla are looking for is pretty incompatible with "but there's a cap on the royalties."

hence the <video> situation we were in even before last week and google's blog post.

(comment deleted)
The cap is $6.5 million for the next five years and then there is no limit on how much it could be raised after that. The royalties can't be raised more than 10% every five year period, but the cap isn't under that restriction.
I would suggest the reason they have a cap is because a) it is anti-competitive towards smaller firms moving into the market to challenge the incumbents who make up MPEG-LA b) it would make economic sense for a bunch of big companies to get together and produce a competitive codec and compete with H.264 if they didn't structure their fees in this way.

They are after all a government sanctioned cartel. You don't get away with this kind of anti-competitive price discrimination otherwise.

Which is why it always confuses me that people talk about what an excellent codec it is. How do you know it's excellent when the legal structures are set up specifically to exclude competition, and therefore comparison?

Dear friends, do you want to have some different things? Whether you want to give your relatives and friends, take a few different exotic gifts? Whether you want to buy some cheap benefits of thing? So please, let us begin now!

Click on our website

(=== http://www.onseeking.com =====)

Will bring you different surprise

=== http://www.onseeking.com =====

=== http://www.onseeking.com =====

=== http://www.onseeking.com =====

=== http://www.onseeking.com =====

=== http://www.onseeking.com =====

=== http://www.onseeking.com =====

=== http://www.onseeking.com ======

Could this be a move to get the H.264 patent holders to release their patent hold on the standard?
It's definitely putting pressure on them to provide a royalty-free version, which might include some of the less performant profiles. This was in the mix originally when H.264 was being developed, but major patent holders refused to go along with a 'royalty-free baseline' profile.
No way that could happen. It would have be unanimous to be of any use at all.
Why is it necessary for HTML5 to specify a codec for the <video> and <audio> tag when no format is specified for <img>?

(I'm not being rhetorical)

It's not really necessary, and so far the HTML5 spec hasn't done so. However, it would be useful if all browsers had at least one codec in common. GIF and JPEG became de facto standard image formats because all browsers supported them, even though there was no de jure requirement.
(comment deleted)
I believe this is to date the sanest article related to the H.264/WebM debacle; it clearly, and with detail, brings out the fact that H.264 is open. However, it fails to clarify a very important detail to this discussion, surrounding the royalty issue: H.264 coding/decoding is free-of-charge to implement, distribute and use in non-commercial contexts.

Google's, Mozilla's and Opera's actions and careless, uninformed (or possibly intentionally slanderous) responses in this discussion clearly show that their issue with H.264 isn't that H.264 isn't open enough, but rather that it isn't gratis enough.