> discard information which will contain the information with high frequency components. Now if you convert back to your regular x-y coordinates, you'll find that the resulting image looks similar to the original but has lost some of the fine details.
I would expect also the edges in the image to become more blurred, as edges correspond to high-frequency content. However, this only seems to be slightly the case in the example images.
You can see exactly that with the speaker grill and the text (This type of transformation is notoriously bad at compressing images of text, and is why you shouldn't use jpg for pictures of text)
In this context, the edges of, say, the macbook are not "high frequency" content, since they only feature one change (low to high luminosity) in a given block rather than several (high-low-high-low-high) like for the grill.
Sure, but also keep in mind that the technology hasn't changed much over time. Even HEVC, which causes extreme gains in compression on high-res video with minimal loss in quality, is still mostly the same algorithm as H.264 but with larger blocks, slightly more flexible coding units rather than frame-wide interpolation changes, and 35 rather than 9 directions recognized for predictions.
I was a bit disappointed in this article for the same reason: this is a great primer for people new to MPEG video compression, but it doesn't have anything to do with H.264.
I was hoping the author would write about H.264 specifically, for instance, how it was basically the "dumping ground" of all the little tweaks and improvements that were pulled out of MPEG-4 for one reason or another (usually because they were too computationally expensive), and why, as a result, it has thousands of different combinations of features that are extremely complicated to support, which is why it had to be grouped into "profiles" (e.g., Baseline, Main, High): http://blog.mediacoderhq.com/h264-profiles-and-levels/
I was also hoping that he would at least touch on the features that make H.264 unique from previous MPEG standards, like in-loop deblocking, CABAC Entropy Coding, etc..
Again, it's fine as an introduction to video encoding, but there's nothing in here specific to H.264.
The lossy transform is important, but I think what's actually most important in video compression is getting rid of redundancy --- H.264 actually has a lossless mode in which that transform is not used, and it still compresses rather well (especially for noiseless scenes like a screencast.) You can see the difference if you compare with something like MJPEG which is essentially every frame independently encoded as a JPEG.
The key idea is to encode differences; even in an I-frame, macroblocks can be encoded as differences from previous macroblocks, and with various filterings applied: https://www.vcodex.com/h264avc-intra-precition/ This reduces the spatial redundancies within a frame, and motion compensation reduces the temporaral redundancies between frames.
You can sometimes see this when seeking through video that doesn't contain many I-frames, as all the decoder can do is try to decode and apply differences to the last full frame; if that isn't the actual preceding frame, you will see the blocks move around and change in odd ways to create sometimes rather amusing effects, until it reaches the next I-frame. The first example I found on the Internet shows this clearly, likely resulting from jumping immediately into the middle of a file: http://i.imgur.com/G4tbmTo.png That frame contains only the differences from the previous one.
As someone who has written a JPEG decoder just for fun and learning purposes, I'm probably going to try a video decoder next; although I think starting from something simpler like H.261 and working upwards from there would be much easier than starting immediately with H.264. The principles are not all that different, but the number of modes/configurations the newer standards have --- essentially for the purpose of eliminating more redundancies from the output --- can be overwhelming. H.261 only supports two frame sizes, no B-frames, and no intra-prediction. It's certainly a fascinating area to explore if you're interested in video and compression in general.
This is really interesting and the imgur picture you linked (with your explanation) explains it really clearly!
But when seeking, why wouldn't any local media playback seek backwards and reconstruct the full frame? It's not like the partial frame after seeking is useful - I'd rather wait 2 seconds while it scrambles (i mean "hurries up") to show me a proper seek, wouldn't everyone?
What was your Internet search for finding that imgur frame? What is this effect called?
>why wouldn't any local media playback seek backwards and reconstruct the full frame?
Most codecs/players do. VLC used to be criticized for being different in that regard. One possible advantage is istantaneous seeking, as there's no need to decode all the needed frames (which could amount to several seconds of video) between the nearest I-frames[1] (the complete reference pictures) and the desired one.
[1]: plural, because prediction can also be bidirectional in time
The use of incomplete video frame data for artistic purposes is called "datamoshing".
I try to use VLC when I can because it offers intuitive playlist support, but for high-resolution H.264 and friends I usually have to switch to Media Player Classic.
VLC is willing to let my entire screen look like a blob of grey alien shit for 10 seconds instead of just taking a moment to reconstruct frames.
And its hardware acceleration for newer codecs is balls. Sucks because otherwise, it's right up there with f2k for me.
>VLC is willing to let my entire screen look like a blob of grey alien shit for 10 seconds instead of just taking a moment to reconstruct frames.
Yes, this is what I was talking about, and yes, specifically for VLC. Plus it's not like playback is so taxing that all cores are pegged at 100% during playback. When I seek, VLC should get off its ass and scramble to come up with the correct full frame then. I'll wait.
I recently bought a camera that has 4k video recording. VLC just gives up playing the video. Even Windows Media Player can handle it. No idea what's going on, but I was really surprised and disappointed with VLC.
See if you can cut a small segment and submit it as a sample to ffmpeg. Hell, see if ffprobe and ffmpeg can play it. Happy to help, if you've got enough upstream bandwidth.
I stopped using VLC when I found mpv [0]. I really like it because it exposes everything from the CLI, so once you're familiarized with the flags you're interested in using, it's easy to play anything. For everyday usage it "just works" too, as expected of any video player.
Does it include all the codecs by default? I think this was a major reason VLC succeeded the way it did. With all other players (BPlayer anyone?) you needed to find and install tons of codecs while in VLC it just worked.
Isn't the other advantage that VLC can play incomplete movie files? Any other players I have tried 'crash' on incomplete torrents, when VLC just fails until it finds the next I frame.
In a course I taught (2010) on music visualizations that's the term I used.
The example I used in the lecture where datamoshing came up was the music video for Charlift's "Evident Utensil"[1]; I always thought this was a neat example.
> MJPEG which is essentially every frame independently encoded as a JPEG.
"essentially" makes it sound like it isn't precisely true. MJPEG is literally just a stream of JPEG images. The framing of the stream varies a bit, but many implementations are just literal JPEG images bundled one after the other into a MIME "multipart/x-mixed-replace" message.
A real fun read. Had an assignment a couple of weeks ago where we used the most k most significant singular values of matrices (from picture of Marilyn M.) to compress the image. H.264 is on a whole other level, though ;)
I love how you can edit photos from people to correct some skin imperfections without loosing the touch that the image is real (and not that blurred, plastic look) when you decompose it in wavelets and just edit some frequencies.
Don't know in photoshop, but in Gimp there's a plugin called "wavelet decomposer" that does that.
The part about entropy encoding only seems explain run-length encoding (RLE). Isn't the interesting aspect of making use of entropy in compression rather to represent rarer events with longer longer code strings?
The fair coin flip is also an example of a process that cannot be compressed well at all because (1) the probably of the same event happening in a row is not as high as for unfair coins (RLE is minimally effective) and (2) the uniform distribution has maximal entropy, so there is no advantage in using different code lengths to represent the events. (Since the process has a binary outcome, there is also nothing to gain in terms of code lengths for unfair coins.)
> H.264 is protected by patents owned by various parties. A license covering most (but not all) patents essential to H.264 is administered by patent pool MPEG LA.[2] Commercial use of patented H.264 technologies requires the payment of royalties to MPEG LA and other patent owners. MPEG LA has allowed the free use of H.264 technologies for streaming internet video that is free to end users, and Cisco Systems pays royalties to MPEG LA on behalf of the users of binaries for its open source H.264 encoder.
It is an open standard. Anyone can purchase and implement it, and it was developed by ISO. The technologies are not royalty free in the US. Don't conflate the two. *
Edit: I emphasize this mainly because the terms have a specific meaning in standards jargon but also because it places the blame for software patent abuses on the wrong parties (the standards developers rather than the lawyers and legislators).
blame for software patent abuses on the wrong parties (the standards developers)
Uh, anyone familiar with the MPEG process will assure you that the companies involved love (let me restate that: PREFER) to bring in technology on which they own the patents so they get a good cut of the resulting patent pool.
Sometimes this is even done even though it technically makes no sense. Best example: hybrid filter-bank in MP3.
The process also provides no protection or discouragement from patents from semi-involved industry partners appearing later on, etc.
This difference in approach is a stark contrast to the IETF, which is why Opus work, and future AV1 work are happening under the IETF rather than the MPEG groups.
I wonder if across a lot of videos, the frequency domain representations look similar and if instead of masking in a circle we could mask with other (pre-determined) shapes to keep more information (this would require decoders to know them, of course).
Or maybe this article is too high-level and it's not possible to "shape" the frequencies.
It's certainly possible to use any arbitrary shape. The way it really works is that there is a quantization matrix - which essentially is a configurable mask for your frequency domain signal.
Yes, I've dumbed it down in the article to a simple circle to illustrate the point.
Anyone who likes this would probably also enjoy the Daala technology demos at https://xiph.org/daala/ for a little taste of some newer, and more experimental, techniques in video compression.
I'm not sure if there's been an official announcement, but I had assumed that AV1 was going to be adopted/ratified as NETVC so that it's got a standards body rubber stamp, as well as the practical adoption/support from browser vendors, GPU manufacturers, streaming sites etc.
Suppose I have a table of 8-digit numbers that I need to add and subtract for various reasons. Do I A: have a child, train them how to read numbers, add, and subtract, and then have the child do it or B: use a calculator purpose built to add and subtract numbers?
Neural nets are always expensive to train. You'd better be getting something from them that you can't get some other way.
Yes, you don't need the machinery of learning when you already have an algorithm you're happy with. Adding a table of numbers, I don't think anyone hopes to do much better than we already do with our circuits and computer architectures.
With video compression, I think most would agree that there might be better architectures/algorithms that we haven't stumbled upon yet. Whether specifically "neural networks" will be the shape of a better architecture, I don't know. But almost surely some meta-algorithm that can try out tons of different parameters/data-pipeline-topologies for something that vaguely resembles h.264 might find something better than h.264.
Neural nets are expensive to train. But so is designing h.264.
In my anecdotal experience, h265 gets me 50-60% improvements in file sizes at the same quality for fairly low quality targets and the gains drop off rather quickly as you increase the quality. For videos where you don't care about the quality all that much, it's superb.
I was under the impression that the first 100,000 units are free, and then 20c per unit afterwards to a max of $25m.
H264 drops to 10c per unit after 5m units, to a max of $6.5m.
You need to be shipping 125 million units annually to hit the full $25m.
Yes it's more, but it's not quite ten times. And notably if the chip maker pays the royalties, then the content creators don't need to (though that was excepted indefinitely with H264).
Parts regurgitated from a quick google for reference [1]
It is actually more then 10x. The annual cap for H.264 royalty fees is 6.5M from MPEG-LA. For H.265 it is 25M from MPEG-LA, AND 50M from HEVC-Advance. That is a total of 75M. And like others have pointed out there are Technicolor patents fees not included.
So it depends how these royalty works in details. If only the chip manufacture are paying, Mediatek, Qualcomm, Samsung, Intel, AMD, Nvidia, Apple. That is at least 10 players paying maximum. And if you consider small players, the total contribution of Royalty fees to HEVC is 1 Billion / Year. ONE BILLION!! In the life time of a Video Codec that typically run at least a decade, these patents are 10 Billions.
Do you think that is a fair price, i think everyone should decide for their selves.
This is a great overview and the techniques are similar to those of h264.
I found it invaluable to get up to speed when I had to do some work on the screen content coding extensions of hevc in Argon Streams. They are a set of bit streams to verify hevc and vp9, take a look, it is a very innovative technique:
Very well explained. But I could have understood it all without the bro-approach to the reader. You see where I am going with this? Get it? See where I am going? Ok!
I remember loving this style when I was a novice, e.g. Beej's networking tutorial. Not a big fan anymore, either, but certainly valuable for (part of) the target audience, I think.
"See how the compressed one does not show the holes in the speaker grills in the MacBook Pro? If you don't zoom in, you would even notice the difference. "
Ehm, what?! The image on the right looks really bad and the missing holes was the first thing I noticed. No zooming needed.
And that's exactly my problem with the majority of online video (iTunes store, Netflix, HBO etc). Even when it's called "HD", there are compression artefacts and gradient banding everywhere.
I understand there must be compromises due to bandwidth, but I don't agree on how much that compromise currently is.
There's another false claim like that a bit below. I can only assume that the author is close to legally blind, or uses a VGA-res display to watch the page.
Of course while reading the article you are going to be very conscious of detail and image quality because that is the subject matter of the post.
However if that MacBook Pro image was placed on the side of an article where the primary content was the text you were reading, you'd glance at the image and your brain would fill in the details for you. You probably wouldn't notice the difference in that context.
For most use cases, there likely is very little functional difference between the two images. At least, that was how I understood it.
The first thing I noticed was the ringing, which is an artifact of low-pass filtering so it's a nice opportunity to go into problems with that kind of filtering. Other than that I think it was an ok teaser that gives an idea of how compression is done and what the trade-offs are.
I find the 480p setting on my plex server at home actually looks better than most of the 1080p HD streams on the internet.
Although to be fair, I suspect that a lot of times what I'm looking at are mpeg videos that have been recompressed a half dozen or more times with different encoders. Each encoder having prioritized different metrics. So, the the quality gets worse until it doesn't really matter how good the compression algorithm is. Each new re-compression is basically spending 3/4 of its bits maintaining the compression artifacts from the previous two passes.
Yep, me too - more like, if I was blind, I wouldn't notice the difference. Which is why the bitrate is always the first thing I look at when sourcing video.
But given other settings with even h.264 vs. h.265 and the source content, that isn't always a valid metric either.
I mean for fast action scenes, I rarely notice the difference between 720p and 1080p at 10ft away... but different encoding and sources, not just size alone can make significant differences.
Ugh. Comparing the file size difference between a lossless PNG and a LOSSY H.264 video of a STATIC PAGE is absurd. Calling it "300 times the amount of data," when it's a STATIC IMAGE is insulting in the extreme. It really doesn't matter if the rest of the article has insights, because you lost me already.
It's a very high level example of the concept, which admittedly is absurd, but drives home the point for people who might not be familiar with the scope of the numbers involved.
Thanks for the feedback! Sorry if I was unclear. The comparison with PNG is very intentional to illustrate the vast difference in the compression efficiencies involved. I do state the difference clearly here though:
> This concept of throwing away bits you don't need to save space is called lossy compression. H.264 is a lossy codec - it throws away less important bits and only keeps the important bits.
> PNG is a lossless codec. It means that nothing is thrown away. Bit for bit, the original source image can be recovered from a PNG encoded image.
"Uncompressed" video has been stored, even before computers, as YUV (or some variant), not RGB as this author states. It makes me doubt everything else he says.
b) Leave codec packs in 2000 where they belong. They are a great malware vector and also good at messing with settings they shouldn't.
>KCP utilizes the following components:
MPC-HC - A robust DirectShow media player.
madVR - High quality gpu assisted video renderer. Included as an alternative to EVR-CP.
xy-vsfilter / XySubFilter(future) - Superior subtitle renderer.
LAV-Filters - A package with the fastest and most actively developed DirectShow Media Splitter and Decoders.
(Optional) ReClock - Addresses the problem of audio judder by adapting media for smooth playback OR utilized for bit perfect audio.
I'm actually using MPC-HC and AC3Filter to deal with some files where I couldn't hear the centre channel on VLC (on stereo speakers). Everything else isn't really needed.
oh crap it's the topic police. I use it specifically for madVR and interpolating frames for high-quality low FPS anime. It looks really great. The best I've found for this particular purpose. Be nice.
Can someone explain how the frequency domain stuff works? I've never really understood that, and the article just waves it away with saying it's like converting from binary to hex.
It's a bad analogy. Binary and hex are just different formats for representing the same number. Spatial domain and frequency domain are different views of a complex data set. In the spatial domain, you are looking at the intensity of different points of the image. In the frequency domain, you are looking at the frequencies of intensity changes in patterns in the image.
As for why you want to do this: throwing away bits in the spatial domain eliminates distinctions between similar intensities, making things look blocky. In the frequency domain, however, you can throw away high-frequency information, which tends to soften patterns like the speaker grills in the MBP image that the human eye isn't that sensitive to to begin with.
In the most basic terms, not even talking about frequency, the mechanics of this is that one series of numbers (pixel values, audio samples, etc.) is replaced, according to some recipe or another, with a different series of numbers from which the original can be recovered (using a similar "inverse" recipe). The benefit of doing this comes from the discovery that this new series has more redundancy in it and can be compressed more efficiently than the original, and even if some of the data are thrown away at this point, the purpose of which is to make compression even more effective, the original can still be recovered with high fidelity.
Basically we can represent any signal as an infinite sum of sinusoids. If you know about Taylor expansion of a function, then you know that the first order term is the most important, then the second and so on. Same principle with the sinusoids. So if we remove the sinusoids with very high frequency we remove the terms with least information.
The JPEG article also has a very good, step by step example of the DCT, followed by quantization and entropy coding: https://en.wikipedia.org/wiki/JPEG
Well done. The only thing that could make this better is an interactive model/app for me to play around with. The frequency spectrum can probably be used while retouching images as well.
A video on youtube led me to Joofa Mac Photoshop FFT/Inverse FFT plugins [1] which was worth a try. I was unable to register it, as have others. Then I came across ImageJ [2], which is a really great tool (with FFT/IFFT).
Edit: if anyone checks out ImageJ, there's a bundled app called Fiji [3] that makes installation easier and has all the plugins.
If anyone has other apps/plugins to consider, please comment.
229 comments
[ 3.5 ms ] story [ 189 ms ] threadIt would be really cool to further extend it showing actually how the various tiles are encoded and between frames, something along the lines of: http://jvns.ca/blog/2013/10/24/day-16-gzip-plus-poetry-equal...
H.265 can even do deltas between blocks in the same frame, IIRC, and is excellent for still image compression too.
I would expect also the edges in the image to become more blurred, as edges correspond to high-frequency content. However, this only seems to be slightly the case in the example images.
In this context, the edges of, say, the macbook are not "high frequency" content, since they only feature one change (low to high luminosity) in a given block rather than several (high-low-high-low-high) like for the grill.
http://web.cs.ucla.edu/classes/fall03/cs218/paper/H.264_MPEG...
How does DCT work:
https://www.youtube.com/watch?v=Q2aEzeMDHMA&
For example if you replace H.264 with a much older technology like mpeg-1 (from 1993) every sentence stays correct, except this:
"It is the result of 30+ years of work" :)
Still, it's a good overview of generic video compression.
https://en.wikipedia.org/wiki/MPEG-1#Macroblocks
Did you miss the third paragraph?
As someone who knew nothing about it before, I found it lived up to it's goal.
I was hoping the author would write about H.264 specifically, for instance, how it was basically the "dumping ground" of all the little tweaks and improvements that were pulled out of MPEG-4 for one reason or another (usually because they were too computationally expensive), and why, as a result, it has thousands of different combinations of features that are extremely complicated to support, which is why it had to be grouped into "profiles" (e.g., Baseline, Main, High): http://blog.mediacoderhq.com/h264-profiles-and-levels/
I was also hoping that he would at least touch on the features that make H.264 unique from previous MPEG standards, like in-loop deblocking, CABAC Entropy Coding, etc..
Again, it's fine as an introduction to video encoding, but there's nothing in here specific to H.264.
The key idea is to encode differences; even in an I-frame, macroblocks can be encoded as differences from previous macroblocks, and with various filterings applied: https://www.vcodex.com/h264avc-intra-precition/ This reduces the spatial redundancies within a frame, and motion compensation reduces the temporaral redundancies between frames.
You can sometimes see this when seeking through video that doesn't contain many I-frames, as all the decoder can do is try to decode and apply differences to the last full frame; if that isn't the actual preceding frame, you will see the blocks move around and change in odd ways to create sometimes rather amusing effects, until it reaches the next I-frame. The first example I found on the Internet shows this clearly, likely resulting from jumping immediately into the middle of a file: http://i.imgur.com/G4tbmTo.png That frame contains only the differences from the previous one.
As someone who has written a JPEG decoder just for fun and learning purposes, I'm probably going to try a video decoder next; although I think starting from something simpler like H.261 and working upwards from there would be much easier than starting immediately with H.264. The principles are not all that different, but the number of modes/configurations the newer standards have --- essentially for the purpose of eliminating more redundancies from the output --- can be overwhelming. H.261 only supports two frame sizes, no B-frames, and no intra-prediction. It's certainly a fascinating area to explore if you're interested in video and compression in general.
But when seeking, why wouldn't any local media playback seek backwards and reconstruct the full frame? It's not like the partial frame after seeking is useful - I'd rather wait 2 seconds while it scrambles (i mean "hurries up") to show me a proper seek, wouldn't everyone?
What was your Internet search for finding that imgur frame? What is this effect called?
Most codecs/players do. VLC used to be criticized for being different in that regard. One possible advantage is istantaneous seeking, as there's no need to decode all the needed frames (which could amount to several seconds of video) between the nearest I-frames[1] (the complete reference pictures) and the desired one.
[1]: plural, because prediction can also be bidirectional in time
The use of incomplete video frame data for artistic purposes is called "datamoshing".
VLC is willing to let my entire screen look like a blob of grey alien shit for 10 seconds instead of just taking a moment to reconstruct frames.
And its hardware acceleration for newer codecs is balls. Sucks because otherwise, it's right up there with f2k for me.
Yes, this is what I was talking about, and yes, specifically for VLC. Plus it's not like playback is so taxing that all cores are pegged at 100% during playback. When I seek, VLC should get off its ass and scramble to come up with the correct full frame then. I'll wait.
[0] https://mpv.io/
* Sane defaults (encodings and fonts, scaletempo for audio)
* instantaneous play of next and previous videos
* navigation in random playlist actually works
* Easy always on top key binding
* Most mplayer key bindings work
I'll definitely keep on trying it for a while.
I think I can find some use for this in certain situations. Still lacks a good playlist building schema.
The example I used in the lecture where datamoshing came up was the music video for Charlift's "Evident Utensil"[1]; I always thought this was a neat example.
[1]: https://www.youtube.com/watch?v=mvqakws0CeU
Kanye West's "Welcome to Heartbreak" (https://www.youtube.com/watch?v=wMH0e8kIZtE)
A$AP Mob's "Yamborghini High" (https://www.youtube.com/watch?v=tt7gP_IW-1w)
"essentially" makes it sound like it isn't precisely true. MJPEG is literally just a stream of JPEG images. The framing of the stream varies a bit, but many implementations are just literal JPEG images bundled one after the other into a MIME "multipart/x-mixed-replace" message.
Don't know in photoshop, but in Gimp there's a plugin called "wavelet decomposer" that does that.
http://registry.gimp.org/node/11742
There was a question about retouching photos some while ago (http://photo.stackexchange.com/questions/48999/how-do-i-take...) that using wavelets was a good use of it.
The fair coin flip is also an example of a process that cannot be compressed well at all because (1) the probably of the same event happening in a row is not as high as for unfair coins (RLE is minimally effective) and (2) the uniform distribution has maximal entropy, so there is no advantage in using different code lengths to represent the events. (Since the process has a binary outcome, there is also nothing to gain in terms of code lengths for unfair coins.)
> H.264 is protected by patents owned by various parties. A license covering most (but not all) patents essential to H.264 is administered by patent pool MPEG LA.[2] Commercial use of patented H.264 technologies requires the payment of royalties to MPEG LA and other patent owners. MPEG LA has allowed the free use of H.264 technologies for streaming internet video that is free to end users, and Cisco Systems pays royalties to MPEG LA on behalf of the users of binaries for its open source H.264 encoder.
Edit: I emphasize this mainly because the terms have a specific meaning in standards jargon but also because it places the blame for software patent abuses on the wrong parties (the standards developers rather than the lawyers and legislators).
Uh, anyone familiar with the MPEG process will assure you that the companies involved love (let me restate that: PREFER) to bring in technology on which they own the patents so they get a good cut of the resulting patent pool.
Sometimes this is even done even though it technically makes no sense. Best example: hybrid filter-bank in MP3.
The process also provides no protection or discouragement from patents from semi-involved industry partners appearing later on, etc.
This difference in approach is a stark contrast to the IETF, which is why Opus work, and future AV1 work are happening under the IETF rather than the MPEG groups.
Of course, this also only applies in countries which enforce software patents.
Yes, I've dumbed it down in the article to a simple circle to illustrate the point.
Previously Daala was presented as a candidate for NETVC but apparently this didn't go anywhere? https://en.wikipedia.org/wiki/NETVC
[1] http://cs.stanford.edu/people/eroberts/courses/soco/projects...
Neural nets are always expensive to train. You'd better be getting something from them that you can't get some other way.
With video compression, I think most would agree that there might be better architectures/algorithms that we haven't stumbled upon yet. Whether specifically "neural networks" will be the shape of a better architecture, I don't know. But almost surely some meta-algorithm that can try out tons of different parameters/data-pipeline-topologies for something that vaguely resembles h.264 might find something better than h.264.
Neural nets are expensive to train. But so is designing h.264.
I was under the impression that the first 100,000 units are free, and then 20c per unit afterwards to a max of $25m.
H264 drops to 10c per unit after 5m units, to a max of $6.5m.
You need to be shipping 125 million units annually to hit the full $25m.
Yes it's more, but it's not quite ten times. And notably if the chip maker pays the royalties, then the content creators don't need to (though that was excepted indefinitely with H264).
Parts regurgitated from a quick google for reference [1]
[1] http://www.theregister.co.uk/2014/10/03/hevc_patent_terms_th...
Said group's demands are basically the reason Netflix started considering VP9.
So it depends how these royalty works in details. If only the chip manufacture are paying, Mediatek, Qualcomm, Samsung, Intel, AMD, Nvidia, Apple. That is at least 10 players paying maximum. And if you consider small players, the total contribution of Royalty fees to HEVC is 1 Billion / Year. ONE BILLION!! In the life time of a Video Codec that typically run at least a decade, these patents are 10 Billions.
Do you think that is a fair price, i think everyone should decide for their selves.
This is a great overview and the techniques are similar to those of h264.
I found it invaluable to get up to speed when I had to do some work on the screen content coding extensions of hevc in Argon Streams. They are a set of bit streams to verify hevc and vp9, take a look, it is a very innovative technique:
http://www.argondesign.com/products/argon-streams-hevc/ http://www.argondesign.com/products/argon-streams-vp9/
Ehm, what?! The image on the right looks really bad and the missing holes was the first thing I noticed. No zooming needed.
And that's exactly my problem with the majority of online video (iTunes store, Netflix, HBO etc). Even when it's called "HD", there are compression artefacts and gradient banding everywhere.
I understand there must be compromises due to bandwidth, but I don't agree on how much that compromise currently is.
Isn't the images above the text a zoomeed version?
>Here is a close-up of the original...
However if that MacBook Pro image was placed on the side of an article where the primary content was the text you were reading, you'd glance at the image and your brain would fill in the details for you. You probably wouldn't notice the difference in that context.
For most use cases, there likely is very little functional difference between the two images. At least, that was how I understood it.
Although to be fair, I suspect that a lot of times what I'm looking at are mpeg videos that have been recompressed a half dozen or more times with different encoders. Each encoder having prioritized different metrics. So, the the quality gets worse until it doesn't really matter how good the compression algorithm is. Each new re-compression is basically spending 3/4 of its bits maintaining the compression artifacts from the previous two passes.
I mean for fast action scenes, I rarely notice the difference between 720p and 1080p at 10ft away... but different encoding and sources, not just size alone can make significant differences.
No he didn't explain "right after that." He rambled on and on, and even after all of that, he STILL doesn't bring up JPG.
It's an inherently stupid comparison to make. You can't polish a turd.
> This concept of throwing away bits you don't need to save space is called lossy compression. H.264 is a lossy codec - it throws away less important bits and only keeps the important bits.
> PNG is a lossless codec. It means that nothing is thrown away. Bit for bit, the original source image can be recovered from a PNG encoded image.
b) Leave codec packs in 2000 where they belong. They are a great malware vector and also good at messing with settings they shouldn't.
>KCP utilizes the following components: MPC-HC - A robust DirectShow media player. madVR - High quality gpu assisted video renderer. Included as an alternative to EVR-CP. xy-vsfilter / XySubFilter(future) - Superior subtitle renderer. LAV-Filters - A package with the fastest and most actively developed DirectShow Media Splitter and Decoders. (Optional) ReClock - Addresses the problem of audio judder by adapting media for smooth playback OR utilized for bit perfect audio.
I'm actually using MPC-HC and AC3Filter to deal with some files where I couldn't hear the centre channel on VLC (on stereo speakers). Everything else isn't really needed.
(Though for images it's in 2D, not 1D which is more commonly done)
Along with the Wikipedia article and the obvious Internet search, there's a lot of good stuff that has been on HN: https://hn.algolia.com/?query=fourier%20transform&sort=byPop...
A good way to develop an intuition for the fourier space is to look at simple images and their DFT transforms: http://web.cs.wpi.edu/~emmanuel/courses/cs545/S14/slides/lec... (3/4 of the way through the slide deck).
This analysis of a "bell pepper" image and its transform is also helpful: https://books.google.com/books?id=6TOUgytafmQC&pg=PA116&lpg=....
As for why you want to do this: throwing away bits in the spatial domain eliminates distinctions between similar intensities, making things look blocky. In the frequency domain, however, you can throw away high-frequency information, which tends to soften patterns like the speaker grills in the MBP image that the human eye isn't that sensitive to to begin with.
Or in this case, a real data set.
The JPEG article also has a very good, step by step example of the DCT, followed by quantization and entropy coding: https://en.wikipedia.org/wiki/JPEG
A video on youtube led me to Joofa Mac Photoshop FFT/Inverse FFT plugins [1] which was worth a try. I was unable to register it, as have others. Then I came across ImageJ [2], which is a really great tool (with FFT/IFFT).
Edit: if anyone checks out ImageJ, there's a bundled app called Fiji [3] that makes installation easier and has all the plugins.
If anyone has other apps/plugins to consider, please comment.
[1] http://www.djjoofa.com/download
[2] https://imagej.nih.gov/ij/download.html
[3] http://fiji.sc/
https://github.com/0x09/dspfun