FFMPEG from Zero to Hero (ffmpegfromzerotohero.com)

It would be hard to be expressive enough considering there’s a lot of exceptions where things don’t really quite work.

eg copying without conversion between different formats isn’t really possible when it looks like it should be, because there’s a lot of incorrect handling of timestamps both in libavformat and in files themselves.

If you don't perform a task frequently, a discoverable interface (e.g. GUI) is dramatically more productive than reading documentation.

There's a continuum, and yes that applies to human languages, both natural and constructed, and to other skills. But where you can choose (so, not natural languages really) you can design things to make this easier, or, I guess, harder.

For example English orthography is a horrible mess. The inventory of squiggles needed to write English isn't too bad, but the correspondence between the words you know and the correct sequence of squiggles to write them is unnecessarily complicated. No benefit accrues to us from this, and in some other written languages it's much easier.

I'd place ffmpeg somewhere in the not-bad but not-great part of the continuum. As with English orthography of course the problem is if you change things with the intent to make them better you actually introduce a cost for existing users which may be impossible to sustain.

You don’t lose it completely. It rusts over, but I had occasion recently to resurrect my 6502 assembler, and it was still in muscle memory.

Kinda like running into an old friend you haven’t seen in decades.

> wouldn't that apply to anything you don't use regularly?

Doesn't apply to well-designed GUI programs. In good GUIs features are discoverable, you don't need prior knowledge or documentation to use software.

Experience speeds things up, e.g. keyboard shortcuts are faster than menus or buttons. That's optional though, one can still use the software without these shortcuts.

I have a $HOME/bin folder full of bash scripts invoking imagemagick or ffmpeg one liners.

  #!/bin/bash

  size=${1:-"1920x1080"}
  offset=${2:-"0,0"}
  name=${3:-"video"}

  ffmpeg -video_size $size -framerate 25 -f x11grab -i :0.0+$offset -c:v libx264 -crf 0 -preset ultrafast "$name.mkv"
  ffmpeg -i "$name.mkv" -movflags faststart -pix_fmt yuv420p "$name.mp4"

    echo ‘!!’ > ~/bin/name-of-script

I have shortcuts to save last (sl) executed command and another to grep that file.

    alias sl='fc -ln -1 | sed "s/^\s*//" >> ~/.saved_commands.txt'
    alias slg='< ~/.saved_commands.txt grep'

You can also add a comment to the command before saving. For ex:

    $ foo_cmd args #this command does this awesome thing
    $ sl

At this point I'd prefer it if ffmpeg included Lua and allowed you to pass Lua scripts, kinda like what ZFS did[1].

[1]: https://zfsonlinux.org/manpages/0.8.3/man8/zfs-program.8.htm...

    The entire script is executed atomically, with no other administrative operations taking effect concurrently. 

     If a fatal error is returned, the channel program may have not executed at all, may have partially executed, or may have fully executed but failed to pass a return value back to userland. 

I find that ffmpeg's regular command lines are easier to mentally parse if I include additional \ to separate things out onto their own lines. That's what I've done in several shell scripts that do complicated things with ffmpeg, along with comment lines and echoing commentary on what it's doing to the terminal.

ffmpeg-python is great! easier to construct complex commands, and read / edit later

I’ve heard this a few times. I’m the opposite, I think I learned it once and never forgot it, it seems impossible. Do you understand regex in terms of a state machine? Draw it out. It’s a very simple language.

So machine learning

Please elaborate, is this a work or hobby project, which space craft?

The GPU encoders aren't GPGPU programs, but just hardware encoders that happen to be on the GPU. The reason they're worse is because the algorithms they use are worse than x264, which had much more development time and better developers. (who have since retired to become professional cosplayers.)

I haven't used OBS, but OBS is more a "producer" app, not very consumer oriented, so it mught be a pain to setup it you just want to layer to independent streams at once for a one time thing

I think that a process of clicking a link (meshstreams.io/streamer1,streamer2) would let you download a script that would install FFMPEG and open both streams in VLC

I don’t know of any book like that. I have a CinemaDNG processing workflow: create a camera/lens profile using a ColorChecker and DCamProf, batch develop DNGs using RawTherapee into 16-bit TIFFs, export TIFF sequence as a movie using DaVinci Resolve (30 FPS if delivering to popular services like Instagram, where 24 FPS causes subtle stuttering). I’ve been meaning to script this all, Resolve can be easily replaced with ffmpeg, but didn’t get around to that yet. For basic VFX I’ve used Blender (which can track the scene allowing to incorporate 3D objects, and can act as a decent NLE) but again not in a scripted way. It’s just personal projects so there’s not enough pressure to automate.

This is the only time I'll admit this, but my favorite book on FFMPEG is called google.com

Just like the rest of the internet, there's probably very little that you could do right now, that someone else has not already done or struggled with as well. Most of the time, it's reading about what people have tried that did not work before which is still incredibly useful in and of itself. It sucks while your under pressure of a deadline, but you eventually you just sort of "get it".

The trick is, you have to use it frequently, and not just every now and then when some task comes up. But that's no different than any other tool. Yes, the commands can get harry and scary looking, but so can SQL statements.

That's indeed true - and why for subjective eyeball moving picture tests you should also do things like cut out a 20 second piece of a raw/y4m file, encode it with various x265 settings, and then put those 4 or 5 pieces in a VLC playlist so you can quickly compare them in series.

For objective comparisons there is of course VMAF, which is an essential tool for doing automated comparisons of videos vs their uncompressed original. One of the reasons why VMAF was created is that subjective eyeball evaluation of codecs (whether still or moving) will vary between person to person, and is very labor intensive.

But still image PNGs of frames also serve a useful purpose when dealing with lower bitrate videos, for your personal opinion comparison of blockiness, color banding, blobs of color in areas that are mostly the same color, etc.

But why is that multi-step process even necessary?

There should be an quick command-line utility to concatenate multiple video files according to exactly the timestamps the user has provided. It's such a common operation.

There's no reason that the tool can't simply do a streaming decode of multiple different file formats and concatenate the video and sub-second precision. If input video resolutions are different, scaling the smaller video to the largest resolution is what the user almost always wants.

I get that FFMPEG is a "plumbing" CLI tool, but a "porcelain" wrapper would be amazing!

I battled with this a lot with https://github.com/umaar/video-everyday and still haven't found a better solution.

What I don't understand is, how can professional video editing tools trim accurately (and very quickly)? What are they doing differently to ffmpeg?

If do things the "fast way" with ffmpeg, the exported video has random black frames which I think is related to the keyframe issue you mention. If I do things the "slow way" (e.g. accurately) with ffmpeg, it takes a huge amount of time (at least with large 4k videos). But I don't understand how I can drop that same 4k video into Screenflow, trim 1 second out of it and export it in a matter of seconds.

Yeah, it is literally not possible to not seek to keyframe if you are stream copying.

There are however, plenty of great software that can but at any frame and only re-encode the frames that are outside the whole GOP. Most of them are commercial though, I haven't find one that is free and good.

----

Also, seeking in FFMPEG in practice, is actually more complicated than the guide [1] you linked. Below is a note I keep for own reference for keyframe-copy. Hope someone will find it useful.

How to keyframe-cut video properly with FFMPEG

FFMPEG supports "input seeking" and "output seeking". The output seeking is very slow (it needs to decode the whole video until the timestamp of your -ss) so you want to avoid it if unnecessary.

However, while -ss (seek start) works fine with input seeking, "-to/-t" (seek ending) is somehow vastly inaccurate in input seeking for FFMPEG. It could be off by a few seconds, or sometimes straight up does not work (for some mepeg-ts files recorded from TV).

The best of the two worlds is to use input seeking for -ss and then output seeking for -to. However, this way, the timestamp will restart from 0 in output seeking. So instead of using -to, you should calculate -t (duration) yourself by subtracting -ss from -to, and use `-t duration` instead. Below is a quick Python script to do so.

https://gist.github.com/fireattack/9a100c5a200154937babd1823...

(You can also try to use -copyts to keep timestamp, but not recommended because it doesn't work if the video file has non-zero start time.)

the "video itself" doesn't really exist. If you're on Windows and use MPC, you can test this yourself - use WMV9 as a renderer, take a PNG screenshot, then switch to NVENC, and take a screenshot at the same timestamp. You'll notice that on most videos, the screenshots are not the same (with NVEnc introducing macroblocking in darkness, which WMV9 gradients out properly). Using the rendered video as a source of truth instead of the source material itself would be as big of a mistake as using Photoshop on a JPEG.

It's ffmpeg's fault that the correct implementation is overly complex. You will think that your code is correct, but actually there is a flag somewhere you have to set, or you have to copy something from one buffer to another. Due to the drastically large input space of different codecs and settings for each codec and how corruption is handled can make it hard to test. Instead of there being a simple API, there are all sorts of things you just have to know you should do. The examples given by the ffmpeg project are not enough. It's a case of there being many unknown unknowns.

FFMPEG's API is far from perfect. In order to work with a video in a memory buffer you literally have to write part of a virtual filesystem driver. That is not perfect API design. You should just let me pass in a memory buffer and length. I am not really bothered with how verbose it can be since it is rather low level. What I am bothered by are undocumented pitfalls in demuxing, decoding, and handling color spaces. Ideally their should be an API that handles taking care of all the pitfalls correctly or there should be documentation explaining the pitfalls and how to avoid them.