Seems like an effort to stick video en/decoding behind graphics drivers as a vendor neutral approach instead of relying on VDPAU or VA-API. These efforts should equally apply to Windows and Android as well as Linux.
Unless targeting flagship devices, most likely OpenGL ES drivers are more stable than Vulkan.
Hence why OpenGL ES on Vulkan is now a thing since Android 14, that is the only way Google has found out to force OEMs to up their game on Vulkan drivers.
It basically does seem to replace them, albeit only for more modern codecs (which should be OK; I suspect nobody really cares about MPEG-2 acceleration anymore.) Like others are pointing out, it is cross-platform and vendor-neutral.
I am curious to see how this evolves over time. Not sure if there are Vulkan Video implementations for things that are not GPUs yet; if we start to see them, this interface may eventually also wind up replacing v4l2-based codec acceleration, too. With increasingly robust support for Vulkan Video in FFMPEG it's seeming likely that it will soon be the defacto way of deciding and encoding modern codecs using hardware acceleration on Linux, at least with GPUs.
I'm afraid I don't know enough about hardware-accelerated video encoding/decoding to really understand what this implies. From what I gather, the Vulkan API gives you the ability to queue video encode/decode operations, which seem to pertain to e.g. a frame of h264 video, passing in all of the required state. So I assume at a low level, how a driver would have to handle this is by processing these commands, dispatching units of work to relevant hardware units, flipping registers and etc. as needed, and doing whatever can't be done in hardware in software. And I guess what you mean when you say Qualcomm's video acceleration is "stateful", it means that you can't just simply dispatch a single-frame encode job, the video encoding process is some stateful thing where the per-codec state is not all exposed and cannot be processed out-of-order, so there's no logical way to implement a Vulkan driver.
If so, that's a bummer.
I wonder if it might be possible to "cheat" by having a driver that tries to use stateful acceleration in the happy path, but falls back to software encoding/decoding when things go out of order or there are no more hardware resources. Probably not. At least, hopefully, future hardware designs will account for this. Until those existing devices are outside of their useful lifespan, though, it seems Vulkan Video will probably not be the one hardware video coding API to rule them all.
> And I guess what you mean when you say Qualcomm's video acceleration is "stateful", it means that you can't just simply dispatch a single-frame encode job
Does this essentially bring AMD video enc/dec up to scratch with Intel's QuikSync?
Thats long been one of my reasons for having a home server run on Intel, and if that moat is now crossed then there is basically no reason for me to go Intel in the future.
15 comments
[ 3.0 ms ] story [ 45.6 ms ] thread> Great to see this milestone for better exposing this cross-vendor, multi-platform open video encode/decode API.
The others were designed separately by each vendor. https://wiki.archlinux.org/title/Hardware_video_acceleration feels like a decent enough, brief explainer.
The lack of AMD GPUs in Android devices?
Hence why OpenGL ES on Vulkan is now a thing since Android 14, that is the only way Google has found out to force OEMs to up their game on Vulkan drivers.
I am curious to see how this evolves over time. Not sure if there are Vulkan Video implementations for things that are not GPUs yet; if we start to see them, this interface may eventually also wind up replacing v4l2-based codec acceleration, too. With increasingly robust support for Vulkan Video in FFMPEG it's seeming likely that it will soon be the defacto way of deciding and encoding modern codecs using hardware acceleration on Linux, at least with GPUs.
Qualcomm video acceleration is stateful, and as such doesn't fit with the design of the Vulkan Video API (which is stateless)
If so, that's a bummer.
I wonder if it might be possible to "cheat" by having a driver that tries to use stateful acceleration in the happy path, but falls back to software encoding/decoding when things go out of order or there are no more hardware resources. Probably not. At least, hopefully, future hardware designs will account for this. Until those existing devices are outside of their useful lifespan, though, it seems Vulkan Video will probably not be the one hardware video coding API to rule them all.
Yup it's exactly that
Thats long been one of my reasons for having a home server run on Intel, and if that moat is now crossed then there is basically no reason for me to go Intel in the future.
It would have been enough to just stop amdgpu crashing my system all the time.