I can't see any configuration in that repo for testing different asm variants.
There's cpu = 'i686' in the config file which would in fact seem to imply none of the modern archs mentioned in a different subthread here ("x86_64(SSE3, AVX2), ARMv7 and ARMv8") are fuzzed.
That's for x86_64(SSE3, AVX2), ARMv7 and ARMv8 right? Is the 186kLoC equally distributed between those arch or is there one arch that has most of the optimization focus?
Is it all hand-written, or do you have some kind of script/macros to create a good part of this assembly (like OpenSSL does IIRC)?
No. Most encoders/decoders need assembly to get last drop of performance. Most of the times it comes down to when you give up. The team here was quite relentless to build a usable AV1 decoder. A few years ago this was a pipe dream and jbk and team have done a tremendous job.
"[..] code has now around 186 kLoC of hand-written assembly in .S and .asm files…"
This reads to me as it being a PITA to maintain. Cross platform code is usually a pain, cross platform with assembly optimizations is more of a pain. Optimization nearly always makes things harder to maintain, and this sounds like it was optimized to hell and back...
In addition to other noting that decoders and encoders almost all have an enormous amount of hand optimized code because they are literally the most demanding applications most people run (which is why most codecs today are hardware accelerated and they are used in almost all CPU benchmarks) 186k LOC (given that it's assembler which is nearly always "vertical code") is also relatively small for decoders.
People don't really get how complex video codecs are.
It's not they're writing a library full of assembly for the sake of it. Reading the readme, one can discover they have dedicated optimised implementations for a large range of processors, which in turns means there's a lot of assembly involved
Using the Netflix 1080P AV1 2Mbps Test clip, my old Macbook 2015 with a Intel Core i5-5287U ( Broadwell Dual Core 3.3Ghz ) just manage to play that at about 70% of the total CPU.
>We have much better results with similar hardware...
I should have included in the original comment this machines needs 60% to 70% CPU to play VP9 on similar bitrate / bit / frames. i.e Decoding on VP9 and AV1 are on a similar level of CPU usage. Considering the complexity of AV1, I thought this was an incredible achievement. ( It actually blows my mind this can be done )
I tested it on with 8 and 10 bit content the result aren't that much different. ( This is just eyeballing CPU usage in Activity Monitor ) The 3.3 Ghz was turbo, so it really should be 2.9Ghz Dual Core Broadwell CPU.
I use either aom for trying to make "optimized" encodes of things to get quality at the smallest file size or svt for fast encodes that aren't really any more efficient than h265, just with the better codec.
I just transcoded some movies with vmaf of 98% from 4k hdr blurays this last week. Takes about two days of mostly single core work with cpu-used=4 on a 5950x but they get average bitrates of around 4400 kbps. Which is like, really, really good for hitting that quality target.
The trick is that since its mostly single core I just do 16 movies at a time, or 24 on my server.
Best way to encode video is to use a scene detect algorithm to chunk up the video into multiple parts, configure your encoder to encode those parts with a single CPU thread, and spin off one thread per scene to your CPU's max amount of threads. You then just concat the scenes together afterwards.
SVT-AV1 has massively improved encoding speed. I was getting ~4x encode speed with some heavy compression (barely any noticeable quality drop) on 1080p/24fps footage on my Ryzen 7 3700X just yesterday.
For cpu I don't know but h265 hardware acceleration ASICs are mainstream contrary to AV1. Also h265 can reach higher compression level than AV1 although the difference is relatively small apparently.
However technologically AV1 is largely obscolete since h266 stable has been released.
Every study I've seen shows AV1 beats H.265 on VMAF at all bitrates. H.265 never got significant browser support, and I don't expect H.266 to either. https://caniuse.com/hevc
I have seen a scientific paper a few weeks ago stating h265 was ~7% smaller than av1 on tested dataset. Although maybe it's dataset sensitive or differ because of the metric? (maybe it was PSNR or SSIM?)
Anyway any data on AV1 hardware decoding energy consumption vs h265? That is probably the main differentiating metric of interest here
svtav1 is the best I've found, personally. Unlike libaom, it's parallelized, which obviously helps massively. I use it at quality profile 20 (as given to ffmpeg params), but I'm not an expert here.
Alder lake's AVX-512 comes with a significant asterisk. It's only in the P-cores (so can only be turned on if you disable the E-cores), isn't officially supported, and can't be turned on in some motherboards IIRC.
We should be seeing better AVX-512 support with CPUs in the coming years though.
According to that link, Zen 4 will only have a very limited kind of AVX-512 instructions, for working with half-precision floats. It won't even have the base AVX-512 instructions (the F set).
Icelake and tigerlake. On AWS, this is m6i/c6i. [edit] earlier AWS machines (c5/m5) have AVX512 also, but not the subsets required by dav1d's assembly.
libdav1d when it was included in MPV was such an incredible experience. Prior to, my QX9300 Thinkpad didn't manage double digit fps with anything AV1 regardless of bitrate. The love and attention put into supporting older Vector extensions is insane and suddenly it managed playback of 8bit content without framedrops. Though I'm sure anything high-bitrate will smother that QX9300.
I'm thinking about buying the latest Roku Ultra because it says it support AV1 media format. Does it have a hardware decoder or how does it manage to do it when even my nvidia Shield Pro doesn't direct play av1 content?
AV1 is for h265 and now h266 is "ready", I guess AV2 should see the light soon, shouldn't it?
In the future, once mainstream CPUs cannot software decode in realtime compressed video/audio and only a specialized hardware block will be fast enough, will it be a "patent encumbered" format with tons of royalties?
I was refering to the future: when the silicium will have hit its "best" (in a mass production context) and the video/audio compressed format of this future would not be able to be "software decoded" in real time by such "best silicium" CPUs: only a specialized hardware block would be able to decode such video/audio compressed format in real time.
The danger is this block only implementing the decoding of a non-royalty free/patent encumbered format à la mpeg, presuming the "intellectual property" was not globally fixed and still toxic like nowdays.
I think that for most of these libraries software-only performance is a moot point: the library is there for other software to use to be able to parse the files, and the library/decoder will simply offload the heavy compute to the (hopefully eventually-available) decoding hardware instructions.
To a certain extent it's just a place holder so folks can start writing software against it so that when the CPU vendors ship everyone can hit the ground running.
I'll try to dig up the slide where they demonstrated it, but basically faster at equal quality than any software decoder of any codec.
Which is cool, but if you have hardware decode then you may not care, though software decode gets used in all sorts of weird places that you might not expect.
Yeah, the key is that newer generations of codecs use more time to encode, but often the decode time goes up less than the encode time. Combine that with greater design for parallelism and you find yourself in the slightly counterintuitive situation where decoding the same quality of video is faster in the newest codecs (at the one time cost of longer encodes).
I dont think it was one single slide I was thinking of but you can for example find comparisons of vp8, vp9, h264 by the same videolan/ffmpeg devs that demonstrate this general principle, the decode fps is surprisingly flat between software decodes of different codec families and generations.
The earlier generations often hit bottlenecks though, I think ffmpeg VP9 decode (built by the same people as dav1d) was the leader for a while but hits a wall at 4 cores due to format limitations. HEVC and especially dav1d have some gains when you can throw a modern amount of cores at it, and even phones often have 8.
Depends on SoC. A15? or low end UniSoC SoC? But generally speaking a modern Smartphone should be able to play 1080P 25fps 1mbps AV1 video on their phone without dropping frames. You will just be burning away your battery life.
It's bad. As a reference, I tried decoding a measly 2 mbit/s test clip on an older Core 2 Duo MacBook Pro (which has no issues playing back the same in H.264 purely with software decoding), and it stuttered a lot. I don't see this thing taking off until there's decent GPU acceleration in place. "Fast decoder" is highly relative.
Yes, most are. That Core 2 Duo machine is the weakest hardware I have that I know can decode H.264 lower bitrate FullHD content in software. It made for a good benchmarking baseline.
And those machine cannot decode H265 in software… Sorry but you cannot say dav1d software decoding is bad on phones because some hardware from 15 years ago who can barely decode H264 cannot decode it.
As all software decoders… So you compare software to hardware decoders?
H265 or VP9 in software are the same… Here, we managed to have a sw decoder for a format that is one order of magnitude more complex than those codecs and yet, consume less CPU.
So is an H.264/H.265 video clip at 5 or even 10 mbit/s on my 4 years old iPhone, at about 2 watts of power consumption instead of 35 watts. Can you see the advantage?
I like open and free standards, and I dislike royalties, but the political aspect is best foregone when one of the alternatives really is objectively so much better for everyone.
One of the alternatives is clearly not objectively so much better for everyone. If they were so much better then VP9 and AV1 wouldn't exist. Here's what video streamers say:
There are no emotions involved here, with the exception of your childish choice to try misconstrue my argument. I consider the MPEG family as objectively better on the technical merit of GPU/VPU hardware decoding being available everywhere - it's fast and it's energy efficient.
You're very stubborn with telling me what my position and my own arguments are. I'll give you the benefit of the doubt and presume you're not just looking for people to rile up, but rather have the bad habit of leaping ahead and making other people's arguments and stances up on your own to better fit your desired counterpoint, so I'll clarify: I am certainly not against new development or progress, and I don't think that it's pointless to put an effort into AV1 or VP9. As I wrote elsewhere in this discussion, I just don't see this taking off until there's hardware decoding available, which I fully expect there to be. I'm merely underlining that the MPEG family is the state of the art.
> I just don't see this taking off until there's hardware decoding available
Hardware decoding is available. And dav1d is a very fast software decoder. I don't have AV1 hardware and yet I play back AV1 on YouTube just fine via dav1d.
> I'm merely underlining that the MPEG family is the state of the art.
So you're saying that a given 4k source compressed to Xmbit will look better if it's not downscaled to 1080p before compression, but instead downscaled by the display (I don't believe there are any phones with a 2160 line display)?
What they're saying is that a 1080p source upscaled to 4K and then uploaded to youtube and then downscaled by the client to a 1080p screen will look better than a 1080p upload because it has more bits.
But doesn't YouTube negotiate a "channel sized" rate? It seems like it switches between bitrates based on what you can tolerate. All of this is to say that it seems like a tough-to-nail-down metric to evaluate.
Of course it is, a 1080p video at 10mbit will almost always look better than a 4k video at 1mbit, and a 4k video at 10mbit will almost always look better than a 1080p video at 1mbit.
The far more interesting question is if you accept you have Xmbit to play with, what is better on a given platform (screen size, resolution, viewing situation, how well compression works, how much battery is used in decoding, etc)
And bitrate goes up for 4k tiers. Even if bitrate only doubles for 4x the amount of pixels, that'll probably look better with modern codecs (depending on where in the quantization curve you are), and most services use at least 3x the bitrate for 4k over 2k, across the same codec.
Compressed by what? Are you just talking about videos you record on your phone?
I do not know what phone you have but an iPhone 12 ( as an example ) is under 1300 pixels on the short dimension. So you are not getting much more than 1080 pixels no matter what. It seems any experiential difference would have more to do with compression quality than with resolution.
Speaking for myself, I do not think I could tell the difference between 4K and 1080p on a phone ( on a decent AV1 clip ).
SMPTE suggest that 4k for viewing is only worthwhile for people with 20/20 vision once the viewing angle increases beyond 30 degrees.
I'm currently looking at a 40cm wide screen, I'm 80cm away from it, which is about 30 degrees.
For my phone to fill that amount of space it would have to be about 10cm in front of my eyes due to binocular vision (if I just use one eye it's a bit further
When I watch something on my phone it's typically 50cm away, about 15-20 degrees viewing. 720p is more than enough at that size, 1080p if you have particularly good eyesight.
Unrelated, but I'm curious if console emulators could gain performance by hand-writing certain parts of it in assembly (the JIT recompiler, for example).
As the number of code paths go up, the percentage of run-once code does too. Most findings agree that optimization increasingly tends towards a small number of hot loops which could benefit from hand optimization.
In modern codebases his has the effect of micro-optimization of the "rewrite in assembly" sort mattering a great deal in heavy lifting library code like codecs, CPU emulation, rasterizers, VMs etc, and not at all in other areas. Emulation is a decent candidate in this regard but it also massively benefits from higher level simplifying assumptions(e.g. not polling for I/O changes at the true device frequency). It can be hard to notice differences in emulation quality where high level techniques are applied well.
On old 8-bit architectures it was rather the opposite: you couldn't be high level about anything because you had to design towards micro-efficiency. Therefore most of your problems were solved with very simple data structures and algorithms that heavily favored either space(by recomputing the intermediate results) or time (by precomputing all answers in a LUT), and then given a very tight cycle-shaving, code-golfing implementation.
Apple is a "Founding Member" of Alliance for Open Media (which doesn't really mean they are actually a founding member). I think Apple main issues with VP9 was that it was a Google codec by all means.
Apple already supports FLAC and Opus in macOS and iOS, and VP9 in Safari.
It could be that they are waiting to see if AV1 will actually get a marketshare outside Youtube or not.
...to bad that they (in the usual Apple fashion) don't support it in a standard container which everyone else supports and require you to package it in their special snowflake .caf container, so even though in theory everyone supports Opus you still need to use MP3 if you don't want to deal with having to support multiple formats/codecs at the same time.
VP9 is an open format but can still be encumbered by patents. $20 says Google gave Apple an indemnity agreement for VP9 support for YouTube only which is why VP9 is locked down so hard in iOS. So if Apple enables VP9 and gets sued for patent infringement for their VP9 implementation Google will have to pay for some or all of the legal costs and/or judgements resulting from them.
I'd say they are, but the real issue to Apple is how this or that format will affect their users and how it will portray their products. That is, whether or not they can decode the format without the device running hot, noisy and out of battery in 45 minutes. They like H.264/H.265 because users can decode them with high power-efficiency due to GPU- and dedicated hardware support. This is one of the greater benefits of formats that the whole world accepts and adopts.
What you should really look at on dav1d is the fact that the code has now around 186 kLoC of hand-written assembly in .S and .asm files…
I think this is quite a feat (this is more asm than the whole FFmpeg) and this is very rare those days to write so much asm.