QSV AV1 encoding quality poorer than QSV HEVC, with low-latency settings #375
Description
I've been evaluating which is best, NVENC or QSV particularly for AV1, and I setup some tests using the command line utilities QSVenc and NVenc.
My aim was to find out how they compare for AV1/HEVC, across multiple bitrates, with a particular focus on tuning for lowest latency. To compare, I used a 30s sports clip, 1080p50 10-bit, with VMAF used to compare the encoding results.
I have found that for AV1, NVENC is clearly superior, whereas for HEVC QSV wins. Counterintuitively, QSV AV1 is significantly poorer than QSV HEVC.
Preparing source for duration 30s
Transcoding source...
Performing comparison @ 1000 kbps
Encoding AV1 NVENC...
Calculating VMAF...
VMAF mean: 80.093806 frames: 1500
Encoding AV1 QSV...
Calculating VMAF...
VMAF mean: 77.161206 frames: 1500
Encoding HEVC NVENC...
Calculating VMAF...
VMAF mean: 74.353500 frames: 1500
Encoding HEVC QSV...
Calculating VMAF...
VMAF mean: 78.154373 frames: 1500
Performing comparison @ 2000 kbps
Encoding AV1 NVENC...
Calculating VMAF...
VMAF mean: 90.399081 frames: 1500
Encoding AV1 QSV...
Calculating VMAF...
VMAF mean: 86.942477 frames: 1500
Encoding HEVC NVENC...
Calculating VMAF...
VMAF mean: 86.989102 frames: 1500
Encoding HEVC QSV...
Calculating VMAF...
VMAF mean: 88.506360 frames: 1500
Performing comparison @ 3000 kbps
Encoding AV1 NVENC...
Calculating VMAF...
VMAF mean: 94.576788 frames: 1500
Encoding AV1 QSV...
Calculating VMAF...
VMAF mean: 91.642781 frames: 1500
Encoding HEVC NVENC...
Calculating VMAF...
VMAF mean: 92.427293 frames: 1500
Encoding HEVC QSV...
Calculating VMAF...
VMAF mean: 93.286833 frames: 1500
Performing comparison @ 4000 kbps
Encoding AV1 NVENC...
Calculating VMAF...
VMAF mean: 96.523973 frames: 1500
Encoding AV1 QSV...
Calculating VMAF...
VMAF mean: 94.433429 frames: 1500
Encoding HEVC NVENC...
Calculating VMAF...
VMAF mean: 95.398390 frames: 1500
Encoding HEVC QSV...
Calculating VMAF...
VMAF mean: 95.962763 frames: 1500
Performing comparison @ 5000 kbps
Encoding AV1 NVENC...
Calculating VMAF...
VMAF mean: 97.615519 frames: 1500
Encoding AV1 QSV...
Calculating VMAF...
VMAF mean: 96.119752 frames: 1500
Encoding HEVC NVENC...
Calculating VMAF...
VMAF mean: 97.052344 frames: 1500
Encoding HEVC QSV...
Calculating VMAF...
VMAF mean: 97.335620 frames: 1500
All tests complete.
Here is my batch file:
@echo off
mkdir "outputs" 2>nul
mkdir "outputs\metrics" 2>nul
set "DURATION=30"
set "FPS=50"
set /a FRAMES=%DURATION% * %FPS%
setlocal enabledelayedexpansion
echo --------------------------------------------------
echo Preparing source for duration !DURATION!s
echo --------------------------------------------------
echo Transcoding source...
ffmpeg -hide_banner -y -i "football.mp4" -t !DURATION! -c:v libx265 -pix_fmt yuv420p10le -preset veryfast -crf 0 -c:a copy "outputs\orig.mp4" > "outputs\metrics\orig_ffmpeg.log" 2>&1
for %%b in (1000 2000 3000 4000 5000) do call :run_test %%b
echo All tests complete.
endlocal
exit /b 0
:run_test
set "BITRATEK=%~1"
set "BITRATE=!BITRATEK!k"
echo --------------------------------------------------
echo Performing comparison @ !BITRATEK! kbps
echo --------------------------------------------------
echo Encoding AV1 NVENC...
NVEncC64.exe --input "outputs\orig.mp4" --codec av1 --output-depth 10 --cbr !BITRATEK! --preset quality --gop-len 999999 --tune ultralowlatency --output "outputs\av1_nvenc_!BITRATEK!k.mp4" > "outputs\metrics\av1_nvenc_!BITRATEK!k_nvencc_log.txt" 2>&1
echo Calculating VMAF...
ffmpeg -hide_banner -y -i "outputs\orig.mp4" -i "outputs\av1_nvenc_!BITRATEK!k.mp4" -lavfi "[0:v][1:v]libvmaf=log_fmt=json:log_path=outputs/metrics/av1_nvenc_!BITRATEK!k_vmaf.json:n_threads=8" -f null - > "outputs\metrics\av1_nvenc_!BITRATEK!k_vmaf_ffmpeg.log" 2>&1
if exist "append_vmaf_summary.ps1" powershell -NoProfile -ExecutionPolicy Bypass -File "append_vmaf_summary.ps1" "outputs\metrics\av1_nvenc_!BITRATEK!k_vmaf.json" "outputs\metrics\av1_nvenc_!BITRATEK!k_vmaf_ffmpeg.log"
echo Encoding AV1 QSV...
QSVEncC64.exe --input "outputs\orig.mp4" --codec av1 --output-depth 10 --cbr !BITRATEK! --async-depth 1 --quality best --bframes 0 --gop-ref-dist 1 --gop-len 999999 --output "outputs\av1_qsv_!BITRATEK!k.mp4" > "outputs\metrics\av1_qsv_!BITRATEK!k_qsvencc_log.txt" 2>&1
echo Calculating VMAF...
ffmpeg -hide_banner -y -i "outputs\orig.mp4" -i "outputs\av1_qsv_!BITRATEK!k.mp4" -lavfi "[0:v][1:v]libvmaf=log_fmt=json:log_path=outputs/metrics/av1_qsv_!BITRATEK!k_vmaf.json:n_threads=8" -f null - > "outputs\metrics\av1_qsv_!BITRATEK!k_vmaf_ffmpeg.log" 2>&1
if exist "append_vmaf_summary.ps1" powershell -NoProfile -ExecutionPolicy Bypass -File "append_vmaf_summary.ps1" "outputs\metrics\av1_qsv_!BITRATEK!k_vmaf.json" "outputs\metrics\av1_qsv_!BITRATEK!k_vmaf_ffmpeg.log"
echo Encoding HEVC NVENC...
NVEncC64.exe --input "outputs\orig.mp4" --codec hevc --output-depth 10 --cbr !BITRATEK! --preset quality --gop-len 999999 --tune ultralowlatency --output "outputs\hevc_nvenc_!BITRATEK!k.mp4" > "outputs\metrics\hevc_nvenc_!BITRATEK!k_nvencc_log.txt" 2>&1
echo Calculating VMAF...
ffmpeg -hide_banner -y -i "outputs\orig.mp4" -i "outputs\hevc_nvenc_!BITRATEK!k.mp4" -lavfi "[0:v][1:v]libvmaf=log_fmt=json:log_path=outputs/metrics/hevc_nvenc_!BITRATEK!k_vmaf.json:n_threads=8" -f null - > "outputs\metrics\hevc_nvenc_!BITRATEK!k_vmaf_ffmpeg.log" 2>&1
if exist "append_vmaf_summary.ps1" powershell -NoProfile -ExecutionPolicy Bypass -File "append_vmaf_summary.ps1" "outputs\metrics\hevc_nvenc_!BITRATEK!k_vmaf.json" "outputs\metrics\hevc_nvenc_!BITRATEK!k_vmaf_ffmpeg.log"
echo Encoding HEVC QSV...
QSVEncC64.exe --input "outputs\orig.mp4" --codec hevc --output-depth 10 --cbr !BITRATEK! --async-depth 1 --quality best --bframes 0 --gop-ref-dist 1 --gop-len 999999 --output "outputs\hevc_qsv_!BITRATEK!k.mp4" > "outputs\metrics\hevc_qsv_!BITRATEK!k_qsvencc_log.txt" 2>&1
echo Calculating VMAF...
ffmpeg -hide_banner -y -i "outputs\orig.mp4" -i "outputs\hevc_qsv_!BITRATEK!k.mp4" -lavfi "[0:v][1:v]libvmaf=log_fmt=json:log_path=outputs/metrics/hevc_qsv_!BITRATEK!k_vmaf.json:n_threads=8" -f null - > "outputs\metrics\hevc_qsv_!BITRATEK!k_vmaf_ffmpeg.log" 2>&1
if exist "append_vmaf_summary.ps1" powershell -NoProfile -ExecutionPolicy Bypass -File "append_vmaf_summary.ps1" "outputs\metrics\hevc_qsv_!BITRATEK!k_vmaf.json" "outputs\metrics\hevc_qsv_!BITRATEK!k_vmaf_ffmpeg.log"
goto :eof
Tests were on done on a Windows 11 machine with a discrete RTX 5050, and integrated ARC140T.
Arc driver version is 32.0.101.8247
Nvidia driver version is 581.80
NVEnc version 9.0.7
QSVenc version 8.0.3
I'm curious if there is any plans to improve the quality of the AV1 encodes in particular on the ARC hardware, for low-latency scenarios, the above results (worse than QSV HEVC) seem disappointing.