Last revision FFMPEG cedrus264 patched: review and test request

twdragon · October 1, 2021

Hello everyone!

I tried to enable cedrus264 encoder/decoder on my old OrangePI Lite 2 (revision 1 if I am right).

I used stulluk's repository as the base to patch the last FFMpeg revision and to add cedrus264 encoder using the new FFMpeg API. The result is built and tested. Unfortunately, I have had no success: the encoder says:

ffmpeg -f video4linux2 -input_format mjpeg -video_size 1280x720 -i /dev/video1 -c:v cedrus264 -pix_fmt nv12 -f mp4 video.mp4
ffmpeg version N-103904-gb2538ce578 Copyright (c) 2000-2021 the FFmpeg developers
  built with clang version 12.0.1 (https://github.com/llvm/llvm-project.git fed41342a82f5a3a9201819a82bf7a48313e296b)
  configuration: --enable-gpl --enable-nonfree --enable-shared --enable-frei0r --enable-gcrypt --enable-gmp --enable-gnutls --enable-ladspa --enable-libass --enable-libcaca --enable-libdc1394 --enable-libfdk-aac --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmp3lame --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-librtmp --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-opengl --enable-libdrm --enable-vdpau --enable-version3 --enable-libspeex --enable-libxvid --enable-libvpx --prefix=/util/ffmpeg --extra-cflags='-O3 -mllvm -polly -I/util/include -I/util/ffmpeg/include' --extra-ldflags='-L/util/lib -L/util/ffmpeg/lib' --enable-encoder=cedrus264 --cc=clang --cxx=clang++
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59.  9.101 / 59.  9.101
  libavformat    59.  5.100 / 59.  5.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 10.100 /  8. 10.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, video4linux2,v4l2, from '/dev/video1':
  Duration: N/A, start: 17593.181597, bitrate: N/A
  Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1000k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (cedrus264))
Press [q] to stop, [?] for help
[swscaler @ 0x17450b90] [swscaler @ 0x1745e680] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x17450b90] [swscaler @ 0x17496ea0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x17450b90] [swscaler @ 0x174ce7f0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x17450b90] [swscaler @ 0x17506140] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x17450b90] [swscaler @ 0x1753da90] deprecated pixel format used, make sure you did set range correctly
[cedrus264 @ 0x17440360] VE Open error.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

It seems the encoder is not enabled in the Armbian kernel 5.10.60-sunxi64 because the device descriptor is not available.

However, I will be glad to see the code reviews, test results and any other feedback. Sorry, but I am not so familiar with FFMpeg API to send pull requests.

Please visit the FFMPEG fork repository here, the default branch is cedrus264:

https://github.com/twdragon/FFmpeg

woodfe · October 17, 2021

Hi,

I've tested your code on an OrangepiP. It can encode h264 1080p at 28fps.

root@orangepipc:~/FFmpeg# ./ffmpeg -s 1920x1080 -pix_fmt nv12 -f rawvideo -stream_loop -1 -i xilinx_1080p_nv12.yuv -pix_fmt nv12 -c:v cedrus264 -f null -
ffmpeg version N-103905-gafd516bd9c Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-nonfree --enable-gpl --enable-version3 --enable-libx264 --enable-libmp3lame --enable-libpulse --enable-libv4l2
libavutil      57. 7.100 / 57. 7.100
libavcodec     59. 9.101 / 59. 9.101
libavformat    59. 5.100 / 59. 5.100
libavdevice    59. 0.101 / 59. 0.101
libavfilter     8. 10.100 / 8. 10.100
libswscale      6. 1.100 / 6. 1.100
libswresample   4. 0.100 / 4. 0.100
libpostproc    56. 0.100 / 56. 0.100
[rawvideo @ 0x27d72c0] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from 'xilinx_1080p_nv12.yuv':
Duration: 00:00:04.12, start: 0.000000, bitrate: 622080 kb/s
Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12, 1920x1080, 622080 kb/s, 25 tbr, 25 tbn
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (cedrus264))
Press [q] to stop, [?] for help
[VDPAU SUNXI] VE version 0x1680 opened.
Output #0, null, to 'pipe:':
Metadata:
    encoder         : Lavf59.5.100
Stream #0:0: Video: h264, nv12(progressive), 1920x1080, q=2-31, 200 kb/s, 25 fps, 25 tbn
    Metadata:
      encoder         : Lavc59.9.101 cedrus264
frame= 465 fps= 28 q=-0.0 Lsize=N/A time=00:00:18.60 bitrate=N/A speed=1.12x
video:5560kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

twdragon · October 17, 2021

@woodfecould you please to provide a detailed specs of your Orange Pi board and kernel to understand better about how to enable Cedrus VDPAU?

woodfe · October 19, 2021

On 10/17/2021 at 4:58 PM, twdragon said:

Orange

I was following this posts:

and:

https://wvthoog.nl/nanopi-ov5640-camera/

1. decompile dtb to dts

2. modify dts: 1) add reversed-memory, 2) disable cedrus, add cedar_ve

3. compile dts to dtb, reboot

4. /dev/cedar_ve appear

5. run ffmpeg

twdragon · October 19, 2021

Thanks for your answer. I will try again to enable

lunixoid · November 1, 2021

I run it on the Banan Pi M2U (R40 cpu). Despite to the ffmpeg errors I got h.264 encoded video file

Spoiler

ffmpeg -f v4l2 -input_format mjpeg -video_size 1280x720 -i /dev/video1 -c:v cedrus264 -pix_fmt nv12 -f mp4 /tmp/video.mp4
ffmpeg version git-2021-09-15-767a2da72f Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 9.4.0 (Buildroot 2021.02.5)
  configuration: --enable-cross-compile --cross-prefix=/work/buildroot/output/host/bin/arm-buildroot-linux-gnueabihf- --sysroot=/work/buildroot/output/host/arm-buildroot-linux-gnueabihf/sysroot --host-cc=/usr/bin/gcc --arch=arm --target-os=linux --disable-stripping --pkg-config=/work/buildroot/output/host/bin/pkg-config --disable-static --enable-shared --prefix=/usr --enable-avfilter --disable-version3 --enable-logging --enable-optimizations --disable-extra-warnings --enable-avdevice --enable-avcodec --enable-avformat --enable-network --disable-gray --enable-swscale-alpha --disable-small --enable-dct --enable-fft --enable-mdct --enable-rdft --disable-crystalhd --disable-dxva2 --enable-runtime-cpudetect --disable-hardcoded-tables --disable-mipsdsp --disable-mipsdspr2 --disable-msa --enable-hwaccels --disable-cuda --disable-cuvid --disable-nvenc --disable-avisynth --disable-frei0r --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-libdc1394 --disable-libgsm --disable-libilbc --disable-libvo-amrwbenc --disable-symver --disable-doc --enable-gpl --enable-nonfree --enable-ffmpeg --enable-ffplay --enable-libv4l2 --enable-swresample --enable-ffprobe --disable-libxcb --enable-postproc --enable-swscale --enable-indevs --enable-alsa --enable-outdevs --enable-pthreads --enable-zlib --disable-bzlib --disable-libfdk-aac --disable-libcdio --enable-gnutls --disable-openssl --enable-libdrm --enable-libopenh264 --enable-vaapi --enable-vdpau --disable-mmal --disable-omx --disable-omx-rpi --disable-libopencv --disable-libopus --enable-libvpx --disable-libass --disable-libbluray --disable-libmfx --disable-librtmp --disable-libmp3lame --disable-libmodplug --enable-libspeex --disable-libtheora --disable-iconv --enable-libfreetype --enable-fontconfig --enable-libopenjpeg --enable-libx264 --disable-libx265 --disable-libdav1d --disable-x86asm --disable-mmx --disable-sse --disable-sse2 --disable-sse3 --disable-ssse3 --disable-sse4 --disable-sse42 --disable-avx --disable-avx2 --enable-armv6 --enable-vfp --enable-neon --disable-altivec --extra-libs=-latomic --enable-pic --cpu=cortex-a7 --enable-encoder=cedrus264
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59. 12.100 / 59. 12.100
  libavformat    59.  8.100 / 59.  8.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 16.100 /  8. 16.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
[mjpeg @ 0x77ba0] unable to decode APP fields: Invalid data found when processing input
Input #0, video4linux2,v4l2, from '/dev/video1':
  Duration: N/A, start: 1955.027528, bitrate: N/A
  Stream #0:0: Video: mjpeg (Baseline), yuvj422p(pc, bt470bg/unknown/unknown), 1280x720, 30 fps, 30 tbr, 1000k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (cedrus264))
Press [q] to stop, [?] for help
[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
[swscaler @ 0x93ec0] [swscaler @ 0x9d670] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x93ec0] [swscaler @ 0xd8240] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x93ec0] [swscaler @ 0x112b30] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x93ec0] [swscaler @ 0x14d420] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x93ec0] [swscaler @ 0x187740] deprecated pixel format used, make sure you did set range correctly
[VDPAU SUNXI] VE version 0x1701 opened.
Output #0, mp4, to '/tmp/video.mp4':
  Metadata:
    encoder         : Lavf59.8.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), nv12(tv, bt470bg/unknown/unknown, progressive), 1280x720, q=2-31, 200 kb/s, 30 fps, 15360 tbn
    Metadata:
      encoder         : Lavc59.12.100 cedrus264
[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input=21.7x
    Last message repeated 1 times
frame=   92 fps= 70 q=-0.0 size=    2048kB time=00:00:03.03 bitrate=5531.0kbits/s dup=89 drop=0 speed[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 5 times
frame=  104 fps= 55 q=-0.0 size=    2304kB time=00:00:03.43 bitrate=5497.4kbits/s dup=95 drop=0 speed[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 5 times
frame=  116 fps= 47 q=-0.0 size=    2560kB time=00:00:03.83 bitrate=5470.8kbits/s dup=101 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 5 times
frame=  128 fps= 42 q=-0.0 size=    2816kB time=00:00:04.23 bitrate=5449.3kbits/s dup=107 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 5 times
frame=  140 fps= 38 q=-0.0 size=    3072kB time=00:00:04.63 bitrate=5431.5kbits/s dup=113 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 4 times
frame=  150 fps= 36 q=-0.0 size=    3328kB time=00:00:04.96 bitrate=5489.2kbits/s dup=118 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 3 times
frame=  158 fps= 33 q=-0.0 size=    3584kB time=00:00:05.23 bitrate=5610.2kbits/s dup=122 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  164 fps= 31 q=-0.0 size=    3584kB time=00:00:05.43 bitrate=5403.7kbits/s dup=125 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 3 times
frame=  172 fps= 29 q=-0.0 size=    3840kB time=00:00:05.70 bitrate=5518.8kbits/s dup=129 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 3 times
frame=  180 fps= 28 q=-0.0 size=    4096kB time=00:00:05.96 bitrate=5623.7kbits/s dup=133 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  186 fps= 26 q=-0.0 size=    4096kB time=00:00:06.16 bitrate=5441.3kbits/s dup=136 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  192 fps= 25 q=-0.0 size=    4352kB time=00:00:06.36 bitrate=5599.7kbits/s dup=139 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 3 times
frame=  200 fps= 24 q=-0.0 size=    4352kB time=00:00:06.63 bitrate=5374.6kbits/s dup=143 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 3 times
frame=  209 fps= 24 q=-0.0 size=    4608kB time=00:00:06.93 bitrate=5444.5kbits/s dup=148 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 3 times
frame=  223 fps= 24 q=-0.0 size=    5120kB time=00:00:07.40 bitrate=5668.0kbits/s dup=158 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  239 fps= 24 q=-0.0 size=    5376kB time=00:00:07.93 bitrate=5551.3kbits/s dup=171 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  253 fps= 24 q=-0.0 size=    5632kB time=00:00:08.40 bitrate=5492.5kbits/s dup=182 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  267 fps= 24 q=-0.0 size=    6144kB time=00:00:08.86 bitrate=5676.5kbits/s dup=193 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  281 fps= 24 q=-0.0 size=    6400kB time=00:00:09.33 bitrate=5617.4kbits/s dup=204 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  297 fps= 24 q=-0.0 size=    6912kB time=00:00:09.86 bitrate=5738.8kbits/s dup=217 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  313 fps= 24 q=-0.0 size=    7168kB time=00:00:10.40 bitrate=5646.2kbits/s dup=230 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  329 fps= 24 q=-0.0 size=    7680kB time=00:00:10.93 bitrate=5754.4kbits/s dup=243 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  341 fps= 24 q=-0.0 size=    7936kB time=00:00:11.33 bitrate=5736.3kbits/s dup=252 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  355 fps= 24 q=-0.0 size=    8192kB time=00:00:11.80 bitrate=5687.2kbits/s dup=263 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  369 fps= 24 q=-0.0 size=    8704kB time=00:00:12.26 bitrate=5812.8kbits/s dup=274 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  383 fps= 24 q=-0.0 size=    8960kB time=00:00:12.73 bitrate=5764.4kbits/s dup=285 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  403 fps= 24 q=-0.0 size=    9472kB time=00:00:13.40 bitrate=5790.6kbits/s dup=302 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  419 fps= 24 q=-0.0 size=    9984kB time=00:00:13.93 bitrate=5870.0kbits/s dup=315 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  439 fps= 24 q=-0.0 size=   10496kB time=00:00:14.60 bitrate=5889.3kbits/s dup=332 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  456 fps= 24 q=-0.0 size=   11008kB time=00:00:15.16 bitrate=5945.8kbits/s dup=346 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  474 fps= 24 q=-0.0 size=   11264kB time=00:00:15.76 bitrate=5852.5kbits/s dup=361 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame=  488 fps= 24 q=-0.0 size=   11776kB time=00:00:16.23 bitrate=5942.6kbits/s dup=373 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  510 fps= 25 q=-0.0 size=   12288kB time=00:00:16.96 bitrate=5933.0kbits/s dup=392 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame=  524 fps= 25 q=-0.0 size=   12544kB time=00:00:17.43 bitrate=5894.5kbits/s dup=404 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  540 fps= 25 q=-0.0 size=   13056kB time=00:00:17.96 bitrate=5953.0kbits/s dup=417 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  556 fps= 25 q=-0.0 size=   13568kB time=00:00:18.50 bitrate=6008.1kbits/s dup=430 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  576 fps= 25 q=-0.0 size=   14080kB time=00:00:19.16 bitrate=6017.9kbits/s dup=447 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  596 fps= 25 q=-0.0 size=   14592kB time=00:00:19.83 bitrate=6027.1kbits/s dup=464 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  614 fps= 25 q=-0.0 size=   14848kB time=00:00:20.43 bitrate=5952.8kbits/s dup=479 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  632 fps= 25 q=-0.0 size=   15360kB time=00:00:21.03 bitrate=5982.4kbits/s dup=494 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  654 fps= 25 q=-0.0 size=   15872kB time=00:00:21.76 bitrate=5973.5kbits/s dup=513 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  673 fps= 25 q=-0.0 size=   16384kB time=00:00:22.40 bitrate=5991.9kbits/s dup=529 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame=  687 fps= 26 q=-0.0 size=   16896kB time=00:00:22.86 bitrate=6053.0kbits/s dup=541 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  709 fps= 26 q=-0.0 size=   17152kB time=00:00:23.60 bitrate=5953.8kbits/s dup=560 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  731 fps= 26 q=-0.0 size=   17664kB time=00:00:24.33 bitrate=5946.7kbits/s dup=579 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  747 fps= 26 q=-0.0 size=   18176kB time=00:00:24.86 bitrate=5987.8kbits/s dup=592 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  767 fps= 26 q=-0.0 size=   18432kB time=00:00:25.53 bitrate=5913.6kbits/s dup=609 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  789 fps= 26 q=-0.0 size=   18944kB time=00:00:26.26 bitrate=5908.2kbits/s dup=628 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame=  803 fps= 26 q=-0.0 size=   19200kB time=00:00:26.73 bitrate=5883.5kbits/s dup=640 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  821 fps= 26 q=-0.0 size=   19712kB time=00:00:27.33 bitrate=5907.8kbits/s dup=655 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame=  835 fps= 26 q=-0.0 size=   19968kB time=00:00:27.80 bitrate=5884.1kbits/s dup=667 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  853 fps= 26 q=-0.0 size=   20224kB time=00:00:28.40 bitrate=5833.6kbits/s dup=682 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame=  869 fps= 26 q=-0.0 size=   20480kB time=00:00:28.93 bitrate=5798.6kbits/s dup=696 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  888 fps= 26 q=-0.0 size=   20992kB time=00:00:29.56 bitrate=5816.2kbits/s dup=712 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame=  903 fps= 26 q=-0.0 size=   21248kB time=00:00:30.06 bitrate=5789.3kbits/s dup=725 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  924 fps= 27 q=-0.0 size=   21760kB time=00:00:30.76 bitrate=5793.9kbits/s dup=743 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  944 fps= 27 q=-0.0 size=   22016kB time=00:00:31.43 bitrate=5737.7kbits/s dup=760 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame=  958 fps= 27 q=-0.0 size=   22272kB time=00:00:31.90 bitrate=5719.5kbits/s dup=772 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  978 fps= 27 q=-0.0 size=   22784kB time=00:00:32.56 bitrate=5731.2kbits/s dup=789 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame=  996 fps= 27 q=-0.0 size=   23296kB time=00:00:33.16 bitrate=5754.0kbits/s dup=804 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1018 fps= 27 q=-0.0 size=   23552kB time=00:00:33.90 bitrate=5691.4kbits/s dup=823 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1032 fps= 27 q=-0.0 size=   23808kB time=00:00:34.36 bitrate=5675.1kbits/s dup=835 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1054 fps= 27 q=-0.0 size=   24320kB time=00:00:35.10 bitrate=5676.1kbits/s dup=854 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1074 fps= 27 q=-0.0 size=   24832kB time=00:00:35.76 bitrate=5687.5kbits/s dup=871 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1088 fps= 27 q=-0.0 size=   25088kB time=00:00:36.23 bitrate=5672.2kbits/s dup=883 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1108 fps= 27 q=-0.0 size=   25600kB time=00:00:36.90 bitrate=5683.3kbits/s dup=900 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1122 fps= 27 q=-0.0 size=   25856kB time=00:00:37.36 bitrate=5668.5kbits/s dup=912 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1134 fps= 27 q=-0.0 size=   26112kB time=00:00:37.76 bitrate=5664.0kbits/s dup=922 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1153 fps= 27 q=-0.0 size=   26368kB time=00:00:38.40 bitrate=5625.2kbits/s dup=938 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1169 fps= 27 q=-0.0 size=   26880kB time=00:00:38.93 bitrate=5655.8kbits/s dup=952 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1183 fps= 27 q=-0.0 size=   27136kB time=00:00:39.40 bitrate=5642.1kbits/s dup=964 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1203 fps= 27 q=-0.0 size=   27392kB time=00:00:40.06 bitrate=5600.5kbits/s dup=981 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1215 fps= 27 q=-0.0 size=   27648kB time=00:00:40.46 bitrate=5597.0kbits/s dup=991 drop=0 spee[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
More than 1000 frames duplicated
[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
frame= 1237 fps= 27 q=-0.0 size=   28160kB time=00:00:41.20 bitrate=5599.2kbits/s dup=1010 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1253 fps= 27 q=-0.0 size=   28672kB time=00:00:41.73 bitrate=5628.1kbits/s dup=1024 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1275 fps= 27 q=-0.0 size=   28928kB time=00:00:42.46 bitrate=5580.3kbits/s dup=1043 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1295 fps= 27 q=-0.0 size=   29440kB time=00:00:43.13 bitrate=5591.3kbits/s dup=1060 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1317 fps= 27 q=-0.0 size=   29952kB time=00:00:43.86 bitrate=5593.5kbits/s dup=1079 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1339 fps= 27 q=-0.0 size=   30464kB time=00:00:44.60 bitrate=5595.5kbits/s dup=1098 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1355 fps= 27 q=-0.0 size=   30720kB time=00:00:45.13 bitrate=5575.9kbits/s dup=1112 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1368 fps= 27 q=-0.0 size=   30976kB time=00:00:45.56 bitrate=5568.9kbits/s dup=1123 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1390 fps= 27 q=-0.0 size=   31488kB time=00:00:46.30 bitrate=5571.3kbits/s dup=1142 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1412 fps= 27 q=-0.0 size=   32000kB time=00:00:47.03 bitrate=5573.6kbits/s dup=1161 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1426 fps= 27 q=-0.0 size=   32256kB time=00:00:47.50 bitrate=5563.0kbits/s dup=1173 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1448 fps= 27 q=-0.0 size=   32512kB time=00:00:48.23 bitrate=5521.9kbits/s dup=1192 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1470 fps= 28 q=-0.0 size=   33024kB time=00:00:48.96 bitrate=5524.8kbits/s dup=1211 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1494 fps= 28 q=-0.0 size=   33536kB time=00:00:49.76 bitrate=5520.3kbits/s dup=1232 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1514 fps= 28 q=-0.0 size=   34048kB time=00:00:50.43 bitrate=5530.5kbits/s dup=1249 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1534 fps= 28 q=-0.0 size=   34560kB time=00:00:51.10 bitrate=5540.4kbits/s dup=1266 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1554 fps= 28 q=-0.0 size=   34816kB time=00:00:51.76 bitrate=5509.6kbits/s dup=1283 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1574 fps= 28 q=-0.0 size=   35328kB time=00:00:52.43 bitrate=5519.5kbits/s dup=1300 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1597 fps= 28 q=-0.0 size=   35840kB time=00:00:53.20 bitrate=5518.8kbits/s dup=1320 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
frame= 1613 fps= 28 q=-0.0 size=   36096kB time=00:00:53.73 bitrate=5503.1kbits/s dup=1334 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1633 fps= 28 q=-0.0 size=   36608kB time=00:00:54.40 bitrate=5512.7kbits/s dup=1351 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1655 fps= 28 q=-0.0 size=   36864kB time=00:00:55.13 bitrate=5477.4kbits/s dup=1370 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1675 fps= 28 q=-0.0 size=   37376kB time=00:00:55.80 bitrate=5487.2kbits/s dup=1387 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1697 fps= 28 q=-0.0 size=   37888kB time=00:00:56.53 bitrate=5490.2kbits/s dup=1406 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1717 fps= 28 q=-0.0 size=   38144kB time=00:00:57.20 bitrate=5462.9kbits/s dup=1423 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 2 times
frame= 1739 fps= 28 q=-0.0 size=   38656kB time=00:00:57.93 bitrate=5466.1kbits/s dup=1442 drop=0 spe[mjpeg @ 0x78a40] unable to decode APP fields: Invalid data found when processing input
frame= 1745 fps= 28 q=-0.0 Lsize=   38982kB time=00:00:58.13 bitrate=5493.3kbits/s dup=1447 drop=0 speed=0.929x
video:38974kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.020328%
Exiting normally, received signal 2.

And CPU usage

Spoiler

Software encoding was incredible bad and I got only 720p@5fps

lunixoid · November 1, 2021

I did next few moves:

Change sun8i-r40.dtsi

Spoiler

// Add section befor the "soc"	
    reserved-memory {
		#address-cells = <1>;
		#size-cells = <1>;
		ranges;

		cma_pool: cma@4a000000 {
			compatible = "shared-dma-pool";
			size = <0x6000000>;
			alloc-ranges = <0x4a000000 0x6000000>;
			reusable;
			linux,cma-default;
		};
	};

// Completely remove video-codec
-		video-codec@1c0e000 {
-			compatible = "allwinner,sun8i-r40-video-engine";
-			reg = <0x01c0e000 0x1000>;
-			clocks = <&ccu CLK_BUS_VE>, <&ccu CLK_VE>,
-			<&ccu CLK_DRAM_VE>;
-			clock-names = "ahb", "mod", "ram";
-			resets = <&ccu RST_BUS_VE>;
-			interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
-			allwinner,sram = <&ve_sram 1>;
-		};

// Add new codec befor "crypto" node
		ve: video-engine@01c0e000 {
			compatible = "allwinner,sunxi-cedar-ve";
			reg = <0x01c0e000 0x1000>,
					<0x01c00000 0x10>,
					<0x01c20000 0x800>;
			memory-region = <&cma_pool>;
			syscon = <&syscon>;
  			clocks = <&ccu CLK_BUS_VE>, <&ccu CLK_VE>,
					<&ccu CLK_DRAM_VE>;
			clock-names = "ahb", "mod", "ram";
			resets = <&ccu RST_BUS_VE>;
			interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
			allwinner,sram = <&ve_sram 1>;
		};

Change sun8i-r40-bananapi-m2-ultra.dts
Spoiler
```
&ve {
	status = "okay";
};
```
Compile ffmpeg (options you can get from log message above)
Compile codec driver sunxi-cedar
Remove cma option from kernel command line (it cause conflict with SRAM described in DTS)

twdragon · December 8, 2021

19.10.2021 в 09:42, woodfe сказал:

I was following this posts:

and:

https://wvthoog.nl/nanopi-ov5640-camera/

1. decompile dtb to dts

2. modify dts: 1) add reversed-memory, 2) disable cedrus, add cedar_ve

3. compile dts to dtb, reboot

4. /dev/cedar_ve appear

5. run ffmpeg

Tried to do, no success, will dig into https://github.com/orangepi-xunlong/linux-orangepi/commit/58e01f39aa375f551e009c8206a57073c040347a to obtain possibly proper CSI kernel bindings

shauuun · February 16, 2022

Can cedrus264 do decoding as well? Anyone has tried? Thanks.

lunixoid · February 16, 2022

2 hours ago, shauuun said:

Can cedrus264 do decoding as well? Anyone has tried? Thanks.

Yes, hw decoding works out of the box for mainline linux kernel.

shauuun · February 16, 2022

4 hours ago, lunixoid said:

Yes, hw decoding works out of the box for mainline linux kernel.

I am using the 5.15.18-sunxi kernel that comes with Armbian. However, I was not able to get /dev/cedar_ve to appear even after editing the dtb file.

Can you elaborate more on which kernel you are using and how did you get it done?

ning · February 17, 2022

why not go LibreELEC, and check their code, to understand how to enable accelerated video decoding?

twdragon · April 5, 2022

17.02.2022 в 15:10, ning сказал:

why not go LibreELEC, and check their code, to understand how to enable accelerated video decoding?

Can someone try to compile the kernel to have working both camera and HW encoder?

Alex LanS · April 13, 2023

Hello. I am using Nano pi duo2 2 as a sip video door bell. I will use baresip.

But i cant do hardware acceleration with OV5640 camera in baresip.

I have found that repo https://github.com/twdragon/FFmpeg - with this ffmpeg and i have working acceleration.

But i dont understand how i can integrate this acceleration with baresip.

I using Ubuntu 18.04.6 LTS with kernel Linux NanoPi-Duo2 3.4.39-h3, because only on this kernel 3.4 works Video OUT on Nano PI Duo 2.

Can i integrate hardware acceleration Cedrus to baresip , when compiling baresip?

twdragon · April 13, 2023

Цитата

But i dont understand how i can integrate this acceleration with baresip.

I using Ubuntu 18.04.6 LTS with kernel Linux NanoPi-Duo2 3.4.39-h3, because only on this kernel 3.4 works Video OUT on Nano PI Duo 2.

Can i integrate hardware acceleration Cedrus to baresip , when compiling baresip?

No, you should rewrite the baresip decoder source to access and activate cedrus264 in the FFMPEG API as a hardware decoding accelerator.

Edited April 13, 2023 by twdragon

Alex LanS · April 14, 2023

I install fresh system, install https://github.com/twdragon/FFmpeg

and clone repo of baresip 3.0.0.

Baresip installed, but i get error

dl: mod: /usr/local/lib/baresip/modules/avcodec.so (/usr/local/lib/baresip/modules/avcodec.so: undefined symbol: avpriv_vga16_font)
module avcodec.so: No such file or directory [2]

Is the any other compiling options for the baresip i should to use? or for the ffmpeg?

config if baresip is used

# Video
video_source            v4l2,/dev/video0
videnc_format           yuv420p
# Video source modules
module                  v4l2.so

Edited April 14, 2023 by Alex LanS

twdragon · April 14, 2023

It seemed the API version conflict problem, you should examine the functions baresip calls from the FFMPEG API or the parts of FFMPEG it exploits. I am not familiar with baresip code, so sadly I cannot help you here...

Alex LanS · April 18, 2023

I have recompile ffmpeg from https://github.com/twdragon/FFmpeg with option

./configure --prefix=/usr --enable-nonfree --enable-gpl --enable-libx264 --enable-libmp3lame --enable-libpulse --enable-libv4l2 --enable-libx265 --enable-shared --arch=armv7l --target-os=linux --enable-version3

After this baresip is compile and can use cedrus264.

avcodec: using H.264 encoder 'cedrus264' -- Cedrus H.264 Encoder
avcodec: using H.264 decoder 'h264' -- H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
avcodec: using H.265 encoder 'libx265' -- libx265 H.265 / HEVC
avcodec: using H.265 decoder 'hevc' -- HEVC (High Efficiency Video Coding)

But on video call to another sip agent - there is no video on other side. on console i see that ftp is 30. but no video.

If a set option selfview - i see my video, and video of other side. on other side - only his video.

May other side dont understand this codec?

Deoptim · February 2

Usage ffmpeg with motioneye on Allwinner H3 board ( 6.1.63-current-sunxi kernel )

https://4pda-to.translate.goog/forum/index.php?showtopic=750921&st=10260&_x_tr_sl=ru&_x_tr_tl=en&_x_tr_pto=wapp#entry128032933

Sign In

Last revision FFMPEG cedrus264 patched: review and test request

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Join the conversation

Similar Content

Important Information