AndreVallestero

Been a few months since I've worked on trying to get VPU support on mainline but I was able to get pretty close working with xmixahlx on the Pine64 IRC. He made a forum post specifying all the resources we used: https://forum.pine64.org/showthread.php?tid=9171. I'll give some insight into my process: I started with the Manjaro ARM kernel (5.5 which was very close to vanilla mainline) and applied patches created by diffing it against specific parts of - https://github.com/bbrezillon/linux/tree/rk/vdec-h264-v9%2Bhantro-h264 - https://github.com/Kwiboo/linux-rockchip/tree/v4l2-5.5-from-list-v5.5-rkvdec - https://github.com/LibreELEC/LibreELEC.tv/tree/master/projects/Rockchip/patches/linux/default such that I would be able to compile - https://github.com/pH5/libva/tree/hantro-h264 - https://github.com/pH5/libva-v4l2-request - https://github.com/Kwiboo/FFmpeg/tree/v4l2-request-hwaccel-4.2.2 without any errors. With this done, I began testing. MPEG2 was working with Hantro however H264 didn't have a codec for profile 100 (High) which is what most content is encoded in. mpv -hwdec bbb_sunflower_native_60fps_normal.mp4 >>> [ffmpeg/video] h264: No support for codec h264 profile 100. xmixahlx Has kept the thread I mentioned up to date on any new progress he finds so I suggest checking it out.

I'm trying to get VPU acceleration on Manjaro ARM and I'm having trouble figuring out how Armbian is able to leverage the VPU with Hantro still incomplete. I know it uses RKMPP but I don't know which driver or patches I need to enable it in the Kernel and DTS. Any help is appreciated, thanks in advance.

Found the an MPP github issue pertaining to my exact problem. https://github.com/rockchip-linux/mpp/issues/52

Did you set it to armv7 in CMakeLists? I tried doing the same (since the rk3399 reports as aarch64 which doesn't match any of the compare strings) and it wasn't able to compile.

The assertion error happens on line 154 in this file: https://github.com/rockchip-linux/mpp/blob/67e80ebfe46558c2eb50218e57b50d51d6df2be2/mpp/hal/rkdec/h264d/hal_h264d_api.c MPP_RET hal_h264d_init(void *hal, MppHalCfg *cfg) { MppHalApi *p_api = NULL; MPP_RET ret = MPP_ERR_UNKNOW; H264dHalCtx_t *p_hal = (H264dHalCtx_t *)hal; VpuHardMode hard_mode = MODE_NULL; RK_U32 hard_platform = 0; INP_CHECK(ret, NULL == p_hal); memset(p_hal, 0, sizeof(H264dHalCtx_t)); p_api = &p_hal->hal_api; p_hal->frame_slots = cfg->frame_slots; p_hal->packet_slots = cfg->packet_slots; p_hal->fast_mode = cfg->fast_mode; //!< choose hard mode { RK_U32 mode = 0; RK_U32 vcodec_type = 0; mpp_env_get_u32("use_mpp_mode", &mode, 0); vcodec_type = mpp_get_vcodec_type(); mpp_assert(vcodec_type & (HAVE_RKVDEC | HAVE_VPU1 | HAVE_VPU2)); if ((mode <= RKVDEC_MODE) && (vcodec_type & HAVE_RKVDEC)) { hard_mode = RKVDEC_MODE; hard_platform = HAVE_RKVDEC; } else if (vcodec_type & HAVE_VPU1) { hard_mode = VDPU1_MODE; hard_platform = HAVE_VPU1; } else if (vcodec_type & HAVE_VPU2) { hard_mode = VDPU2_MODE; hard_platform = HAVE_VPU2; } H264D_DBG(H264D_DBG_HARD_MODE, "set_mode=%d, hw_spt=%08x, use_mode=%d\n", mode, vcodec_type, hard_mode); } switch (hard_mode) { case RKVDEC_MODE: p_api->init = rkv_h264d_init; p_api->deinit = rkv_h264d_deinit; p_api->reg_gen = rkv_h264d_gen_regs; p_api->start = rkv_h264d_start; p_api->wait = rkv_h264d_wait; p_api->reset = rkv_h264d_reset; p_api->flush = rkv_h264d_flush; p_api->control = rkv_h264d_control; cfg->device_id = HAL_RKVDEC; break; case VDPU1_MODE: p_api->init = vdpu1_h264d_init; p_api->deinit = vdpu1_h264d_deinit; p_api->reg_gen = vdpu1_h264d_gen_regs; p_api->start = vdpu1_h264d_start; p_api->wait = vdpu1_h264d_wait; p_api->reset = vdpu1_h264d_reset; p_api->flush = vdpu1_h264d_flush; p_api->control = vdpu1_h264d_control; cfg->device_id = HAL_VDPU; break; case VDPU2_MODE: p_api->init = vdpu2_h264d_init; p_api->deinit = vdpu2_h264d_deinit; p_api->reg_gen = vdpu2_h264d_gen_regs; p_api->start = vdpu2_h264d_start; p_api->wait = vdpu2_h264d_wait; p_api->reset = vdpu2_h264d_reset; p_api->flush = vdpu2_h264d_flush; p_api->control = vdpu2_h264d_control; cfg->device_id = HAL_VDPU; break; default: mpp_err_f("hard mode error, value=%d\n", hard_mode); mpp_assert(0); break; } The value of `hard_mode` is 0 when it fails. The error goes on to say "failed to find device for coding 7 type 0".

I'm trying to make a script to run on the Pinebook Pro and I'm having mpp errors while trying to leverage it in ffmpeg. I'm not sure what might be causing the error but according to THIS THREAD , compiling with `-DHAVE_DRM='ON` should resolve it (which it doesn't in my case). I'm using the 20171218 release of rkmpp and ffmpeg 4.2. Any help is appreciated. mmp build: cmake -DHAVE_DRM='ON' -DCMAKE_BUILD_TYPE:STRING='Release' -DCMAKE_INSTALL_PREFIX:PATH='/usr' \ -DRKPLATFORM:BOOL='ON' -Wno-dev \ -DAVSD_TEST:BOOL='OFF' -DH264D_TEST:BOOL='OFF' -DH265D_TEST:BOOL='OFF' -DJPEGD_TEST:BOOL='OFF' \ -DMPI_DEC_TEST:BOOL='OFF' -DMPI_ENC_TEST:BOOL='OFF' -DMPI_RC2_TEST:BOOL='OFF' \ -DMPI_RC_TEST:BOOL='OFF' -DMPI_TEST:BOOL='OFF' -DMPP_BUFFER_TEST:BOOL='OFF' \ -DMPP_ENV_TEST:BOOL='OFF' -DMPP_INFO_TEST:BOOL='OFF' -DMPP_LOG_TEST:BOOL='OFF' \ -DMPP_MEM_TEST:BOOL='OFF' -DMPP_PACKET_TEST:BOOL='OFF' -DMPP_PLATFORM_TEST:BOOL='OFF' \ -DMPP_TASK_TEST:BOOL='OFF' -DMPP_THREAD_TEST:BOOL='OFF' -DVP9D_TEST:BOOL='OFF' \ -DVPU_API_TEST:BOOL='OFF' .. make -j$THREADS sudo make install ffmpeg build: ./configure --prefix=/usr --enable-gpl --enable-version3 --enable-nonfree --enable-static --enable-gmp \ --enable-gnutls --enable-libass --enable-libbluray --enable-libcdio --enable-libfdk-aac \ --enable-libfreetype --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 \ --enable-libxcb --enable-opencl --enable-libdrm --arch='armv8-a+crc+crypto' \ --cpu='cortex-a72.cortex-a53' --enable-rkmpp --enable-lto --enable-hardcoded-tables \ --disable-debug make -j$THREADS sudo make install sudo ldconfig error: sudo ffmpeg -benchmark -vcodec h264_rkmpp -i demo.mkv -f null - ffmpeg version 4.2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 9.1.0 (GCC) configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-nonfree --enable-static --enable-gmp --enable-gnutls --enable-libass --enable-libbluray --enable-libcdio --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxcb --enable-opencl --enable-libdrm --arch=armv8-a+crc+crypto --cpu=cortex-a72.cortex-a53 --enable-rkmpp --enable-lto --enable-hardcoded-tables --disable-debug libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 Input #0, matroska,webm, from 'demo.mkv': Metadata: COMPATIBLE_BRANDS: iso6avc1mp41 MAJOR_BRAND : dash MINOR_VERSION : 0 ENCODER : Lavf58.29.100 Duration: 00:05:13.80, start: -0.007000, bitrate: 2887 kb/s Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default) Metadata: HANDLER_NAME : ISO Media file produced by Google Inc. DURATION : 00:05:13.780000000 Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default) Metadata: DURATION : 00:05:13.801000000 mpi: mpp version: hal_h264d_api: Assertion vcodec_type & ((0x00000200) | (0x00000001) | (0x00000002)) failed at hal_h264d_init:119 hal_h264d_api: hal_h264d_init hard mode error, value=0 hal_h264d_api: Assertion 0 failed at hal_h264d_init:169 mpp_device: mpp_device_init failed to find device for coding 0 type 7 hal_h264d_api: p_hal->vpu_socket <= 0 mpp_hal: mpp_hal_init hal h264d_rkdec init failed ret -2 mpp_hal: mpp_hal_init could not found coding type 7 mpp_dec: mpp_dec_init could not init hal mpp_rt: NOT found ion allocator mpp_rt: found drm allocator mpp: error found on mpp initialization mpp: WARNING: setup buffer group before decoder init mpp: command 310002 param 0xaaab0831a360 ret -1 [h264_rkmpp @ 0xaaab0833b720] Failed to assign buffer group (code = -1) [h264_rkmpp @ 0xaaab0833b720] Failed to initialize RKMPP decoder. Stream mapping: Stream #0:0 -> #0:0 (h264 (h264_rkmpp) -> wrapped_avframe (native)) Stream #0:1 -> #0:1 (opus (native) -> pcm_s16le (native)) Error while opening decoder for input stream #0:0 : Unknown error occurred bench: maxrss=23880kB

Did you manage to figure out how to properly utilize hardware acceleration using ffmpeg + RockChip MPP? I'm working on a similar project that would benefit greatly from this and an update on your work would be very appreciated.