Jump to content

Kwiboo

Members
  • Posts

    42
  • Joined

  • Last visited

Everything posted by Kwiboo

  1. Kwiboo

    Mainline VPU

    Thanks for testing!, kodi has just moved to ffmpeg 4.3 so LibreELEC will follow as soon as all ffmpeg patches has been rebased. Do you have a sample to share? Also note that only Profile 0 is supported and videos using frame resize is expected to have issues. I do not expect these formats to work, the kodi drmprime video codec and renderer sw decoding parts have mainly been tested with 8-bit video. The crash should probably be fixed, unfortunately 10/12-bit sw decoding is not high on priority list, in future when ffmpeg filters are supported in kodi drmprime video codec converting to a 8-bit pixel format may be possible.
  2. Kwiboo

    Mainline VPU

    Thanks for testing, normally I try to keep ffmpeg_v4l2-request-hwaccel-master and ffmpeg_v4l2-request-hwaccel-4.2.2 in sync but I have not yet updated or tested the -master branch, I will try to update it in next few days. Background on my different ffmpeg branches: v4l2-request-hwaccel-x.y.z: Main branch that most of the time matches the patches used in LibreELEC master, x.y.z (4.2.2) matches the ffmpeg version used in Kodi master. This branch is usually tested on Rockchip/hantro/rkvdec by me and on Allwinner/cedrus by @jernej, fixup commits usually gets squashed right before we pick out a new patchset for use in LibreELEC. v4l2-request-hwaccel-master: This branch should contain the patches in v4l2-request-hwaccel-x.y.z on top of ffmpeg master, may see some delays at times. v4l2-request-hwaccel-4.2.2-rkvdec: work-in-progress branch for parts that currently only is expected to work on rockchip/rkvdec, e.g. 10-bit hevc decoding on cedrus stops working in this branch due to patches to handle rkvdec 10-bit decoding, patches should eventually flow to -x.y.z/-master branches once ready.
  3. I have some work-in-progress patches at https://github.com/Kwiboo/linux-rockchip/compare/next-20200501...next-20200501-drm-rockchip that should enable up to 4k30hz hdmi modes on rk3399. My focus switched to rkvdec so these patches have been on hold for some time, they need to be synced with some of my older rk3328 patches for 420-mode support before they can be sent upstream. With 10-bit decoding now working I am expecting readying 4k/10-bit hdmi patches to be more fun :-)
  4. Kwiboo

    Mainline VPU

    Correct, the device-tree changes is not part of my media_tree branch, it is already merged for 5.8 and not something I care for in my patchset (I do not use this kernel branch as-is, only the diff from media tree master + linux-next and/or a mix of merged/work-in-progress patches). No this is not the "small artifacts while decoding" issue, this is due to the sw pixel format conversion being done, the sw pixel format may incorrectly be signaled as nv12 but is instead "nv15" for 10-bit content also there is no nv15/nv20 converter. Correct, I recommend you use kodi-gbm or mpv if you want to render video as they both have support for zero-copy rendering to a dedicated drm plane, with ffmpeg command the cpu will be busy doing pixel format conversion (sw pixel conversion is also activated with null output).
  5. Kwiboo

    Mainline VPU

    No problems! :-) I can recommend you to look at https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-work-in-progress that is the branch that will continue to see updates as I prepare ongoing work for upstream. With this updated branch (work rebased on top of media_tree master) and latest 4.2.2-rkvdec ffmpeg branch vp9, h264 (high 10 and high 422) and hevc (main 10) should be usable. There is still one bit not properly being configured for hevc so there are some videos that produce small artifacts while decoding. Hoping to have initial patchset with focus on rkvdec fixes and rkvdec/hantro support for h264 interlaced/field encoded on media mailing list before the weekend is over :-)
  6. Kwiboo

    Mainline VPU

    I have some patches for activating h264 decoding using the hantro vpu on rk3328/rk3399 at https://github.com/Kwiboo/linux-rockchip/commits/v4l2-5.4-work-in-progress-v5.4, they only ever reached mailing list as a RFC. Hoping to find some time soon to rework the field encoded content (interlaced) support and also to make it possible to use the hantro vpu for h264 decoding on rk3328/rk3399. Hantro h264 does seem to be limited to 1080p on those SoCs so will probably not bring any benefit over rkvdec.
  7. Kwiboo

    Mainline VPU

    Thanks! I have now updated the https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-hwaccel-4.2.2 (and the -master) branch, commits have been squashed, updated and split apart to hopefully make it easier to use with mainline linux without the work-in-progress cedurs/hantro patches that has not yet been upstreamed. I would recommend you to test the commits up to and including "HACK: hwcontext_drm: do not require drm device", that should be safe to use on v5.5 or newer kernel without out-of-tree patches for cedrus/hantro/rkvdec. Also added the initial vp9 hwaccel, picked from https://github.com/bbrezillon/FFmpeg/commits/v4l2-vp9, but have not yet had time to test it more then a simple compile test. I am expecting that I will make some minor updates to the vp9 hwaccel once I have run some tests.
  8. Kwiboo

    Mainline VPU

    That branch was just something I played with for stateful decoding on RPi and Amlogic and not something that was finished nor do I expect it to be working. https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-hwaccel-4.2.2 is the branch we use in LibreELEC. I was able to hw decode h264 with that branch and vanilla linux 5.6.6 on my RK3288 Tinker Board S a few hours ago. Also you should not need the --enable-libv4l2 configure flag, --enable-libdrm --enable-v4l2-request --enable-libudev should be enough to enable the request api hwaccels. What device are you testing on? I have not done any testing with rkvdec driver and there is no rk3328/rk3399 hantro driver for h264 in upstream media tree. Mpeg-2 and VP8 should work across rk3288/rk3328/rk3399. I do not expect h264 rkvdec to be working without changes to ffmpeg since there was some flags that got changed and now do not match between upstream rkvdec driver and my ffmpeg branch. I am hoping to get some time this weekend to update ffmpeg to work with rkvdec and hopefully submit the initial hantro h264 decoder for rk3328/rk3399 vpu2 to upstream.
  9. I have not synced any possible update from @jernej or the Raspberry Pi team yet, the latest ffmpeg that works with the cedrus/hantro patches we use in LibreELEC is located at https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-hwaccel-4.2.2 That version is still not compatible with the rkvdec driver targeted for 5.7 due to incompatible use of flags, I will update the ffmpeg code to work with both hantro and rkvdec when I rebase my patches for 5.6. The main diff between upstream hantro and the patches we use in LibreELEC is support for field encoded (interlaced) H264. The ffmpeg v4l2 request api hwaccel is not yet upstreamed and the main reason is that it depends on private kernel headers not yet part of the uapi.
  10. Yes, I have had VPU working on 5.4 kernel since the new year started, my test images from 31 dec / 1 jan at http://kwiboo.libreelec.tv/test/ should "work", at least for MPEG-2, H264 and VP8 codecs on RK3288, RK3328 and RK3399. The VPU patchset used in LibreELEC has not been touched/rebased for 5.5/5.6 yet, but I am hoping to get some time to work on that later this weekend and next week.
  11. @usual user thanks, I have updated the ffmpeg branch and updated my post to reflect the correct --enable-libudev https://github.com/FFmpeg/FFmpeg/compare/master...Kwiboo:v4l2-request-hwaccel-master.patch should apply clean on latest FFmpeg master, there may be some v4l2 commits in v5.5 that may be required for the ffmpeg branch to compile on linux v5.4, I forgot about them. https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Allwinner/patches/linux/0001-backport-from-5.5.patch or https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Rockchip/patches/linux/default/linux-0011-v4l2-from-5.5.patch should contain the patches needed for build on v5.4, the Allwinner patch is less intrusive, for Rockchip I picked every possible related v4l2 patch.
  12. @jerryg for hantro driver on RK3399 (or RK3328) I would recommend you to try the patches from https://github.com/Kwiboo/linux-rockchip/compare/v4l2-5.4-from-5.5-v5.4...v4l2-5.4-work-in-progress-v5.4, those add support for decoding of field encoded content and adds a hw variant for RK3399 on top of all hantro patches merged for 5.5. For ffmpeg I have two active branches that adds required bits for v4l2 stateless decoding: v4l2-request-hwaccel-4.0.4 (for use with ffmpeg in libreelec/kodi) and v4l2-request-hwaccel-master (for use with ffmpeg master), see https://github.com/FFmpeg/FFmpeg/compare/master...Kwiboo:v4l2-request-hwaccel-master for patches, build ffmpeg with --enable-v4l2-request --enable-libdrm --enable-libudev I plan on having these ffmpeg branches updated with latest stateless support until it can be upstreamed once the required private kernel headers have moved to uapi. mpv and kodi-gbm should be able to use the hw decoders as long as ffmpeg have been compiled with v4l2 request api enabled.
  13. Kwiboo

    The VPU driver

    Yep, the vpu1 is a Hantro G1 (or at least based on it). G1 is also used in IMX8 and details are in the IMX8M reference documentation. If you look for rk PX3 trm you also have some docs on the vpu1. Vpu2 is very similar to vpu1 but with regs in different positions. And imx8 g1 decoder code at https://github.com/CliveLau1990/imx-vpu-hantro/tree/master/decoder_sw/software/source is interesting.
  14. Kwiboo

    The VPU driver

    There are other people working on porting h264/vp8 to mainline driver, e.g. https://github.com/bbrezillon/linux/commits/rk-vpu1-h264, nothing I have tested yet.
  15. Kwiboo

    The VPU driver

    Great! The VPU in rk3288 (vpu1) should have VP8 support, same as the VPU in rk3328/rk3399 (vpu2) rk3288 has a second VPU block that can decode hevc rk3328/rk3399 has a second VPU block (rkvdec) that can decode h264/hevc/vp9 (they have two different vpu blocks that can decode h264) Just to let you know, I have moved to v5.1 now and latest RK VPU patchset v5 on top of linux v5.1 can be found at end of https://github.com/Kwiboo/linux-rockchip/compare/v5.1...rockchip-5.1-v4l2-from-list-v5.1 (along with a lot of probably unneccecery v4l2 patches)
  16. Kwiboo

    The VPU driver

    Yes, looks like the script used one of the branches, I do not have one branch with all patches, I tend to group patches into multiple branches to make it easier rebasing and review changes. If you keep the hdmi-sound base as you have and add https://github.com/Kwiboo/linux-rockchip/compare/rockchip-5.x-hdmi-sound...rockchip-5.x-vpu-v2.patch as a patch you should have both branches. The 1080p H264, VP8 samples I have tested on my tinker board and even my rock64 running libreelec / kodi-gbm seems to play back fine using sw decoding. There are some libreelec test images at http://kwiboo.libreelec.tv/test/ for testing, built out of https://github.com/Kwiboo/LibreELEC.tv/commits/rockchip-5.x
  17. Kwiboo

    The VPU driver

    Almost, that is the ffmpeg 4.0 branch I use myself, the ML patches was the ones at https://github.com/Kwiboo/FFmpeg/compare/bb5efd1727eeecc9be8f1402810c7ab72344eed3...v4l2-request-hwaccel-master.patch (same patches rebased on ffmpeg master), the FIXUP commit that I have added was just to do a rename as suggested by ML comments You can use https://github.com/Kwiboo/FFmpeg/compare/21c13d04e624aabc1c78998f910782db673070e6...583677006f15ee59600f6f30f5e79aa5f81935b9.patch if you want the 4.0 version or the link above for a ffmpeg master patch. I have tested https://samplemedia.linaro.org/MPEG2/ very much, the only sample I have had issue with was a field encoded mpeg2 sample been trying to find other field encoded samples in order to test but they are hard to come by.
  18. Kwiboo

    The VPU driver

    I have not yet tested lima or panfrost drivers since libmali is working better for my use case, will switch to lima and panfrost in future. For midgard mali driver I am using https://github.com/Kwiboo/mali-rockchip/commits/midgard-r28p0 (contains a restore 10.6 api commit that makes it work with rk r14p0 libmali) See https://github.com/Kwiboo/LibreELEC.tv/blob/rockchip-5.x/projects/Rockchip/packages/mali-midgard-rockchip/package.mk for how I build the midgard driver. The midgard rk platform code is doing some unnecessary dev_info logging, but driver runs with mainline device trees on both rk3288 and rk3399. I am not sure the v4l2 testing tools by bootlin etc is working with rockchip vpu driver (nothing I have ever tested). The ffmpeg hwaccel from https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-hwaccel and mpv patched with https://github.com/mpv-player/mpv/pull/6461 should work for both cedrus and rockchip vpu driver. kodi-gbm should also work as long as ffmpeg used is patched and is built with --enable-v4l2-request --enable-libudev --enable-libdrm
  19. Kwiboo

    The VPU driver

    @Myy I pushed some updated kernel branches to my linux-rockchip repo today that may help you get the vpu driver running. Earlier this week I sent out my ffmpeg V4L2 request API hwaccel as RFC on ffmpeg-devel mailing list, see http://ffmpeg.org/pipermail/ffmpeg-devel/2019-April/242316.html Important device tree and clk patches have already been merged or is queued for v5.1/v5.2. If you apply both of these two patches on top of v5.0.7 you should have a kernel with working vpu mpeg-2 driver for rk3288, rk3328 and rk3399: https://github.com/Kwiboo/linux-rockchip/compare/v5.0.7...rockchip-5.x-hdmi-sound.patch - contains all mmind/for-next patches merged/queued for v5.1/v5.2 + few patches from mailing list + my hdmi sound and cec patches https://github.com/Kwiboo/linux-rockchip/compare/v5.0.7...rockchip-5.x-vpu-v2.patch - contains all v4l2 related media tree patches merged/queued for v5.1/v5.2 + Ezequiel rockchip vpu v2 patchset + my additions for rk3288 and rk3328 If you do not care for hdmi sound or cec improvements you can instead use the fromlist patch: https://github.com/Kwiboo/linux-rockchip/compare/v5.0.7...rockchip-5.x-fromlist.patch - contains all mmind/for-next patches merged/queued for v5.1/v5.2 + few patches from mailing list For mali kernel driver I recommend https://github.com/Kwiboo/mali-rockchip/commits/utgard-r9p0, see below how I build it: make_target() { kernel_make -C $(kernel_path) M=$PKG_BUILD/driver/src/devicedrv/mali \ MALI_PLATFORM_FILES=platform/rk/rk.c GIT_REV="" \ EXTRA_CFLAGS="-DCONFIG_MALI400=1 -DCONFIG_MALI450=1 -DCONFIG_MALI470=1 -DMALI_FAKE_PLATFORM_DEVICE=1 -DCONFIG_MALI_DMA_BUF_MAP_ON_ATTACH -DCONFIG_MALI_DT" \ CONFIG_MALI400=m CONFIG_MALI450=y CONFIG_MALI470=y CONFIG_MALI_DMA_BUF_MAP_ON_ATTACH=y CONFIG_MALI_DT=y \ modules } makeinstall_target() { kernel_make -C $(kernel_path) M=$PKG_BUILD/driver/src/devicedrv/mali \ INSTALL_MOD_PATH=$INSTALL/$(get_kernel_overlay_dir) INSTALL_MOD_STRIP=1 DEPMOD=: \ modules_install }
  20. Kwiboo

    The VPU driver

    Sounds like it is linking to the static ffmpeg? I added the libudev dependancy to ffmpeg and only tested it on LibreELEC buildsystem where proper pkg-config file exists for libudev and -ludev is added correctly. Do you have full ffmpeg debug log? On my Tinker Board there is one media and three video devices and the v4l2_request probe code will try all media and related video devices until a video device supporting requested format is found. On Tinker Board the jpeg encoder is probed before the decoder and one output format not found is expected. Run mpv with --msg-level=ffmpeg=trace to get ffmpeg debug log. Output from v4l2-ctl --device /dev/video0 --all (and the other 2-3 video devices could be helpful to see if the MP2S format is supported).
  21. Kwiboo

    The VPU driver

    I never tested v4l2-request-test only peeked at the code to try and learn how to call the request api. The mpeg2/h264/hevc v4l2 ctrls currently lives in private kernel headers causing some troubles for user-space. My kernel vpu patcheset includes a revert to make the mpeg2 ctrl public and the hwaccel expects this, I plan to bundle the ctrl headers in next update to hwaccel.
  22. Kwiboo

    The VPU driver

    FFmpeg will need the following configure flags: --enable-v4l2-request --enable-libdrm --enable-libudev For mvp https://github.com/LibreELEC/LibreELEC.tv/pull/3256/files have some flags and commands that worked on LibreELEC buildsystem. kodi-gbm should also work without modifications as long as the mpeg2_v4l2request hwaccel is enabled, any recent version of kodi pre v18-v18.1. I have only tested this with libmali gbm files at https://github.com/LibreELEC/libmali and mali-utgard/mali-midgard kernel drivers. https://github.com/Kwiboo/LibreELEC.tv/tree/rockchip-5.x/projects/Rockchip/packages also have the package.pk with build commans I have used to build utgard/midgard drivers out-of-tree, but lima/panfrost should also "work" I have been told.
  23. Kwiboo

    The VPU driver

    @Myy The patches I have in my rockchip-5.x-vpu branch is on top of v5.0-rc6 + mmind/for-next and not a rockchip tree. All vpu related patches needed on top of v5.0-rc6: https://github.com/Kwiboo/linux-rockchip/compare/v5.0-rc6...rockchip-5.x-vpu (patch) List of mmind/for-next patches rebased on v5.0-rc6: https://github.com/Kwiboo/linux-rockchip/compare/v5.0-rc6...rockchip-5.x (patch) VPU related patches on top of mmind/for-next: https://github.com/Kwiboo/linux-rockchip/compare/rockchip-5.x...rockchip-5.x-vpu (patch) I have started to send some of my patches to linux mailing list and more will follow.
  24. Kwiboo

    The VPU driver

    I was familiar with the vpu2 hw regs that needed to be set from creating an experimental mpeg-2 hwaccel for the rk vcodec kernel driver some time ago. After collaboration with @jernej to create a v4l2 request api hwaccel and getting it working with the Allwinner cedrus driver I learned enough v4l2 to get the rockchip vpu MPEG-2 decoder to work on my Rock64. Ezecquiel Garcia was then very helpful and got my initial work (decoder boilerplate was copied from encoder and chromium os) ready for upstream and submitted the MPEG-2 decoder for rk3399 on top of his decoder boilerplate work. RK3288 and RK3328 was left out as they requires clk and drm changes to work properly, patches are being prepared to be submitted. The rockchip-vpu-regtool was created to help set correct hw regs for both vpu1 and vpu2, mpeg2.txt was created based on mpp hal code, some imx-vpu-hantro code along with some docs was also useful to get more insights into the rockchip vpu.
  25. Kwiboo

    The VPU driver

    https://github.com/Kwiboo/rockchip-vpu-regtool may also be interesting, it generated the template vpu code used for the rk mpeg-2 decoder. I will push an update including txt-files with hw regs that needs to be set for other codecs later, buffer and reference frame handling code will need to be created by hand.
×
×
  • Create New...