duo2 6.1.34-sunxi ov5640 video capture kernel oops

[schunckt]

Hi there!
I've recently configured the ov5640 camera overlay and was able to capture videos either with mjpg-streamer of ffmpeg.

Now on 6.1.34 both causes now a kernel oops.
From the stack trace it looks like it happens inside v4l2 "media_pad_remote_pad_first"

  

Unable to handle kernel NULL pointer dereference at virtual address 00000030
...
...
PC is at media_pad_remote_pad_first+0xa/0x58

 

I don't think v4l2 is the root cause because this may mean it fails alle the time which is unlikely.
Looks like a null ptr gets passed in into media_pad_remote_pad_first

 

ffmpeg used like this, but other parameters all resulting in the kernel oops at the same location.

sudo media-ctl --device /dev/media1 --set-v4l2 '"ov5640 2-003c":0[fmt:UYVY8_2X8/1280x720@1/15]'
ffmpeg -hwaccel_device 0 -framerate 15 -f v4l2 -video_size 1280x720 -i /dev/video1 -r 15 -pix_fmt yuv420p -b:v 1280k test1280-15.mp4
WORKS on 4.14.x (was video0 and media0 here)

 

mjpeg streamer testing

./mjpg_streamer -i "./input_uvc.so -d /dev/video0 -y 1 -r 1920x1080 -f 5 -q 30 -n " -o "./output_http.so -w ./www"

 

 

Possible root causes:

1) Something is now wrong inside my DT overlay due to kernel changes? I'm thinking about the i2c and spi relationships

2) Something is broken inside ov5640 driver

3) Something is broken inside v4l

 

Maybe someone can provide some hints to point me in the right direction for further investigation?

 

Attached are the dmesg output and overlay source.

    Thilo

mjpg-streamer with camera overlay oops 06.08.2023 output.txt sun8i-h3-ov5640.dts

Edited August 7, 2023 by schunckt
added ffmpeg which also failed

[Gunjan Gupta]

Could you please reproduce the issue and then run `armbianmonitor -u` and share the link for the uploaded logs? I don't have a camera myself and I can't help until I see atleast the full oops message. 

 

Edit: Sorry it seems its still too early for me. Didn't saw your attached files. Going through the logs now will update in a bit.

[schunckt]

Done. Here is the link  https://paste.armbian.com/ufihilozam

 

 

  Thilo

[Gunjan Gupta]

I see you modified something in kernel config. Could you please let me know what were those modifications?

[Gunjan Gupta]

Also could you please try with latest nightly/rolling image from the download page just to see if this is already resolved in newer kernels?

[schunckt]

Just FYI i did update/upgrade recently and just NOW again but the result is the same.

 

Will test latest image just in case.

Thanks!

 

 

  Thilo

[schunckt]

Update: 

after boot simple

 v4l2-ctl -d /dev/video1 --all

also causes the crash.

 

 

Output ends here in the middle:

Driver Info:
        Driver name      : sun6i-csi
        Card type        : sun6i-csi
        Bus info         : platform:1cb0000.camera
        Driver version   : 6.1.34
        Capabilities     : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : sun6i-csi
        Model            : Allwinner A31 CSI Device
        Serial           :
        Bus info         : platform:1cb0000.camera
        Media version    : 6.1.34
        Hardware revision: 0x00000000 (0)
        Driver version   : 6.1.34
Interface Info:
        ID               : 0x03000003
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : sun6i-csi
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Sink, Must Connect
          Link 0x02000007: from remote pad 0x1000006 of entity 'ov5640 2-003c' (Camera Sensor): Data, Enabled
Priority: 2
Video input : 0 (Camera: ok)
Format Video Capture:
        Width/Height      : 1280/720
        Pixel Format      : 'BA81' (8-bit Bayer BGBG/GRGR)
        Field             : None
        Bytes per Line    : 1280
        Size Image        : 921600
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             :

 

 

Thilo

[Gunjan Gupta]

Interesting. I think it might have something to do with the driver for ov5640. Will get one for testing, but it will take about a week to get delivered. Will update once the driver arrives.

[going]

@Gunjan Gupta The problem may already be solved on the latest kernels. It may be enough to pull down some patches from megouse.

Compare:

megous-linux> gitk -100 origin/cam-6.1
megous-linux> gitk -100 origin/cam-6.4

And accept compliments in your address. You work very carefully. I adore.

[schunckt]

I've a armbian VM build environment ready, did build once successfully but I'm not experienced in linux.

Is there an easy way to find and integrate the changes mentioned above?

Then I could give it a try, maybe just the code to be pasted into the respecitve files...

 

 

   Thilo

[going]

1 час назад, schunckt сказал:

I've a armbian VM build environment ready, did build once successfully but I'm not experienced in linux.

Is there an easy way to find and integrate the changes mentioned above?

Then I could give it a try, maybe just the code to be pasted into the respecitve files...

Just build EDGE or install the EDGE kernel ready to confirm or refute my assumptions.

 

P.S. BRANCH=edge

Today it is the core v6.4.8

[going]

2 часа назад, schunckt сказал:

but I'm not experienced in linux

 

Check the kernels available for installation directly on the device

sudo apt update
apt search linux-image | grep sunxi

 

[schunckt]

Did build 'edge'  have now

Linux nanopiduo2 6.4.8-sunxi #1 SMP Thu Aug  3 08:26:15 UTC 2023 armv7l armv7l armv7l GNU/Linux

No more crashes 😀 but now no video🙄. mjpg_streamer says "cant enable video"

dmesg shows    sun6i-csi 1cb0000.camera: invalid input/output format combination

tried sveral params which did work in the past, no luck as yet. Will do further testing later on.

v4l2-ctl -d /dev/video1 --all

Driver Info:
        Driver name      : sun6i-csi
        Card type        : sun6i-csi-capture
        Bus info         : platform:1cb0000.camera
        Driver version   : 6.4.8
        Capabilities     : 0xa4200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x24200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : sun6i-csi
        Model            : Allwinner A31 CSI Device
        Serial           :
        Bus info         : platform:1cb0000.camera
        Media version    : 6.4.8
        Hardware revision: 0x00000000 (0)
        Driver version   : 6.4.8
Interface Info:
        ID               : 0x0300000e
        Type             : V4L Video
Entity Info:
        ID               : 0x0000000c (12)
        Name             : sun6i-csi-capture
        Function         : V4L2 I/O
        Pad 0x0100000d   : 0: Sink, Must Connect
          Link 0x02000010: from remote pad 0x1000003 of entity 'sun6i-csi-bridge' (Video Interface Bridge): Data, Enabled, Immutable
Priority: 2
Video input : 0 (sun6i-csi-capture: ok)
Format Video Capture:
        Width/Height      : 640/480
        Pixel Format      : 'YUYV' (YUYV 4:2:2)
        Field             : None
        Bytes per Line    : 1280
        Size Image        : 614400
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             :

looks good but that "sun6i-csi-bridge" seems to be different!?!

 

also i have spi-spidev overlay enabled but there is no /dev/spi*  

Maybe I made something wrong when building the image!?!?

 

ov5640 load shows

root@nanopiduo2:~# dmesg | grep ov5640
[    8.767826] ov5640 2-003c: supply DOVDD not found, using dummy regulator
[    8.768164] ov5640 2-003c: supply AVDD not found, using dummy regulator
[    8.768320] ov5640 2-003c: supply DVDD not found, using dummy regulator
[    8.775980] ov5640 2-003c: Consider updating driver ov5640 to match on endpoints

The last line is new i think. Need to figure out what that means...

 

 

   Thilo

Edited August 8, 2023 by schunckt
missed adding param_spidev_spi_bus=1 to armbianEnv.txt. Now SPI is visible in /dev

[schunckt]

Progress

still on nanopiduo2 6.4.8-sunxi

Meaniwhile i tried ustreamer which now WORKS at least with 640x480  like so:

v4l2-ctl  --device /dev/video1 --set-fmt-video=width=640,height=480,pixelformat=YUYV
media-ctl --device /dev/media1 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/640x480]'
./ustreamer -d /dev/video1 --host 192.168.179.55 -r 640x480

 

and VLC plays     http://192.168.179.55:8080/stream

 

will now go ahead trying ffmpeg and will test ustreamer on 6.1.x (But I'd prefer using latest build anyway!)

Thanks!

Thilo

[going]

OV5640_datasheet.pdf

 

support for output formats: RAW RGB,
RGB565/555/444, CCIR656, YUV422/420,
YCbCr422, and compression

 

maximum image transfer rate:
QSXGA (2592x1944): 15 fps
1080p: 30 fps
1280x960: 45 fps
720p: 60 fps
VGA (640x480): 90 fps
QVGA (320x240): 120 fps

 

And if you change the frequency, it is possible to increase the resolution. And how to do it?

[schunckt]

Did further play around with ffmpeg and v4l pipeline.

This now WORKS also 6.4.8-sunxi  (Armbian_23.08.0-trunk_Nanopiduo2_jammy_edge_6.4.8_minimal  build from 07-Aug)

v4l2-ctl --device /dev/video1 --set-fmt-video=width=1280,height=720,pixelformat=YUYV
media-ctl --device /dev/media1 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1280x720]'
ffmpeg -i /dev/video1 -q 10 -pix_fmt yuv420p -video_size 1280x720 -r 5 -listen 1 -f avi http://192.168.179.55:8080/stream

VLC
http://192.168.179.55:8080/stream

ffmpeg is  version 4.4.2-0ubuntu0.22.04.1

 

This means, there was a problem "behind the scenes" in 6.1x  which is now fixed.

Other parameters which did work in the past and now failing might be a side effect of internal improvments.

I'll figure out anyway because this kernel now also contains the cedrus video engine and I want to use that for encoding and decoding.

 

So from my point of view the issue is solved.

 

@going the highest resolution 2592x1944 did work in friendlycore xenial 4.14 so the driver WAS able to do so.

Don't know yet if something is now broken - but as said above need to play around anyways.

 

 

Thanks @all!

 

    Thilo

 

[schunckt]

@going it seems the higest resolution 2592x1944  fails because of this:

 

[  593.498632] cma: cma_alloc: reserved: alloc failed, req-size: 1846 pages, ret: -16
[  593.498684] sun6i-csi 1cb0000.camera: dma alloc of size 7561216 failed

 

already increased cma to 256M but doesnt help.

Anyway. Will proceed next with the encoder/decoder and fullhd 1920x1080 (works), that's more important for me.

 

  Thilo

[going]

1 час назад, schunckt сказал:

[  593.498632] cma: cma_alloc: reserved: alloc failed, req-size: 1846 pages, ret: -16
[  593.498684] sun6i-csi 1cb0000.camera: dma alloc of size 7561216 failed

I am a bad advisor in this kernel subsystem. But in dts from manufacturers,

I have seen lines that reserve a certain amount of memory in a fixed region for a specific purpose.
Check the processor's data sheet. Maybe it's a hardware limitation?

[Gunjan Gupta]

I got my camera delivered yesterday. Will test and fix its working on 6.1 kernel.

 

@schunckt Could you please try building the image with BRANCH=legacy and see if the included 5.15 kernel works with 2592x1944 resolution? Asking this as 6.1+ kernels introduced maple-tree for memory management and I have seen some cases where memory allocation is failing for 32-bit Allwinner boards.

[schunckt]

@going "Maybe it's a hardware limitation?"  this can't be true because highest resolution works with the "original" friendlyCore 4.14 image/kernel.

 

@Gunjan Gupta im quite sure 2592x1944 worked in Armbian_21.08.1_Nanopiduo2_focal_current_5.10.60

but i can retry using the prepared SD card (I hope this still is there 🙂)

About 6.1:    I don't mind if it only works on the latest 6.4x series, I'm fine staying there and proceed with other development like hwaccell and other stuff, like spi display retest and the like.

 

FYI using the "friendly" I already was able to capture and playback on fbi / spi display, other i2c sensors are working, ola DMX output works, .net6 works with swagger UI ...

 

Goal is have this all working in latest armbian plus hwaccel (if possible)

 

btw. if you really find the root cause in 6.1 don't hesitate to share the source code of the problematic area. I'm curious... Thanks!

 

   Thilo

Edited August 16, 2023 by schunckt

[Gunjan Gupta]

@schunckt I have disabled patch/kernel/archive/sunxi-6.1/patches.megous/media-sun6i-csi-Pass-on-g_parm-s_parm-to-the-subdev.patch as that was the code causing the kernel oops. You should be able to use the camera now with 6.1 kernel as well 

[schunckt]

Great!

 

I'll give it a try, likely next weekend.

 

  Thilo

[schunckt]

I can confirm 6.1.47 now works!

One caveat: Highest resolution 2592x1944 still has the cma/dma memory error like  6.4.x.

But as of now I don't mind.

 

I'll focus next on the harware encoding and other hardware resting,

 

 

   Thilo

[Gunjan Gupta]

@schunckt Thanks for confirmation. Could you please vote one of the responses as answer using '^' symbol on the left of the comment . That will make the question appeared as answered (green check mark) in the forum