0
@lex

OV5640 on mainline kernel

Recommended Posts

(edited)

:)Today I had a chance to test OV5640 on mainline kernel 4.17.2 and see the status of OV5640 and CSI drivers, thanks to FE work and the author of the driver (help name here...). I tested on NanoPi K1 Plus (H5) to verify the images in very low light conditions, so don't expect good quality.

I could take some images using fswebcam and you should expect basic v4l2 functionality already works if not all. I think motion (did not test  / had time to test it) can work with current OV5640 on mainline kernel.

Grabbing Image is very fast, currently, Image size I can get are 320x240 and 640x480 pixels, above this i get a dark image. I have not looked at the driver source code to see if it is implemented or not.  

 

Basically, you need to add the sun6i_csi and ov5640 drivers to the kernel and adjust DT to have the endpoint. FE has already done it and left H5 for homework, but is the same as H3.

 

To check the functionality, install v4l2-utils and when the driver sun6i_cs loads, it creates the device node /dev/video0 and you can check that:

 

sudo v4l2-ctl -d /dev/video0 -D
Driver Info (not using libv4l2):
    Driver name   : sun6i-video
    Card type     : sun6i-csi
    Bus info      : platform:camera
    Driver version: 4.17.2
    Capabilities  : 0x84200001
        Video Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps   : 0x04200001
        Video Capture
        Streaming
        Extended Pix Format

You should see the modules loaded like this:

lsmod
Module                  Size  Used by
sun6i_csi              24576  0
videobuf2_dma_contig    20480  1 sun6i_csi
videobuf2_memops       16384  1 videobuf2_dma_contig
ov5640                 36864  1
videobuf2_v4l2         24576  1 sun6i_csi
videobuf2_common       40960  2 videobuf2_v4l2,sun6i_csi
v4l2_fwnode            20480  2 ov5640,sun6i_csi
v4l2_common            16384  1 ov5640
videodev              196608  6 v4l2_fwnode,v4l2_common,ov5640,videobuf2_v4l2,sun6i_csi,videobuf2_common
sunxi_cir              16384  0
media                  36864  3 videodev,ov5640,sun6i_csi
rc_core                40960  2 sunxi_cir
sch_fq_codel           20480  6
8189es               1118208  0

So time to revisit the camera drivers on mainline.

Here are some images (remember, low light condition!):

 

*update: All possible window sizes are working for streaming!

 

 

 

cam320x240.jpg

cam640x480.jpg

Edited by @lex
Image resolutions

Share this post


Link to post
Share on other sites

If anyone is wondering how is the performance of the driver, just take a look:

---- cap parameters -----
width: 1920
height: 1080
v4l2 buffers: 4
exposure: -999
hflip: -1
vflip: -1
Mode: V4L2_MODE_VIDEO
Driver: "sun6i-video"
Card: "sun6i-csi"
Bus: "platform:camera"
Version: 1.0
Capabilities: 84200001
Input: 0
V4L2 pixel formats:
0: [0x59565955] 'UYVY' (UYVY 4:2:2)
1: [0x32314D48] 'HM12' (YUV 4:2:0 (16x16 Macroblocks))
2: [0x3231564E] 'NV12' (Y/CbCr 4:2:0)
3: [0x3132564E] 'NV21' (Y/CrCb 4:2:0)
4: [0x32315559] 'YU12' (Planar YUV 4:2:0)
5: [0x32315659] 'YV12' (Planar YVU 4:2:0)
6: [0x3631564E] 'NV16' (Y/CbCr 4:2:2)
7: [0x3136564E] 'NV61' (Y/CrCb 4:2:2)
8: [0x50323234] '422P' (Planar YUV 4:2:2)
9: [0x56595559] 'YUYV' (YUYV 4:2:2)
10: [0x32314D48] 'HM12' (YUV 4:2:0 (16x16 Macroblocks))
11: [0x3231564E] 'NV12' (Y/CbCr 4:2:0)
12: [0x3132564E] 'NV21' (Y/CrCb 4:2:0)
13: [0x32315559] 'YU12' (Planar YUV 4:2:0)
14: [0x32315659] 'YV12' (Planar YVU 4:2:0)
15: [0x3631564E] 'NV16' (Y/CbCr 4:2:2)
16: [0x3136564E] 'NV61' (Y/CrCb 4:2:2)
17: [0x50323234] '422P' (Planar YUV 4:2:2)
V4L2 pixel sizes:

Sensor size: 1920x1080 pixels
Pixel Format: V4L2_PIX_FMT_YUV420 [0x32315559]
Frame #10 will be saved!
Length: 3110400     Bytesused: 3110400     Address: 0x55b01b9670
FPS[0]: 25.77
Length: 3110400     Bytesused: 3110400     Address: 0x55b01b9680
FPS[1]: 30.06
Length: 3110400     Bytesused: 3110400     Address: 0x55b01b9690
FPS[2]: 30.01
Length: 3110400     Bytesused: 3110400     Address: 0x55b01b96a0
FPS[3]: 29.79
Length: 3110400     Bytesused: 3110400     Address: 0x55b01b9670
FPS[4]: 30.32
Length: 3110400     Bytesused: 3110400     Address: 0x55b01b9680
FPS[5]: 30.04
Length: 3110400     Bytesused: 3110400     Address: 0x55b01b9690
FPS[6]: 29.95
Length: 3110400     Bytesused: 3110400     Address: 0x55b01b96a0
FPS[7]: 30.14
Length: 3110400     Bytesused: 3110400     Address: 0x55b01b9670
FPS[8]: 29.81
Length: 3110400     Bytesused: 3110400     Address: 0x55b01b9680
FPS[9]: 0.72
------- Avg FPS: 30.02 --------

 

Share this post


Link to post
Share on other sites
On 6/17/2018 at 11:45 PM, @lex said:

:)Today I had a chance to test OV5640 on mainline kernel 4.17.2 and see the status of OV5640 and CSI drivers, thanks to FE work and the author of the driver (help name here...). I tested on NanoPi K1 Plus (H5) to verify the images in very low light conditions, so don't expect good quality. [...]

 

 

This sounds very interesting!
But ... I can not find the interface driver (sun6i_csi). This is the Maxime Ripard's patch to mailine kernel or another one?

For the testing: I've a Orange Pi PC2 H5 and a OV5647 NoIR CSI.

Share this post


Link to post
Share on other sites

Here is a diff patch for mainline kernel.

Apply from the kernel root tree. Enter menuconfig to Enable VIDEO / CSI , don't know about your camera. or how you wire it.

 

patch -p0 -i sun6i_csi.dif.patch 

 

sun6i_csi.dif.patch.tar.gz

Share this post


Link to post
Share on other sites

It looks like Yong Deng's patch for V3s CSI.

 

+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing)
+ * All rights reserved.
+ * Author: Yong Deng <yong.deng@magewell.com>
+ */

 

Here:

https://www.spinics.net/lists/linux-media/msg130006.html

http://linux-sunxi.org/Linux_mainlining_effort

Share this post


Link to post
Share on other sites

where did you get the ov5640 driver?

 

trying now to get ov5640 on a allwinner A33 running.

 

my status till now:

flashed new armbian stretch build on sdcard ( 4.14.69-sunxi)

 

found out that sun6i_csi works only with v4l2 interface with kernel 4.15+

upgraded to 4.18.5 (linux-dtb-dev-sunxi_5.59.180904_armhf.deb, linux-headers-dev-sunxi_5.59.180904_armhf.deb, linux-image-dev-sunxi_5.59.180904_armhf.deb) from https://mirror.applebred.net/armbian-beta/pool/main/l/linux-4.18.5-sunxi/

 

was able to build sun6i_csi, however it looks like its already buildin, now stuck with loading / compiling the ov5640 driver to get a /dev/video device

 

Anyone an idea where to get this driver?

 

yours,
Manuel

 

 

Share this post


Link to post
Share on other sites

Hi,

 

I'm struggling to get the driver working on my V3s. Could you please show me your devicetree files and which command you used to capture the images? I tried fswebcam, but I only get a greenish image however I can see the shapes of things.

 

thanks

Share this post


Link to post
Share on other sites

The device tree: https://github.com/armbian/build/commit/10e206519089b8ff81b0dd6a6f3caa6c1adba04d#diff-5ec21c7daf434027694931230901ca60R96

If I recall correctly the driver (specific to FE boards) delivers YUV so you need to instruct fswebcam and cap-v4l2 to receive V4L2_PIX_FMT_YUYV. You can try mjpg-streamer that does that by default. Try 640x480 window size first and then move to 720p or higher.

 

 

I have noticed there is (or there was) a slightly new/modified driver on mainline kernel 4.19.0 but did not have time to check the changes. That mainline driver has been originally written to v3s, you may check linux-sunxi. I don't know v3s so can't help much.

 

 

Share this post


Link to post
Share on other sites

@@lex, can you show the result (dt_full.dts) of the command: dtc -I fs -O dts -o dt_full.dts /sys/firmware/devicetree/base/ , please?

I think this patch work only for nano_pi, and on my orangepi i have other DT description. (i am a newbie)

Share this post


Link to post
Share on other sites

The patch only adds the c source code to the linux kernel. As I posted on your thread you need to rebuild the kernel and change the devicetree source files to support the CSI /DVP and the ADV7611.

Share this post


Link to post
Share on other sites

Hello. Can you give an common advise.

In order to use ov5640 you add the sun6i_csi and ov5640 drivers and adjust DT to have the endpoint.  Did you take the ov5640 driver from mainline kernel. Right? 

 

So, main task is to use ov2710. The sun6i_csi driver I can load with modprobe (the mainline kernel has it). But where can i find the ov2710 driver. 

I found 2 drivers for ov2710.

The first is from linux-3.4-sunxi and it has vfe functions.  It turns out, that I have to rewrite that driver (remove vfe) in order to compile it. Do I get it right?  

The second driver is from tegra. That ov2710 driver does not contain vfe.  But I don't know if that driver suit for me. What do you think about it?

Thanks. 

Share this post


Link to post
Share on other sites

so the csi camera is now working in mainline kernel? woow

The device tree stuff is implemented,csi, i2c and clock are supported?

 

as to this vfe_v4l2 driver, this confuses me too, i think it originates from qualcomm. (video front end), rather complicated.

 

mainline uses the videobuf2 interface, so drivers need to be modified, i guess.

Share this post


Link to post
Share on other sites

i have had success with OV5640 on FE HW (H3,H5,A64, possibly H2+)

You should check camera schematic of your board if it matches the same way FE handle the camera. I think I2C must be tied to CSI or you cannot communicate with the sensor. 

 

vfe_v4l2 driver seems to be from Android and has additional Image Processing in it. (ISP)  (Kernel 3.x and 4.4.y, perhaps 4.9.y).

 

I think there is still no support for OPIs, BPIs and some others for the mainline OV5640 / OV5647 and you have to use the Android kernel way or until someone writes the code.

Share this post


Link to post
Share on other sites

Hai @lex 

In my orange pi pc2, i'm load sun6i_csi and ov5640 module on boot. Now to connect ov5640, need adjust DT Script. Can i see DT Script for ov5640 ?

I already tried to adjust DT but on dmesg, i see error : ov5640_read_reg: error: reg=300a. how to fix this ?

Share this post


Link to post
Share on other sites

You can refer to this patch: https://lkml.org/lkml/2018/11/27/1041  https://patchwork.kernel.org/patch/10783429/

If this works for banana pi it will work for orange pi.

DO NOT MERGE is there for a reason.

 

ov5640_read_reg: error: reg=300a.

It usually means it cannot communicate with the sensor.

 

Use this:

PE12/CSI_SCK/TWI2_SCK ==> CSI-SCK
PE13/CSI_SDA/TWI2_SDA ==> CSI-SDA
PE14 ==> CSI-RESET#
PE15 ==> CSI-STBY-EN

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
0