Banana Pi M64 (Allwinner A64): Getting the OV5640 camera to work under armbian with kernel 5.8

[@lex]

1 hour ago, Levent Erenler said:

i thought csi camera uses GPU to grab image.

No GPU/VPU involved.

 

1 hour ago, Levent Erenler said:

Cpu usage around %70 with opencv V4l2 capture fmt:UYVY8_2X8/640x480@30FPS and %60 with JPEG_1X8/640x480@30FPS

I think there is a conversion from YUV to rgb in opencv and possible a decompression from jpeg to rgb, i am not an opencv expert, maybe someone can give more details about what's going on inside opencv. There is still room to optimize the JPEG_1x8.

 

1 hour ago, Levent Erenler said:

Usb ov5640 cpu usage around %25

That's pretty good. There must be no conversion in the image format to achieve this. It is interesting to find out more about it. Can you share more about your application and your setup and usb camera?

[Levent Erenler]

5 minutes ago, @lex said:

There must be no conversion in the image format to achieve this. It is interesting to find out more about it. Can you share more about your application and your setup and usb camera?

yes,  u are right that means there is no conversion. The app is very simple;  open camera with opencv videocapturer and grap image with opencv(v3.4.10) and calculate fps (no display). Setup Pine64+ 1GB A64 Kernel 5.9.3 with HBV1318 ov5640 usb camera

 

cpu usage is %6-8 at each core and total cpu usage %25 -31 ( 640*480*30FPS) 

 

device listed as 

doccamera: doccamera (usb-1c1a000.usb-1):
        /dev/video4
        /dev/video5
 

[@lex]

Ok. 

In the JPEG_1X8/640x480@30FPS case you should expect a bit more cpu usage than the USB camera, say ~5% more cpu usage and not the ~35% increase you see. The reason is you get 640x480x3 pixels from the sensor instead of only the compressed image.

I would build opencv with debugging info and try to find the bottleneck. And why there is an image conversion in the DVP case.

Push the limits to 720P/30fps or even 1080P and see what you get.

 

 

[Levent Erenler]

On 11/16/2020 at 4:50 PM, @lex said:

I would build opencv with debugging info and try to find the bottleneck. And why there is an image conversion in the DVP case.

Push the limits to 720P/30fps or even 1080P and see what you get.

Thanks for your support, I prefer 640*480 cause opencv face or tag recognition consume lots of cpu power,  640*480@30 is acceptable. I try a few more resolution and its getting worst

 

[fmt:JPEG_1X8/1280x720]   %100 cpu usage limited 15 FPS, could not reach more fps,  cpu usage at single core that means not multithreaded
[fmt:UYVY8_2X8/1280x720]  // %190 cpu usage @30 fps ,multi threaded
[fmt:UYVY8_2X8/1980x1080] // could not handle 30fps ,%200 cpu usage at 15FPS 
 

 

UPDATE: my best scenario is fmt:UYVY8_2X8 and opencv CV_CAP_PROP_FOURCC property is CV_FOURCC('Y', 'U', '1', '2') Cpu usage ~%52

[@lex]

Can you post the format info for your USB camera?

 

v4l2-ctl -d 4 --list-formats

or

v4l2-ctl -d 5 --list-formats

 

I have seen some USB camera has a YUY2 format.

 

Cpu usage ~52% looks good for OpenCV.

[FeiJi]

I try to Enable OV5640 on Banana Pi M64 too;used  armbian with kernel 5.10; but when kernel start OV5640 has an error

[    6.771712] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/1 wl0: Mar 30 2016 11:30:56 version 7.45.77.h8.4 FWID 01-ee8a6268
[    7.185164] ov5640 1-003c: ov5640_read_reg: error: reg=300a
[    7.185180] ov5640 1-003c: ov5640_set_power_on: failed to read chip identifier
[    7.904788] Adding 985724k swap on /dev/zram0.  Priority:5 extents:1 across:985724k SSFS
[    9.133355] zram1: detected capacity change from 0 to 52428800

 I added this to sun50i-a64-bananapi-m64.dts:

i2c-csi {
		compatible = "i2c-gpio";
		sda-gpios = <&pio 4 13 GPIO_ACTIVE_HIGH>; /* PE13 */
		scl-gpios = <&pio 4 12 GPIO_ACTIVE_HIGH>; /* PE12 */
		i2c-gpio,delay-us = <5>;
		#address-cells = <1>;
		#size-cells = <0>;

		ov5640: camera@3c {
			compatible = "ovti,ov5640";
			reg = <0x3c>;
			pinctrl-names = "default";
			pinctrl-0 = <&csi_mclk_pin>;
			clocks = <&ccu CLK_CSI_MCLK>;
			clock-names = "xclk";

			AVDD-supply = <&reg_dldo3>;
			DOVDD-supply = <&reg_aldo1>;
			DVDD-supply = <&reg_eldo3>;
			reset-gpios = <&pio 4 16 GPIO_ACTIVE_LOW>; /* PE16 */
			powerdown-gpios = <&pio 4 17 GPIO_ACTIVE_HIGH>; /* PE17 */

			port {
				ov5640_ep: endpoint {
					remote-endpoint = <&csi_ep>;
					bus-width = <8>;
					hsync-active = <1>; /* Active high */
					vsync-active = <0>; /* Active low */
					data-active = <1>;  /* Active high */
					pclk-sample = <1>;  /* Rising */
				};
			};
		};
	};

&csi {
	status = "okay";

	port {
		#address-cells = <1>;
		#size-cells = <0>;

		csi_ep: endpoint {
			remote-endpoint = <&ov5640_ep>;
			bus-width = <8>;
			hsync-active = <1>; /* Active high */
			vsync-active = <0>; /* Active low */
			data-active = <1>;  /* Active high */
			pclk-sample = <1>;  /* Rising */
		};
	};
};

and try change I2C to:

        sda-gpios = <&pio 4 13 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN|GPIO_PULL_UP)>; /* CSI0-SDA: PE13 */
        scl-gpios = <&pio 4 12 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN|GPIO_PULL_UP)>; /* CSI0-SCK: PE12 */

but the result is the same

 

I just check my I2C setting,it seems right

root@bananapim64:~# cat /boot/config-`uname -r` | grep I2C | grep GPIO
CONFIG_I2C_ARB_GPIO_CHALLENGE=m
CONFIG_I2C_MUX_GPIO=m
CONFIG_I2C_CBUS_GPIO=m
CONFIG_I2C_GPIO=m
# CONFIG_I2C_GPIO_FAULT_INJECTOR is not set
# I2C GPIO expanders
# end of I2C GPIO expanders

 

also I have add sun6i_csi and ov5640

root@bananapim64:~# lsmod|grep ov
ov5640                 28672  1
v4l2_fwnode            24576  2 ov5640,sun6i_csi
videodev              233472  9 sunxi_cedrus,sun8i_rotate,v4l2_fwnode,ov5640,videobuf2_v4l2,sun8i_di,sun6i_csi,videobuf2_common,v4l2_mem2mem
mc                     49152  7 sunxi_cedrus,videodev,ov5640,videobuf2_v4l2,sun6i_csi,videobuf2_common,v4l2_mem2mem

root@bananapim64:~# dmesg | grep sun6i-csi
[    5.985118] sun6i-csi 1cb0000.csi: creating ov5640 1-003c:0 -> sun6i-csi:0 link
root@bananapim64:~# 

 

[Levent Erenler]

On 11/21/2020 at 4:41 PM, @lex said:

Can you post the format info for your USB camera?

 

v4l2-ctl -d 4 --list-formats

or

v4l2-ctl -d 5 --list-formats

 

I have seen some USB camera has a YUY2 format.

 

Cpu usage ~52% looks good for OpenCV.

 

Sorry @@lex for late reply, i did not see any notification until last message. Thank you

usb camera formats are;

ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'MJPG' (Motion-JPEG, compressed)
        [1]: 'YUYV' (YUYV 4:2:2)
 

[Levent Erenler]

hi @FeiJi i dont have a bpi m64, so i dont have a chance to test it but i will try on pine64 with 5.10 kernel. A few months ago, I have a similar problem because of broken camera. Everything were looking normal but i was getting error from driver when i checked dmesg. I change cam and it works. Do you have chance to try different camera.  

[Eemeien]

Hi @Robert Heller how did you connect your camera to your board? could you please share a photo? I tried all things in this topic but there is no success. ı see "i2c-gpio i2c-csi: using lines 141 (SDA) and 140 (SCL)" this output in dmesg and lsmod outputs but when ı try "v4l2-ctl --list-devices", I had error "/dev/video0 invalid argument". Did you use linux default ov5640 driver or anything?