OV5640 device tree overlay for OrangePi One H3


rreignier
 Share

6 6

Recommended Posts

Donate and support the project!

@MikePooh, I think there's still some schematic-reading involved at the moment, if you're willing to do that though it should be do-able. I don't know anything about other boards yet so this is the process followed. I think for "it just works" you'll need to wait a bit more. I'm assuming a lot here, e.g. active high and fixed-voltage gpio-switched regulators, so there is risk in following this.


The first step is getting i2c working, the driver won't be able to do anything until you can communicate with the camera. I'll use the h5 in my Orange Pi Zero Plus2 as an example but I believe the process should be identical for h3 boards. Looking at the schematic (https://linux-sunxi.org/images/f/f6/ORANGE_PI-ZERO-PLUS2_V1_0.pdf), on page 9 is the csi connector. We want AFCC_EN, CSI_EN and CSI-PWR-EN to be '1' to power the camera. They might be named differently on different boards, but generally you're looking for the ones that aren't connected to pins (on page 6,  chip U1D) with a CSI_* function (e.g. PE9/CSI_D5/TS_D5). I'm looking for AFCC_EN and CSI-PWR-EN which link to page 6. So AFCC_EN comes from pin PA7/SIM_CLK/PA_EINT7 and CSI-PWR-EN from PA8/SIM_DATA/PA_EINT8. We need to remember those PXX numbers.
 

rreignier said their camera was on the i2c1 bus, so we need to confirm that too. Following CSI-SCK and CSI-SDA from page 9 to page 6 reveals the i2c interface. CSI-SCK for me connects to "PE12/CSI_SCK/TWI2_SCK" and CSI-SDA to "PE13/CSI_SDA/TWI2_SDA". Maybe someone can confirm, but I think TWIx_SDA equals I2Cx. You can test by enabling the i2c bus overlays one after the other and running i2cdetect while measuring with a multimeter on low range (200mv) A/C voltage between ground and the SCK pin on the camera connector - the correct bus will generate a voltage for a short while. We need to remember that bus number.

Note the reset-gpios and powerdown-gpios the same way.


Next is the change to the device tree. In Armbian checkout the linux source  with "apt install linux-source-<kernel version>-current-sunxi" (or linux-source-<kernel version>-current-sunxi64 for h5) which will leave you with a tar in /usr/src of your kernel source. extract it (do it in a new folder or it will trash your current one) and then copy your kernel config in (cp /boot/config-* ./.config). Edit your .dts (./arch/arm/boot/dts/ for h3, ./arch/arm64/boot/dts/allwinner/ for h5) in line with my previous comment, swapping in your values for regulators, i2c bus and reset/powerdown. for a quick bodge, which pin goes in which regulator doesn't actually matter - my change just forced them all on anyway. As lex said, there's supposed to be an order, my change doesn't respect that order but it has worked so far. Then run "make dtbs" from the top level of the kernel source. That should produce a .dtb file in the same folder as the .dts.


backup your current dtb and copy the new one to /boot/dtb/allwinner/*.dtb (that's the h5 path, not sure of the h3 path sorry, might be /boot/dtb/). I think you can alternatively specify this as fdtfile in the armbianEnv.txt. The armbianEnv.txt way will probably survive an update better.


Finally, add a line to /etc/modules that says ov5640. Reboot and you might be good to go. If you don't have a /dev/video0 device we'll have to debug.

Obviously if I've got something wrong feel free to chip in.

Has anyone had success with resolutions other than 640x480? The driver seems to have some code for it, but I haven't gotten anything to work.

Link to post
Share on other sites

On 6/21/2020 at 9:26 PM, gsumner said:

XVCLK (CLK_CSI_MCLK in the device tree) needs to be running before accessing registers. I couldn't see an a/c voltage on the csi-mclk pin so figured that was the issue. The fix for me was connecting the clock parent in the device tree. I'll post my dtb changes. I'm not sure this is all correct, but it lets me open the camera without having to mess with any gpios etc after boot.

Thank you! That's the part that i was missing. 

Now i'm able to run ov5640 on OrangePiPcPlus with kernel 5.4.45. My dts: https://pastebin.com/gdQnnrxQ

/dev/video1 appears after reboot (no manual modprobe ov5640 needed). ov5640 for some reason does not leave any footprint in dmesg.

I've kept compatible = "allwinner,sun6i-a31-i2c" in i2c2@1c2b400 to be able to check chip id by i2cdetect. And nothing was working without compatible = "allwinner,sun8i-h3-csi" and other stuff inside csi@1cb0000.

 

Would be nice to pull the solution into official build (maybe as overlay) ;) . Ov5640 module is one of very few official orangepi accessories

 

16 hours ago, gsumner said:

Has anyone had success with resolutions other than 640x480?

 

according to https://linux-sunxi.org/CSI

media-ctl --device /dev/media1 --set-v4l2 '"ov5640 1-003c":0[fmt:UYVY/1920x1080]'

1920x1080 and 1280x760 are working well for me

fswebcam -d /dev/video1 -r 1920x1080 -D 3 --jpeg 100 /mnt/sd/aaa.jpg
--- Opening /dev/video1...
Trying source module v4l2...
/dev/video1 opened.
No input was specified, using the first.
Delaying 3 seconds.
--- Capturing frame...
Captured frame in 0.00 seconds.
--- Processing captured image...
Setting output format to JPEG, quality 100
Writing JPEG image to '/mnt/sd/aaa.jpg'.

 

Link to post
Share on other sites

8 hours ago, mostly said:

according to https://linux-sunxi.org/CSI


media-ctl --device /dev/media1 --set-v4l2 '"ov5640 1-003c":0[fmt:UYVY/1920x1080]'

 

@mostly awesome, thanks! I tried a couple others for anyone else wondering. e.g.:

media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:JPEG_1X8/1920x1080]'
v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=JPEG
fswebcam -d /dev/video0 -r 1920x1080 -D 0 --jpeg 100 /tmp/test.jpg

The colour balance and noise are terrible, but it's a start! I see there's something in media-ctl to change colourspace, but it's the first time I've heard of media-ctl and I've got no idea what I'm doing yet. Still no luck with 2592x1944 which I'd like to get going though.
 

@jgauthier I'm not sure you need a bitbang i2c, the schematic for your board (https://drive.google.com/drive/u/0/folders/0B4PAo2nW2KfnflVqbjJGTFlFTTd1b1o1OUxDNk5ackVDM0RNUjBpZ0FQU19SbDk1MngzZWM?tid=0B4PAo2nW2Kfndjh6SW9MS2xKSWs) says they connect to TWI2.  If the TWIx == I2Cx thing holds, I2C2 might work for you(?).  It looks like CSI-PWR-EN controls all the power to your camera too, that net might be strongly pulled-up next to U9 too. So maybe try enable PD14 for your regulator and I2C2 for the bus.

Link to post
Share on other sites

On 6/25/2020 at 10:53 PM, gsumner said:

The colour balance and noise are terrible, but it's a start! I see there's something in media-ctl to change colourspace, but it's the first time I've heard of media-ctl and I've got no idea what I'm doing yet. Still no luck with 2592x1944 which I'd like to get going though.

Yes noise is annoying. Maybe "5.6 color interpolation (CIP)" mentioned in ov5640 datasheet can deal with it (CIP seems to be already enabled in driver initialization, maybe tuning of CIP parameters can help).

Also not sure what to do with autofocus. The driver also has nothing for it, but i have not found in the datasheet how to enable it.

 

One problem found with 1920x1080 resolution: still image capture with fswebcam works but attempt to get 1080p video with ffmpeg gives noisy greenish picture where only contours of objects can be recognized, exactly like in that message:

 

1280x720 is fine.

Link to post
Share on other sites

I think I understand that media-ctl is setting the resolution/format (or group of formats) that will be able to be successfully opened by a program. I'm confused by the media-ctl, v4l2-ctl and program order of operations though, it seems like you have to run your program first, see it fail, run v4l2-ctl -V to see the (last?) format it tried to open, then you go back and set something compatible with media-ctl and try again... is that correct? Surely it should see what the program is asking for and reconfigure itself?
 

With fswebcam (which seems to want JPEG format), running: 

media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:JPEG_1X8/1920x1080]'
fswebcam -d /dev/video0 -r 1920x1080 -D 0 --jpeg 100 /tmp/test.jpg

there's a weird issue where 1 in 3 or 4 images has a green tinge. All the detail is there, just greenish. Does anyone have the same issue?
 

I'm also having an issue with vlc, it seems to want planar yuv (YUYV8_2X8), but I can't get a successful stream out of it. cvlc complains "rawvideo decoder warning: invalid frame size (3110400 < 4665600)", which agrees with the "Size Image" field of v4l2-ctl -V.  I wonder if the frame size error is related to the green/purple image problem or separate, 3110400 means 12bit/pixel while 4665600 is 18bit/pixel. The format and v4l2-ctl -V agree on 3110400 so I wonder where 4665600 comes from...  and I also don't understand how to differentiate yuv422 and yuv420 from media-ctl --known-mbus-fmts.


@mostly, ISP CONTROL 03 "Draw window for AFC enable" looks interesting for autofocus. I might try enabling that to see what happens. I wonder if the autofocus is a oneshot thing that the driver would need to fire off every time or constantly focusing. I'm also having the same issues with YUV green/purple images that you mention, if you run ffmpeg with "-input_format mjpeg" you can at least get back to the "green tinge" problem.

I tried signing the driver I built previously with the keys I found in the Linux source package but I keep getting exec format error, I was hoping to avoid rebuilding the whole kernel. Are the signing keys available? Maybe that defeats the purpose of keys though!

I guess if swapping between mjpeg and yuv is only reconfiguring the soc and not the camera module itself then we should look at sun6i_csi for these format and frame size issues(?). 

Link to post
Share on other sites

On 6/30/2020 at 7:33 PM, gsumner said:

there's a weird issue where 1 in 3 or 4 images has a green tinge. All the detail is there, just greenish. Does anyone have the same issue?

I'm not experiencing that, all still image captures are ok, both at 720p and 1080p. But problem with green 1080p video capture still here.

On 6/30/2020 at 7:33 PM, gsumner said:

ISP CONTROL 03 "Draw window for AFC enable" looks interesting for autofocus

Somebody was trying this register but it seems to be not the right one: https://patchwork.kernel.org/patch/10342907/

There is one interesting commit claiming adding autofocus: https://github.com/D3Engineering/linux_kernel_qcomlt/blob/7b3c6e86d236db6d0c523dbd3297deb7563d7bf1/drivers/media/i2c/ov5640.c but it's for legacy 

 

On 6/30/2020 at 7:33 PM, gsumner said:

but I keep getting exec format error

This way of building modules works for me: https://yoursunny.com/t/2018/one-kernel-module/ + touch .scmversion

Link to post
Share on other sites

@mostly Thanks, not sure why it wasn't working before, I can build modules now and signing turned out to not be needed.

I've been looking at the sun6i_csi, sun6i_video and ov5640 drivers, at the moment I think there is a problem in the clock generation in the ov5640 code. I'll share my test patch:

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 18dd2d717..7b5166b11 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -470,54 +470,106 @@ static const struct reg_value ov5640_setting_PAL_720_576[] = {
 };
 
 static const struct reg_value ov5640_setting_720P_1280_720[] = {
-	{0x3c07, 0x07, 0, 0},
-	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
-	{0x3814, 0x31, 0, 0},
-	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
-	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
-	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0},
-	{0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0},
-	{0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x72, 0, 0}, {0x3a0e, 0x01, 0, 0},
-	{0x3a0d, 0x02, 0, 0}, {0x3a14, 0x02, 0, 0}, {0x3a15, 0xe4, 0, 0},
-	{0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0},
-	{0x4407, 0x04, 0, 0}, {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0},
-	{0x3824, 0x04, 0, 0}, {0x5001, 0x83, 0, 0},
+	{0x3612, 0x29, 0, 0}, /* ??? between VCM and timing control */
+	{0x3618, 0x00, 0, 0}, /* ??? between VCM and timing control */
+
+	{0x3708, 0x64, 0, 0}, /* ??? between VCM and timing control */
+	{0x3709, 0x52, 0, 0}, /* ??? between VCM and timing control */
+	{0x370c, 0x03, 0, 0}, /* ??? between VCM and timing control */
+
+	{0x3800, 0x00, 0, 0}, /* TIMING HS */
+	{0x3801, 0x00, 0, 0}, /* TIMING HS */
+	{0x3802, 0x00, 0, 0}, /* TIMING VS */
+	{0x3803, 0xfa, 0, 0}, /* TIMING VS */
+	{0x3804, 0x0a, 0, 0}, /* TIMING HW */
+	{0x3805, 0x3f, 0, 0}, /* TIMING HW */
+	{0x3806, 0x06, 0, 0}, /* TIMING VH */
+	{0x3807, 0xa9, 0, 0}, /* TIMING VH */
+	{0x3810, 0x00, 0, 0}, /* TIMING HOFFSET */
+	{0x3811, 0x10, 0, 0}, /* TIMING_HOFFSET */
+	{0x3812, 0x00, 0, 0}, /* TIMING VOFFSET */
+	{0x3813, 0x04, 0, 0}, /* TIMING VOFFSET */
+	{0x3814, 0x31, 0, 0}, /* TIMING X INC */
+	{0x3815, 0x31, 0, 0}, /* TIMING Y INC */
+	{0x3824, 0x04, 0, 0}, /* DVP PCLK divider for VFIFO CTRL0C */
+
+	{0x3a02, 0x02, 0, 0}, /* AEC MAX EXPO (60HZ) */
+	{0x3a03, 0xe4, 0, 0}, /* AEC MAX EXPO (60HZ) */
+	{0x3a08, 0x01, 0, 0}, /* AEC B50 STEP */
+	{0x3a09, 0xbc, 0, 0}, /* AEC B50 STEP */
+	{0x3a0a, 0x01, 0, 0}, /* AEC B60 STEP */
+	{0x3a0b, 0x72, 0, 0}, /* AEC B60 STEP */
+	{0x3a0e, 0x01, 0, 0}, /* AEC CTRL0E */
+	{0x3a0d, 0x02, 0, 0}, /* AEC CTRL0D */
+	{0x3a14, 0x02, 0, 0}, /* AEC MAX EXPO (50HZ) */
+	{0x3a15, 0xe4, 0, 0}, /* AEC MAX EXPO (50HZ) */
+
+	{0x3c07, 0x07, 0, 0}, /* LIGHT METER1 THRESHOLD */
+	{0x3c09, 0x1c, 0, 0}, /* LIGHT METER2 THRESHOLD */
+	{0x3c0a, 0x9c, 0, 0}, /* SAMPLE NUMBER */
+	{0x3c0b, 0x40, 0, 0}, /* SAMPLE NUMBER */
+
+	{0x4001, 0x02, 0, 0}, /* BLC CTRL01 */
+	{0x4004, 0x02, 0, 0}, /* BLC CTRL04 */
+
+	{0x4407, 0x04, 0, 0}, /* JPEG CTRL07 */
+
+	{0x460b, 0x37, 0, 0}, /* DEBUG MODE */
+	{0x460c, 0x20, 0, 0}, /* VFIFO CTRL0C */
+
+	{0x5001, 0x83, 0, 0}, /* ISP CONTROL 01 */
 };
 
 static const struct reg_value ov5640_setting_1080P_1920_1080[] = {
-	{0x3008, 0x42, 0, 0},
-	{0x3c07, 0x08, 0, 0},
-	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
-	{0x3814, 0x11, 0, 0},
-	{0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
-	{0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
-	{0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
-	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
-	{0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
-	{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
-	{0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0},
-	{0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
-	{0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0},
-	{0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0},
-	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
-	{0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0},
-	{0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0},
-	{0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0},
-	{0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0},
-	{0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0},
-	{0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0},
-	{0x3a0e, 0x03, 0, 0}, {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x04, 0, 0},
-	{0x3a15, 0x60, 0, 0}, {0x4407, 0x04, 0, 0},
-	{0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0}, {0x3824, 0x04, 0, 0},
-	{0x4005, 0x1a, 0, 0}, {0x3008, 0x02, 0, 0},
+	{0x3612, 0x2b, 0, 0}, /* ??? between VCM and timing control */
+	{0x3618, 0x04, 0, 0}, /* ??? between VCM and timing control */
+	{0x3708, 0x64, 0, 0}, /* ??? between VCM and timing control */
+	{0x3709, 0x12, 0, 0}, /* ??? between VCM and timing control */
+	{0x370c, 0x00, 0, 0}, /* ??? between VCM and timing control */
+
+	{0x3800, 0x01, 0, 0}, /* TIMING HS */
+	{0x3801, 0x50, 0, 0}, /* TIMING HS */
+	{0x3802, 0x01, 0, 0}, /* TIMING VS */
+	{0x3803, 0xb2, 0, 0}, /* TIMING VS */
+	{0x3804, 0x08, 0, 0}, /* TIMING HW */
+	{0x3805, 0xef, 0, 0}, /* TIMING HW */
+	{0x3806, 0x05, 0, 0}, /* TIMING VH */
+	{0x3807, 0xf1, 0, 0}, /* TIMING VH */
+	{0x3810, 0x00, 0, 0}, /* TIMING HOFFSET */
+	{0x3811, 0x10, 0, 0}, /* TIMING HOFFSET */
+	{0x3812, 0x00, 0, 0}, /* TIMING VOFFSET */
+	{0x3813, 0x04, 0, 0}, /* TIMING VOFFSET */
+	{0x3814, 0x11, 0, 0}, /* TIMING X INC */
+	{0x3815, 0x11, 0, 0}, /* TIMING Y INC */
+	{0x3824, 0x04, 0, 0}, /* DVP PCLK divider for VFIFO CTRL0C */
+
+	{0x3a02, 0x04, 0, 0}, /* AEC MAX EXPO (60HZ) */
+	{0x3a03, 0x60, 0, 0}, /* AEC MAX EXPO (60HZ) */
+	{0x3a08, 0x01, 0, 0}, /* AEC B50 STEP */
+	{0x3a09, 0x50, 0, 0}, /* AEC B50 STEP */
+	{0x3a0a, 0x01, 0, 0}, /* AEC B60 STEP */ 
+	{0x3a0b, 0x18, 0, 0}, /* AEC B60 STEP */ 
+	{0x3a0d, 0x04, 0, 0}, /* AEC CTRL0D */
+	{0x3a0e, 0x03, 0, 0}, /* AEC CTRL0E */
+	{0x3a14, 0x04, 0, 0}, /* AEC MAX EXPO (50HZ) */
+	{0x3a15, 0x60, 0, 0}, /* AEC MAX EXPO (50HZ) */
+
+	{0x3c07, 0x07, 0, 0}, /* LIGHT METER1 THRESHOLD */
+	{0x3c08, 0x00, 0, 0}, /* LIGHT METER2 THRESHOLD */
+	{0x3c09, 0x1c, 0, 0}, /* LIGHT METER2 THRESHOLD */
+	{0x3c0a, 0x9c, 0, 0}, /* SAMPLE NUMBER */
+	{0x3c0b, 0x40, 0, 0}, /* SAMPLE NUMBER */
+
+	{0x4001, 0x02, 0, 0}, /* BLC CTRL01 */
+	{0x4004, 0x06, 0, 0}, /* BLC CTRL04 */
+	{0x4005, 0x1a, 0, 0}, /* BLC CTRL05 */
+
+	{0x4407, 0x04, 0, 0}, /* JPEG CTRL07 */
+
+	{0x460b, 0x37, 0, 0}, /* DEBUG MODE */
+	{0x460c, 0x20, 0, 0}, /* VFIFO CTRL0C */
+
+	{0x5001, 0x83, 0, 0}, /* ISP CONTROL 01 */
 };
 
 static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
@@ -787,7 +839,7 @@ static int ov5640_mod_reg(struct ov5640_dev *sensor, u16 reg,
  * This is supposed to be ranging from 1 to 16, but the value is
  * always set to either 1 or 2 in the vendor kernels.
  */
-#define OV5640_SYSDIV_MIN	1
+#define OV5640_SYSDIV_MIN	2
 #define OV5640_SYSDIV_MAX	16
 
 /*
@@ -848,7 +900,7 @@ static unsigned long ov5640_calc_sys_clk(struct ov5640_dev *sensor,
 					 u8 *sysdiv)
 {
 	unsigned long best = ~0;
-	u8 best_sysdiv = 1, best_mult = 1;
+	u8 best_sysdiv = OV5640_SYSDIV_MIN, best_mult = OV5640_PLL_MULT_MIN;
 	u8 _sysdiv, _pll_mult;
 
 	for (_sysdiv = OV5640_SYSDIV_MIN;
@@ -874,7 +926,7 @@ static unsigned long ov5640_calc_sys_clk(struct ov5640_dev *sensor,
 			 * We have reached the maximum allowed PLL1 output,
 			 * increase sysdiv.
 			 */
-			if (!rate)
+			if (!_rate)
 				break;
 
 			/*
@@ -993,14 +1045,19 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor,
 {
 	unsigned long _rate = rate * OV5640_PLL_ROOT_DIV * OV5640_BIT_DIV *
 				OV5640_PCLK_ROOT_DIV;
+	unsigned long pclk;
 
 	_rate = ov5640_calc_sys_clk(sensor, _rate, pll_prediv, pll_mult,
 				    sysdiv);
 	*pll_rdiv = OV5640_PLL_ROOT_DIV;
 	*bit_div = OV5640_BIT_DIV;
 	*pclk_div = OV5640_PCLK_ROOT_DIV;
+	pclk = _rate / *pll_rdiv / *bit_div / *pclk_div;
 
-	return _rate / *pll_rdiv / *bit_div / *pclk_div;
+	dev_warn(&sensor->i2c_client->dev, "ov5640_calc_pclk sysclk: %lu pclk: %lu\n",
+				 _rate, pclk);
+
+	return pclk;
 }
 
 static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor, unsigned long rate)
@@ -1014,6 +1071,10 @@ static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor, unsigned long rate)
 	if (bit_div == 2)
 		bit_div = 8;
 
+	dev_warn(&sensor->i2c_client->dev, "ov5640_set_dvp_pclk bit_div: %u" \
+ 		", sysdiv: %u, mult: %u, prediv: %u, pll_rdiv: %u, pclk_div: %u\n",
+		bit_div, sysdiv, mult, prediv, pll_rdiv, pclk_div);
+
 	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL0,
 			     0x0f, bit_div);
 	if (ret)
@@ -1078,6 +1139,10 @@ static int ov5640_set_timings(struct ov5640_dev *sensor,
 			return ret;
 	}
 
+	dev_warn(&sensor->i2c_client->dev, 
+		"ov5640_set_timings setting hact: %u, vact: %u, htot: %u, vtot: %u", 
+		mode->hact, mode->vact, mode->htot, mode->vtot);
+
 	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, mode->hact);
 	if (ret < 0)
 		return ret;

The 720p registers are just sorted, but the 1080p ones had a lot of repeated addresses. I've assumed they aren't stateful and only kept the last instance and then sorted them. I also removed a few values that were mentioned in mainline commits. It think it's functionally the same as before but has helped debugging and might help someone else. I don't think they're the current issue anyway.

With the patch (minus the change to OV5640_SYSDIV_MIN) and 720p you'll see in dmesg:

ov5640 2-003c: ov5640_set_dvp_pclk bit_div: 8, sysdiv: 2, mult: 85, prediv: 3, pll_rdiv: 2, pclk_div: 1

But with 1080p sysdiv notably jumps to 1:

ov5640 2-003c: ov5640_set_dvp_pclk bit_div: 8, sysdiv: 1, mult: 84, prediv: 3, pll_rdiv: 2, pclk_div: 1

If you add the change to OV5640_SYSDIV_MIN and test using the OV5640's pattern generator:

v4l2-ctl --set-ctrl test_pattern=2
media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080]'
ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output.mjpg

You should get a perfect 1080p test pattern. I can't get anything but a black screen and lines if I turn the pattern off and try to capture a real image,  but if you compare the pattern to the pattern without my change I think you'll agree there's a clocking problem. The documentation on the clock structure in the code is the only useful source I've found unfortunately.

There is also a kernel warning in dmesg with 1080p regarding "dma_alloc_coherent of size 4149248 failed" but I think(?) it's a red herring - it's asking for 32 buffers and it looks like only the 32nd one fails. The capture continues without it.

Hopefully this helps get us closer to working 1080p!

@rreignier Did you get your board working? I guess we did end up hijacking your thread anyway sorry.

Link to post
Share on other sites

47 minutes ago, gsumner said:

@rreignier Did you get your board working? I guess we did end up hijacking your thread anyway sorry.

Hi @gsumner

No, I did not have time to try it out on my board yet :(

But I am very glad that you have managed to get this camera working on mainline and shared your progress. I am following with great attention!

Thanks!

 

Did you have a look at the blog post about PinePhone camera which happen to be an OV5640 too?

https://blog.brixit.nl/camera-on-the-pinephone/

Link to post
Share on other sites

5 hours ago, gsumner said:

There is also a kernel warning in dmesg with 1080p regarding "dma_alloc_coherent of size 4149248 failed" but I think(?) it's a red herring - it's asking for 32 buffers and it looks like only the 32nd one fails.

 

Usually V4l2 applications map 4 buffers (you can map 8 buffers if you have enough memory to get some optimization).

For the size of 1920x1080 and YUV420 format you get a buffer size of 1920*1080*2 = 4147200 mapped in memory and the sensor delivers an image of size 1920*1080*3/2 = 3110400.

Looks like your VLC is addding some extra bytes for any reason. 32 buffers is around 128 MB, try to increase CMA, or change and recompile it for 8 buffers.

 

Regarding the green tinge and pinkish or grainy image i think is related to light condition, if you take shoots at day light (outside) you get better image just like the PinePhone.

 

Can you show an Image 1920x1090 with daylight (outside) and one with indirect light (inside)?

The current driver is able to sustain a 1920x1080 ~10 FPS on a nanopi air, 1.2Ghz. I think on A64 you can get ~15FPS. 

 

Link to post
Share on other sites

Hi Lex, I think I've been talking about two unrelated issues, the greenish image showed up every n images but I only tested indoors. I think I now know that was only when the camera was in jpeg mode. Unfortunately my driver is in a sorry state at the moment for taking new images. This drive link has the old images: https://drive.google.com/drive/folders/1kjn-D02tEXLiDAMPOVFVVTlrnqgGFkro?usp=sharing

The other two images represent the issue that mostly and I are talking about, which is when the camera is in yuv mode. When using yuv in 1080p (I'm testing using "v4l2-ctl --set-ctrl test_pattern=2") you get the heavily distorted green/pink images. I currently believe they are because of an issue in the clock calculations I mentioned above. I can get a reasonable looking 1080p test pattern by forcing sysdiv to more than 1 but can't get real images. I'm guessing that means that the SCLK is a reasonable value (the ISP must generate the pattern??) but the PCLK is clearly not good. I wonder if everyone else using the ov5640 driver uses csi instead of parallel...

I think if I start looking into the cma alloc issue at the same time my head might explode! I'll have to look after the yuv-1080p-clock issue. Regarding VLC, I'm wondering if the issue is because vlc is opening the camera in yuv420 mode, but in the sun6i_csi driver I can see in "get_csi_input_format" that yuv420 isn't supported? Would that explain the frame size difference?

@rreignier Thanks, that looks helpful. So they're using an Allwinner, ov5640 and parallel camera too. I'll have to look to see if they've already solved this.

Link to post
Share on other sites

1 hour ago, gsumner said:

Would that explain the frame size difference?

i think so.

 

I tested your pattern sample on A64 and both pattern looked fine (exactly the same)

ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output_yuyv422.mjpg

ffmpeg -input_format yuv420p -s 1920x1080 -i /dev/video0 /tmp/output_yuyv420.mjpg

which i think should fail for the yuyv422 on fmt:YUYV8_2X8/1920x1080

 

Image quality is similar to what you got with your test_0.

I would advise to move on to latest ov5640 driver or if you find the ov5640 driver (PinePhone) with additional fix , let us know the diff.

Maybe @rreignier can have different results.

Hope this information will be helpful. Good luck.

 

 

Link to post
Share on other sites

@mostly and anyone else interested, I have gotten normal looking yuv422 video at 1920x1080 working (I think). I only have part of the final fix though, because I don't understand yet why the rate is out by two. I thought it might have been dynamic and only broken when sysdiv went to 1x, but I can capture fine looking video at 720p and 480p with this change too. The minimal change is below:

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 18dd2d717..342e80d21 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -992,7 +992,7 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor,
 				      u8 *pll_rdiv, u8 *bit_div, u8 *pclk_div)
 {
 	unsigned long _rate = rate * OV5640_PLL_ROOT_DIV * OV5640_BIT_DIV *
-				OV5640_PCLK_ROOT_DIV;
+				OV5640_PCLK_ROOT_DIV / 2;
 
 	_rate = ov5640_calc_sys_clk(sensor, _rate, pll_prediv, pll_mult,
 				    sysdiv);

Could someone please confirm that they had the pink/green image problem and that this fixes it at 1080p for them? Tested with:

media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080]'
ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output.mjpg

I've been sticking with yuv422 so far because the sun6i_csi driver mentions not supporting yuv420 and the ov5640 seems to have some 16bit/pixel parts hardcoded. I'd like to understand the clock structure better, but as I said previously I'm struggling to find details.

Link to post
Share on other sites

I started having a look at 2592x1944 video, for anyone else trying the same thing and failing with "Unable to setup formats: Invalid argument (22)" from media-ctl, the issue appears to be the framerate/interval. 2592x1944 only works at 15fps but the command (@1/15 is the frame interval, inverse of framerate):

media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/2592x1944@1/15]'

seems to have its order of operation backwards, it tries 2592x1944 at the current framerate, and sets the new framerate after, which fails. To work around it you can do the following:

media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080@1/15]'
media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/2592x1944]'

1080p works at 30fps, which is why you can switch to 15fps in the same command, then you can move the resolution to 2592x1944 once already at 15fps.

I can't get it to capture any video at 2592x1944 yet, but hopefully this helps anyone else stuck where I was.

Link to post
Share on other sites

Hi, I think I have 2592x1944 working (with the above workaround) yuv422 and jpeg. If you just want the change (needs to be combined with the device tree change from earlier in the thread if you're starting from scratch) it's below:

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 18dd2d717..6c8d00c30 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -535,7 +535,7 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
 	{0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
 	{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
 	{0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0},
-	{0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+	{0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x20, 0, 0},
 	{0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 70},
 };
 
@@ -992,7 +992,7 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor,
 				      u8 *pll_rdiv, u8 *bit_div, u8 *pclk_div)
 {
 	unsigned long _rate = rate * OV5640_PLL_ROOT_DIV * OV5640_BIT_DIV *
-				OV5640_PCLK_ROOT_DIV;
+				OV5640_PCLK_ROOT_DIV / OV5640_BIT_DIV;
 
 	_rate = ov5640_calc_sys_clk(sensor, _rate, pll_prediv, pll_mult,
 				    sysdiv);

The explanation is that I think the parallel bus calculation (mipi csi on other SoCs unaffected) is missing an extra bit_div which is in the diagram but not the calculation. The 2592x1944 register 0x460c setting 0x22 overrides that extra OV5640_BIT_DIV I've added with the value in register 0x3824, so 0x20 brings it back in line with 720p and 1080p using "auto" calculation. Lower resolutions use the override too but they worked when I tested them so I haven't messed with it.

I'm concerned that lex has 1080p working on an a64 without the change though, I thought the fix would be independent of SoC and affect all parallel buses with the ov5640. Lex, could you confirm it was a64 + ov5640 + no driver changes that worked for you?

(Assuming the a64 works with my change) We could have something Armbian could apply soon, could you post your Allwinner board+SoC, camera and whether this change works for you?

I think we should also make a change to try the framerate before/after the resolution so we don't need the workaround in the previous post.

@Igor I think I nearly have a change to get 1080p and 1944p working with ov5640 and parallel bus (as opposed to mipi csi). I haven't done this with Armbian before, how much testing do I need? Do I need to find all boards with an ov5640 and parallel bus? I need to test some more formats/resolutions on my board before I'm confident at least anyway. There's also a separate change for the device tree from earlier in the thread that I think could also be submitted with some more testing. I'd like to help contribute this to Armbian if I can get these tested on some other boards and in the right format for you.

Link to post
Share on other sites

32 minutes ago, gsumner said:

Lex, could you confirm it was a64 + ov5640 + no driver changes that worked for you?

 

Untoched kernel driver works for A64 and H2/H3 on the following win size:

* 320x240 (30 FPS)

* 640x480 (30 FPS)

* 1280x720 ( < 30 FPS)

* 1920x1080 ( < 30 FPS)

 

Quality is acceptable (outdoor).

 

Does not work:

60 FPS, 2592x1944

 

I think you are right about 2592x1944 ,  i will test your changes and report back.

 

Link to post
Share on other sites

@mostly, it's apparently been a productive Sunday, I had a look at the Auto Focus commit you linked and did a quick hack job to bring it into the current driver. I've shown it working here: https://drive.google.com/drive/folders/1kjn-D02tEXLiDAMPOVFVVTlrnqgGFkro?usp=sharing (I can only play the video with ubuntu default video app??).

 

The problem is that (I believe) the large array they're reading in to address 0x8000 is firmware for the microcontroller to run, so there was no AF feature to enable before the mcu code was loaded. With this hack it seems to be in a constant-refocus mode by default, but there are untested v4l2 controls for other modes.

 

I'll attach the patch but I'd like to stress that it was chopped into the driver with no care, I've already seen instability playing with it. I think it could be close enough to working to be a starting point for a real change though, I might start looking at improving it. I don't like that it's a black box though, debugging it will be a pain. I also think it might be loading the firmware everytime you record, which seems over the top.

ov5640_AF.patch

Link to post
Share on other sites

On 7/18/2020 at 12:04 AM, gsumner said:

I have gotten normal looking yuv422 video at 1920x1080 working (I think). I only have part of the final fix though, because I don't understand yet why the rate is out by two

Could someone please confirm that they had the pink/green image problem and that this fixes it at 1080p for them? Tested with:

media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080]'
ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output.mjpg

Tried it and got almost adequate FHD picture with OV5640_PCLK_ROOT_DIV / 2 . If using ffmpeg, the only problem that i see is that there is more green than expected.

In the meantime, i brought up h264 hw encoding from uboborov sunxi-cedar-mainline + uvc_h264_lb_allwinner (turned on writing to file and FPS calculation in settings inside main.c).  It works well with 1280x720@15fps (real 15fps in h264 file), but with 1920x1080 OV5640_PCLK_ROOT_DIV / 2 it gives some artifacts with colors and adds shadows to dark objects on white background. Not sure yet if this problem is from hw encoder side or ov5640.

On 7/19/2020 at 12:55 PM, gsumner said:

1080p works at 30fps

Were you able to get real 30fps? uvc_h264_lb_allwinner shows actual FPS, and 1280x720 framerate is always 15 and 1920x1080 framerate is always 7 regardless of what set in media-ctl format @ 1/

On 7/19/2020 at 3:36 PM, gsumner said:

I'm concerned that lex has 1080p working on an a64 without the change though

So a64 does not need that patch, and also does not need "connecting the clock parent in the device tree" in dts. Since we see these strange clocks behavior, can this custom clock source be not suitable for some modes?...

 

 

Link to post
Share on other sites

Lex, when you test a64 could you try with/without the OV5640_PCLK_ROOT_DIV change I proposed at a few resolutions including 2592x1944? My H5 was working at 720p without the change so I'm wondering if the higher resolutions require a more 'correct' clock but will still function at lower res? If your boards only work at 2592x1944 with the change or if they work both with/without the change I think there could still be hope that it is correct, otherwise I think we'll have to look more on the SoC side at what differs between SoCs. btw, I'm very confused that H3 works for you without changes, they share a lot of DTB and driver code for this - it might narrow the search if H3 works and H5 doesn't.

I was thinking the clock change was the perfect answer given it wasn't in the calculation but was in the diagram and that it was the difference between green/purple image and perfect-ish 1080p/1944p for me. But if it at makes other SoCs perform worse I think I'll accept that it's wrong, the camera shouldn't have SoC specific code.

Link to post
Share on other sites

Looking at the BSP code and the driver, I think my change to the clock can be ignored now - it's coupled to the framerate, so even if it had been correct I think it would halve the framerate. I think I will start looking at clocking on the SoC, after that I could only guess at electrical design issues(?). @mostly has an H3 and I have an H5 which have issues, but Lex has an H3 and A64 which work, which makes me worry it's an electrical design issue...

I think the auto focus patch should still work if the couple of lines changing clocks are removed though.

Lex, thanks, I think you don't need to worry about testing the change. I'll make a thread for ov5640 improvements for things like the auto focus after looking at this issue (SoC/board specific corrupted green/pink images) for a bit more. I haven't been concerned with framerate, just looking at what ffmpeg output. The rates I mentioned before were just the configured rate in media-ctl.

@mostly No, I've never seen 30fps at 1080p sorry, I was referring to the configured rate. I only get 2 or 3 fps at 1080p but I've realised that's because the clock change is wrong. I agree that it might be time to look at clocking on the SoC side. I've ordered an Orange Pi One with an H3 to see if I can get different behaviour from an H3 on a different board.

Link to post
Share on other sites

I have updated fswebcam for the mainline kernel. You get the FPS printed on the picture, all seems to be fine.

Tested with A64, H3 and H2 with  your:

- {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0}, 
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x20, 0, 0},

 

Samples:

H3

https://github.com/avafinger/fswebcam/tree/mainline-kernel/nanopi-air

H2

https://github.com/avafinger/fswebcam/tree/mainline-kernel/bpi-m2z

A64

https://github.com/avafinger/fswebcam/tree/mainline-kernel/nanopi-a64

 

You should focus on the AF improvements. Looking forward to your progress with AF.

Unfortunately i was removing old files and directories to free some space and did a sudo rm -rf  *  in the wrong root directory and now i can't recover important stuff, s* happens, no backup. The inodes are marked free now. 

Link to post
Share on other sites

17 hours ago, gsumner said:

makes me worry it's an electrical design issue

Even if there is something bad in electrical design, it is somehow solvable from software side - year ago i had both 1080p and 2592x1944 working without any artifacts on kernel 3.4.113 on the same board OrangePiPC+.

 

17 hours ago, gsumner said:

I only get 2 or 3 fps at 1080p but I've realised that's because the clock change is wrong

I believe 2-3fps is due to some bottleneck in ffmpeg (you'll likely see 100% core utilization by ffmpeg). I'm getting poor fps in ffmpeg at 720p and 1080p, but optimized uvc_h264_lb_allwinner with hw encoding gives pretty stable 15fps at 720p (no artifacts at all) and 7fps at 1080p (with /2 patch, some color artifacts are visible).

Link to post
Share on other sites

On 7/27/2020 at 2:06 AM, mostly said:

Even if there is something bad in electrical design, it is somehow solvable from software side - year ago i had both 1080p and 2592x1944 working without any artifacts on kernel 3.4.113 on the same board OrangePiPC+.

Thanks, I didn't know that. There must be a software problem/solution then.

 

I started having a look at clocks, trying to see if setting mclk parent/rate in the dtb was having the effect I thought, there's a feature to print the clock structure:

sudo cat /sys/kernel/debug/clk/clk_summary
                                 enable  prepare  protect                                duty
   clock                          count    count    count        rate   accuracy phase  cycle
---------------------------------------------------------------------------------------------
 iosc                                 0        0        0    16000000  300000000     0  50000
 ext_osc32k                           1        1        0       32768      50000     0  50000
    osc32k                            1        1        0       32768      50000     0  50000
       ir                             0        0        0       32768      50000     0  50000
       osc32k-out                     0        0        0       32768      50000     0  50000
 osc24M                              15       15        0    24000000      50000     0  50000
    ar100                             1        1        0    24000000      50000     0  50000
       ahb0                           1        1        0    24000000      50000     0  50000
          apb0                        1        1        0    24000000      50000     0  50000
             apb0-twd                 0        0        0    24000000      50000     0  50000
             apb0-i2c                 0        0        0    24000000      50000     0  50000
             apb0-uart                0        0        0    24000000      50000     0  50000
             apb0-timer               0        0        0    24000000      50000     0  50000
             apb0-ir                  0        0        0    24000000      50000     0  50000
             apb0-pio                 1        1        0    24000000      50000     0  50000
    csi-mclk                          0        0        0    24000000      50000     0  50000
    hdmi-ddc                          2        2        0    24000000      50000     0  50000
    avs                               0        0        0    24000000      50000     0  50000
    csi-misc                          0        0        0    24000000      50000     0  50000
    usb-ohci3                         1        1        0    24000000      50000     0  50000
    usb-ohci2                         1        1        0    24000000      50000     0  50000
    usb-ohci1                         0        0        0    24000000      50000     0  50000
    usb-ohci0                         0        0        0    24000000      50000     0  50000
    usb-phy3                          1        1        0    24000000      50000     0  50000
    usb-phy2                          1        1        0    24000000      50000     0  50000
    usb-phy1                          0        0        0    24000000      50000     0  50000
    usb-phy0                          1        1        0    24000000      50000     0  50000
    spi1                              0        0        0    24000000      50000     0  50000
    spi0                              0        0        0    24000000      50000     0  50000
    ce                                0        0        0    24000000      50000     0  50000
    ts                                0        0        0    24000000      50000     0  50000
    nand                              0        0        0    24000000      50000     0  50000
    ths                               1        1        0    24000000      50000     0  50000
    apb2                              2        2        0    24000000      50000     0  50000
       bus-scr1                       0        0        0    24000000      50000     0  50000
       bus-scr0                       0        0        0    24000000      50000     0  50000
       bus-uart3                      0        0        0    24000000      50000     0  50000
       bus-uart2                      0        0        0    24000000      50000     0  50000
       bus-uart1                      0        0        0    24000000      50000     0  50000
       bus-uart0                      1        1        0    24000000      50000     0  50000
       bus-i2c2                       1        1        0    24000000      50000     0  50000
       bus-i2c1                       0        0        0    24000000      50000     0  50000
       bus-i2c0                       0        0        0    24000000      50000     0  50000
    pll-de                            1        1        0   432000000      50000     0  50000
       de                             2        2        0   432000000      50000     0  50000
          wb-div                      0        0        0   432000000      50000     0  50000
             wb                       0        0        0   432000000      50000     0  50000
          mixer1-div                  0        0        0   432000000      50000     0  50000
             mixer1                   0        0        0   432000000      50000     0  50000
          mixer0-div                  1        1        0   432000000      50000     0  50000
             mixer0                   1        1        0   432000000      50000     0  50000
       tve                            0        0        0   432000000      50000     0  50000
    pll-periph1                       0        0        0   600000000      50000     0  50000
    pll-gpu                           1        1        0   384000000      50000     0  50000
       gpu                            1        1        0   384000000      50000     0  50000
    pll-periph0                       6        6        0   600000000      50000     0  50000
       mmc1                           1        1        0    50000000      50000     0  50000
       mmc2                           1        1        0    50000000      50000     0  50000
       mmc0                           1        1        0    50000000      50000     0  50000
       csi-sclk                       0        0        0   300000000      50000     0  50000
       deinterlace                    0        0        0   600000000      50000     0  50000
       ahb2                           4        4        0   300000000      50000     0  50000
          bus-ohci3                   2        2        0   300000000      50000     0  50000
          bus-ohci2                   2        2        0   300000000      50000     0  50000
          bus-ohci1                   0        0        0   300000000      50000     0  50000
          bus-ehci3                   2        2        0   300000000      50000     0  50000
          bus-ehci2                   2        2        0   300000000      50000     0  50000
          bus-ehci1                   0        0        0   300000000      50000     0  50000
          bus-emac                    0        0        0   300000000      50000     0  50000
       ahb1                          11       12        0   200000000      50000     0  50000
          bus-dbg                     0        0        0   200000000      50000     0  50000
          bus-ephy                    0        0        0   200000000      50000     0  50000
          bus-spinlock                0        0        0   200000000      50000     0  50000
          bus-msgbox                  1        1        0   200000000      50000     0  50000
          bus-gpu                     1        1        0   200000000      50000     0  50000
          bus-de                      2        2        0   200000000      50000     0  50000
             bus-wb                   0        0        0   200000000      50000     0  50000
             bus-mixer1               0        0        0   200000000      50000     0  50000
             bus-mixer0               1        1        0   200000000      50000     0  50000
          bus-hdmi                    2        2        0   200000000      50000     0  50000
          bus-tve                     0        0        0   200000000      50000     0  50000
          bus-csi                     0        1        0   200000000      50000     0  50000
          bus-deinterlace             0        0        0   200000000      50000     0  50000
          bus-tcon1                   0        0        0   200000000      50000     0  50000
          bus-tcon0                   1        1        0   200000000      50000     0  50000
          bus-ve                      0        0        0   200000000      50000     0  50000
          bus-ohci0                   0        0        0   200000000      50000     0  50000
          bus-ehci0                   0        0        0   200000000      50000     0  50000
          bus-otg                     1        1        0   200000000      50000     0  50000
          bus-spi1                    0        0        0   200000000      50000     0  50000
          bus-spi0                    0        0        0   200000000      50000     0  50000
          bus-hstimer                 0        0        0   200000000      50000     0  50000
          bus-ts                      0        0        0   200000000      50000     0  50000
          bus-dram                    0        0        0   200000000      50000     0  50000
          bus-nand                    0        0        0   200000000      50000     0  50000
          bus-mmc2                    1        1        0   200000000      50000     0  50000
          bus-mmc1                    1        1        0   200000000      50000     0  50000
          bus-mmc0                    1        1        0   200000000      50000     0  50000
          bus-dma                     1        1        0   200000000      50000     0  50000
          bus-ce                      0        0        0   200000000      50000     0  50000
          apb1                        2        2        0   100000000      50000     0  50000
             bus-i2s2                 0        0        0   100000000      50000     0  50000
             bus-i2s1                 0        0        0   100000000      50000     0  50000
             bus-i2s0                 0        0        0   100000000      50000     0  50000
             bus-ths                  1        1        0   100000000      50000     0  50000
             bus-pio                  1        1        0   100000000      50000     0  50000
             bus-spdif                0        0        0   100000000      50000     0  50000
             bus-codec                0        0        0   100000000      50000     0  50000
       pll-periph0-2x                 1        1        0  1200000000      50000     0  50000
          mbus                        1        1        0   300000000      50000     0  50000
    pll-ddr                           1        1        0  1248000000      50000     0  50000
       dram                           1        1        0  1248000000      50000     0  50000
          dram-ts                     0        0        0  1248000000      50000     0  50000
          dram-deinterlace            0        0        0  1248000000      50000     0  50000
          dram-csi                    0        0        0  1248000000      50000     0  50000
          dram-ve                     0        0        0  1248000000      50000     0  50000
    pll-ve                            0        0        0   297000000      50000     0  50000
       ve                             0        0        0   297000000      50000     0  50000
    pll-video                         2        2        0   294000000      50000     0  50000
       hdmi-phy-clk                   1        1        0    73500000      50000     0  50000
       hdmi                           1        1        0   294000000      50000     0  50000
       tcon                           0        0        0   294000000      50000     0  50000
    pll-audio-base                    0        0        0    98285714      50000     0  50000
       pll-audio-8x                   0        0        0   196571428      50000     0  50000
          i2s2                        0        0        0   196571428      50000     0  50000
          i2s1                        0        0        0   196571428      50000     0  50000
          i2s0                        0        0        0   196571428      50000     0  50000
       pll-audio-4x                   0        0        0    98285714      50000     0  50000
       pll-audio-2x                   0        0        0    49142857      50000     0  50000
       pll-audio                      0        0        0    98285714      50000     0  50000
          ac-dig                      0        0        0    98285714      50000     0  50000
          spdif                       0        0        0    98285714      50000     0  50000
    pll-cpux                          1        1        0   480000000      50000     0  50000
       cpux                           1        1        0   480000000      50000     0  50000
          axi                         0        0        0   120000000      50000     0  50000

All of our csi clocks have no statistics even after a 1080p (green/pink) capture. I know that changing the mclk rate in the dtb reached the driver because it complained once it passed the limits, but changing the clock parent doesn't reflect in this output. I don't know what to make of this though, I assume the Linux part it working well, so are we calling/reporting clock changes?

 

On 7/20/2020 at 1:59 PM, mostly said:

Since we see these strange clocks behavior, can this custom clock source be not suitable for some modes?...

Looking at the BSP code (searching "CSI_MASTER_CLK_24M_SRC" in bsp code), I think as long as the mclk rate is 6/12/24MHz the 24MHz clock is the same one they would configure.

I think a clock config problem on the SoC side could be a tidy answer, it would explain why it already works for some with the same SoC. I'll keep looking in this direction for now.

Link to post
Share on other sites

kernal configs:
 

CONFIG_I2C_GPIO=m

CONFIG_V4L_PLATFORM_DRIVERS=y

CONFIG_VIDEO_SUN6I_CSI=m

CONFIG_V4L_MEM2MEM_DRIVERS=y

CONFIG_VIDEO_MEM2MEM_DEINTERLACE=m

CONFIG_VIDEO_OV5640=m

 

Hi all,

 I did changes for kernal configs

 

After that what should I need to change in  sun8i-h3-nanopi-duo2.dts or/and add overlay to enable the camera

As I am using Nano pi duo 2 with H3 chipset running under armhf(32bit) Armbian Focal (mainline based kernel 5.10.y)  attached with OV5640 camera 

 

Thanks in advance

Link to post
Share on other sites

Hello!

 

Guys, I need your help!

Recently I've been trying for 2 weeks to make ov5640 work on my OrangePi-PC (Armbian 21.05.0-trunk Buster with Linux 5.10.19-sunxi) and yet did not suceed.

 

I'm currently using DeviceTree overlay:

/dts-v1/;
/plugin/;

/ {
	compatible = "allwinner,sun8i-h3";

	fragment@0 {
		target-path = "/";
		__overlay__ {
			/* DOVDD power supply. */
			reg_cam_dovdd: cam-dovdd-1v8 {
				compatible = "regulator-fixed";
				regulator-name = "cam-dovdd-1v8";
				regulator-min-microvolt = <1800000>;
				regulator-max-microvolt = <1800000>;
				enable-active-high;

				gpio = <&pio 6 13 0>; /* DOVDD: AFCC_EN: PG13, GPIO_ACTIVE_HIGH: 0 */

				regulator-boot-on;
				regulator-always-on;
			};

			/* AVDD power supply. */
			reg_cam_avdd: cam-avdd-2v8 {
				compatible = "regulator-fixed";
				regulator-name = "cam-avdd-2v8";
				regulator-min-microvolt = <2800000>;
				regulator-max-microvolt = <2800000>;
				enable-active-high;

				gpio = <&pio 0 17 0>; /* AVDD: CSI_PWR_EN: PA17, GPIO_ACTIVE_HIGH: 0 */

				regulator-boot-on;
				regulator-always-on;
			};

			/* DVDD power supply. */
			reg_cam_dvdd: cam-dvdd-1v5 {
				compatible = "regulator-fixed";
				regulator-name = "cam-dvdd-1v5";
				regulator-min-microvolt = <1500000>;
				regulator-max-microvolt = <1500000>;
				enable-active-high;

				gpio = <&pio 6 11 0>; /* CSI_EN: PG11, GPIO_ACTIVE_HIGH: 0 */

				/* startup-delay-us = <5000>; */ /* 5ms delay. */

				regulator-boot-on;
				regulator-always-on;
			};
		};
	};

	fragment@1 {
		target = <&pio>;
		__overlay__ {
			csi_mclk_pin: csi-mclk-pin {
				pins = "PE1";
				function = "csi";
			};
		};
	};

	fragment@2 {
		target = <&csi>;
		__overlay__ {
			status = "okay";

			port {
				#address-cells = <1>;
				#size-cells = <0>;

				/* Parallel bus endpoint */
				csi_ep: endpoint {
					remote-endpoint = <&ov5640_ep>;
					bus-width = <8>;

					hsync-active = <1>; /* Active high */
					vsync-active = <0>; /* Active low */
					pclk-sample = <1>;  /* Rising */
				};
			};
		};
	};

	fragment@3 {
		target = <&i2c2_pins>;
		__overlay__ {
			bias-pull-up;
		};
	};

	fragment@4 {
		target = <&i2c2>;
		__overlay__ {
			status = "okay";

			ov5640: camera@3c {
				compatible = "ovti,ov5640";
				reg = <0x3c>;
				pinctrl-names = "default";
				pinctrl-0 = <&csi_mclk_pin>;
				clocks = <&ccu 107>; /* CLK_CSI_MCLK: 107 */
				clock-names = "xclk";

				AVDD-supply = <&reg_cam_avdd>;
				DOVDD-supply = <&reg_cam_dovdd>;
				DVDD-supply = <&reg_cam_dvdd>;

				reset-gpios = <&pio 4 14 1>; /* CSI-RESET#: PE14, GPIO_ACTIVE_LOW: 1 */
				powerdown-gpios = <&pio 4 15 0>; /* CSI-STBY-EN: PE15, GPIO_ACTIVE_HIGH: 0 */

				port {
					/* Parallel bus endpoint. */
					ov5640_ep: endpoint {
						remote-endpoint = <&csi_ep>;
						bus-width = <8>;
						data-shift = <2>; /* lines 9:2 are used */

						hsync-active = <1>; /* Active high */
						vsync-active = <0>; /* Active low */
						/* data-active = <1>; */ /* Active high */
						pclk-sample = <1>;  /* Rising */
					};
				};
			};
		};
	};
};

 

With this overlay /dev/video1 is created, ov5640 driver loads, no complaints from driver in dmesg whatsoever, camera is powered on and I/O from /dev/video1 somehow hits the camera (I slightly modified drivers/media/i2c/ov5640.c to see in dmesg, that something is being written/read on camera registers).

 

However, I am not able to use the camera (In my C++ application ioctl(VIDIOC_DQBUF, &buffer) function falls into endless loop, yet the same app works perfectly on my laptop with built-in camera).

Also command

$ v4l2-ctl -d /dev/video1 --list-ctrls

shows nothing.

 

If I enable v4l2 debug with

# echo 0x1f > /sys/class/video4linux/video1/dev_debug

and then try v4l2-ctl command, dmesg is spammed with VIDIOC_QUERYCTRL: error -25 ... .

And if I try using my app I have VIDIOC_DQBUF: error -512 in dmesg.

 

I've also tried applying @gsumner's solution with modified .dts file and 'assigned-*' properties within &ccu node and it gave me the same result.

 

So I'm out of ideas, what am I doing wrong and what to do next...

 

Please, help. And thank you in advance.

Link to post
Share on other sites

Hi @rdeyes

 

On an Orange Pi One (which might be close to your Orange Pi PC), I have managed to get the OV5640 thanks to the amazing work of the contributors of this thread.

 

Last time I have tried was on Armbian 20.08.17 with Linux 5.8.16.

 

The device tree overlay I use is this one (you may have to change some pins) :

/dts-v1/;
/plugin/;

/ {
        compatible = "allwinner,sun8i-h3";
};

&ccu {
        assigned-clocks = <&ccu 107>;
        assigned-clock-parents = <&osc24M>;
        assigned-clock-rates = <24000000>;
};

&pio {
        csi_mclk_pin: csi-mclk-pin {
                pins = "PE1";
                function = "csi";
        };
};

&i2c2_pins {
        bias-pull-up;
};

&i2c2 {
        status = "okay";

        ov5640: camera@3c {
                compatible = "ovti,ov5640";
                reg = <0x3c>;
                pinctrl-names = "default";
                pinctrl-0 = <&csi_mclk_pin>;
                clocks = <&ccu 107>;
                clock-names = "xclk";

                AVDD-supply = <&reg_vcc_af_csi>;
                DOVDD-supply = <&reg_vdd_1v5_csi>;
                DVDD-supply = <&reg_vcc_csi>;
                reset-gpios = <&pio 4 14 1>; /* CSI-RST-R: PE14 */
                powerdown-gpios = <&pio 4 15 0>; /* CSI-STBY-R: PE15 */

                port {
                        ov5640_ep: endpoint {
                                remote-endpoint = <&csi_ep>;
                                bus-width = <8>;
                                data-shift = <2>; /* lines 9:2 are used */
                                hsync-active = <1>; /* Active high */
                                vsync-active = <0>; /* Active low */
                                data-active = <1>;  /* Active high */
                                pclk-sample = <1>;  /* Rising */
                        };
                };
        };
};

&csi {
        status = "okay";

        port {
                #address-cells = <1>;
                #size-cells = <0>;

                csi_ep: endpoint {
                        remote-endpoint = <&ov5640_ep>;
                        bus-width = <8>;
                        hsync-active = <1>; /* Active high */
                        vsync-active = <0>; /* Active low */
                        data-active = <1>;  /* Active high */
                        pclk-sample = <1>;  /* Rising */
                };
        };
};

 

To enable it, I use this command:

sudo armbian-add-overlay csi-ov5640.dts

And then, to take a picture:

media-ctl --device /dev/media1 --set-v4l2 '"ov5640 1-003c":0[fmt:JPEG_1X8/1920x1080]'
fswebcam -d /dev/video0 -r 1920x1080 -D 0 --jpeg 100 /tmp/test.jpg

 

I hope this can help you and others.

Link to post
Share on other sites

Hi all,

 Thank you for the code. I have tried it in nano pi duo 2  with ov5640 in kernal version 5.10.21 armbian focal.
 can you give us the output for 

 media-ctl --device /dev/media1 --print-topology

media-ctl --device /dev/video0 --print-topology
 

I have added the overlay by sudo armbian-add-overlay csi-ov5640.dts and did reboot my device (nano pi duo 2)

 

my output for,

 

root@nanopiduo2:~# ls /dev/
autofs           loop3         rtc0    tty24  tty46   ttyS0     vcsa3
block            loop4         shm     tty25  tty47   ttyS1     vcsa4
btrfs-control    loop5         snd     tty26  tty48   ttyS2     vcsa5
bus              loop6         stderr  tty27  tty49   ttyS3     vcsa6
char             loop7         stdin   tty28  tty5    ttyS4     vcsu
console          loop-control  stdout  tty29  tty50   ttyS5     vcsu1
cpu_dma_latency  mapper        tty     tty3   tty51   ttyS6     vcsu2
cuse             media0        tty0    tty30  tty52   ttyS7     vcsu3
disk             mem           tty1    tty31  tty53   ubi_ctrl  vcsu4
ecryptfs         mmcblk0       tty10   tty32  tty54   uhid      vcsu5
fd               mmcblk0p1     tty11   tty33  tty55   uinput    vcsu6
full             mqueue        tty12   tty34  tty56   uleds     video0
fuse             net           tty13   tty35  tty57   urandom   watchdog
gpiochip0        null          tty14   tty36  tty58   v4l       watchdog0
gpiochip1        ppp           tty15   tty37  tty59   vcs       zero
hwrng            ptmx          tty16   tty38  tty6    vcs1      zram0
initctl          pts           tty17   tty39  tty60   vcs2      zram1
input            ram0          tty18   tty4   tty61   vcs3      zram2
kmem             ram1          tty19   tty40  tty62   vcs4
kmsg             ram2          tty2    tty41  tty63   vcs5
log              ram3          tty20   tty42  tty7    vcs6
loop0            random        tty21   tty43  tty8    vcsa
loop1            rfkill        tty22   tty44  tty9    vcsa1
loop2            rtc           tty23   tty45  ttyGS0  vcsa2



here I have media0 and video0 so, I tried

root@nanopiduo2:~# media-ctl --device /dev/media0 --set-v4l2 '"ov5640 1-003c":0[fmt:JPEG_1X8/1920x1080]'
Unable to setup formats: Invalid argument (22)

root@nanopiduo2:~# fswebcam -d /dev/video0 -r 1920x1080 -D 0 --jpeg 100 /tmp/test.jpg
-bash: fswebcam: command not found

can you also give me the I2C detect,

i2cdetect -y 0
i2cdetect -y 1
i2cdetect -y 2


I found I2C 2 which is connect to camera is not working

root@nanopiduo2:~# ls -l /sys/class/i2c-adapter/
total 0

I found my I2C2 is disable how can I enable?

root@nanopiduo2:~# fdtget /boot/dtb/sun8i-h3-nanopi-duo2.dtb /soc/i2c@1c2b400 status
disabled

 

Thanks in advance 


 

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

6 6