MikePooh Posted June 23, 2020 Posted June 23, 2020 Could you please share the full step-by-step how-to enable the OV5640 camera on mainline kernel on H3 devices. Including CSI, drivers etc. Thanks in advance. 1 Quote
gsumner Posted June 24, 2020 Posted June 24, 2020 @MikePooh, I think there's still some schematic-reading involved at the moment, if you're willing to do that though it should be do-able. I don't know anything about other boards yet so this is the process followed. I think for "it just works" you'll need to wait a bit more. I'm assuming a lot here, e.g. active high and fixed-voltage gpio-switched regulators, so there is risk in following this. The first step is getting i2c working, the driver won't be able to do anything until you can communicate with the camera. I'll use the h5 in my Orange Pi Zero Plus2 as an example but I believe the process should be identical for h3 boards. Looking at the schematic (https://linux-sunxi.org/images/f/f6/ORANGE_PI-ZERO-PLUS2_V1_0.pdf), on page 9 is the csi connector. We want AFCC_EN, CSI_EN and CSI-PWR-EN to be '1' to power the camera. They might be named differently on different boards, but generally you're looking for the ones that aren't connected to pins (on page 6, chip U1D) with a CSI_* function (e.g. PE9/CSI_D5/TS_D5). I'm looking for AFCC_EN and CSI-PWR-EN which link to page 6. So AFCC_EN comes from pin PA7/SIM_CLK/PA_EINT7 and CSI-PWR-EN from PA8/SIM_DATA/PA_EINT8. We need to remember those PXX numbers. rreignier said their camera was on the i2c1 bus, so we need to confirm that too. Following CSI-SCK and CSI-SDA from page 9 to page 6 reveals the i2c interface. CSI-SCK for me connects to "PE12/CSI_SCK/TWI2_SCK" and CSI-SDA to "PE13/CSI_SDA/TWI2_SDA". Maybe someone can confirm, but I think TWIx_SDA equals I2Cx. You can test by enabling the i2c bus overlays one after the other and running i2cdetect while measuring with a multimeter on low range (200mv) A/C voltage between ground and the SCK pin on the camera connector - the correct bus will generate a voltage for a short while. We need to remember that bus number. Note the reset-gpios and powerdown-gpios the same way. Next is the change to the device tree. In Armbian checkout the linux source with "apt install linux-source-<kernel version>-current-sunxi" (or linux-source-<kernel version>-current-sunxi64 for h5) which will leave you with a tar in /usr/src of your kernel source. extract it (do it in a new folder or it will trash your current one) and then copy your kernel config in (cp /boot/config-* ./.config). Edit your .dts (./arch/arm/boot/dts/ for h3, ./arch/arm64/boot/dts/allwinner/ for h5) in line with my previous comment, swapping in your values for regulators, i2c bus and reset/powerdown. for a quick bodge, which pin goes in which regulator doesn't actually matter - my change just forced them all on anyway. As lex said, there's supposed to be an order, my change doesn't respect that order but it has worked so far. Then run "make dtbs" from the top level of the kernel source. That should produce a .dtb file in the same folder as the .dts. backup your current dtb and copy the new one to /boot/dtb/allwinner/*.dtb (that's the h5 path, not sure of the h3 path sorry, might be /boot/dtb/). I think you can alternatively specify this as fdtfile in the armbianEnv.txt. The armbianEnv.txt way will probably survive an update better. Finally, add a line to /etc/modules that says ov5640. Reboot and you might be good to go. If you don't have a /dev/video0 device we'll have to debug. Obviously if I've got something wrong feel free to chip in. Has anyone had success with resolutions other than 640x480? The driver seems to have some code for it, but I haven't gotten anything to work. 4 Quote
jgauthier Posted June 25, 2020 Posted June 25, 2020 I tried this (and gpio line to bitbang i2c) on my banana pi m2 zero and had no luck. 0 Quote
mostly Posted June 25, 2020 Posted June 25, 2020 On 6/21/2020 at 9:26 PM, gsumner said: XVCLK (CLK_CSI_MCLK in the device tree) needs to be running before accessing registers. I couldn't see an a/c voltage on the csi-mclk pin so figured that was the issue. The fix for me was connecting the clock parent in the device tree. I'll post my dtb changes. I'm not sure this is all correct, but it lets me open the camera without having to mess with any gpios etc after boot. Thank you! That's the part that i was missing. Now i'm able to run ov5640 on OrangePiPcPlus with kernel 5.4.45. My dts: https://pastebin.com/gdQnnrxQ /dev/video1 appears after reboot (no manual modprobe ov5640 needed). ov5640 for some reason does not leave any footprint in dmesg. I've kept compatible = "allwinner,sun6i-a31-i2c" in i2c2@1c2b400 to be able to check chip id by i2cdetect. And nothing was working without compatible = "allwinner,sun8i-h3-csi" and other stuff inside csi@1cb0000. Would be nice to pull the solution into official build (maybe as overlay) . Ov5640 module is one of very few official orangepi accessories 16 hours ago, gsumner said: Has anyone had success with resolutions other than 640x480? according to https://linux-sunxi.org/CSI media-ctl --device /dev/media1 --set-v4l2 '"ov5640 1-003c":0[fmt:UYVY/1920x1080]' 1920x1080 and 1280x760 are working well for me fswebcam -d /dev/video1 -r 1920x1080 -D 3 --jpeg 100 /mnt/sd/aaa.jpg --- Opening /dev/video1... Trying source module v4l2... /dev/video1 opened. No input was specified, using the first. Delaying 3 seconds. --- Capturing frame... Captured frame in 0.00 seconds. --- Processing captured image... Setting output format to JPEG, quality 100 Writing JPEG image to '/mnt/sd/aaa.jpg'. 2 Quote
rreignier Posted June 25, 2020 Author Posted June 25, 2020 Wow! That's very nice! Thanks a lot for the help. I did not have the time to try it yet. But for sure, a dts overlay should be pushed to armbian! 0 Quote
gsumner Posted June 25, 2020 Posted June 25, 2020 8 hours ago, mostly said: according to https://linux-sunxi.org/CSI media-ctl --device /dev/media1 --set-v4l2 '"ov5640 1-003c":0[fmt:UYVY/1920x1080]' @mostly awesome, thanks! I tried a couple others for anyone else wondering. e.g.: media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:JPEG_1X8/1920x1080]' v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=JPEG fswebcam -d /dev/video0 -r 1920x1080 -D 0 --jpeg 100 /tmp/test.jpg The colour balance and noise are terrible, but it's a start! I see there's something in media-ctl to change colourspace, but it's the first time I've heard of media-ctl and I've got no idea what I'm doing yet. Still no luck with 2592x1944 which I'd like to get going though. @jgauthier I'm not sure you need a bitbang i2c, the schematic for your board (https://drive.google.com/drive/u/0/folders/0B4PAo2nW2KfnflVqbjJGTFlFTTd1b1o1OUxDNk5ackVDM0RNUjBpZ0FQU19SbDk1MngzZWM?tid=0B4PAo2nW2Kfndjh6SW9MS2xKSWs) says they connect to TWI2. If the TWIx == I2Cx thing holds, I2C2 might work for you(?). It looks like CSI-PWR-EN controls all the power to your camera too, that net might be strongly pulled-up next to U9 too. So maybe try enable PD14 for your regulator and I2C2 for the bus. 0 Quote
mostly Posted June 29, 2020 Posted June 29, 2020 On 6/25/2020 at 10:53 PM, gsumner said: The colour balance and noise are terrible, but it's a start! I see there's something in media-ctl to change colourspace, but it's the first time I've heard of media-ctl and I've got no idea what I'm doing yet. Still no luck with 2592x1944 which I'd like to get going though. Yes noise is annoying. Maybe "5.6 color interpolation (CIP)" mentioned in ov5640 datasheet can deal with it (CIP seems to be already enabled in driver initialization, maybe tuning of CIP parameters can help). Also not sure what to do with autofocus. The driver also has nothing for it, but i have not found in the datasheet how to enable it. One problem found with 1920x1080 resolution: still image capture with fswebcam works but attempt to get 1080p video with ffmpeg gives noisy greenish picture where only contours of objects can be recognized, exactly like in that message: 1280x720 is fine. 0 Quote
gsumner Posted June 30, 2020 Posted June 30, 2020 I think I understand that media-ctl is setting the resolution/format (or group of formats) that will be able to be successfully opened by a program. I'm confused by the media-ctl, v4l2-ctl and program order of operations though, it seems like you have to run your program first, see it fail, run v4l2-ctl -V to see the (last?) format it tried to open, then you go back and set something compatible with media-ctl and try again... is that correct? Surely it should see what the program is asking for and reconfigure itself? With fswebcam (which seems to want JPEG format), running: media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:JPEG_1X8/1920x1080]' fswebcam -d /dev/video0 -r 1920x1080 -D 0 --jpeg 100 /tmp/test.jpg there's a weird issue where 1 in 3 or 4 images has a green tinge. All the detail is there, just greenish. Does anyone have the same issue? I'm also having an issue with vlc, it seems to want planar yuv (YUYV8_2X8), but I can't get a successful stream out of it. cvlc complains "rawvideo decoder warning: invalid frame size (3110400 < 4665600)", which agrees with the "Size Image" field of v4l2-ctl -V. I wonder if the frame size error is related to the green/purple image problem or separate, 3110400 means 12bit/pixel while 4665600 is 18bit/pixel. The format and v4l2-ctl -V agree on 3110400 so I wonder where 4665600 comes from... and I also don't understand how to differentiate yuv422 and yuv420 from media-ctl --known-mbus-fmts. @mostly, ISP CONTROL 03 "Draw window for AFC enable" looks interesting for autofocus. I might try enabling that to see what happens. I wonder if the autofocus is a oneshot thing that the driver would need to fire off every time or constantly focusing. I'm also having the same issues with YUV green/purple images that you mention, if you run ffmpeg with "-input_format mjpeg" you can at least get back to the "green tinge" problem. I tried signing the driver I built previously with the keys I found in the Linux source package but I keep getting exec format error, I was hoping to avoid rebuilding the whole kernel. Are the signing keys available? Maybe that defeats the purpose of keys though! I guess if swapping between mjpeg and yuv is only reconfiguring the soc and not the camera module itself then we should look at sun6i_csi for these format and frame size issues(?). 0 Quote
mostly Posted July 7, 2020 Posted July 7, 2020 On 6/30/2020 at 7:33 PM, gsumner said: there's a weird issue where 1 in 3 or 4 images has a green tinge. All the detail is there, just greenish. Does anyone have the same issue? I'm not experiencing that, all still image captures are ok, both at 720p and 1080p. But problem with green 1080p video capture still here. On 6/30/2020 at 7:33 PM, gsumner said: ISP CONTROL 03 "Draw window for AFC enable" looks interesting for autofocus Somebody was trying this register but it seems to be not the right one: https://patchwork.kernel.org/patch/10342907/ There is one interesting commit claiming adding autofocus: https://github.com/D3Engineering/linux_kernel_qcomlt/blob/7b3c6e86d236db6d0c523dbd3297deb7563d7bf1/drivers/media/i2c/ov5640.c but it's for legacy On 6/30/2020 at 7:33 PM, gsumner said: but I keep getting exec format error This way of building modules works for me: https://yoursunny.com/t/2018/one-kernel-module/ + touch .scmversion 0 Quote
gsumner Posted July 11, 2020 Posted July 11, 2020 @mostly Thanks, not sure why it wasn't working before, I can build modules now and signing turned out to not be needed. I've been looking at the sun6i_csi, sun6i_video and ov5640 drivers, at the moment I think there is a problem in the clock generation in the ov5640 code. I'll share my test patch: diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c index 18dd2d717..7b5166b11 100644 --- a/drivers/media/i2c/ov5640.c +++ b/drivers/media/i2c/ov5640.c @@ -470,54 +470,106 @@ static const struct reg_value ov5640_setting_PAL_720_576[] = { }; static const struct reg_value ov5640_setting_720P_1280_720[] = { - {0x3c07, 0x07, 0, 0}, - {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0}, - {0x3814, 0x31, 0, 0}, - {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0}, - {0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0}, - {0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0}, - {0x3810, 0x00, 0, 0}, - {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0}, - {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0}, - {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0}, - {0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0}, - {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x72, 0, 0}, {0x3a0e, 0x01, 0, 0}, - {0x3a0d, 0x02, 0, 0}, {0x3a14, 0x02, 0, 0}, {0x3a15, 0xe4, 0, 0}, - {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, - {0x4407, 0x04, 0, 0}, {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0}, - {0x3824, 0x04, 0, 0}, {0x5001, 0x83, 0, 0}, + {0x3612, 0x29, 0, 0}, /* ??? between VCM and timing control */ + {0x3618, 0x00, 0, 0}, /* ??? between VCM and timing control */ + + {0x3708, 0x64, 0, 0}, /* ??? between VCM and timing control */ + {0x3709, 0x52, 0, 0}, /* ??? between VCM and timing control */ + {0x370c, 0x03, 0, 0}, /* ??? between VCM and timing control */ + + {0x3800, 0x00, 0, 0}, /* TIMING HS */ + {0x3801, 0x00, 0, 0}, /* TIMING HS */ + {0x3802, 0x00, 0, 0}, /* TIMING VS */ + {0x3803, 0xfa, 0, 0}, /* TIMING VS */ + {0x3804, 0x0a, 0, 0}, /* TIMING HW */ + {0x3805, 0x3f, 0, 0}, /* TIMING HW */ + {0x3806, 0x06, 0, 0}, /* TIMING VH */ + {0x3807, 0xa9, 0, 0}, /* TIMING VH */ + {0x3810, 0x00, 0, 0}, /* TIMING HOFFSET */ + {0x3811, 0x10, 0, 0}, /* TIMING_HOFFSET */ + {0x3812, 0x00, 0, 0}, /* TIMING VOFFSET */ + {0x3813, 0x04, 0, 0}, /* TIMING VOFFSET */ + {0x3814, 0x31, 0, 0}, /* TIMING X INC */ + {0x3815, 0x31, 0, 0}, /* TIMING Y INC */ + {0x3824, 0x04, 0, 0}, /* DVP PCLK divider for VFIFO CTRL0C */ + + {0x3a02, 0x02, 0, 0}, /* AEC MAX EXPO (60HZ) */ + {0x3a03, 0xe4, 0, 0}, /* AEC MAX EXPO (60HZ) */ + {0x3a08, 0x01, 0, 0}, /* AEC B50 STEP */ + {0x3a09, 0xbc, 0, 0}, /* AEC B50 STEP */ + {0x3a0a, 0x01, 0, 0}, /* AEC B60 STEP */ + {0x3a0b, 0x72, 0, 0}, /* AEC B60 STEP */ + {0x3a0e, 0x01, 0, 0}, /* AEC CTRL0E */ + {0x3a0d, 0x02, 0, 0}, /* AEC CTRL0D */ + {0x3a14, 0x02, 0, 0}, /* AEC MAX EXPO (50HZ) */ + {0x3a15, 0xe4, 0, 0}, /* AEC MAX EXPO (50HZ) */ + + {0x3c07, 0x07, 0, 0}, /* LIGHT METER1 THRESHOLD */ + {0x3c09, 0x1c, 0, 0}, /* LIGHT METER2 THRESHOLD */ + {0x3c0a, 0x9c, 0, 0}, /* SAMPLE NUMBER */ + {0x3c0b, 0x40, 0, 0}, /* SAMPLE NUMBER */ + + {0x4001, 0x02, 0, 0}, /* BLC CTRL01 */ + {0x4004, 0x02, 0, 0}, /* BLC CTRL04 */ + + {0x4407, 0x04, 0, 0}, /* JPEG CTRL07 */ + + {0x460b, 0x37, 0, 0}, /* DEBUG MODE */ + {0x460c, 0x20, 0, 0}, /* VFIFO CTRL0C */ + + {0x5001, 0x83, 0, 0}, /* ISP CONTROL 01 */ }; static const struct reg_value ov5640_setting_1080P_1920_1080[] = { - {0x3008, 0x42, 0, 0}, - {0x3c07, 0x08, 0, 0}, - {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0}, - {0x3814, 0x11, 0, 0}, - {0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0}, - {0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0}, - {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0}, - {0x3810, 0x00, 0, 0}, - {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0}, - {0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0}, - {0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0}, - {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0}, - {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0}, - {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0}, - {0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, - {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0}, - {0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0}, - {0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0}, - {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0}, - {0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0}, - {0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0}, - {0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0}, - {0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0}, - {0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0}, - {0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0}, - {0x3a0e, 0x03, 0, 0}, {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x04, 0, 0}, - {0x3a15, 0x60, 0, 0}, {0x4407, 0x04, 0, 0}, - {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0}, {0x3824, 0x04, 0, 0}, - {0x4005, 0x1a, 0, 0}, {0x3008, 0x02, 0, 0}, + {0x3612, 0x2b, 0, 0}, /* ??? between VCM and timing control */ + {0x3618, 0x04, 0, 0}, /* ??? between VCM and timing control */ + {0x3708, 0x64, 0, 0}, /* ??? between VCM and timing control */ + {0x3709, 0x12, 0, 0}, /* ??? between VCM and timing control */ + {0x370c, 0x00, 0, 0}, /* ??? between VCM and timing control */ + + {0x3800, 0x01, 0, 0}, /* TIMING HS */ + {0x3801, 0x50, 0, 0}, /* TIMING HS */ + {0x3802, 0x01, 0, 0}, /* TIMING VS */ + {0x3803, 0xb2, 0, 0}, /* TIMING VS */ + {0x3804, 0x08, 0, 0}, /* TIMING HW */ + {0x3805, 0xef, 0, 0}, /* TIMING HW */ + {0x3806, 0x05, 0, 0}, /* TIMING VH */ + {0x3807, 0xf1, 0, 0}, /* TIMING VH */ + {0x3810, 0x00, 0, 0}, /* TIMING HOFFSET */ + {0x3811, 0x10, 0, 0}, /* TIMING HOFFSET */ + {0x3812, 0x00, 0, 0}, /* TIMING VOFFSET */ + {0x3813, 0x04, 0, 0}, /* TIMING VOFFSET */ + {0x3814, 0x11, 0, 0}, /* TIMING X INC */ + {0x3815, 0x11, 0, 0}, /* TIMING Y INC */ + {0x3824, 0x04, 0, 0}, /* DVP PCLK divider for VFIFO CTRL0C */ + + {0x3a02, 0x04, 0, 0}, /* AEC MAX EXPO (60HZ) */ + {0x3a03, 0x60, 0, 0}, /* AEC MAX EXPO (60HZ) */ + {0x3a08, 0x01, 0, 0}, /* AEC B50 STEP */ + {0x3a09, 0x50, 0, 0}, /* AEC B50 STEP */ + {0x3a0a, 0x01, 0, 0}, /* AEC B60 STEP */ + {0x3a0b, 0x18, 0, 0}, /* AEC B60 STEP */ + {0x3a0d, 0x04, 0, 0}, /* AEC CTRL0D */ + {0x3a0e, 0x03, 0, 0}, /* AEC CTRL0E */ + {0x3a14, 0x04, 0, 0}, /* AEC MAX EXPO (50HZ) */ + {0x3a15, 0x60, 0, 0}, /* AEC MAX EXPO (50HZ) */ + + {0x3c07, 0x07, 0, 0}, /* LIGHT METER1 THRESHOLD */ + {0x3c08, 0x00, 0, 0}, /* LIGHT METER2 THRESHOLD */ + {0x3c09, 0x1c, 0, 0}, /* LIGHT METER2 THRESHOLD */ + {0x3c0a, 0x9c, 0, 0}, /* SAMPLE NUMBER */ + {0x3c0b, 0x40, 0, 0}, /* SAMPLE NUMBER */ + + {0x4001, 0x02, 0, 0}, /* BLC CTRL01 */ + {0x4004, 0x06, 0, 0}, /* BLC CTRL04 */ + {0x4005, 0x1a, 0, 0}, /* BLC CTRL05 */ + + {0x4407, 0x04, 0, 0}, /* JPEG CTRL07 */ + + {0x460b, 0x37, 0, 0}, /* DEBUG MODE */ + {0x460c, 0x20, 0, 0}, /* VFIFO CTRL0C */ + + {0x5001, 0x83, 0, 0}, /* ISP CONTROL 01 */ }; static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = { @@ -787,7 +839,7 @@ static int ov5640_mod_reg(struct ov5640_dev *sensor, u16 reg, * This is supposed to be ranging from 1 to 16, but the value is * always set to either 1 or 2 in the vendor kernels. */ -#define OV5640_SYSDIV_MIN 1 +#define OV5640_SYSDIV_MIN 2 #define OV5640_SYSDIV_MAX 16 /* @@ -848,7 +900,7 @@ static unsigned long ov5640_calc_sys_clk(struct ov5640_dev *sensor, u8 *sysdiv) { unsigned long best = ~0; - u8 best_sysdiv = 1, best_mult = 1; + u8 best_sysdiv = OV5640_SYSDIV_MIN, best_mult = OV5640_PLL_MULT_MIN; u8 _sysdiv, _pll_mult; for (_sysdiv = OV5640_SYSDIV_MIN; @@ -874,7 +926,7 @@ static unsigned long ov5640_calc_sys_clk(struct ov5640_dev *sensor, * We have reached the maximum allowed PLL1 output, * increase sysdiv. */ - if (!rate) + if (!_rate) break; /* @@ -993,14 +1045,19 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor, { unsigned long _rate = rate * OV5640_PLL_ROOT_DIV * OV5640_BIT_DIV * OV5640_PCLK_ROOT_DIV; + unsigned long pclk; _rate = ov5640_calc_sys_clk(sensor, _rate, pll_prediv, pll_mult, sysdiv); *pll_rdiv = OV5640_PLL_ROOT_DIV; *bit_div = OV5640_BIT_DIV; *pclk_div = OV5640_PCLK_ROOT_DIV; + pclk = _rate / *pll_rdiv / *bit_div / *pclk_div; - return _rate / *pll_rdiv / *bit_div / *pclk_div; + dev_warn(&sensor->i2c_client->dev, "ov5640_calc_pclk sysclk: %lu pclk: %lu\n", + _rate, pclk); + + return pclk; } static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor, unsigned long rate) @@ -1014,6 +1071,10 @@ static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor, unsigned long rate) if (bit_div == 2) bit_div = 8; + dev_warn(&sensor->i2c_client->dev, "ov5640_set_dvp_pclk bit_div: %u" \ + ", sysdiv: %u, mult: %u, prediv: %u, pll_rdiv: %u, pclk_div: %u\n", + bit_div, sysdiv, mult, prediv, pll_rdiv, pclk_div); + ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL0, 0x0f, bit_div); if (ret) @@ -1078,6 +1139,10 @@ static int ov5640_set_timings(struct ov5640_dev *sensor, return ret; } + dev_warn(&sensor->i2c_client->dev, + "ov5640_set_timings setting hact: %u, vact: %u, htot: %u, vtot: %u", + mode->hact, mode->vact, mode->htot, mode->vtot); + ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, mode->hact); if (ret < 0) return ret; The 720p registers are just sorted, but the 1080p ones had a lot of repeated addresses. I've assumed they aren't stateful and only kept the last instance and then sorted them. I also removed a few values that were mentioned in mainline commits. It think it's functionally the same as before but has helped debugging and might help someone else. I don't think they're the current issue anyway. With the patch (minus the change to OV5640_SYSDIV_MIN) and 720p you'll see in dmesg: ov5640 2-003c: ov5640_set_dvp_pclk bit_div: 8, sysdiv: 2, mult: 85, prediv: 3, pll_rdiv: 2, pclk_div: 1 But with 1080p sysdiv notably jumps to 1: ov5640 2-003c: ov5640_set_dvp_pclk bit_div: 8, sysdiv: 1, mult: 84, prediv: 3, pll_rdiv: 2, pclk_div: 1 If you add the change to OV5640_SYSDIV_MIN and test using the OV5640's pattern generator: v4l2-ctl --set-ctrl test_pattern=2 media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080]' ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output.mjpg You should get a perfect 1080p test pattern. I can't get anything but a black screen and lines if I turn the pattern off and try to capture a real image, but if you compare the pattern to the pattern without my change I think you'll agree there's a clocking problem. The documentation on the clock structure in the code is the only useful source I've found unfortunately. There is also a kernel warning in dmesg with 1080p regarding "dma_alloc_coherent of size 4149248 failed" but I think(?) it's a red herring - it's asking for 32 buffers and it looks like only the 32nd one fails. The capture continues without it. Hopefully this helps get us closer to working 1080p!@rreignier Did you get your board working? I guess we did end up hijacking your thread anyway sorry. 0 Quote
rreignier Posted July 11, 2020 Author Posted July 11, 2020 47 minutes ago, gsumner said: @rreignier Did you get your board working? I guess we did end up hijacking your thread anyway sorry. Hi @gsumner No, I did not have time to try it out on my board yet But I am very glad that you have managed to get this camera working on mainline and shared your progress. I am following with great attention! Thanks! Did you have a look at the blog post about PinePhone camera which happen to be an OV5640 too? https://blog.brixit.nl/camera-on-the-pinephone/ 0 Quote
@lex Posted July 11, 2020 Posted July 11, 2020 5 hours ago, gsumner said: There is also a kernel warning in dmesg with 1080p regarding "dma_alloc_coherent of size 4149248 failed" but I think(?) it's a red herring - it's asking for 32 buffers and it looks like only the 32nd one fails. Usually V4l2 applications map 4 buffers (you can map 8 buffers if you have enough memory to get some optimization). For the size of 1920x1080 and YUV420 format you get a buffer size of 1920*1080*2 = 4147200 mapped in memory and the sensor delivers an image of size 1920*1080*3/2 = 3110400. Looks like your VLC is addding some extra bytes for any reason. 32 buffers is around 128 MB, try to increase CMA, or change and recompile it for 8 buffers. Regarding the green tinge and pinkish or grainy image i think is related to light condition, if you take shoots at day light (outside) you get better image just like the PinePhone. Can you show an Image 1920x1090 with daylight (outside) and one with indirect light (inside)? The current driver is able to sustain a 1920x1080 ~10 FPS on a nanopi air, 1.2Ghz. I think on A64 you can get ~15FPS. 0 Quote
gsumner Posted July 11, 2020 Posted July 11, 2020 Hi Lex, I think I've been talking about two unrelated issues, the greenish image showed up every n images but I only tested indoors. I think I now know that was only when the camera was in jpeg mode. Unfortunately my driver is in a sorry state at the moment for taking new images. This drive link has the old images: https://drive.google.com/drive/folders/1kjn-D02tEXLiDAMPOVFVVTlrnqgGFkro?usp=sharing The other two images represent the issue that mostly and I are talking about, which is when the camera is in yuv mode. When using yuv in 1080p (I'm testing using "v4l2-ctl --set-ctrl test_pattern=2") you get the heavily distorted green/pink images. I currently believe they are because of an issue in the clock calculations I mentioned above. I can get a reasonable looking 1080p test pattern by forcing sysdiv to more than 1 but can't get real images. I'm guessing that means that the SCLK is a reasonable value (the ISP must generate the pattern??) but the PCLK is clearly not good. I wonder if everyone else using the ov5640 driver uses csi instead of parallel... I think if I start looking into the cma alloc issue at the same time my head might explode! I'll have to look after the yuv-1080p-clock issue. Regarding VLC, I'm wondering if the issue is because vlc is opening the camera in yuv420 mode, but in the sun6i_csi driver I can see in "get_csi_input_format" that yuv420 isn't supported? Would that explain the frame size difference?@rreignier Thanks, that looks helpful. So they're using an Allwinner, ov5640 and parallel camera too. I'll have to look to see if they've already solved this. 0 Quote
@lex Posted July 11, 2020 Posted July 11, 2020 1 hour ago, gsumner said: Would that explain the frame size difference? i think so. I tested your pattern sample on A64 and both pattern looked fine (exactly the same) ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output_yuyv422.mjpg ffmpeg -input_format yuv420p -s 1920x1080 -i /dev/video0 /tmp/output_yuyv420.mjpg which i think should fail for the yuyv422 on fmt:YUYV8_2X8/1920x1080 Image quality is similar to what you got with your test_0. I would advise to move on to latest ov5640 driver or if you find the ov5640 driver (PinePhone) with additional fix , let us know the diff. Maybe @rreignier can have different results. Hope this information will be helpful. Good luck. 0 Quote
gsumner Posted July 17, 2020 Posted July 17, 2020 @mostly and anyone else interested, I have gotten normal looking yuv422 video at 1920x1080 working (I think). I only have part of the final fix though, because I don't understand yet why the rate is out by two. I thought it might have been dynamic and only broken when sysdiv went to 1x, but I can capture fine looking video at 720p and 480p with this change too. The minimal change is below: diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c index 18dd2d717..342e80d21 100644 --- a/drivers/media/i2c/ov5640.c +++ b/drivers/media/i2c/ov5640.c @@ -992,7 +992,7 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor, u8 *pll_rdiv, u8 *bit_div, u8 *pclk_div) { unsigned long _rate = rate * OV5640_PLL_ROOT_DIV * OV5640_BIT_DIV * - OV5640_PCLK_ROOT_DIV; + OV5640_PCLK_ROOT_DIV / 2; _rate = ov5640_calc_sys_clk(sensor, _rate, pll_prediv, pll_mult, sysdiv); Could someone please confirm that they had the pink/green image problem and that this fixes it at 1080p for them? Tested with: media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080]' ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output.mjpg I've been sticking with yuv422 so far because the sun6i_csi driver mentions not supporting yuv420 and the ov5640 seems to have some 16bit/pixel parts hardcoded. I'd like to understand the clock structure better, but as I said previously I'm struggling to find details. 0 Quote
gsumner Posted July 19, 2020 Posted July 19, 2020 I started having a look at 2592x1944 video, for anyone else trying the same thing and failing with "Unable to setup formats: Invalid argument (22)" from media-ctl, the issue appears to be the framerate/interval. 2592x1944 only works at 15fps but the command (@1/15 is the frame interval, inverse of framerate): media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/2592x1944@1/15]' seems to have its order of operation backwards, it tries 2592x1944 at the current framerate, and sets the new framerate after, which fails. To work around it you can do the following: media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080@1/15]' media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/2592x1944]' 1080p works at 30fps, which is why you can switch to 15fps in the same command, then you can move the resolution to 2592x1944 once already at 15fps. I can't get it to capture any video at 2592x1944 yet, but hopefully this helps anyone else stuck where I was. 0 Quote
gsumner Posted July 19, 2020 Posted July 19, 2020 Hi, I think I have 2592x1944 working (with the above workaround) yuv422 and jpeg. If you just want the change (needs to be combined with the device tree change from earlier in the thread if you're starting from scratch) it's below: diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c index 18dd2d717..6c8d00c30 100644 --- a/drivers/media/i2c/ov5640.c +++ b/drivers/media/i2c/ov5640.c @@ -535,7 +535,7 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = { {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0}, {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0}, {0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, - {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0}, + {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x20, 0, 0}, {0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 70}, }; @@ -992,7 +992,7 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor, u8 *pll_rdiv, u8 *bit_div, u8 *pclk_div) { unsigned long _rate = rate * OV5640_PLL_ROOT_DIV * OV5640_BIT_DIV * - OV5640_PCLK_ROOT_DIV; + OV5640_PCLK_ROOT_DIV / OV5640_BIT_DIV; _rate = ov5640_calc_sys_clk(sensor, _rate, pll_prediv, pll_mult, sysdiv); The explanation is that I think the parallel bus calculation (mipi csi on other SoCs unaffected) is missing an extra bit_div which is in the diagram but not the calculation. The 2592x1944 register 0x460c setting 0x22 overrides that extra OV5640_BIT_DIV I've added with the value in register 0x3824, so 0x20 brings it back in line with 720p and 1080p using "auto" calculation. Lower resolutions use the override too but they worked when I tested them so I haven't messed with it. I'm concerned that lex has 1080p working on an a64 without the change though, I thought the fix would be independent of SoC and affect all parallel buses with the ov5640. Lex, could you confirm it was a64 + ov5640 + no driver changes that worked for you? (Assuming the a64 works with my change) We could have something Armbian could apply soon, could you post your Allwinner board+SoC, camera and whether this change works for you? I think we should also make a change to try the framerate before/after the resolution so we don't need the workaround in the previous post.@Igor I think I nearly have a change to get 1080p and 1944p working with ov5640 and parallel bus (as opposed to mipi csi). I haven't done this with Armbian before, how much testing do I need? Do I need to find all boards with an ov5640 and parallel bus? I need to test some more formats/resolutions on my board before I'm confident at least anyway. There's also a separate change for the device tree from earlier in the thread that I think could also be submitted with some more testing. I'd like to help contribute this to Armbian if I can get these tested on some other boards and in the right format for you. 0 Quote
@lex Posted July 19, 2020 Posted July 19, 2020 32 minutes ago, gsumner said: Lex, could you confirm it was a64 + ov5640 + no driver changes that worked for you? Untoched kernel driver works for A64 and H2/H3 on the following win size: * 320x240 (30 FPS) * 640x480 (30 FPS) * 1280x720 ( < 30 FPS) * 1920x1080 ( < 30 FPS) Quality is acceptable (outdoor). Does not work: 60 FPS, 2592x1944 I think you are right about 2592x1944 , i will test your changes and report back. 0 Quote
gsumner Posted July 19, 2020 Posted July 19, 2020 @mostly, it's apparently been a productive Sunday, I had a look at the Auto Focus commit you linked and did a quick hack job to bring it into the current driver. I've shown it working here: https://drive.google.com/drive/folders/1kjn-D02tEXLiDAMPOVFVVTlrnqgGFkro?usp=sharing (I can only play the video with ubuntu default video app??). The problem is that (I believe) the large array they're reading in to address 0x8000 is firmware for the microcontroller to run, so there was no AF feature to enable before the mcu code was loaded. With this hack it seems to be in a constant-refocus mode by default, but there are untested v4l2 controls for other modes. I'll attach the patch but I'd like to stress that it was chopped into the driver with no care, I've already seen instability playing with it. I think it could be close enough to working to be a starting point for a real change though, I might start looking at improving it. I don't like that it's a black box though, debugging it will be a pain. I also think it might be loading the firmware everytime you record, which seems over the top. ov5640_AF.patch 0 Quote
mostly Posted July 20, 2020 Posted July 20, 2020 On 7/18/2020 at 12:04 AM, gsumner said: I have gotten normal looking yuv422 video at 1920x1080 working (I think). I only have part of the final fix though, because I don't understand yet why the rate is out by two Could someone please confirm that they had the pink/green image problem and that this fixes it at 1080p for them? Tested with: media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080]' ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output.mjpg Tried it and got almost adequate FHD picture with OV5640_PCLK_ROOT_DIV / 2 . If using ffmpeg, the only problem that i see is that there is more green than expected. In the meantime, i brought up h264 hw encoding from uboborov sunxi-cedar-mainline + uvc_h264_lb_allwinner (turned on writing to file and FPS calculation in settings inside main.c). It works well with 1280x720@15fps (real 15fps in h264 file), but with 1920x1080 OV5640_PCLK_ROOT_DIV / 2 it gives some artifacts with colors and adds shadows to dark objects on white background. Not sure yet if this problem is from hw encoder side or ov5640. On 7/19/2020 at 12:55 PM, gsumner said: 1080p works at 30fps Were you able to get real 30fps? uvc_h264_lb_allwinner shows actual FPS, and 1280x720 framerate is always 15 and 1920x1080 framerate is always 7 regardless of what set in media-ctl format @ 1/ On 7/19/2020 at 3:36 PM, gsumner said: I'm concerned that lex has 1080p working on an a64 without the change though So a64 does not need that patch, and also does not need "connecting the clock parent in the device tree" in dts. Since we see these strange clocks behavior, can this custom clock source be not suitable for some modes?... 0 Quote
gsumner Posted July 21, 2020 Posted July 21, 2020 Lex, when you test a64 could you try with/without the OV5640_PCLK_ROOT_DIV change I proposed at a few resolutions including 2592x1944? My H5 was working at 720p without the change so I'm wondering if the higher resolutions require a more 'correct' clock but will still function at lower res? If your boards only work at 2592x1944 with the change or if they work both with/without the change I think there could still be hope that it is correct, otherwise I think we'll have to look more on the SoC side at what differs between SoCs. btw, I'm very confused that H3 works for you without changes, they share a lot of DTB and driver code for this - it might narrow the search if H3 works and H5 doesn't. I was thinking the clock change was the perfect answer given it wasn't in the calculation but was in the diagram and that it was the difference between green/purple image and perfect-ish 1080p/1944p for me. But if it at makes other SoCs perform worse I think I'll accept that it's wrong, the camera shouldn't have SoC specific code. 0 Quote
@lex Posted July 21, 2020 Posted July 21, 2020 I will test with and without asap and give some feedback. Perhaps you should open a new thread, something like "Improving ov5640 (mainline)" and push your code/patch there (up to you). How do you test your FPS? Maybe you could get inspired by the BSP code: https://github.com/avafinger/ov5640/blob/A64/ov5640.c#L4577 0 Quote
gsumner Posted July 26, 2020 Posted July 26, 2020 Looking at the BSP code and the driver, I think my change to the clock can be ignored now - it's coupled to the framerate, so even if it had been correct I think it would halve the framerate. I think I will start looking at clocking on the SoC, after that I could only guess at electrical design issues(?). @mostly has an H3 and I have an H5 which have issues, but Lex has an H3 and A64 which work, which makes me worry it's an electrical design issue... I think the auto focus patch should still work if the couple of lines changing clocks are removed though. Lex, thanks, I think you don't need to worry about testing the change. I'll make a thread for ov5640 improvements for things like the auto focus after looking at this issue (SoC/board specific corrupted green/pink images) for a bit more. I haven't been concerned with framerate, just looking at what ffmpeg output. The rates I mentioned before were just the configured rate in media-ctl. @mostly No, I've never seen 30fps at 1080p sorry, I was referring to the configured rate. I only get 2 or 3 fps at 1080p but I've realised that's because the clock change is wrong. I agree that it might be time to look at clocking on the SoC side. I've ordered an Orange Pi One with an H3 to see if I can get different behaviour from an H3 on a different board. 0 Quote
@lex Posted July 26, 2020 Posted July 26, 2020 I have updated fswebcam for the mainline kernel. You get the FPS printed on the picture, all seems to be fine. Tested with A64, H3 and H2 with your: - {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0}, + {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x20, 0, 0}, Samples: H3 https://github.com/avafinger/fswebcam/tree/mainline-kernel/nanopi-air H2 https://github.com/avafinger/fswebcam/tree/mainline-kernel/bpi-m2z A64 https://github.com/avafinger/fswebcam/tree/mainline-kernel/nanopi-a64 You should focus on the AF improvements. Looking forward to your progress with AF. Unfortunately i was removing old files and directories to free some space and did a sudo rm -rf * in the wrong root directory and now i can't recover important stuff, s* happens, no backup. The inodes are marked free now. 0 Quote
mostly Posted July 27, 2020 Posted July 27, 2020 17 hours ago, gsumner said: makes me worry it's an electrical design issue Even if there is something bad in electrical design, it is somehow solvable from software side - year ago i had both 1080p and 2592x1944 working without any artifacts on kernel 3.4.113 on the same board OrangePiPC+. 17 hours ago, gsumner said: I only get 2 or 3 fps at 1080p but I've realised that's because the clock change is wrong I believe 2-3fps is due to some bottleneck in ffmpeg (you'll likely see 100% core utilization by ffmpeg). I'm getting poor fps in ffmpeg at 720p and 1080p, but optimized uvc_h264_lb_allwinner with hw encoding gives pretty stable 15fps at 720p (no artifacts at all) and 7fps at 1080p (with /2 patch, some color artifacts are visible). 0 Quote
gsumner Posted July 28, 2020 Posted July 28, 2020 On 7/27/2020 at 2:06 AM, mostly said: Even if there is something bad in electrical design, it is somehow solvable from software side - year ago i had both 1080p and 2592x1944 working without any artifacts on kernel 3.4.113 on the same board OrangePiPC+. Thanks, I didn't know that. There must be a software problem/solution then. I started having a look at clocks, trying to see if setting mclk parent/rate in the dtb was having the effect I thought, there's a feature to print the clock structure: sudo cat /sys/kernel/debug/clk/clk_summary enable prepare protect duty clock count count count rate accuracy phase cycle --------------------------------------------------------------------------------------------- iosc 0 0 0 16000000 300000000 0 50000 ext_osc32k 1 1 0 32768 50000 0 50000 osc32k 1 1 0 32768 50000 0 50000 ir 0 0 0 32768 50000 0 50000 osc32k-out 0 0 0 32768 50000 0 50000 osc24M 15 15 0 24000000 50000 0 50000 ar100 1 1 0 24000000 50000 0 50000 ahb0 1 1 0 24000000 50000 0 50000 apb0 1 1 0 24000000 50000 0 50000 apb0-twd 0 0 0 24000000 50000 0 50000 apb0-i2c 0 0 0 24000000 50000 0 50000 apb0-uart 0 0 0 24000000 50000 0 50000 apb0-timer 0 0 0 24000000 50000 0 50000 apb0-ir 0 0 0 24000000 50000 0 50000 apb0-pio 1 1 0 24000000 50000 0 50000 csi-mclk 0 0 0 24000000 50000 0 50000 hdmi-ddc 2 2 0 24000000 50000 0 50000 avs 0 0 0 24000000 50000 0 50000 csi-misc 0 0 0 24000000 50000 0 50000 usb-ohci3 1 1 0 24000000 50000 0 50000 usb-ohci2 1 1 0 24000000 50000 0 50000 usb-ohci1 0 0 0 24000000 50000 0 50000 usb-ohci0 0 0 0 24000000 50000 0 50000 usb-phy3 1 1 0 24000000 50000 0 50000 usb-phy2 1 1 0 24000000 50000 0 50000 usb-phy1 0 0 0 24000000 50000 0 50000 usb-phy0 1 1 0 24000000 50000 0 50000 spi1 0 0 0 24000000 50000 0 50000 spi0 0 0 0 24000000 50000 0 50000 ce 0 0 0 24000000 50000 0 50000 ts 0 0 0 24000000 50000 0 50000 nand 0 0 0 24000000 50000 0 50000 ths 1 1 0 24000000 50000 0 50000 apb2 2 2 0 24000000 50000 0 50000 bus-scr1 0 0 0 24000000 50000 0 50000 bus-scr0 0 0 0 24000000 50000 0 50000 bus-uart3 0 0 0 24000000 50000 0 50000 bus-uart2 0 0 0 24000000 50000 0 50000 bus-uart1 0 0 0 24000000 50000 0 50000 bus-uart0 1 1 0 24000000 50000 0 50000 bus-i2c2 1 1 0 24000000 50000 0 50000 bus-i2c1 0 0 0 24000000 50000 0 50000 bus-i2c0 0 0 0 24000000 50000 0 50000 pll-de 1 1 0 432000000 50000 0 50000 de 2 2 0 432000000 50000 0 50000 wb-div 0 0 0 432000000 50000 0 50000 wb 0 0 0 432000000 50000 0 50000 mixer1-div 0 0 0 432000000 50000 0 50000 mixer1 0 0 0 432000000 50000 0 50000 mixer0-div 1 1 0 432000000 50000 0 50000 mixer0 1 1 0 432000000 50000 0 50000 tve 0 0 0 432000000 50000 0 50000 pll-periph1 0 0 0 600000000 50000 0 50000 pll-gpu 1 1 0 384000000 50000 0 50000 gpu 1 1 0 384000000 50000 0 50000 pll-periph0 6 6 0 600000000 50000 0 50000 mmc1 1 1 0 50000000 50000 0 50000 mmc2 1 1 0 50000000 50000 0 50000 mmc0 1 1 0 50000000 50000 0 50000 csi-sclk 0 0 0 300000000 50000 0 50000 deinterlace 0 0 0 600000000 50000 0 50000 ahb2 4 4 0 300000000 50000 0 50000 bus-ohci3 2 2 0 300000000 50000 0 50000 bus-ohci2 2 2 0 300000000 50000 0 50000 bus-ohci1 0 0 0 300000000 50000 0 50000 bus-ehci3 2 2 0 300000000 50000 0 50000 bus-ehci2 2 2 0 300000000 50000 0 50000 bus-ehci1 0 0 0 300000000 50000 0 50000 bus-emac 0 0 0 300000000 50000 0 50000 ahb1 11 12 0 200000000 50000 0 50000 bus-dbg 0 0 0 200000000 50000 0 50000 bus-ephy 0 0 0 200000000 50000 0 50000 bus-spinlock 0 0 0 200000000 50000 0 50000 bus-msgbox 1 1 0 200000000 50000 0 50000 bus-gpu 1 1 0 200000000 50000 0 50000 bus-de 2 2 0 200000000 50000 0 50000 bus-wb 0 0 0 200000000 50000 0 50000 bus-mixer1 0 0 0 200000000 50000 0 50000 bus-mixer0 1 1 0 200000000 50000 0 50000 bus-hdmi 2 2 0 200000000 50000 0 50000 bus-tve 0 0 0 200000000 50000 0 50000 bus-csi 0 1 0 200000000 50000 0 50000 bus-deinterlace 0 0 0 200000000 50000 0 50000 bus-tcon1 0 0 0 200000000 50000 0 50000 bus-tcon0 1 1 0 200000000 50000 0 50000 bus-ve 0 0 0 200000000 50000 0 50000 bus-ohci0 0 0 0 200000000 50000 0 50000 bus-ehci0 0 0 0 200000000 50000 0 50000 bus-otg 1 1 0 200000000 50000 0 50000 bus-spi1 0 0 0 200000000 50000 0 50000 bus-spi0 0 0 0 200000000 50000 0 50000 bus-hstimer 0 0 0 200000000 50000 0 50000 bus-ts 0 0 0 200000000 50000 0 50000 bus-dram 0 0 0 200000000 50000 0 50000 bus-nand 0 0 0 200000000 50000 0 50000 bus-mmc2 1 1 0 200000000 50000 0 50000 bus-mmc1 1 1 0 200000000 50000 0 50000 bus-mmc0 1 1 0 200000000 50000 0 50000 bus-dma 1 1 0 200000000 50000 0 50000 bus-ce 0 0 0 200000000 50000 0 50000 apb1 2 2 0 100000000 50000 0 50000 bus-i2s2 0 0 0 100000000 50000 0 50000 bus-i2s1 0 0 0 100000000 50000 0 50000 bus-i2s0 0 0 0 100000000 50000 0 50000 bus-ths 1 1 0 100000000 50000 0 50000 bus-pio 1 1 0 100000000 50000 0 50000 bus-spdif 0 0 0 100000000 50000 0 50000 bus-codec 0 0 0 100000000 50000 0 50000 pll-periph0-2x 1 1 0 1200000000 50000 0 50000 mbus 1 1 0 300000000 50000 0 50000 pll-ddr 1 1 0 1248000000 50000 0 50000 dram 1 1 0 1248000000 50000 0 50000 dram-ts 0 0 0 1248000000 50000 0 50000 dram-deinterlace 0 0 0 1248000000 50000 0 50000 dram-csi 0 0 0 1248000000 50000 0 50000 dram-ve 0 0 0 1248000000 50000 0 50000 pll-ve 0 0 0 297000000 50000 0 50000 ve 0 0 0 297000000 50000 0 50000 pll-video 2 2 0 294000000 50000 0 50000 hdmi-phy-clk 1 1 0 73500000 50000 0 50000 hdmi 1 1 0 294000000 50000 0 50000 tcon 0 0 0 294000000 50000 0 50000 pll-audio-base 0 0 0 98285714 50000 0 50000 pll-audio-8x 0 0 0 196571428 50000 0 50000 i2s2 0 0 0 196571428 50000 0 50000 i2s1 0 0 0 196571428 50000 0 50000 i2s0 0 0 0 196571428 50000 0 50000 pll-audio-4x 0 0 0 98285714 50000 0 50000 pll-audio-2x 0 0 0 49142857 50000 0 50000 pll-audio 0 0 0 98285714 50000 0 50000 ac-dig 0 0 0 98285714 50000 0 50000 spdif 0 0 0 98285714 50000 0 50000 pll-cpux 1 1 0 480000000 50000 0 50000 cpux 1 1 0 480000000 50000 0 50000 axi 0 0 0 120000000 50000 0 50000 All of our csi clocks have no statistics even after a 1080p (green/pink) capture. I know that changing the mclk rate in the dtb reached the driver because it complained once it passed the limits, but changing the clock parent doesn't reflect in this output. I don't know what to make of this though, I assume the Linux part it working well, so are we calling/reporting clock changes? On 7/20/2020 at 1:59 PM, mostly said: Since we see these strange clocks behavior, can this custom clock source be not suitable for some modes?... Looking at the BSP code (searching "CSI_MASTER_CLK_24M_SRC" in bsp code), I think as long as the mclk rate is 6/12/24MHz the 24MHz clock is the same one they would configure. I think a clock config problem on the SoC side could be a tidy answer, it would explain why it already works for some with the same SoC. I'll keep looking in this direction for now. 0 Quote
srinath Posted March 11, 2021 Posted March 11, 2021 kernal configs: CONFIG_I2C_GPIO=m CONFIG_V4L_PLATFORM_DRIVERS=y CONFIG_VIDEO_SUN6I_CSI=m CONFIG_V4L_MEM2MEM_DRIVERS=y CONFIG_VIDEO_MEM2MEM_DEINTERLACE=m CONFIG_VIDEO_OV5640=m Hi all, I did changes for kernal configs After that what should I need to change in sun8i-h3-nanopi-duo2.dts or/and add overlay to enable the camera As I am using Nano pi duo 2 with H3 chipset running under armhf(32bit) Armbian Focal (mainline based kernel 5.10.y) attached with OV5640 camera Thanks in advance 0 Quote
rdeyes Posted March 15, 2021 Posted March 15, 2021 Hello! Guys, I need your help! Recently I've been trying for 2 weeks to make ov5640 work on my OrangePi-PC (Armbian 21.05.0-trunk Buster with Linux 5.10.19-sunxi) and yet did not suceed. I'm currently using DeviceTree overlay: /dts-v1/; /plugin/; / { compatible = "allwinner,sun8i-h3"; fragment@0 { target-path = "/"; __overlay__ { /* DOVDD power supply. */ reg_cam_dovdd: cam-dovdd-1v8 { compatible = "regulator-fixed"; regulator-name = "cam-dovdd-1v8"; regulator-min-microvolt = <1800000>; regulator-max-microvolt = <1800000>; enable-active-high; gpio = <&pio 6 13 0>; /* DOVDD: AFCC_EN: PG13, GPIO_ACTIVE_HIGH: 0 */ regulator-boot-on; regulator-always-on; }; /* AVDD power supply. */ reg_cam_avdd: cam-avdd-2v8 { compatible = "regulator-fixed"; regulator-name = "cam-avdd-2v8"; regulator-min-microvolt = <2800000>; regulator-max-microvolt = <2800000>; enable-active-high; gpio = <&pio 0 17 0>; /* AVDD: CSI_PWR_EN: PA17, GPIO_ACTIVE_HIGH: 0 */ regulator-boot-on; regulator-always-on; }; /* DVDD power supply. */ reg_cam_dvdd: cam-dvdd-1v5 { compatible = "regulator-fixed"; regulator-name = "cam-dvdd-1v5"; regulator-min-microvolt = <1500000>; regulator-max-microvolt = <1500000>; enable-active-high; gpio = <&pio 6 11 0>; /* CSI_EN: PG11, GPIO_ACTIVE_HIGH: 0 */ /* startup-delay-us = <5000>; */ /* 5ms delay. */ regulator-boot-on; regulator-always-on; }; }; }; fragment@1 { target = <&pio>; __overlay__ { csi_mclk_pin: csi-mclk-pin { pins = "PE1"; function = "csi"; }; }; }; fragment@2 { target = <&csi>; __overlay__ { status = "okay"; port { #address-cells = <1>; #size-cells = <0>; /* Parallel bus endpoint */ csi_ep: endpoint { remote-endpoint = <&ov5640_ep>; bus-width = <8>; hsync-active = <1>; /* Active high */ vsync-active = <0>; /* Active low */ pclk-sample = <1>; /* Rising */ }; }; }; }; fragment@3 { target = <&i2c2_pins>; __overlay__ { bias-pull-up; }; }; fragment@4 { target = <&i2c2>; __overlay__ { status = "okay"; ov5640: camera@3c { compatible = "ovti,ov5640"; reg = <0x3c>; pinctrl-names = "default"; pinctrl-0 = <&csi_mclk_pin>; clocks = <&ccu 107>; /* CLK_CSI_MCLK: 107 */ clock-names = "xclk"; AVDD-supply = <®_cam_avdd>; DOVDD-supply = <®_cam_dovdd>; DVDD-supply = <®_cam_dvdd>; reset-gpios = <&pio 4 14 1>; /* CSI-RESET#: PE14, GPIO_ACTIVE_LOW: 1 */ powerdown-gpios = <&pio 4 15 0>; /* CSI-STBY-EN: PE15, GPIO_ACTIVE_HIGH: 0 */ port { /* Parallel bus endpoint. */ ov5640_ep: endpoint { remote-endpoint = <&csi_ep>; bus-width = <8>; data-shift = <2>; /* lines 9:2 are used */ hsync-active = <1>; /* Active high */ vsync-active = <0>; /* Active low */ /* data-active = <1>; */ /* Active high */ pclk-sample = <1>; /* Rising */ }; }; }; }; }; }; With this overlay /dev/video1 is created, ov5640 driver loads, no complaints from driver in dmesg whatsoever, camera is powered on and I/O from /dev/video1 somehow hits the camera (I slightly modified drivers/media/i2c/ov5640.c to see in dmesg, that something is being written/read on camera registers). However, I am not able to use the camera (In my C++ application ioctl(VIDIOC_DQBUF, &buffer) function falls into endless loop, yet the same app works perfectly on my laptop with built-in camera). Also command $ v4l2-ctl -d /dev/video1 --list-ctrls shows nothing. If I enable v4l2 debug with # echo 0x1f > /sys/class/video4linux/video1/dev_debug and then try v4l2-ctl command, dmesg is spammed with VIDIOC_QUERYCTRL: error -25 ... . And if I try using my app I have VIDIOC_DQBUF: error -512 in dmesg. I've also tried applying @gsumner's solution with modified .dts file and 'assigned-*' properties within &ccu node and it gave me the same result. So I'm out of ideas, what am I doing wrong and what to do next... Please, help. And thank you in advance. 0 Quote
rreignier Posted March 15, 2021 Author Posted March 15, 2021 Hi @rdeyes On an Orange Pi One (which might be close to your Orange Pi PC), I have managed to get the OV5640 thanks to the amazing work of the contributors of this thread. Last time I have tried was on Armbian 20.08.17 with Linux 5.8.16. The device tree overlay I use is this one (you may have to change some pins) : /dts-v1/; /plugin/; / { compatible = "allwinner,sun8i-h3"; }; &ccu { assigned-clocks = <&ccu 107>; assigned-clock-parents = <&osc24M>; assigned-clock-rates = <24000000>; }; &pio { csi_mclk_pin: csi-mclk-pin { pins = "PE1"; function = "csi"; }; }; &i2c2_pins { bias-pull-up; }; &i2c2 { status = "okay"; ov5640: camera@3c { compatible = "ovti,ov5640"; reg = <0x3c>; pinctrl-names = "default"; pinctrl-0 = <&csi_mclk_pin>; clocks = <&ccu 107>; clock-names = "xclk"; AVDD-supply = <®_vcc_af_csi>; DOVDD-supply = <®_vdd_1v5_csi>; DVDD-supply = <®_vcc_csi>; reset-gpios = <&pio 4 14 1>; /* CSI-RST-R: PE14 */ powerdown-gpios = <&pio 4 15 0>; /* CSI-STBY-R: PE15 */ port { ov5640_ep: endpoint { remote-endpoint = <&csi_ep>; bus-width = <8>; data-shift = <2>; /* lines 9:2 are used */ hsync-active = <1>; /* Active high */ vsync-active = <0>; /* Active low */ data-active = <1>; /* Active high */ pclk-sample = <1>; /* Rising */ }; }; }; }; &csi { status = "okay"; port { #address-cells = <1>; #size-cells = <0>; csi_ep: endpoint { remote-endpoint = <&ov5640_ep>; bus-width = <8>; hsync-active = <1>; /* Active high */ vsync-active = <0>; /* Active low */ data-active = <1>; /* Active high */ pclk-sample = <1>; /* Rising */ }; }; }; To enable it, I use this command: sudo armbian-add-overlay csi-ov5640.dts And then, to take a picture: media-ctl --device /dev/media1 --set-v4l2 '"ov5640 1-003c":0[fmt:JPEG_1X8/1920x1080]' fswebcam -d /dev/video0 -r 1920x1080 -D 0 --jpeg 100 /tmp/test.jpg I hope this can help you and others. 1 Quote
srinath Posted March 16, 2021 Posted March 16, 2021 Hi all, Thank you for the code. I have tried it in nano pi duo 2 with ov5640 in kernal version 5.10.21 armbian focal. can you give us the output for media-ctl --device /dev/media1 --print-topology media-ctl --device /dev/video0 --print-topology I have added the overlay by sudo armbian-add-overlay csi-ov5640.dts and did reboot my device (nano pi duo 2) my output for, root@nanopiduo2:~# ls /dev/ autofs loop3 rtc0 tty24 tty46 ttyS0 vcsa3 block loop4 shm tty25 tty47 ttyS1 vcsa4 btrfs-control loop5 snd tty26 tty48 ttyS2 vcsa5 bus loop6 stderr tty27 tty49 ttyS3 vcsa6 char loop7 stdin tty28 tty5 ttyS4 vcsu console loop-control stdout tty29 tty50 ttyS5 vcsu1 cpu_dma_latency mapper tty tty3 tty51 ttyS6 vcsu2 cuse media0 tty0 tty30 tty52 ttyS7 vcsu3 disk mem tty1 tty31 tty53 ubi_ctrl vcsu4 ecryptfs mmcblk0 tty10 tty32 tty54 uhid vcsu5 fd mmcblk0p1 tty11 tty33 tty55 uinput vcsu6 full mqueue tty12 tty34 tty56 uleds video0 fuse net tty13 tty35 tty57 urandom watchdog gpiochip0 null tty14 tty36 tty58 v4l watchdog0 gpiochip1 ppp tty15 tty37 tty59 vcs zero hwrng ptmx tty16 tty38 tty6 vcs1 zram0 initctl pts tty17 tty39 tty60 vcs2 zram1 input ram0 tty18 tty4 tty61 vcs3 zram2 kmem ram1 tty19 tty40 tty62 vcs4 kmsg ram2 tty2 tty41 tty63 vcs5 log ram3 tty20 tty42 tty7 vcs6 loop0 random tty21 tty43 tty8 vcsa loop1 rfkill tty22 tty44 tty9 vcsa1 loop2 rtc tty23 tty45 ttyGS0 vcsa2 here I have media0 and video0 so, I tried root@nanopiduo2:~# media-ctl --device /dev/media0 --set-v4l2 '"ov5640 1-003c":0[fmt:JPEG_1X8/1920x1080]' Unable to setup formats: Invalid argument (22) root@nanopiduo2:~# fswebcam -d /dev/video0 -r 1920x1080 -D 0 --jpeg 100 /tmp/test.jpg -bash: fswebcam: command not found can you also give me the I2C detect, i2cdetect -y 0 i2cdetect -y 1 i2cdetect -y 2 I found I2C 2 which is connect to camera is not working root@nanopiduo2:~# ls -l /sys/class/i2c-adapter/ total 0 I found my I2C2 is disable how can I enable? root@nanopiduo2:~# fdtget /boot/dtb/sun8i-h3-nanopi-duo2.dtb /soc/i2c@1c2b400 status disabled Thanks in advance 0 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.