Search the Community

Looking at the BSP code and the driver, I think my change to the clock can be ignored now - it's coupled to the framerate, so even if it had been correct I think it would halve the framerate. I think I will start looking at clocking on the SoC, after that I could only guess at electrical design issues(?). @mostly has an H3 and I have an H5 which have issues, but Lex has an H3 and A64 which work, which makes me worry it's an electrical design issue... I think the auto focus patch should still work if the couple of lines changing clocks are removed though. Lex, thanks, I think you don't need to worry about testing the change. I'll make a thread for ov5640 improvements for things like the auto focus after looking at this issue (SoC/board specific corrupted green/pink images) for a bit more. I haven't been concerned with framerate, just looking at what ffmpeg output. The rates I mentioned before were just the configured rate in media-ctl. @mostly No, I've never seen 30fps at 1080p sorry, I was referring to the configured rate. I only get 2 or 3 fps at 1080p but I've realised that's because the clock change is wrong. I agree that it might be time to look at clocking on the SoC side. I've ordered an Orange Pi One with an H3 to see if I can get different behaviour from an H3 on a different board.

Yes, it is. On nanopi neo air with armbian 20.08.0 (5.7.8) it works. FFmpeg with 960x720 input from our camera* encode mp4 by h264_omx with 15-20 fps. If i use cedrus264 as codec, ffmpeg can easy provide 30 fps. With default cam500b i get 60 fps 640x480 (that is maximum of the camera). Fps numbers i get from ffmpeg log and i think this results is accurate enough. * our camera isn't ov5640 default camera and 30 fps likely it's maximum fps. We haven't camera's datasheet.

I will test with and without asap and give some feedback. Perhaps you should open a new thread, something like "Improving ov5640 (mainline)" and push your code/patch there (up to you). How do you test your FPS? Maybe you could get inspired by the BSP code: https://github.com/avafinger/ov5640/blob/A64/ov5640.c#L4577

Tried it and got almost adequate FHD picture with OV5640_PCLK_ROOT_DIV / 2 . If using ffmpeg, the only problem that i see is that there is more green than expected. In the meantime, i brought up h264 hw encoding from uboborov sunxi-cedar-mainline + uvc_h264_lb_allwinner (turned on writing to file and FPS calculation in settings inside main.c). It works well with 1280x720@15fps (real 15fps in h264 file), but with 1920x1080 OV5640_PCLK_ROOT_DIV / 2 it gives some artifacts with colors and adds shadows to dark objects on white background. Not sure yet if this problem is from hw encoder side or ov5640. Were you able to get real 30fps? uvc_h264_lb_allwinner shows actual FPS, and 1280x720 framerate is always 15 and 1920x1080 framerate is always 7 regardless of what set in media-ctl format @ 1/ So a64 does not need that patch, and also does not need "connecting the clock parent in the device tree" in dts. Since we see these strange clocks behavior, can this custom clock source be not suitable for some modes?...

Hi, I think I have 2592x1944 working (with the above workaround) yuv422 and jpeg. If you just want the change (needs to be combined with the device tree change from earlier in the thread if you're starting from scratch) it's below: diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c index 18dd2d717..6c8d00c30 100644 --- a/drivers/media/i2c/ov5640.c +++ b/drivers/media/i2c/ov5640.c @@ -535,7 +535,7 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = { {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0}, {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0}, {0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, - {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0}, + {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x20, 0, 0}, {0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 70}, }; @@ -992,7 +992,7 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor, u8 *pll_rdiv, u8 *bit_div, u8 *pclk_div) { unsigned long _rate = rate * OV5640_PLL_ROOT_DIV * OV5640_BIT_DIV * - OV5640_PCLK_ROOT_DIV; + OV5640_PCLK_ROOT_DIV / OV5640_BIT_DIV; _rate = ov5640_calc_sys_clk(sensor, _rate, pll_prediv, pll_mult, sysdiv); The explanation is that I think the parallel bus calculation (mipi csi on other SoCs unaffected) is missing an extra bit_div which is in the diagram but not the calculation. The 2592x1944 register 0x460c setting 0x22 overrides that extra OV5640_BIT_DIV I've added with the value in register 0x3824, so 0x20 brings it back in line with 720p and 1080p using "auto" calculation. Lower resolutions use the override too but they worked when I tested them so I haven't messed with it. I'm concerned that lex has 1080p working on an a64 without the change though, I thought the fix would be independent of SoC and affect all parallel buses with the ov5640. Lex, could you confirm it was a64 + ov5640 + no driver changes that worked for you? (Assuming the a64 works with my change) We could have something Armbian could apply soon, could you post your Allwinner board+SoC, camera and whether this change works for you? I think we should also make a change to try the framerate before/after the resolution so we don't need the workaround in the previous post. @Igor I think I nearly have a change to get 1080p and 1944p working with ov5640 and parallel bus (as opposed to mipi csi). I haven't done this with Armbian before, how much testing do I need? Do I need to find all boards with an ov5640 and parallel bus? I need to test some more formats/resolutions on my board before I'm confident at least anyway. There's also a separate change for the device tree from earlier in the thread that I think could also be submitted with some more testing. I'd like to help contribute this to Armbian if I can get these tested on some other boards and in the right format for you.

I started having a look at 2592x1944 video, for anyone else trying the same thing and failing with "Unable to setup formats: Invalid argument (22)" from media-ctl, the issue appears to be the framerate/interval. 2592x1944 only works at 15fps but the command (@1/15 is the frame interval, inverse of framerate): media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/2592x1944@1/15]' seems to have its order of operation backwards, it tries 2592x1944 at the current framerate, and sets the new framerate after, which fails. To work around it you can do the following: media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080@1/15]' media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/2592x1944]' 1080p works at 30fps, which is why you can switch to 15fps in the same command, then you can move the resolution to 2592x1944 once already at 15fps. I can't get it to capture any video at 2592x1944 yet, but hopefully this helps anyone else stuck where I was.

@mostly and anyone else interested, I have gotten normal looking yuv422 video at 1920x1080 working (I think). I only have part of the final fix though, because I don't understand yet why the rate is out by two. I thought it might have been dynamic and only broken when sysdiv went to 1x, but I can capture fine looking video at 720p and 480p with this change too. The minimal change is below: diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c index 18dd2d717..342e80d21 100644 --- a/drivers/media/i2c/ov5640.c +++ b/drivers/media/i2c/ov5640.c @@ -992,7 +992,7 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor, u8 *pll_rdiv, u8 *bit_div, u8 *pclk_div) { unsigned long _rate = rate * OV5640_PLL_ROOT_DIV * OV5640_BIT_DIV * - OV5640_PCLK_ROOT_DIV; + OV5640_PCLK_ROOT_DIV / 2; _rate = ov5640_calc_sys_clk(sensor, _rate, pll_prediv, pll_mult, sysdiv); Could someone please confirm that they had the pink/green image problem and that this fixes it at 1080p for them? Tested with: media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080]' ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output.mjpg I've been sticking with yuv422 so far because the sun6i_csi driver mentions not supporting yuv420 and the ov5640 seems to have some 16bit/pixel parts hardcoded. I'd like to understand the clock structure better, but as I said previously I'm struggling to find details.

Hi all. I have a board nanopi neo air with armbian linux on it. We with my colleague try to setting up ffmpeg with cedrus264 for hardware accelerated capturing video from ov5640-like camera device. There is my armbianmonitor. There are my v4l2-ctl -d /dev/videox -D out: After some reconfiguration of dts, we can successfully capture video in low fps with ffmpeg -f v4l2 -framerate 25 -video_size 960x720 -i /dev/video1 output.mkv but with ffmpeg -f v4l2 -channel 0 -video_size 960x720 -i /dev/video1 -pix_fmt nv12 -r 30 -b:v 64k -c:v cedrus264 test.mp4 ffmpeg failed: Ffmpeg was buided from FFmpeg-Cedrus, also we try follow this and use this binaries - result haven't changed. After some inspecting sources of ffmpeg-cedrus we found that ffmpeg try to open /dev/cedar_dev device, which don't exists in our system. Ffmpeg fails on opening device. If we replace /dev/cedar_dev by /dev/video0, where is our cedrus driver, ffmpeg fails on ioctl IOCTL_GET_ENV_INFO. In source, it is 99 or 101 line. We check kernel config for configuration, provided by Sunxi-Cedrus and determine that cedrus was configured right. Also we have tried all described actions on official armbian buster image for nanopi neo air and we have got the same result. What we does wrong?

i think so. I tested your pattern sample on A64 and both pattern looked fine (exactly the same) ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output_yuyv422.mjpg ffmpeg -input_format yuv420p -s 1920x1080 -i /dev/video0 /tmp/output_yuyv420.mjpg which i think should fail for the yuyv422 on fmt:YUYV8_2X8/1920x1080 Image quality is similar to what you got with your test_0. I would advise to move on to latest ov5640 driver or if you find the ov5640 driver (PinePhone) with additional fix , let us know the diff. Maybe @rreignier can have different results. Hope this information will be helpful. Good luck.

Hi Lex, I think I've been talking about two unrelated issues, the greenish image showed up every n images but I only tested indoors. I think I now know that was only when the camera was in jpeg mode. Unfortunately my driver is in a sorry state at the moment for taking new images. This drive link has the old images: https://drive.google.com/drive/folders/1kjn-D02tEXLiDAMPOVFVVTlrnqgGFkro?usp=sharing The other two images represent the issue that mostly and I are talking about, which is when the camera is in yuv mode. When using yuv in 1080p (I'm testing using "v4l2-ctl --set-ctrl test_pattern=2") you get the heavily distorted green/pink images. I currently believe they are because of an issue in the clock calculations I mentioned above. I can get a reasonable looking 1080p test pattern by forcing sysdiv to more than 1 but can't get real images. I'm guessing that means that the SCLK is a reasonable value (the ISP must generate the pattern??) but the PCLK is clearly not good. I wonder if everyone else using the ov5640 driver uses csi instead of parallel... I think if I start looking into the cma alloc issue at the same time my head might explode! I'll have to look after the yuv-1080p-clock issue. Regarding VLC, I'm wondering if the issue is because vlc is opening the camera in yuv420 mode, but in the sun6i_csi driver I can see in "get_csi_input_format" that yuv420 isn't supported? Would that explain the frame size difference? @rreignier Thanks, that looks helpful. So they're using an Allwinner, ov5640 and parallel camera too. I'll have to look to see if they've already solved this.

Hi @gsumner No, I did not have time to try it out on my board yet But I am very glad that you have managed to get this camera working on mainline and shared your progress. I am following with great attention! Thanks! Did you have a look at the blog post about PinePhone camera which happen to be an OV5640 too? https://blog.brixit.nl/camera-on-the-pinephone/

@mostly Thanks, not sure why it wasn't working before, I can build modules now and signing turned out to not be needed. I've been looking at the sun6i_csi, sun6i_video and ov5640 drivers, at the moment I think there is a problem in the clock generation in the ov5640 code. I'll share my test patch: diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c index 18dd2d717..7b5166b11 100644 --- a/drivers/media/i2c/ov5640.c +++ b/drivers/media/i2c/ov5640.c @@ -470,54 +470,106 @@ static const struct reg_value ov5640_setting_PAL_720_576[] = { }; static const struct reg_value ov5640_setting_720P_1280_720[] = { - {0x3c07, 0x07, 0, 0}, - {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0}, - {0x3814, 0x31, 0, 0}, - {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0}, - {0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0}, - {0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0}, - {0x3810, 0x00, 0, 0}, - {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0}, - {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0}, - {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0}, - {0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0}, - {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x72, 0, 0}, {0x3a0e, 0x01, 0, 0}, - {0x3a0d, 0x02, 0, 0}, {0x3a14, 0x02, 0, 0}, {0x3a15, 0xe4, 0, 0}, - {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, - {0x4407, 0x04, 0, 0}, {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0}, - {0x3824, 0x04, 0, 0}, {0x5001, 0x83, 0, 0}, + {0x3612, 0x29, 0, 0}, /* ??? between VCM and timing control */ + {0x3618, 0x00, 0, 0}, /* ??? between VCM and timing control */ + + {0x3708, 0x64, 0, 0}, /* ??? between VCM and timing control */ + {0x3709, 0x52, 0, 0}, /* ??? between VCM and timing control */ + {0x370c, 0x03, 0, 0}, /* ??? between VCM and timing control */ + + {0x3800, 0x00, 0, 0}, /* TIMING HS */ + {0x3801, 0x00, 0, 0}, /* TIMING HS */ + {0x3802, 0x00, 0, 0}, /* TIMING VS */ + {0x3803, 0xfa, 0, 0}, /* TIMING VS */ + {0x3804, 0x0a, 0, 0}, /* TIMING HW */ + {0x3805, 0x3f, 0, 0}, /* TIMING HW */ + {0x3806, 0x06, 0, 0}, /* TIMING VH */ + {0x3807, 0xa9, 0, 0}, /* TIMING VH */ + {0x3810, 0x00, 0, 0}, /* TIMING HOFFSET */ + {0x3811, 0x10, 0, 0}, /* TIMING_HOFFSET */ + {0x3812, 0x00, 0, 0}, /* TIMING VOFFSET */ + {0x3813, 0x04, 0, 0}, /* TIMING VOFFSET */ + {0x3814, 0x31, 0, 0}, /* TIMING X INC */ + {0x3815, 0x31, 0, 0}, /* TIMING Y INC */ + {0x3824, 0x04, 0, 0}, /* DVP PCLK divider for VFIFO CTRL0C */ + + {0x3a02, 0x02, 0, 0}, /* AEC MAX EXPO (60HZ) */ + {0x3a03, 0xe4, 0, 0}, /* AEC MAX EXPO (60HZ) */ + {0x3a08, 0x01, 0, 0}, /* AEC B50 STEP */ + {0x3a09, 0xbc, 0, 0}, /* AEC B50 STEP */ + {0x3a0a, 0x01, 0, 0}, /* AEC B60 STEP */ + {0x3a0b, 0x72, 0, 0}, /* AEC B60 STEP */ + {0x3a0e, 0x01, 0, 0}, /* AEC CTRL0E */ + {0x3a0d, 0x02, 0, 0}, /* AEC CTRL0D */ + {0x3a14, 0x02, 0, 0}, /* AEC MAX EXPO (50HZ) */ + {0x3a15, 0xe4, 0, 0}, /* AEC MAX EXPO (50HZ) */ + + {0x3c07, 0x07, 0, 0}, /* LIGHT METER1 THRESHOLD */ + {0x3c09, 0x1c, 0, 0}, /* LIGHT METER2 THRESHOLD */ + {0x3c0a, 0x9c, 0, 0}, /* SAMPLE NUMBER */ + {0x3c0b, 0x40, 0, 0}, /* SAMPLE NUMBER */ + + {0x4001, 0x02, 0, 0}, /* BLC CTRL01 */ + {0x4004, 0x02, 0, 0}, /* BLC CTRL04 */ + + {0x4407, 0x04, 0, 0}, /* JPEG CTRL07 */ + + {0x460b, 0x37, 0, 0}, /* DEBUG MODE */ + {0x460c, 0x20, 0, 0}, /* VFIFO CTRL0C */ + + {0x5001, 0x83, 0, 0}, /* ISP CONTROL 01 */ }; static const struct reg_value ov5640_setting_1080P_1920_1080[] = { - {0x3008, 0x42, 0, 0}, - {0x3c07, 0x08, 0, 0}, - {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0}, - {0x3814, 0x11, 0, 0}, - {0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0}, - {0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0}, - {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0}, - {0x3810, 0x00, 0, 0}, - {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0}, - {0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0}, - {0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0}, - {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0}, - {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0}, - {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0}, - {0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, - {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0}, - {0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0}, - {0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0}, - {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0}, - {0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0}, - {0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0}, - {0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0}, - {0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0}, - {0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0}, - {0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0}, - {0x3a0e, 0x03, 0, 0}, {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x04, 0, 0}, - {0x3a15, 0x60, 0, 0}, {0x4407, 0x04, 0, 0}, - {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0}, {0x3824, 0x04, 0, 0}, - {0x4005, 0x1a, 0, 0}, {0x3008, 0x02, 0, 0}, + {0x3612, 0x2b, 0, 0}, /* ??? between VCM and timing control */ + {0x3618, 0x04, 0, 0}, /* ??? between VCM and timing control */ + {0x3708, 0x64, 0, 0}, /* ??? between VCM and timing control */ + {0x3709, 0x12, 0, 0}, /* ??? between VCM and timing control */ + {0x370c, 0x00, 0, 0}, /* ??? between VCM and timing control */ + + {0x3800, 0x01, 0, 0}, /* TIMING HS */ + {0x3801, 0x50, 0, 0}, /* TIMING HS */ + {0x3802, 0x01, 0, 0}, /* TIMING VS */ + {0x3803, 0xb2, 0, 0}, /* TIMING VS */ + {0x3804, 0x08, 0, 0}, /* TIMING HW */ + {0x3805, 0xef, 0, 0}, /* TIMING HW */ + {0x3806, 0x05, 0, 0}, /* TIMING VH */ + {0x3807, 0xf1, 0, 0}, /* TIMING VH */ + {0x3810, 0x00, 0, 0}, /* TIMING HOFFSET */ + {0x3811, 0x10, 0, 0}, /* TIMING HOFFSET */ + {0x3812, 0x00, 0, 0}, /* TIMING VOFFSET */ + {0x3813, 0x04, 0, 0}, /* TIMING VOFFSET */ + {0x3814, 0x11, 0, 0}, /* TIMING X INC */ + {0x3815, 0x11, 0, 0}, /* TIMING Y INC */ + {0x3824, 0x04, 0, 0}, /* DVP PCLK divider for VFIFO CTRL0C */ + + {0x3a02, 0x04, 0, 0}, /* AEC MAX EXPO (60HZ) */ + {0x3a03, 0x60, 0, 0}, /* AEC MAX EXPO (60HZ) */ + {0x3a08, 0x01, 0, 0}, /* AEC B50 STEP */ + {0x3a09, 0x50, 0, 0}, /* AEC B50 STEP */ + {0x3a0a, 0x01, 0, 0}, /* AEC B60 STEP */ + {0x3a0b, 0x18, 0, 0}, /* AEC B60 STEP */ + {0x3a0d, 0x04, 0, 0}, /* AEC CTRL0D */ + {0x3a0e, 0x03, 0, 0}, /* AEC CTRL0E */ + {0x3a14, 0x04, 0, 0}, /* AEC MAX EXPO (50HZ) */ + {0x3a15, 0x60, 0, 0}, /* AEC MAX EXPO (50HZ) */ + + {0x3c07, 0x07, 0, 0}, /* LIGHT METER1 THRESHOLD */ + {0x3c08, 0x00, 0, 0}, /* LIGHT METER2 THRESHOLD */ + {0x3c09, 0x1c, 0, 0}, /* LIGHT METER2 THRESHOLD */ + {0x3c0a, 0x9c, 0, 0}, /* SAMPLE NUMBER */ + {0x3c0b, 0x40, 0, 0}, /* SAMPLE NUMBER */ + + {0x4001, 0x02, 0, 0}, /* BLC CTRL01 */ + {0x4004, 0x06, 0, 0}, /* BLC CTRL04 */ + {0x4005, 0x1a, 0, 0}, /* BLC CTRL05 */ + + {0x4407, 0x04, 0, 0}, /* JPEG CTRL07 */ + + {0x460b, 0x37, 0, 0}, /* DEBUG MODE */ + {0x460c, 0x20, 0, 0}, /* VFIFO CTRL0C */ + + {0x5001, 0x83, 0, 0}, /* ISP CONTROL 01 */ }; static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = { @@ -787,7 +839,7 @@ static int ov5640_mod_reg(struct ov5640_dev *sensor, u16 reg, * This is supposed to be ranging from 1 to 16, but the value is * always set to either 1 or 2 in the vendor kernels. */ -#define OV5640_SYSDIV_MIN 1 +#define OV5640_SYSDIV_MIN 2 #define OV5640_SYSDIV_MAX 16 /* @@ -848,7 +900,7 @@ static unsigned long ov5640_calc_sys_clk(struct ov5640_dev *sensor, u8 *sysdiv) { unsigned long best = ~0; - u8 best_sysdiv = 1, best_mult = 1; + u8 best_sysdiv = OV5640_SYSDIV_MIN, best_mult = OV5640_PLL_MULT_MIN; u8 _sysdiv, _pll_mult; for (_sysdiv = OV5640_SYSDIV_MIN; @@ -874,7 +926,7 @@ static unsigned long ov5640_calc_sys_clk(struct ov5640_dev *sensor, * We have reached the maximum allowed PLL1 output, * increase sysdiv. */ - if (!rate) + if (!_rate) break; /* @@ -993,14 +1045,19 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor, { unsigned long _rate = rate * OV5640_PLL_ROOT_DIV * OV5640_BIT_DIV * OV5640_PCLK_ROOT_DIV; + unsigned long pclk; _rate = ov5640_calc_sys_clk(sensor, _rate, pll_prediv, pll_mult, sysdiv); *pll_rdiv = OV5640_PLL_ROOT_DIV; *bit_div = OV5640_BIT_DIV; *pclk_div = OV5640_PCLK_ROOT_DIV; + pclk = _rate / *pll_rdiv / *bit_div / *pclk_div; - return _rate / *pll_rdiv / *bit_div / *pclk_div; + dev_warn(&sensor->i2c_client->dev, "ov5640_calc_pclk sysclk: %lu pclk: %lu\n", + _rate, pclk); + + return pclk; } static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor, unsigned long rate) @@ -1014,6 +1071,10 @@ static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor, unsigned long rate) if (bit_div == 2) bit_div = 8; + dev_warn(&sensor->i2c_client->dev, "ov5640_set_dvp_pclk bit_div: %u" \ + ", sysdiv: %u, mult: %u, prediv: %u, pll_rdiv: %u, pclk_div: %u\n", + bit_div, sysdiv, mult, prediv, pll_rdiv, pclk_div); + ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL0, 0x0f, bit_div); if (ret) @@ -1078,6 +1139,10 @@ static int ov5640_set_timings(struct ov5640_dev *sensor, return ret; } + dev_warn(&sensor->i2c_client->dev, + "ov5640_set_timings setting hact: %u, vact: %u, htot: %u, vtot: %u", + mode->hact, mode->vact, mode->htot, mode->vtot); + ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, mode->hact); if (ret < 0) return ret; The 720p registers are just sorted, but the 1080p ones had a lot of repeated addresses. I've assumed they aren't stateful and only kept the last instance and then sorted them. I also removed a few values that were mentioned in mainline commits. It think it's functionally the same as before but has helped debugging and might help someone else. I don't think they're the current issue anyway. With the patch (minus the change to OV5640_SYSDIV_MIN) and 720p you'll see in dmesg: ov5640 2-003c: ov5640_set_dvp_pclk bit_div: 8, sysdiv: 2, mult: 85, prediv: 3, pll_rdiv: 2, pclk_div: 1 But with 1080p sysdiv notably jumps to 1: ov5640 2-003c: ov5640_set_dvp_pclk bit_div: 8, sysdiv: 1, mult: 84, prediv: 3, pll_rdiv: 2, pclk_div: 1 If you add the change to OV5640_SYSDIV_MIN and test using the OV5640's pattern generator: v4l2-ctl --set-ctrl test_pattern=2 media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:YUYV8_2X8/1920x1080]' ffmpeg -input_format yuyv422 -s 1920x1080 -i /dev/video0 /tmp/output.mjpg You should get a perfect 1080p test pattern. I can't get anything but a black screen and lines if I turn the pattern off and try to capture a real image, but if you compare the pattern to the pattern without my change I think you'll agree there's a clocking problem. The documentation on the clock structure in the code is the only useful source I've found unfortunately. There is also a kernel warning in dmesg with 1080p regarding "dma_alloc_coherent of size 4149248 failed" but I think(?) it's a red herring - it's asking for 32 buffers and it looks like only the 32nd one fails. The capture continues without it. Hopefully this helps get us closer to working 1080p! @rreignier Did you get your board working? I guess we did end up hijacking your thread anyway sorry.

I'm not experiencing that, all still image captures are ok, both at 720p and 1080p. But problem with green 1080p video capture still here. Somebody was trying this register but it seems to be not the right one: https://patchwork.kernel.org/patch/10342907/ There is one interesting commit claiming adding autofocus: https://github.com/D3Engineering/linux_kernel_qcomlt/blob/7b3c6e86d236db6d0c523dbd3297deb7563d7bf1/drivers/media/i2c/ov5640.c but it's for legacy This way of building modules works for me: https://yoursunny.com/t/2018/one-kernel-module/ + touch .scmversion

I think I understand that media-ctl is setting the resolution/format (or group of formats) that will be able to be successfully opened by a program. I'm confused by the media-ctl, v4l2-ctl and program order of operations though, it seems like you have to run your program first, see it fail, run v4l2-ctl -V to see the (last?) format it tried to open, then you go back and set something compatible with media-ctl and try again... is that correct? Surely it should see what the program is asking for and reconfigure itself? With fswebcam (which seems to want JPEG format), running: media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:JPEG_1X8/1920x1080]' fswebcam -d /dev/video0 -r 1920x1080 -D 0 --jpeg 100 /tmp/test.jpg there's a weird issue where 1 in 3 or 4 images has a green tinge. All the detail is there, just greenish. Does anyone have the same issue? I'm also having an issue with vlc, it seems to want planar yuv (YUYV8_2X8), but I can't get a successful stream out of it. cvlc complains "rawvideo decoder warning: invalid frame size (3110400 < 4665600)", which agrees with the "Size Image" field of v4l2-ctl -V. I wonder if the frame size error is related to the green/purple image problem or separate, 3110400 means 12bit/pixel while 4665600 is 18bit/pixel. The format and v4l2-ctl -V agree on 3110400 so I wonder where 4665600 comes from... and I also don't understand how to differentiate yuv422 and yuv420 from media-ctl --known-mbus-fmts. @mostly, ISP CONTROL 03 "Draw window for AFC enable" looks interesting for autofocus. I might try enabling that to see what happens. I wonder if the autofocus is a oneshot thing that the driver would need to fire off every time or constantly focusing. I'm also having the same issues with YUV green/purple images that you mention, if you run ffmpeg with "-input_format mjpeg" you can at least get back to the "green tinge" problem. I tried signing the driver I built previously with the keys I found in the Linux source package but I keep getting exec format error, I was hoping to avoid rebuilding the whole kernel. Are the signing keys available? Maybe that defeats the purpose of keys though! I guess if swapping between mjpeg and yuv is only reconfiguring the soc and not the camera module itself then we should look at sun6i_csi for these format and frame size issues(?).

Yes noise is annoying. Maybe "5.6 color interpolation (CIP)" mentioned in ov5640 datasheet can deal with it (CIP seems to be already enabled in driver initialization, maybe tuning of CIP parameters can help). Also not sure what to do with autofocus. The driver also has nothing for it, but i have not found in the datasheet how to enable it. One problem found with 1920x1080 resolution: still image capture with fswebcam works but attempt to get 1080p video with ffmpeg gives noisy greenish picture where only contours of objects can be recognized, exactly like in that message: 1280x720 is fine.

@mostly awesome, thanks! I tried a couple others for anyone else wondering. e.g.: media-ctl --device /dev/media0 --set-v4l2 '"ov5640 2-003c":0[fmt:JPEG_1X8/1920x1080]' v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=JPEG fswebcam -d /dev/video0 -r 1920x1080 -D 0 --jpeg 100 /tmp/test.jpg The colour balance and noise are terrible, but it's a start! I see there's something in media-ctl to change colourspace, but it's the first time I've heard of media-ctl and I've got no idea what I'm doing yet. Still no luck with 2592x1944 which I'd like to get going though. @jgauthier I'm not sure you need a bitbang i2c, the schematic for your board (https://drive.google.com/drive/u/0/folders/0B4PAo2nW2KfnflVqbjJGTFlFTTd1b1o1OUxDNk5ackVDM0RNUjBpZ0FQU19SbDk1MngzZWM?tid=0B4PAo2nW2Kfndjh6SW9MS2xKSWs) says they connect to TWI2. If the TWIx == I2Cx thing holds, I2C2 might work for you(?). It looks like CSI-PWR-EN controls all the power to your camera too, that net might be strongly pulled-up next to U9 too. So maybe try enable PD14 for your regulator and I2C2 for the bus.

Thank you! That's the part that i was missing. Now i'm able to run ov5640 on OrangePiPcPlus with kernel 5.4.45. My dts: https://pastebin.com/gdQnnrxQ /dev/video1 appears after reboot (no manual modprobe ov5640 needed). ov5640 for some reason does not leave any footprint in dmesg. I've kept compatible = "allwinner,sun6i-a31-i2c" in i2c2@1c2b400 to be able to check chip id by i2cdetect. And nothing was working without compatible = "allwinner,sun8i-h3-csi" and other stuff inside csi@1cb0000. Would be nice to pull the solution into official build (maybe as overlay) . Ov5640 module is one of very few official orangepi accessories according to https://linux-sunxi.org/CSI media-ctl --device /dev/media1 --set-v4l2 '"ov5640 1-003c":0[fmt:UYVY/1920x1080]' 1920x1080 and 1280x760 are working well for me fswebcam -d /dev/video1 -r 1920x1080 -D 3 --jpeg 100 /mnt/sd/aaa.jpg --- Opening /dev/video1... Trying source module v4l2... /dev/video1 opened. No input was specified, using the first. Delaying 3 seconds. --- Capturing frame... Captured frame in 0.00 seconds. --- Processing captured image... Setting output format to JPEG, quality 100 Writing JPEG image to '/mnt/sd/aaa.jpg'.

@MikePooh, I think there's still some schematic-reading involved at the moment, if you're willing to do that though it should be do-able. I don't know anything about other boards yet so this is the process followed. I think for "it just works" you'll need to wait a bit more. I'm assuming a lot here, e.g. active high and fixed-voltage gpio-switched regulators, so there is risk in following this. The first step is getting i2c working, the driver won't be able to do anything until you can communicate with the camera. I'll use the h5 in my Orange Pi Zero Plus2 as an example but I believe the process should be identical for h3 boards. Looking at the schematic (https://linux-sunxi.org/images/f/f6/ORANGE_PI-ZERO-PLUS2_V1_0.pdf), on page 9 is the csi connector. We want AFCC_EN, CSI_EN and CSI-PWR-EN to be '1' to power the camera. They might be named differently on different boards, but generally you're looking for the ones that aren't connected to pins (on page 6, chip U1D) with a CSI_* function (e.g. PE9/CSI_D5/TS_D5). I'm looking for AFCC_EN and CSI-PWR-EN which link to page 6. So AFCC_EN comes from pin PA7/SIM_CLK/PA_EINT7 and CSI-PWR-EN from PA8/SIM_DATA/PA_EINT8. We need to remember those PXX numbers. rreignier said their camera was on the i2c1 bus, so we need to confirm that too. Following CSI-SCK and CSI-SDA from page 9 to page 6 reveals the i2c interface. CSI-SCK for me connects to "PE12/CSI_SCK/TWI2_SCK" and CSI-SDA to "PE13/CSI_SDA/TWI2_SDA". Maybe someone can confirm, but I think TWIx_SDA equals I2Cx. You can test by enabling the i2c bus overlays one after the other and running i2cdetect while measuring with a multimeter on low range (200mv) A/C voltage between ground and the SCK pin on the camera connector - the correct bus will generate a voltage for a short while. We need to remember that bus number. Note the reset-gpios and powerdown-gpios the same way. Next is the change to the device tree. In Armbian checkout the linux source with "apt install linux-source-<kernel version>-current-sunxi" (or linux-source-<kernel version>-current-sunxi64 for h5) which will leave you with a tar in /usr/src of your kernel source. extract it (do it in a new folder or it will trash your current one) and then copy your kernel config in (cp /boot/config-* ./.config). Edit your .dts (./arch/arm/boot/dts/ for h3, ./arch/arm64/boot/dts/allwinner/ for h5) in line with my previous comment, swapping in your values for regulators, i2c bus and reset/powerdown. for a quick bodge, which pin goes in which regulator doesn't actually matter - my change just forced them all on anyway. As lex said, there's supposed to be an order, my change doesn't respect that order but it has worked so far. Then run "make dtbs" from the top level of the kernel source. That should produce a .dtb file in the same folder as the .dts. backup your current dtb and copy the new one to /boot/dtb/allwinner/*.dtb (that's the h5 path, not sure of the h3 path sorry, might be /boot/dtb/). I think you can alternatively specify this as fdtfile in the armbianEnv.txt. The armbianEnv.txt way will probably survive an update better. Finally, add a line to /etc/modules that says ov5640. Reboot and you might be good to go. If you don't have a /dev/video0 device we'll have to debug. Obviously if I've got something wrong feel free to chip in. Has anyone had success with resolutions other than 640x480? The driver seems to have some code for it, but I haven't gotten anything to work.

Could you please share the full step-by-step how-to enable the OV5640 camera on mainline kernel on H3 devices. Including CSI, drivers etc. Thanks in advance.

@rreignier, I've just gotten an ov5640 going with my Orange Pi Zero Plus2 at 640x480 on mainline, Buster 5.4.43 - maybe the following applies to you too. I had bus locked errors until I powered up the camera, on the Zero it was PA8 and PA7 for autofocus (which I turned on just in case). Looking at the One schematic (https://linux-sunxi.org/images/7/7e/ORANGE_PI-ONE-V1_1.pdf) it looks like it's PA17, PG11 and PG13 for autofocus. After that I still wasn't getting an answer on i2c2, the ov5640 datasheet (https://cdn.sparkfun.com/datasheets/Sensors/LightImaging/OV5640_datasheet.pdf) says XVCLK (CLK_CSI_MCLK in the device tree) needs to be running before accessing registers. I couldn't see an a/c voltage on the csi-mclk pin so figured that was the issue. The fix for me was connecting the clock parent in the device tree. I'll post my dtb changes. I'm not sure this is all correct, but it lets me open the camera without having to mess with any gpios etc after boot. Inside the "model = "OrangePi Zero Plus2";" braces: reg_vdd_1v5_csi: vdd-1v5-csi { compatible = "regulator-fixed"; regulator-name = "vdd1v5-csi"; regulator-min-microvolt = <1500000>; regulator-max-microvolt = <1500000>; gpio = <&pio 0 8 GPIO_ACTIVE_HIGH>; /* PA8 */ enable-active-high; regulator-boot-on; regulator-always-on; }; reg_vcc_csi: vcc-csi { compatible = "regulator-fixed"; regulator-name = "vcc-csi"; regulator-min-microvolt = <2800000>; regulator-max-microvolt = <2800000>; gpio = <&pio 0 8 GPIO_ACTIVE_HIGH>; /* PA8 */ enable-active-high; regulator-boot-on; regulator-always-on; }; reg_vcc_af_csi: vcc-af-csi { compatible = "regulator-fixed"; regulator-name = "vcc-af-csi"; regulator-min-microvolt = <2800000>; regulator-max-microvolt = <2800000>; gpio = <&pio 0 7 GPIO_ACTIVE_HIGH>; /* PA7 */ enable-active-high; regulator-boot-on; regulator-always-on; }; At top level: &ccu { /* Use a stable clock source with known fixed rate for MCLK */ assigned-clocks = <&ccu CLK_CSI_MCLK>; assigned-clock-parents = <&osc24M>; assigned-clock-rates = <24000000>; }; &csi { status = "okay"; port { #address-cells = <1>; #size-cells = <0>; /* Parallel bus endpoint */ csi_ep: endpoint { remote-endpoint = <&ov5640_ep>; bus-width = <8>; hsync-active = <1>; /* Active high */ vsync-active = <0>; /* Active low */ data-active = <1>; /* Active high */ pclk-sample = <1>; /* Rising */ }; }; }; &i2c2_pins { bias-pull-up; }; &i2c2 { status = "okay"; ov5640: camera@3c { compatible = "ovti,ov5640"; reg = <0x3c>; pinctrl-names = "default"; pinctrl-0 = <&csi_mclk_pin>; clocks = <&ccu CLK_CSI_MCLK>; clock-names = "xclk"; AVDD-supply = <&reg_vcc_af_csi>; DOVDD-supply = <&reg_vdd_1v5_csi>; DVDD-supply = <&reg_vcc_csi>; reset-gpios = <&pio 4 14 GPIO_ACTIVE_LOW>; /* CSI-RST-R: PE14 */ powerdown-gpios = <&pio 4 15 GPIO_ACTIVE_HIGH>; /* CSI-STBY-R: PE15 */ port { ov5640_ep: endpoint { remote-endpoint = <&csi_ep>; bus-width = <8>; data-shift = <2>; /* lines 9:2 are used */ hsync-active = <1>; /* Active high */ vsync-active = <0>; /* Active low */ data-active = <1>; /* Active high */ pclk-sample = <1>; /* Rising */ }; }; }; }; I also added csi_mclk_pin to sunxi-h3-h5.dtsi (after "csi_pins: csi-pins {") but I suspect you could reference the pin directly in the dtb and skip this: /omit-if-no-ref/ csi_mclk_pin: csi-mclk-pin { pins = "PE1"; function = "csi"; }; I'm currently having issues with only being able to use 640x480, vlc reporting incorrect frame size and crashes when streaming from motion, but I don't want to derail your thread unless you have the same issues. Hope that helps, Greg

Well, I tried the i2c overlay, along with implementing the dts posted above. This is an H2+ but is pin-for-pin identical to the H3. I verified the schematics are the same between the two board. I still have nothing when i load the ov5640 driver. I'm not sure what else to try.

Yes, I have been trying on an Orange Pi One. I also have Orange Pi PC Plus and Orange Pi Lite2 (H6) on which I could also try. But since I have received the ov5640 board, I did not try the driver, only the i2cdetect.

Interesting, indeed. But avafinger does not appear to be a contributor of the mainline driver : here. I think he has worked the old one, for kernel 3.4. Interestingly, the PineTab device tree uses an ov5640 here. On a A64 that uses the same sun6i_csi driver that the H3. So I am sure there is still hope. What device tree are you using? Are you using the device tree overlay I have posted above?

OV5640 is ready and works in mainline kernel. /dev/video0 will be created if the driver can find the sensor using the i2c line (generally speaking). You can check the schematic of your board if the sensor is attached to i2c. Compare it to NanoPi Neo Air schematic and you will see the nuances.

Gotcha. The way I understand it, sun4i_csi should create /dev/video. It doesn't. Doing some kernel level debugging I discovered that the probe() function is never called when the module loads. I learned that probe() is only called if there is hardware to match the name of the device. https://stackoverflow.com/questions/16259580/probe-in-linux-devfreq-driver-not-being-called Furthermore, I found a thread from past January from the author of the OV5640 code that discusses it not working in 5. kernels yet. https://github.com/avafinger/bananapi-zero-ubuntu-base-minimal/issues/17 I have no idea what it takes, but if he says it's not ready for the mainline, I am taking that as the authoritative source.