[Kevincoooool]

Problem

The OV3640 driver was derived from the OV3660 one, so set_framesize() changes resolution by writing the 0x3800..0x3815 windowing block and the OV3660 PLL block. The OV3640 has none of those registers, so the DVP output never moved away from the QVGA 320x240 that sensor_default_regs sets up. The host sizes its frame buffer from the requested framesize, so any other size - 96x96, 240x240, VGA - gets mismatched line widths and comes out smeared, together with cam_hal: FB-OVF spam.

QVGA only worked by coincidence: it is exactly the size the init table is tuned for.

Fix 1: output size chain

Program the size chain the datasheet documents (4.2 / 5.7 / table 7-1):

crop HS/VS/HW/VH (0x3020..0x3027)
  -> zoom out SIZE_IN (0x335F..0x3361) / SIZE_OUT (0x3362..0x3364)
    -> ISP output (0x3088..0x308B), which is what the DVP port sends

• Scale enable is 0x3302[5] on this part (0x5001, used by the OV3660 code path, does not exist here).
• The crop is centred inside the readout window, which is read back from the sensor first, so the configured window is never exceeded. The datasheet default printed for 0x3024 (0x1800 = 6144) is not what the part actually reports (1787 on my module), hence the read-back.
• SIZE_OUT stays slightly above the requested output, as the application notes recommend, but strictly below SIZE_IN, which 5.7 requires.

Fix 2: crop aspect ratio

When the readout window is narrower or shorter than the requested aspect ratio, set_window() clamped only that dimension and the zoom block stretched the image to the output size. A 4:3 request in a 1787-wide window produced a 1787x1536 crop (ratio 1.163) that came out horizontally stretched at 320x240.

Keep the ratio by shrinking the other dimension as well, so the 4:3 request becomes 1787x1340:

if ((uint32_t)crop_w * aspect_h > (uint32_t)crop_h * aspect_w) {
    crop_w = (uint32_t)crop_h * aspect_w / aspect_h;
} else if ((uint32_t)crop_w * aspect_h < (uint32_t)crop_h * aspect_w) {
    crop_h = (uint32_t)crop_w * aspect_h / aspect_w;
}

Square requests (96x96) are unaffected because both dimensions clamp equally.

Fix 3: clock registers

The old non-JPEG path called the inherited set_pll(), which writes 0x303a..0x303d, 0x3824 and 0x460c. On this part 0x303A..0x303D are the AEC metering window (AVS[15:8], AVS[7:0], AHW[11:8], AHW[7:0]), so every frame size change silently rewrote auto exposure (the window width went from 512 to 259 pixels), and 0x3824 / 0x460c do not exist at all. The frame rate never changed, so the whole block only did harm.

The real clock is 0x300e (Rx_PLL, PLLDiv = 64 - Rx_PLL), 0x300f (FreqDiv / Bit8Div / bypass / InDiv) and 0x3010 (DvpDiv / LaneDiv / SensorDiv / ScaleDiv). Program those with the vendor defaults instead.

Worth noting for anyone trying to raise the frame rate: the frame time is HTS (0x3028) x VTS (0x302a) / core clock and does not depend on the output size, because the full QXGA array is always read out. Measured on a 20MHz XCLK board, frame rate ~= 0.414 x PLLDiv:

PLLDiv	Rx_PLL	core clock	frame rate
7	0x39	10.8 MHz	2.9 fps
14	0x32	21.6 MHz	5.8 fps
26	0x26	40.0 MHz	10.7 fps
32	0x20	49.3 MHz	13.2 fps

Raising PLLDiv also raises the DVP clock, and above ~25 MHz some hosts start dropping pixels, so the vendor default is kept here. 0x3012[6:4] (XGA 2x2 subsampling) would halve the readout instead of the clock, but on this part it only changes the readout window and the Bayer phase and needs the full vendor XGA init sequence, so it is pinned to QXGA mode.

Testing

ESP32-S3 with a Korvo OV3640 module, PIXFORMAT_RGB565, fb_location = CAMERA_FB_IN_PSRAM, fb_count = 2.

96x96 verified on hardware:

ov3640: Crop 410,11 1536x1536 (readout 285,10 1787x1538) -> zoom 104x100 -> DVP 96x96

Before fix 1 the same build produced a 2x-magnified corner of the scene; after it the full square field of view is shown, no FB-OVF.

QVGA 320x240 after fix 2:

ov3640: Crop 285,109 1787x1340 (readout 285,10 1787x1538) -> zoom 328x244 -> DVP 320x240

1340 / 1787 = 0.75, i.e. 4:3, and the preview fills the panel without stretching. Measured 5.8 fps at the default clock, with auto exposure now metering over the correct window.