Raspberry Pi Camera module 3 noir autofocus not working well under IR light

[m4mmon]

Hi,
While using the Raspberry Pi Camera module 3 with rpicam-hello, the camera autofocus works fine with visible light.
As long as the scene is in the dark with 850 nm light source, the autofocus struggles, pumps and does not get proper fixes.

The IR light source is more than enough bright, tested with two different IP cameras with their own IR LEDs off (a Reolink E1 Zoom, and a Eufy E220 running thingino), a modified Logitech C922 (had to "screw" a little the lens to get AF working nice under IR), a modified Canon DSLR 700D. So that's not the problem.
The autofocus works perfectly fine with visible light.
I tried tuning the json settings with the help from members in the rpi forum here, with no success, except showing there is a problem with PDAF. CDAF gives better results, but only at cam initialization, so no AF.

There I learnt that the autofocus was actually not exactly a hardware feature, but something working along the software. That's why I am opening this issue.

As of today, I did a test on a Pi 4 8 GB, running a fully updated bookworm:
uname -a
Linux pi4n001 6.12.25+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.12.25-1+rpt1 (2025-04-30) aarch64 GNU/Linux

cat /etc/rpi-issue
Raspberry Pi reference 2024-11-19
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 891df1e21ed2b6099a2e6a13e26c91dea44b34d4, stage4

At first, I was running libcamera provided with the OS and updated through apt.
Now I am running a build following these instructions, no new commits since I followed that guide.

[naushir]

I believe @njhollinghurst is investigating this.

[njhollinghurst]

Thanks for the details. Meanwhile I've been doing some experiments, also with an 850nm light.

PDAF does seem to behave badly (or at least, very differently) under IR light. Probably, diffraction effects cause the shielded pixels to have a different "beam pattern" at 850nm than for green light. I don't think software can entirely fix it but we should at least be able to detect when it's misbehaving.

I hope to re-write the algorithm to do this, have better hand-off between PDAF and CDAF, allow Continuous AF to use CDAF, and tidy up some of the accumulated cruft.

(Though if you come up with any improvements sooner, I'd be happy to consider a PR on this repo before forwarding it to libcamera.)

[njhollinghurst]

Some experimental changes, nowhere near ready for PR, are in this branch. It tries to detect IR lighting and falls back to CDAF (which is slower) in that case or when PDAF is low-confidence. The 're-trigger' condition needs improving as it can sometimes get stuck.

The new code sits alongside the old. To test it, you need to edit the tuning file to change "rpi.af" to "rpi.new.af".

[m4mmon]

Wow :)
I just came back to the issue and read your messages.
First thing, thank you for your replies.

Second thing, I just built the af_proto branch, did the changes in the tuning file, and tada! Thank you again, it is working.
I haven't used it for long with yet, so I did not met the conditions that get the algorithm stuck.
My tests were done from very close distance 15 cm to 1 m, and up to 3 meters, and really it appears good enough to me.

I really appreciate your help, and wish I could do anything to help back.
Thanks again :)

PS: for people reading this, and wanting to try it, the instructions to build the repository in the RPi documentation are very easy to follow, you just have to replace the libcamera clone url with the one from the link in njhollinghurst's previous message.
Just take care to change the branch after cloning the repo, a 'git checkout -b remotes/origin/af_proto origin/af_proto' is all it takes.
The traces can also be activated with LIBCAMERA_LOG_LEVELS=RPiNewAf:0 .

[njhollinghurst]

You're welcome, and thanks for testing! Please let me know if you have any more test results, good or bad.

Let's keep this issue open until there is a PR.

[m4mmon]

Sure!

[m4mmon]

@njhollinghurst
Just to let you know, in my somehow limited tests, I didn't encounter any issues. Sometimes it feels slower or struggling a little to find a fix, compared to the official version in visible light. But I haven't seen it stuck.
So I guess it is not perfect, but works in my experience.

PS: I just figured out I was already exchanging with you on the rpi forum 🤦

[njhollinghurst]

This should get fixed (amongst other issues) here: #270

The latest version uses rpi.af again and doesn't require any tuning file hacks, but it does need an updated tuning file with "check_for_ir": true for the NoIR cameras.