RL does not use transpose when calculating correct

[bnorthan]

Hi Marvin

Great talking to you last night. As I mentioned I was wondering if the RL code takes the transpose of the forward model when calculating the correction here

You may have already read this, but a good explanation of how 'transpose' is used in multiview deconvolution is here. The transpose of the set of PSFs is implied (forward each PSF is applied individually to each produce an image, but inverse they are summed to create one correction factor. However you have to also reverse each PSF (complex conjugate in the frequency domain)

If you look closely at the formula as written in Wikipedia you see the flipped PSF is used when applying the correction. Of course, it doesn't make a difference if using a symmetrical PSF.

I also noticed this note "due to scattering, the psf strongly varies within each view" here

Technically scattering is random, not a property of the PSF. I think the way you are handling it is good (with a mask it seems), but just be careful of extracting PSFs from highly scattering images. I believe we touched on that briefly when talking this week but I have found PSFs extracted from biological samples to be unreliable because of uneven background. It seems reasonable that a PSF extracted from nearby the sample would be the most accurate (because it by definition is taken under the exact same conditions as the sample), however if a random inconsistency causes a poor PSF, the poor PSF may not be reliable even in nearby locations, and wouldn't be reliable to use for other images.

A final note, I checked over https://github.com/PreibischLab/multiview-reconstruction looking for the part where they do the update, it seems they refactored the code a lot, and I think it does do the 'flipped' PSF properly when calculating the correction factor. However I noticed, as you guys mentioned, there is the option of calculating the combined correction factor with a multiplication. That seems like it could cause noise amplification, especially if there is lots of scatter. I haven't tried that method, but there was recently an acceleration method published in Nature Methods that many people found caused issues after a few iterations see this discussion. I wonder if something similar could happen with multiplicative acceleration

[m-albert]

Hi Brian, it was great meeting you last week. Thanks a lot for your comments here!

As I mentioned I was wondering if the RL code takes the transpose of the forward model when calculating the correction here

You're completely right that the psf is not transposed in the code. However, as you noted, since the used psfs used are symmetrical this doesn't have an effect.

I also noticed this note "due to scattering, the psf strongly varies within each view" here
Technically scattering is random, not a property of the PSF. I think the way you are handling it is good (with a mask it seems), but just be careful of extracting PSFs from highly scattering images. I believe we touched on that briefly when talking this week but I have found PSFs extracted from biological samples to be unreliable because of uneven background. It seems reasonable that a PSF extracted from nearby the sample would be the most accurate (because it by definition is taken under the exact same conditions as the sample), however if a random inconsistency causes a poor PSF, the poor PSF may not be reliable even in nearby locations, and wouldn't be reliable to use for other images.

Thanks a lot for the word of caution, I agree that extracting psfs from close to the sample boundaries would be problematic. The comment in the code is slightly misleading though: actually, currently there's no functionality to extract psfs from the input images, instead an approximation to the theoretical psf is used. What the comment intends to express is that the actual psf typically strongly varies across the sample.

A final note, I checked over https://github.com/PreibischLab/multiview-reconstruction looking for the part where they do the update, it seems they refactored the code a lot, and I think it does do the 'flipped' PSF properly when calculating the correction factor. However I noticed, as you guys mentioned, there is the option of calculating the combined correction factor with a multiplication. That seems like it could cause noise amplification, especially if there is lots of scatter.

Will look into that soon, thanks!

I haven't tried that method, but there was recently an acceleration method published in Nature Methods that many people found caused issues after a few iterations see this discussion. I wonder if something similar could happen with multiplicative acceleration

Yes we've been experimenting with the approach they presented, and initial observations are promising. Thanks to your link I've seen that now there's a repo wrapping the publication's code by Talley Lambert. We'll have a look whether this can be useful in our context. Thanks a lot for the hint!

Thanks again for your comments! I might come back to you with some question :)