Add counter 1/f and common mode subtraction to preprocess

[17-sugiyama]

This PR modifies the scripts to enable fancy counter 1/f filter and common mode subtraction.

The changes for counter 1/f filter:
The filter function 'counter_1_over_f' is modified to accept the array-like input of parameters.
This enables to apply independent 1/f parameters to individual detectors.

The changes for common mode subtraction:
The PCAFilter is modified to have signal and model_signal independently.
This enables to estimate PCA model from the down sampled signal, and then subtract it from the signal.
For the fancy operations, I introduced get_common_noise_params to estimate knee frequency of the common noise.
The estimated knee frequency is applied to the down sample frequency of model_signal .
However, this deployment requires many changes in fft_ops.
(As common mode aman doesn't have 'dets' LabelAxis, aman.dets is changed to aman[LabelAxis].)
These changes can be discarded if they are too much.

The following lines are the example of the process pipeline after the demodulation.
I have confirmed the processes run with this config.

process_pipe:
    # correct boresight rotation to zero
    - name : "rotate_focal_plane"
      process:
        hwp: True

    # telescope coordinate
    - name : "rotate_qu"
      process:
        sign: 1 
        offset: 0 
        update_focal_plane: True

    # radial coordinate
    - name : "rotate_qu"
      process:
        sign: 1 
        offset: 0 
        radial: True
        update_focal_plane: True

    # calculate psd
    - name: "psd"
      skip_on_sim: False
      signal: "demodQ"
      wrap: "psdQ"
      calc:
        max_samples: 524288 #2**19
        merge: False
        nperseg: 65536 #2**16
        noverlap: 0
        full_output: True
      save: True

    # fit psd
    - name: "noise"
      skip_on_sim: False
      psd: "psdQ"
      fit: True
      calc:
        fwhite: [1, 2]
        f_max: 2
        merge_name: 'noise_fit_statsQ'
      save:
        wrap_name: "noise_fitQ"

    # apply counter 1/f
    - name: "fourier_filter"
      skip_on_sim: False
      signal_name: "demodQ"
      wrap_name: "demodQ"
      process:
        filt_function: "counter_1_over_f"
        filter_params:
          noise_fit_stats: "noise_fit_statsQ"
          
    # get common mode and its fknee
    - name: "get_common_mode"
      noise_fit: True
      f_max: 2
      wrap_name: "demodQ_commonmode"
      calc:
        signal: "demodQ"
        method: "median"
      save: True

    # get signal downsampled to fknee
    - name: "fourier_filter"
      signal_name: "demodQ"
      wrap_name: "lpf_demodQ"
      process:
        noise_fit_array: "demodQ_commonmode"
        filters:
          - name: "low_pass_butter4"

    # estimate and subtract common noise estimated from 
    # the down sampled signal
    - name: "pca_filter"
      signal: "demodQ"
      model_signal: "lpf_demodQ"
      process:
        n_modes: 3

[17-sugiyama]

I guess it would be better to modify fft_ops.build_hpf_params_dict following the changes in counter_1_over_f filter, but I haven't changed this yet.

[msilvafe]

Just a few requests and questions inline

[msilvafe]

You need to add a description for this new LabelAxis variable in the docstring. Also do not use capitals in the function arguments that's reserved for class definitions.

[msilvafe]

It's also valid to pass in a precomputed common mode signal with shape 1 x nsamps. That option should be explained here in the docstring.

[msilvafe]

I think if you should raise an error if all 3 (fk, n and noise_fit_stats) are None at this point in the code and describe this combination of inputs in the docstring.

[msilvafe]

Why did you add this isrealobj check? What issue were you running into?

[17-sugiyama]

It doesn't make any issue as long as the input signal is real object.
I thought that we need this kind of change if we want to derive the common mode of Q+iU, for example.
But I didn't implement an option for the complex common mode derivation in the pipeline so far.

[msilvafe]

This noise_fit option should be described better in the docstring here.

[msilvafe]

In the following places, here:

The changes for common mode subtraction: The PCAFilter is modified to have signal and model_signal independently. This enables to estimate PCA model from the down sampled signal

and here:

# get signal downsampled to fknee
- name: "fourier_filter"
  signal_name: "demodQ"
  wrap_name: "lpf_demodQ"
  process:
    noise_fit_array: "demodQ_commonmode"
    filters:
      - name: "low_pass_butter4"

You mention downsampling but that's not implemented here do you just mean demodulated and/or low pass filtered?

[msilvafe]

Also why do you need these 3 different rotation steps, especially for telescope vs radial Q/U rotation shouldn't it be one or the other not both?

# correct boresight rotation to zero
- name : "rotate_focal_plane"
  process:
    hwp: True

# telescope coordinate
- name : "rotate_qu"
  process:
    sign: 1 
    offset: 0 
    update_focal_plane: True

# radial coordinate
- name : "rotate_qu"
  process:
    sign: 1 
    offset: 0 
    radial: True
    update_focal_plane: True

[17-sugiyama]

Also why do you need these 3 different rotation steps, especially for telescope vs radial Q/U rotation shouldn't it be one or the other not both?

These steps operate,

• Rotate the focal plane by the boresight angle, and then set boresight value to zero
• Rotate the detectors to the telescope coordinate (horizontal, vertical angle is Q)
• Rotate the detectors to the radial coordinate (radial, tangential is Q)

respectively.
This function coords.demod.rotate_demodQU cannot convert the detector coordinate to the radial coordinate at a time.
I can modify pipeline.py to apply multiple rotations like the Fourier filter function.

[17-sugiyama]

You mention downsampling but that's not implemented here do you just mean demodulated and/or low pass filtered?

I'm sorry! Exactly, downsample is a mistake for low pass filtering.