Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to correct readout pattern for extended objects with continuum seems to result in a poor readout pattern correction (amp bias correction?) #27

Open
kfkaplan opened this issue Jan 30, 2024 · 7 comments

Comments

@kfkaplan
Copy link
Contributor

kfkaplan commented Jan 30, 2024

I am currently working on a target that is extended and has a lot of continuum and fills the IGRINS slit and the PLP's readout pattern removal code seems to overcorrect for the readout pattern (row amplifier bias? not really clear to me exactly what it is correcting). Below are some images of my target in the H band showing the overcorrection where the continuum is bright. I suspect stray light from the bright continuum hitting the detector beyond the order edges is causing this.

Screenshot 2024-01-29 at 8 19 05 PM
Screenshot 2024-01-29 at 8 19 14 PM

I tracked the issue down to the PLP's readout pattern removal code (in the qlook and reimplement_cr_reject branches). Specifically in
igrins/procedures/readout_pattern_helper.py the following function

    if mask is None:
        mask = np.zeros(d.shape, dtype=bool)
    else:
        mask = mask.copy()

    mask[:4] = True
    mask[-4:] = True

    p = [pipes[k] for k in ['p64_1st_order',
                            'col_wise_bias_c64',
                            'amp_wise_bias_r2',
                            'col_wise_bias']]

    return apply_pipe(d, p, mask=mask)

where amp_wise_bias_r2 appears to be the cause.

I tracked the "pipe" for amp_wise_bias_r2 to igrins/procedures/readout_pattern.py where it points to the following function

class PatternAmpP2(PatternBase):
    @classmethod
    def get(kls, d, mask=None):
        """
        returns a tuple of two 32 element array. First is per-amp bias values.
        The second is the [1,-1] amplitude for each amp.
        """
        d = np.ma.array(d, mask=mask).filled(np.nan)

        do = d.reshape(32, 32, 2, -1)
        av = np.nanmedian(do, axis=[1, 3])

        amp_bias_mean = np.mean(av, axis=1)
        amp_bias_amp = av[:, 0] - amp_bias_mean

        return amp_bias_mean, amp_bias_amp

    @classmethod
    def broadcast(kls, d, av_p):
        av, p = av_p
        k = p[:, np.newaxis] * np.array([1, -1])
        v = np.zeros((32, 32, 2, 1)) + k[:, np.newaxis, :, np.newaxis]
        avv = av.reshape(32, 1, 1, 1) + v
        return avv.reshape(2048, 1)

To try to resolve the issue, I tried to simply comment out amp_wise_bias_r2 in igrins/procedures/readout_pattern_helper.py like so:

    if mask is None:
        mask = np.zeros(d.shape, dtype=bool)
    else:
        mask = mask.copy()

    mask[:4] = True
    mask[-4:] = True

    p = [pipes[k] for k in ['p64_1st_order',
                            'col_wise_bias_c64',
                            #'amp_wise_bias_r2',
                            'col_wise_bias']]

    return apply_pipe(d, p, mask=mask)

and the result is a stacked H band frame that looks much better here when I blink it with what I had before:

ds9

It is very tempting to just leave amp_wise_bias_r2 turned off as a solution, but my main worry is that I am disabling pattern correction or bias subtraction that should be left on, so I am not sure this is the solution. My attempts to modify class PatternAmpP2(PatternBase) did not result in any improvement.

@ericasaw
Copy link

is this from the new pattern noise removal we are trying to implement for Gemini South data? or was there an old pattern removal section in the plp?

@kfkaplan
Copy link
Contributor Author

kfkaplan commented Jan 30, 2024

This is from the old pattern removal section in the PLP. It is unrelated to the additional correction we are adding. (As a side note, the new additional correction is pretty much done, I was going to commit it when I ran into this issue and spent the day trying to solve it).

@ericasaw
Copy link

Do we need to have both running when the new correction is implemented?

@kfkaplan
Copy link
Contributor Author

Yes I found we need both. My new correction gets rid of the small scale "wavy" pattern while the PLP's existing readout pattern correction gets rid of the larger scale pattern so both working together gives the best result.

@ericasaw
Copy link

ericasaw commented Jan 30, 2024 via email

@kfkaplan
Copy link
Contributor Author

We will need to test more but I've only seen this in the extended data I am working with. Specifically it seems to only affect it if there is a lot of extended continuum. It doesn't affect 99.9% of targets, hence why we never really noticed it before.

@kfkaplan
Copy link
Contributor Author

Similar issues seem to arise when only A (on) nods are reduced without the B (off) nod subtracted in the K-band. The K-band has stray light background that falls on the inter-order pixels. It seems the pattern removal is getting confused by the flux on the inter-order pixels and that is what might be causing this problem. In the above example, Uranus has bright enough continuum filling the slit, the light might be leaking into the inter-order pixels and since it is not subtracted in the OFF, the H-band where Uranus has a lot of bright continuum shows this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants