cosmicrays: Remove cosmic rays using flux ratio algorithm

Package: crutil

Usage

cosmicrays input output

Parameters

input: List of input images in which to detect cosmic rays.

output: List of output images in which the detected cosmic rays will be replaced by an average of neighboring pixels. If the output image name differs from the input image name then a copy of the input image is made with the detected cosmic rays replaced. If no output images are specified then the input images are modified in place. In place modification of an input image also occurs when the output image name is the same as the input image name.

crmasks = "": List of cosmic ray mask files to be created; one for each input image. If no file names are given then no cosmic ray mask is created. If an existing mask is specified the newly detected cosmic rays will be added.

threshold = 25.: Detection threshold above the mean of the surrounding pixels for cosmic rays. The threshold will depend on the noise characteristics of the image and how weak the cosmic rays may be for detection. A typical value is 5 or more times the sigma of the background.

fluxratio = 2.: The ratio (as a percent) of the mean neighboring pixel flux to the candidate cosmic ray pixel for rejection. The value depends on the seeing and the characteristics of the cosmic rays. Typical values are in the range 2 to 10 percent. This value may be reset interactively from a plot or defined by identifying selected objects as stars or cosmic rays.

npasses = 5: Number of cosmic ray detection passes. Since only the locally strongest pixel is considered a cosmic ray, multiple detection passes are needed to detect and replace cosmic ray events with multiple neighboring pixels.

window = 5: Size of cosmic ray detection window. A square window of either 5 by 5 or 7 by 7 is used to detect cosmic rays. The smaller window allows detection in the presence of greater background gradients but is less sensitive at discriminating multiple event cosmic rays from stars. It is also marginally faster.

interactive = yes: Examine parameters interactively? A plot of the mean flux within the detection window (x100) vs the flux ratio (x100) is plotted and the user may set the flux ratio threshold, delete and undelete specific events, and examine specific events. This is useful for new data in which one is uncertain of an appropriate flux ratio threshold. Once determined the task need not be used interactively.

train = no: Define the flux ratio threshold by using a set of objects identified as stars (or other astronomical objects) or cosmic rays?

objects = "": Cursor list of coordinates of training objects. If null (the null string "") then the image display cursor will be read. The user is responsible for first displaying the image. Otherwise a file containing cursor coordinates may be given. The format of the cursor file is "x y wcs key" where x and y are the pixel coordinates, wcs is an arbitrary number such as 1, and key may be 's' for star or 'c' for cosmic ray.

savefile = "": File to save (by appending) the training object coordinates. This is of use when the objects are identified using the image display cursor. The saved file can then be input as the object cursor list for repeating the execution.

plotfile: If a plot file is specified then the graph of the flux ratio (x100) vs the mean flux (x100) is recorded as metacode. This may be spooled or examined later.

graphics = "stdgraph": Interactive graphic output device for interactive examination of the detection parameters.

cursor = "": Interactive graphics cursor input. If null the graphics display cursor is used, otherwise a file containing cursor input may be specified.

answer: This parameter is used for interactive queries when processing a list of images. The responses may be "no", "yes", "NO", or "YES". The upper case responses permanently enable or disable the interactive review while the lower case reponses allow selective examination of certain input images. This parameter should not be specified on the command line. If it is then the value will be ignored and the task will act as if the answer "yes" is given for each image; i.e. it will enter the interactive phase without prompting.

Image cursor commands

?       Help
c       Identify the object as a cosmic ray
s       Identify the object as a star
g       Switch to the graphics plot
q       Quit and continue with the cleaning

GRAPHICS CURSOR COMMANDS

?       Help
a       Toggle between showing all candidates and only the training points
d       Mark candidate for replacement (applys to '+' points)
e       Mark candidates in a region for replacement (applys to '+' points)
q       Quit and return to image cursor or replace the selected pixels
r       Redraw the graph
s       Make a surface plot for the candidate nearest the cursor
t       Set the flux ratio threshold at the y cursor position
u       Mark candidate to not be replaced (applys to 'x' points)
v       Mark candidates in a region to not be replaced (applys to 'x' points)
w       Adjust the graph window (see gtools)
<space> Print the pixel coordinates

There are no colon commands except those for the windowing options (type :\help or see gtools).

Description

Cosmic ray events in each input image are detected and replaced by the average of the four neighbors. The replacement may be performed directly on the input image if no output image is specified or if the output image name is the same as the input image name. If a new image is created it is a copy of the input image except for the replaced pixels. Optional output includes a plot file showing the parameters of the detected cosmic ray candidates and the flux ratio threshold used, a cosmic ray mask identifying the cosmic rays found, and a file of training objects marked with the image display cursor. The cosmic ray mask may be used for display purposes, combined with other masks, and with crfix.

This task may be applied to an image previously processed to detect additional cosmic rays.

The cosmic ray detection algorithm consists of the following steps. First a pixel must be the brightest pixel within the specified detection window (either 5x5 or 7x7). The mean flux in the surrounding pixels with the second brightest pixel excluded (which may also be a cosmic ray event) is computed and the candidate pixel must exceed this mean by the amount specified by the parameter threshold. A plane is fit to the border pixels of the window and the fitted background is subtracted. The mean flux (now background subtracted) and the ratio of this mean to the cosmic ray candidate (the brightest pixel) are computed. The mean flux (x100) and the ratio (x100) are recorded for interactive examination if desired.

Once the list of cosmic ray candidates has been created and a threshold for the flux ratio established (by the parameter fluxratio, by the "training" method, or by using the graphics cursor in the interactive plot) the pixels with ratios below the threshold are replaced in the output by the average of the four neighboring pixels (with the second strongest pixel in the detection window excluded if it is one of these pixels). Additonal pixels may then be detected and replaced in further passes as specified by the parameter npasses. Note that only pixels in the vicinity of replaced pixels need be considered in further passes.

The division between the peaks of real objects and cosmic rays is made based on the flux ratio between the mean flux (excluding the center pixel and the second strongest pixel) and the candidate pixel. This threshold depends on the point spread function and the distribution of multiple cosmic ray events and any additional neighboring light caused by the events. This threshold is not strongly coupled to small changes in the data so that once it is set for a new type of image data it may be used for similar images. To set it initially one may examine the scatter plot of the flux ratio as a function of the mean flux. This may be done interactively or from the optional plot file produced.

After the initial list of cosmic ray candidates has been created and before the final replacing cosmic rays there are two optional steps to allow examining the candidates and setting the flux ratio threshold dividing cosmic rays from real objects. The first optional step is define the flux ratio boundary by reference to user specified classifications; that is "training". To do this step the train parameter must be set to yes. The user classified objects are specified by a cursor input list. This list can be an actual file or the image display cursor as defined by the objects parameter. The savefile parameter is also used during the training to record the objects specified. The parameter specifies a file to append the objects selected. This is useful when the objects are defined by interactive image cursor and does not make much sense when using an input list.

If the objects parameter is specified as a null string then the image display cursor will be repeatedly read until a 'q' is entered. The user first displays the image and then when the task reads the display cursor the cursor shape will change. The user points at objects and types 's' for a star (or other astronomical object) and 'c' for a cosmic ray. Note that this input is used to search for the matching object in the cosmic ray candidate list and so it is possible the selected object is not in the list though it is unlikely. The selection will be quietly ignored in that case. To exit the interactive selection of training objects type 'q'.

If 'g' is typed a graph of all the candidates is drawn showing "flux" vs. "flux ratio" (see below for more). Training objects will be shown with a box and the currently set flux ratio threshold will also be shown. Exiting the plot will return to entering more training objects. The plot will remain and additional objects will immediately be shown with a new box. Thus, if one wants to see the training objects identified in the plot as one selects them from the image display first type a 'g' to draw the initial plot. Also by switching to the plot with 'g' allows you to draw surface plots (with 's') or get the pixel coordinates of a candidate (the space key) to be found in the display using the coordinate readout of the display. Note that the display interaction is simpler than might be desired because this task does not directly connect to the display.

The most likely use for training is with the interactive image display. However one may prepare an input list by other means, one example is with rimcursor, and then specify the file name. The savefile may also be used a cursor input to repeat the cosmic ray operation (but be careful not to have the cursor input and save file be the same file!).

The flux ratio threshold is determined from the training objects by finding the point with the minimum number of misclassifications (stars as cosmic rays or cosmic rays as stars). The threshold is set at the lowest value so that it will always go through one of the cosmic ray objects. There should be at least one of each type of object defined for this to work. The following option of examining the cosmic ray candidates and parameters may still be used to modify the derived flux ratio threshold. One last point about the training objects is that even if some of the points lie on the wrong side of the threshold they will remain classified as cosmic ray or non-cosmic ray. In other words, any object classified by the user will remain in that classification regardless of the final flux ratio threshold.

After the training step the user will be queried to examine the candidates in the flux vs flux ratio plane if the interactive flag is set. Responses may be made for specific images or for all images by using lower or upper case answers respectively. When the parameters are examined interactively the user may change the flux ratio threshold ('t' key). Changes made are stored in the parameter file and, thus, learned for further images. Pixels to be deleted are marked by crosses and pixels which are peaks of objects are marked by pluses. The user may explicitly delete or undelete any point if desired but this is only for special cases near the threshold. In the future keys for interactive display of the specific detections will be added. Currently a surface plot of any candidate may be displayed graphically in four 90 degree rotated views using the 's' key. Note that the initial graph does not show all the points some of which are clearly cosmic rays because they have negative mean flux or flux ratio. To view all data one must rewindow the graph with the 'w' key or ":/" commands (see gtools).

Examples

1. To replace cosmic rays in a set of images ccd* without training:

cl> cosmicrays ccd* new//ccd*
ccd001: Examine parameters interactively? (yes):
[A scatter plot graph is made.  One can adjust the threshold.]
[Looking at a few points using the 's' key can be instructive.]
[When done type 'q'.]
ccd002: Examine parameters interactively? (yes): NO
[No further interactive examination is done.]

After cleaning one typically displays the images and possibly blinks them. A difference image or mask image may also be created.

2. To use the interactive training method for setting the flux ratio threshold:

# First display the image.
cl> display ccd001 1
z1 = 123.45 z2= 543.21
cl> cosmicrays ccd001 ccd001cr train+
[After the cosmic ray candidates are found the image display
[cursor will be activated.  Mark a cosmic ray with 'c' and
[a star with 's'.  Type 'g' to get a plot showing the two
[points with boxes.  Type 'q' to go back to the image display.
[As each new object is marked a box will appear in the plot and
[the threshold may change.  To find the location of an object
[seen in the plot use 'g' to go to the graph, space key to find
[the pixel coordinates, 'q' to go back to the image display,
[and the image display coordinate box to find the object.
[When done with the training type 'q'.
ccd001: Examine parameters interactively? (yes): no

3. To create a mask image a bad pixel file must be specified.

cl> cosmicrays ccd001 ccd001 crmask=crccd001