ccdproc: Process mosaic exposures

Package: mscred

Usage

ccdproc images

Parameters

images: List of mosaic or multiple amplifier CCD data to process. The data is typically in multiextension files though simple single images are allowed. If the input includes processed data requiring no further processing it will be silently skipped. Calibration data may be included in the input list and it will be selected and processed as needed provided the data has a keyword identifying the type of data. However, more commonly the calibration data is specified separately with the calibration data parameters.

output = "": List of output processed data. If no list is given then the processing will replace the input images with the processed images, possibly after making a backup of the input if the package "bkuproot" parameter is defined. If a list is given it must match the input list. Note that dependent calibration data requiring processing will be processed in-place (with optional backup).

bpmasks = "": List of output bad pixel files or directories to contain bad pixel masks created for each input. If the input is a single image then the output is a bad pixel file while if the input is a multiextension file then the output is a directory to contain a bad pixel mask file for each extension. If no list is specified then no output masks will be produced. The output mask will be a combination of pixels specified in the "fixfile" parameter and identified as saturated or bleed trail pixels.

ccdtype = "": CCD type to select from the input list. If no type is given then all input will be selected. The recognized types are described in ccdtypes.

noproc = no: Only list processing steps to be performed for each input file?

PROCESSING SWITCHES

xtalkcor = no: Apply a crosstalk correction? The crosstalk file is specified by the "xtalkfile" parameter.

fixpix = yes: Fix bad CCD pixels by linear interpolation from neighboring lines and columns? If a file is specified by the "fixfile" parameter then the identified pixels will be interpolated upon input either along lines or columns depending on the mask value and dimensions of the regions. If saturated or bleed trail pixels are defined in this task, these will be interpolated on output (i.e. after all other processing) along lines.

overscan = yes: Apply overscan or prescan bias correction? If yes then the overscan section must be specified with the "biassec" parameter.

trim = yes: Trim the image of the overscan region and bad edge lines and columns? If yes then the data section must be specified with the "trimsec" parameter.

zerocor = yes: Apply zero level correction? If yes a zero level image must be specified with the "zero" parameter.

darkcor = yes: Apply dark count correction? If yes a dark count image must be specified with the "dark" parameter.

flatcor = yes: Apply flat field correction? If yes flat field images must be specified with the "flat" parameter.

sflatcor = no: Apply sky flat field correction? If yes sky flat field images must be specified with the "sflat" parameter.

merge = yes: Merge amplifiers from the same CCD? If yes then the amplifier extensions with the same CCD name will be merged into a single extension with the header and extension name of the first amplifier extension in the file. If only a single extension results from the merging then a simple image file is produced. If the input has only one amplifier per CCD then nothing is done. The merging also creates new bad pixel masks if an output bad pixel mask is specified and if the merged masks differ from the current bad pixel masks.

PROCESSING PARAMETERS

The parameters, "xtalkfile", "fixfile", "saturation", "bleed", "biassec", "trimsec", "zero", "dark", "flat", and "sflat" may reference keywords containing the desired value by preceding the keyword name with '!'. This allows each image or image extension in each input to have different values. Note that keyword name specified may be translated through the instrument file to another keyword or to a default value.

xtalkfile = "": Crosstalk file for the crosstalk correction. Only one crosstalk file may be specified and it applies to all the input data being processed. A keyword reference may be used to specify the file by preceding the keyword name with '!'.

fixfile = "": Bad pixel mask, image, or file. specified in the image header or instrument translation file. A bad pixel mask is a compact format (".pl" extension) with zero values indicating good pixels and non-zero values indicating bad pixels. A bad pixel image is a regular image in which zero values are good pixels and non-zero values are bad pixels. A bad pixel file specifies bad pixels or rectangular bad pixel regions as described later. The direction of interpolation is determined by the mask value with a value of two interpolating across columns, a value of three interpolating across lines, and any other non-zero value interpolating along the narrowest dimension. A keyword reference may be used to specify the mask by preceding the keyword name with '!'. The special value "BPM" may also be used reference the standard BPM keyword for a bad pixel mask.

saturation = "INDEF": Pixels with values equal to or greater than this value in the input data are identified as saturated by the mask value 4. The saturation value is specified by two words. The first word is a number giving the saturation pixel value. The value INDEF is equivalent to positive infinity and will identify no pixels as saturated. The second word is the units which may be "ADUs" or "electrons". If the units are "electrons" then the conversion from ADUs to electrons (in electrons per ADU) will be obtained from the "gain" keyword (which may be translated to some other keyword in the instrument file. The units may abbreviated or be omitted, which then defaults to "ADUs". If the first word is not a number (with or without a preceding '!') then the word is considered to be a keyword reference. The value of the keyword is interpreted in the same way as a number with optional units. Note that numeric keywords cannot not have a units specification so they will always be understood as being in ADUs. Since there is only one parameter value a keyword is the way to provide different saturation values for the extensions and list of input data.

sgrow = 0: Number of neighboring pixels along rows and columns from a saturated pixel which are also identified as saturated pixels.

bleed = "INDEF"

Threshold for identifying bleed trail pixels. This may be specified in the same way as the saturation value including use of "ADUs" and "electrons" and reference to a header keyword. In addition the value may be set in relation to the saturation value or the mean of the data with one of the following specifications

saturation-X, saturation/X, mean+X, mean*X

where X is a number and the values are in ADU. For example the value "mean+5000" would define candidate bleed trail pixels as those which are 5000 counts above the mean. Note that for a pixel to actually be identified as a bleed pixel there must be a consecutive number of pixels (parameter btrail) along a column which are above this threshold.

btrail = 20: Number of consecutive pixels with values above the bleed pixel threshold along a column to qualify as a bleed trail. The threshold is specified by the bleed parameter.

bgrow = 0: Number of neighboring pixels along rows and columns from a bleed trail pixel which are also identified as bleed trail pixels.

Radius

biassec = "": Overscan bias image section. Only the part of the bias section along the lines is used. The column length of the bias region fit is defined by the trim section. If one wants to limit the region of the overscan used in the fit to be less than that of the trim section then the sample region parameter, sample, should be used. It is an error if no section or the whole image is specified. A keyword reference may be used to specify the file by preceding the keyword name with '!'. The older form of the special word "image" to reference the keyword BIASSEC is also allowed.

trimsec = "": Image section defining the trimmed output. A keyword reference may be used to specify the file by preceding the keyword name with '!'. The older form of the special word "image" to reference the keyword TRIMSEC is also allowed.

fixfile = "": Bad pixel mask, image, or file. specified in the image header or instrument translation file. A bad pixel mask is a compact format (".pl" extension) with zero values indicating good pixels and non-zero values indicating bad pixels. A bad pixel image is a regular image in which zero values are good pixels and non-zero values are bad pixels. A bad pixel file specifies bad pixels or rectangular bad pixel regions as described later. The direction of interpolation is determined by the mask value with a value of two interpolating across columns, a value of three interpolating across lines, and any other non-zero value interpolating along the narrowest dimension. A keyword reference may be used to specify the mask by preceding the keyword name with '!'. The special value "BPM" may also be used reference the standard BPM keyword for a bad pixel mask.

zero = "": List of zero level calibration files. The first image or image extension matching the amplifier of the input image to be calibrated is used. The CCD type and subset are not checked for these images. If no calibration image is found as specified by this parameter then the input list is checked for files of the appropriate CCD type. The zero level calibration images may be one or two dimensional. If the calibration file has not been processed it is processed as approprate for this type of calibration using the same parameters as for the input data being processed. A keyword reference may be used to specify the file by preceding the keyword name with '!'.

dark = "": List of dark count calibration files. The first image or image extension matching the amplifier of the input image to be calibrated is used. The CCD type and subset are not checked for these images. If no calibration image is found as specified by this parameter then the input list is checked for files of the appropriate CCD type. If the calibration file has not been processed it is processed as approprate for this type of calibration using the same parameters as for the input data being processed. A keyword reference may be used to specify the file by preceding the keyword name with '!'.

flat = "": List of flat field calibration files. The first image or image extension matching the amplifier and subset of the input image to be calibrated is used. The CCD type and subset are not checked for these images. If no calibration image is found as specified by this parameter then the input list is checked for files of the appropriate CCD type. If the calibration file has not been processed it is processed as approprate for this type of calibration using the same parameters as for the input data being processed. The flat field images may be one or two dimensional. A keyword reference may be used to specify the file by preceding the keyword name with '!'.

sflat = "": List of sky flat field calibration files. The first image or image extension matching the amplifier and subset of the input image to be calibrated is used. The CCD type and subset are not checked for these images. If no calibration image is found as specified by this parameter then the input list is checked for files of the appropriate CCD type. If the calibration file has not been processed it is processed as approprate for this type of calibration using the same parameters as for the input data being processed. The sky flat field images may be one or two dimensional. A keyword reference may be used to specify the file by preceding the keyword name with '!'.

minreplace = 1.: When processing flat fields, pixel values below this value (after all other processing such as overscan, zero, and dark corrections) are replaced by this value. This allows flat fields processed by ccdproc to be certain to avoid divide by zero problems when applied to object images.

OVERSCAN BIAS FITTING PARAMETERS

There are two types of overscan (or prescan) bias determinations. One determines a independent bias value for each line. The other averages the overscan columns to make an overscan vector, fits a smooth bias function to the vector, and then evaluates the bias function to get the bias at each line. The line-by-line bias determination only uses the function parameter. The bias function determination uses the icfit procedure with the following parameters.

interactive = no: Fit the overscan bias vector interactively? If yes and the bias function type is one of the icfit types then the average overscan bias vector is fit interactively using the icfit package. If no then the fitting parameters are used in a non-interactive fit.

function = "legendre"

Line-by-line determination of the bias is specified by:

  mean - the mean of the biassec columns at each line
median - the median of the biassec columns at each line
minmax - the mean at each line with the min and max excluded

The bias vector may be fit by one of the functions:

 legendre - legendre polynomial
chebyshev - chebyshev polynomial
  spline1 - linear spline
  spline3 - cubic spline

order = 1: Number of polynomial terms or spline pieces in the overscan fit. To simply use the average bias use a polynomial function of order 1.

sample = "*": Sample points to use in the overscan bias fit. The string "*" specifies all points otherwise an icfit range string is used.

naverage = 1: Number of points to average or median to form fitting points. Positive numbers specify averages and negative numbers specify medians.

niterate = 1: Number of rejection interations to remove deviant points from the overscan fit. If 0 then no points are rejected.

low_reject = 3., high_reject = 3.: Low and high sigma rejection factors for rejecting deviant points from the overscan fit.

grow = 0.: One dimensional growing radius for rejection of neighbors to deviant points.

PACKAGE PARAMETERS

pixeltype = "real real": Output pixel datatype and calculation datatype. When images are processed or created, the output pixel datatype is the highest precision of the input pixel datatype and the specified output datatype. The allowed datatypes and order of precision are "short", "ushort", "int", "long", "real", or "double". The calculation datatype may either be short or real. Real is the default if no calculation type is specified.

verbose = no: Print log information to the standard output?

logfile = "logfile": Logfile to append log information. If no filename is specified then no logfile is kept.

plotfile = "": Metacode plotfile for appending plots of the overscan bias fits. If no filename is specified then no metacode plotfile is kept.

backup = "once" (none|once|all): Backup the input data when the input file is replaced by the processed data? If the value is "none" then no backup of the input data is made. If the value is "once" then only the first backup of the input is made. If the value is "all" than if the input is repeatedly replaced by additional processing then additional backups will be made.

bkuproot = "Raw/": When a backup of the input data is made the string given by this parameter is used as a prefix to the original input data filename. If the root is a directory name (ends with '$' or '/') the directory will be created if needed and the input data moved to the directory. When the backup type is "all" and a second version of the input is backed up a digit is prepended to the input filename.

instrument = "": CCD instrument file. See help for instrument.

ampfile = "amps": The "amp" keyword (which may be translated in the instrument file) produces a string identifying the amplifier for each image. A mapping between the full string and a short version (based on the first word) is stored in this file.

ssfile = "subsets": The "subset" keyword (which may be translated in the instrument file) produces a string identifying a subset for each image. A mapping between the full string and a short version (based on the first word) is stored in this file.

im_bufsize = 0.065536: When a line of an image is read a larger block of data is actually read. This parameter defines the block size in megabytes. For large images this I/O buffering often makes the processing more efficient. Note however that setting this to the size of the image does not necessarily make the processing faster. Once the block size reaches an optimal size for the disk I/O system it does not improve performance further and might actually degrade performance if too much memory is tied up.

graphics = "stdgraph": Graphics output device for interactive graphics.

cursor = "": Graphics cursor input. If null the standard terminal graphics cursor is used.

version: Package version string.

Description

Ccdproc applies various calibrations and corrections to CCD data in multiextension (mosaic or multiamplifier) or single image formats. The calibrations and corrections are for amplifier crosstalk, detector defects, electronic bias, zero level bias, dark counts, and pixel responses. The task also identifies saturated pixels and bleed trails, trims unwanted edge lines and columns, merges multiple amplfiers from the same CCD into single images, and changes the pixel datatype.

The task is designed to be efficient and easy to use. All one has to do is set the parameters and begin processing the data. The task takes care of most of the record keeping and automatically does the prerequisite processing of calibration images. Beneath this simplicity there is much going on. In this section a brief description of the usage is given. The following sections present detailed discussions on the different operations performed and the order and logic of the processing steps.

One begins by setting the task parameters. There are many parameters but they may be easily reviewed and modified using "eparam". The CCD data to be processed is specified with the "input" parameter list as a combination of filenames, filename templates, and @files. Previously processed data are silently ignored and calibration files are recognized provided the CCD image types are identified in the image headers (see instruments and ccdtypes). Therefore it is permissible to use simple image templates such as "*.fits". However, it is recommended that calibration data by specified explicitly with the appropriated parameters.

The "ccdtype" parameter may be used to select only certain types of CCD data to process. If the data does not contain a CCD type identification keyword then the parameter can be set to the null string "". In this case it is the user's responsibility to select the correct processing steps for the type of data, and the calibration data cannot be determined automatically from the input list.

The names for processed data are specified by the "output" parameter list of names which are matched in order against the input list. However, if no output list is given the processed data replaces the input data with an option to make a backup of the original input file (see the package "bkuproot" parameter). The output file will be in the same format as the input file except that if a multiextension input consists of multiple amplifiers from a single CCD and the amplifiers are merged, a single simple image will be produced.

Other (optional) output includes pixel masks and processing log information. Output pixel masks are specified by the "bpmasks" parameter. The masks merge any input pixel mask data with identification of saturated or non-linear pixels and bleed trails. The processing information consists of a logfile and/or terminal output for text and a plotfile for plots of the overscan bias fitting. These are select with the package "logfile", "verbose", and "plotfile" parameters.

The processing operations are selected by boolean (yes/no) parameters. When the input data includes CCD type identifications the processing options may be set for object data and only the appropriate subset of operations will be performed on the calibration data. Any combination of operations may be specified. While it is possible to do operations in separate steps some sets of operations are done in a single pass through the data and will be more efficiently performed if done at the same time.

The processing steps selected have related parameters which must be specified. These are things like image sections defining the electronic bias overscan and trim regions, parameters for identifying saturated pixels and bleed trails, and calibration files. There are a number of parameters used for fitting the overscan or prescan electronic bias data. These are parameters used by the standard IRAF curve fitting package icfit.

Calibration data are specified by task parameters and/or in the input list. The task paramters are lists so more than one calibration file may be specified. Zero and dark count calibrations generally only need one file but flat field calibrations need one for each subset which is typically the filter. When more than one calibration file is specified then the first one encountered that matches the input is used and a warning is issued for the extra files. Calibration files specified by task parameters take precedence over calibration files in the input list.

In addition to the task parameters there are package parameters which affect ccdproc. These include the instrument, amplifier, and subset files, the verbose, text and plot output log settings, the output and calculation pixel datatype, the amount of memory to use for image I/O buffering, and the backup option. The instrument file is used to define the keywords to be used, translations of CCD type strings to a standard set, and defaults for missing keywords. The amplifier and subset files translate arbitrary keyword values for the amplifier and subset to short one word identifiers. Users may edit these files to change the mapping. The image I/O buffering may be increased to improve I/O efficiency. Note that this is just how much is read in one I/O request and is not a means to cache an image in memory. The backup option allows input files to be saved with a new name or in a directory when the processed data replaces the input. One may backup once, every time, or not at all. When a backup is requested the prefix string is added to the input name or the input is moved to the backup directory. The datatype parameter determines the type of the output pixel and the calculation mode. Typically raw CCD data is in short integers and processed data is saved as real (32-bit floating point) values.

When an input file is processed the task first determines the processing parameters and calibration files. If a requested operation has been done it is skipped and if all requested operations have been completed then no processing takes place. When it determines that a calibration file is required it checks for the file from the task parameter and then for a calibration file of the proper type in the input list. Having selected a calibration file it checks if it has been processed. If it has not been processed, based on the current settings of the processing options appropriate for that type of calibration, it is processed automatically. Once the processing parameters and calibration files have been determined the input file is processed. The output processed data will include keywords identifying the processing steps and calibration files used.

Xtalkcor: amplifier crosstalk correction

When multiple amplifiers are readout, such as occurs when using multiple amplifiers in a single CCD or multiple CCDs in a mosaic, there is the possibilty of crosstalk in the controller electronics. The crosstalk causes pixel values produced by one amplifier to be affected by the signal in another amplifier. There are many ways this crosstalk may affect the data. Ccdproc includes a way to correct pixels based on a simple crosstalk model.

In this model the signal for a pixel in one amplifier, which we call the "source", adds or subtracts a small amount to the pixel value read at the same time in another amplifier, called the "victim". A correction is obtained by multiplying the pixel value of the source image by a crosstalk coefficient and adding or subtracting it from the matching pixel in the victim image.

Note that it is possible that a source may also be a victim and that a victim may be affected by multiple sources. In our simple model each pair of source and victim are treated independently and the source pixel values used to correct a victim are treated as unaffected by other amplifiers.

The crosstalk coefficients are given by a crosstalk calibration file. This may be specified explicitly through reference to a keyword. The correction is performed by the task xtalkcor which is called from ccdproc. Information about the format of the crosstalk calibration file and details of the algorithm are found in the description for that task. The crosstalk coefficients may provided by the observatory as a standard calibration file or they may be estimated from the data using the task xtcoeff.

The crosstalk correction is performed before any other operation. The simple model of the crosstalk is that the raw data from the amplifier readout is used. Therefore the correction should generally be applied only to the raw data.

Saturated pixels

Saturated pixels are identified as those pixels with values above a fixed threshold in the input image before they are modified by any other calibration. Any pixels identified as bad in a pixel file given by the "\Ifixfile" parameter are excluded. Neighboring pixels, those within a distance of "sgrow" pixels along lines or columns, of the threshold selected saturation pixels are also identified as saturated.

To identify saturated pixels a saturation threshold is specified by the "saturation" parameter. The saturation value may be given in units of digital counts as recorded in the image data or as electrons related to the digital counts through a gain keyword in the header. The parameter description explains how to specify the saturation threshold. The term "saturated" can really be used to apply to any pixels which are non-linear and not correctable. Thus the saturation threshold need not be the actual saturation of the CCD but some lower value where the pixels become uncorrectably non-linear.

The identified pixels are recorded in the output bad pixel mask specified by the "bpmasks" parameter with a mask value of 4. If the "fixpix" processing option is selected the saturated pixels are replaced by linear interpolation along lines. If a pixel identified as bad in an input mask or file touches a saturated pixel it is also interpolated. This is done to avoid funny effects where the bad pixel is first interpolated using data which has not yet been identified as a bleed trail or saturated pixel and which is not subsequently replaced by more reasonable data values.

Note that if no output pixel mask or pixel replacement are specified then the saturated pixels will have no effect. Therefore, the identification of such pixels is not done by the task even if the other parameters are set to identify saturated pixels. This operation does not apply to data identified as zero, dark, or flat.

Bleed trails

Bleed trails are identifed as regions with some minimum number of consecutive pixels along a columns having values above a fixed threshold. The pixel values are before they are modified by any other calibration. Neighboring pixels, those within a distance of "bgrow" pixels along lines or columns, of the threshold selected bleed trail pixels are also identified as part of the bleed trail. Any pixels identified as bad in a pixel file given by the "\Ifixfile" parameter are excluded.

To identify bleed trails a threshold is specified by the "bleed" parameter. The value may be given in units of digital counts as recorded in the image data or as electrons related to the digital counts through a gain keyword in the header. The parameter description explains how to specify the bleed threshold. In addition to an explicit value specified by the parameter or in the header the threshold may be specified in relation to the saturation threshold or to the mean value in the data.

Note that it is not individual pixels above a threshold but a consecutive number of pixels. This means the threshold can be fairly low provided the minimum bleed trail length, specified by the "btrail" parameter, is greater than would occur in objects. For this reason specifying the threshold as some number times the mean or above the mean is very useful. A recommendation is to use "mean+5000" when the data in counts are from 15 or 16 bit A/D converters.

The identified pixels are recorded in the output bad pixel mask specified by the "bpmasks" parameter with a mask value of 5. If the "fixpix" processing option is selected the bleed trails are replaced by linear interpolation along lines. If pixel identified as bad in an input mask or file touches the bleed trail it is also interpolated. This is done to avoid funny effects where the bad pixel is first interpolated using data which has not yet been identified as a bleed trail or saturated pixel and which is not subsequently replaced by more reasonable data values.

Note that if no output pixel mask or pixel replacement are specified then the bleed trails will have no effect. Therefore, the identification of such pixels is not done by the task even if the other parameters are set to identify saturated pixels. This operation does not apply to data identified as zero, dark, or flat.

Output pixel masks

An output pixel mask is created when a name is specified with the "bpmasks" parameter and the mask does not exist. If the processing does not involved any modification to the input data then only the mask will be produced. The mask is a combination of the input mask specified by the "fixfile" parameter and pixels identified as saturated and bleed trails. Note that the "fixfile" parameter is used even if "fixpix" is not set.

An input bad pixel mask is not required and if none is specified then the output will be just the pixels identified as bleed trails or saturated. If the saturated pixels and bleed trails are not identified and no input mask is specified then the output will simply be an empty mask.

The specified output mask is currently used as a directory name. It is created if it is not found. The individual bad pixel masks, in pixel list format, are created in this directory. In a future version the multiple pixel masks will be stored as extensions in the multiextension file specified by the output mask name.

Fixpix: replacing bad pixels by interpolation

Regions of bad lines and columns may be replaced by linear interpolation from neighboring lines and columns when the parameter fixpix is set. This algorithm is the same as used in the task fixpix. The bad pixels may be specified by a pixel mask, an image, or a text file. For a mask or image, values of zero indicate good pixels and other values indicate bad pixels to be replaced.

A text file consists of lines with four fields, the starting and ending columns and the starting and ending lines. Any number of regions may be specified. Comment lines beginning with the character '#' may be included. The description applies directly to the input image (before trimming) so different files are needed for previously trimmed or subsection readouts. The data in this file is internally turned into the same description as a bad pixel mask with values of two for regions which are narrower or equal across the columns and a value of three for regions narrower across lines.

The direction of interpolation is determined from the values in the mask, image, or the converted text file. A value of two interpolates across columns, a value of three interpolates across lines, and any other value interpolates across the narrowest dimension of bad pixels and using column interpolation if the two dimensions are equal.

The bad pixel description may be specified explicitly or by reference to a keyword with the name. The special value "BPM" or "image" references the keyword BPM.

Overscan: removing electronic bias using overscan/prescan data

If an overscan or prescan correction is specified (overscan parameter) then the image section (biassec parameter) defines the overscan region.

There are two types of overscan (or prescan) determinations. One determines a independent overscan value for each line and is only available for a readaxis of 1. The other averages the overscan along the readout direction to make an overscan vector, fits a smoothing function to the vector, and then evaluate and then evaluates the smooth function at each readout line or column.

The line-by-line determination provides an mean, median, or mean with the minimum and maximum values excluded. The median is lowest value of the middle two when the number of overscan columns is even rather than the mean.

The smoothed overscan vector determination uses the icfit options including interactive fitting. The fitting function is generally either a constant (polynomial of 1 term) or a high order function which fits the large scale shape of the overscan vector. Bad pixel rejection is also available to eliminate cosmic ray events. The function fitting may be done interactively using the standard icfit iteractive graphical curve fitting tool. Regardless of whether the fit is done interactively, the overscan vector and the fit may be recorded for later review in a metacode plot file named by the parameter ccdred.plotfile. The mean value of the bias function is also recorded in the image header and log file.

Trim: trimming unwanted data

When the parameter trim is set the input image will be trimmed to the image section given by the parameter trimsec. This trim should, of course, be the same as that used for the calibration images.

Zerocor: applying a zero bias calibration

After the readout bias is subtracted, as defined by the overscan or prescan region, there may still be a zero level bias. This level may be two dimensional or one dimensional (the same for every readout line). A zero level calibration is obtained by taking zero length exposures; generally many are taken and combined. To apply this zero level calibration the parameter zerocor is set. In addition if the zero level bias is only readout dependent then the parameter readcor is set to reduce two dimensional zero level images to one dimensional images. The zero level images may be specified by the parameter zero or given in the input image list (provided the CCD image type is defined).

When the zero level image is needed to correct an input image it is checked to see if it has been processed and, if not, it is processed automatically. Processing of zero level images consists of bad pixel replacement, overscan correction, trimming, and averaging to one dimension if the readout correction is specified.

Darkcor: applying a dark count calibration

Dark counts are subtracted by scaling a dark count calibration image to the same exposure time as the input image and subtracting. The exposure time used is the dark time which may be different than the actual integration or exposure time. A dark count calibration image is obtained by taking a very long exposure with the shutter closed; i.e. an exposure with no light reaching the detector. The dark count correction is selected with the parameter darkcor and the dark count calibration image is specified either with the parameter dark or as one of the input images. The dark count image is automatically processed as needed. Processing of dark count images consists of bad pixel replacement, overscan and zero level correction, and trimming.

Flatcor: applying a flat field calibration

The relative detector pixel response is calibrated by dividing by a scaled flat field calibration image. A flat field image is obtained by exposure to a spatially uniform source of light such as an lamp or twilight sky. Flat field images may be corrected for the spectral signature in spectroscopic images (see response and apnormalize), or for illumination effects (see mkillumflat or mkskyflat). For more on flat fields and illumination corrections see flatfields. The flat field response is dependent on the wavelength of light so if different filters or spectroscopic wavelength coverage are used a flat field calibration for each one is required. The different flat fields are automatically selected by a subset parameter (see subsets).

Flat field calibration is selected with the parameter flatcor and the flat field images are specified with the parameter flat or as part of the input image list. The appropriate subset is automatically selected for each input image processed. The flat field image is automatically processed as needed. Processing consists of bad pixel replacement, overscan subtraction, zero level subtraction, dark count subtraction, and trimming. Also if a scan mode is used and the parameter scancor is specified then a scan mode correction is applied (see below). The processing also computes the mean of the flat field image which is used later to scale the flat field before division into the input image. For scan mode flat fields the ramp part is included in computing the mean which will affect the level of images processed with this flat field. Note that there is no check for division by zero in the interest of efficiency. If division by zero does occur a fatal error will occur. The flat field can be fixed by replacing small values using a task such as imreplace or during processing using the minreplace parameter. Note that the minreplace parameter only applies to flat fields processed by ccdproc.

Sflatcor: applying a sky flat field calibration

A sky flat field calibration is just a second flat field derived from data which has been flat fielded by the first flat field. Typically a sky flat field is created from sky data. This is either exposures of the twilight sky or combinations of dark sky observations where objects are eliminated by stacking disregistered exposures. The operation is similar to the primary flat field in that a scaling is determined from the CCDMEAN information in the image or by computing a mean value. The calibration data is scaled and divided into the input data.

Merge: merging amplifiers from the same ccd

When an input file consists of multiple amplifiers from the same CCD they may be merged together into a single image or extension. If the input file has only one CCD then the output is a simple single image otherwise it is a multiextension file with fewer extensions. The image header of the merged output is from the first amplifier encountered for each CCD. For multiextension output the merged extension name will be the extension name of the first amplifier.

If an output mask is specified then the input masks will also be merged. In cases where the masks for the input data are already in merged form, where the masks for all the extensions to be merged are the same mask, the task will not create a new mask.

Examples

The user's guide presents a tutorial in the use of this task.

1. In general all that needs to be done is to set the task parameters and enter

cl> ccdproc *.imh &

This will run in the background and process all images which have not been processed previously.

Time requirements

o SUN-3, 15 MHz 68020 with 68881 floating point hardware (no FPA)
o 8 Mb RAM, 2 Fuji Eagle disks.
o Input images = 544 x 512 short
o Output image = 500 x 500 real
o Operations are overscan subtraction (O), trimming to 500x500 (T),
  zero level subtraction (Z), dark count scaling and subtraction (D),
  and flat field scaling and subtraction (F).
o UNIX statistics
  (user, system, and clock time, and misc. memory and i/o statistics):

[OTF] One calibration image and 9 object images:
No caching:  110.6u 25.5s 3:18 68% 28+ 40K 3093+1645io   9pf+0w
Caching:     111.2u 23.0s 2:59 74% 28+105K 2043+1618io   9pf+0w

[OTZF] Two calibration images and 9 object images:
No caching:  119.2u 29.0s 3:45 65% 28+ 50K 4310+1660io   9pf+0w
Caching:     119.3u 23.0s 3:07 75% 28+124K 2179+1601io   9pf+0w

[OTZDF] Three calibration images and 9 object images:
No caching:  149.4u 31.6s 4:41 64% 28+ 59K 5501+1680io  19pf+0w
Caching:     151.5u 29.0s 4:14 70% 27+227K 2346+1637io 148pf+0w

[OTZF] 2 calibration images and 20 images processed:
No caching:  272.7u 63.8u 8:47 63% 28+ 50K 9598+3713io  12pf+0w
Caching:     271.2u 50.9s 7:00 76% 28+173K 4487+3613io  51pf+0w

Revisions

CCDPROC: MSCRED - V4.5: March 19, 2001: This help page describes the options for the above version of MSCRED.