IRAF V2.16 Release Notes
- Authors:
IRAF Group
- Date:
March 22, 2012
- Abstract:
These notes provide a summary of the major changes in the V2.16 release. V2.16 includes a number of bugfixes and new tasks, but is primarily a platform-support release to add new Virtual Observatory (VO) capabilities to the core system. Full integration of VO capabilities requires new functionality to be available in all tasks and not simply in new VO applications. In particular, tasks are now all able to deal with remote data as easily as local disk files, and VO-specific formats are supported as transparently as other long-used table formats. Specifics about the new core capabilities are described in the highlights section below.
These new core capabilities have been fairly extensively tested and are considered stable, however we expect there will no doubt be instances that cause trouble that should be reported. Because bugs in new core capabilities will affect all tasks, a simplified update procedure is also featured in this release to assist in applying patches to the entire system. This follows the earlier model for installing external packages where the following ‘make’ commands in the $iraf directory will suffice to bring a system up to date following a bug-fix release:
make latest # Update entire system (core + extpkgs)
make latest_src # Update only source code
make latest_core # Update only the core iraf system
make check_latest # Check is system is latest released version
We expect that in the months immediately following the initial release there will need to be one or more updates to the entire system.
More detailed technical documentation of all system changes will be
found in the notes.v215
file in the iraf$local directory
,
detailed revisions notes for each application package are in the package
directories in a file called Revisions
, e.g. onedspec$Revisions
.
Highlights of This Release:
64-bit Bug Fixes
A number of 64-bit bugs remaining in the v2.15 release have been fixed, both in specific applications as well as system-level interfaces such as IMIO. As of this writing, there are no known 64-bit issues in the system and we now consider these architectures to be stable.
All users of 64-bit systems are STRONGLY ENCOURAGED TO UPDATE to v2.16 in order to fix these bugs in their installation of IRAF. Specific bugs fixed by this release are given below, detailed changes
“Free” Licenses
Previous releases of IRAF were seen as being “non-free” due to legacy license retrictions in the system. In particular, the NCAR graphics code restricted re-distribution of IRAF by parties other than NOAO, and comments in the XYACC source directory were taken to mean that an original BSD source license was required.
We are happy to report that both restrictions have been removed from the IRAF system: The XYACC code was rewritten prior to the v2.15.1a release and is covered under the OpenSolaris CDDL license. Agreements between NOAO and UCAR were reached to allow the NCAR code in IRAF to (retroactively) be covered by the current UCAR license allowing redistribution.
A new COPYRIGHTS file and the text of the licenses now in use can be found in the $iraf/local directory as well as the distrubution directory on the iraf.noao.edu server.
A new VO-CL
IRAF v2.16 features a modified version of the CL, called VOCL, that provides new builtin functions to support VO task development. This version of the CL is fully backward compatible with the earlier ECL (which remains available) and since it is required for the proper operation of the VO package tasks, will be the version started by default.
In addition to VO data and service integration, the VOCL is able to exchange messages with other desktop applications using the SAMP protocol. Proprietary message types are implemented that allow arbitrary execution of IRAF commands, and which set/get both parameter and environment values. Custom messages may also be defined by users to permit any two IRAF sessions on the same machine to exchange messages. This level of interoperability allows other applications designed for e.g. table or spectrum visualization to be used seemlessly within the IRAF environment.
VO Package
IRAF v2.16 is now distributed with a new VO external package as part of the core release (similar to how the NOAO external package is considered part of the ‘core’). This package provides utility tasks and applications specifically for accessing an analyzing VO data and is meant to provide a toolbox for scientists to incorporate VO data into their reductions.
The VO package is expected to evolve quickly as new tasks and features are added, and due to limited testing, is considered to be only a Beta-quality release at this stage. A detailed listing of tasks in the VO package is given below.
Enhance @-file Capabilities
IRAF has long had the ability to the ‘@-files’ as a mean of passing in lists of images/files to tasks. For v2.16, the concept was expanded to include other things that logically have as lists, e.g. a column in a table of remote data URLS, the implicit list of images in an MEF file, etc.
In these enhanced templates, things like MEF files and tables can be expanded automatically using the @-file operator to let any task that takes an image list process the extensions automagically. Additionally, selection of images based on image header keywrods can also be done dynamically using expressions in the template strings. For example, consider the following templates:
@file* expand all files beginning w/ 'file'
@file//".fits" append ".fits" to contents of 'file'
@mef.fits expand image extensions of an MEF file
@mef.fits[SCI] select SCI extensions from MEF file
@mef.fits[SCI,2][noinherit] select v2 SCI extns, add kernel param
@mef.fits[1-16x2] select range of extensions from MEF file
@mef.fits[+1-8] create a list of extensions for an MEF
*.fits[1:100,1:100] append section to all FITS images
@@file[1:100,1:100] append section to expanded MEFs in file
*.fits[filter?='V'] select images where FILTER contains 'V'
@*.fits[gain==3.0] select extns where GAIN keyword is 3.0
*.fits[filter?='V';gain==2.5] select using multiple OR's expressions
These templates could be used e.g. to run IMSTAT on all the extensions of a FITS file with a single command, as in
Enhanced @-files Old @-files
---------------- -----------
cl> imext mef.fits output=file > list.dat
cl> imstat @mef.fits cl> if (imext.nimages > 0)
>>> imstat @list.dat
cl> del list.dat
where before it would have been necessary to expand the extensions explicitly using a second task into a standard @file. Notice how the other examples above provide even more refined selection of the extensions, either within a single MEF or across multiple images.
These changes are all fully backward compatible but provide new syntax to give users a powerful and compact way to dynamically build image lists for use by all tasks. These new features will be especially useful for script developers tired of managing temp files of image lists, or those who just need to quickly examine MEF files or a directory of images.
System File Cache
The system file cache is used to provide local storage for URL files,
i.e. when a URL is encountered it is downloaded and put into the
cache. If that same URL is used again (e.g. from within a script),
the cached file is passed to the task instead of accessing the URL a
second time. The cache directory used is determined by the ‘cache’
environment variable set in the user’s login.cl file, or by the
hlib$zzsetenv.def
file. Upon login, files in the cache older than
‘cache_age’ days will automatically be removed. This value may be
changed in the login.cl file or the hlib$zzsetenv.def
as needed.
URL Support by all tasks
Because both queries for VO data, and access to remote files/images is typically done using HTTP, the use of a URL in place of a local file/image name is now supported by all tasks. This feature transparently allows use of remote data from archives as input to all tasks (use of URLs for output from tasks is not supported at this time). For example, from the CL commandline
cl> imstat http://iraf.noao.edu/votest/dpix.fits
cl> tinfo http://iraf.noao.edu/votest/usno-b.xml
cl> type http://iraf.noao.edu/index.html
or programmatically from a script such as
s1 = "http://archive.stsci.edu/wuppe/search.php?RA=0.0&DEC=0.0&SR=30."
type (s1)
The file returned by the URL is automatically placed in the system file cache meaning the URL can be used repeatedly without actually downloading the file each time it is used. This can be especially important in scripts where a URL may be constructed or retrieved from a result table and be used multiple times within the script. For example, in the use of ‘s1’ above, the URL is actually a catalog query string that returns a VOTable document; When the TYPE task is called, we are able to execute the query and print the results in a single step as opposed to having a separate download step. Further, because the URL is now cached we can continue to use the ‘s1’ script variable to refer to the result table in other tasks without having to re-execute the query each time.
VOTable Support by all tasks
Part of the core VO support in v2.16 is the ability for all tasks that expect tablular input (e.g anything using the TABLES interfaces) to also now accept the VOTable XML documents used by the VO. When a VOTable is encountered, it is automatically converted internally to a FITS binary table and placed in the system file cache, this cached file is actually what is being manipulated by the task.
In some cases, tasks in the VO external package that are specifically VOTable-aware will be able to use (and expect) the raw XML document directly. In many cases it will be possible to use VOTables as input to VO-aware tasks without having to manage the XML yourself, in other cases common tasks such as extracting a row/column from the table will behave as they always have.
SAMP Support by all tasks
SAMP is the Simple Applications Messaging Protocol, an XML-RPC messaging system for desktop applications developed within the VO. The new VOCL is able to send an receive SAMP messages in order to allow IRAF to interoperate with other VO-enabled applications.
URLGET Task (New!)
A new URLGET task is available in the SYSTEM package that may be used in scripts much as one might use the host WGET task to access a URL (see the help page). In particular, this task is used by the package update mechanism to provide a download capability on systems where e.g. ‘wget’ is not available.
FCACHE Task (New!)
A new FCACHE task is available to manipulate and clean the system file cache. In particular, a command such as
cl> fcache init
can be used to completely re-initialize the file cache when the disk begins to fill or in some cases to remedy problems encountered with use of URLs.
Simplified Build From Source
At the request of users, a toplevel ‘Makefile’ is now available for building or configuring the system with a single command. This Makefile is a simple driver for scripts that do all the work using conventional IRAF commands. Allowed ‘make’ command targets include:
all alias for 'update'
sysgen do a complete sysgen
update update system since last sysgen
updatex update with debugging flags enabled
src clean system of current binaries
clean clean system of current binaries
pristine clean system of all binaries
tables compile the TABLES package
noao compile the NOAO package
summary print core/noao/tables spool file summaries
showarch show currently configure arch
<arch> reconfigure for named architecture
latest Update entire system to latest patch release
latest_src Update only source code
latest_core Update only the core iraf system
check_latest Check is system is latest released version
Simplified Download/Install Process
IRAF systems now come in single-file downloads for the most common configurations:
iraf-src.tar.gz Source code only
iraf-all.tar.gz Source + all supported platform binaries
iraf-linux.tar.gz Source + linux/linux64 platform binaries
iraf-macosx.tar.gz Source + macosx/macintel platform binaries
iraf.lnux.x86.tar.gz Source + 32-bit Linux binaries
iraf.lnux.x86_64.tar.gz Source + 64-bit Linux binaries
iraf.macx.uni.tar.gz Source + 32-bit OSX binaries
iraf.macx.x86_64.tar.gz Source + 64-bit OSX binaries
With these distributions files, installation is a simple matter of unpacking the tarball in the desired IRAF root directory (typically some place like /iraf/iraf) and running the install script. The traditional multi-part distribution files also remain available.
Simplified External Package Installation
External packages may be installed and defined dynamically and no longer need to be manually configured. The $iraf/exern directory contains a ‘configure’ script to create the files needed for a single-command installation of an external package (and it’s dependencies). For example,
% cd $iraf/extern # go to extern directory
% ./configure # configure system (only once)
% make mscred # install the MSCRED package
Packages defined in this way will be available the next time you login to the system. See the $iraf/extern/README file for details.
Improved Documentation
Thanks to Jason Quinn for continued suggestions on improved wording of help pages and careful proofreading of the docs.
Platform Support:
IRAF V2.16 supports the following platforms:
PC-IRAF
32-bit Linux (LNUX.X86)
32-bit Mac OS X 10.4 and higher (ppc and intel) (MACX.UNI)
64-bit Linux (LNUX.X86_64)
64-bit Mac OS X 10.4 and higher (intel) (MACX.X86_64)
Note that PC platforms not mentioned here specifically may still be supported by one or more of the distributions (e.g. Ubuntu can use LNUX).
CORE IRAF REVISIONS SUMMARY
This section describes changes to tasks in the IRAF core system other than routine bug fixes.
New Tasks
VO-CL: A new version of the CL call ‘vocl’ is now the default CL on startup. This version supports both SAMP messaging to interopate with other VO applications, as well as new builtin functions to query VO data and services.
system.urlget: Native IRAF task to retrieve and HTTP URL
system.fcache: List, clean or manipulate the system file cache
Existing Tasks with New Parameters or New Parameter Defaults
images.imcoords.ccmap: A new option “tweak” was added to the values for the “refpoint” parameter to allow controlling whether to tweak the input tangent point.
images.imcoords.ccmap: New parameters xref and yref can be set to a value or header keyword in order to constrain the solution to the specified reference pixel.
Existing Tasks with New Capabilities
All Tasks: Now able to use an HTTP URL in places where an input image/file is expected.
All Tasks Using Lists: Support for enhanced @-files
All Tasks Expecting Tabular Input: Now able to use VOTable XML documents
images.imcoords.ccdmap: Changes to allow constraining WCS solutions to specified tangent point parameters (reference pixel and reference coordinate). This adds parameters so potentially requires users to update scripts.
NOAO PACKAGE REVISIONS SUMMARY
This section describes changes to tasks in the NOAO package tasks other than routine bug fixes.
New NOAO Package Tasks
noao.onedspec:
hirescal - Apply HIRES wavelengths to flux data to produce an IRAF file.
Existing Packages and Tasks with New Parameters or New Parameter Defaults
N/A
Existing Tasks with New Capabilities
artdata.mkobjects: Added Sersic model profiles. This was done using only the model name to avoid any additional parameters.
astutil.observatory: Added entries for SOAR, Baker Observatory, McDonald Obs. HET, the Jack C. Davis Observatory and Langkawai National Observatory
onedspec/splot: Added overplotting of individual components to deblending code.
VO PACKAGE REVISIONS SUMMARY
New VO Package Tasks
The VO package is new to the IRAF v2.16 release. Current contents of the package include:
Toplevel apps:
registry: Query the VO Registry
Toolbox sub-package:
Data Query/Access Tools
getcat: Query catalog data services in the VO
getimg: Query image data services in the VO
getspec: Query spectral data services in the VO (NYI)
getlines: Query spectral line data services in the VO (NYI)
vodata: General purpose query of VO data service
Image Utilities
dss: Display a DSS2 image of a named field
imgcat: Create a catalog of detections in an image
wcsinfo: Summarize the WCS information of an image
dispname: Get the currently displayed image name
VO Service Tools
sesame: Resolve an object name to a position
Simple Catalog Tools
nedoverlay: Overlay NED objects in the image display
obslogoverlay: Overlay an observation catalog (HST, XMM, etc)
radiooverlay: Overlay NVSS radio contours in the image display
xrayoverlay: Overlay RASS3 X-Ray contours in the image display
Registry Tools
mkregdb - Create a local VO Registry database
regdb - Manage/Query a local VO Registry database
regmetalist - List the metadata fields of a Registry record
Votable Utility Tools
votcopy: Copy a VOTable to another format
votget: Download data referenced in a VOTable
votpos: Extract the main positional columns from a VOTable
votsize: Get the size of a VOTable
Table Utilities
colbyid: Identify VOTable column by ID attribute
colbyucd: Identify VOTable column by UCD attribute
colbyname: Identify VOTable column by NAME attribute
tabclip: Clip a table to given boundaries
taboverlay: General table overlay in the image display
External Applications
aladin: Start/Stop/Status of the Aladin image display application
hub: Start/Stop/Status of the SAMP Hub
topcat: Start/Stop/Status of the TOPCAT table display application
SAMP Message Handlers
overhandler: Default SAMP handler for image overlays
tblhandler: Default SAMP handler for table loading messages
imghandler: Default SAMP handler for image loading messages
Hidden Tasks
qstring: Generate a query string URL
makewcs: Create an IRAF WCS from a plate solution
prettystr: Pretty-print a long string
Existing Packages and Tasks with New Parameters or New Parameter Defaults
N/A
Existing Tasks with New Capabilities
N/A
BUG LOGS FIXED BY THIS RELEASE
The following buglog entries are fixed by the this V2.16 release:
NUMBER: 574
MODULE: daophot.psf
SYSTEM: -V2.14
DATE: Tue Apr 13 22:01:52 MST 2010
FROM: fitz
BUG: The ":function" command was not properly saving the new function
when refitting is done with the 'f' keystroke. This is because the
fitting function reinitializes the parameters to the startup
values without first saving the modified function.
STATUS: Fixed for the next release
NUMBER: 575
MODULE: all tasks using the icfit tools
SYSTEM: - V2.14
DATE: Mon Jun 28 14:08:48 MST 2010
FROM: valdes
BUG: The icfit tools are used in many tasks involving 1D function fitting.
These include onedspec tasks like continuum and identify. The
tools provide for a grow radius where any sigma rejected points
have neighbors also rejected. The logic was wrong
in two ways; one where if a neighbor was also a rejected point
it did not also reject neighbors of that point, and another where
the grow radius units were used both as in pixels and in user
coordinates. In reality the grow is supposed to be in user
coordinate units. In addition some tasks, like continuum, incorrectly
described the units adding to the confusion.
STATUS: Fixed for the next IRAF release.
NUMBER: 576
MODULE: imcombine
SYSTEM: V2.14
DATE: Wed Nov 17 15:20:16 MST 2010
FROM: valdes
BUG: The addition of the image names using imcmb="$I" does not work for
input images with a square bracket; e.g. foo[1], foo[im1], foo[*,*].
The IMCMB value, in order to allow long filenames, is stripped of
any path. For an obscure reason related to VMS directories this
code failed to find a rootname.
STATUS: This has been fixed for the next release.
NUMBER: 577
MODULE: dohydra, dofibers, doargus, do3fiber
SYSTEM: -V2.15.1
DATE: Fri Feb 11 12:30:46 MST 2011
FROM: valdes
BUG: The tasks will shorten root input image names to six characters by
using the first five and last characters. Depending on the style
of image names this can result in name conflicts. The reason for
this shortening is no longer known so it is now considered a bug.
Workarounds are to be aware of this and rename image names to avoid
conflicts.
STATUS: This is fixed in the next release. The fix is to modify the file
$iraf/noao/imred/src/fibers/proc.cl as shown (replace lines 125 to
129 with "extn = extn // ".ms"). If you don't have permission
to make this change then copy the file to your iraf "home$"
directory, edit it, load the desired package, and then override
the definition of the file with "redefine proc = home$proc.cl".
125,129c125
< i = strlen (extn)
< if (i < 7)
< extn = extn // ".ms"
< else
< extn = substr (extn, 1, 5) // substr (extn, i, i) // ".ms"
---
> extn = extn // ".ms"
NUMBER: 578
MODULE: splot, scombine, fxcor, identify tasks, dispcor, disptrans
SYSTEM: v2.15-V2.15.1a (64-bit platforms only)
DATE: Tue Mar 8 21:57:38 MST 2011
FROM: fitz
BUG: The 64-bit port changes to smw.h improperly added a P2R() macro to
the APLOW/APHIGH macro declarations. This was causing tasks with
2-D data to make an out-of-bounds request for data and leading to
and error message such as
ERROR: Pixel subscript out of bounds (spec.fits)
Normal onedspec data or use on 32-bit platforms is not affected.
STATUS: Fixed for the next release. A re-application of the v2.15.1a patch
file will replace the affected binaries on 'linux64' and 'macintel'
platforms.
NUMBER: 579
MODULE: onedspec.specplot
SYSTEM: V2.15
DATE: Thu Mar 31 10:41:56 MST 2011
FROM: fitz
BUG: SPECPLOT can sometimes throw a segmentation violation or not
recognize valid input spectra due to an incorrect macro definition
on 64-bit platforms (linux64 and macintel only).
STATUS: This has been fixed for the next release. Patched x_onedspec.e
binaries can be installed from
ftp://iraf.noao.edu/iraf/v215/support/<arch>/x_onedspec.e
where the <arch> is either 'linux64' or 'macintel'.
NUMBER: 580
MODULE: imcombine and variants
SYSTEM: -V2.15.1
DATE: Fri Apr 1 10:53:41 MST 2011
FROM: valdes
BUG: When the grow options is used with masks or partially overlapping
data a segmentation could occur. This is because when data is
absent (because of non-overlap) or excluded (because of mask) an
identifier value was not initialized. The only workaround is to
not use the grow options.
STATUS: Fixed for future patches and releases.
NUMBER: 581
MODULE: splot
SYSTEM: V2.15-
DATE: Mon Jun 6 17:21:27 MST 2011
FROM: valdes
BUG: When using the deblending options a memory free error occurs with
64-bit versions. This is caused by allocating an integer array and
freeing it as a real array.
STATUS: Fixed in future patches and releases.
NUMBER: 582
MODULE: utilities.curfit
SYSTEM: -V2.15.1
DATE: Fri Jul 29 12:40:08 MST 2011
FROM: valdes
BUG: For input data with two or more values having the same x value
there is an arithmetic exception when setting the niterate parameter
greater than zero during interactive fitting. This occurs because
a check for the distance between two points for the purpose of the
grow option divides by the distance. This is done even if no growing
is requested (grow=0). The workaround is to edit the input so that
the values are not exactly the same.
STATUS: This condition has been eliminated for the next release.
NUMBER: 583
MODULE: apall,
SYSTEM: V2.15
DATE: Mon Mar 5 08:51:03 MST 2012
FROM: valdes
BUG: The optimal extraction for significantly tilted spectra, the Marsh
algorithm, has bug which manifests only under 64-bit architectures.
The symptom is a crash, usually a memory or segmentation panic.
The only workarounds are 1) go to an 32-bit system or 2) don't
use the optimal extraction option.
STATUS: Fixed for V2.16.