urlget: Get a (http) URL to a named file

Package: system

Usage

urlget url fname

Parameters

url

The URL to download.

fname

The name of the file to create containing the URL contents. If not specified, the trailing part of the URL is used as the filename.

use_cache = yes

Use the system file cache? If 'yes' and the file already exists in the cache, the cached file will be copied to the output filename. If 'no' then the URL will be downloaded again.

extn = ""

Optional filename extension to put on the cached filename. This can be used to force files to be saved as a particular type in the cache.

verbose = no

Print verbose output?

cache = "cache$"

Logical cache directory name.

Description

The URLGET task is used to download a URL (HTTP protocol only) to a local file named by the fname parameter. If no fname is given, a filename is constructed from the last part of the URL (i.e. characters trailing any of the '?', '/', or '&' delimiters). Because the URL may not point to a static file, use of the fname parameter is recommended.

If the use_cache parameter is set, the URL will only be downloaded if it does not already exist in the file cache pointed to by the cache parameter, otherwise the cached file will be copied to the output filename. The extn parameter may be to used to force a specific file extension to be appended to the filename in the cache, this allows a URL to be passed to a task and treated as if it were a file of a specific type.

Examples

1. Download a FITS image from a URL (these are equivalent):

cl> urlget http://iraf.noao.edu/foo.fits
cl> urlget http://iraf.noao.edu/foo.fits foo.fits

2. Force a URL to be downloaded again:

cl> urlget http://iraf.noao.edu/foo.fits use_cache=no

3. Download a URL with special characters:

    cl> urlget http://iraf.noao.edu/scripts/tget?f=foo.fits
or
    cl> s1 = "http://iraf.noao.edu/scripts/tget?f=foo.fits"
    cl> urlget(s1)
or
    cl> s1 = "http://iraf.noao.edu/scripts/tget?f=foo.fits&d=/iraf/web"
    cl> urlget(s1,"foo.fits",verbose+)

Escaping special characters isn't required from the commandline since the URL is assumed to be whitespace or comma delimited.

Bugs

See also