astropy:docs

What’s New in Astropy 0.3?

Overview

Astropy 0.3 is a major release that adds significant new functionality since the 0.2.x series of releases. This release sees the addition of the Modeling sub-package, which provides an initial framework for fitting models to data, and the Virtual Observatory sub-package, which implements cone searches. In addition, there have been many improvements to the existing sub-packages, such as a much faster implementation of quantities for Numpy arrays in the Units and Quantities sub-package, a better integration of units and quantities in other sub-packages (such as the Cosmology sub-package), new Table functionality related to joining and aggregation, support for arrays in the Coordinates sub-package, and a re-structuring of the Convolution code to provide many build-in parameterized kernels. We describe the new sub-packages and the main improvements below.

For a detailed list of all the changes, including deprecated features and backward-incompatible changes, please take a look at the full Full Changelog.

Modeling

The Models And Fitting (astropy.modeling) sub-package defines many built-in 1-D and 2-D models and provides an extensible framework for fitting these to data. The following example demonstrates how to easily define a Gaussian model and fit it to a previously defined dataset:

import numpy as np
from astropy.modeling import models, fitting

# Generate fake data
np.random.seed(0)
x = np.linspace(-5., 5., 200)
y = 3 * np.exp(-0.5 * (x - 1.3)**2 / 0.8**2)
y += np.random.normal(0., 0.2, x.shape)

# Fit the data
g_init = models.Gaussian1D(amplitude=1., mean=0, stddev=1.)
f2 = fitting.NonLinearLSQFitter()
g = f2(g_init, x, y)

# Plot the results
plt.figure(figsize=(8,5))
plt.plot(x, y, 'ko')
plt.plot(x, g(x), 'r-', lw=2)
plt.xlabel('Position')
plt.ylabel('Flux')

(Source code, png, hires.png, pdf)

../_images/0-3-1.png

Note

Since this is a new sub-package, it should be considered experimental and will be significantly improved in future. In particular, the ability to fit composite models (such as the sum of two models) will be added in the next major version (in the mean time, users can define their own models for such cases). More complete examples and a full description of the sub-package is provided in Models And Fitting (astropy.modeling).

Virtual Observatory

The new Virtual Observatory Access (astropy.vo) sub-package currently provides the ability to execute cone searches. To see a list of available catalogs, you can first import the cone search module:

>>> from astropy.vo.client import conesearch

then make use of the list_catalogs() function:

>>> conesearch.list_catalogs()
[u'Guide Star Catalog 2.3 1',
 u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 1',
 u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 2',
 u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 3',
 u'SDSS DR7 - Sloan Digital Sky Survey Data Release 7 4',
 u'SDSS DR8 - Sloan Digital Sky Survey Data Release 8 1',
 u'SDSS DR8 - Sloan Digital Sky Survey Data Release 8 2',
 u'The HST Guide Star Catalog, Version 1.1 (Lasker+ 1992) 1',
 u'The HST Guide Star Catalog, Version 1.2 (Lasker+ 1996) 1',
 u'The HST Guide Star Catalog, Version GSC-ACT (Lasker+ 1996-99) 1',
 u'The PMM USNO-A1.0 Catalogue (Monet 1997) 1',
 u'The USNO-A2.0 Catalogue (Monet+ 1998) 1',
 u'Two Micron All Sky Survey (2MASS) 1',
 u'Two Micron All Sky Survey (2MASS) 2',
 u'USNO-A2 Catalogue 1',
 u'USNO-A2.0 1']

The following example shows how to execute a cone search for a radius of 0.1 degrees around M31 from the 2MASS catalog. First, we can extract the coordinates for M31 from SIMBAD:

>>> from astropy import coordinates as coords
>>> c = coords.ICRS.from_name('M31')
>>> c
<ICRS RA=10.68471 deg, Dec=41.26875 deg>

and we can then execute the query:

>>> from astropy import units as u
>>> twomass = 'Two Micron All Sky Survey (2MASS) 1'
>>> result = conesearch.conesearch(c, 0.1 * u.degree, catalog_db=twomass)
Trying http://wfaudata.roe.ac.uk/twomass-dsa/DirectCone?DSACAT=TWOMASS&...
Downloading ...

The result is returned as a astropy.io.votable.tree.Table instance (note that this is different from the generic astropy.table.table.Table class):

>>> result.url
u'http://wfaudata.roe.ac.uk/twomass-dsa/DirectCone?DSACAT=TWOMASS&DSATAB=twomass_psc&'
>>> result.array.size
2008
>>> result.array['ra']
masked_array(data = [10.620983 10.672264 10.651166 ..., 10.805599],
             mask = [False False False ..., False],
       fill_value = 1e+20)

Converting to an astropy.table.table.Table is straightforward:

>>> table = result.to_table()
>>> print(table)
      cx             cy             cz       ... coadd_key coadd
-------------- -------------- -------------- ... --------- -----
0.739345466303 0.138832922929 0.658857876152 ...   1590591    33
 0.73937055914 0.138481636014 0.658903644522 ...    577809    33
0.739301283105 0.138762751963 0.658922234764 ...   1590591    33
           ...            ...            ... ...       ...   ...
0.739389950758 0.139376220174 0.658693229028 ...   1590591    33
0.739099087292 0.140726588258 0.658732545516 ...   1590631   232
0.737999550267 0.140950272324 0.659916422388 ...   1590632   244

Units and Quantities

Quantity instances in the Units and Quantities (astropy.units) sub-package are now fully-fledged Numpy arrays, and common Numpy functions (such as numpy.mean, numpy.cos, numpy.log10, etc.) will now correctly treat the units:

>>> q = np.array([1., 2., 3., 4.]) * u.m / u.s
>>> np.mean(q)
<Quantity 2.5 m / s>
>>> np.std(q)
<Quantity 1.11803398875 m / s>

This includes functions that only accept specific units such as angles:

>>> q = 30. * u.deg
>>> np.sin(q)
<Quantity 0.5 >

or dimensionless quantities:

>>> nu = 3 * u.GHz
>>> T = 30 * u.K
>>> np.exp(- h * nu / (k_B * T))
<Quantity 0.995212254619 >

Note

Not all Numpy functions (in particular non-ufuncs) and functions outside Numpy will treat units correctly, so be aware that units may be implicitly dropped without a warning. Always check that the function you are using treats the units correctly.

Another change is that imperial units are not enabled by default. To enable them, use:

>>> from astropy.units import imperial
>>> imperial.enable()

Coordinates

Coordinate objects from the Astronomical Coordinate Systems (astropy.coordinates) sub-package can now store arrays of coordinates instead of just a single coordinate. This dramatically speeds up coordinate conversions when many coordinates are used. The following example shows how one can combine this with functionality from the Table class to read in arrays of coordinates and convert them to a different coordinate frame. First, we can read in a table which contains coordinates both in decimal and string form:

>>> from astropy.table import Table
>>> t = Table.read('2mass.tbl', format='ascii.ipac')
>>> print(t)
    ra        dec         sra           sdec     ...  h_k   j_k
---------- ---------- ------------ ------------- ... ----- -----
274.429506 -13.870547 18h17m43.08s -13d52m13.97s ... 0.791 3.048
274.423821  -13.86974 18h17m41.72s -13d52m11.06s ... 0.867 3.034
274.424587 -13.739629 18h17m41.90s -13d44m22.66s ...  0.94    --
       ...        ...          ...           ... ...   ...   ...
274.870009 -13.817775 18h19m28.80s -13d49m03.99s ... 1.557    --
274.735323 -13.941575 18h18m56.48s -13d56m29.67s ...    --    --
274.866294 -13.841778 18h19m27.91s -13d50m30.40s ... 1.146    --

Then we can create the coordinate object either with the decimal floating-point values:

>>> from astropy import units as u
>>> from astropy import coordinates as coords
>>> c = coords.FK5(t['ra'], t['dec'], unit=(u.deg, u.deg))
>>> c[0]
<FK5 RA=274.42951 deg, Dec=-13.87055 deg>
>>> c[1]
<FK5 RA=274.42382 deg, Dec=-13.86974 deg>

or the string values:

>>> c = coords.FK5(t['sra'], t['sdec'], unit=(u.deg, u.deg))
>>> c[0]
<FK5 RA=274.42950 deg, Dec=-13.87055 deg>
>>> c[1]
<FK5 RA=274.42383 deg, Dec=-13.86974 deg>

Note that in the second case, the initialization will be slower due to the parsing of the strings. These coordinates can then easily be converted to other frames:

>>> cgal = c.transform_to(coords.Galactic)
>>> cgal[0]
<Galactic l=16.77345 deg, b=0.99446 deg>
>>> cgal[1]
<Galactic l=16.77155 deg, b=0.99968 deg>

For coordinate arrays, accessing attributes such as ra or dec will now return Numpy arrays.

As shown above, all the coordinate classes have now been renamed to drop the Coordinates suffix (e.g. ICRS instead of ICRSCoordinates). In addition, HorizontalCoordinates has now been renamed to AltAz.

Coordinate objects now support matching one set of coordinates to another using the array coordinates functionality described above. See match_to_catalog_sky and match_to_catalog_3d for more details.

Finally, coordinate objects now have a to_string method that allows easy conversion to string representations. For example, in the case of the previous coordinates used above:

>>> c.to_string()
[u'18h17m43.08s -13d52m13.97s',
 u'18h17m41.72s -13d52m11.06s',
 ...
 u'18h18m56.48s -13d56m29.67s',
 u'18h19m27.91s -13d50m30.4s']

Table

In addition to many bug fixes and usability improvements, the key new feature in the Data Tables (astropy.table) sub-package is the addition of high level Table operations that can be used to generate a new table from one or more input tables:

Documentation Description Function
Grouped operations Group tables and columns by keys group_by
Stack vertically Concatenate input tables along rows vstack
Stack horizontally Concatenate input tables along columns hstack
Join Database-style join of two tables join

Grouping is a useful concept that allows you to divide a table into sub-groups based on certain key values and create new tables based on computed properties of those sub-groups. As an example, if you have a table containing photometric observations of multiple sources over multiple epochs, it would be possible to compute a mean magnitude for each unique object. In addition to this aggregation operation, the grouping interface also allows filtering operations where certain groups are excluded from the resultant table.

Even more powerful is the ability to do database-style joins of tables. For instance, if you have distinct tables with photometry in different wavebands for a set of objects, you join these points into a single table with one row for each source.

Time

The Time and Dates (astropy.time) sub-package has received attention in filling in the details from the initial release in astropy 0.2. This includes adding array indexing and supporting various arithmetic operations involving arrays, constants, and Quantity objects with time units. In addition the initial infrastructure was added to allow use of International Earth Rotation and Reference Systems Service tables so that automatic calculation of UT1 becomes possible.

One very significant improvement is an overhaul of the internal time manipulations so that arithmetic with Time and TimeDelta objects maintain sub-nanosecond precision over a time span longer than the age of the universe. This is done by carefully managing the way the time is represented and manipulated using two 64-bit floats.

Finally, three new time formats were added:

  • datetime: standard library datetime.datetime objects.
  • plot_date: dates compatible with the matplotlib.pyplot.plot_date function.
  • gps: seconds since 1980-01-01 00:00:00 UTC including leap seconds.

ASCII Tables

The ASCII Tables (astropy.io.ascii) sub-package now includes functionality to write IPAC format tables.

The read() and write() functions now allow a format keyword argument for specifying the file format as a string. This replaces the deprecated Reader and Writer keywords which required supplying a fully-qualified class type. To convert existing code that uses Reader or Writer, change the class name to all lower case with underscores between words. For instance:

>>> from astropy.io import ascii
>>> data = [' name         age ',
            '-----------   ----',
            'Jane Doe       31 ',
            'John Smith     45 ']
>>> t = ascii.read(data, Reader=ascii.FixedWidthTwoLine)  # OLD
>>> t = ascii.read(data, format='fixed_width_two_line')  # NEW

Unified File Read/Write Interface

FITS format tables can now be read and written via the Unified file read/write interface. All of the ASCII table formats are now supported as well. When using the unified file interface for ASCII tables the format defined in the ASCII Tables (astropy.io.ascii) package is prefixed with 'ascii.'. Thus the previous example would be written:

>>> from astropy.table import Table
>>> t = Table.read(data, format='ascii.fixed_width_two_line')

Some formats such as cds or latex will work without the 'ascii.' prefix but this is deprecated and will be removed in the next major release.

The full list of available formats is now available via the get_formats function:

>>> from astropy.io import registry
>>> print registry.get_formats()
Data class    Format    Read Write Auto-identify Deprecated
---------- ------------ ---- ----- ------------- ----------
     Table        ascii  Yes   Yes            No
     Table ascii.aastex  Yes   Yes            No
     Table  ascii.basic  Yes   Yes            No
     Table    ascii.cds  Yes    No            No
       ...          ...  ...   ...           ...        ...
     Table      daophot  Yes    No            No        Yes
     Table         ipac  Yes   Yes            No        Yes
     Table        latex  Yes   Yes            No        Yes
     Table          rdb  Yes   Yes            No        Yes

If you make a mistake and specify an unavailable or incorrect format, the error message will now list all the available formats.

Convolution

The convolution functionality that was originally included in the N-dimensional datasets (astropy.nddata) sub-package has now been moved to the new Convolution and filtering (astropy.convolution) sub-package. As part of a Google Summer of Code project it has been refactored to include a framework that provides common built-in kernels:

import numpy as np
from astropy.convolution import convolve, Gaussian2DKernel

# Generate data
np.random.seed(0)
image = np.random.random((128, 128))

# Create kernel
g = Gaussian2DKernel(width=1)

# Convolve data
image_new = convolve(image, g, boundary='extend')

# Plot the results
plt.figure(figsize=(8,3))
plt.subplot(1,2,1)
plt.imshow(image, interpolation='none', origin='lower', vmin=0., vmax=1.)
plt.title('Reference')
plt.subplot(1,2,2)
plt.imshow(image_new, interpolation='none', origin='lower', vmin=0., vmax=1.)
plt.title('Convolved')

(Source code, png, hires.png, pdf)

../_images/0-3-2.png

A number of different 1-D and 2-D kernels are provided, based on models defined in Models And Fitting (astropy.modeling). The discretization of the kernels can be handled in various ways (e.g. oversampling, interpolation, etc.) which are described in more detail in Convolution and filtering (astropy.convolution).

Cosmology

The Cosmological Calculations (astropy.cosmology) sub-package now includes support for including massive neutrinos in the cosmology classes, and the Planck 2013 cosmology has been updated to use this. In addition, Quantity objects are now used wherever appropriate:

>>> from astropy.cosmology import WMAP9
>>> WMAP9.H0
<Quantity 69.32 km / (Mpc s)>
>>> WMAP9.lookback_time(3)
<Quantity 11.590618401420071 Gyr>
>>> WMAP9.luminosity_distance(3)
<Quantity 26015.607762091513 Mpc>

Statistics

The Astrostatistics Tools (astropy.stats) sub-package includes a number of new common statistical functions, for example related to binomial statistics and bootstrapping.

WCS

When reading FITS headers with the World Coordinate System (astropy.wcs) sub-package, warnings will now be displayed about any non-standard WCS keywords that were fixed to become standard compliant.

For users who have Scipy installed, the WCS class features a new method all_world2pix() for converting from world coordinates to pixel space including the inversion of astrometric distortion corrections.

The included version of wcslib has been upgraded to version 4.19. The relevant changes for astropy users are:

  • Implemented the butterfly projection (XPH), being the polar form of the HEALPix projection with (H,K) = (4,3).
  • Bug fix in celfix() when translating GLS to SFL with non-zero reference point.
  • A number of memory handling and stability fixes.

VO Tables

The VOTable XML handling (astropy.io.votable) sub-package now includes support for the VOTable 1.3 proposed recommendation. Notably, this includes a new binary representation that supports masking of any data type.

Logger

The Astropy logger will now no longer log exceptions by default, and will also no longer log any warning emitted outside of Astropy. In addition, logging to the Astropy log file (located at ~/.astropy/config/astropy.log by default on MacOS X and Linux) has also been disabled by default. This functionality is automatically disabled for new users, but in order to see the new default behavior, previous users of Astropy will need to edit the Astropy configuration file (located at ~/.astropy/config/astropy.cfg by default on MaxOS X and Linux) and change the following two lines as follows:

# Whether to log exceptions before raising them
log_exceptions = False

# Whether to always log messages to a log file
log_to_file = False

Deprecation and backward-incompatible changes

For a full-list of deprecated features and backward-incompatible changes, please take a look at the full Full Changelog.

In Python 2.7 and above, deprecation warnings are disabled by default. If you want to make sure you see these warnings, you can run your Python scripts with:

$ python -Wd script.py

In addition to deprecation warnings, Astropy will also raise warnings (by default) about changes that are not backward-compatible. These can be disabled by doing:

import warnings
from astropy.utils.exceptions import AstropyBackwardsIncompatibleChangeWarning
warnings.simplefilter('ignore', AstropyBackwardsIncompatibleChangeWarning)