AipyEnhancementProposal

From AstroBaki
Revision as of 12:02, 15 February 2010 by WikiSysop (talk | contribs) (Created page with '= AIPY Enhancement Proposals (APEPs) = Please add below any enhancements you would like to see made to AIPY. An APEP should consist of: * A summary of the proposed enhancment *…')
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

AIPY Enhancement Proposals (APEPs)[edit]

Please add below any enhancements you would like to see made to AIPY. An APEP should consist of:

  • A summary of the proposed enhancment
  • A detailed description of the change, including:
    • How the feature in question used to work (if applicable)
    • How it will work after the enhancement
    • Why the enhancement is better than what it replaces
  • Whether the described enhancement is currently being worked on, and if so, by whom.

APEP 001: Unit Tests[edit]

Summary[edit]

Unit tests, using the standard Python package [[1]], should be developed for all major methods of AIPY classes. Additionally, a standard dataset and processing pipeline should be developed that exercises the major scripts in AIPY. These enhancements will help ensure that functionality is not broken by future changes, and that an AIPY installation can be certified as fully functional.

Description[edit]

To avoid many of the pitfalls of distributed development, AIPY needs unit tests that ensure core functionality is maintained when source code is modified. PyUnit is a standard Python tool for implementing such unit tests. Unit tests will live separately from in a "tests" subdirectory. Each AIPY class will have a separate file devoted to testing all of the mid- to high-level methods (i.e. all components that are intended to be accessed by something external to the class). Once unit tests are established, any code change should be subjected to these tests before being merged into the primary branch of AIPY. All effort should be made to maintain compatibility with unit tests. Changes that intentionally break a unit test should include a replacement unit test. Such changes will be treated increasingly conservatively as the user base of AIPY expands.

So similarly ensure that functionality is preserved in basic AIPY scripts, a standard UV dataset is proposed to be established using a set of documented parameters to mdlvis.py. This dataset will serve to characterize all AIPY scripts. For each script, a set of parameters will be defined, and the result of processing the standard dataset (or one derived from the standard using documented processing steps) will be hashed and recorded. Several sets of parameters/hashes may be defined for a script to exercise its various components.

AIPY users will be able to ensure they have a full functional installation by running unit tests and by processing the standard dataset with various scripts. A master test script should be written that automatically runs all defined tests for AIPY code. This master test script will be used to verify the compatibility of all subsequent AIPY modifications.

Assigned to: Aaron Parsons[edit]

Status:[edit]

Done for core modules.

APEP 002: Catalogs[edit]

Summary[edit]

AIPY should support multiple catalogs and adding catalogs should be simple (or at least well documented).

Description[edit]

The basic unit is the RadioCatalog returned by the protypical get_catalog, function. AIPY currently leaves this to the user via the loc.get_catalog function. This allows the user to impliment any catalogs within the location file. Catalog functionality should not depend on location. The get_catalog function implimented in Danny's pgb564.py location file supports 3c and Helmboldt with the following features.

  • Entire catalogs may be selected by using identifiers ('3Ca' or 'H')
  • individual source names are paired with their catalogs by matching prefix (eg '3c' vs 'J')

This can probably be implemented by merely moving around the classes and changing the way they are imported.

Assigned to: Aaron Parsons[edit]

Status:[edit]

Done.

APEP 003: Sky Model Database[edit]

Summary[edit]

The sky model of an all-sky telescope must contain hundreds of sources. To account for ionospheric and other variations, these sources must be tracked at varying temporal rates. This model is part of a continually refining self-calibration that is a recursive process. In order to correctly diagnose this loop the entire history of the refinement of the sky model must be accessible. A single database can keep track of all of this without loss of data or need to print/parse/find files.

Description[edit]

Use sqlite and the sqlite3 python API to read and write source records in a database file. One possible table structure is:

name, time_begin,time_end,tolerance,iteration count, str,ndex,dra,ddec,index #

The index # introduced here is a unique key assigned to groups of solutions with idea that one can specify a complete calibration by listing index #s.

Assigned to: Danny Jacobs[edit]

Status:[edit]

To be implemented through cal files and not included explicitly in AIPY.

APEP 004: UVFITS Interface[edit]

Summary[edit]

It's time that AIPY demonstrated interoperability with an existing interferometry package other than MIRIAD. A good starting place for this would be the UVFITS standard used by AIPS. Although this standard may be becoming less used, it is nonetheless an important one for interfacing to legacy data sets.

Description[edit]

The UVFITS format sits as a layer on top of the standard FITS format and so this interface should use a standard FITS reader like pyfits or cfitsio. Included with basic routines for data I/O using UVFITS file and numpy arrays, AIPY should include scripts that perform conversion between all supported UV data formats.

Assigned to: Aaron Parsons[edit]

Status:[edit]

Progress made on PFITS--a python library for reading FITS files that can support the table structure in UVFITS files.

APEP 005: A Refereed AIPY Paper[edit]

Summary[edit]

Write an overview paper about AIPY that users may cite in their own papers.

Description[edit]

A sketch of an outline: "AIPY: a flexible package for synthesis imaging"

  1. Intro -- the need for a new package
  2. Methods -- how implementation was approached, algorithms used, etc.
  3. How-to -- an extremely brief tutorial for several interesting test-cases (maybe with a url pointing to test data-sets and scripts)
  4. Results -- examples of AIPY's use in PAPER data and also, to emphasize its utility to other people, for some publicly available VLA data (say, the FIRST survey, or something)
  5. Summary and Conclusions -- summary and some manifesto points

Assigned to: Aaron Parsons[edit]

Status:[edit]

Pending adoption of SPEAD protocol in AIPY core.