autowisp.data_reduction.data_reduction_file module

Class Inheritance Diagram

Define a class for worknig with data reduction files.

class autowisp.data_reduction.data_reduction_file.DataReductionFile(*args, **kwargs)

Bases: HDF5FileDatabaseStructure

Interface for working with the pipeline data reduction (DR) files.

_product

The pipeline key of the HDF5 product. In this case: ‘data_reduction’

Type:

str

_key_io_tree_to_dr

A dictionary specifying the correspondence between the keys used in astrowisp.IOTree to store quantities and the element key in the DR file.

Type:

dict

_dtype_dr_to_io_tree

A dictionary specifying the correspondence between data types for entries in DR files and data types in astrowisp.IOTree.

Type:

dict

__init__(*args, **kwargs)

Open or create a data reduction file.

Parameters:

• arguments (See HDF5File.__init__() for description of)

• however

• fname (instead of)

• header (a DataReductionFile can be specified by the)

• object (of the frame it corresponds to (or at least a dict-like)

• template). (defining the header keywords required by the DR filename)

classmethod _get_root_tag_name()

The name of the root tag in the layout configuration.

_prepare_source_iter(dataset_key, column_substitution_name, **path_substitutions)

Return required head and tail of paths identifying source collection.

Parameters:

get_sources(). (See)

Returns:

The path to the parent group containing all source columns.

str: The string that must be in the beginning of each path for it

to be considered part of the source collection.

str: The string that must be in the end of each path for it to be

considered part of the source collection.

Return type:

str

add_frame_header(header, **substitutions)

Add the header of the corresponding FITS frame to DR file.

add_hat_astrometry(filenames, configuration, **path_substitutions)

Add astrometry derived by fistar, and anmatch to the DR file.

Parameters:

• filanemes (dict) – The files containing the astrometry results. Should have the following keys: ‘fistar’, ‘trans’, ‘match’, ‘catalogue’.

• configuration – An object with attributes containing the configuraiton of how astormetry was performed.

• path_substitutions – See get_source_count()

Returns:

None

add_magnitude_fitting(*, fitted_magnitudes, fit_statistics, magfit_configuration, missing_indices, **path_substitutions)

Add a magnitude fitting iteration to the DR file.

Parameters:

• fitted_magnitudes (numpy.array) – The differential photometry corrected magnitudes of the sources.

• fit_statistics (dict) – Summary statistics about how the fit went. It should define at least the following keys: initial_src_count, final_src_count, and residual.

• magfit_configuration – The configuration structure with which magnitude fitting was performed.

• missing_indices – A list of indices within the file of sources for which no entries are included in fitted_magnitudes.

Returns:

None

add_sources(data, dataset_key, column_substitution_name, *, parse_ids=False, ascii_columns=(), **path_substitutions)

Creates datasets out of the fields in an array of sources.

Parameters:

• data (structured numpy.array) – The data about the sources to add.

• dataset_key (str) – The pipeline key for the dataset to add.

• column_substitution_name (str) – The %-subsittution variable to distinguish between the column in the array.

• parse_ids (bool) – Should self.parse_hat_source_id() be used to translate string IDs to datasets to insert?

• string_columns ([str]) – A list of column names to convert to ascii strings before saving.

Returns:

None

delete_sources(dataset_key, column_substitution_name, **path_substitutions)

Delete all columns of a given source collection.

fname_template = None

get_dataset_creation_args(dataset_key, **path_substitutions)

See HDF5File.get_dataset_creation_args(), but handle srcextract.

get_dtype(element_key)

Return numpy data type for the element with by the given key.

classmethod get_fname_from_header(header)

Return the filename of the DR file for the given header.

get_frame_header(**substitutions)

Return the header of the corresponding FITS frame.

get_matched_sources(**path_substitutions)

Get combined catalogue and extracted matched sources.

get_num_apertures(**path_substitutions)

Return the number of apertures used for aperture photometry.

get_num_magfit_iterations(**path_substitutions)

Return how many magnitude fitting iterations are in the file.

Parameters:

path_substitutions – See get_source_count().

Returns:

The number of magnitude fitting iterations performed on the set of photometry measurements identified by the path_substitutions argument.

Return type:

int

get_source_count(**path_substitutions)

Return the number of sources for the given tool versions.

Parameters:

path_substitutions – Values to substitute in the paths to the datasets and attributes containing shape fit informaiton (usually versions of various components).

Returns:

The number of projected sources in the databasets reached by the given substitutions.

Return type:

int

get_source_data(*, magfit_iterations='all', shape_fit=True, apphot=True, string_source_ids=True, all_numeric_source_ids=False, background=True, **path_substitutions)

Extract available photometry from the data reduction file.

Parameters:

• magfit_iterations (iterable) – The set of magnitude fitting iterations to include in the result. 0 is the raw photometry (i.e. no magnitude fitting), 1 is single reference frame fit, 1 is the first re-fit etc. Use 'all' to get all iterations. Negative numbers have the same interpretation as python list indices. For example -1 is the final iteration.

• shape_fit (bool) – Should the result include shape fit photometry measurements. If True and no shape fit is present, still excludes shape fit columns.

• apphot (bool) – Should the result include aperture photometry measurements.

• string_source_ids (bool) – Should source IDs be formatted as strings (True) or a set of integers (False)?

• background (bool) – Should the result include information about the background behind the sources?

• path_substitutions – See get_source_count().

Returns:

The photometry information in the current data reduction file.

The columns always included are:

• ID(set as index): an array of sources IDs in the given DR file. Either a string (if string_source_ids) or 1- or 3-column composite index depending on ID type.

• <catalogue quantity> (dtype as needed): one entry for each catalogue column.

• x (numpy.float64): The x coordinates of the sources

• y (numpy.float64): The y coordinates of the sources

The following columns are included if the corresponding input argument is set to True:

• bg (numpy.float64): The background estimates for the sources

• bg_err (numpy.float64): Error estimate for ‘bg’

• bg_npix (numpy.uint): The number of pixel background extraction was based on.

• mag (2-D numpy.float64 array): measured magnitudes. The first dimension is the index within the magfit_iterations argument and the second index iterates over photometry, starting with shape fitting (if the shape_fit argument is True), followed by the aperture photometry measurement for each aperture (if the apphot argument is True).

• mag_err (numpy.float64): Error estimate for mag. Same shape and order.

• phot_flag: The quality flag for the photometry. Same shape and order as mag.

Return type:

pandas.Dataframe

get_source_ids(string_source_ids=True, **path_substitutions)

Return the IDs of the sources in the given DR file.

Parameters:

• string_source_ids – Should source IDs be formatted as strings (True) or a set of integers (False)?

• path_substitutions – See get_source_count().

Returns:

See ID field of result in get_source_data().

Return type:

numpy.array

get_sources(dataset_key, column_substitution_name, **path_substitutions)

Return a collection of sources previously stored in the DR file.

Parameters:

• dataset_key (str) – The pipeline key for the dataset to return.

• column_substitution_name (str) – The %-subsittution variable to distinguish between the column in the array.

Returns:

The keys are the columns of the sources stored and the values are 1-D numpy arrays containing the data.

Return type:

dict

has_shape_fit(accept_zeropsf=True, **path_substitutions)

True iff shape fitting photometry exists for path_substitutions.

parse_hat_source_id(source_id)

Return the prefix ID, field number, and source number.

save_source_extracted_psf_map(*, fit_results, fit_configuration, **path_substitutions)

Create the datasets and attributes holding the fit results.

autowisp.data_reduction.data_reduction_file.init_dr_process(configuration)

Initialize the process with the given configuration.