autowisp.light_curves.correction module

Class Inheritance Diagram

Define base class for all LC de-trend algorithms.

class autowisp.light_curves.correction.Correction(fit_datasets, mark_progress, *, observation_id=None, qc_exclude_file=None, exclusion_rule=None, **iterative_fit_config)

Bases: object

Functionality and interface shared by all LC de-trending corrections.

Variables:

• fit_datasets – See __init__().

• iterative_fit_config (dict) – Configuration to use for iterative fitting. See iterative_fit() for details.

__init__(fit_datasets, mark_progress, *, observation_id=None, qc_exclude_file=None, exclusion_rule=None, **iterative_fit_config)

Configure the fitting.

Parameters:

• fit_datasets ([]) – A list of 3-tuples of pipeline keys corresponding to each variable identifying a dataset to fit and correct, an associated dictionary of path substitutions, and a pipeline key for the output dataset. Configurations of how the fitting was done and the resulting residual and non-rejected points are added to configuration datasets generated by removing the tail of the destination and adding ‘.cfg.’ + <parameter name> for configurations and just ‘.’ + <parameter name> for fitting statistics. For example, if the output dataset key is

• 'shapefit.epd.magnitude'

• look (the configuration datasets will)

• 'shapefit.epd.cfg.fit_terms' (like)

• and

• 'shapefit.epd.residual'.

• observation_id ([str]) – The pipeline keys of the datasets whose values identify an observation. Only needed to match observations across lightcurves or with qc_exclude_file.

• qc_exclude_file (str or None) – The observations to leave out of the fit while still correcting them: one per line, each given by the values of the observation_id datasets separated by white space, blank lines ignored. A line may end with the photometry it applies to, shapefit or ap followed by an aperture index; without one it applies to every photometry. None excludes nothing.

• exclusion_rule (str or None) – The rule that produced qc_exclude_file, recorded with the fit. None if the list was not produced by a rule.

• iterative_fit_config – Any other arguments to pass directly to iterative_fit().

Returns:

None

_find_qc_excluded(light_curve, fit_target, num_points)

Flag the points of a lightcurve to leave out of the fit.

Parameters:

• light_curve (LightCurveFile) – The lightcurve being corrected.

• fit_target (tuple) – The entry of fit_datasets being fit: the pipeline key of the input dataset, the substitutions resolving it, and the pipeline key of the output dataset. The first two give the photometry, and the substitutions also resolve the observation ID datasets.

• num_points (int) – The number of points in the lightcurve.

Returns:

For each point, whether its observation is in the exclusion list for the photometry being fit.

Return type:

numpy.array(dtype=bool)

_fix_substitutions(*, light_curve, photometry_mode, fit_points, substitutions, in_place=False)

Fix magfit iteration in substitutions if negative.

static _get_config_key_prefix(fit_target)

Return the prefix of the pipeline key for storing configuration.

_get_fit_data(light_curve, get_fit_dataset, fit_target, fit_points)

Return the lightcurve points to detrend.

Parameters:

• light_curve (LightCurveFile) – The lightcurve being corrected.

• get_fit_dataset (callable) – See EPDCorrection.__call__().

• fit_target ((str, dict)) – The dataset key and substitutions identifying a unique dataset in the lightcurve to fit.

• fit_points (bool array) – Flags selecting the points to return.

Returns:

The selected points of the dataset to apply the correction to.

numpy.array:

The selected points of the dataset to derive the correction from: a separate array, even when get_fit_dataset returns a single dataset to use for both.

Return type:

numpy.array

_get_io_iterative_fit_config(pipeline_key_prefix)

Return the iterative fit portion of the configuration to save in the LC.

Parameters:

pipeline_key_prefix (str) – The part of the pipeline key specifying which configuration is being defined (i.e. everything except the last item in the key).

Returns:

A list of tuples of the configuration options contained in iterative_fit_config.

Return type:

[()]

_get_observation_ids(light_curve, substitutions)

Return the observation IDs from the given light curve.

static _get_photometry(dset_key, substitutions)

Return the photometry id of a fit dataset, as exclusion lists name it.

Parameters:

• dset_key (str) – The pipeline key of the dataset.

• substitutions (dict) – The substitutions resolving it.

Returns:

shapefit_photometry for the shape fit, and the

aperture index for aperture photometry.

Return type:

int

_is_qc_excluded(observation_ids, dset_key, substitutions)

Flag the observations the exclusion list leaves out of one dataset.

Those listed with no photometry, and those listed under the photometry of the dataset.

Parameters:

• observation_ids (structured array) – The observations to check, as returned by _get_observation_ids().

• dset_key (str) – The pipeline key of the input dataset being fit.

• substitutions (dict) – The substitutions resolving it.

Returns:

For each observation, whether it is in the exclusion list for the photometry of the dataset.

Return type:

numpy.array(dtype=bool)

_match_exclusions(observation_ids, photometry)

Flag the observations listed under one photometry of the list.

Parameters:

• observation_ids (structured array) – The observations to check, as returned by _get_observation_ids().

• photometry (int or None) – The photometry the list names them under, None for the lines naming none.

Returns:

For each observation, whether it is listed there.

Return type:

numpy.array(dtype=bool)

static _process_fit(*, fit_results, raw_values, predictors, fit_index, result, num_extra_predictors)

Incorporate the results of a single fit in final result of __call__().

Parameters:

• fit_results – The return value of the iterative_fit() used to calculate the correction

• raw_values (1-D array) – The data to apply the correction to. Should already exclude any points not selected for correction.

• predictors (2-D array) – The predictors to use for the correction (e.g. the templates for TFA fitting).

• fit_index (int) – The index of the dataset being fit within the list of datasets that will be fit for this lightcurve.

• result – The result variable for the parent update for this fit.

• num_extra_predictors (int) – How many extra predictors are there.

Returns:

The same structure as fit_results, but ready to pass to self._save_result().

Return type:

dict

static _read_exclusions(exclusion_fname, num_id_values)

Return the observations an exclusion list names.

Parameters:

• exclusion_fname (str or None) – See qc_exclude_file argument to __init__().

• num_id_values (int) – How many values identify an observation.

Returns:

{photometry: [(str, ...), ...]}: the values identifying each listed observation, as written, under the photometry id its line names, or under None for a line naming none, which applies to every photometry.

Return type:

dict

_save_result(*, fit_index, corrected_values, fit_residual, non_rejected_points, fit_points, qc_excluded, configuration, light_curve)

Stores the de-treneded results and configuration to the light curve.

Parameters:

• fit_index (int) – The index of the dataset for which a correction was applied in within the list of datasets specified at init.

• corrected_values (array) – The corrected data to save.

• fit_residual (float) – The residual from the fit, calculated as specified at init.

• non_rejected_points (int) – The number of points used in the last iteration of the itaritive fit.

• fit_points (bool array) – Flags indicating for each entry in the input (uncorrected) dataset, whether it is represented in corrected_values.

• qc_excluded (bool array) – Flags indicating for each entry in the input (uncorrected) dataset, whether it was left out of the fit. See _find_qc_excluded().

• configuration ([]) – The configuration used for the fit, properly formatted to be converted to an entry in the configurations argument to LightCurveFile.add_configurations().

• light_curve (LightCurveFile) – A light curve file opened for writing.

Returns:

None

static get_result_dtype(num_photometries, extra_predictors=None, id_size=1)

Return the data type for the result of __call__.