autowisp.magnitude_fitting.base module

Class Inheritance Diagram

Magnitude fitting interface.

class autowisp.magnitude_fitting.base.MagnitudeFit(*, reference, config, magfit_collector=None, source_name_format)

Bases: ABC

A base class for all classes doing magnitude fitting.

Takes care of adding fitted magnitudes to data reduction files and updating the database.

config

See config argument to __init__().

logger

A python logging logger for emitting messages on the progress and status of magnitude fitting.

_dr_fname

The name of the data reduction file currently undergoing magnitude fitting.

_magfit_collector

See magfit_collector argument to __init__().

_reference

See reference argument to __init__().

_catalogue

See master_catalogue argument to __init__().

_source_name_format

See source_name_format argument to __init__().

__call__(dr_fname, mark_start, mark_end, **dr_path_substitutions)

Performs the fit for the latest magfit iteration for a single frame.

Parameters:

• dr_fname – The name of the data reductio file to fit.

• dr_path_substitutions – See path_substitutions argument to DataReduction.get_source_data().

Returns:

The non-rejected photometry for the frame.

array:

The magfit corrected non-rejected photometry for the frame.

Return type:

array

__init__(*, reference, config, magfit_collector=None, source_name_format)

Initializes a magnditude fitting object.

Parameters:

• reference (dict) – the reference against which fitting is done. Should be indexed by source and contain entries implementing the dict interface with keys ‘mag’, ‘mag_err’ and optionally ‘x’ and ‘y’ if the sub-pixel position of the source in the reference is to be used in magnitude fitting.

• #master_catalogue (pandas.DataFrame) – should be indexed by source

• information. (# id and contain relevant catalog)

• config –

  An object with attributes configuring how to perform magnitude fitting. It should provide at least the following attributes:

  • fit_source_condition: An expression involving catalogue, reference and/or photometry variables which evaluates to zero if a source should be excluded and any non-zero value if it should be included in the magnitude fit.

  • reference_subpix: Should the magnitude fitting correction depend on the sub-pixel position of the source in the reference frame.

  • grouping: An expressions using catalogue, and/or photometry variables which evaluates to a tuple of boolean values. Each distinct tuple defines a separate fitting group (i.e. a group of sources which participate in magnitude fitting together, excluding sources belonging to other groups).

• magfit_collector (MasterPhotrefCollector) – Object collecting fitted magnitedes for generating statistics of the scatter after magnitude fitting.

_add_fit_to_db(coefficients, **fit_diagnostics)

Record the given best fit coefficient and diagnostics in the database.

Parameters:

• coefficients (iterable of values) – The best fit coefficients for the magnitude fitting of the current data reduction file.

• fit_diagnostics – Any information about the fit that should be recorded in the database. The names of the arguments are assumed to correspond to column names in the magnitude fitting table.

Returns:

None

For now disabled.

Code from HATpipe:

def _update_db(self,
               values,
               apind,
               fit_res,
               start_src_count,
               final_src_count):

    if self.database is None : return
    args=((self._header['STID'],
           self._header['FNUM'],
           self._header['CMPOS'],
           self._header['PROJID'],
           self._header['SPRID'],
           apind,
           self.config.version,
           start_src_count,
           final_src_count,
           (None if fit_res is None else float(fit_res)))
          +
          tuple((None if v is None else float(v)) for v in values))
    statement=('REPLACE INTO `'+self.config.dest_table+
               '` (`station_id`, `fnum`, `cmpos`, `project_id`, '
               '`sphotref_id`, `aperture`, `magfit_version`, '
               '`input_src`, `non_rej_src`, `rms_residuals`, `'+
               '`, `'.join(self._db_columns)+'`) VALUES (%s'+
               ', %s'*(len(args)-1)+')')
    self._log_to_file(
        'Inserting into DB:\n'
        +
        '\t' + statement + '\n'
        +
        '\targs: ' + repr(args) + '\n'
        +
        '\targ types: ' + repr([type(v) for v in args]) + '\n'
    )
    self.database(statement, args)

abstractmethod _apply_fit(phot, fit_results)

Return corrected magnitudes using best fit magfit coefficients.

Parameters:

• phot – The current photometry being fit, including catalogue information.

• fit_results – The best fit parameters derived using _fit().

Returns:

The magnitude fit corrected magnitudes.

Return type:

numpy.array (number sources x number photometry methods)

static _combine_fit_statistics(fit_results)

Combine the statistics summarizing how the fit went from all groups.

Properly combines values from the individual group fits into single numbers for each photometry method. The quantities processed are: residual, initial_src_count, final_src_count.

Parameters:

fit_results – The best fit results for this photometry.

Returns:

The derived fit statistics. Keys are residual, initial_src_count, and final_src_count, with one entry for each input photometry method.

Return type:

dict

_downgrade_calib_status()

Deal with bad photometry for a frame.

Decrements the calibration status of the given file to astrometry and deletes the raw photometry file.

For now disabled.

Code from HATpipe:

sys.stderr.write(
    'bad photometry encountered: ' + str(self._header) + '\n'
)
if(self.database is None):
    return
self._log_to_file(
    'Downgrading status of header: ' + str(self._header) + '\n'
)
sys.stderr.write(
    'downgrading calibration status of ' + str(self._header) + '\n'
)
self.database(
    'UPDATE `'+raw_db_table(self._header['IMAGETYP'])+'` SET '
    '`calib_status`=%s WHERE `station_id`=%s AND `fnum`=%s AND '
    '`cmpos`=%s',
    (
        object_status['good_astrometry'],
        self._header['STID'], self._header['FNUM'],
        self._header['CMPOS']
    )
)
sys.stderr.write('removing:'+self._fit_file+'\n')
os.remove(self._fit_file)

abstractmethod _fit(fit_data)

Perform a fit for the magfit correction.

Parameters:

fit_data (numpy structured array) – The current photometry being fitted. It should contain the source information from the frame being fit, the photometry from the reference (and optionally reference position if used) and catalogue information for each source.

Returns:

For each photometry (i.e. different aperture of PSF fitting) a list of dictionaries is returned containing a fit result dictionary for each group of sources with the following entries:

coefficients:

The best fit coefficients after the rejection iterations of the fit has converged.

residual:

The residual of the latest iteration of the fit,

initial_src_count:

The number of stars the fit started with before any rejections,

final_src_count:

The number of stars left in the fit after the final rejection iteration,

group_id:

The ID of the group of sources this fit result corresponds to.

Return type:

[[dict]]

_get_fit_indices(phot, evaluator, no_catalogue)

Return a list of the indices within phot of sources to use for mag fit.

Exclude sources based on their catalague information.

Parameters:

• phot – See return of DataReductionFile.get_photometry().

• evaluator (Evaluator) – An object capable of evaluating expressions involving fields from phot.

• no_catalogue – The list of source indices for which no catalogue information is available (all are rejected).

Returns:

A list of the indices within phot of the sources which pass all

catalogue requirements for inclusion in the magnitude fit.

int:

The number of dropped sources.

tuple:

The ID of one of the sources dropped, None if no sources were dropped.

Return type:

[int]

_match_to_reference(phot, no_catalogue, evaluator)

Add photometric reference information, and filter per self.config.

Parameters:

• phot – See return of add_catalogue_info()

• no_catalogue (list) – The source indices from phot for which no catalogue information was available, so default values were used. All identified sources are omitted from the result.

• evaluator (Evaluator) – An object capable of evaluating expressions involving photometry and catalogue columns.

Returns:

a photometry structure like phot but for each aperture ‘ref mag’ is added - the magnitude the source has in the reference - and sources that should not be used in the fit because they are not in the reference or do not satisfy the criteria based on catalogue quantities are removed.

_set_group(evaluator, result)

Set the fit_group column in result per grouping configuration.

_solved(*, data_reduction, deleted_phot_indices, num_phot, num_sources, dr_path_substitutions)

Return fitted mags if fit is already in DR, None otherwise.

_update_calib_status()

Record in the database that the current header has been magfitted.

For now disabled.

Code from HATpipe:

self.database(
    'UPDATE `' + raw_db_table(self._header['IMAGETYP'])
    + '` SET `calib_status`=%s WHERE `station_id`=%s AND `fnum`=%s '
    'AND `cmpos`=%s',
    (
        self.config.calib_status,
        self._header['STID'],
        self._header['FNUM'],
        self._header['CMPOS']
    )
)