autowisp.image_calibration.master_flat_maker module

Class Inheritance Diagram

Define classes for creating master flat frames.

class autowisp.image_calibration.master_flat_maker.MasterFlatMaker(*, stamp_statistics_config=None, stamp_select_config=None, large_scale_smoother=None, cloud_check_smoother=None, master_stack_config=None)

Bases: MasterMaker

Specialize MasterMaker for making master flat frames.

stamp_statistics_config

Dictionary configuring how stamps statistics for stamp-based selection are extracted from the frames.

stamp_select_config

Dictionary configuring how stamp-based selection is performed. See keyword only arguments of configure_stamp_selection() for details.

large_scale_smoother

A autowisp.image_smoothing.ImageSmoother instance applied to the ratio of a frame to the reference large scale structure before applying it to the frame.

cloud_check_smoother

autowisp.image_smoothing.ImageSmoother instance used for cloud detection performed on the full flat frames after smoothing to the master large scale structure.

master_stack_config

Dictionary configuring how to stack the individual frames into a master.

_master_large_scale

The large scale structure imposed on all master flats. Dictionary with keys values, stdev, mask and header. If empty dictionary, nothing is imposed.

Examples

>>> import scipy.ndimage.filters
>>> from autowisp.image_smoothing import\
>>>     PolynomialImageSmoother,\
>>>     SplineImageSmoother,\
>>>     ChainSmoother,\
>>>     WrapFilterAsSmoother
>>>
>>> #Stamp statistics configuration:
>>> #  * stamps span half the frame along each dimension
>>> #  * stamps are detrended by a bi-quadratic polynomial with at most
>>> #    one rejection iteration, discarding more than 3-sigma outliers.
>>> #  * for each stamp a iterative rejection mean and variance are
>>> #    calculated with up to 3 iterations rejecting three or more
>>> #    sigma outliers.
>>> stamp_statistics_config = dict(
>>>     fraction=0.5,
>>>     smoother=PolynomialImageSmoother(num_x_terms=3,
>>>                                      num_y_terms=3,
>>>                                      outlier_threshold=3.0,
>>>                                      max_iterations=3),
>>>     average='mean',
>>>     outlier_threshold=3.0,
>>>     max_iter=3
>>> )
>>>
>>> #Stamp statistics based selection configuration:
>>> #  * Stamps with more than 0.1% of their pixels saturated are
>>> #    discarded
>>> #  * if variance vs mean quadratic fit has residual of more than 5k
>>> #    ADU^2, the entire night is considered cloudy.
>>> #  * individual frames with stamp mean and variance deviating more
>>> #    than 2*(fit_residual) are discarded as cloudy.
>>> #  * high master flat will be generated from frames with stamp mean
>>> #    > 25 kADU, and low master flat from frames with stamp mean < 15
>>> #    kADU (intermediate frames are discarded).
>>> stamp_select_config = dict(max_saturated_fraction=1e-4,
>>>                            var_mean_fit_threshold=2.0,
>>>                            var_mean_fit_iterations=2,
>>>                            cloudy_night_threshold=5e3,
>>>                            cloudy_frame_threshold=2.0,
>>>                            min_high_mean=2.5e4,
>>>                            max_low_mean=1.5e4)
>>>
>>> #Large scale structure smoothing configuration. For each frame, the
>>> #large scale struture is corrected by taking the ratio of the frame
>>> #to the reference (median of all input frames), smoothing this ratio
>>> #and then dividing by it. The following defines how the smoothing is
>>> #performed:
>>> #  * shrink by a factor of 4 in each direction (16 pixels gets
>>> #    averaged to one).
>>> #  * Performa a box-filtering with a half-size of 6-pixels using
>>> #    median averaging
>>> #  * Perform a bi-cubic spline interpolation smoothing of the
>>> #    box-filtered image.
>>> #  * Discard more than 5-sigma outliers if any and re-smooth (no
>>> #    further iterations allowed)
>>> #  * Re-interpolate the image back to its original size, using
>>> #    bicubic interpolation (see zoom_image()).
>>> #  * The resulting image is scaled to have a mean of 1 (no
>>> #    configuration for that).
>>> large_scale_smoother = ChainSmoother(
>>>     WrapFilterAsSmoother(scipy.ndimage.filters.median_filter,
>>>                          size=12),
>>>     SplineImageSmoother(num_x_nodes=3,
>>>                         num_y_nodes=3,
>>>                         outlier_threshold=5.0,
>>>                         max_iter=1),
>>>     bin_factor=4,
>>>     zoom_interp_order=3
>>> )
>>>
>>> #Configuration for smoothnig when checking for clouds.
>>> #After smoothing to the master large scale structure:
>>> #  * extract a central stamp is extracted from each flat covering
>>> #    3/4 of the frame along each dimension
>>> #  * shrink the fractional deviation of that stamp from the master
>>> #    by a factor of 4 in each dimension
>>> #  * smooth by median box-filtering with half size of 4 shrunk
>>> #    pixels
>>> #  * zoom the frame back out by a factor of 4 in each dimension
>>> #    (same factor as shrinking, no separater config), using
>>> #    bi-quadratic interpolation.
>>> cloud_check_smoother = WrapFilterAsSmoother(
>>>     scipy.ndimage.filters.median_filter,
>>>     size=8,
>>>     bin_factor=4,
>>>     zoom_interp_order=3
>>> )
>>>
>>> #When stacking masters require:
>>> #  * At least 10 input frames for a high intensity master and at
>>> #    least 5 for a low intensity one.
>>> #  * When creating the stack used to match large-scale structure,
>>> #    use median averaging with outlier rejection of more than
>>> #    4-sigma outliers with at most one reject/re-fit iteration.
>>> #  * When creating the final master use median averaging with
>>> #    outlier rejection of more than 2-sigma outliers in the positive
>>> #    and more than 3-sigma in the negative direction with at most 2
>>> #    reject/re-fit iterations. Create the master compressed and
>>> #    raise an exception if a file with that name already exists.
>>> master_stack_config = dict(
>>>     min_pointing_separation=150.0,
>>>     large_scale_deviation_threshold=0.05,
>>>     min_high_combine=10,
>>>     min_low_combine=5,
>>>     large_scale_stack_options=dict(
>>>         outlier_threshold=4,
>>>         average_func=numpy.nanmedian,
>>>         min_valid_values=3,
>>>         max_iter=1,
>>>         exclude_mask=MasterMaker.default_exclude_mask
>>>     ),
>>>     master_stack_options=dict(
>>>         outlier_threshold=(2, -3),
>>>         average_func=numpy.nanmedian,
>>>         min_valid_values=3,
>>>         max_iter=2,
>>>         exclude_mask=MasterMaker.default_exclude_mask,
>>>         compress=True,
>>>         allow_overwrite=False
>>>     )
>>> )
>>>
>>> #Create an object for stacking calibrated flat frames to master
>>> #flats. In addition to the stamp-based rejections:
>>> #  * reject flats that point within 40 arcsec of each other on the
>>> #    sky.
>>> #  * Require at least 10 frames to be combined into a high master
>>> #    and at least 5 for a low master.
>>> #  * if the smoothed cloud-check image contains pixels with absolute
>>> #    value > 5% the frame is discarded as cloudy.
>>> make_master_flat = MasterFlatMaker(
>>>     stamp_statistics_config=stamp_statistics_config,
>>>     stamp_select_config=stamp_select_config,
>>>     large_scale_smoother=large_scale_smoother,
>>>     cloud_check_smoother=cloud_check_smoother,
>>>     master_stack_config=master_stack_config
>>> )
>>>
>>> #Create master flat(s) from the given raw flat frames. Note that
>>> #zero, one or two master flat frames can be created, depending on
>>> #the input images. Assume that the raw flat frames have names like
>>> #10-<fnum>_2.fits.fz, with fnum ranging from 1 to 30 inclusive.
>>> make_master_flat(
>>>     ['10-%d_2.fits.fz' % fnum for fnum in range(1, 31)],
>>>     high_master_fname='high_master_flat.fits.fz',
>>>     low_master_fname='low_master_flat.fits.fz'
>>> )

__call__(frame_list, high_master_fname, low_master_fname, *, compress=True, allow_overwrite=False, stamp_statistics_config=None, stamp_select_config=None, master_stack_config=None, custom_header=None)

Attempt to create high & low master flat from the given frames.

Parameters:

• frame_list – A list of the frames to create the masters from (FITS filenames).

• high_master_fname – The filename to save the generated high intensity master flat if one is successfully created.

• low_master_fname – The filename to save the generated low intensity master flat if one is successfully created.

• compress – Should the final result be compressed?

• allow_overwrite – See same name argument to autowisp.image_calibration.fits_util.create_result() .

• stamp_statistics_config (dict) – Overwrite the configuration for extracting stamp statistics for this set of frames only.

• stamp_select_config (dict) – Overwrite the stamp selection configuration for this set of frames only.

• master_stack_config (dict) – Overwrite the configuration for how to stack frames to a master for this set of frames only.

• custom_header – See same name argument to autowisp.image_calibration.master_maker.MasterMaker.__call__() .

Returns:

A dictionary splitting the input list of frames into

high:

All entries from frame_list which were deemed suitable for inclusion in a master high flat.

low:

All entries from frame_list which were deemed suitable for inclusion in a master low flat.

medium:

All entries from frame_list which were of intermediate intensity and thus not included in any master, but for which no issues were detected.

colocated:

All entries from frame_list which were excluded because they were not sufficiently isolated from their closest neighbor to guarantee that stars do not overlap.

cloudy:

All entries from frame_list which were flagged as cloudy either based on their stamps or on the final full-frame cloud check.

Return type:

dict

__init__(*, stamp_statistics_config=None, stamp_select_config=None, large_scale_smoother=None, cloud_check_smoother=None, master_stack_config=None)

Create object for creating master flats out of calibrated flat frames.

Parameters:

• stamp_statistics_config – A dictionary mith arguments to pass to configure_stamp_statistics().

• stamp_select_cofig – A dictionary with arguments to pass to configure_stamp_selection().

• large_scale_smoother – An ImageSmoother instance used when matching large scale structure of individual flats to master.

• cloud_check_smoother – An ImageSmoother instance used when checkng the full frames for clouds (after stamps are checked).

• master_stack_config – Configuration of how to stack frames to a master after matching their large scale structure. See example in class doc-string for details.

Returns:

None

_classify_from_stamps(frame_list, stamp_statistics_config, stamp_select_config)

Classify frames by intensity: (high/low/ntermediate), or flag as cloudy.

Parameters:

frame_list – The list of frames to create masters from.

Returns:

filename list:

The list of high intensity non-cloudy frames.

filename list:

The list of low intensity non-cloudy frames.

filename list:

The list of medium intensity non-cloudy frames.

filename list:

The list of frames suspected of containing clouds in their central stamps.

Return type:

(tuple)

_find_colocated(frame_list)

Split the list of frames into well isolated ones and co-located ones.

Parameters:

frame_list – A list of the frames to create the masters from (FITS filenames).

Returns:

filename list:

The ist of frames sufficiently far in pointing from all other frames.

filename list:

The list of frames which have at least one other frame too close in pointing to them.

Return type:

(tuple)

_get_stamp_statistics(frame_list, **stamp_statistics_config)

Get relevant information from the stamp of a single input flat frame.

Parameters:

frame_list – The list of frames for which to extract stamp statistics.

Returns:

An array with fields called mean, variance and num_averaged with the obvious meanings.

Return type:

numpy field array

configure_stamp_selection(*, max_saturated_fraction=None, var_mean_fit_threshold=None, var_mean_fit_iterations=None, cloudy_night_threshold=None, cloudy_frame_threshold=None, min_high_mean=None, max_low_mean=None)

Configure stamp-based frame selection and high/low split.

Parameters:

• max_saturated_fraction – The maximum fraction of stamp pixels allowed to be saturated before discarding the frame.

• cloudy_night_threshold – The maximum residual of the variance vs mean quadratic fit before a night is declared cloudy. If None, this check is disabled.

• cloudy_frame_threshold – The maximum deviation in units of the RMS residual of the fit an individual frame’s variance vs mean from the var(mean) quadratic fit before the frame is declared cloudy.

• min_high_mean – The minimum mean of the stamp pixels in order to consider the frame high intensity.

• max_low_mean – The maximum mean of the stamp pixels in order to consider the frame low intensity. Must not overlap with min_high_mean.

Returns:

None

configure_stamp_statistics(*, fraction=None, smoother=None, outlier_threshold=None, max_iter=None, average=None)

Configure extraction of stamp satistics for rejection & high/low split.

Any arguments left as None are not updated.

Parameters:

• fraction – The fraction of the frame size that is included in the stamp along each dimension (i.e. fraction=0.5 means 1/4 of all frame pixels will be incruded in the stamp).

• smoother – An ImageSmoother instance used used for de-trending the stamps before extracting statistics

• outlier_threshold – The threshold in units of RMS deviation from the average above which pixels are considered outliers from the de-trending function and discarded from its fit.

• max_iter – The maximum number of fit/reject iterations to perform before declaring the de-trending function final.

• average – How to compute the average. Should be either 'mean' or 'median'.

Returns:

None

prepare_for_stacking(image)

Match image large scale to self._master_large_scale if not empty.

Parameters:

image – The image to transform large scale structure of.

Returns:

If _master_large_scale is an empty dictionary, this is just image. Otherwise, image is transformed to have the same large scale structure as _master_large_scale, while the small scale structure is preserved. image is also checked for clouds, and discarded if cloudy.

Return type:

2-D array