**************************** How a setting gets its value **************************** :doc:`wisp_options` lists every setting there is and says what each one means. It does not say how AutoWISP decides what a setting *is* for any particular image, which is a separate question and occasionally a puzzling one -- settings can hold different values for different images, some never appear in your configuration at all, and leaving one blank does not mean what you might think it means. Settings belong to the project ============================== A setting is not something you pass to a command. It is a row in the project's database, written when the project is created: AutoWISP asks each processing stage what it can be told, and records every answer. That is why the options reference is generated from a freshly made project rather than written by hand, and why a setting exists whether or not you ever give it a value. The important consequence is that **a setting belongs to the project, not to a stage, and many settings are used by several stages at once**. There is one :option:`verbose`, and it decides how much every stage writes to its log; likewise the rest of the logging settings. There is one :option:`num-parallel-processes`, and it decides how much work at a time each stage that can use several processes will do. That extends to settings that matter a great deal more than logging does: * :option:`data-reduction-fname` says where each image's data reduction file lives. It is shared by ``find_stars``, ``fit_star_shape`` and ``measure_aperture_photometry`` -- and it has to be, since one writes what the others go on to read. Were they able to disagree, the later stages would simply not find the files. * :option:`variables` names the light curve quantities that :option:`lc-points-filter-expression` and the fitting expressions are written in terms of. It is shared by all four detrending stages: EPD, TFA and the two that report on how well each of them did. If the reporting stages understood the names differently from the correcting ones, the statistics would describe something other than what was corrected. Sharing is the point -- you say a thing once and every stage that needs it is told -- but it does mean there is no such thing as changing a setting "for one stage". Change it and everything using it changes. This is worth keeping in mind because the configuration editor invites the opposite impression. It can be narrowed to a single stage, so that you see only the settings that stage uses instead of several hundred at once. That filter decides what is shown and nothing else: values are stored against the setting, not against the stage you happened to be looking at, so editing one while filtered to ``tfa`` changes it for every other stage that uses it too. Running a stage yourself with ``wisp-calibrate`` and the like is the exception to all of this. Those commands read a configuration file and their own command line, not the database, which is what makes them useful for one-off runs but also means a value you set there affects nothing else. A value, and when it applies ============================ Each value is stored together with a **condition**: a set of expressions over the image header, all of which must hold for that value to be used. Every project starts with one condition whose expression is simply ``True``, so a value stored against it applies to every image -- which is what a setting with a single, unconditional value amounts to. Give a setting more than one value and you are choosing between them by condition. When a stage is about to run on an image, AutoWISP evaluates the conditions' expressions against that image's header. A particular value applies if everything its condition asks for came out true. The values are tried **in the order they are listed**. The first that applies wins, so a general fallback belongs last. If you put it first, nothing after it is ever reached. If no value applies at all, the run stops with an error naming the setting, rather than guessing. Two things about the matching are worth knowing. It happens per image, so conditions can key off anything in the header -- the target, the exposure length, the observing session. And it happens per colour channel as well, so the same setting can take different values for the red and blue channels of the same image. Where the values come from, and what "no value" means ===================================================== Every setting has a value from the moment a project exists. As the project is created each one is given the value its stage would have used by itself, and that is what goes into the database. You will want to change a fair number of them, and most of this documentation is about which ones and why; the point is that none of them are empty waiting for you, and the stored value is what applies until you replace it. For a fair number of settings, the value a stage would have used is *nothing at all*. That is a real state in its own right, and not the same as an empty box. Such a setting is left out of the configuration handed to the stage, and the stage then decides for itself when it runs. :option:`epd-datasets` and :option:`tfa-datasets` are the clearest example. Absent, they do not detrend nothing -- they detrend every aperture the photometry produced, worked out from the photometric reference at the time. The distinction matters because only one of the two can be typed. The editor's value boxes hold text, so clearing one leaves an empty string, which is a value like any other rather than an absence. For settings where an empty value means nothing sensible, that is an error and not a fallback. Emptying :option:`tfa-datasets` does not restore the behaviour described above, it stops the step with a complaint that ``''`` cannot be made sense of as a dataset to detrend. Restoring a value to absent is therefore not something the value boxes can do. It has to be done through the exported configuration described next, where an absence appears as ``null``. Carrying configuration between projects ======================================= The configuration editor can write the whole configuration out as a JSON file and read one back in. Importing merges what the file contains into what the project already has and hands you the result in the editor, so you can look it over -- and, if it is not what you wanted, leave without saving. Nothing changes until you save. That is the way to set up a second project like the first, to keep a copy of a configuration that works, and to make the sort of edit the value boxes cannot express. **Use the JSON rather than a configuration file for this.** The two are not equivalent. A configuration file has one line per setting, and no way to say "this value under these conditions"; the JSON is a tree, and the conditions are the branches. Anything conditional survives the JSON round trip and cannot be written in a configuration file at all. The same limitation applies to the configuration you can paste in when creating a project. It is read as ordinary configuration lines, so everything it sets is unconditional -- convenient for carrying the settings of an instrument to a new project, but not the conditions you may have built on top of them. Those have to come from an import afterwards. The configuration files that the individual ``wisp-*`` commands take are different again: each is a flat list of settings for one run, which is all such a command needs, since it works on what you hand it rather than choosing per image. Settings you will not find in your configuration ================================================ Some of what a stage reads is never stored as a setting at all. Anything naming a master file -- the master bias, the single photometric reference, the catalog collected for detrending -- is filled in after your configuration has been read, by looking up which master matches the image at hand in the tables AutoWISP maintains for the purpose. Those settings appear in the options reference, because the stages accept them, but not in the configuration you edit, and there is nothing to set: the answer depends on the image. This is also why such a setting behaves differently when you run a stage by hand. There is no master lookup on the command line, so :option:`single-photref-dr-fname` and its like have to be given explicitly, and the test data's configuration file duly gives them. What the values themselves are ============================== Every value is stored as text, and what becomes of that text depends on the setting. Most are used as they stand, or converted to a number or a list. Some -- those identifying equipment, the observing time, the target -- are expressions evaluated against the image header, which is what lets them cope with headers that use different keywords for the same thing. :doc:`bringing_your_own_data` covers those, including the quoting needed to make one produce a fixed string rather than read a keyword. A note on versions ================== Values carry a version number, and the configuration editor offers a version to work on. The intent is that a project's configuration can move forward while what earlier runs used stays on record, so that results can be traced back to the settings that produced them. Treat it as unfinished. It has had no real use and is not exercised by the tests, so it is not something to build a way of working around yet. Leaving the version alone and editing the configuration in place is the trodden path.