panoptes.utils.config package

Submodules

panoptes.utils.config.cli module

panoptes.utils.config.client module

panoptes.utils.config.client.get_config(key=None, default=None, host=None, port=None, endpoint='get-config', parse=True, verbose=False)[source]

Get a config item from the config server.

Return the config entry for the given key. If key=None (default), return the entire config.

Nested keys can be specified as a string, as per scalpl.

Examples:

>>> get_config(key='name')
'Testing PANOPTES Unit'

>>> get_config(key='location.horizon')
<Quantity 30. deg>

>>> # With no parsing, the raw string (including quotes) is returned.
>>> get_config(key='location.horizon', parse=False)
'"30 deg"'
>>> get_config(key='cameras.devices[1].model')
'canon_gphoto2'

>>> # Returns `None` if key is not found.
>>> foobar = get_config(key='foobar')
>>> foobar is None
True

>>> # But you can supply a default.
>>> get_config(key='foobar', default='baz')
'baz'

>>> # key and default are first two parameters.
>>> get_config('foobar', 'baz')
'baz'

>>> # Can use Quantities as well.
>>> from astropy import units as u
>>> get_config('foobar', 42 * u.meter)
<Quantity 42. m>

Notes

By default all calls to this function will log at the trace level because there are some calls (e.g. during POCS operation) that will be quite noisy.

Setting verbose=True changes those to debug log levels for an individual call.

Parameters:

  • key (str) – The key to update, see Examples in get_config() for details.

  • default (str, optional) – The config server port, defaults to 6563.

  • host (str, optional) – The config server host. First checks for PANOPTES_CONFIG_HOST env var, defaults to ‘localhost’.

  • port (str or int, optional) – The config server port. First checks for PANOPTES_CONFIG_HOST env var, defaults to 6563.

  • endpoint (str, optional) – The relative url endpoint to use for getting the config items, default ‘get-config’. See server_is_running() for example of usage.

  • parse (bool, optional) – If response should be parsed by panoptes.utils.serializers.from_json(), default True.

  • verbose (bool, optional) – Determines the output log level, defaults to True (i.e. debug log level). See notes for details.

Returns:

The corresponding config entry.

Return type:

dict

Raises:

Exception – Raised if the config server is not available.

panoptes.utils.config.client.server_is_running(*args, **kwargs)[source]

Thin-wrapper to check server.

panoptes.utils.config.client.set_config(key, new_value, host=None, port=None, parse=True)[source]

Set config item in config server.

Given a key entry, update the config to match. The key is a dot accessible string, as given by scalpl. See Examples in get_config() for details.

Examples:

>>> from astropy import units as u

>>> # Can use astropy units.
>>> set_config('location.horizon', 35 * u.degree)
{'location.horizon': <Quantity 35. deg>}

>>> get_config(key='location.horizon')
<Quantity 35. deg>

>>> # String equivalent works for 'deg', 'm', 's'.
>>> set_config('location.horizon', '30 deg')
{'location.horizon': <Quantity 30. deg>}
Parameters:

  • key (str) – The key to update, see Examples in get_config() for details.

  • new_value (scalar|object) – The new value for the key, can be any serializable object.

  • host (str, optional) – The config server host. First checks for PANOPTES_CONFIG_HOST env var, defaults to ‘localhost’.

  • port (str or int, optional) – The config server port. First checks for PANOPTES_CONFIG_HOST env var, defaults to 6563.

  • parse (bool, optional) – If response should be parsed by panoptes.utils.serializers.from_json(), default True.

Returns:

The updated config entry.

Return type:

dict

Raises:

Exception – Raised if the config server is not available.

panoptes.utils.config.helpers module

panoptes.utils.config.helpers.load_config(config_files: Path | List | None = None, parse: bool = True, load_local: bool = True)[source]

Load configuration information.

Note

This function is used by the config server and normal config usage should be via a running config server.

This function supports loading of a number of different files. If no options are passed to config_files then the default $PANOPTES_CONFIG_FILE will be loaded.

config_files is a list and loaded in order, so the second entry will overwrite any values specified by similarly named keys in the first entry.

config_files should be specified by an absolute path, which can exist anywhere on the filesystem.

Local versions of files can override built-in versions and are automatically loaded if they exist alongside the specified config path. Local files have a <>_local.yaml name, where <> is the built-in file.

Given the following path:

/path/to/dir
|- my_conf.yaml
|- my_conf_local.yaml

You can do a load_config('/path/to/dir/my_conf.yaml') and both versions of the file will be loaded, with the values in the local file overriding the non-local. Typically the local file would also be ignored by git, etc.

For example, the panoptes.utils.config.server.config_server will always save values to a local version of the file so the default settings can always be recovered if necessary.

Local files can be ignored (mostly for testing purposes or for recovering default values) with the load_local=False parameter.

Parameters:

  • config_files (list, optional) – A list of files to load as config, see Notes for details of how to specify files.

  • parse (bool, optional) – If the config file should attempt to create objects such as dates, astropy units, etc.

  • load_local (bool, optional) – If local files should be used, see Notes for details.

Returns:

A dictionary of config items.

Return type:

dict

panoptes.utils.config.helpers.parse_config_directories(directories: Dict[str, str])[source]

Parse the config dictionary for common objects.

Given a base entry that corresponds to the absolute path of a directory, prepend the base to all other relative directory entries.

The base directory must exist or an exception is rasied.

If the base entry is not given the current working directory is used.

>>> dirs_config = dict(base='/tmp', foo='bar', baz='bam', app='/app')
>>> parse_config_directories(dirs_config)
{'base': '/tmp', 'foo': '/tmp/bar', 'baz': '/tmp/bam', 'app': '/app'}

>>> # If base doesn't exist an exception is raised.
>>> dirs_config = dict(base='/panoptes', foo='bar', baz='bam', app='/app')
>>> parse_config_directories(dirs_config)
Traceback (most recent call last):
...
panoptes.utils.error.NotFound: NotFound: Base directory does not exist: /panoptes
Parameters:

directories (dict) – The dictionary of directory information. Usually comes from the “directories” entry in the config.

Returns:

The same directory but with relative directories resolved.

Return type:

dict

Raises:

panoptes.utils.error.NotFound – if the ‘base’ entry is given but does not exist.

panoptes.utils.config.helpers.save_config(save_path: Path, config: dict, overwrite: bool = True)[source]

Save config to local yaml file.

Parameters:

  • save_path (str) – Path to save, can be relative or absolute. See Notes in load_config.

  • config (dict) – Config to save.

  • overwrite (bool, optional) – True if file should be updated, False to generate a warning for existing config. Defaults to True for updates.

Returns:

If the save was successful.

Return type:

bool

Raises:

FileExistsError – If the local path already exists and overwrite=False.

panoptes.utils.config.server module

panoptes.utils.config.server.config_server(config_file, host=None, port=None, load_local=True, save_local=False, auto_start=True, access_logs=None, error_logs='logger')[source]

Start the config server in a separate process.

A convenience function to start the config server.

Parameters:

  • config_file (str or None) – The absolute path to the config file to load.

  • host (str, optional) – The config server host. First checks for PANOPTES_CONFIG_HOST env var, defaults to ‘localhost’.

  • port (str or int, optional) – The config server port. First checks for PANOPTES_CONFIG_HOST env var, defaults to 6563.

  • load_local (bool, optional) – If local config files should be used when loading, default True.

  • save_local (bool, optional) – If setting new values should auto-save to local file, default False.

  • auto_start (bool, optional) – If server process should be started automatically, default True.

  • access_logs (‘default’ or logger or File-like or None, optional) – Controls access logs for the gevent WSGIServer. The default string will cause access logs to go to stderr. The string logger will use the panoptes logger. A File-like will write to file. The default None will turn off all access logs.

  • error_logs (‘default’ or ‘logger’ or File-like or None, optional) – Same as access_logs except we use our logger as the default.

Returns:

The process running the config server.

Return type:

multiprocessing.Process

panoptes.utils.config.server.get_config_entry()[source]

Get config entries from server.

Endpoint that responds to GET and POST requests and returns configuration item corresponding to provided key or entire configuration. The key entries should be specified in dot-notation, with the names corresponding to the entries stored in the configuration file. See the scalpl documentation for details on the dot-notation.

The endpoint should receive a JSON document with a single key named "key" and a value that corresponds to the desired key within the configuration.

For example, take the following configuration:

{
    'location': {
        'elevation': 3400.0,
        'latitude': 19.55,
        'longitude': 155.12,
    }
}

To get the corresponding value for the elevation, pass a JSON document similar to:

'{"key": "location.elevation"}'
Returns:

The json string for the requested object if object is found in config. Otherwise a json string with status and msg keys will be returned.

Return type:

str

panoptes.utils.config.server.heartbeat()[source]

A simple echo service to be used for a heartbeat.

Defaults to looking for the ‘config_server.running’ bool value, although a different key can be specified in the POST.

panoptes.utils.config.server.reset_config()[source]

Reset the configuration.

An endpoint that accepts a POST method. The json request object must contain the key reset (with any value).

The method will reset the configuration to the original configuration files that were used, skipping the local (and saved file).

Note

If the server was originally started with a local version of the file, those will be skipped upon reload. This is not ideal but hopefully this method is not used too much.

Returns:

A json string object containing the keys success and msg that indicate success or failure.

Return type:

str

panoptes.utils.config.server.set_config_entry()[source]

Sets an item in the config.

Endpoint that responds to GET and POST requests and sets a configuration item corresponding to the provided key.

The key entries should be specified in dot-notation, with the names corresponding to the entries stored in the configuration file. See the scalpl documentation for details on the dot-notation.

The endpoint should receive a JSON document with a single key named "key" and a value that corresponds to the desired key within the configuration.

For example, take the following configuration:

{
    'location': {
        'elevation': 3400.0,
        'latitude': 19.55,
        'longitude': 155.12,
    }
}

To set the corresponding value for the elevation, pass a JSON document similar to:

'{"location.elevation": "1000 m"}'
Returns:

If method is successful, returned json string will be a copy of the set values. On failure, a json string with status and msg keys will be returned.

Return type:

str

Module contents