panoptes.utils.database package

Submodules

panoptes.utils.database.base module

class panoptes.utils.database.base.AbstractPanDB(db_name=None, **kwargs)

Bases: object

Abstract base class for PANOPTES database implementations.

This class defines the interface that all database implementations must follow for storing and retrieving observational data.

abstractmethod clear_current(type)

Clear the current record of a certain type

Parameters:

type (str) – The type of entry in the current collection that should be cleared.

abstractmethod find(collection, obj_id)

Find an object by it’s identifier.

Parameters:

• collection (str) – Collection to search for object.

• obj_id (ObjectID|str) – Record identifier returned earlier by insert or insert_current.

Returns:

Object matching identifier or None.

Return type:

dict|None

abstractmethod get_current(collection)

Returns the most current record for the given collection

Parameters:

collection (str) – Name of the collection to get most current from

Returns:

Current object of the collection or None.

Return type:

dict|None

abstractmethod insert(collection, obj)

Insert an object into the collection provided.

The obj to be stored in a collection should include the type and date metadata as well as a data key that contains the actual object data. If these keys are not provided then obj will be wrapped in a corresponding object that does contain the metadata.

Parameters:

• collection (str) – Name of valid collection within the db.

• obj (dict or str) – Object to be inserted.

Returns:

identifier of inserted record in collection.

Returns None if unable to insert into the collection.

Return type:

str

abstractmethod insert_current(collection, obj, store_permanently=True)

Insert an object into both the current collection and the collection provided.

Parameters:

• collection (str) – Name of valid collection within the db.

• obj (dict or str) – Object to be inserted.

• store_permanently (bool) – Whether to also update the collection, defaults to True.

Returns:

identifier of inserted record. If store_permanently is True, will

be the identifier of the object in the collection, otherwise will be the identifier of object in the current collection. These may or may not be the same. Returns None if unable to insert into the collection.

Return type:

str

class panoptes.utils.database.base.PanDB(db_type='memory', db_name=None, *args, **kwargs)

Bases: object

Simple class to load the appropriate DB type based on the config.

We don’t actually create instances of this class, but instead create an instance of the ‘correct’ type of db.

classmethod permanently_erase_database(db_type, db_name, storage_dir=None, really=False, dangerous=False, *args, **kwargs)

Permanently delete the contents of the identified database.

panoptes.utils.database.base.create_storage_obj(collection, data, obj_id)

Wrap the data in a dict along with the id and a timestamp.

panoptes.utils.database.base.get_db_class(module_name='file')

Load the main DB class for the module of the given name.

Note

This is used by the PanDB constructor to determine the correct database type. Normal DB instantiation should be done via the PanDB() class with the desired db_type parameter set. See example in PanDB below.

Parameters:

module_name (str) – Name of module, one of: file (default), ‘memory’.

Returns:

An instance of the db class for the correct database type.

Return type:

panoptes.utils.database.PanDB

Raises:

Exception – If an unsupported database type string is passed.

panoptes.utils.database.file module

class panoptes.utils.database.file.PanFileDB(db_name='panoptes', storage_dir='json_store', **kwargs)

Bases: AbstractPanDB

Stores collections as files of JSON records.

clear_current(record_type)

Clears the current record of the given type.

Parameters:

record_type (str) – The record type, e.g. ‘weather’, ‘environment’, etc.

find(collection, obj_id)

Find object by ID in collection.

Parameters:

• collection (str) – Collection name to search in.

• obj_id (str) – Object ID to find.

Returns:

Found object, or None if not found.

Return type:

dict or None

get_current(collection)

Get current object from collection.

Parameters:

collection (str) – Collection name to get current from.

Returns:

Current object in collection, or None if not found.

Return type:

dict or None

insert(collection, obj)

Insert object into collection.

Parameters:

• collection (str) – Collection name to insert into.

• obj – Object to insert.

Returns:

Object ID of inserted item.

Return type:

str

insert_current(collection, obj, store_permanently=True)

Insert object as current item in collection.

Parameters:

• collection (str) – Collection name to insert into.

• obj – Object to insert.

• store_permanently (bool) – Whether to also store in permanent collection.

Returns:

Object ID of inserted item.

Return type:

str

classmethod permanently_erase_database(db_name, storage_dir=None)

Permanently erase the database.

For testing purposes only. Removes all JSON files from storage directory.

Parameters:

• db_name (str) – Database name.

• storage_dir (str, optional) – Storage directory path.

panoptes.utils.database.memory module

class panoptes.utils.database.memory.PanMemoryDB(**kwargs)

Bases: AbstractPanDB

In-memory store of serialized objects.

We serialize the objects in order to test the same code path used when storing in an external database.

active_dbs = <WeakValueDictionary>

clear_current(entry_type)

Clear current entry for specified type.

Parameters:

entry_type (str) – Entry type to clear.

find(collection, obj_id)

Find object by ID in collection.

Parameters:

• collection (str) – Collection name to search in.

• obj_id (str) – Object ID to find.

Returns:

Found object, or None if not found.

Return type:

dict or None

get_current(collection)

Get current object from collection.

Parameters:

collection (str) – Collection name to get current from.

Returns:

Current object in collection, or None if not found.

Return type:

dict or None

classmethod get_or_create(db_name=None, **kwargs)

Returns the named db, creating if needed.

This method exists because PanDB gets called multiple times for the same database name. With mongo or a file store where the storage is external from the instance, that is not a problem, but with PanMemoryDB the instance is the store, so the instance must be shared.

insert(collection, obj)

Insert object into collection.

Parameters:

• collection (str) – Collection name to insert into.

• obj – Object to insert.

Returns:

Object ID of inserted item.

Return type:

str

insert_current(collection, obj, store_permanently=True)

Insert object as current item in collection.

Parameters:

• collection (str) – Collection name to insert into.

• obj – Object to insert.

• store_permanently (bool) – Whether to also store in permanent collection.

Returns:

Object ID of inserted item.

Return type:

str

classmethod permanently_erase_database(*args, **kwargs)

Permanently erase the database.

For testing purposes only. Erases all data and references.

Parameters:

• *args – Positional arguments (ignored).

• **kwargs – Keyword arguments (ignored).

Module contents