panoptes.utils.database package

Submodules

panoptes.utils.database.base module

class panoptes.utils.database.base.AbstractPanDB(db_name=None, **kwargs)

Bases: object

__init__(db_name=None, **kwargs)

Init base class for db instances.

Parameters:db_name – Name of the database, typically ‘panoptes’ or ‘panoptes_testing’.

clear_current(type)

Clear the current record of a certain type

Parameters:type (str) – The type of entry in the current collection that should be cleared.

find(collection, obj_id)

Find an object by it’s identifier.

Parameters:

• collection (str) – Collection to search for object.
• obj_id (ObjectID|str) – Record identifier returned earlier by insert or insert_current.

Returns:

Object matching identifier or None.

Return type:

dict|None

get_current(collection)

Returns the most current record for the given collection

Parameters:collection (str) – Name of the collection to get most current from Returns:Current object of the collection or None. Return type:dict|None

insert(collection, obj)

Insert an object into the collection provided.

The obj to be stored in a collection should include the type and date metadata as well as a data key that contains the actual object data. If these keys are not provided then obj will be wrapped in a corresponding object that does contain the metadata.

Parameters:

• collection (str) – Name of valid collection within the db.
• obj (dict or str) – Object to be inserted.

Returns:

identifier of inserted record in collection.

Returns None if unable to insert into the collection.

Return type:

str

insert_current(collection, obj, store_permanently=True)

Insert an object into both the current collection and the collection provided.

Parameters:

• collection (str) – Name of valid collection within the db.
• obj (dict or str) – Object to be inserted.
• store_permanently (bool) – Whether to also update the collection, defaults to True.

Returns:

identifier of inserted record. If store_permanently is True, will

be the identifier of the object in the collection, otherwise will be the identifier of object in the current collection. These may or may not be the same. Returns None if unable to insert into the collection.

Return type:

str

class panoptes.utils.database.base.PanDB

Bases: object

Simple class to load the appropriate DB type based on the config.

We don’t actually create instances of this class, but instead create an instance of the ‘correct’ type of db.

classmethod permanently_erase_database(db_type, db_name, storage_dir=None, really=False, dangerous=False, *args, **kwargs)

Permanently delete the contents of the identified database.

panoptes.utils.database.base.create_storage_obj(collection, data, obj_id)

Wrap the data in a dict along with the id and a timestamp.

panoptes.utils.database.base.get_db_class(module_name='file')

Load the main DB class for the module of the given name.

Note

This is used by the PanDB constructor to determine the correct database type. Normal DB instantiation should be done via the PanDB() class with the desired db_type parameter set. See example in PanDB below.

Parameters:module_name (str) – Name of module, one of: file (default), ‘memory’. Returns:An instance of the db class for the correct database type. Return type:panoptes.utils.database.PanDB Raises:Exception – If an unsupported database type string is passed.

panoptes.utils.database.file module

class panoptes.utils.database.file.PanFileDB(db_name='panoptes', storage_dir='json_store', **kwargs)

Bases: panoptes.utils.database.base.AbstractPanDB

Stores collections as files of JSON records.

__init__(db_name='panoptes', storage_dir='json_store', **kwargs)

Flat file storage for json records.

This will simply store each json record inside a file corresponding to the type. Each entry will be stored in a single line.

Parameters:

• db_name (str, optional) – Name of the database containing the collections.
• storage_dir (str, optional) – The name of the directory in $PANDIR where the database files will be stored. Default is json_store for backwards compatibility.

clear_current(record_type)

Clears the current record of the given type.

Parameters:record_type (str) – The record type, e.g. ‘weather’, ‘environment’, etc.

find(collection, obj_id)

Find an object by it’s identifier.

Parameters:

• collection (str) – Collection to search for object.
• obj_id (ObjectID|str) – Record identifier returned earlier by insert or insert_current.

Returns:

Object matching identifier or None.

Return type:

dict|None

get_current(collection)

Returns the most current record for the given collection

Parameters:collection (str) – Name of the collection to get most current from Returns:Current object of the collection or None. Return type:dict|None

insert(collection, obj)

Insert an object into the collection provided.

The obj to be stored in a collection should include the type and date metadata as well as a data key that contains the actual object data. If these keys are not provided then obj will be wrapped in a corresponding object that does contain the metadata.

Parameters:

• collection (str) – Name of valid collection within the db.
• obj (dict or str) – Object to be inserted.

Returns:

identifier of inserted record in collection.

Returns None if unable to insert into the collection.

Return type:

str

insert_current(collection, obj, store_permanently=True)

Insert an object into both the current collection and the collection provided.

Parameters:

• collection (str) – Name of valid collection within the db.
• obj (dict or str) – Object to be inserted.
• store_permanently (bool) – Whether to also update the collection, defaults to True.

Returns:

identifier of inserted record. If store_permanently is True, will

be the identifier of the object in the collection, otherwise will be the identifier of object in the current collection. These may or may not be the same. Returns None if unable to insert into the collection.

Return type:

str

classmethod permanently_erase_database(db_name, storage_dir='json_store')

panoptes.utils.database.memory module

class panoptes.utils.database.memory.PanMemoryDB(**kwargs)

Bases: panoptes.utils.database.base.AbstractPanDB

In-memory store of serialized objects.

We serialize the objects in order to test the same code path used when storing in an external database.

active_dbs = <WeakValueDictionary>

clear_current(entry_type)

Clear the current record of a certain type

Parameters:type (str) – The type of entry in the current collection that should be cleared.

find(collection, obj_id)

Find an object by it’s identifier.

Parameters:

• collection (str) – Collection to search for object.
• obj_id (ObjectID|str) – Record identifier returned earlier by insert or insert_current.

Returns:

Object matching identifier or None.

Return type:

dict|None

get_current(collection)

Returns the most current record for the given collection

Parameters:collection (str) – Name of the collection to get most current from Returns:Current object of the collection or None. Return type:dict|None

classmethod get_or_create(db_name=None, **kwargs)

Returns the named db, creating if needed.

This method exists because PanDB gets called multiple times for the same database name. With mongo or a file store where the storage is external from the instance, that is not a problem, but with PanMemoryDB the instance is the store, so the instance must be shared.

insert(collection, obj)

Insert an object into the collection provided.

The obj to be stored in a collection should include the type and date metadata as well as a data key that contains the actual object data. If these keys are not provided then obj will be wrapped in a corresponding object that does contain the metadata.

Parameters:

• collection (str) – Name of valid collection within the db.
• obj (dict or str) – Object to be inserted.

Returns:

identifier of inserted record in collection.

Returns None if unable to insert into the collection.

Return type:

str

insert_current(collection, obj, store_permanently=True)

Insert an object into both the current collection and the collection provided.

Parameters:

• collection (str) – Name of valid collection within the db.
• obj (dict or str) – Object to be inserted.
• store_permanently (bool) – Whether to also update the collection, defaults to True.

Returns:

identifier of inserted record. If store_permanently is True, will

be the identifier of the object in the collection, otherwise will be the identifier of object in the current collection. These may or may not be the same. Returns None if unable to insert into the collection.

Return type:

str

classmethod permanently_erase_database(*args, **kwargs)

Module contents