General architecture

The storage of data is an important aspect of the AiiDA system. The design for this subsystem is illustrated below.

../../_images/storage-uml.svg

Fig. 28 UML diagram of the storage architecture.

Blue indicates frontend classes, red indicates backend classes, and green indicates singletons.

Separate data is stored per Profile, forming a single provenance graph. A Profile instance represents a dictionary that includes the configuration details for accessing the storage for that profile, such as a database URI, etc. Multiple Profile can be stored in a Config instance, which is stored in the configuration file (config.json).

Within a single Python process, a single Manager instance can be loaded, to manage access to a globally loaded Profile and its StorageBackend instance.

The storage API subsystem is based on an Object Relational Mapper (ORM) and is divided into two main parts: the frontend and the backend. The frontend is responsible for the user interface, and is agnostic of any particular storage technologies, and the backend is responsible for implementing interfaces with specific technologies (such as SQL databases).

Frontend ORM

The frontend ORM comprises of a number of Collection and Entity subclasses, representing access to a single ORM type.

User

Represents the author of a particular entity.

Node

Represents a node in a provenance graph, containing data for a particular process (ProcessNode) or process input/output (Data). Nodes are connected by links, that form an acyclic graph. Nodes also have a Repository instance, which is used to store binary data of the node (see also Repository).

Comment

Represents a comment on a node, by a particular user.

Log

Represents a log message on a ProcessNode, by a particular user.

Group

Represents a group of nodes. A single node can be part of multiple groups (i.e. a one-to-many relationship).

Computer

Represents a compute resource on which a process is executed. A single computer can be attached to multiple ProcessNode (i.e. a one-to-many relationship).

AuthInfo

Represents a authentication information for a particular computer and user.

The QueryBuilder allows for querying of specific entities and their associated data.

Backend Implementations

Backend implementations must implement the classes outlines in aiida.orm.implementation.

There are currently two core backend implementations:

Storage maintenance and profile locking

The maintain() method is allows for maintenance operations on the storage (for example, to optimise memory usage), and is called by verdi storage maintain.

During “full” maintenance, to guarantee the safety of its procedures, it may be necessary that the storage is not accessed by other processes. The :py:class`~aiida.manage.profile_access.ProfileAccessManager` allows for profile access requests, and locking of profiles during such procedures. request_access() is called within get_profile_storage().