aiida.tools.importexport package

Provides import/export functionalities.

To see history/git blame prior to the move to aiida.tools.importexport, explore tree: https://github.com/aiidateam/aiida-core/tree/eebef392c81e8b130834a92e1d7abf5e2e30b3ce Functionality: <tree>/aiida/orm/importexport.py Tests: <tree>/aiida/backends/tests/test_export_and_import.py

aiida.tools.importexport.export(what, outfile='export_data.aiida.tar.gz', overwrite=False, silent=False, **kwargs)

Export the entries passed in the ‘what’ list to a file tree.

Parameters

• what (list) – a list of entity instances; they can belong to different models/entities.

• outfile (str) – the filename (possibly including the absolute path) of the file on which to export.

• overwrite (bool) – if True, overwrite the output file without asking, if it exists. If False, raise an ArchiveExportError if the output file already exists.

• silent (bool) – suppress prints.

• allowed_licenses (list) – List or function. If a list, then checks whether all licenses of Data nodes are in the list. If a function, then calls function for licenses of Data nodes expecting True if license is allowed, False otherwise.

• forbidden_licenses (list) – List or function. If a list, then checks whether all licenses of Data nodes are in the list. If a function, then calls function for licenses of Data nodes expecting True if license is allowed, False otherwise.

• include_comments (bool) – In-/exclude export of comments for given node(s) in what. Default: True, include comments in export (as well as relevant users).

• include_logs (bool) – In-/exclude export of logs for given node(s) in what. Default: True, include logs in export.

• kwargs – graph traversal rules. See aiida.common.links.GraphTraversalRules what rule names are toggleable and what the defaults are.

Raises

• ArchiveExportError – if there are any internal errors when exporting.

• LicensingException – if any node is licensed under forbidden license.

aiida.tools.importexport.export_zip(what, outfile='testzip', overwrite=False, silent=False, use_compression=True, **kwargs)

Export in a zipped folder

Parameters

• what (list) – a list of entity instances; they can belong to different models/entities.

• outfile (str) – the filename (possibly including the absolute path) of the file on which to export.

• overwrite (bool) – if True, overwrite the output file without asking, if it exists. If False, raise an ArchiveExportError if the output file already exists.

• silent (bool) – suppress prints.

• use_compression (bool) – Whether or not to compress the zip file.

• allowed_licenses (list) – List or function. If a list, then checks whether all licenses of Data nodes are in the list. If a function, then calls function for licenses of Data nodes expecting True if license is allowed, False otherwise.

• forbidden_licenses (list) – List or function. If a list, then checks whether all licenses of Data nodes are in the list. If a function, then calls function for licenses of Data nodes expecting True if license is allowed, False otherwise.

• include_comments (bool) – In-/exclude export of comments for given node(s) in what. Default: True, include comments in export (as well as relevant users).

• include_logs (bool) – In-/exclude export of logs for given node(s) in what. Default: True, include logs in export.

• kwargs – graph traversal rules. See aiida.common.links.GraphTraversalRules what rule names are toggleable and what the defaults are.

Raises

• ArchiveExportError – if there are any internal errors when exporting.

• LicensingException – if any node is licensed under forbidden license.

aiida.tools.importexport.import_data(in_path, group=None, silent=False, **kwargs)

Import exported AiiDA archive to the AiiDA database and repository.

Proxy function for the backend-specific import functions. If in_path is a folder, calls extract_tree; otherwise, tries to detect the compression format (zip, tar.gz, tar.bz2, …) and calls the correct function.

Parameters

• in_path (str) – the path to a file or folder that can be imported in AiiDA.

• group (Group) – Group wherein all imported Nodes will be placed.

• silent (bool) – suppress prints.

• extras_mode_existing (str) – 3 letter code that will identify what to do with the extras import. The first letter acts on extras that are present in the original node and not present in the imported node. Can be either: ‘k’ (keep it) or ‘n’ (do not keep it). The second letter acts on the imported extras that are not present in the original node. Can be either: ‘c’ (create it) or ‘n’ (do not create it). The third letter defines what to do in case of a name collision. Can be either: ‘l’ (leave the old value), ‘u’ (update with a new value), ‘d’ (delete the extra), or ‘a’ (ask what to do if the content is different).

• extras_mode_new (str) – ‘import’ to import extras of new nodes or ‘none’ to ignore them

• comment_mode (str) – Comment import modes (when same UUIDs are found). Can be either: ‘newest’ (will keep the Comment with the most recent modification time (mtime)) or ‘overwrite’ (will overwrite existing Comments with the ones from the import file).

Returns

New and existing Nodes and Links.

Return type

dict

Raises

ArchiveImportError – if there are any internal errors when importing.

class aiida.tools.importexport.Archive(filepath)

Bases: object

Utility class to operate on exported archive files or directories.

The main usage should be to construct the class with the filepath of the export archive as an argument. The contents will be lazily unpacked into a sand box folder which is constructed upon entering the instance within a context and which will be automatically cleaned upon leaving that context. Example:

with Archive('/some/path/archive.aiida') as archive:
    archive.version

FILENAME_DATA = 'data.json'

FILENAME_METADATA = 'metadata.json'

__dict__ = mappingproxy({'__module__': 'aiida.tools.importexport.common.archive', '__doc__': "Utility class to operate on exported archive files or directories.\n\n The main usage should be to construct the class with the filepath of the export archive as an argument.\n The contents will be lazily unpacked into a sand box folder which is constructed upon entering the instance\n within a context and which will be automatically cleaned upon leaving that context. Example::\n\n with Archive('/some/path/archive.aiida') as archive:\n archive.version\n\n ", 'FILENAME_DATA': 'data.json', 'FILENAME_METADATA': 'metadata.json', '__init__': <function Archive.__init__>, '__enter__': <function Archive.__enter__>, '__exit__': <function Archive.__exit__>, 'ensure_within_context': <AdapterWrapper at 0x7fef14fbb180 for function>, 'ensure_unpacked': <AdapterWrapper at 0x7fef14fbb1e8 for function>, 'unpack': <FunctionWrapper at 0x7fef14fbb250 for function>, 'filepath': <property object>, 'folder': <property object>, 'data': <property object>, 'meta_data': <property object>, 'unpacked': <property object>, 'get_info': <FunctionWrapper at 0x7fef14fbb3f0 for function>, 'get_data_statistics': <FunctionWrapper at 0x7fef14fbb458 for function>, 'version_aiida': <property object>, 'version_format': <property object>, 'conversion_info': <property object>, '_read_json_file': <FunctionWrapper at 0x7fef14fbb798 for FunctionWrapper>, '__dict__': <attribute '__dict__' of 'Archive' objects>, '__weakref__': <attribute '__weakref__' of 'Archive' objects>})

__enter__()

Instantiate a SandboxFolder into which the archive can be lazily unpacked.

__exit__(exc_type, exc_value, traceback)

Clean the sandbox folder if it was instatiated.

__init__(filepath)

Initialize self. See help(type(self)) for accurate signature.

__module__ = 'aiida.tools.importexport.common.archive'

__weakref__

list of weak references to the object (if defined)

_read_json_file(filename)

Read the contents of a JSON file from the unpacked archive contents.

Parameters

filename – the filename relative to the sandbox folder

Returns

a dictionary with the loaded JSON content

property conversion_info

Return information about migration events that were applied to this archive.

Returns

list of conversion notifications

property data

Return the loaded content of the data file

Returns

dictionary with contents of data file

ensure_unpacked(instance, args, kwargs)

Decorator to ensure that the archive is unpacked before entering the decorated function.

ensure_within_context(instance, args, kwargs)

Decorator to ensure that the instance is called within a context manager.

property filepath

Return the filepath of the archive

Returns

the archive filepath

property folder

Return the sandbox folder

Returns

sandbox folder aiida.common.folders.SandboxFolder

get_data_statistics()

Return dictionary with statistics about data content, i.e. how many entries of each entity type it contains.

Returns

a dictionary with basic details

get_info()

Return a dictionary with basic information about the archive.

Returns

a dictionary with basic details

property meta_data

Return the loaded content of the meta data file

Returns

dictionary with contents of meta data file

unpack()

Unpack the archive and store the contents in a sandbox.

property unpacked

Return whether the archive has been unpacked into the sandbox folder.

property version_aiida

Return the version of AiiDA the archive was created with.

Returns

version number

property version_format

Return the version of the archive format.

Returns

version number

aiida.tools.importexport.extract_zip(infile, folder, nodes_export_subfolder=None, silent=False)

Extract the nodes to be imported from a zip file.

Parameters

• infile (str) – file path

• folder (SandboxFolder) – a temporary folder used to extract the file tree

• nodes_export_subfolder (str) – name of the subfolder for AiiDA nodes

• silent (bool) – suppress debug print

Raises

• TypeError – if parameter types are not respected

• CorruptArchive – if the archive misses files or files have incorrect formats

aiida.tools.importexport.extract_tar(infile, folder, nodes_export_subfolder=None, silent=False)

Extract the nodes to be imported from a (possibly zipped) tar file.

Parameters

• infile (str) – file path

• folder (SandboxFolder) – a temporary fodler used to extract the file tree

• nodes_export_subfolder (str) – name of the subfolder for AiiDA nodes

• silent (bool) – suppress debug print

Raises

• TypeError – if parameter types are not respected

• CorruptArchive – if the archive misses files or files have incorrect formats

aiida.tools.importexport.extract_tree(infile, folder)

Prepare to import nodes from plain file system tree by copying in the given sandbox folder.

Note

the contents of the unpacked archive directory are copied into the sandbox folder, because the files will anyway haven to be copied to the repository at some point. By copying the contents of the source directory now and continuing operation only on the sandbox folder, we do not risk to modify the source files accidentally. During import then, the node files from the sandbox can be moved to the repository, so they won’t have to be copied again in any case.

Parameters

• infile (str) – absolute filepath point to the unpacked archive directory

• folder (SandboxFolder) – a temporary folder to which the archive contents are copied

exception aiida.tools.importexport.ExportImportException

Bases: aiida.common.exceptions.AiidaException

Base class for all AiiDA export/import module exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'

exception aiida.tools.importexport.ArchiveExportError

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA export exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'

exception aiida.tools.importexport.ArchiveImportError

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA import exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'

exception aiida.tools.importexport.CorruptArchive

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Raised when an operation is applied to a corrupt export archive, e.g. missing files or invalid formats.

__module__ = 'aiida.tools.importexport.common.exceptions'

exception aiida.tools.importexport.IncompatibleArchiveVersionError

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Raised when trying to import an export archive with an incompatible schema version.

__module__ = 'aiida.tools.importexport.common.exceptions'

exception aiida.tools.importexport.ExportValidationError

Bases: aiida.tools.importexport.common.exceptions.ArchiveExportError

Raised when validation fails during export, e.g. for non-sealed ProcessNode s.

__module__ = 'aiida.tools.importexport.common.exceptions'

exception aiida.tools.importexport.ImportUniquenessError

Bases: aiida.tools.importexport.common.exceptions.ArchiveImportError

Raised when the user tries to violate a uniqueness constraint.

Similar to UniquenessError.

__module__ = 'aiida.tools.importexport.common.exceptions'

exception aiida.tools.importexport.ImportValidationError

Bases: aiida.tools.importexport.common.exceptions.ArchiveImportError

Raised when validation fails during import, e.g. for parameter types and values.

__module__ = 'aiida.tools.importexport.common.exceptions'

exception aiida.tools.importexport.ArchiveMigrationError

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA export archive migration exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'

exception aiida.tools.importexport.MigrationValidationError

Bases: aiida.tools.importexport.common.exceptions.ArchiveMigrationError

Raised when validation fails during migration of export archives.

__module__ = 'aiida.tools.importexport.common.exceptions'

exception aiida.tools.importexport.DanglingLinkError

Bases: aiida.tools.importexport.common.exceptions.MigrationValidationError

Raised when an export archive is detected to contain dangling links when importing.

__module__ = 'aiida.tools.importexport.common.exceptions'

Subpackages

• aiida.tools.importexport.common package
  • Submodules
• aiida.tools.importexport.dbexport package
  • Submodules
• aiida.tools.importexport.dbimport package
  • Subpackages
    • aiida.tools.importexport.dbimport.backends package
      • Subpackages
        • aiida.tools.importexport.dbimport.backends.django package
        • aiida.tools.importexport.dbimport.backends.sqla package
          • Submodules
      • Submodules
• aiida.tools.importexport.migration package
  • Submodules