aiida.tools.importexport.common package

Common utility functions, classes, and exceptions

exception aiida.tools.importexport.common.ArchiveExportError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA export exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ArchiveImportError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA import exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ArchiveMigrationError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA export archive migration exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.CorruptArchive[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Raised when an operation is applied to a corrupt export archive, e.g. missing files or invalid formats.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.DanglingLinkError[source]

Bases: aiida.tools.importexport.common.exceptions.MigrationValidationError

Raised when an export archive is detected to contain dangling links when importing.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ExportImportException[source]

Bases: aiida.common.exceptions.AiidaException

Base class for all AiiDA export/import module exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ExportValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveExportError

Raised when validation fails during export, e.g. for non-sealed ProcessNode s.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ImportUniquenessError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveImportError

Raised when the user tries to violate a uniqueness constraint.

Similar to UniquenessError.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ImportValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveImportError

Raised when validation fails during import, e.g. for parameter types and values.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.IncompatibleArchiveVersionError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Raised when trying to import an export archive with an incompatible schema version.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.MigrationValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveMigrationError

Raised when validation fails during migration of export archives.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ProgressBarError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Something is wrong with setting up the tqdm progress bar

__module__ = 'aiida.tools.importexport.common.exceptions'
aiida.tools.importexport.common.extract_tar(infile, folder, nodes_export_subfolder=None, check_files='data.json', 'metadata.json', **kwargs)[source]

Extract the nodes to be imported from a (possibly zipped) tar file.

Parameters

  • infile (str) – file path

  • folder (SandboxFolder) – a temporary fodler used to extract the file tree

  • nodes_export_subfolder (str) – name of the subfolder for AiiDA nodes

  • check_files – list of files to check are present

  • silent (bool) – suppress progress bar

Raises

  • TypeError – if parameter types are not respected

  • CorruptArchive – if the archive misses files or files have incorrect formats

aiida.tools.importexport.common.extract_tree(infile, folder)[source]

Prepare to import nodes from plain file system tree by copying in the given sandbox folder.

Note

the contents of the unpacked archive directory are copied into the sandbox folder, because the files will anyway haven to be copied to the repository at some point. By copying the contents of the source directory now and continuing operation only on the sandbox folder, we do not risk to modify the source files accidentally. During import then, the node files from the sandbox can be moved to the repository, so they won’t have to be copied again in any case.

Parameters

  • infile (str) – absolute filepath point to the unpacked archive directory

  • folder (SandboxFolder) – a temporary folder to which the archive contents are copied

aiida.tools.importexport.common.extract_zip(infile, folder, nodes_export_subfolder=None, check_files='data.json', 'metadata.json', **kwargs)[source]

Extract the nodes to be imported from a zip file.

Parameters

  • infile (str) – file path

  • folder (SandboxFolder) – a temporary folder used to extract the file tree

  • nodes_export_subfolder (str) – name of the subfolder for AiiDA nodes

  • check_files – list of files to check are present

  • silent (bool) – suppress progress bar

Raises

  • TypeError – if parameter types are not respected

  • CorruptArchive – if the archive misses files or files have incorrect formats

Submodules

Utility functions and classes to interact with AiiDA export archives.

aiida.tools.importexport.common.archive.extract_tar(infile, folder, nodes_export_subfolder=None, check_files='data.json', 'metadata.json', **kwargs)[source]

Extract the nodes to be imported from a (possibly zipped) tar file.

Parameters

  • infile (str) – file path

  • folder (SandboxFolder) – a temporary fodler used to extract the file tree

  • nodes_export_subfolder (str) – name of the subfolder for AiiDA nodes

  • check_files – list of files to check are present

  • silent (bool) – suppress progress bar

Raises

  • TypeError – if parameter types are not respected

  • CorruptArchive – if the archive misses files or files have incorrect formats

aiida.tools.importexport.common.archive.extract_tree(infile, folder)[source]

Prepare to import nodes from plain file system tree by copying in the given sandbox folder.

Note

the contents of the unpacked archive directory are copied into the sandbox folder, because the files will anyway haven to be copied to the repository at some point. By copying the contents of the source directory now and continuing operation only on the sandbox folder, we do not risk to modify the source files accidentally. During import then, the node files from the sandbox can be moved to the repository, so they won’t have to be copied again in any case.

Parameters

  • infile (str) – absolute filepath point to the unpacked archive directory

  • folder (SandboxFolder) – a temporary folder to which the archive contents are copied

aiida.tools.importexport.common.archive.extract_zip(infile, folder, nodes_export_subfolder=None, check_files='data.json', 'metadata.json', **kwargs)[source]

Extract the nodes to be imported from a zip file.

Parameters

  • infile (str) – file path

  • folder (SandboxFolder) – a temporary folder used to extract the file tree

  • nodes_export_subfolder (str) – name of the subfolder for AiiDA nodes

  • check_files – list of files to check are present

  • silent (bool) – suppress progress bar

Raises

  • TypeError – if parameter types are not respected

  • CorruptArchive – if the archive misses files or files have incorrect formats

Configuration file for AiiDA Import/Export module

Module that defines the exceptions thrown by AiiDA’s export/import module.

Note: In order to not override the built-in ImportError, both ImportError and ExportError are prefixed with

Archive.

exception aiida.tools.importexport.common.exceptions.ArchiveExportError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA export exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ArchiveImportError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA import exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ArchiveMigrationError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA export archive migration exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.CorruptArchive[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Raised when an operation is applied to a corrupt export archive, e.g. missing files or invalid formats.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.DanglingLinkError[source]

Bases: aiida.tools.importexport.common.exceptions.MigrationValidationError

Raised when an export archive is detected to contain dangling links when importing.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ExportImportException[source]

Bases: aiida.common.exceptions.AiidaException

Base class for all AiiDA export/import module exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ExportValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveExportError

Raised when validation fails during export, e.g. for non-sealed ProcessNode s.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ImportUniquenessError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveImportError

Raised when the user tries to violate a uniqueness constraint.

Similar to UniquenessError.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ImportValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveImportError

Raised when validation fails during import, e.g. for parameter types and values.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.IncompatibleArchiveVersionError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Raised when trying to import an export archive with an incompatible schema version.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.MigrationValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveMigrationError

Raised when validation fails during migration of export archives.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ProgressBarError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Something is wrong with setting up the tqdm progress bar

__module__ = 'aiida.tools.importexport.common.exceptions'

Utility functions for import/export of AiiDA entities

class aiida.tools.importexport.common.utils.HTMLGetLinksParser(filter_extension=None)[source]

Bases: html.parser.HTMLParser

If a filter_extension is passed, only links with extension matching the given one will be returned.

__init__(filter_extension=None)[source]

Initialize and reset this instance.

If convert_charrefs is True (the default), all character references are automatically converted to the corresponding Unicode characters.

__module__ = 'aiida.tools.importexport.common.utils'

Return the links that were found during the parsing phase.

handle_starttag(tag, attrs)[source]

Store the urls encountered, if they match the request.

aiida.tools.importexport.common.utils.export_shard_uuid(uuid)[source]

Sharding of the UUID for the import/export

Parameters

uuid (str) – UUID to be sharded (v4)

Returns

Sharded UUID as a subfolder path

Return type

str

Open the given URL, parse the HTML and return a list of valid links where the link file has a .aiida extension.

aiida.tools.importexport.common.utils.schema_to_entity_names(class_string)[source]

Mapping from classes path to entity names (used by the SQLA import/export) This could have been written much simpler if it is only for SQLA but there is an attempt the SQLA import/export code to be used for Django too.

aiida.tools.importexport.archive.migrations package aiida.tools.importexport.dbexport package