aiida.tools.importexport.common package¶
Common utility functions, classes, and exceptions
-
class
aiida.tools.importexport.common.
Archive
(filepath)[source]¶ Bases:
object
Utility class to operate on exported archive files or directories.
The main usage should be to construct the class with the filepath of the export archive as an argument. The contents will be lazily unpacked into a sand box folder which is constructed upon entering the instance within a context and which will be automatically cleaned upon leaving that context. Example:
with Archive('/some/path/archive.aiida') as archive: archive.version
-
FILENAME_DATA
= 'data.json'¶
-
FILENAME_METADATA
= 'metadata.json'¶
-
__dict__
= mappingproxy({'__module__': 'aiida.tools.importexport.common.archive', '__doc__': "Utility class to operate on exported archive files or directories.\n\n The main usage should be to construct the class with the filepath of the export archive as an argument.\n The contents will be lazily unpacked into a sand box folder which is constructed upon entering the instance\n within a context and which will be automatically cleaned upon leaving that context. Example::\n\n with Archive('/some/path/archive.aiida') as archive:\n archive.version\n\n ", 'FILENAME_DATA': 'data.json', 'FILENAME_METADATA': 'metadata.json', '__init__': <function Archive.__init__>, '__enter__': <function Archive.__enter__>, '__exit__': <function Archive.__exit__>, 'ensure_within_context': <AdapterWrapper at 0x7fef14fbb180 for function>, 'ensure_unpacked': <AdapterWrapper at 0x7fef14fbb1e8 for function>, 'unpack': <FunctionWrapper at 0x7fef14fbb250 for function>, 'filepath': <property object>, 'folder': <property object>, 'data': <property object>, 'meta_data': <property object>, 'unpacked': <property object>, 'get_info': <FunctionWrapper at 0x7fef14fbb3f0 for function>, 'get_data_statistics': <FunctionWrapper at 0x7fef14fbb458 for function>, 'version_aiida': <property object>, 'version_format': <property object>, 'conversion_info': <property object>, '_read_json_file': <FunctionWrapper at 0x7fef14fbb798 for FunctionWrapper>, '__dict__': <attribute '__dict__' of 'Archive' objects>, '__weakref__': <attribute '__weakref__' of 'Archive' objects>})¶
-
__module__
= 'aiida.tools.importexport.common.archive'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
_read_json_file
(filename)[source]¶ Read the contents of a JSON file from the unpacked archive contents.
- Parameters
filename – the filename relative to the sandbox folder
- Returns
a dictionary with the loaded JSON content
-
property
conversion_info
¶ Return information about migration events that were applied to this archive.
- Returns
list of conversion notifications
-
property
data
¶ Return the loaded content of the data file
- Returns
dictionary with contents of data file
-
ensure_unpacked
(instance, args, kwargs)[source]¶ Decorator to ensure that the archive is unpacked before entering the decorated function.
-
ensure_within_context
(instance, args, kwargs)[source]¶ Decorator to ensure that the instance is called within a context manager.
-
property
filepath
¶ Return the filepath of the archive
- Returns
the archive filepath
-
property
folder
¶ Return the sandbox folder
- Returns
sandbox folder
aiida.common.folders.SandboxFolder
-
get_data_statistics
()[source]¶ Return dictionary with statistics about data content, i.e. how many entries of each entity type it contains.
- Returns
a dictionary with basic details
-
get_info
()[source]¶ Return a dictionary with basic information about the archive.
- Returns
a dictionary with basic details
-
property
meta_data
¶ Return the loaded content of the meta data file
- Returns
dictionary with contents of meta data file
-
property
unpacked
¶ Return whether the archive has been unpacked into the sandbox folder.
-
property
version_aiida
¶ Return the version of AiiDA the archive was created with.
- Returns
version number
-
property
version_format
¶ Return the version of the archive format.
- Returns
version number
-
-
aiida.tools.importexport.common.
extract_zip
(infile, folder, nodes_export_subfolder=None, silent=False)[source]¶ Extract the nodes to be imported from a zip file.
- Parameters
infile (str) – file path
folder (
SandboxFolder
) – a temporary folder used to extract the file treenodes_export_subfolder (str) – name of the subfolder for AiiDA nodes
silent (bool) – suppress debug print
- Raises
TypeError – if parameter types are not respected
CorruptArchive – if the archive misses files or files have incorrect formats
-
aiida.tools.importexport.common.
extract_tar
(infile, folder, nodes_export_subfolder=None, silent=False)[source]¶ Extract the nodes to be imported from a (possibly zipped) tar file.
- Parameters
infile (str) – file path
folder (
SandboxFolder
) – a temporary fodler used to extract the file treenodes_export_subfolder (str) – name of the subfolder for AiiDA nodes
silent (bool) – suppress debug print
- Raises
TypeError – if parameter types are not respected
CorruptArchive – if the archive misses files or files have incorrect formats
-
aiida.tools.importexport.common.
extract_tree
(infile, folder)[source]¶ Prepare to import nodes from plain file system tree by copying in the given sandbox folder.
Note
the contents of the unpacked archive directory are copied into the sandbox folder, because the files will anyway haven to be copied to the repository at some point. By copying the contents of the source directory now and continuing operation only on the sandbox folder, we do not risk to modify the source files accidentally. During import then, the node files from the sandbox can be moved to the repository, so they won’t have to be copied again in any case.
- Parameters
infile (str) – absolute filepath point to the unpacked archive directory
folder (
SandboxFolder
) – a temporary folder to which the archive contents are copied
-
exception
aiida.tools.importexport.common.
ExportImportException
[source]¶ Bases:
aiida.common.exceptions.AiidaException
Base class for all AiiDA export/import module exceptions.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.
ArchiveExportError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ExportImportException
Base class for all AiiDA export exceptions.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.
ArchiveImportError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ExportImportException
Base class for all AiiDA import exceptions.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.
CorruptArchive
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ExportImportException
Raised when an operation is applied to a corrupt export archive, e.g. missing files or invalid formats.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.
IncompatibleArchiveVersionError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ExportImportException
Raised when trying to import an export archive with an incompatible schema version.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.
ExportValidationError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ArchiveExportError
Raised when validation fails during export, e.g. for non-sealed
ProcessNode
s.-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.
ImportUniquenessError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ArchiveImportError
Raised when the user tries to violate a uniqueness constraint.
Similar to
UniquenessError
.-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.
ImportValidationError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ArchiveImportError
Raised when validation fails during import, e.g. for parameter types and values.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.
ArchiveMigrationError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ExportImportException
Base class for all AiiDA export archive migration exceptions.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.
MigrationValidationError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ArchiveMigrationError
Raised when validation fails during migration of export archives.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.
DanglingLinkError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.MigrationValidationError
Raised when an export archive is detected to contain dangling links when importing.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
Submodules¶
Utility functions and classes to interact with AiiDA export archives.
-
class
aiida.tools.importexport.common.archive.
Archive
(filepath)[source]¶ Bases:
object
Utility class to operate on exported archive files or directories.
The main usage should be to construct the class with the filepath of the export archive as an argument. The contents will be lazily unpacked into a sand box folder which is constructed upon entering the instance within a context and which will be automatically cleaned upon leaving that context. Example:
with Archive('/some/path/archive.aiida') as archive: archive.version
-
FILENAME_DATA
= 'data.json'¶
-
FILENAME_METADATA
= 'metadata.json'¶
-
__dict__
= mappingproxy({'__module__': 'aiida.tools.importexport.common.archive', '__doc__': "Utility class to operate on exported archive files or directories.\n\n The main usage should be to construct the class with the filepath of the export archive as an argument.\n The contents will be lazily unpacked into a sand box folder which is constructed upon entering the instance\n within a context and which will be automatically cleaned upon leaving that context. Example::\n\n with Archive('/some/path/archive.aiida') as archive:\n archive.version\n\n ", 'FILENAME_DATA': 'data.json', 'FILENAME_METADATA': 'metadata.json', '__init__': <function Archive.__init__>, '__enter__': <function Archive.__enter__>, '__exit__': <function Archive.__exit__>, 'ensure_within_context': <AdapterWrapper at 0x7fef14fbb180 for function>, 'ensure_unpacked': <AdapterWrapper at 0x7fef14fbb1e8 for function>, 'unpack': <FunctionWrapper at 0x7fef14fbb250 for function>, 'filepath': <property object>, 'folder': <property object>, 'data': <property object>, 'meta_data': <property object>, 'unpacked': <property object>, 'get_info': <FunctionWrapper at 0x7fef14fbb3f0 for function>, 'get_data_statistics': <FunctionWrapper at 0x7fef14fbb458 for function>, 'version_aiida': <property object>, 'version_format': <property object>, 'conversion_info': <property object>, '_read_json_file': <FunctionWrapper at 0x7fef14fbb798 for FunctionWrapper>, '__dict__': <attribute '__dict__' of 'Archive' objects>, '__weakref__': <attribute '__weakref__' of 'Archive' objects>})¶
-
__module__
= 'aiida.tools.importexport.common.archive'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
_read_json_file
(filename)[source]¶ Read the contents of a JSON file from the unpacked archive contents.
- Parameters
filename – the filename relative to the sandbox folder
- Returns
a dictionary with the loaded JSON content
-
property
conversion_info
¶ Return information about migration events that were applied to this archive.
- Returns
list of conversion notifications
-
property
data
¶ Return the loaded content of the data file
- Returns
dictionary with contents of data file
-
ensure_unpacked
(instance, args, kwargs)[source]¶ Decorator to ensure that the archive is unpacked before entering the decorated function.
-
ensure_within_context
(instance, args, kwargs)[source]¶ Decorator to ensure that the instance is called within a context manager.
-
property
filepath
¶ Return the filepath of the archive
- Returns
the archive filepath
-
property
folder
¶ Return the sandbox folder
- Returns
sandbox folder
aiida.common.folders.SandboxFolder
-
get_data_statistics
()[source]¶ Return dictionary with statistics about data content, i.e. how many entries of each entity type it contains.
- Returns
a dictionary with basic details
-
get_info
()[source]¶ Return a dictionary with basic information about the archive.
- Returns
a dictionary with basic details
-
property
meta_data
¶ Return the loaded content of the meta data file
- Returns
dictionary with contents of meta data file
-
property
unpacked
¶ Return whether the archive has been unpacked into the sandbox folder.
-
property
version_aiida
¶ Return the version of AiiDA the archive was created with.
- Returns
version number
-
property
version_format
¶ Return the version of the archive format.
- Returns
version number
-
-
aiida.tools.importexport.common.archive.
extract_zip
(infile, folder, nodes_export_subfolder=None, silent=False)[source]¶ Extract the nodes to be imported from a zip file.
- Parameters
infile (str) – file path
folder (
SandboxFolder
) – a temporary folder used to extract the file treenodes_export_subfolder (str) – name of the subfolder for AiiDA nodes
silent (bool) – suppress debug print
- Raises
TypeError – if parameter types are not respected
CorruptArchive – if the archive misses files or files have incorrect formats
-
aiida.tools.importexport.common.archive.
extract_tar
(infile, folder, nodes_export_subfolder=None, silent=False)[source]¶ Extract the nodes to be imported from a (possibly zipped) tar file.
- Parameters
infile (str) – file path
folder (
SandboxFolder
) – a temporary fodler used to extract the file treenodes_export_subfolder (str) – name of the subfolder for AiiDA nodes
silent (bool) – suppress debug print
- Raises
TypeError – if parameter types are not respected
CorruptArchive – if the archive misses files or files have incorrect formats
-
aiida.tools.importexport.common.archive.
extract_tree
(infile, folder)[source]¶ Prepare to import nodes from plain file system tree by copying in the given sandbox folder.
Note
the contents of the unpacked archive directory are copied into the sandbox folder, because the files will anyway haven to be copied to the repository at some point. By copying the contents of the source directory now and continuing operation only on the sandbox folder, we do not risk to modify the source files accidentally. During import then, the node files from the sandbox can be moved to the repository, so they won’t have to be copied again in any case.
- Parameters
infile (str) – absolute filepath point to the unpacked archive directory
folder (
SandboxFolder
) – a temporary folder to which the archive contents are copied
Configuration file for AiiDA Import/Export module
Module that defines the exceptions thrown by AiiDA’s export/import module.
- Note: In order to not override the built-in ImportError, both ImportError and ExportError are prefixed with
Archive.
-
exception
aiida.tools.importexport.common.exceptions.
ExportImportException
[source]¶ Bases:
aiida.common.exceptions.AiidaException
Base class for all AiiDA export/import module exceptions.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.exceptions.
ArchiveExportError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ExportImportException
Base class for all AiiDA export exceptions.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.exceptions.
ArchiveImportError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ExportImportException
Base class for all AiiDA import exceptions.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.exceptions.
CorruptArchive
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ExportImportException
Raised when an operation is applied to a corrupt export archive, e.g. missing files or invalid formats.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.exceptions.
IncompatibleArchiveVersionError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ExportImportException
Raised when trying to import an export archive with an incompatible schema version.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.exceptions.
ExportValidationError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ArchiveExportError
Raised when validation fails during export, e.g. for non-sealed
ProcessNode
s.-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.exceptions.
ImportUniquenessError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ArchiveImportError
Raised when the user tries to violate a uniqueness constraint.
Similar to
UniquenessError
.-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.exceptions.
ImportValidationError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ArchiveImportError
Raised when validation fails during import, e.g. for parameter types and values.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.exceptions.
ArchiveMigrationError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ExportImportException
Base class for all AiiDA export archive migration exceptions.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.exceptions.
MigrationValidationError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.ArchiveMigrationError
Raised when validation fails during migration of export archives.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
-
exception
aiida.tools.importexport.common.exceptions.
DanglingLinkError
[source]¶ Bases:
aiida.tools.importexport.common.exceptions.MigrationValidationError
Raised when an export archive is detected to contain dangling links when importing.
-
__module__
= 'aiida.tools.importexport.common.exceptions'¶
-
Utility functions for import/export of AiiDA entities
-
class
aiida.tools.importexport.common.utils.
HTMLGetLinksParser
(filter_extension=None)[source]¶ Bases:
html.parser.HTMLParser
If a filter_extension is passed, only links with extension matching the given one will be returned.
-
__init__
(filter_extension=None)[source]¶ Initialize and reset this instance.
If convert_charrefs is True (the default), all character references are automatically converted to the corresponding Unicode characters.
-
__module__
= 'aiida.tools.importexport.common.utils'¶
-
-
aiida.tools.importexport.common.utils.
export_shard_uuid
(uuid)[source]¶ Sharding of the UUID for the import/export
-
aiida.tools.importexport.common.utils.
get_valid_import_links
(url)[source]¶ Open the given URL, parse the HTML and return a list of valid links where the link file has a .aiida extension.
-
aiida.tools.importexport.common.utils.
schema_to_entity_names
(class_string)[source]¶ Mapping from classes path to entity names (used by the SQLA import/export) This could have been written much simpler if it is only for SQLA but there is an attempt the SQLA import/export code to be used for Django too.