aiida.storage.sqlite_zip.migrations.legacy package¶
Legacy migrations,
using the old data.json format for storing the database.
These migrations simply manipulate the metadata and data in-place.
Submodules¶
Migration from v0.4 to v0.5, used by verdi export migrate command.
The migration steps are named similarly to the database migrations for Django and SQLAlchemy. In the description of each migration, a revision number is given, which refers to the Django migrations. The individual Django database migrations may be found at:
aiida.storage.djsite.db.migrations.00XX_<migration-name>.py
Where XX are the numbers in the migrations’ documentation: REV. 1.0.XX And migration-name is the name of the particular migration. The individual SQLAlchemy database migrations may be found at:
aiida.storage.psql_dos.migrations.versions.<id>_<migration-name>.py
Where id is a SQLA id and migration-name is the name of the particular migration.
- aiida.storage.sqlite_zip.migrations.legacy.v04_to_v05.migrate_v4_to_v5(metadata: dict, data: dict) None[source]¶
Migration of archive files from v0.4 to v0.5
This is from migration 0034 (drop_node_columns_nodeversion_public) and onwards
- aiida.storage.sqlite_zip.migrations.legacy.v04_to_v05.migration_drop_computer_transport_params(metadata, data)[source]¶
Apply migration 0036 - REV. 1.0.36 Drop the column transport_params from the Computer model
- aiida.storage.sqlite_zip.migrations.legacy.v04_to_v05.migration_drop_node_columns_nodeversion_public(metadata, data)[source]¶
Apply migration 0034 - REV. 1.0.34 Drop the columns nodeversion and public from the Node model
- aiida.storage.sqlite_zip.migrations.legacy.v04_to_v05.remove_fields(metadata, data, entities, fields)[source]¶
Remove fields under entities from data.json and metadata.json.
- Parameters
metadata – the content of an export archive metadata.json file
data – the content of an export archive data.json file
entities – list of ORM entities
fields – list of fields to be removed from the export archive files
Migration from v0.5 to v0.6, used by verdi export migrate command.
The migration steps are named similarly to the database migrations for Django and SQLAlchemy. In the description of each migration, a revision number is given, which refers to the Django migrations. The individual Django database migrations may be found at:
aiida.storage.djsite.db.migrations.00XX_<migration-name>.py
Where XX are the numbers in the migrations’ documentation: REV. 1.0.XX And migration-name is the name of the particular migration. The individual SQLAlchemy database migrations may be found at:
aiida.storage.psql_dos.migrations.versions.<id>_<migration-name>.py
Where id is a SQLA id and migration-name is the name of the particular migration.
- aiida.storage.sqlite_zip.migrations.legacy.v05_to_v06.migrate_deserialized_datetime(data, conversion)[source]¶
Deserialize datetime strings from export archives, meaning to reattach the UTC timezone information.
- aiida.storage.sqlite_zip.migrations.legacy.v05_to_v06.migrate_v5_to_v6(metadata: dict, data: dict) None[source]¶
Migration of archive files from v0.5 to v0.6
- aiida.storage.sqlite_zip.migrations.legacy.v05_to_v06.migration_migrate_legacy_job_calculation_data(data)[source]¶
Apply migration 0038 - REV. 1.0.38
Migrates legacy JobCalculation data to the new process system. Essentially old JobCalculation nodes, which have already been migrated to CalcJobNodes, are missing important attributes process_state, exit_status and process_status. These are inferred from the old state attribute, which is then discarded as its values have been deprecated.
- aiida.storage.sqlite_zip.migrations.legacy.v05_to_v06.migration_serialize_datetime_objects(data)[source]¶
Apply migration 0037 - REV. 1.0.37
Migrates the node attributes and extras from the EAV schema to JSONB columns. Since JSON does not support datetime objects, and the EAV did, existing datetime objects have to be serialized to strings. Just like the database migration they were serialized to the standard ISO format, except that they were first converted to UTC timezone and then the stored without a timezone reference. Since existing datetimes in the attributes and extras in the database were timezone aware and have been migrated to an ISO format string including the timezone information we should now add the same timezone information to datetime attributes and extras in existing export archives. All that one needs to do for this is to append the +00:00 suffix, which signifies the UTC timezone.
Since the datetime objects were the only types being serialized in the attributes and extras, after the reinstating of the timeonze information, there is no longer a need for the de/serialization dictionaries for each node, stored in node_attributes_conversion and node_extras_conversion, respectively. They are no longer added to new archives and so they can and should be removed from existing archives, reducing the size enormously.
Migration from v0.6 to v0.7, used by verdi export migrate command.
The migration steps are named similarly to the database migrations for Django and SQLAlchemy. In the description of each migration, a revision number is given, which refers to the Django migrations. The individual Django database migrations may be found at:
aiida.storage.djsite.db.migrations.00XX_<migration-name>.py
Where XX are the numbers in the migrations’ documentation: REV. 1.0.XX And migration-name is the name of the particular migration. The individual SQLAlchemy database migrations may be found at:
aiida.storage.psql_dos.migrations.versions.<id>_<migration-name>.py
Where id is a SQLA id and migration-name is the name of the particular migration.
- aiida.storage.sqlite_zip.migrations.legacy.v06_to_v07.data_migration_legacy_process_attributes(data)[source]¶
Apply migration 0040 - REV. 1.0.40 Data migration for some legacy process attributes.
Attribute keys that are renamed:
_sealed -> sealed
Attribute keys that are removed entirely:
_finished
_failed
_aborted
_do_abort
Finally, after these first migrations, any remaining process nodes are screened for the existence of the process_state attribute. If they have it, it is checked whether the state is active or not, if not, the sealed attribute is created and set to True.
- Raises
CorruptStorage – if a Node, found to have attributes, cannot be found in the list of exported entities.
CorruptStorage – if the ‘sealed’ attribute does not exist and the ProcessNode is in an active state, i.e. process_state is one of (‘created’, ‘running’, ‘waiting’). A log-file, listing all illegal ProcessNodes, will be produced in the current directory.
- aiida.storage.sqlite_zip.migrations.legacy.v06_to_v07.migrate_v6_to_v7(metadata: dict, data: dict) None[source]¶
Migration of archive files from v0.6 to v0.7
- aiida.storage.sqlite_zip.migrations.legacy.v06_to_v07.remove_attribute_link_metadata(metadata)[source]¶
Remove Attribute and Link from metadata.json This is not a database migration, but purely to do with the import/export schema.
The “entities” ATTRIBUTE_ENTITY_NAME and LINK_ENTITY_NAME were introduced in v0.3 or v0.4 of the import/export schema. However, they were never used - or the usage has been reverted. They will be removed from metadata.json, slightly simplifying the import functions as well.
Migration from v0.7 to v0.8, used by verdi export migrate command.
The migration steps are named similarly to the database migrations for Django and SQLAlchemy. In the description of each migration, a revision number is given, which refers to the Django migrations. The individual Django database migrations may be found at:
aiida.storage.djsite.db.migrations.00XX_<migration-name>.py
Where XX are the numbers in the migrations’ documentation: REV. 1.0.XX And migration-name is the name of the particular migration. The individual SQLAlchemy database migrations may be found at:
aiida.storage.psql_dos.migrations.versions.<id>_<migration-name>.py
Where id is a SQLA id and migration-name is the name of the particular migration.
- aiida.storage.sqlite_zip.migrations.legacy.v07_to_v08.migrate_v7_to_v8(metadata: dict, data: dict) None[source]¶
Migration of archive files from v0.7 to v0.8.
- aiida.storage.sqlite_zip.migrations.legacy.v07_to_v08.migration_default_link_label(data: dict)[source]¶
Apply migration 0043 - REV. 1.0.43
Rename all link labels _return to result.
Migration from v0.8 to v0.9, used by verdi export migrate command.
The migration steps are named similarly to the database migrations for Django and SQLAlchemy. In the description of each migration, a revision number is given, which refers to the Django migrations. The individual Django database migrations may be found at:
aiida.storage.djsite.db.migrations.00XX_<migration-name>.py
Where XX are the numbers in the migrations’ documentation: REV. 1.0.XX And migration-name is the name of the particular migration. The individual SQLAlchemy database migrations may be found at:
aiida.storage.psql_dos.migrations.versions.<id>_<migration-name>.py
Where id is a SQLA id and migration-name is the name of the particular migration.
- aiida.storage.sqlite_zip.migrations.legacy.v08_to_v09.migrate_v8_to_v9(metadata: dict, data: dict) None[source]¶
Migration of archive files from v0.8 to v0.9.
- aiida.storage.sqlite_zip.migrations.legacy.v08_to_v09.migration_dbgroup_type_string(data)[source]¶
Apply migration 0044 - REV. 1.0.44
Rename the type_string columns of all Group instances.
Migration from v0.9 to v0.10, used by verdi export migrate command.
- aiida.storage.sqlite_zip.migrations.legacy.v09_to_v10.migrate_v9_to_v10(metadata: dict, data: dict) None[source]¶
Migration of archive files from v0.9 to v0.10.
Migration from v0.10 to v0.11, used by verdi archive migrate command.
This migration applies the name change of the Computer attribute name to label.
- aiida.storage.sqlite_zip.migrations.legacy.v10_to_v11.migrate_v10_to_v11(metadata: dict, data: dict) None[source]¶
Migration of export files from v0.10 to v0.11.
Migration from v0.11 to v0.12, used by verdi archive migrate command.
This migration is necessary after the core. prefix was added to entry points shipped with aiida-core.