Utility commands

The preparation of the environment is carried out using the luna_prepare script, to which specific commands are passed, enabling for detailed configuration of the preparation process.

List of available commands:

• check — Command that enables you to check the existing environment in accordance with the specified information about the LUNA PLATFORM version.
• list — Command that enables you to show a list of available versions of LUNA PLATFORM for preparing the environment.
• prepare — Main command, which enables you to prepare the environment in accordance with the specified LUNA PLATFORM version information.
• load_dump — Command that enables you to load settings from a user dump file into the Configurator database.
• logs — Command that enables you to display or save to a file special logs received during the environment preparation stage.

Each command except the list command has its own arguments. A list of possible arguments with their basic description can be obtained using the help argument --help.

For an extended description of the arguments, see "Named arguments" section.

In addition, the prepare command has two types of arguments — positional and named (see "Prepare command") and you can also get help information for each type of argument.

For example, you can get a list of arguments for the load_dump command with the following command:

docker run \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/storages:v.0.4.6 \
bash -c "luna_prepare load_dump --help"

Important: It is highly recommended that you use help arguments when working with the Storages utility.

List command

The Storages utility does not work with all versions of LUNA PLATFORM. The list command enables you to get a list of supported versions of the environment preparation run.

Important: The version list also contains releases intended for internal use. Only public versions of LUNA PLATFORM must be used.

Example of a command to get a list of versions:

docker run \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/storages:v.0.4.6 \
bash -c "luna_prepare list"

Example of successful command execution:

[2024-02-07 16:20:50+0300] [storages] [INFO]: Available platform versions for surroundings preparation:
[2024-02-07 16:20:50+0300] [storages] [INFO]: v.5.46.1, v.5.47.1, v.5.47.4, v.5.49.1, v.5.51.0, v.5.51.4, v.5.51.6, v.5.53.0, v.5.54.0, v.5.55.0, v.5.56.0, v.5.57.0

Check command

The check command enables you to check the current environment and get information about what is needed to install an environment of a different version. Using this command, for example, you can verify whether the user really has a certain version of the environment installed.

The following arguments are available for the check command:

• --config
• --luna-config
• --profile
• --platform_version
• --s3-buckets
• --local-buckets

Example of a command to check the current environment:

docker run \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/storages:v.0.4.6 \
bash -c "luna_prepare check \
    --platform_version=v.5.53.0 \
    --profile=common \
    --luna-config=http://10.16.5.177:5070"

The example checks the environment for all LUNA PLATFORM v.5.53.0 services, except for the Backport 3 and Backport 4 services, and also checks the availability of buckets using the corresponding settings "LUNA_IMAGE_STORE_bucket_name_ADDRESS" from the Configurator service launched at the address from the --luna-config argument.

If the installed environment matches the version specified in the --platform_version argument, then a message similar to this will appear in the logs:

[2024-02-06 13:57:01+0300] [storages] [INFO]: The surrounding state is completely prepared for platform version 'v.5.53.0'

If the installed environment does not match the version specified in the --platform_version argument, then a message similar to this will appear in the logs:

[2024-02-08 18:19:48+0300] [storages] [INFO]: luna_tasks[v.3.18.0] will be added database entity: tasks_database_migration with revision 75d45d56f7f7
[2024-02-08 18:19:48+0300] [storages] [INFO]: luna_tasks[v.3.18.0] will be updated configs migration version from '6a3d8839' to 'e7490433'

This means that when the prepare command is executed, the Tasks database will be migrated and the Configurator settings migration revision will be updated to the specified one.

Prepare command

The prepare command prepares the environment according to the specified version of LUNA PLATFORM.

The preparation of the environment is understood as follows:

• Preparing buckets in InfluxDB for monitoring work.
• Preparing buckets for the Image Store service, enabling you to store user data (images, metadata, archives, etc.).
• Preparing the InfluxDB for collection of aggregated statistics by the Admin service (see "Requests and estimations statistics gathering" section in the administrator manual).
• Preparing databases, adding VLMatch functionality, creating and managing migration scripts for LUNA PLATFORM databases (PostgreSQL or Oracle).
• Performing migration/loading settings into the Configurator database.

All of the above tasks are specified as separate command arguments (see below).

Note: If necessary, you can record environment preparation execution logs to solve specific problems that require contacting VisionLabs specialists. To do this, you need to specify additional data in the environment preparation command. By default, logs are not saved. See "Logs command" for more information.

Important: The Storages utility will not prepare the environment for services that are disabled in the "ADDITIONAL_SERVICES_USAGE" setting.

Types of passed arguments

For the prepare command, you can pass additional arguments that enable you to configure the preparation of the environment in detail.

Arguments can be:

• Positional (mandatory)
• Named (optional, having a default value)

The positional argument must appear exactly after the prepare command. Named arguments appear after positional ones.

The table below provides a list of possible positional and named arguments for the prepare command.

Positional argument	Description
	Preparing entities
lis_bucket	Enables you to create buckets in the Image Store container. Bucket settings are specified in the "LUNA_IMAGE_STORE_bucket_name_ADDRESS" sections in the Storages utility settings (--config) or in the Configurator service settings (--luna-config). List of available named arguments: --help, --verbose, --config, --luna-config, --ignore-integrity, --platform_version, --s3-buckets, --local-buckets, --profile, --ADDITIONAL_SERVICES_USAGE, arguments for passing configuration tags with bucket addresses.
s3_bucket	Enables you to create buckets in S3 storage without accessing the Image Store service. Bucket settings are specified in the "S3" section, and bucket names in the "LUNA_IMAGE_STORE_bucket_name_ADDRESS" sections in the Storages utility settings (--config) or in the Configurator service settings (--luna-config). List of available named arguments: --help, --verbose, --config, --luna-config, --ignore-integrity, --platform_version, --profile, --ADDITIONAL_SERVICES_USAGE, --LAMBDA_S3.
influx_bucket	Enables you to create buckets in the InfluxDB. List of available named arguments: --help, --verbose, --config, --luna-config, --ignore-integrity, --platform_version, --profile, --LUNA_MONITORING. |
aggregated_influx_bucket	Allows you to create a "luna_monitoring_aggregated" bucket in the Influx database and enable statistics collection (see section "Requests and estimations statistics gathering" in the administrator manual). List of available named arguments: --help, --verbose, --config, --luna-config, --ignore-integrity, --platform_version, --profile, --dry, --LUNA_MONITORING. |
database	Enables creating databases, adding VLMatch functionality, creating and managing migration scripts for LUNA PLATFORM databases (PostgreSQL or Oracle). List of available named arguments: --help, --verbose, --config, --luna-config, --ignore-integrity, --platform_version, --profile, --db-password, --db-user, --ADDITIONAL_SERVICES_USAGE, Arguments for passing database settings tags.
configs	Migrates Configurator service settings. List of available named arguments: --help, --verbose, --config, --luna-config, --ignore-integrity, --platform_version, --profile, --configs-revision.
all_entities	Uses positional arguments lis_bucket, influx_bucket, aggregated_influx_bucket, database and configs. List of available named arguments: all of the above and --dump-file.
	Specifying service for preparing environment using one positional argument from the list above
<service_name>	Name of the service (configurator, remote_sdk, etc.) for which you need to prepare the environment. The entity for preparing the environment is specified separately in the named argument --entity, available for each service. See the luna_prepare prepare <service> --help command for a list of available named arguments.

A description of all named arguments is given in the table "Named arguments".

Important: There is no positional argument that prepares the environment for all services. The profile (a link to a list of services) is specified in the named argument --profile, which is available for all positional arguments except <service_name>. For the positional argument <service_name> there is a separate logic for selecting the environment for preparation, specified in the --entity flag. If the named argument --entity is not specified, then an environment for all entities will be prepared.

See the example of the prepare command in the "Upgrade environment scenario" section.

Load_dump command

The load_dump command enables you to load user settings into the Configurator service.

The following arguments are available for the load_dump command:

• --help
• --verbose
• --config
• --clear-database
• --dump-file

For example, you can load the supplied custom settings using the following command:

docker run \
--rm \
--network=host \
-v /var/lib/luna/current/extras/conf/platform_settings.json:/srv/platform_settings.json \
dockerhub.visionlabs.ru/luna/storages:v.0.4.6 \
bash -c "luna_prepare load_dump \
    -v \
    --dump-file=/srv/platform_settings.json"

Example of successful command execution:

[2024-02-07 16:25:33+0300] [storages] [INFO]: getting settings-dump file
[2024-02-07 16:25:33+0300] [storages] [INFO]: start updating settings
[2024-02-07 16:25:34+0300] [storages] [INFO]: update setting with name 'ADDITIONAL_SERVICES_USAGE'
[2024-02-07 16:25:34+0300] [storages] [INFO]: update setting with name 'LICENSE_VENDOR'
...
[2024-02-07 16:25:34+0300] [storages] [INFO]: applied settings: 20 out of 20
[2024-02-07 16:25:34+0300] [storages] [INFO]: database is ready to use

Logs command

The logs command enables you to get special logs in JSON format created during the environment preparation stage. Each log contains the following information about the action performed:

• migration_action — Performed action (create, update, downgrade or delete).
• migration_datetime — Time of execution of the action.
• migration_problems — Problems encountered.
• migration_source — Source (for example, the source revision version).
• migration_target — Target (for example, the revision version to which the migration was performed).
• target_version — Version of LUNA PLATFORM corresponding to the action being performed.

Not all log fields may be filled in (for example, migration_problems': None). Also, some logs may be missing if no problems occurred (for example, entity preparation logs influx_bucket).

Receiving logs is optional. It is recommended to configure the receipt of logs in case of problems that require contacting VisionLabs specialists.

Logs are saved in a separate file in SQLite database format.

The following arguments are available for the logs command:

• --save-file
• --tail

To ensure that logs are recorded during environment preparation, the following is required:

• Create a directory to store the SQLite database file.
• Change the owner and group for the specified directory.
• In the environment preparation command, mount the created directory.
• In the environment preparation command, set the environment variable "DB_PATH", which will indicate the path to the mounted directory (by default "/srv/sqlite_db/data").

Example of Prepare command with logging enabled

Create a directory to store the SQLite database file:

mkdir -p /var/lib/luna/sqlite_db/data

Change the owner and group for the specified directory:

chown -R 1001:0 /var/lib/luna/sqlite_db/data

Prepare the environment and save logs to a SQLite database file:

docker run \
--rm \
--env=DB_PATH="/srv/my_data" \
-v /var/lib/luna/sqlite_db/data/:/srv/my_data \
--network=host \
dockerhub.visionlabs.ru/luna/storages:v.0.4.6 \
bash -c "luna_prepare prepare all_entities \
    --platform_version=v.5.57.0"

Here:

• -v /root/my_data/:/srv/sqlite_db/data \ — Mount the directory for storing the SQLite database file.
• --env=DB_PATH="/srv/my_data" \ — Specifying an environment variable containing the path to the directory with the SQLite database file inside the container.

Example of Logs command to get environment preparation logs

After preparing the environment, you can get logs using the following command:

docker run \
--rm \
--network=host \
--env=DB_PATH="/srv/sqlite_db/data" \
-v /var/lib/luna/sqlite_db/data/:/srv/sqlite_db/data \
dockerhub.visionlabs.ru/luna/storages:v.0.4.6 \
bash -c "luna_prepare logs \
    --save-file=/srv/sqlite_db/data/storages_logs.log \
    --tail=30"

Here:

• --save-file=/srv/sqlite_db/data/storages_logs.log — Address inside the container where the log file should be saved. In this case, the file will be saved in a mounted directory and will be available on the host machine.
• --tail=30 — Number of lines with logs in the storages_logs.log file.