Before upgrade#

Make sure that you are the root user before upgrade!

Before upgrading the LUNA PLATFORM, you must perform the following actions:

See key changes from previous versions if you are upgrading from a version other than LUNA PLATFORM v.5.78.0.
Create backups.
Prepare to change the version of the neural network to extract descriptors if necessary.
Delete old symbolic link.
Unpack the distribution of the new version of LUNA PLATFORM.
Create new symbolic link.
Change group and owner for new directories.
Save user configurations of the Configurator service if they have changed.
Create log directories for new services if you have previously used logging to file.
Update license if necessary.
Set up GPU computing if you plan to use GPU.
Remove old containers if necessary.

Key changes from previous versions#

Note: When updating LUNA PLATFORM from a previous version, skip this section.

The following are the key changes from previous versions that you should pay attention to when updating from older versions of LUNA PLATFORM. Some of these changes require mandatory actions to be performed, otherwise the LUNA PLATFORM may not start or function incorrectly.

Not all changes are listed in the table below. See the LUNA PLATFORM release notes for details on all changes.

Version	Changes	Mandatory actions
5.84.0	The 110th neural network model for extracting body descriptors has been removed from the Remote SDK container. Now, by default, the 116th neural network model is used for extracting body descriptors.	Read the information described in the section "Prepare to change the neural network version", and perform certain actions before launching Remote SDK, because the default version has changed.
5.76.0	The "INFLUX_MONITORING" section has been renamed to "LUNA_MONITORING" section.	When updating the environment using Storages with different entities specified (e.g. "luna_prepare configs", "luna_prepare database", etc.), you must update the environment for InfluxDB (arguments "aggregated_influx_bucket" and "influx_bucket") strictly after migrating the settings (argument "configs"). Otherwise, the environment will not be prepared correctly. When performing the entire environment using the argument "all_entities", no additional actions are required.
5.72.1	The 107th neural network model for extracting body descriptors has been removed from the Remote SDK container.	Read the information described in the section "Prepare to change the neural network version" and perform certain actions before launching the Remote SDK if the 107th model was used.
5.67.0	The threshold value from the "redetect_score_threshold" setting of the "LUNA_REMOTE_SDK_FACE_DETECTOR_SETTINGS" section has been updated from "0.3" to "0.5".	Update the threshold value according to user logic.
5.62.3	Now, by default, the neural network model 62 for extracting face descriptors is used.	Read the information described in the section "Prepare to change the neural network version", and perform certain actions before launching Remote SDK, because the default version has changed.
5.53.0	The VisionLabs image for PostgreSQL has been updated from version 12 to version 16.	If this image was previously used, then you need to perform the migration yourself according to official documentation.

Create backups#

It is recommended to create the following backups:

Backups of all databases used with LUNA PLATFORM.
Backup of Image Store buckets.
Backup of LUNA PLATFORM services configurations.

Creating backups will enable you to restore in case of any problems during the migration process.

Backup of PostgreSQL DMBS#

A PostgreSQL database backup is performed using the pg_dumpall or pg_dump utilities.

Use the official instructions to perform a backup.

Backup of Influx database#

An InfluxDB backup is performed using the influxd backup command.

Use the official instructions to perform a backup.

Backup of Image Store buckets#

Create a backup of buckets using the following command:

cp -r /var/lib/luna/image_store /var/lib/luna/BACKUP_image_store

Dump file with service settings#

Custom values of settings for LUNA PLATFORM services (all except the Configurator service) are automatically migrated using the Configurator service migration mechanism.

If the migration of the service for some reason has lost the user configuration or the user just wants to store the old service settings for different LP versions, then you can create a dump file.

To create a dump file, use the following command (may be executed from anywhere on your server):

wget -O /var/lib/luna/BACKUP_settings_dump.json 127.0.0.1:5070/1/dump

or

curl 127.0.0.1:5070/1/dump > /var/lib/luna/BACKUP_settings_dump.json

Important: This file will not be used during the normal installation of the LUNA PLATFORM. To apply the dumped settings use the db_create.py script with the --dump-file command line argument (followed with the created dump file name): base_scripts/db_create.py --dump-file settings_dump.json. You can apply full settings dump on an empty database only. See the detailed information in the "Settings dump" section of the administrator manual.

Prepare to change the neural network version#

In some LUNA PLATFORM builds, neural network models for extracting face and body descriptors are removed, and the default model usage settings are changed. See the section "Key changes from previous versions" for more information about these changes.

If you are updating from a version where neural networks were removed, and in the previous build one of the deleted models was specified in the "DEFAULT_FACE_DESCRIPTOR_VERSION" or "DEFAULT_HUMAN_DESCRIPTOR_VERSION" settings, then the Remote SDK service will not start.

The current build of LUNA PLATFORM supports neural network models for extracting descriptors:

Object from which descriptor is extracted	Neural network models	Default model
Face	59, 60, 62	62
Body	116	116

It is necessary to perform one of the additional actions depending on the following scenarios of the work:

Continuation of the use of missing neural networks

Request to VisionLabs an old neural network model and prepare it for transfer to the new Remote SDK container after its launch (see instructions in the section "Use non-delivery neural network model" of administrator manual).

Switching to new version of the neural network with continuation of the use of old descriptors

You can switch to a new version while continuing to use old templates by completing the Additional extraction task. This will convert old templates to the new version of the neural network.

In LUNA PLATFORM releases, support for a model is first added, and only then, after several versions, the default model is updated, which ensures that the new model is available in the container when upgrading from a previous version. However, if you are upgrading from an old version of LUNA PLATFORM, it is important to make sure that the old version of the Remote SDK container has the required model to perform the Additional extraction task. The model can be requested from VisionLabs specialists or transferred from a newer version of the Remote SDK container.

Important: An exception is the process of additional extraction body descriptors, which allows you to use the Additional extraction task after launching a new Remote SDK container, which is guaranteed to have a new model. In this case, you do not need to worry about whether the neural network model is in the old container.

Steps to migrate to a new version of the neural network while continuing to use old descriptors:

Perform preliminary actions if you are not upgrading from a previous version of LUNA PLATFORM:
- additional extraction for body descriptors: If you prefer to run the Additional extraction task before upgrading, be sure to check that the old Remote SDK container has the new neural network model. If you prefer to run the Additional extraction task after upgrading (recommended), no additional actions are required.
- additional extraction for face descriptors: Be sure to check that the Remote SDK container has the new neural network model.
You can check that the model is present using the following command:

bash docker exec -it luna-remote-sdk ls fsdk/data

The directory should contain files like cnndescriptor_<nn_version>.conf.
Run the Additional extraction task:
- additional extraction for body descriptors: before or after the update
- additional extraction for face descriptors: before the update

See the "Additional extraction task" section in the administrator manual.

Specify the new neural network version in the "DEFAULT_FACE_DESCRIPTOR_VERSION" or "DEFAULT_HUMAN_DESCRIPTOR_VERSION" settings according to the "Changing the neural network version in Configurator" section:
- additional extraction for body descriptors:
  - after the update if the Additional extraction task is run before the update
  - before the update if the Additional extraction task is run after the update (otherwise the Remote SDK service may not start if the model was removed from the container)
- additional extraction for face descriptors: after the update

Switching to new version of the neural network with cessation of the use of old descriptors

Specify the new version of the neural network in the settings "DEFAULT_FACE_DESCRIPTOR_VERSION" or "DEFAULT_HUMAN_DESCRIPTOR_VERSION" before launching the Remote SDK service in accordance with the section "Change neural network version using Configurator".

If it is not necessary to extract body descriptors, then you can disable the use of the neural network using the command --env=EXTEND_CMD="--enable-body-descriptor-estimator=0" when launching Remote SDK container (see the detailed information in the section "Enable/disable several estimators and detectors" of the administrator manual).

Change neural network version using Configurator#

To change the version of the neural network, you must perform the following steps:

Open the Configurator user interface http://<configurator_server_ip>:5070.
Enter the name of the setting "DEFAULT_FACE_DESCRIPTOR_VERSION" or "DEFAULT_HUMAN_DESCRIPTOR_VERSION" in the "Setting name" field and click "Apply Filters".
Set the necessary neural network model in the "DEFAULT_FACE_DESCRIPTOR_VERSION" or "DEFAULT_HUMAN_DESCRIPTOR_VERSION" setting.
Save the changes by clicking the "Save" button.

Delete old symbolic link#

Delete the symbolic link to the previous minor version directory using the following command:

rm -f /var/lib/luna/current

Distribution unpacking#

The distribution package is an archive luna_v.5.84.0, where v.5.84.0 is a numerical identifier, describing the current LUNA PLATFORM version.

The archive includes configuration files, required for installation and exploitation. It does not include Docker images for the services. They should be downloaded from the Internet.

Move the distribution package to the directory on your server before the installation. For example, move the files to /root/ directory. The directory should not contain any other distribution or license files except the target ones.

Move the distribution to the created directory.

mv /root/luna_v.5.84.0.zip /var/lib/luna

Install the unzip archiver if it is necessary.

yum install -y unzip

Go to the folder with distribution.

cd /var/lib/luna

Unzip files.

unzip luna_v.5.84.0.zip

Symbolic link creation#

Create a symbolic link.

The link indicates that the current version of the distribution file is used to run LUNA PLATFORM.

ln -s luna_v.5.84.0 current

Changing group and owner for directories#

LP services are launched inside the containers by the "luna" user. Therefore, it is required to set permissions for this user to use the mounted volumes.

Go to the LP "example-docker" directory.

cd /var/lib/luna/current/example-docker/

Create a directory to store settings.

mkdir luna_configurator/used_dumps

Set permissions for the user with UID 1001 and group 0 to use the mounted directories.

chown -R 1001:0 luna_configurator/used_dumps

Save user configurations of the Configurator#

Note: Skip this step if the Configurator settings have not been changed.

The configurations of the Configurator service are not automatically migrated, unlike the configurations of all other services.

If your previous LP version was used with non-default Configurator service configurations, back up your "luna_configurator_postgres.conf" config file in the separate directory on your server.

cp /var/lib/luna/<your_previous_lp_version>/example-docker/luna_configurator/configs/luna_configurator_postgres.conf /var/lib/luna/BACKUP_luna_configurator_postgres.conf

This backup must be mounted to the Configurator service container that is being run.

If you are not sure if the Configurator service configurations have changed, you can compare the created backup with the Configurator configurations from the current distribution using the following command:

diff /var/lib/luna/current/example-docker/luna_configurator/configs/luna_configurator_postgres.conf /var/lib/luna/BACKUP_luna_configurator_postgres.conf

Create log directory for new services#

Skip this section if no logs were previously stored on the server.

In the new version of LUNA PLATFORM, new services could appear, for which you need to create directories with logs. It depends on the version from which you are upgrading. For example, version 5.30.0 introduced the Accounts service.

See "Logging to server" section if you have not previously used logging to a file, but want to enable it.

Following are the commands to create directories for all existing services. These commands will create and assign permissions only to missing directories.

mkdir -p /tmp/logs/configurator /tmp/logs/image-store /tmp/logs/accounts /tmp/logs/faces /tmp/logs/licenses /tmp/logs/events /tmp/logs/python-matcher /tmp/logs/handlers /tmp/logs/remote-sdk /tmp/logs/tasks /tmp/logs/tasks-worker /tmp/logs/sender /tmp/logs/api /tmp/logs/admin /tmp/logs/backport3 /tmp/logs/backport4 /tmp/logs/luna-video-agent /tmp/logs/luna-video-manager

chown -R 1001:0 /tmp/logs/configurator /tmp/logs/image-store /tmp/logs/accounts /tmp/logs/faces /tmp/logs/licenses /tmp/logs/events /tmp/logs/python-matcher /tmp/logs/handlers /tmp/logs/remote-sdk /tmp/logs/tasks /tmp/logs/tasks-worker /tmp/logs/sender /tmp/logs/api /tmp/logs/admin /tmp/logs/backport3 /tmp/logs/backport4 /tmp/logs/luna-video-agent /tmp/logs/luna-video-manager

If you need to use the Python Matcher Proxy service, then you need to additionally create the /tmp/logs/python-matcher-proxy directory and set its permissions.

License update#

To update the license, follow these steps:

Follow the steps from license activation manual.
Set settings for HASP license or Guardant license before starting Licenses container.

Actions from License activation manual#

Open the license activation manual and follow the necessary steps.

Note: This action is mandatory. The license will not work without following the steps to activate the license from the corresponding manual.

Note. When updating Guardant Control Center, you must re-issue the license key.

Calculations using GPU#

You can use GPU for the general calculations performed by Remote SDK.

Skip this section if you are not going to utilize GPU for your calculations.

You need to install NVIDIA Container Toolkit to use GPU with Docker containers. The example of the installation is given below.

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)

curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.repo | tee /etc/yum.repos.d/nvidia-docker.repo

yum install -y nvidia-container-toolkit

systemctl restart docker

Check the NVIDIA Container toolkit operating by running a base CUDA container (this container is not provided in the LP distribution and should be downloaded from the Internet):

docker run --rm --gpus all nvidia/cuda:11.4.3-base-centos7 nvidia-smi

See the NVIDIA documentation for additional information.

Attributes extraction on the GPU is engineered for maximum throughput. The input images are processed in batches. This reduces computation cost per image but does not provide the shortest latency per image.

GPU acceleration is designed for high load applications where request counts per second consistently reach thousands. It won’t be beneficial to use GPU acceleration in non-extensively loaded scenarios where latency matters.

Remove old containers (optional)#

Note: Removing old containers is not necessary when preparing your environment using the Storages utility. It is enough to simply limit the number of queries to the database, but in this case it is necessary to understand certain consequences. See the "Recommendations services behavior during environment preparation" section in the Storages utility manual.

Important: The Image Store service is an exception. During the preparation of the bucket environment, Storages makes a request to the Image Store service to obtain information about the buckets. If the Image Store container is removed, the request will fail and a migration error will occur. To migrate buckets when the Image Store container is removed, you must explicitly tell the Storages utility the location of the buckets using the --local-buckets or --s3-buckets arguments. When using local buckets, you must also mount the directory with the buckets to the Storages container.

Before launching the containers of the current minor version, stop all LUNA PLATFORM related containers of the previous minor version. It is not necessary to delete third-party services containers.

For example, to remove LP containers only use the following command:

docker container rm -f luna-configurator luna-backport3 luna-backport4 luna-sender luna-tasks luna-handlers luna-remote-sdk luna-python-matcher luna-events luna-licenses luna-faces luna-image-store luna-ui-3 luna-ui-4 luna-admin luna-api luna-tasks-worker luna-accounts luna-lambda

To see the containers names or IDs, use the following command:

docker ps -a

It is also recommended to delete old images of the containers to free space. You can use the following command to delete all unused images.

If there is enough space on the server it is recommended to perform this action only after new version of LP is successfully launched.

The command deletes all the unused images, not only the images related to LP.

docker image prune -a -f