Before upgrade#
Make sure that you are the root user before upgrade!
Before upgrading the LUNA PLATFORM, you must perform the following actions:
- See key changes from previous versions if you are upgrading from a version other than LUNA PLATFORM 5.53.0.
- Create backups.
- Prepare to change the version of the neural network to extract descriptors if necessary.
- Delete old symbolic link.
- Unpack the distribution of the new version of LUNA PLATFORM.
- Create new symbolic link.
- Change group and owner for new directories.
- Move data if you are upgrading from version 5.38.3 and below.
- Save user configurations of the Configurator service if they have changed.
- Configure SELinux and Firewall if not previously configured.
- Create log directories for new services if you have previously used logging to file.
- Activate license if necessary.
- Set up GPU computing if you plan to use GPU.
- Login to VisionLabs registry if authorization was not previously performed.
- Remove old containers.
Key changes from previous versions#
Note: When updating LUNA PLATFORM from a previous version, skip this section.
The following are the key changes from previous versions that you should pay attention to when updating from older versions of LUNA PLATFORM. Some of these changes require mandatory actions to be performed, otherwise the LUNA PLATFORM may not start or function incorrectly.
Not all changes are listed in the table below. See the LUNA PLATFORM release notes for details on all changes.
| Version | Changes | Mandatory actions |
|---|---|---|
| 5.53.0 | The VisionLabs image for PostgreSQL has been updated from version 12 to version 16. | If this image was previously used, then you need to perform the migration yourself according to official documentation. |
| 5.46.0 | The functionality for working with neural networks (detection, estimation and extraction) has been transferred from the Handlers service to the new Remote SDK service. | - |
| The 105th neural network model for extracting body descriptors has been removed from the Remote SDK container. Now, by default, the 110th neural network model is used for extracting body descriptors. | Read the information described in the section "Prepare to change the neural network version", and perform certain actions before launching Remote SDK, because the default version has changed. | |
| 5.45.1 | The default value of the "score_threshold" setting in the "LUNA_HANDLERS_FACE_DETECTOR_SETTINGS" section of the Configurator service has been changed from 0.42 to 0.5. | Check the face recognition logic if the "score_threshold" value is used that is different from the default value. |
| 5.40.0 | The PostgreSQL, InfluxDB and Image Store container launch commands now prescribe the paths of the directories for mounting located in the root directory /var/lib/luna/<db_or_bucket_folder>, unlike previous versions, where paths were prescribed for a specific version of the LUNA PLATFORM /var/lib/luna/current/example-docker/<db_or_bucket_folder>. This enables you not to move data with each update. |
Move the old PostgreSQL, InfluxDB and Image Store data to the root directory (see "Move data"), and then delete and recreate the containers by specifying new directory paths for mounting. |
| 5.38.3 | The default value of the "score_threshold" setting in the "LUNA_HANDLERS_BODY_DETECTOR_SETTINGS" section of the Configurator service has been changed from 0.3 to 0.5. | Check the body recognition logic if the "score_threshold" value is used that is different from the default value. |
| The default value of the "redetect_face_target_size" setting in the "LUNA_HANDLERS_FACE_DETECTOR_SETTINGS" section of the Configurator service has been changed from 45 to 64. | Check the face recognition logic if the "redetect_face_target_size" value is used that is different from the default value. | |
| 5.36.5 | The address of the license server must now be set in the Configurator settings before starting the Licenses container. | Perform the steps described in the "Specify license server address using Configurator" section. |
| 5.35.0 | Support for old Services for index building and searching by index has been discontinued. | - |
| 5.34.0 | The 54, 56 and 57 neural network models for extracting face descriptors have been removed from the Handlers container. | Read the information described in the section "Prepare to change the neural network version", and perform certain actions before launching Handlers if one of the listed models was used. |
| The 104 and 106 neural network models for extracting body descriptors has been removed from the Handlers container. Now the 107 model is used by default. | Read the information described in the section "Prepare to change the neural network version", and perform certain actions before launching Handlers, because the default version has changed. | |
| Support for the Liveness V1 service has been discontinued. | - | |
| 5.30.0 | Vendor libraries for HASP key and HASP utility have been updated (from 111186 to 30147). | Update the HASP service and issue a new license (see "License key activation"). |
| The mechanism for creating and managing accounts has been changed. | Perform actions marked with the phrase Note. Accounts migration (upgrading from 5.2.0...5.28.0 only) | |
| 5.28.0 | The default value of the "score_threshold" setting in the "FACE_DETECTOR_V3" section of the Configurator service has been changed from 0.89 to 0.42. | Check the face recognition logic if the "score_threshold" value is used that is different from the default value. |
| 5.26.0 | The presence of a mask on the chin from this version refers to the "missing" state. In previous versions, it referred to the "medical_mask" state. | Check the logic of the mask estimation. |
| 5.24.0 | Support for the Vertica database for the Events service has been discontinued. | - |
| 5.23.0 | The default thresholds (recommended) have been updated for the following checks in the “face_quality” section and the “/iso” resource: "mouth_occluded" (old values: min=0, max=0.3; new values: min=0, max=0.5) and "mouth_open" (old values: min=0, max=0.64; new values: min=0, max=0.5) | Check the logic of "mouth_occluded" and "mouth_open" if the default values were used. |
| 5.14.0 | The Liveness V2 algorithm has been updated. The default thresholds for “liveness_threshold” and “quality_threshold” are now “0.5". The recommended threshold “liveness_threshold” is now “0.5” instead of “0.88". | Check the logic of the Liveness V2 algorithm. |
| 5.6.0 | Now, by default, the neural network model 59 for extracting face descriptors is used. | Read the information described in the section "Prepare to change the neural network version", and perform certain actions before launching Handlers, because the default version has changed. |
| The 101 model of the neural network for extracting body descriptors has been removed from the Handlers container. Now the 104 model is used by default. | Read the information described in the section "Prepare to change the neural network version", and perform certain actions before launching Handlers, because the default version has changed. | |
| 5.3.0 | The logic of launching LUNA PLATFORM services inside containers has been changed. Now applications are launched not from the "root" user, but from the "luna" user. | - |
| Support for FaceDetV1 and FaceDetV2 detectors has been discontinued. | - |
Create backups#
It is recommended to create the following backups:
- Backups of all databases used with LUNA PLATFORM.
- Backup of Image Store buckets.
- Backup of LUNA PLATFORM services configurations.
Creating backups will enable you to restore in case of any problems during the migration process.
Below are the backup commands for versions v.5.40.0 and higher. If you are upgrading to a lower version, then the path to the unpacked previous LP version must be added to the location of the LP data directories, for example, to create a backup copy of the PostgreSQL database for version v.5.38.1, run the following command:
cp -r /var/lib/luna/luna_v.5.38.1/example-docker/postgresql /var/lib/luna/BACKUP_postgresql
Backup of PostgreSQL database#
Create a backup of the PostgreSQL database using the following command:
cp -r /var/lib/luna/postgresql /var/lib/luna/BACKUP_postgresql
Backup of InfluxDB database#
Create a backup of the InfluxDB database using the following command:
cp -r /var/lib/luna/influx /var/lib/luna/BACKUP_influx
Backup of Image Store Buckets#
Create a backup of buckets using the following command:
cp -r /var/lib/luna/image_store /var/lib/luna/BACKUP_image_store
Dump file with service settings#
Custom values of settings for LUNA PLATFORM services (all except the Configurator service) are automatically migrated using the Configurator service migration mechanism described in the section "Configurator database migration".
If the migration of the service for some reason has lost the user configuration or the user just wants to store the old service settings for different LP versions, then you can create a dump file.
To create a dump file, use the following command (may be executed from anywhere on your server):
wget -O /var/lib/luna/BACKUP_settings_dump.json 127.0.0.1:5070/1/dump
or
curl 127.0.0.1:5070/1/dump > /var/lib/luna/BACKUP_settings_dump.json
This file will not be used during the normal installation of the LUNA PLATFORM. To apply the dumped settings use the db_create.py script with the --dump-file command line argument (followed with the created dump file name): base_scripts/db_create.py --dump-file settings_dump.json. You can apply full settings dump on an empty database only. See the detailed information in the "Settings dump" section of the administrator manual.
Prepare to change the neural network version#
In some LUNA PLATFORM builds, neural network models for extracting face and body descriptors are removed, and the default model usage settings are changed. See the section "Key changes from previous versions" for more information about these changes.
If you are upgrading from versions 5.2.0 and higher and in the previous build, one of the removed models was specified in the settings "DEFAULT_FACE_DESCRIPTOR_VERSION" or "DEFAULT_HUMAN_DESCRIPTOR_VERSION", then the launch of the Remote SDK service will fail if you do not perform the additional steps.
There are available models of neural networks for extracting descriptors in the current LUNA PLATFORM build:
| Object from which descriptor is extracted | Neural network models | Default model |
|---|---|---|
| Face | 59, 60, 62 | 59 |
| Body | 107, 110 | 110 |
It is necessary to perform one of the additional actions depending on the following scenarios of the work:
Continuation of the use of missing neural networks
Request to VisionLabs an old neural network model and prepare it for transfer to the new Remote SDK container after its launch (see instructions in the section "Use non-delivery neural network model" of administrator manual).
Switching to new version of the neural network with continuation of the use of old descriptors
- Run the "Additional extraction" task before the update (see "Additional extraction task" in the administrator manual). This will convert old descriptors to a new version of the neural network.
- Specify the new version of the neural network in the settings "DEFAULT_FACE_DESCRIPTOR_VERSION" or "DEFAULT_HUMAN_DESCRIPTOR_VERSION" before launching the Remote SDK service (see section "Change the neural network model for extracting descriptors").
Switching to new version of the neural network with cessation of the use of old descriptors
Specify the new version of the neural network in the settings "DEFAULT_FACE_DESCRIPTOR_VERSION" or "DEFAULT_HUMAN_DESCRIPTOR_VERSION" before launching the Remote SDK service (see section "Change the neural network model for extracting descriptors").
If it is not necessary to extract body descriptors, then you can disable the use of the neural network using the command
--env=EXTEND_CMD="--enable-body-descriptor-estimator=0"when launching Remote SDK container (see the detailed information in the section "Enable/disable several estimators and detectors" of the administrator manual).
Delete old symbolic link#
Delete the symbolic link to the previous minor version directory using the following command:
rm -f /var/lib/luna/current
Distribution unpacking#
The distribution package is an archive luna_v.5.54.0, where v.5.54.0 is a numerical identifier, describing the current LUNA PLATFORM version.
The archive includes configuration files, required for installation and exploitation. It does not include Docker images for the services. They should be downloaded from the Internet.
Move the distribution package to the directory on your server before the installation. For example, move the files to /root/ directory. The directory should not contain any other distribution or license files except the target ones.
Move the distribution to the created directory.
mv /root/luna_v.5.54.0.zip /var/lib/luna
Install the unzip archiver if it is necessary.
yum install -y unzip
Go to the folder with distribution.
cd /var/lib/luna
Unzip files.
unzip luna_v.5.54.0.zip
Symbolic link creation#
Create a symbolic link.
The link indicates that the current version of the distribution file is used to run LUNA PLATFORM.
ln -s luna_v.5.54.0 current
Changing group and owner for directories#
LP services are launched inside the containers by the "luna" user. Therefore, it is required to set permissions for this user to use the mounted volumes.
Go to the LP "example-docker" directory.
cd /var/lib/luna/current/example-docker/
Create a directory to store settings.
mkdir luna_configurator/used_dumps
Set permissions for the user with UID 1001 and group 0 to use the mounted directories.
chown -R 1001:0 luna_configurator/used_dumps
Move data (versions 5.38.3 and below)#
Note: Perform these actions only if you are upgrading from version 5.38.3 and below, because starting with LUNA PLATFORM v.5.40.0, the PostgreSQL, InfluxDB and Image Store container launch commands prescribe the paths of the directories for mounting located in the root directory /var/lib/luna/<db_or_bucket_folder>, unlike previous versions, where paths were prescribed for a specific version of the LUNA PLATFORM /var/lib/luna/current/example-docker/<db_or_bucket_folder>. Thus, there is no need to move data in new versions.
Move the data from your version to the directory /var/lib/luna/. Below are examples of commands for version 5.38.3.
Move Image Store buckets#
The following step is required if you are storing Image Store buckets in the default directory.
Copy the "image_store" folder with all its buckets.
mv /var/lib/luna/luna_v.5.38.3/example-docker/image_store /var/lib/luna/
Set rights for the luna user.
chown -R 1001:0 /var/lib/luna/image_store
Move PostgreSQL data#
The following step is required if you are using PostgreSQL in Docker container.
Copy the "data" folder.
mv /var/lib/luna/luna_v.5.38.3/example-docker/postgresql /var/lib/luna/
Move InfluxDB Data#
The following step is required if you are using InfluxDB in Docker container.
Copy the "influx" folder with all its buckets.
mv /var/lib/luna/luna_v.5.38.3/example-docker/influx /var/lib/luna/
Starting from version 5.9.0, InfluxDB OSS version 2 is used by default. This version enables you to visualize monitoring data using Grafana. If necessary, you can migrate data from InfluxDB version 1 to InfluxDB version 2. See Influx documentation for more details.
Save user configurations of the Configurator#
Note: Skip this step if the Configurator settings have not been changed.
The configurations of the Configurator service are not automatically migrated, unlike the configurations of all other services.
If your previous LP version was used with non-default Configurator service configurations, back up your "luna_configurator_postgres.conf" config file in the separate directory on your server.
cp /var/lib/luna/<your_previous_lp_version>/example-docker/luna_configurator/configs/luna_configurator_postgres.conf /var/lib/luna/BACKUP_luna_configurator_postgres.conf
This backup must be mounted to the Configurator service container that is being run.
If you are not sure if the Configurator service configurations have changed, you can compare the created backup with the Configurator configurations from the current distribution using the following command:
diff /var/lib/luna/current/example-docker/luna_configurator/configs/luna_configurator_postgres.conf /var/lib/luna/BACKUP_luna_configurator_postgres.conf
SELinux and Firewall#
You must configure SELinux and Firewall so that they do not block LUNA PLATFORM services.
SELinux and Firewall configurations are not described in this guide.
If SELinux and Firewall are not configured, the installation cannot be performed.
Create log directory for new services#
Skip this section if no logs were previously stored on the server.
In the new version of LUNA PLATFORM, new services could appear, for which you need to create directories with logs. It depends on the version from which you are upgrading. For example, version 5.30.0 introduced the Accounts service.
See "Logging to server" section if you have not previously used logging to a file, but want to enable it.
Following are the commands to create directories for all existing services. These commands will create and assign permissions only to missing directories.
mkdir -p /tmp/logs/configurator /tmp/logs/image-store /tmp/logs/accounts /tmp/logs/faces /tmp/logs/licenses /tmp/logs/events /tmp/logs/python-matcher /tmp/logs/handlers /tmp/logs/remote-sdk /tmp/logs/tasks /tmp/logs/tasks-worker /tmp/logs/sender /tmp/logs/api /tmp/logs/admin /tmp/logs/backport3 /tmp/logs/backport4
chown -R 1001:0 /tmp/logs/configurator /tmp/logs/image-store /tmp/logs/accounts /tmp/logs/faces /tmp/logs/licenses /tmp/logs/events /tmp/logs/python-matcher /tmp/logs/handlers /tmp/logs/remote-sdk /tmp/logs/tasks /tmp/logs/tasks-worker /tmp/logs/sender /tmp/logs/api /tmp/logs/admin /tmp/logs/backport3 /tmp/logs/backport4
If you need to use the Python Matcher Proxy service, then you need to additionally create the /tmp/logs/python-matcher-proxy directory and set its permissions.
License activation#
To activate/upgrade the license, follow these steps:
- Follow the steps from license activation manual.
- Set settings for HASP license or Guardant license before starting Licenses container.
Actions from License activation manual#
Open the license activation manual and follow the necessary steps.
Note: This action is mandatory. The license will not work without following the steps to activate the license from the corresponding manual.
Calculations using GPU#
You can use GPU for the general calculations performed by Remote SDK.
Skip this section if you are not going to utilize GPU for your calculations.
You need to install NVIDIA Container Toolkit to use GPU with Docker containers. The example of the installation is given below.
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.repo | tee /etc/yum.repos.d/nvidia-docker.repo
yum install -y nvidia-container-toolkit
systemctl restart docker
Check the NVIDIA Container toolkit operating by running a base CUDA container (this container is not provided in the LP distribution and should be downloaded from the Internet):
docker run --rm --gpus all nvidia/cuda:11.4.3-base-centos7 nvidia-smi
See the NVIDIA documentation for additional information.
Attributes extraction on the GPU is engineered for maximum throughput. The input images are processed in batches. This reduces computation cost per image but does not provide the shortest latency per image.
GPU acceleration is designed for high load applications where request counts per second consistently reach thousands. It won’t be beneficial to use GPU acceleration in non-extensively loaded scenarios where latency matters.
Login to registry#
When launching containers, you should specify a link to the image required for the container launching. This image will be downloaded from the VisionLabs registry. Before that, you should login to the registry.
Login and password can be requested from the VisionLabs representative.
Enter login
docker login dockerhub.visionlabs.ru --username <username>
After running the command, you will be prompted for a password. Enter password.
In the
docker logincommand, you can enter the login and password at the same time, but this does not guarantee security because the password can be seen in the command history.
Remove old containers#
Before launching the containers of the current minor version, stop all LUNA PLATFORM related containers of the previous minor version and third-party software containers.
It is also necessary to delete the PostgreSQL and InfluxDB containers in order to recreate them with new mounted directories (see "Move data").
To delete a container use the following command:
docker container rm -f [container_name]
where [container_name] is the service docker container name or ID.
For example, to remove LP containers only use the following command:
docker container rm -f luna-configurator luna-backport3 luna-backport4 luna-sender luna-tasks luna-handlers luna-remote-sdk luna-python-matcher luna-events luna-licenses luna-faces luna-image-store luna-ui-3 luna-ui-4 luna-admin luna-api luna-tasks-worker luna-accounts
To delete PostgreSQL and InfluxDB containers, use the following command:
docker container rm -f postgres influxdb
If you are upgrading from version 5.38.3 and below, you must delete the PostgreSQL and InfluxDB containers.
To see the containers names or IDs, use the following command:
docker ps -a
It is also recommended to delete old images of the containers to free space. You can use the following command to delete all unused images.
If there is enough space on the server it is recommended to perform this action only after new version of LP is successfully launched.
The command deletes all the unused images, not only the images related to LP.
docker image prune -a -f