Services launch#

This section gives examples for:

Databases tables migration
Buckets creation
Launching of containers

LUNA PLATFORM services must be launched in the following sequence:

Databases, Balancers, HASP service and other third-party party software
Configurator
Image Store
Accounts
Licenses
Faces
Events
Python Matcher
Python Matcher Proxy. The service is disabled by default.
Handlers
Tasks
Sender
API
Admin

Next, you need to launch the Backport 3 service and its user interface:

It is recommended to launch containers one by one and wait for the container status to become "up" (use the docker ps command).

Some of these services are optional and you can disable their use. It is recommended to use Events, Tasks, Sender and Admin services by default. See the "Optional services usage" section for details.

When launching each service, certain parameters are used, for example, --detach, --network, etc. See the section "Launching parameters description" for more detailed information about all launch parameters of LUNA PLATFORM services and databases.

See the "Docker commands" section for details about working with containers.

Monitoring configuration#

Monitoring LUNA PLATFORM services requires running the Influx 2.0.8-alpine database. Below are the commands to launch the Influx container.

For more information, see the "Monitoring" section in the administrator manual.

If necessary, you can configure the visualization of monitoring data using the LUNA Dashboards service, which includes a configured Grafana data visualization system. In addition, you can launch the Grafana Loki tool for advanced work with logs. See the instructions for launching LUNA Dashboards and Grafana Loki in the "Monitoring and logs visualization using Grafana" section.

InfluxDB OSS 2#

Note! If you already have InfluxDB 2.0.8-alpine installed, skip this step.

Use the docker run command with these parameters:

docker run \
-e DOCKER_INFLUXDB_INIT_MODE=setup \
-e DOCKER_INFLUXDB_INIT_BUCKET=luna_monitoring \
-e DOCKER_INFLUXDB_INIT_USERNAME=luna \
-e DOCKER_INFLUXDB_INIT_PASSWORD=password \
-e DOCKER_INFLUXDB_INIT_ORG=luna \
-e DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=kofqt4Pfqjn6o0RBtMDQqVoJLgHoxxDUmmhiAZ7JS6VmEnrqZXQhxDhad8AX9tmiJH6CjM7Y1U8p5eSEocGzIA== \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/luna/current/example-docker/influx:/var/lib/influxdb2 \
--restart=always \
--detach=true \
--network=host \
--name influxdb \
dockerhub.visionlabs.ru/luna/influxdb:2.0.8-alpine

If you need to set the custom settings of the Influx DB (for example, set the IP address and port when launching Influx on separate server), then you need to change them in the configurations of each LUNA PLATFORM service. See the section "Set custom Influx settings" for more information.

Run third-party services#

This section describes the launching of databases and message queues in docker containers. They must be launched before LP services.

Redis#

If you already have Redis installed, skip this step.

Use the following command to launch Redis.

docker run \
-v /etc/localtime:/etc/localtime:ro \
--name=redis \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/redis:7.0.5-alpine3.16

Configurator#

Optional services usage#

The listed below services are not mandatory for LP. You can disable them if their functionality is not required for your tasks.

Use the "ADDITIONAL_SERVICES_USAGE" section in the API service settings in the Configurator service to disable unnecessary services.

Optional services

Service	Dependent services
Events	API, Admin, Handlers, Tasks.
Tasks	API, Admin, Handlers
Sender	API, Admin, Handlers
Python Matcher Proxy (disabled by default)	API, Admin, Handlers, Tasks.

The API service does not depend on the state of the Admin, Backport3, Backport4, User Interface 3 and User Interface 4 services. These services do not need to be changed in the Configurator service, it is enough just not to launch them.

You can use the dump file provided in the distribution package to enable/disable services from the "Optional services" table before Configurator launch.

vi /var/lib/luna/current/extras/conf/platform_settings.json

You must restart a dependent service after changing its parameters if the service was already launched.

Configurator DB creation#

Create the database for the Configurator service. It is assumed that the DB user is already created.

Go to the directory.

cd /var/

Create the database
Grant privileges to the database user
Allow user to authorize in the DB

runuser -u postgres -- psql -c 'CREATE DATABASE luna_configurator;'

runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_configurator TO luna;'

runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'

Configurator DB tables creation#

Use the docker run command with these parameters to create the Configurator database tables.

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/luna/current/example-docker/luna_configurator/configs/luna_configurator_postgres.conf:/srv/luna_configurator/configs/config.conf \
-v /var/lib/luna/current/extras/conf/platform_settings.json:/srv/luna_configurator/used_dumps/platform_settings.json \
--network=host \
-v /tmp/logs/configurator:/srv/logs \
--rm \
--entrypoint bash \
dockerhub.visionlabs.ru/luna/luna-configurator:v.2.1.5 \
-c "python3 ./base_scripts/db_create.py; cd /srv/luna_configurator/configs/configs/; python3 -m configs.migrate --config /srv/luna_configurator/configs/config.conf head; cd /srv; python3 ./base_scripts/db_create.py --dump-file /srv/luna_configurator/used_dumps/platform_settings.json"

/var/lib/luna/current/extras/conf/platform_settings.json - enables you to specify the path to the dump file with LP configurations.

./base_scripts/db_create.py; - creates database structure.

python3 -m configs.migrate head; - performs settings migrations in Configurator DB and sets revision for migration. The revision will be required during the upgrade to the new LP5 build.

--dump-file /srv/luna_configurator/used_dumps/platform_settings.json - updates settings in the Configurator DB with values from the provided file.

Run Configurator container#

Use the docker run command with these parameters to launch Configurator:

docker run \
--env=PORT=5070 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/luna/current/example-docker/luna_configurator/configs/luna_configurator_postgres.conf:/srv/luna_configurator/configs/config.conf \
-v /tmp/logs/configurator:/srv/logs \
--name=luna-configurator \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-configurator:v.2.1.5

At this stage, you can activate logging to file if you need to save them on the server (see the "Logging to server" section).

Changing the neural network version#

For successful migration, it is necessary to change the version of the neural network to extract body descriptors in the "DEFAULT_HUMAN_DESCRIPTOR_VERSION" setting of the Handlers service.

Follow these steps:

open the Configurator user interface http://<configurator_server_ip>:5070;
enter the name of the setting "DEFAULT_HUMAN_DESCRIPTOR_VERSION" in the "Setting name" field and click "Apply";
set the neural network model "107" in the "DEFAULT_HUMAN_DESCRIPTOR_VERSION" setting;
save the changes by clicking the "Save" button.

Migration from LUNA PLATFORM 3 to Backport 3#

This section describes accounts, descriptors, and persons migration from LUNA PLATFORM 3 databases to LUNA PLATFORM 5 databases.

Edit configuration file#

You need to set up the following configuration file before starting the migration:

vi /var/lib/luna/current/extras/conf/migration_config.conf

Enter the following information:

API, Faces, Broker services general parameters (LUNA PLATFORM 3);
Backport 3 and Faces services general parameters (LUNA PLATFORM 5);
Luna Image Store general parameters;
Logging parameters.

Before migration#

The server where the migrations script is launched should have a connection to all the specified databases and the Broker service.
Make sure that the Broker service (LUNA PLATFORM 3) is launched. It is used to get descriptors from the database.

It is not necessary to launch the LUNA PLATFORM 5 services to perform migrations.

The databases with all the required tables should be created for Backport3 and Faces services (LUNA PLATFORM 5) before running migrations. See the installation manuals of corresponding services.

The Faces (LUNA PLATFORM 3) and Faces (LUNA PLATFORM 5) database names are similar by default (luna_faces). You should fix it using one of the following ways:

You can use another PostgreSQL for creating a new database for the Faces service (LUNA PLATFORM 5) hence the "luna_faces" name will not be changed. In this case, you should specify the new address of the PostgreSQL database in the "LUNA_FACES_DB" section of the Faces configuration file/Configurator.
You can change the default name of the Faces DB (LUNA PLATFORM 5) in the "LUNA_FACES_DB" section of the Faces configuration file/Configurator and database creation commands. For example, set the "luna_faces_5" name.

The second way is described in this manual. The "luna_faces_5" name is set for the base by default. See the "Faces DB creation" section.

Note! Make sure that the databases for the Backport 3 service (LUNA PLATFORM 5) and the Faces service (LUNA PLATFORM 5) are empty (there are no entries) before starting the migration.

Note! Do not launch the creation of database tables for the Faces service before changing the database name/PostgreSQL address in the "LUNA_FACES_DB" section of the Faces service configuration file/Configurator. Otherwise, you can lose the data stored in the Faces database of LUNA PLATFORM 3.

Faces DB creation#

Create the database for the Faces service. It is assumed that the DB user is already created.

Go to the directory.

cd /var/

Create a database
Grant privileges to the database and the user
Enable the user to login into DB

runuser -u postgres -- psql -c 'CREATE DATABASE luna_faces;'

runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_faces TO luna;'

runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'

Change the utilized DB#

Now you should specify the "luna_faces_5" DB name in the settings of the Faces service.

Go to the Configurator service user interface (http://<server_name>:5070).
Select "luna-faces" in the "Service name" filter.
Find the "LUNA_FACES_DB" group of settings.
Set the "db_name": "luna_faces_5" parameter.
Press the [Save] button.

Faces DB tables creation#

Use the following command to create the Faces DB tables:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/faces:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-faces:v.4.8.2 \
python3 ./base_scripts/db_create.py --luna-config http://localhost:5070/1

Backport 3 DB creation#

Create the database for the Backport 3 service. It is assumed that the DB user is already created.

Go to the directory.

cd /var/

The sequence of actions corresponds to the commands below:

Create a database
Grant privileges to the database and the user
Enable the user to login into DB

runuser -u postgres -- psql -c 'CREATE DATABASE luna_backport3;'

runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_backport3 TO luna;'

runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'

Backport 3 DB tables creation#

Use the following command to create DB tables for Backport 3:

docker run \
-v /etc/localtime:/etc/localtime:ro  \
-v /tmp/logs/backport3:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-backport3:v.0.7.2 \
python3 ./base_scripts/db_create.py --luna-config http://localhost:5070/1

Accounts DB creation#

Create the database for the Accounts service. It is assumed that the DB user is already created.

Go to the directory.

cd /var/

Create the database
Grant privileges to the database user
Allow user to authorize in the DB

runuser -u postgres -- psql -c 'CREATE DATABASE luna_accounts;'

runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_accounts TO luna;'

runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'

Accounts DB tables creation#

Use the following command to create Accounts DB tables:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/accounts:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-accounts:v.0.1.2 \
python3 ./base_scripts/db_create.py --luna-config http://localhost:5070/1

Migration launch#

All the migration procedures are performed using the "start_migration.py" script. See the "Migration script description" section below for additional information about the script.

Add the "luna" owner for the "/var/lib/luna/luna_v.3.3.8/luna-image-store/luna_image_store/local_storage".

chown -R 1001:0 /var/lib/luna/luna_v.3.3.8/luna-image-store/luna_image_store/local_storage

Run the script using the Backport 3 container:

docker run \
--rm -t \
-v /tmp/logs/backport3:/srv/logs \
-v /var/lib/luna/current/extras/conf/migration_config.conf:/srv/base_scripts/migrate_backport3/config/config.conf \
-v /var/lib/luna/luna_v.3.3.8/luna-image-store/luna_image_store/local_storage:/local_storage \
--network=host \
--entrypoint bash dockerhub.visionlabs.ru/luna/luna-backport3:v.0.7.2 -c "cd ./base_scripts/migrate_backport3 && pip3 install -r requirements.txt && python3 ./start_migration.py"

You can optionally use the --skip_missing_descriptors parameter, which will enable you to ignore missing descriptors in the LP 3 database.

-v /tmp/logs/backport3:/srv/logs - the script is saving logs to the /srv/logs directory by default. Then the logs are moved to the /tmp/logs/backport3 directory on your server.

The configuration file /var/lib/luna/current/extras/conf/migration_config.conf is added to the container for the script launching.

-v /var/lib/luna/luna_v.3.3.8/luna-image-store/luna_image_store/local_storage:/local_storage - the local storage with Image Store samples is added to the container. It is required for the samples migration.

cd ./base_scripts/migrate_backport3 && pip3 install -r requirements.txt && python3 ./start_migration.py - the "requirements.txt" file is required to run the "start_migration.py" script with a set of specified dependencies.

Migration script description#

Data transfer will be performed in the following order:

database migration from API (LUNA PLATFORM 3) service to Backport 3 (LUNA PLATFORM 5)
database migration from Faces (LUNA PLATFORM 3) to Backport 3 (LUNA PLATFORM 5)
database migration from Faces (LUNA PLATFORM 3) to Faces (LUNA PLATFORM 5)
updating descriptors in the Faces database (LUNA PLATFORM 5) from the CORE (LUNA PLATFORM 3) database
add accounts to samples and optionally portraits in the Luna Image Store storage

When migrating API and Faces service databases, all LP 3 accounts will be migrated and stored in the database of the Accounts service. All migrated accounts will be of type "user". The fields "password", "email" and "organization_name" will be transferred to the "account" table of the Accounts database under new names - "password", "login" and "description" respectively. Tokens will stored in the Backport3 database, but their identifier will also be entered in the Accounts database, where the necessary permissions will be automatically set.

When the script is launched, the "luna-backport3_ERROR_migration.txt" and "luna-backport3_WARNING_migration.txt" files are created. The files include information about all the errors and warnings that occurred during the migration process.

The files are saved to "/srv/logs/" directory of the container.

Run the following script to get help:

docker run --rm -t \
--network=host \
--entrypoint bash dockerhub.visionlabs.ru/luna/luna-backport3:v.0.7.2 -c "cd ./base_scripts/migrate_backport3 && pip3 install -r requirements.txt && python3 ./start_migration.py --help"

To start individual migration steps, pass the --migrate command line argument.

The argument takes the following parameters:

stage_1 - migrate from API (LUNA PLATFORM 3) and Faces (LUNA PLATFORM 3) databases to Backport 3 (LUNA PLATFORM 5) database
stage_2 - migrate from Faces (LUNA PLATFORM 3) database to Faces (LUNA PLATFORM 5) database
stage_3 - update descriptors in the Faces (LUNA PLATFORM 5) from the CORE (LUNA PLATFORM 3) database
stage_4 - add accounts for samples and optionally portraits (--migrate_portraits flag) in the Luna-Image-Store storage
all - perform all the above steps

Code example:

docker run \
--rm -t \
-v /tmp/logs/backport3:/srv/logs \
-v /var/lib/luna/current/extras/conf/migration_config.conf:/srv/base_scripts/migrate_backport3/config/config.conf \
-v /var/lib/luna/luna_v.3.3.8/luna-image-store/luna_image_store/local_storage:/local_storage \
--network=host \
--entrypoint bash dockerhub.visionlabs.ru/luna/luna-backport3:v.0.7.2 -c "cd ./base_scripts/migrate_backport3 && pip3 install -r requirements.txt && python3 ./start_migration.py --migrate stage_1"

If you are migrating from stage_3, check that the face and attribute tables in the Faces (LUNA PLATFORM 5) database have entries.

If something went wrong during stage_3, use the argument --lower_boundary to specify the last failed face ID to continue migration. The face ID can be found in migration logs.

For example:

docker run --rm -t -v \
/tmp/logs/backport3:/srv/logs \
-v /var/lib/luna/current/extras/conf/migration_config.conf:/srv/base_scripts/migrate_backport3/config/config.conf \
--network=host \
--entrypoint bash dockerhub.visionlabs.ru/luna/luna-backport3:v.0.7.2 -c "cd ./base_scripts/migrate_backport3 && pip3 install -r requirements.txt && python3 ./start_migration.py --lower_boundary 02e7b0db-b3c3-4446-bbdd-0f0d9a566058"

Stop LUNA PLATFORM 3 services#

Stop and disable all the LUNA PLATFORM 3 services

systemctl stop luna-image-store luna-faces luna-broker luna-extractor@1 luna-matcher@1 luna-stat-lpse.service luna-stat-sm.service luna-api luna-admin_back luna-admin_tasks aerospike

systemctl disable luna-image-store luna-faces luna-broker luna-extractor@1 luna-matcher@1 luna-stat-lpse.service luna-stat-sm.service luna-api luna-admin_back luna-admin_tasks aerospike

systemctl status luna-image-store luna-faces luna-broker luna-extractor@1 luna-matcher@1 luna-stat-lpse.service luna-stat-sm.service luna-api luna-admin_back luna-admin_tasks aerospike

Image Store#

Samples migrations#

Samples migration is required to add an account for each sample.

Create a backup of all the samples buckets before launching the following script.

Implied that Luna-Image-Store storage from LUNA PLATFORM 3 will be used with LUNA PLATFORM 5. Storage transfer not provided during the migration.

You should use existing buckets during the LUNA PLATFORM 5 Image Store launching.

Change the default bucket used for the samples storage to "visionlabs-warps".

Go to the Configurator service user interface (http://<server_name>:5070).
Enter "LUNA_IMAGE_STORE_FACES_SAMPLES_ADDRESS" in the "Setting name" filter.
Specify the "bucket" name:

"bucket": "visionlabs-warps"

By default, the samples bucket in LUNA PLATFORM 3 was called "visionlabs-warps".

Press the [Save] button.

You should specify the current directory with buckets ("/var/lib/luna/luna_v.3.3.8/luna-image-store/luna_image_store/local_storage/" by default) instead of "/var/lib/luna/current/example-docker/image_store/" during the Image Store container launching: -v /var/lib/luna/current/example-docker/image_store/:/srv/local_storage/ See section Image Store container launch.

Portraits migration#

Backport uses samples as portraits by default. Thus it is not required to store portraits and samples simultaneously.

If samples will not be used as portraits, its required to migrate portraits (run migration with --migrate_portraits flag).

If portraits are required, you should turn off the USE_SAMPLES_AS_PORTRAITS setting of Backport 3.

You should follow one of these steps if you are going to use portraits:

If you store portraits in the IMAGE STORE service (the SEND_TO_LUNA_IMAGE_STORE setting of the API service in LUNA PLATFORM 3 is enabled), you should set up the LUNA_IMAGE_STORE_PORTRAITS_ADDRESS setting. To do this, use the values of LUNA_IMAGE_STORE_ORIGIN and LUNA_IMAGE_STORE_BUCKET settings that can be found in the configuration file of the API service (LUNA PLATFORM 3).
If you use a plugin to store portraits, you should move the existing portraits to the Image Store service of LUNA PLATFORM 5. See the "./docs/ReferenceManuals/ImageStoreReferenceManual.html" document in the LUNA PLATFORM 5 distribution package for details about creating a bucket and saving images to the service.

You can configure all the listed settings in the Configurator service of LUNA PLATFORM 5 or configuration files of the corresponding services (if the Configurator service is not utilized).

Image Store container launch#

Use the following command to launch the Image Store service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5020 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /var/lib/luna/current/example-docker/image_store/:/srv/local_storage/ \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/image-store:/srv/logs \
--name=luna-image-store \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-image-store:v.3.8.2

-v /var/lib/luna/current/example-docker/image_store/:/srv/local_storage/ - the data from the specified folder is added to the Docker container when it is launched. All the data from the specified Docker container folder is saved to this directory.

If you already have a directory with LP buckets you should specify it instead of /var/lib/luna/current/example-docker/image_store/.

Buckets creation#

Buckets are required to store data in Image Store. The Image Store service should be launched before the commands execution.

When upgrading from the previous version, it is recommended to launch the bucket creation commands one more time. Hence you make sure that all the required buckets were created.

If the "error_code":13006,"desc":"Unique constraint error","detail":"Bucket with name 'task-result' already exist"} error appears during launching of the listed above commands, the bucket is already created.

There are two ways to create buckets in LP.

Run the listed below scripts to create buckets

Run this script to create general buckets:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/api:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-api:v.6.13.2 \
python3 ./base_scripts/lis_bucket_create.py -ii --luna-config http://localhost:5070/1

If you are going to use the Tasks service, use the following command to additionally create the "task-result" in the Image Store service:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/tasks:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-tasks:v.3.14.2 \
python3 ./base_scripts/lis_bucket_create.py -ii --luna-config http://localhost:5070/1

If you are going to use the portraits, use the following command to additionally create the "portraits".

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/api:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-backport3:v.0.7.2 \
python3 ./base_scripts/lis_bucket_create.py -ii --luna-config http://localhost:5070/1

Use direct requests to create required buckets.

The curl utility is required for the following requests.

The "visionlabs-samples" bucket is used for face samples storage. The bucket is required for LP utilization.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=visionlabs-samples

The "portraits" bucket is used for portraits storage. The bucket is required for Backport 3 utilization.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=portraits

The "visionlabs-bodies-samples" bucket is used for human bodies samples storage. The bucket is required for LP utilization.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=visionlabs-bodies-samples

The "visionlabs-image-origin" bucket is used for source images storage. The bucket is required for LP utilization.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=visionlabs-image-origin

The "visionlabs-objects" bucket is used for objects storage. The bucket is required for LP utilization.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=visionlabs-objects

The "task-result" bucket for the Tasks service. Do not use it if you are not going to use the Tasks service.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=task-result

Accounts#

Accounts container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5170 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/accounts:/srv/logs \
--name=luna-accounts \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-accounts:v.0.1.2

Licenses#

Specify license server address using Configurator#

To set the license server address, follow these steps:

go to the Configurator service interface http://<configurator_server_ip>:5070/
specify the "LICENSE_VENDOR" value in the "Setting name" field and click "Apply Filters"
set the IP address of the server with your HASP key in the field "server_address"
click "Save"

Licenses container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5120 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/licenses:/srv/logs \
--name=luna-licenses \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-licenses:v.0.6.3

Faces#

Create VLMatch function for Faces DB#

The Faces service requires the VLMatch additional function to be added to the utilized database. LUNA PLATFORM cannot perform matching calculations without this function. The VLMatch function can be added to the PostgreSQL or Oracle database.

The VLMatch library is compiled for your particular database version.

Note! Do not use the library built for another version of DB. For example, the library build for the PostgreSQL of version 12 cannot be used for the PostgreSQL of version 9.6.

This section describes the function creation for PostgreSQL.

The instruction for the Oracle DB is given in the "VLMatch for Oracle" section.

Build VLMatch for PostgreSQL#

You can find all the required files for the VLMatch user-defined extension (UDx) compilation in the following directory:

/var/lib/luna/current/extras/VLMatch/postgres/

The following instruction describes installation for PostgreSQL 12.

For VLMatch UDx function compilation one needs to:

Make sure, that PostgreSQL of the required version is installed and launched.
Install the required PostgreSQL development environment. You can find more information on the official web site.
The llvm-toolset-7-clang is required for postgresql12-devel. Install it from the centos-release-scl-rh repository.

yum -y install centos-release-scl-rh

yum -y --enablerepo=centos-sclo-rh-testing install llvm-toolset-7-clang

Install epel-release for access to extended package repository

yum -y install epel-release

Install the development environment.

yum -y install postgresql12 postgresql12-server postgresql12-devel

Install the gcc-c++ package. The package version 4.8 or higher is required.

yum -y install gcc-c++.x86_64

Install CMAKE. The version 3.5 or higher is required.
Open the make.sh script using a text editor. It includes paths to the currently used PostgreSQL version. Change the following values (if necessary):

SDK_HOME specifies the path to PostgreSQL home directory. The default value is /usr/pgsql-12/include/server;

LIB_ROOT specifies the path to PostgreSQL library root directory. The default value is /usr/pgsql-12/lib.

Go to the make.sh script directory and run it:

cd /var/lib/luna/current/extras/VLMatch/postgres/

chmod +x make.sh

./make.sh

Add VLMatch function to Faces database#

The VLMatch function should be applied to the PostgreSQL DB.

Define the function inside the Faces database:

sudo -u postgres -h 127.0.0.1 -- psql -d luna_faces -c "CREATE FUNCTION VLMatch(bytea, bytea, int) RETURNS float8 AS 'VLMatchSource.so', 'VLMatch' LANGUAGE C PARALLEL SAFE;"

Test function by sending re following request to the service database:

sudo -u postgres -h 127.0.0.1 -- psql -d luna_faces -c "SELECT VLMatch('\x1234567890123456789012345678901234567890123456789012345678901234'::bytea, '\x0123456789012345678901234567890123456789012345678901234567890123'::bytea, 32);"

The result returned by the database must be "0.4765625".

Faces container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5030 \
--env=WORKER_COUNT=2 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/faces:/srv/logs \
--name=luna-faces \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-faces:v.4.8.2

Events#

Events DB creation#

Create the database for the Events service. It is assumed that the DB user is already created.

Go to the directory.

cd /var/

Create database
Grant privileges to database and the user
Enable the user to login into DB

runuser -u postgres -- psql -c 'CREATE DATABASE luna_events;'

runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_events TO luna;'

runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'

Create VLMatch function for Events DB#

The Events service requires the VLMatch additional function to be added to the utilized database. LUNA PLATFORM cannot perform matching calculations without this function. The VLMatch function can be added to the PostgreSQL.

The VLMatch library is compiled for your particular database version.

Note! Do not use the library built for another version of DB. For example, the library build for the PostgreSQL of version 12 cannot be used for the PostgreSQL of version 9.6.

This section describes the function creation for PostgreSQL. If you use the PostgreSQL database, you have already created and moved the created library during the Faces service launch. See section "Build VLMatch for PostgreSQL".

Add VLMatch function to Events database#

The VLMatch function should be applied to the PostgreSQL DB.

Define the function inside the Events database:

sudo -u postgres -h 127.0.0.1 -- psql -d luna_events -c "CREATE FUNCTION VLMatch(bytea, bytea, int) RETURNS float8 AS 'VLMatchSource.so', 'VLMatch' LANGUAGE C PARALLEL SAFE;"

Test function within call:

sudo -u postgres -h 127.0.0.1 -- psql -d luna_events -c "SELECT VLMatch('\x1234567890123456789012345678901234567890123456789012345678901234'::bytea, '\x0123456789012345678901234567890123456789012345678901234567890123'::bytea, 32);"

The result returned by the database must be "0.4765625".

PostGIS installation for Events#

The Events service requires PostGIS for working with coordinates.

This instruction describes the PostGIS installation for PostgreSQL 12 database. The PostGIS version depends on the PostgreSQL version.

Install epel-release for access to extended package repository (necessary for RabbitMQ and other dependencies)

yum -y install epel-release

Install PostGIS:

yum -y install postgis25_12

Activate PostGIS in your database:

sudo -u postgres -h 127.0.0.1 -- psql -d luna_events -c "CREATE EXTENSION postgis;";

See additional information about PostGIS on its website:

https://postgis.net/

Events DB tables creation#

Note. If you are not going to use the Events service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".

Use the following command to create the Events DB tables:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/events:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-events:v.4.8.2 \
python3 ./base_scripts/db_create.py --luna-config http://localhost:5070/1

Events container launch#

Note. If you are not going to use the Events service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5040 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/events:/srv/logs \
--name=luna-events \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-events:v.4.8.2

Python Matcher services#

For matching tasks, you can use either only the Python Matcher service, or additionally use the Python Matcher Proxy service, which redirects matching requests to either the Python Matcher service or matching plugins. This section describes how to use Python Matcher without Python Matcher Proxy.

Note. You need to use the Python Matcher Proxy service only if you are going to use matching plugins. Using Python Matcher Proxy and running the corresponding docker container are described in the "Use Python Matcher with Python Matcher Proxy" section.

See the description and usage of matching plugins in the administrator manual.

Use Python Matcher without Python Matcher Proxy#

The Python Matcher service with matching by the Faces DB is enabled by default after launching.

The Python Matcher service with matching by the Events is also enabled by default. You can disable it by specifying USE_LUNA_EVENTS = 0 in the ADDITIONAL_SERVICES_USAGE settings of Configurator (see "Optional services usage" section). Thus, the Events service will not be used for LUNA PLATFORM.

The Python Matcher that matches using the matcher library is enabled when CACHE_ENABLED is set to "true" in the DESCRIPTORS_CACHE setting.

Note. A single image is downloaded for the Python Matcher service and the Python Matcher Proxy service.

Python Matcher container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5100 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/python-matcher:/srv/logs \
--name=luna-python-matcher \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-python-matcher:v.1.4.2

Handlers#

Handlers DB creation#

Create the database for the Handlers service. It is assumed that the DB user is already created.

Go to the directory.

cd /var/

The sequence of actions corresponds to the commands below:

Create a database
Grant privileges to the database and the user
Enable the user to login into DB

runuser -u postgres -- psql -c 'CREATE DATABASE luna_handlers;'

runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_handlers TO luna;'

runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'

Handlers DB tables creation#

Use the following command to create the Handlers DB tables:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/handlers:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-handlers:v.2.16.2 \
python3 ./base_scripts/db_create.py --luna-config http://localhost:5070/1

Handlers container launch#

You can run the Handlers service utilizing CPU (set by default) or GPU.

By default, the Handlers service is launched with all estimators and detectors enabled. If necessary, you can disable the use of some estimators or detectors when launching the Handlers container. Disabling unnecessary estimators enables you to save RAM or GPU memory, since when the Handlers service launches, the possibility of performing these estimates is checked and neural networks are loaded into memory. If you disable the estimator or detector, you can also remove its neural network from the Handlers container. See the "Enable/disable several estimators and detectors" section of the administrator manual for more information.

Run the Handlers service using one of the following commands according to the utilized processing unit.

Run Handlers utilizing CPU#

Use the following command to launch the Handlers service using CPU:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5090 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/handlers:/srv/logs \
--name=luna-handlers \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-handlers:v.2.16.2

Run Handlers utilizing GPU#

The Handlers service does not utilize GPU by default. If you are going to use the GPU, then you should enable its use for the Handlers service in the Configurator service.

If you need to use the GPU for all estimators and detectors at once, then you need to use the global_device_class parameter in the "LUNA_HANDLERS_RUNTIME_SETTINGS" section. All estimators and detectors will use the value of this parameter if the device_class parameter of their settings like "LUNA_HANDLERS__SETTINGS.runtime_settings" is set to global (by default for all estimators and detectors).

If you need to use the GPU for a specific estimator or detector, then you need to use the device_class parameter in sections like "LUNA_HANDLERS__SETTINGS.runtime_settings".

See section "Calculations using GPU" for additional requirements for GPU utilization.

Use the following command to launch the Handlers service using GPU:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5090 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--gpus device=0 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/handlers:/srv/logs \
--name=luna-handlers \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-handlers:v.2.16.2

--gpus device=0 - the parameter specifies the used GPU device and enables GPU utilization. A single GPU can be utilized per Handlers instance. Multiple GPU utilization per instance is not available.

Tasks#

Note. If you are not going to use the Tasks service, do not launch the Tasks container and the Tasks Worker container. Disable the service utilization in Configurator. See section "Optional services usage".

Tasks DB creation#

Create the database for the Tasks service. It is assumed that the DB user is already created.

Go to the directory.

cd /var/

Create the database
Grant privileges to the database user
Allow user to authorize in the DB

runuser -u postgres -- psql -c 'CREATE DATABASE luna_tasks;'

runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_tasks TO luna;'

runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'

Tasks DB tables creation#

Use the following command to create Tasks DB tables:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/tasks:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-tasks:v.3.14.2 \
python3 ./base_scripts/db_create.py --luna-config http://localhost:5070/1

Tasks and Tasks worker containers launch#

Luna Tasks Docker image includes the Tasks service and the Tasks worker. They both must be launched.

The "task-result" bucket should be created for the Tasks service before the service launch. The buckets creation is described in the "Buckets creation".

If it is necessary to use the Estimator task using a network disk, then you should first mount the directory with images from the network disk into special directories of Tasks and Tasks Worker containers. See the "Estimator task" section in the administrator manual for details.

Tasks Worker launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5051 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--env=SERVICE_TYPE="tasks_worker" \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/tasks-worker:/srv/logs \
--name=luna-tasks-worker \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-tasks:v.3.14.2

Tasks launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5050 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/tasks:/srv/logs \
--name=luna-tasks \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-tasks:v.3.14.2

Sender#

Sender container launch#

Note. If you are not going to use the Sender service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5080 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/sender:/srv/logs \
--name=luna-sender \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-sender:v.2.6.2

API#

API container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5000 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--name=luna-api \
--restart=always \
--detach=true \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/api:/srv/logs \
--network=host \
dockerhub.visionlabs.ru/luna/luna-api:v.6.13.2

Admin#

Admin container launch#

Note. If you are not going to use the Admin service, do not launch this container.

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5010 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/admin:/srv/logs \
--name=luna-admin \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-admin:v.5.1.3

Monitoring data about the number of executed requests is saved in the luna-admin bucket of the InfluxDB. To enable data saving use the following command:

docker exec -it luna-admin python3 ./base_scripts/influx2_cli.py create_usage_task --luna-config http://127.0.0.1:5070/1

Backport 3#

The section describes launching of Backport 3 service.

Note. The service is not mandatory for utilizing LP5 and is required for emulation of LP 3 API only.

Backport 3 container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5140 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--name=luna-backport3 \
--restart=always \
--detach=true \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/backport3:/srv/logs \
--network=host \
dockerhub.visionlabs.ru/luna/luna-backport3:v.0.7.2

User Interface 3#

The User Interface 3 is used with the Backport 3 service only.

User Interface 3 container launch#

Use the following command to launch the service:

docker run \
--env=PORT=4100 \
--env=LUNA_API_URL=http://127.0.0.1:5140 \
--name=luna-ui-3 \
--restart=always \
--detach=true \
--network=host \
-v /etc/localtime:/etc/localtime:ro \
dockerhub.visionlabs.ru/luna/luna3-ui:v.0.5.8

--env=LUNA_API_URL - specifies the URL of the Backport 3 service.

--env=PORT - specifies the port of the User Interface 3 service.