Services launch#

This section gives examples for:

Databases tables migration.
Buckets creation.
Launching of containers.

LUNA PLATFORM services must be launched in the following sequence:

Databases, Balancers, HASP service and other third-party party software.
Configurator
Image Store
Accounts
Licenses
Faces
Events
Python Matcher
Python Matcher Proxy. The service is disabled by default
Remote SDK
Handlers
Tasks
Sender
API
Admin

The Lambda service (disabled by default) can be launched after Licenses and Configurator services.

The following service are used when emulation of LUNA PLATFORM 3 is required only:

The following service are used when emulation of LUNA PLATFORM 4 is required only:

It is recommended to launch containers one by one and wait for the container status to become "up" (use the docker ps command).

Some of these services are optional and you can disable their use. It is recommended to use Events, Tasks, Sender and Admin services by default. See the "Optional services usage" section for details.

When launching each service, certain parameters are used, for example, --detach, --network, etc. See the section "Launching parameters description" for more detailed information about all launch parameters of LUNA PLATFORM services and databases.

See the "Docker commands" section for details about working with containers.

Third-party services#

This section describes the launching of databases in docker containers. They must be launched before LP services.

InfluxDB#

Note: If you haven't deleted the old container, skip this step.

Monitoring LUNA PLATFORM services requires running the Influx 2.0.8-alpine database. Below are the commands to launch the InfluxDB container.

For more information, see the "Monitoring" section in the administrator manual.

If necessary, you can configure the visualization of monitoring data using the LUNA Dashboards service, which includes a configured Grafana data visualization system. In addition, you can launch the Grafana Loki tool for advanced work with logs. See the instructions for launching LUNA Dashboards and Grafana Loki in the "Monitoring and logs visualization using Grafana" section.

Note: If necessary, you can use an external InfluxDB 2.0.8-alpine. In this case, you can skip the command below, but you will have to set custom settings for each LUNA PLATFORM service.

Use the docker run command with these parameters:

docker run \
-e DOCKER_INFLUXDB_INIT_MODE=setup \
-e DOCKER_INFLUXDB_INIT_BUCKET=luna_monitoring \
-e DOCKER_INFLUXDB_INIT_USERNAME=luna \
-e DOCKER_INFLUXDB_INIT_PASSWORD=password \
-e DOCKER_INFLUXDB_INIT_ORG=luna \
-e DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=kofqt4Pfqjn6o0RBtMDQqVoJLgHoxxDUmmhiAZ7JS6VmEnrqZXQhxDhad8AX9tmiJH6CjM7Y1U8p5eSEocGzIA== \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/luna/influx:/var/lib/influxdb2 \
--restart=always \
--detach=true \
--network=host \
--name influxdb \
dockerhub.visionlabs.ru/luna/influxdb:2.0.8-alpine

PostgreSQL#

Migrate PostgreSQL 12 to PostgreSQL 16#

In LUNA PLATFORM v.5.53.0, the VisionLabs image for PostgreSQL has been updated from version 12 to version 16.

If this image was previously used, then you need to perform the migration yourself according to official documentation. If necessary, you can continue using PostgreSQL 12 by specifying the image "postgis-vlmatch:12" in the container launch command.

Mounting PostgreSQL 12 data from the directory "/var/lib/luna/postgres" into a container for PostgreSQL 16 will result in an error.

Launch PostgreSQL#

Note: If you haven't deleted the old container, skip this step.

Use the following command to launch PostgreSQL.

docker run \
--env=POSTGRES_USER=luna \
--env=POSTGRES_PASSWORD=luna \
--shm-size=1g \
-v /var/lib/luna/postgresql/data/:/var/lib/postgresql/data/ \
-v /var/lib/luna/current/example-docker/postgresql/entrypoint-initdb.d/:/docker-entrypoint-initdb.d/ \
-v /etc/localtime:/etc/localtime:ro \
--name=postgres \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/postgis-vlmatch:16

-v /var/lib/luna/current/example-docker/postgresql/entrypoint-initdb.d/:/docker-entrypoint-initdb.d/ \ — The "docker-entrypoint-initdb.d" script includes the commands for the creation of services databases. During database creation, a default username and password are automatically used.

-v /var/lib/luna/current/example-docker/postgresql/data/:/var/lib/postgresql/data/ — The volume command enables you to mount the "data" folder to the PostgreSQL container. The folder on the server and the folder in the container will be synchronized. The PostgreSQL data from the container will be saved to this directory.

--network=host — If you need to change the port for PotgreSQL, you should change this string to -p 5440:5432. Where the first port 5440 is the local port and 5432 is the port used inside the container.

You should create all the databases for LP services manually if you are going to use an already installed PostgreSQL.

Redis#

Note: If you haven't deleted the old container, skip this step.

Use the following command to launch Redis.

docker run \
-v /etc/localtime:/etc/localtime:ro \
--name=redis \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/redis:7.2

Configurator#

Optional services usage#

The listed below services are not mandatory for LP:

Events
Image Store
Tasks
Sender
Handlers
Python Matcher Proxy (disabled by default)
Lambda (disabled by default)

You can disable them if their functionality is not required for your tasks.

Use the "ADDITIONAL_SERVICES_USAGE" section in the API service settings in the Configurator service to disable unnecessary services.

You can use the dump file provided in the distribution package to enable/disable services before Configurator launch.

vi /var/lib/luna/current/extras/conf/platform_settings.json

Disabling any of the services has certain consequences. For more information, see the "Disableable services" section of the administrator manual.

Configurator database migration#

Migration from the previous LP build to LP build v.5.59.0 is described in this section.

The following instruction for Configurator DB migration assumes that you already have settings migration revision set in your database. This revision is set using the configs.migrate head script. This script is included in the installation manuals of LP, starting with build 5.1.1. If you have performed installation according to the manual, you do not need to perform additional actions.Settings migration will be performed automatically.

If there is no revision, you should recreate the database structure. See the LP 5 installation manual section "Configurator DB tables creation" for instructions. Then you should specify all the required settings manually.

The migration from version 5.1.0 is described in "LP_Docker_Upgrade_Manual.html" in the distribution package of version 5.1.1. You can update your Configurator database using this instruction and distribution package of version 5.1.1. Otherwise, you should recreate the database structure for the Configurator service and specify all the required settings manually. See the LP 5 installation manual section "Configurator DB tables creation" for instructions.

Migration from pre-release builds (builds before v.5.1.0) of LP is not provided. You should transfer all the required settings to the Configurator DB of 5.1.1 build manually.

When upgrading the Configurator service database with existing service configurations, a database structure migration must be performed, which will also migrate the saved settings of all LP services except the Configurator service configurations.

Note on migrating Configurator service settings is described in the next section.

Use the docker run command with these parameters to create the Configurator database tables.

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/luna/current/example-docker/luna_configurator/configs/luna_configurator_postgres.conf:/srv/luna_configurator/configs/config.conf \
--network=host \
-v /tmp/logs/configurator:/srv/logs \
--rm \
--entrypoint bash \
dockerhub.visionlabs.ru/luna/luna-configurator:v.2.2.0 \
-c "alembic upgrade head; cd /srv/luna_configurator/configs/configs/; python3 -m configs.migrate --config /srv/luna_configurator/configs/config.conf head;"

Here:

alembic upgrade head; — Upgrades already existing database structure.
python3 -m configs.migrate head; — Performs settings migrations in Configurator DB and sets revision for migration. The revision will be required during the upgrade to the new LP5 build.

Run Configurator container#

Note: Configurator service configurations are not automatically migrated, unlike all other service configurations. If the configurations of the Configurator service were changed in the previous version of LP and you need to save custom values, then in the command below you need to replace the path /var/lib/luna/current/example-docker/luna_configurator/configs/luna_configurator_postgres.conf with the path with a backup copy of the configurations Configurator service /var/lib/luna/BACKUP_luna_configurator_postgres.conf (see "Save user configurations of the Configurator" section).

Use the docker run command with these parameters to launch Configurator:

docker run \
--env=PORT=5070 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/luna/current/example-docker/luna_configurator/configs/luna_configurator_postgres.conf:/srv/luna_configurator/configs/config.conf \
-v /tmp/logs/configurator:/srv/logs \
--name=luna-configurator \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-configurator:v.2.2.0

At this stage, you can activate logging to file if you need to save them on the server (see the "Logging to server" section).

Image Store#

Image Store container launch#

Note: If you are not going to use the Image Store service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".

Use the following command to launch the Image Store service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5020 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /var/lib/luna/image_store/:/srv/local_storage/ \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/image-store:/srv/logs \
--name=luna-image-store \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-image-store:v.3.11.0

Here -v /var/lib/luna/image_store/:/srv/local_storage/ is the data from the specified folder is added to the Docker container when it is launched. All the data from the specified Docker container folder is saved to this directory.

If you already have a directory with LP buckets you should specify it instead of /var/lib/luna/image_store/.

Buckets creation#

Buckets are required to store data in Image Store. The Image Store service should be launched before the commands execution.

When upgrading from the previous version, it is recommended to launch the bucket creation commands one more time. Hence you make sure that all the required buckets were created.

If the error with code 13006 appears during launching of the listed above commands, the bucket is already created.

There are two ways to create buckets in LP.

Run the listed below scripts to create buckets.

Run this script to create general buckets:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/api:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-api:v.6.26.0 \
python3 ./base_scripts/lis_bucket_create.py -ii --luna-config http://localhost:5070/1

If you are going to use the Tasks service, use the following command to additionally create the "task-result" in the Image Store service:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/tasks:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-tasks:v.3.21.0 \
python3 ./base_scripts/lis_bucket_create.py -ii --luna-config http://localhost:5070/1

If you are going to use the portraits, use the following command to additionally create the "portraits".

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/api:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-backport3:v.0.11.0 \
python3 ./base_scripts/lis_bucket_create.py -ii --luna-config http://localhost:5070/1

Use direct requests to create required buckets.

The curl utility is required for the following requests.

The "visionlabs-samples" bucket is used for face samples storage. The bucket is required for LP utilization.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=visionlabs-samples

The "portraits" bucket is used for portraits storage. The bucket is required for Backport 3 utilization.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=portraits

The "visionlabs-bodies-samples" bucket is used for human bodies samples storage. The bucket is required for LP utilization.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=visionlabs-bodies-samples

The "visionlabs-image-origin" bucket is used for source images storage. The bucket is required for LP utilization.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=visionlabs-image-origin

The "visionlabs-objects" bucket is used for objects storage. The bucket is required for LP utilization.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=visionlabs-objects

The "task-result" bucket for the Tasks service. Do not use it if you are not going to use the Tasks service.

curl -X POST http://127.0.0.1:5020/1/buckets?bucket=task-result

Accounts#

Accounts DB tables creation#

Note. Accounts migration (upgrading from 5.2.0...5.28.0 only). Perform the steps below only if you are upgrading from LP 5.2.0...5.28.0, where the Admin service was not used. If you have used the Admin service before, skip this section and perform the steps in the "Accounts database migration" section.

Use the following command to create Accounts DB tables:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/accounts:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-accounts:v.0.3.0 \
python3 ./base_scripts/db_create.py --luna-config http://localhost:5070/1

Accounts database migration#

You need to execute migration scripts to update your Accounts database structure.

It is recommended to create the back up of your database before applying any changes.

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/accounts:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-accounts:v.0.3.0 \
alembic -x luna-config=http://127.0.0.1:5070/1 upgrade head

In LUNA PLATFORM 5.30.0, the database that stores information about accounts has changed. In LUNA PLATFORM versions 5.2.0...5.28.0, information was stored in the Admin database (luna_admin database), and in versions 5.30.0 and higher, information is stored in the Accounts database (luna_accounts database). If you are upgrading from versions 5.2.0...5.28.0 to the current version, then the script above will transform the luna_admin database to work with new accounts (the database name will remain the same).

Accounts container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5170 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/accounts:/srv/logs \
--name=luna-accounts \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-accounts:v.0.3.0

Licenses#

Note: To use a trial license, it is required to launch the Licenses service on the same server where trial license is being used.

Specify license settings using Configurator#

Follow the steps below to set the settings for HASP-key or Guardant-key.

Specify HASP license settings#

Note: Perform these actions only if the HASP key is used. See the "Specify Guardant license settings" section if the Guardant key is used.

To set the license server address, follow these steps:

Go to the Configurator service interface http://<configurator_server_ip>:5070/.
Specify the "LICENSE_VENDOR" value in the "Setting name" field and click "Apply Filters".
Set the IP address of the server with your HASP key in the field "server_address" in the format "127.0.0.1".
Click "Save".

If the license is activated using the HASP key, then two parameters "vendor" and "server_address" must be specified. If you want to change the HASP protection to Guardant, then you need to add the "license_id" field.

Specify Guardant license settings#

Note: Perform these actions only if the Guardant key is used. See the "Specify HASP license settings" section if the HASP key is used.

To set the license server address, follow these steps:

Go to the Configurator service interface http://<configurator_server_ip>:5070/.
Enter the value "LICENSE_VENDOR" in the "Setting name" field and click "Apply Filters".
Set the IP address of the server with your Guardant key in the "server_address" field.
Set the license ID in the format 0x<your_license_id>, obtained in the section "Save license ID" in the License activation manual, in the "license_id" field.
Click "Save".

If the license is activated using the Guardant key, then three parameters "vendor", "server_address" and "license_id" must be specified. If you want to change the Guardant protection to HASP, then you need to delete the "license_id" field.

Licenses container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5120 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/licenses:/srv/logs \
--name=luna-licenses \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-licenses:v.0.10.0

Faces#

Faces database migration#

You need to execute migration scripts to update your Faces database structure.

It is recommended to create the back up of your database before applying any changes.

Run the following command to perform the Faces DB migration.

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/faces:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-faces:v.4.11.0 \
alembic -x luna-config=http://127.0.0.1:5070/1 upgrade head

Faces container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5030 \
--env=WORKER_COUNT=2 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/faces:/srv/logs \
--name=luna-faces \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-faces:v.4.11.0

Events#

Events database migration#

You need to execute migration scripts to update your Events database structure.

It is recommended to create the back up of your database before applying any changes.

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/events:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-events:v.4.12.0 \
alembic -x luna-config=http://127.0.0.1:5070/1 upgrade head

Events container launch#

Note: If you are not going to use the Events service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5040 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/events:/srv/logs \
--name=luna-events \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-events:v.4.12.0

Python Matcher services#

For matching tasks, you can use either only the Python Matcher service, or additionally use the Python Matcher Proxy service, which redirects matching requests to either the Python Matcher service or matching plugins. This section describes how to use Python Matcher without Python Matcher Proxy.

You need to use the Python Matcher Proxy service only if you are going to use matching plugins. Using Python Matcher Proxy and running the corresponding docker container are described in the "Use Python Matcher with Python Matcher Proxy" section.

See the description and usage of matching plugins in the administrator manual.

Use Python Matcher without Python Matcher Proxy#

The Python Matcher service with matching by the Faces DB is enabled by default after launching.

The Python Matcher service with matching by the Events is also enabled by default. You can disable it by specifying "USE_LUNA_EVENTS = 0" in the "ADDITIONAL_SERVICES_USAGE" settings of Configurator (see "Optional services usage" section). Thus, the Events service will not be used for LUNA PLATFORM.

The Python Matcher that matches using the matcher library is enabled when "CACHE_ENABLED" is set to "true" in the "DESCRIPTORS_CACHE" setting.

A single image is downloaded for the Python Matcher service and the Python Matcher Proxy service.

Python Matcher container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5100 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/python-matcher:/srv/logs \
--name=luna-python-matcher \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-python-matcher:v.1.9.0

Remote SDK#

Change the neural network model for extracting descriptors#

Some builds of LUNA PLATFORM remove the neural network models for extracting descriptors of faces and bodies, and also change the default settings for using models.

If you are upgrading from a version where neural networks were removed, and in the previous build one of the deleted models was specified in the "DEFAULT_FACE_DESCRIPTOR_VERSION" or "DEFAULT_HUMAN_DESCRIPTOR_VERSION" settings, then the Remote SDK service will not start unless you do one of the following actions:

request old neural network model from VisionLabs specialists and transfer it to new container
switch to new version of the neural network, having previously completed the Additional extraction task to re-extract existing descriptors using the new version of the neural network for their further use
switch to new version of the neural network with cessation of the use of old descriptors

See "Prepare to change the neural network version" for details.

Remote SDK container launch#

You can run the Remote SDK service utilizing CPU (set by default) or GPU.

By default, the Remote SDK service is launched with all estimators and detectors enabled. If necessary, you can disable the use of some estimators or detectors when launching the Remote SDK container. Disabling unnecessary estimators enables you to save RAM or GPU memory, since when the Remote SDK service launches, the possibility of performing these estimates is checked and neural networks are loaded into memory. If you disable the estimator or detector, you can also remove its neural network from the Remote SDK container. See the "Enable/disable several estimators and detectors" section of the administrator manual for more information.

Run the Remote SDK service using one of the following commands according to the utilized processing unit.

Run Remote SDK utilizing CPU#

Use the following command to launch the Remote SDK service using CPU:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5220 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/remote-sdk:/srv/logs \
--network=host \
--name=luna-remote-sdk \
--restart=always \
--detach=true \
dockerhub.visionlabs.ru/luna/luna-remote-sdk:v.0.5.0

Run Remote SDK utilizing GPU#

The Remote SDK service does not utilize GPU by default. If you are going to use the GPU, then you should enable its use for the Remote SDK service in the Configurator service.

If you need to use the GPU for all estimators and detectors at once, then you need to use the "global_device_class" parameter in the "LUNA_REMOTE_SDK_RUNTIME_SETTINGS" section. All estimators and detectors will use the value of this parameter if the "device_class" parameter of their settings like "LUNA_REMOTE_SDK__SETTINGS.runtime_settings" is set to "global" (by default for all estimators and detectors).

If you need to use the GPU for a specific estimator or detector, then you need to use the "device_class" parameter in sections like "LUNA_REMOTE_SDK__SETTINGS.runtime_settings".

See section "Calculations using GPU" for additional requirements for GPU utilization.

Use the following command to launch the Remote SDK service using GPU:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5220 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--gpus device=0 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/remote-sdk:/srv/logs \
--network=host \
--name=luna-remote-sdk \
--restart=always \
--detach=true \
dockerhub.visionlabs.ru/luna/luna-remote-sdk:v.0.5.0

Here --gpus device=0 is the parameter specifies the used GPU device and enables GPU utilization. A single GPU can be utilized per Remote SDK instance. Multiple GPU utilization per instance is not available.

Run slim version of Remote SDK#

You can run a slim version of the Remote SDK service that contains only configuration files without neural networks. It is assumed that the user himself will add the neural networks he needs to the container.

The launch of the slim version of the Remote SDK service is intended for advanced users.

To successfully launch the Remote SDK container with a custom set of neural networks, you need to perform the following actions:

Request the required neural networks from VisionLabs.
Place neural networks in a folder with LUNA PLATFORM installed.
Assign appropriate rights to neural network files.
Mount neural network files to the /srv/fsdk/data folder of the Remote SDK container.
Using the arguments of the variable "EXTEND_CMD" to explicitly specify which of the neural networks should be used.

Using the "enable-all-estimators-by-default" flag for the "EXTEND_CMD" variable, you can disable the use of all neural networks (estimators) by default, and then use special flags to explicitly specify which neural networks should be used. If you do not specify this flag or set the value "--enable-all-estimators-by-default=1", the Remote SDK service will try to find all neural networks in the container. If one of the neural networks is not found, the Remote SDK service will not start.

List of available estimators:

Argument	Description
--enable-all-estimators-by-default	Enable all estimators by default.
--enable-human-detector	Simultaneous detector of bodies and bodies.
--enable-face-detector	Face detector.
--enable-body-detector	Body detector.
--enable-face-landmarks5-estimator	Face landmarks5 estimator.
--enable-face-landmarks68-estimator	Face landmarks68 estimator.
--enable-head-pose-estimator	Head pose estimator.
--enable-liveness-estimator	Liveness estimator.
--enable-fisheye-estimator	FishEye effect estimator.
--enable-face-detection-background-estimator	Image background estimator.
--enable-face-warp-estimator	Face sample estimator.
--enable-body-warp-estimator	Body sample estimator.
--enable-quality-estimator	Image quality estimator.
--enable-image-color-type-estimator	Face color type estimator.
--enable-face-natural-light-estimator	Natural light estimator.
--enable-eyes-estimator	Eyes estimator.
--enable-gaze-estimator	Gaze estimator.
--enable-mouth-attributes-estimator	Mouth attributes estimator.
--enable-emotions-estimator	Emotions estimator.
--enable-mask-estimator	Mask estimator.
--enable-glasses-estimator	Glasses estimator.
--enable-eyebrow-expression-estimator	Eyebrow estimator.
--enable-red-eyes-estimator	Red eyes estimator.
--enable-headwear-estimator	Headwear estimator.
--enable-basic-attributes-estimator	Basic attributes estimator.
--enable-face-descriptor-estimator	Face descriptor extraction estimator.
--enable-body-descriptor-estimator	Body descriptor extraction estimator.
--enable-body-attributes-estimator	Body attributes estimator.
--enable-people-count-estimator	People count estimator.
--enable-deepfake-estimator	Deepfake estimator.

See the detailed information on enabling and disabling certain estimators in the section "Enable/disable several estimators and detectors " of the administrator manual.

Below is an example of a command to assign rights to a neural network file:

chown -R 1001:0 /var/lib/luna/current/<neural_network_name>.plan

Example of a command to run Remote SDK container with mounting neural networks for face detection and face descriptor extraction:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5220 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--env=EXTEND_CMD="--enable-all-estimators-by-default=0 --enable-face-detector=1 --enable-face-descriptor-estimator=1" \
-v /var/lib/luna/current/cnn59b_cpu-avx2.plan:/srv/fsdk/data/cnn59b_cpu-avx2.plan \
-v /var/lib/luna/current/FaceDet_v3_a1_cpu-avx2.plan:/srv/fsdk/data/FaceDet_v3_a1_cpu-avx2.plan \
-v /var/lib/luna/current/FaceDet_v3_redetect_v3_cpu-avx2.plan:/srv/fsdk/data/FaceDet_v3_redetect_v3_cpu-avx2.plan \
-v /var/lib/luna/current/slnet_v3_cpu-avx2.plan:/srv/fsdk/data/slnet_v3_cpu-avx2.plan \
-v /var/lib/luna/current/LNet_precise_v2_cpu-avx2.plan:/srv/fsdk/data/LNet_precise_v2_cpu-avx2.plan \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/remote-sdk:/srv/logs \
--network=host \
--name=luna-remote-sdk \
--restart=always \
--detach=true \
dockerhub.visionlabs.ru/luna/luna-remote-sdk:v.0.5.0

Handlers#

Note: If you are not going to use the Handlers service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".

Handlers database migration#

You need to execute migration scripts to update your Handlers database structure.

It is recommended to create the back up of your database before applying any changes.

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/handlers:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-handlers:v.3.5.0 \
alembic -x luna-config=http://127.0.0.1:5070/1 upgrade head

Handlers container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5090 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/handlers:/srv/logs \
--name=luna-handlers \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-handlers:v.3.5.0

Tasks#

Note: If you are not going to use the Tasks service, do not launch the Tasks container and the Tasks Worker container. Disable the service utilization in Configurator. See section "Optional services usage".

Tasks database migration#

You need to execute migration scripts to update your Tasks database structure.

It is recommended to create the back up of your database before applying any changes.

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/tasks:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-tasks:v.3.21.0 \
alembic -x luna-config=http://127.0.0.1:5070/1 upgrade head

Tasks and Tasks Worker containers launch#

Tasks service image includes the Tasks service and the Tasks Worker. They both must be launched.

The "task-result" bucket should be created for the Tasks service before the service launch. The buckets creation is described in the "Buckets creation".

If it is necessary to use the Estimator task using a network disk, then you should first mount the directory with images from the network disk into special directories of Tasks and Tasks Worker containers. See the "Estimator task" section in the administrator manual for details.

Tasks Worker launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5051 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--env=SERVICE_TYPE="tasks_worker" \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/tasks-worker:/srv/logs \
--name=luna-tasks-worker \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-tasks:v.3.21.0

Tasks launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5050 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/tasks:/srv/logs \
--name=luna-tasks \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-tasks:v.3.21.0

Sender#

Sender container launch#

Note: If you are not going to use the Sender service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5080 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/sender:/srv/logs \
--name=luna-sender \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-sender:v.2.11.0

API#

API container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5000 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--name=luna-api \
--restart=always \
--detach=true \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/api:/srv/logs \
--network=host \
dockerhub.visionlabs.ru/luna/luna-api:v.6.26.0

Account migration#

Note. Accounts migration (upgrading from 5.2.0...5.28.0 only). If you want to keep the ability to use the "account_id" that was used as the "Luna-Account-Id" header in versions 5.2.0...5.28.0 (without creating an account in the Admin service), then you need to link the old "account_id" to the account being created.

Example of CURL-request to the "create account" resource:

curl --location --request POST 'http://127.0.0.1:5000/6/accounts' \
--header 'Content-Type: application/json' \
--header 'Luna-Account-Id: <your_old_account_id>' \
--data '{
  "login": "user@mail.com",
  "password": "password",
  "account_type": "user",
  "description": "description"
}'

It is necessary to replace the authentication data from the example with your own.

To work with tokens, you must have an account.

Admin#

Admin container launch#

Note: If you are not going to use the Admin service, do not launch this container.

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5010 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/admin:/srv/logs \
--name=luna-admin \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-admin:v.5.6.0

Monitoring data about the number of executed requests is saved in the luna-admin bucket of the InfluxDB. To enable data saving use the following command:

docker exec -it luna-admin python3 ./base_scripts/influx2_cli.py create_usage_task --luna-config http://127.0.0.1:5070/1

Backport 3#

The section describes launching of Backport 3 service.

The service is not mandatory for utilizing LP5 and is required for emulation of LP 3 API only.

Backport 3 database migration#

You need to execute migration scripts to update your Backport 3 DB structure.

It is recommended to create the back up of your database before applying any changes.

Run the following command to perform the Backport 3 DB migration.

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/backport3:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-backport3:v.0.11.0 \
alembic -x luna-config=http://127.0.0.1:5070/1 upgrade head

Accounts and tokens migration#

Note. Accounts migration (upgrading from 5.2.0...5.28.0 only). Use the following command to migrate existing accounts and tokens from Backport 3 previous version:

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/backport3:/srv/logs \
--rm \
--network=host \
--entrypoint bash dockerhub.visionlabs.ru/luna/luna-backport3:v.0.11.0 -c "cd ./base_scripts/migrate_accounts/ && pip3 install -r requirements-postgres.txt && python3 ./run.py --backport_db_url postgres://luna:luna@127.0.0.1:5432/luna_backport3 --accounts_db_url postgres://luna:luna@127.0.0.1:5432/luna_admin"

Here:

python3 ./run.py — Running the script for migrating accounts and tokens.
--backport_db_url postgres://luna:luna@127.0.0.1:5432/luna_backport3 — "luna_backport3" DB address flag.
--accounts_db_url postgres://luna:luna@127.0.0.1:5432/luna_admin — "luna_admin" DB address flag.

When migrating API and Faces service databases, all Backport 3 accounts will be migrated and stored in the database of the Accounts service. All migrated accounts will be of type "user". The fields "password", "email" and "organization_name" will be transferred from the "account" table of the Backport 3 database to the "account" table of the Accounts database under new names — "password", "login" and "description" respectively. Tokens will remain stored in the Backport3 database, but their identifier will also be entered in the Accounts database, where the necessary permissions will be automatically set.

Backport 3 container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5140 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--name=luna-backport3 \
--restart=always \
--detach=true \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/backport3:/srv/logs \
--network=host \
dockerhub.visionlabs.ru/luna/luna-backport3:v.0.11.0

User Interface 3#

The User Interface 3 is used with the Backport 3 service only.

User Interface 3 container launch#

Use the following command to launch the service:

docker run \
--env=PORT=4100 \
--env=LUNA_API_URL=http://127.0.0.1:5140 \
--name=luna-ui-3 \
--restart=always \
--detach=true \
--network=host \
-v /etc/localtime:/etc/localtime:ro \
dockerhub.visionlabs.ru/luna/luna3-ui:v.0.5.11

Here:

--env=LUNA_API_URL — Specifies the URL of the Backport 3 service.
--env=PORT — Specifies the port of the User Interface 3 service.

Backport 4#

The section describes launching of Backport 4 service.

The service is not mandatory for utilizing LP5 and is required for emulation of LP 4 API only.

Backport 4 container launch#

Use the following command to launch the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5130 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--name=luna-backport4 \
--restart=always \
--detach=true \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/backport4:/srv/logs \
--network=host \
dockerhub.visionlabs.ru/luna/luna-backport4:v.1.5.0

User Interface 4#

The User Interface 4 is used with the Backport 4 service only.

User Interface 4 container launch#

Note: You should have the account with account type user before launching the User Interface 4 container. Its login and password in Base64 format will be used to work with the user interface.

Use the following command to launch the service:

docker run \
--env=PORT=4200 \
--env=LUNA_API_URL=http://<server_external_ip>:5130 \
--env=BASIC_AUTH=dXNlckBtYWlsLmNvbTpwYXNzd29yZA== \
--name=luna-ui-4 \
--restart=always \
--detach=true \
--network=host \
-v /etc/localtime:/etc/localtime:ro \
dockerhub.visionlabs.ru/luna/luna4-ui:v.0.1.6

Here:

--env=PORT — Sets the port for running User Interface 4.
--env=BASIC_AUTH — Sets the Basic authorization for the account which data is displayed in the user interface.
--env=LUNA_API_URL — Sets the URL of the Backport 4 service.

You should use the external IP of the service, not localhost.

You should specify the Backport 4 service port ("5130" is set by default).

Lambda#

Note: If you are not going to use the Lambda service, do not run this container.

Working with the Lambda service is possible only when deploying LUNA PLATFORM services in Kubernetes. To use it, you need to deploy LUNA PLATFORM services in Kubernetes yourself or consult VisionLabs specialists. Use the commands below as reference information.

Enable the use of the Lambda service (see the section "Using optional services").

Prepare Docker registry#

It is necessary to prepare a registry for storing Lambda docker images. Transfer the base images and Kaniko executor image to your registry using the following commands.

Upload the images from the remote repository to the local image storage.

docker pull dockerhub.visionlabs.ru/luna/lpa-lambda-base-fsdk:v.0.0.69

docker pull dockerhub.visionlabs.ru/luna/lpa-lambda-base:v.0.0.69

docker pull dockerhub.visionlabs.ru/luna/kaniko-executor:latest

Add new names to the images by replacing new-registry on their own. The names of the base images in the user registry must be the same as in the dockerhub.visionlabs.ru/luna registry.

docker tag dockerhub.visionlabs.ru/luna/lpa-lambda-base-fsdk:v.0.0.69 new-registry/lpa-lambda-base-fsdk:v.0.0.69

docker tag dockerhub.visionlabs.ru/luna/lpa-lambda-base:v.0.0.69 new-registry/lpa-lambda-base:v.0.0.69

docker tag dockerhub.visionlabs.ru/luna/kaniko-executor:latest new-registry/kaniko-executor:latest

Push local images to your remote repository by replacing new-registry on their own.

docker push new-registry/lpa-lambda-base-fsdk:v.0.0.69

docker push new-registry/lpa-lambda-base:v.0.0.69

docker push new-registry/kaniko-executor:latest

Lambda database migration#

You need to execute migration scripts to update your Lambda database structure.

It is recommended to create the back up of your database before applying any changes.

docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/lambda:/srv/logs \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/luna-lambda:v.0.3.0 \
alembic -x luna-config=http://127.0.0.1:5070/1 upgrade head

Lambda container launch#

Use the following command to start the service:

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5210 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/lambda:/srv/logs \
--name=luna-lambda \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-lambda:v.0.3.0