Default ports for services#
Service name | Port |
---|---|
LUNA PLATFORM API | 5000 |
LUNA PLATFORM Admin | 5010 |
LUNA PLATFORM Image Store | 5020 |
LUNA PLATFORM Faces | 5030 |
LUNA PLATFORM Events | 5040 |
LUNA PLATFORM Tasks | 5050 |
LUNA PLATFORM Tasks Worker | 5051 |
LUNA PLATFORM Configurator | 5070 |
LUNA PLATFORM Sender | 5080 |
LUNA PLATFORM Handlers | 5090 |
LUNA PLATFORM Python Matcher | 5100 |
LUNA PLATFORM Licenses | 5120 |
LUNA PLATFORM Backport 4 | 5130 |
LUNA PLATFORM Backport 3 | 5140 |
LUNA PLATFORM Accounts | 5170 |
LUNA PLATFORM Lambda | 5210 |
LUNA PLATFORM Remote SDK | 5220 |
LUNA PLATFORM 3 User Interface | 4100 |
LUNA PLATFORM 4 User Interface | 4200 |
Oracle DB | 1521 |
PostgreSQL | 5432 |
Redis DB | 6379 |
InfluxDB | 8086 |
Grafana | 3000 |
Configuration names for services#
The table below includes the service names in the Configurator service. Use these parameters to configure your services.
Service | Service name in Configurator |
---|---|
API | luna-api |
Licenses | luna-licenses |
Faces | luna-faces |
Image Store | luna-image-store |
Accounts | luna-accounts |
Tasks | luna-tasks |
Events | luna-events |
Sender | luna-sender |
Admin | luna-admin |
Remote SDK | luna-remote-sdk |
Handlers | luna-handlers |
Lambda | luna-lambda |
Python Matcher | luna-python-matcher |
Backport 3 | luna-backport3 |
Backport 4 | luna-backport4 |
Settings for the Configurator service are set in its configuration file.
Introduction#
This document provides example steps for preparing an environment using the Storages utility and then launching LUNA PLATFORM containers.
It is recommended to familiarize yourself with the Storages utility manual before deploying the LUNA PLATFORM.
This document includes an example of LUNA PLATFORM deployment. It implements LUNA PLATFORM minimum power operating for demonstration purposes and cannot be used for the production system.
It is considered that installation is performed on the server with CentOS, where LP was not installed.
Firewall and SELinux should be manually configured on the server by the administrator. Their configuration is not described in this document.
No data backup or databases replication is implemented for LP data in this installation.
To deploy the LUNA PLATFORM, follow the steps in the following sections:
- "Before launch" — Actions to unpack archives, prepare directories, configure a license, etc. Some actions may be optional.
- "Run third-party services" — Launching PostgreSQL, Redis, Influx databases.
- "Prepare environment" — Preparing the environment (creating databases, buckets, etc.).
- "Launch services" — Launching containers with LUNA PLATFORM services.
The "Additional Information" section provides useful information on describing service launch parameters, Docker commands, enabling Grafana for monitoring visualization, etc.
A network license is required to use the LUNA PLATFORM in Docker containers. The license is provided by VisionLabs on request separately from the delivery. The license key is created using the fingerprint of the system. This fingerprint is created based on information about the hardware characteristics of the server. Thus, the received license key will work only on the same server from which the system fingerprint was obtained. LUNA PLATFORM can be activated using one of two utilities — HASP or Guardant. The section "Activate license" provides instructions for activating the license key for each method.
This document does not include a tutorial for Docker usage. Please refer to the Docker documentation to find more information about Docker:
It is recommended to use orchestration services for the commercial usage of LP. Their utilization is not described in this manual.
All the provided commands should be executed using the Bash shell (when you launch commands directly on the server) or in a program for working with network protocols (when you remotely connect to the server), for example, Putty.
Before launch#
Make sure that you are the root user before launch!
Before launching the LUNA PLATFORM, you must perform the following actions:
- Unpack the LUNA PLATFORM distribution.
- Create symbolic link.
- Change group and owner for new directories.
- Configure SELinux and Firewall.
- Activate license.
- Login to VisionLabs registry.
- Install Docker.
- Choose logging method.
- Set up GPU computing if you plan to use GPU.
Distribution unpacking#
The distribution package is an archive luna_v.5.67.0, where v.5.67.0 is a numerical identifier, describing the current LUNA PLATFORM version.
The archive includes configuration files, required for installation and exploitation. It does not include Docker images for the services. They should be downloaded from the Internet.
Move the distribution package to the directory on your server before the installation. For example, move the files to /root/
directory. The directory should not contain any other distribution or license files except the target ones.
Create directory for distribution file unpacking.
mkdir -p /var/lib/luna
Move the distribution to the created directory.
mv /root/luna_v.5.67.0.zip /var/lib/luna
Install the unzip archiver if it is necessary.
yum install -y unzip
Go to the folder with distribution.
cd /var/lib/luna
Unzip files.
unzip luna_v.5.67.0.zip
Symbolic link creation#
Create a symbolic link.
The link indicates that the current version of the distribution file is used to run LUNA PLATFORM.
ln -s luna_v.5.67.0 current
Changing group and owner for directories#
LP services are launched inside the containers by the "luna" user. Therefore, it is required to set permissions for this user to use the mounted volumes.
Go to the LP "example-docker" directory.
cd /var/lib/luna/current/example-docker/
Create a directory to store settings.
mkdir luna_configurator/used_dumps
Set permissions for the user with UID 1001 and group 0 to use the mounted directories.
chown -R 1001:0 luna_configurator/used_dumps
Open the LP root directory.
cd /var/lib/luna/
Create a directory to store Image Store buckets.
mkdir image_store
Set permissions for the user with UID 1001 and group 0 to use the mounted directories.
chown -R 1001:0 image_store
SELinux and Firewall#
You must configure SELinux and Firewall so that they do not block LUNA PLATFORM services.
SELinux and Firewall configurations are not described in this guide.
If SELinux and Firewall are not configured, the installation cannot be performed.
License activation#
To activate the license, follow these steps:
- Follow the steps from license activation manual.
- Set settings for HASP license or Guardant license.
Actions from License activation manual#
Open the license activation manual and follow the necessary steps.
Note: This action is mandatory. The license will not work without following the steps to activate the license from the corresponding manual.
Ways to specify HASP license settings#
For the HASP key, you need to specify the IP address of the licensing server. It can be set in one of two ways:
- In the dump file "platform_settings.json" (see below). The contents of the default settings will be overwritten by the contents of this file when the Configurator service starts.
- In the Licenses service settings in the Configurator user interface (see section "Specify license settings using Configurator").
Choose the most convenient method and follow the steps described in the relevant sections.
Specify HASP license settings using dump file#
Open the "platform_settings.json" file.
vi /var/lib/luna/current/extras/conf/platform_settings.json
Set the server IP address with your HASP key in the "server_address" field.
{
"value": {
"vendor": "hasp",
"server_address": "127.0.0.1"
},
"description":"License vendor config",
"name":"LICENSE_VENDOR",
"tags":[]
},
Save the file.
If the license is activated using the HASP key, then two parameters "vendor" and "server_address" must be specified. If you want to change the HASP protection to Guardant, then you need to add the "license_id" field.
Ways to specify Guardant license settings#
For the Guardant key, you need to specify the IP address of the licensing server and the license ID. The settings can be set in one of two ways:
- In the dump file "platform_settings.json" (see below). The contents of the default settings will be overwritten by the contents of this file at the launch stage of the Configurator service.
- In the Licenses service settings in the Configurator user interface (see the section "Specify license settings using Configurator").
Choose the most convenient method and follow the steps described in the relevant sections.
Specify Guardant license settings using dump file#
Open the file "platform_settings.json".
vi /var/lib/luna/current/extras/conf/platform_settings.json
Enter the following data:
- IP address of the server with your Guardant key in the "server_address" field.
- License ID in the format
0x<your_license_id>
, obtained in the section "Save license ID" of license activation manual, in the "license_id" field.
{
"value": {
"vendor": "guardant",
"server_address": "127.0.0.1",
"license_id": "0x92683BEA"
},
"description":"License vendor config",
"name":"LICENSE_VENDOR",
"tags":[]
},
Save the file.
If the license is activated using the Guardant key, then three parameters "vendor", "server_address" and "license_id" must be specified. If you want to change the Guardant protection to HASP, then you need to delete the "license_id" field.
Login to registry#
When launching containers, you should specify a link to the image required for the container launching. This image will be downloaded from the VisionLabs registry. Before that, you should login to the registry.
Login and password can be requested from the VisionLabs representative.
Enter login
docker login dockerhub.visionlabs.ru --username <username>
After running the command, you will be prompted for a password. Enter password.
In the
docker login
command, you can enter the login and password at the same time, but this does not guarantee security because the password can be seen in the command history.
Docker installation#
The Docker installation is described in the official documentation.
Note: Docker version 25.0.3 was used when testing this instruction. It is not guaranteed to work with higher versions of Docker.
Quick installation commands are listed below.
Check the official documentation for updates if you have any problems with the installation.
Install dependencies:
yum install -y yum-utils device-mapper-persistent-data lvm2
Add repository:
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Install Docker:
yum -y install docker-ce docker-ce-cli containerd.io
Launch Docker:
systemctl start docker
systemctl enable docker
Check Docker status:
systemctl status docker
Choose logging method#
There are two methods to output logs in LUNA PLATFORM:
- Standard log output (stdout).
- Log output to a file.
Log output settings are set in the settings of each service in the <SERVICE_NAME>_LOGGER
section.
If necessary, you can use both methods of displaying logs.
For more information about the LUNA PLATFORM logging system, see the "Logging" section in the administrator manual.
Logging to stdout#
This method is used by default and requires no further action.
It is recommended to configure Docker log rotation to limit log sizes (see "Docker log rotation").
Logging to file#
Note: When you enable saving logs to a file, you should remember that logs occupy a certain place in the storage, and the process of logging to a file negatively affects system performance.
To use this method, you need to perform the following additional actions:
- Before launching the services: Create directories for logs on the server.
- After launching the services: Activate log recording and set the location of log storage inside LP service containers.
- During the launch of services: Configure synchronization of log directories in the container with logs on the server using the
volume
argument at the start of each container.
Examples of container launch commands in this documentation contain arguments for synchronizing log directories.
Note that the above steps must be performed before, during and after starting the services. Saving logs to file will not work if you perform all actions after starting the containers.
See the instructions for enabling logging to files in the "Logging the server" section.
Calculations using GPU#
You can use GPU for the general calculations performed by Remote SDK.
Skip this section if you are not going to utilize GPU for your calculations.
You need to install NVIDIA Container Toolkit to use GPU with Docker containers. The example of the installation is given below.
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.repo | tee /etc/yum.repos.d/nvidia-docker.repo
yum install -y nvidia-container-toolkit
systemctl restart docker
Check the NVIDIA Container toolkit operating by running a base CUDA container (this container is not provided in the LP distribution and should be downloaded from the Internet):
docker run --rm --gpus all nvidia/cuda:11.4.3-base-centos7 nvidia-smi
See the NVIDIA documentation for additional information.
Attributes extraction on the GPU is engineered for maximum throughput. The input images are processed in batches. This reduces computation cost per image but does not provide the shortest latency per image.
GPU acceleration is designed for high load applications where request counts per second consistently reach thousands. It won’t be beneficial to use GPU acceleration in non-extensively loaded scenarios where latency matters.
Run third-party services#
This section describes the launching of databases in docker containers. They must be launched before environment preparation and LP services.
InfluxDB#
Monitoring LUNA PLATFORM services requires running the Influx 2.0.8-alpine database. Below are the commands to launch the InfluxDB container.
For more information, see the "Monitoring" section in the administrator manual.
If necessary, you can configure the visualization of monitoring data using the LUNA Dashboards service, which includes a configured Grafana data visualization system. In addition, you can launch the Grafana Loki tool for advanced work with logs. See the instructions for launching LUNA Dashboards and Grafana Loki in the "Monitoring and logs visualization using Grafana" section.
Note: If necessary, you can use an external InfluxDB 2.0.8-alpine. In this case, you can skip the command below, but you will have to set custom settings for each LUNA PLATFORM service.
Use the docker run
command with these parameters:
docker run \
-e DOCKER_INFLUXDB_INIT_MODE=setup \
-e DOCKER_INFLUXDB_INIT_BUCKET=luna_monitoring \
-e DOCKER_INFLUXDB_INIT_USERNAME=luna \
-e DOCKER_INFLUXDB_INIT_PASSWORD=password \
-e DOCKER_INFLUXDB_INIT_ORG=luna \
-e DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=kofqt4Pfqjn6o0RBtMDQqVoJLgHoxxDUmmhiAZ7JS6VmEnrqZXQhxDhad8AX9tmiJH6CjM7Y1U8p5eSEocGzIA== \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/luna/influx:/var/lib/influxdb2 \
--restart=always \
--detach=true \
--network=host \
--name influxdb \
dockerhub.visionlabs.ru/luna/influxdb:2.0.8-alpine
PostgreSQL#
Note: If necessary, you can use an external PostgreSQL database. In this case, you can skip the command below and perform the actions from the section "External PostgreSQL configuration".
Use the following command to launch PostgreSQL.
docker run \
--env=POSTGRES_USER=luna \
--env=POSTGRES_PASSWORD=luna \
--shm-size=1g \
-v /var/lib/luna/postgresql/data/:/var/lib/postgresql/data/ \
-v /etc/localtime:/etc/localtime:ro \
--name=postgres \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/postgis-vlmatch:16
-v /var/lib/luna/current/example-docker/postgresql/data/:/var/lib/postgresql/data/
— The volume command enables you to mount the "data" folder to the PostgreSQL container. The folder on the server and the folder in the container will be synchronized. The PostgreSQL data from the container will be saved to this directory.
--network=host
— If you need to change the port for PotgreSQL, you should change this string to -p 5440:5432
. Where the first port 5440
is the local port and 5432
is the port used inside the container.
You should create all the databases for LP services manually if you are going to use an already installed PostgreSQL.
Redis#
Note: If you already have Redis installed, skip this step.
Use the following command to launch Redis.
docker run \
-v /etc/localtime:/etc/localtime:ro \
--name=redis \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/redis:7.2
Prepare environment#
The environment is prepared using the Storages service. Using the command below will execute:
- Creating buckets in InfluxDB for monitoring work (if not done previously)
- Creating buckets for the Image Store service (if not done previously)
- Preparing the Influx database for collecting aggregated statistics using the Admin service (if not done previously)
- Creating databases, adding VLMatch functions, creating migration scripts for PostgreSQL databases
- Uploading basic settings to the Configurator database
When preparing the environment, the default configuration file of the Storages service will be used, containing all the default settings for connecting to databases, buckets, etc. If you need to use non-default settings or run LUNA PLATFORM on different servers, you need to edit the configuration file before running the environment preparation command:
vi /var/lib/luna/current/extras/conf/storages_config.conf
If necessary, you can first prepare the environment for the Configurator service, start it, and then prepare the environment for all other services using the settings from the running Configurator service. See the Storages utility manual for examples and detailed information about Storages.
Prepare environment command#
Prepare the environment with the following command:
docker run \
--rm \
--network=host \
-v /var/lib/luna/current/extras/conf/storages_config.conf:/srv/storages_config.conf \
-v /var/lib/luna/image_store/:/srv/local_storage/ \
dockerhub.visionlabs.ru/luna/storages:v.0.2.51 \
bash -c "luna_prepare prepare all_entities \
--platform_version=v.5.67.0 \
--profile=backports \
--config=/srv/storages_config.conf \
--local-buckets=/srv/local_storage"
Here:
luna_prepare prepare all_entities
— Command "prepare" to prepare all entities.--platform_version
— Named argument containing the LUNA PLATFORM version.--profile
— Named argument containing the profile (list of services)backports
, which means that the environment will be prepared for all services, including Backport 3 and Backport 4 services.-v /var/lib/luna/image_store/:/srv/local_storage/
— Command to mount a directory with Image Store buckets.-v /var/lib/luna/current/extras/conf/storages_config.conf:/srv/storages_config.conf
— Command to mount the Storages configuration file.-local-buckets=/srv/local_storage
— Named argument containing the directory where the Image Store buckets should be created.--config=/srv/storages_config.conf
— Named argument containing the address of the configuration file for use by the Storages service.
Optional services usage#
The listed below services are not mandatory for LP:
- Events
- Image Store
- Tasks
- Sender
- Handlers
- Python Matcher Proxy (disabled by default)
- Lambda (disabled by default)
Working with the Lambda service is possible only when deploying LUNA PLATFORM services in Kubernetes. See detailed information in the installation manual using Kubernetes.
You can disable them if their functionality is not required for your tasks.
Use the "ADDITIONAL_SERVICES_USAGE" section in the API service settings in the Configurator service to disable unnecessary services.
You can use the dump file provided in the distribution package to enable/disable services before Configurator launch.
vi /var/lib/luna/current/extras/conf/platform_settings.json
Disabling any of the services has certain consequences. For more information, see the "Disableable services" section of the administrator manual.
Storages service will not prepare the environment for services that are disabled in the "ADDITIONAL_SERVICES_USAGE" setting.
Load dump file#
Run the following command to upload the dump file to the Configurator:
docker run \
--rm \
--network=host \
-v /var/lib/luna/current/extras/conf/platform_settings.json:/srv/platform_settings.json \
dockerhub.visionlabs.ru/luna/storages:v.0.2.51 \
bash -c "luna_prepare load_dump \
--dump-file=/srv/platform_settings.json"
Here:
luna_prepare load_dump
— Command "load_dump", which enables you to upload a dump file to the Configurator database.-v /var/lib/luna/current/extras/conf/platform_settings.json:/srv/platform_settings.json \
— Command to mount the dump fileplatform_settings.json
.--dump-file=/srv/platform_settings.json
— Named argument containing the address of the dump file inside the container.
Services launch#
This section provides examples of commands for launching LUNA PLATFORM services.
LUNA PLATFORM services must be launched in the following sequence:
- Databases, Balancers, HASP service and other third-party party software.
- Configurator
- Image Store
- Accounts
- Licenses
- Faces
- Events
- Python Matcher
- Python Matcher Proxy. The service is disabled by default
- Remote SDK
- Handlers
- Tasks
- Sender
- API
- Admin
The following service are used when emulation of LUNA PLATFORM 3 is required only:
The following service are used when emulation of LUNA PLATFORM 4 is required only:
It is recommended to launch containers one by one and wait for the container status to become "up" (use the docker ps
command).
Some of these services are optional and you can disable their use. It is recommended to use Events, Tasks, Sender and Admin services by default. See the "Optional services usage" section for details.
When launching each service, certain parameters are used, for example, --detach
, --network
, etc. See the section "Launching parameters description" for more detailed information about all launch parameters of LUNA PLATFORM services and databases.
See the "Docker commands" section for details about working with containers.
Configurator#
Run Configurator container#
Use the docker run
command with these parameters to launch Configurator:
docker run \
--env=PORT=5070 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/luna/current/example-docker/luna_configurator/configs/luna_configurator_postgres.conf:/srv/luna_configurator/configs/config.conf \
-v /tmp/logs/configurator:/srv/logs \
--name=luna-configurator \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-configurator:v.2.2.18
At this stage, you can activate logging to file if you need to save them on the server (see the "Logging to server" section).
Image Store#
Image Store container launch#
Note: If you are not going to use the Image Store service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".
Use the following command to launch the Image Store service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5020 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /var/lib/luna/image_store/:/srv/local_storage/ \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/image-store:/srv/logs \
--name=luna-image-store \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-image-store:v.3.14.2
Here -v /var/lib/luna/image_store/:/srv/local_storage/
is the data from the specified folder is added to the Docker container when it is launched. All the data from the specified Docker container folder is saved to this directory.
If you already have a directory with LP buckets you should specify it instead of
/var/lib/luna/image_store/
.
Accounts#
Accounts container launch#
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5170 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/accounts:/srv/logs \
--name=luna-accounts \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-accounts:v.0.3.9
Licenses#
Note: To use a trial license, it is required to launch the Licenses service on the same server where trial license is being used.
Specify license settings using Configurator#
Follow the steps below to set the settings for HASP-key or Guardant-key.
Note: Do not perform the steps described below if you have already specified the license settings in the sections "Specify HASP license settings using dump file" or "Specify Guardant license settings using dump file".
Specify HASP license settings#
Note: Perform these actions only if the HASP key is used. See the "Specify Guardant license settings" section if the Guardant key is used.
To set the license server address, follow these steps:
-
Go to the Configurator service interface
http://<configurator_server_ip>:5070/
. -
Specify the "LICENSE_VENDOR" value in the "Setting name" field and click "Apply Filters".
-
Set the IP address of the server with your HASP key in the field "server_address" in the format "127.0.0.1".
-
Click "Save".
If the license is activated using the HASP key, then two parameters "vendor" and "server_address" must be specified. If you want to change the HASP protection to Guardant, then you need to add the "license_id" field.
Specify Guardant license settings#
Note: Perform these actions only if the Guardant key is used. See the "Specify HASP license settings" section if the HASP key is used.
To set the license server address, follow these steps:
-
Go to the Configurator service interface
http://<configurator_server_ip>:5070/
. -
Enter the value "LICENSE_VENDOR" in the "Setting name" field and click "Apply Filters".
-
Set the IP address of the server with your Guardant key in the "server_address" field.
-
Set the license ID in the format
0x<your_license_id>
, obtained in the section "Save license ID" in the License activation manual, in the "license_id" field. -
Click "Save".
If the license is activated using the Guardant key, then three parameters "vendor", "server_address" and "license_id" must be specified. If you want to change the Guardant protection to HASP, then you need to delete the "license_id" field.
Licenses container launch#
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5120 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/licenses:/srv/logs \
--name=luna-licenses \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-licenses:v.0.10.18
Faces#
Faces container launch#
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5030 \
--env=WORKER_COUNT=2 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/faces:/srv/logs \
--name=luna-faces \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-faces:v.4.11.9
Events#
Events container launch#
Note: If you are not going to use the Events service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5040 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/events:/srv/logs \
--name=luna-events \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-events:v.4.12.9
Python Matcher services#
For matching tasks, you can use either only the Python Matcher service, or additionally use the Python Matcher Proxy service, which redirects matching requests to either the Python Matcher service or matching plugins. This section describes how to use Python Matcher without Python Matcher Proxy.
You need to use the Python Matcher Proxy service only if you are going to use matching plugins. Using Python Matcher Proxy and running the corresponding docker container are described in the "Use Python Matcher with Python Matcher Proxy" section.
See the description and usage of matching plugins in the administrator manual.
Use Python Matcher without Python Matcher Proxy#
The Python Matcher service with matching by the Faces DB is enabled by default after launching.
The Python Matcher service with matching by the Events is also enabled by default. You can disable it by specifying "USE_LUNA_EVENTS = 0" in the "ADDITIONAL_SERVICES_USAGE" settings of Configurator (see "Optional services usage" section). Thus, the Events service will not be used for LUNA PLATFORM.
The Python Matcher that matches using the matcher library is enabled when "CACHE_ENABLED" is set to "true" in the "DESCRIPTORS_CACHE" setting.
A single image is downloaded for the Python Matcher service and the Python Matcher Proxy service.
Python Matcher container launch#
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5100 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/python-matcher:/srv/logs \
--name=luna-python-matcher \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-python-matcher:v.1.9.9
Remote SDK#
Remote SDK container launch#
You can run the Remote SDK service utilizing CPU (set by default) or GPU.
By default, the Remote SDK service is launched with all estimators and detectors enabled. If necessary, you can disable the use of some estimators or detectors when launching the Remote SDK container. Disabling unnecessary estimators enables you to save RAM or GPU memory, since when the Remote SDK service launches, the possibility of performing these estimates is checked and neural networks are loaded into memory. If you disable the estimator or detector, you can also remove its neural network from the Remote SDK container. See the "Enable/disable several estimators and detectors" section of the administrator manual for more information.
Run the Remote SDK service using one of the following commands according to the utilized processing unit.
Run Remote SDK utilizing CPU#
Use the following command to launch the Remote SDK service using CPU:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5220 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/remote-sdk:/srv/logs \
--network=host \
--name=luna-remote-sdk \
--restart=always \
--detach=true \
dockerhub.visionlabs.ru/luna/luna-remote-sdk:v.0.6.0
Run Remote SDK utilizing GPU#
The Remote SDK service does not utilize GPU by default. If you are going to use the GPU, then you should enable its use for the Remote SDK service in the Configurator service.
If you need to use the GPU for all estimators and detectors at once, then you need to use the "global_device_class" parameter in the "LUNA_REMOTE_SDK_RUNTIME_SETTINGS" section. All estimators and detectors will use the value of this parameter if the "device_class" parameter of their settings like "LUNA_REMOTE_SDK_
If you need to use the GPU for a specific estimator or detector, then you need to use the "device_class" parameter in sections like "LUNA_REMOTE_SDK_
See section "Calculations using GPU" for additional requirements for GPU utilization.
Use the following command to launch the Remote SDK service using GPU:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5220 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--gpus device=0 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/remote-sdk:/srv/logs \
--network=host \
--name=luna-remote-sdk \
--restart=always \
--detach=true \
dockerhub.visionlabs.ru/luna/luna-remote-sdk:v.0.6.0
Here --gpus device=0
is the parameter specifies the used GPU device and enables GPU utilization. A single GPU can be utilized per Remote SDK instance. Multiple GPU utilization per instance is not available.
Run slim version of Remote SDK#
You can run a slim version of the Remote SDK service that contains only configuration files without neural networks. It is assumed that the user himself will add the neural networks he needs to the container.
The launch of the slim version of the Remote SDK service is intended for advanced users.
To successfully launch the Remote SDK container with a custom set of neural networks, you need to perform the following actions:
- Request the required neural networks from VisionLabs.
- Place neural networks in a folder with LUNA PLATFORM installed.
- Assign appropriate rights to neural network files.
- Mount neural network files to the
/srv/fsdk/data
folder of the Remote SDK container. - Using the arguments of the variable "EXTEND_CMD" to explicitly specify which of the neural networks should be used.
Using the "enable-all-estimators-by-default" flag for the "EXTEND_CMD" variable, you can disable the use of all neural networks (estimators) by default, and then use special flags to explicitly specify which neural networks should be used. If you do not specify this flag or set the value "--enable-all-estimators-by-default=1", the Remote SDK service will try to find all neural networks in the container. If one of the neural networks is not found, the Remote SDK service will not start.
List of available estimators:
Argument | Description |
---|---|
--enable-all-estimators-by-default | Enable all estimators by default. |
--enable-human-detector | Simultaneous detector of bodies and bodies. |
--enable-face-detector | Face detector. |
--enable-body-detector | Body detector. |
--enable-face-landmarks5-estimator | Face landmarks5 estimator. |
--enable-face-landmarks68-estimator | Face landmarks68 estimator. |
--enable-head-pose-estimator | Head pose estimator. |
--enable-liveness-estimator | Liveness estimator. |
--enable-fisheye-estimator | FishEye effect estimator. |
--enable-face-detection-background-estimator | Image background estimator. |
--enable-face-warp-estimator | Face sample estimator. |
--enable-body-warp-estimator | Body sample estimator. |
--enable-quality-estimator | Image quality estimator. |
--enable-image-color-type-estimator | Face color type estimator. |
--enable-face-natural-light-estimator | Natural light estimator. |
--enable-eyes-estimator | Eyes estimator. |
--enable-gaze-estimator | Gaze estimator. |
--enable-mouth-attributes-estimator | Mouth attributes estimator. |
--enable-emotions-estimator | Emotions estimator. |
--enable-mask-estimator | Mask estimator. |
--enable-glasses-estimator | Glasses estimator. |
--enable-eyebrow-expression-estimator | Eyebrow estimator. |
--enable-red-eyes-estimator | Red eyes estimator. |
--enable-headwear-estimator | Headwear estimator. |
--enable-basic-attributes-estimator | Basic attributes estimator. |
--enable-face-descriptor-estimator | Face descriptor extraction estimator. |
--enable-body-descriptor-estimator | Body descriptor extraction estimator. |
--enable-body-attributes-estimator | Body attributes estimator. |
--enable-people-count-estimator | People count estimator. |
--enable-deepfake-estimator | Deepfake estimator. |
See the detailed information on enabling and disabling certain estimators in the section "Enable/disable several estimators and detectors " of the administrator manual.
Below is an example of a command to assign rights to a neural network file:
chown -R 1001:0 /var/lib/luna/current/<neural_network_name>.plan
Example of a command to run Remote SDK container with mounting neural networks for face detection and face descriptor extraction:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5220 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--env=EXTEND_CMD="--enable-all-estimators-by-default=0 --enable-face-detector=1 --enable-face-descriptor-estimator=1" \
-v /var/lib/luna/current/cnn59b_cpu-avx2.plan:/srv/fsdk/data/cnn59b_cpu-avx2.plan \
-v /var/lib/luna/current/FaceDet_v3_a1_cpu-avx2.plan:/srv/fsdk/data/FaceDet_v3_a1_cpu-avx2.plan \
-v /var/lib/luna/current/FaceDet_v3_redetect_v3_cpu-avx2.plan:/srv/fsdk/data/FaceDet_v3_redetect_v3_cpu-avx2.plan \
-v /var/lib/luna/current/slnet_v3_cpu-avx2.plan:/srv/fsdk/data/slnet_v3_cpu-avx2.plan \
-v /var/lib/luna/current/LNet_precise_v2_cpu-avx2.plan:/srv/fsdk/data/LNet_precise_v2_cpu-avx2.plan \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/remote-sdk:/srv/logs \
--network=host \
--name=luna-remote-sdk \
--restart=always \
--detach=true \
dockerhub.visionlabs.ru/luna/luna-remote-sdk:v.0.6.0
Handlers#
Note: If you are not going to use the Handlers service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".
Handlers container launch#
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5090 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/handlers:/srv/logs \
--name=luna-handlers \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-handlers:v.3.6.0
Tasks#
Note: If you are not going to use the Tasks service, do not launch the Tasks container and the Tasks Worker container. Disable the service utilization in Configurator. See section "Optional services usage".
Tasks and Tasks Worker containers launch#
Tasks service image includes the Tasks service and the Tasks Worker. They both must be launched.
If it is necessary to use the Estimator task using a network disk, then you should first mount the directory with images from the network disk into special directories of Tasks and Tasks Worker containers. See the "Estimator task" section in the administrator manual for details.
Tasks Worker launch#
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5051 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--env=SERVICE_TYPE="tasks_worker" \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/tasks-worker:/srv/logs \
--name=luna-tasks-worker \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-tasks:v.3.22.1
Tasks launch#
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5050 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/tasks:/srv/logs \
--name=luna-tasks \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-tasks:v.3.22.1
Sender#
Sender container launch#
Note: If you are not going to use the Sender service, do not launch this container and disable the service utilization in Configurator. See section "Optional services usage".
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5080 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/sender:/srv/logs \
--name=luna-sender \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-sender:v.2.11.9
API#
API container launch#
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5000 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--name=luna-api \
--restart=always \
--detach=true \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/api:/srv/logs \
--network=host \
dockerhub.visionlabs.ru/luna/luna-api:v.6.29.0
Account creation#
The account is created using an HTTP request to the "create account" resource of the API service.
You can also create an account using the Admin service. This method requires an existing login and password (or the default login and password) and enables you to create an "admin" account. See the "Admin service" section of the administrator manual for details.
To create the account using a request to the API service, you need to provide the following mandatory data:
- "login" — Email address.
- "password" — Password.
- "account_type" — Account type ("user" or "advanced_user").
Create the account using your authentication details.
Example of CURL-request to the "create account" resource:
curl --location --request POST 'http://127.0.0.1:5000/6/accounts' \
--header 'Content-Type: application/json' \
--data '{
"login": "user@mail.com",
"password": "password",
"account_type": "user",
"description": "description"
}'
It is necessary to replace the authentication data from the example with your own.
To work with tokens, you must have an account.
Create GC task schedule#
Before you start working with the LUNA PLATFORM, you can create a schedule for the Garbage collection task.
To do this, make a "create tasks schedule" request to the API service, specifying the necessary rules for the schedule.
An example of a schedule creation command for an account created in section "Account creation" is given below.
The example sets a schedule for the Garbage collection task for events older than 30 days with the removal of the samples and the source images. The task will be repeated once a day at 05:30 am.
curl --location --request POST 'http://127.0.0.1:5000/6/tasks/schedules' \
--header 'Authorization: Basic dXNlckBtYWlsLmNvbTpwYXNzd29yZA==' \
--header 'Content-Type: application/json' \
--data '{
"task": {
"task_type": 4,
"content": {
"target": "events",
"filters": {
"create_time__lt": "now-30d"
},
"remove_samples": true,
"remove_image_origins": true
}
},
"trigger": {"cron": "30 5 * * *", "cron_timezone": "utc"},
"behaviour": {"start_immediately": false, "create_stopped": false}
}'
If necessary, you can create a schedule without automatically activating it. To do this, specify the parameter "create_stopped": "true". In this case, after creating the schedule, it must be activated manually using the "action" = "start" parameter of the "patch tasks schedule" request.
For more information, see the "Running scheduled tasks" section of the administrator manual.
Admin#
Admin container launch#
Note: If you are not going to use the Admin service, do not launch this container.
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5010 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/admin:/srv/logs \
--name=luna-admin \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-admin:v.5.7.1
Backport 3#
The section describes launching of Backport 3 service.
The service is not mandatory for utilizing LP5 and is required for emulation of LP 3 API only.
Backport 3 container launch#
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5140 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--name=luna-backport3 \
--restart=always \
--detach=true \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/backport3:/srv/logs \
--network=host \
dockerhub.visionlabs.ru/luna/luna-backport3:v.0.11.9
User Interface 3#
The User Interface 3 is used with the Backport 3 service only.
User Interface 3 container launch#
Use the following command to launch the service:
docker run \
--env=PORT=4100 \
--env=LUNA_API_URL=http://127.0.0.1:5140 \
--name=luna-ui-3 \
--restart=always \
--detach=true \
--network=host \
-v /etc/localtime:/etc/localtime:ro \
dockerhub.visionlabs.ru/luna/luna3-ui:v.0.5.13
Here:
-
--env=LUNA_API_URL
— Specifies the URL of the Backport 3 service. -
--env=PORT
— Specifies the port of the User Interface 3 service.
Backport 4#
The section describes launching of Backport 4 service.
The service is not mandatory for utilizing LP5 and is required for emulation of LP 4 API only.
Backport 4 container launch#
Use the following command to launch the service:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5130 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--name=luna-backport4 \
--restart=always \
--detach=true \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/backport4:/srv/logs \
--network=host \
dockerhub.visionlabs.ru/luna/luna-backport4:v.1.5.9
User Interface 4#
The User Interface 4 is used with the Backport 4 service only.
User Interface 4 container launch#
Note: You should have the account with account type user before launching the User Interface 4 container. Its login and password in Base64 format will be used to work with the user interface.
Use the following command to launch the service:
docker run \
--env=PORT=4200 \
--env=LUNA_API_URL=http://<server_external_ip>:5130 \
--env=BASIC_AUTH=dXNlckBtYWlsLmNvbTpwYXNzd29yZA== \
--name=luna-ui-4 \
--restart=always \
--detach=true \
--network=host \
-v /etc/localtime:/etc/localtime:ro \
dockerhub.visionlabs.ru/luna/luna4-ui:v.0.1.8
Here:
-
--env=PORT
— Sets the port for running User Interface 4. -
--env=BASIC_AUTH
— Sets the Basic authorization for the account which data is displayed in the user interface. It is necessary to convertlogin:password
created in the section "Account creation" to Base64 format. The account type should be set to user. -
--env=LUNA_API_URL
— Sets the URL of the Backport 4 service.
You should use the external IP of the service, not localhost.
You should specify the Backport 4 service port (5130 is set by default).
Additional information#
This section provides the following additional information:
- Monitoring and logs visualization using Grafana.
- Useful commands for working with Docker.
- Description of the parameters for launching LUNA PLATFORM services and creating databases.
- Ways to change service settings.
- Actions to enable saving LP service logs to files.
- Configuring Docker log rotation.
- Setting custom InfluxDB settings.
- Using Python Matcher service with Python Matcher Proxy service.
- System scaling.
- Improve performance of services in Docker containers.
- Manual creation of databases when using an external PostgreSQL database (including creation of the VLMatch function for the Faces and Events database).
- Compiling the VLMatch library for Oracle.
Monitoring and logs visualization using Grafana#
Monitoring visualization is performed by the LUNA Dashboards service, which contains the Grafana monitoring data visualization platform with configured LUNA PLATFORM dashboards.
If necessary, you can install customized dashboards for Grafana separately. See the "LUNA Dashboards" section in the administrator manual for more information.
Together with Grafana, you can use the Grafana Loki log aggregation system, which enables you to flexibly work with LUNA PLATFORM logs. The Promtail agent is used to deliver LUNA PLATFORM logs to Grafana Loki (for more information, see the "Grafana Loki" section in the administrator manual).
LUNA Dashboards#
Note: To work with Grafana you need to use InfluxDB version 2.
Run LUNA Dashboards container#
Use the docker run
command with these parameters to run Grafana:
docker run \
--restart=always \
--detach=true \
--network=host \
--name=grafana \
-v /etc/localtime:/etc/localtime:ro \
dockerhub.visionlabs.ru/luna/luna-dashboards:v.0.1.0
Use "http://IP_ADDRESS:3000" to go to the Grafana web interface when the LUNA Dashboards and InfluxDB containers are running.
Grafana Loki#
Note: Grafana Loki requires LUNA Dashboards to be running.
Run Grafana Loki container#
Use the docker run
command with these parameters to run Grafana Loki:
docker run \
--name=loki \
--restart=always \
--detach=true \
--network=host \
-v /etc/localtime:/etc/localtime:ro \
dockerhub.visionlabs.ru/luna/loki:2.7.1
Run Promtail container#
Use the docker run
command with these parameters to run Promtail:
docker run \
-v /var/lib/luna/current/example-docker/logging/promtail.yml:/etc/promtail/luna.yml \
-v /var/lib/docker/containers:/var/lib/docker/containers \
-v /etc/localtime:/etc/localtime:ro \
--name=promtail \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/promtail:2.7.1 \
-config.file=/etc/promtail/luna.yml -client.url=http://127.0.0.1:3100/loki/api/v1/push -client.external-labels=job=containerlogs,pipeline_id=,job_id=,version=
Here:
-
-v /var/lib/luna/current/example-docker/logging/promtail.yml:/etc/promtail/luna.yml
— Mounting the configuration file to the Promtail container. -
-config.file=/etc/promtail/luna.yml
— Flag with the address of the configuration file. -
-client.url=http://127.0.0.1:3100/loki/api/v1/push
— Flag with the address of deployed Grafana Loki. -
-client.external-labels=job=containerlogs,pipeline_id=,job_id=,version=
— Static labels to add to all logs sent to Grafana Loki.
Docker commands#
Show containers#
To show the list of launched Docker containers use the command:
docker ps
To show all the existing Docker containers use the command:
docker ps -a
Copy files to container#
You can transfer files into the container. Use the docker cp
command to copy a file into the container.
docker cp <file_location> <container_name>:<folder_inside_container>
Enter container#
You can enter individual containers using the following command:
docker exec -it <container_name> bash
To exit the container, use the command:
exit
Images names#
You can see all the names of the images using the command:
docker images
Delete image#
If you need to delete an image:
- Run the
docker images
command. - Find the required image, for example dockerhub.visionlabs.ru/luna/luna-image-store.
- Copy the corresponding image ID from the IMAGE ID, for example, "61860d036d8c".
- Specify it in the deletion command:
docker rmi -f 61860d036d8c
Delete all the existing images.
docker rmi -f $(docker images -q)
Stop container#
You can stop the container using the command:
docker stop <container_name>
Stop all the containers:
docker stop $(docker ps -a -q)
Delete container#
If you need to delete a container:
- Run the "docker ps" command.
- Stop the container (see Stop container).
- Find the required image, for example dockerhub.visionlabs.ru/luna/luna-image-store.
- Copy the corresponding container ID from the CONTAINER ID column, for example, "23f555be8f3a".
- Specify it in the deletion command:
docker container rm -f 23f555be8f3a
Delete all the containers.
docker container rm -f $(docker container ls -aq)
Check service logs#
You can use the following command to show logs for the service:
docker logs <container_name>
Launching parameters description#
When launching a Docker container for a LUNA PLATFORM service you should specify additional parameters required for the service launching.
The parameters specific for a particular container are described in the section about this container launching.
All the parameters given in the service launching example are required for proper service launching and utilization.
Launching services parameters#
Example command of launching LP services containers:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=<Port_of_the_launched_service> \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/<service>:/srv/logs/ \
--name=<service_container_name> \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/<service-name>:<version>
The following parameters are used when launching LP services containers:
-
docker run
— Command for running the selected image as a new container. -
dockerhub.visionlabs.ru/luna/<service-name>:<version>
— Sets the image required for the container launching.
Links to download the container images you need are available in the description of the corresponding container launching.
-
--network=host
— Sets that a network is not simulated and the server network is used. If you need to change the port for third-party party containers, you should change this string to-p 5440:5432
. Where the first port5440
is the local port and5432
is the port used inside the container. The example is given for PostgreSQL. -
--env=
— Sets the environment variables required to run the container (see the "Service arguments" section). -
--name=<service_container_name>
— Sets the name of the launched container. The name must be unique. If there is a container with the same name, an error will occur. -
--restart=always
— Sets a restart policy. The daemon will always restart the container regardless of the exit status. -
--detach=true
— Run the container in the background mode. -
-v
— Enables you to mount the content of a server folder into a volume in the container. Thus their contents will synchronize. The following general data is mounted: -
/etc/localtime:/etc/localtime:ro
— Sets the current time zone used by the system in the container. -
/tmp/logs/<service>:/srv/logs/
— Enables copying of the folder with service logs to your server/tmp/logs/<service>
directory. You can change the directory where the logs will be saved according to your needs.
Service arguments#
Each service in LUNA PLATFORM has its own launch arguments. These arguments can be passed through:
- Setting a flag for the launch script (
run.py
) of the corresponding service. - Setting environment variables (
--env
) on the Docker command line.
For example, using the --help
flag you can get a list of all available arguments. An example of passing an argument to an API service:
docker run --rm dockerhub.visionlabs.ru/luna/luna-api:v.6.29.0 python3 /srv/luna_api/run.py --help
List of main arguments:
Launch flag |
Environment variable |
Description |
|
|
Port on which the service will listen for connections. |
|
|
Number of workers for the service. |
|
|
Suffix added to log file names (with the option to write logs to a file enabled). |
|
|
Enable automatic configuration reload. See "Automatic configurations reload" in the LUNA PLATFORM 5 administrator manual. |
|
|
Configuration checking period (default 10 seconds). See "Automatic configurations reload" in the LUNA PLATFORM 5 administrator manual. |
|
|
Address of the Configurator service for downloading settings.
For |
|
None |
Path to the file with service configurations. |
|
|
Tag of the specified configuration in the Configurator.
When setting this configuration, the value of the tagged
configuration will be used. Example: Note: You must pre-tag the appropriate configuration in. Configurator. Note: Only works with the |
|
None |
Path to the SSL certificate for launching the service using the HTTPS protocol. |
|
None |
Path to the SSL private key for launching the service using the HTTPS protocol. |
|
None |
Password for the SSL private key for launching the service using the HTTPS protocol. |
The list of arguments may vary depending on the service.
It is also possible to override the settings of services at their start using environment variables.
The VL_SETTINGS
prefix is used to redefine the settings. Examples:
--env=VL_SETTINGS.INFLUX_MONITORING.SEND_DATA_FOR_MONITORING=0
. Using the environment variable from this example will set the "SEND_DATA_FOR_MONITORING" setting for theINFLUX_MONITORING
section to "0".--env=VL_SETTINGS.OTHER.STORAGE_TIME=LOCAL
. For non-compound settings (settings that are located in the "OTHER" section in the configuration file), you must specify the "OTHER" prefix. Using the environment variable from this example will set the value of the "STORAGE_TIME" setting (if the service uses this setting) to "LOCAL".
Passing flags using environment variable
Flags for which an environment variable is not explicitly allocated can be passed using the environment variable EXTEND_CMD
.
For example, you can pass the configurations tag in the following way:
--env=EXTEND_CMD="--INFLUX_MONITORING=TAG_1 --LUNA_EVENTS_DB=TAG_2"
Creating DB parameters#
Example command of launching containers for database migration or database creation:
docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/<service>:/srv/logs/ \
--rm \
--network=host \
dockerhub.visionlabs.ru/luna/<service-name>:<version> \
python3 ./base_scripts/db_create.py --luna-config http://localhost:5070/1
The following parameters are used when launching containers for database migration or database creation:
Here:
-
--rm
— Sets if the container is deleted after all the specified scripts finish processing. -
python3 ./base_scripts/db_create.py
— Sets Python version and a scriptdb_create.py
launched in the container. The script is used for the database structure creation. -
--luna-config http://localhost:5070/1
— Sets where the launched script should receive configurations. By default, the service requests configurations from the Configurator service.
Ways to change service settings#
There are three general ways to change settings stored in the Configurator DB:
- Using Configurator GUI.
- Using Configurator API requests.
- Using dump file.
Settings can be changed after the Configurator service was launched.
The settings of Configurator itself are stored in the "luna_configurator_postgres.conf" file that is downloaded to the Configurator container during its launch.
Configurator Interface
You can enter the configurator GUI and change setting. By default, the following address on localhost are used: <Configurator_server_address>:5070
.
Configurator API
You can use Configurator API to update settings. See the Configurator OpenAPI specification.
Dump file
You can receive a dump file with all the LP services settings. Use one of the following commands:
wget -O /var/lib/luna/settings_dump.json 127.0.0.1:5070/1/dump
or
curl 127.0.0.1:5070/1/dump > /var/lib/luna/settings_dump.json
You should specify correct Configurator service address and port.
You should delete the "limitations" section from the file. You will not be able to apply your updated dump file if the section remains.
"limitations":[
...
],
Edit parameters in the "settings" section.
"settings":[
...
],
Copy the dump file to your Configurator container.
docker cp /var/lib/luna/settings_dump.json luna-configurator:/srv/
Apply the file:
docker exec luna-configurator python3 ./base_scripts/db_create.py --dump-file /srv/settings_dump.json
Logging to server#
To enable saving logs to the server, you should:
- Create directories for logs on the server.
- Activate log recording and set the location of log storage inside LP service containers.
- Configure synchronization of log directories in the container with logs on the server using the
volume
argument at the start of each container.
Create logs directory#
Below are examples of commands for creating directories for saving logs and assigning rights to them for all LUNA PLATFORM services.
mkdir -p /tmp/logs/configurator /tmp/logs/image-store /tmp/logs/accounts /tmp/logs/faces /tmp/logs/licenses /tmp/logs/events /tmp/logs/python-matcher /tmp/logs/handlers /tmp/logs/remote-sdk /tmp/logs/tasks /tmp/logs/tasks-worker /tmp/logs/sender /tmp/logs/api /tmp/logs/admin /tmp/logs/backport3 /tmp/logs/backport4
chown -R 1001:0 /tmp/logs/configurator /tmp/logs/image-store /tmp/logs/accounts /tmp/logs/faces /tmp/logs/licenses /tmp/logs/events /tmp/logs/python-matcher /tmp/logs/handlers /tmp/logs/remote-sdk /tmp/logs/tasks /tmp/logs/tasks-worker /tmp/logs/sender /tmp/logs/api /tmp/logs/admin /tmp/logs/backport3 /tmp/logs/backport4
If you need to use the Python Matcher Proxy service, then you need to additionally create the /tmp/logs/python-matcher-proxy
directory and set its permissions.
Logging activation#
LP services logging activation#
To enable logging to file, you need to set the log_to_file
and folder_with_logs
settings in the <SERVICE_NAME>_LOGGER
section of the settings for each service.
Automatic method (before/after starting Configurator)
To update logging settings, you can use the logging.json
settings file provided with the distribution package.
Run the following command after starting the Configurator service:
docker cp /var/lib/luna/current/extras/conf/logging.json luna-configurator:/srv/luna_configurator/used_dumps/logging.json
Update your logging settings with the copied file.
docker exec -it luna-configurator python3 ./base_scripts/db_create.py --dump-file /srv/luna_configurator/used_dumps/logging.json
Manual method (after starting Configurator)
Go to the Configurator service interface (127.0.0.1:5070
) and set the logs path in the container in the folder_with_logs
parameter for all services whose logs need to be saved. For example, you can use the path /srv/logs
.
Set the log_to_file
option to true
to enable logging to a file.
Configurator service logging activation (before/after Configurator start)#
The Configurator service settings are not located in the Configurator user interface, they are located in the following file:
/var/lib/luna/current/example-docker/luna_configurator/configs/luna_configurator_postgres.conf
You should change the logging parameters in this file before starting the Configurator service or restart it after making changes.
Set the path to the logs location in the container in the FOLDER_WITH_LOGS = ./
parameter of the file. For example, FOLDER_WITH_LOGS = /srv/logs
.
Set the log_to_file
option to true
to enable logging to a file.
Mounting directories with logs when starting services#
The log directory is mounted with the following argument when starting the container:
-v <server_logs_folder>:<container_logs_folder> \
where <server_logs_folder>
is the directory created in the create logs directory step, and <container_logs_folder>
is the directory created in the activate logging step.
Example of command to launch the API service with mounting a directory with logs:
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5000 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--name=luna-api \
--restart=always \
--detach=true \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/api:/srv/logs \
--network=host \
dockerhub.visionlabs.ru/luna/luna-api:v.6.29.0
The example container launch commands in this documentation contain these arguments.
Docker log rotation#
To limit the size of logs generated by Docker, you can set up automatic log rotation. To do this, add the following data to the /etc/docker/daemon.json
file:
{
"log-driver": "json-file",
"log-opts": {
"max-size": "100m",
"max-file": "5"
}
}
This will allow Docker to store up to 5 log files per container, with each file being limited to 100MB.
After changing the file, you need to restart Docker:
systemctl reload docker
The above changes are the default for any newly created container, they do not apply to already created containers.
Set custom InfluxDB settings#
If you are going to use InfluxDB OSS 2, then you need to update the monitoring settings in Configurator service.
There are the following settings for InfluxDB OSS 2:
"send_data_for_monitoring": 1,
"use_ssl": 0,
"flushing_period": 1,
"host": "127.0.0.1",
"port": 8086,
"organization": "<ORGANIZATION_NAME>",
"token": "<TOKEN>",
"bucket": "<BUCKET_NAME>",
"version": <DB_VERSION>
You can update InfluxDB settings in the Configurator service by following these steps:
- Open the following file:
vi /var/lib/luna/current/extras/conf/influx2.json
- Set required data.
- Save changes.
- Copy the file to the influxDB container:
docker cp /var/lib/luna/current/extras/conf/influx2.json luna-configurator:/srv/
- Update settings in the Configurator.
docker exec -it luna-configurator python3 ./base_scripts/db_create.py --dump-file /srv/influx2.json
You can also manually update settings in the Configurator service user interface.
The Configurator service configurations are set separately.
- Open the file with the Configurator configurations:
vi /var/lib/luna/current/example-docker/luna_configurator/configs/luna_configurator_postgres.conf
- Set required data.
- Save changes.
- Restart Configurator:
docker restart luna-configurator
Use Python Matcher with Python Matcher Proxy#
As mentioned earlier, along with the Python Matcher service, you can additionally use the Python Matcher Proxy service, which will redirect matching requests either to the Python Matcher service or to the matching plugins. Plugins may significantly improve matching processing performance. For example, it is possible to organize the storage of the data required for matching operations and additional objects fields in separate storage using plugins, which will speed up access to the data compared to the use of the standard LUNA PLATFORM database.
To use the Python Matcher service with Python Matcher Proxy, you should additionally launch the appropriate container, and then set a certain setting in the Configurator service. Follow the steps below only if you are going to use matching plugins.
See the description and usage of matching plugins in the administrator manual.
Python Matcher proxy container launch#
Use the following command to launch the service:
After starting the container, you need to set the
"luna_matcher_proxy":true
parameter in the "ADDITIONAL_SERVICES_USAGE" section in the Configurator service.
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5110 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
--env=SERVICE_TYPE="proxy" \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/python-matcher-proxy:/srv/logs \
--name=luna-python-matcher-proxy \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-python-matcher:v.1.9.9
After launching the container, you need to set the following value in the Configurator service.
ADDITIONAL_SERVICES_USAGE = "luna_matcher_proxy":true
System scaling#
All LP services are linearly scalable and can be located on several services.
You can run additional containers with LP services to improve performance and fail-safety. The number of services and the characteristics of servers depend on your tasks.
To increase performance, you may either improve the performance of a single server or increase the number of servers used by distributing most resource-intensive components of the system.
Balancers are used for the distribution of requests among the launched service instances. This approach provides the necessary processing speed and the required fail-safety level for specific customer's tasks. In the case of a node failure, the system will not stop: requests will be redirected to another node.
The image below shows two instances of the Faces service balanced by Nginx. Nginx receives requests on port 5030 and routes them to Faces instances. The faces services are launched on ports 5031 and 5032.
It is strongly recommended to regularly back up databases to a separate server regardless of the fail-safety level of the system. It allows you not to lose data in case of unforeseen circumstances.
MQs, databases, and balancers used by LUNA PLATFORM are products of third-party developers. You should configure them according to the recommendations of the corresponding vendors.
The Remote SDK service and the Python Matcher service perform the most resource-intensive operations.
The Remote SDK service performs mathematical image transformations and descriptors extraction. The operations require significant computational resources. Both CPU and GPU can be used for computations.
GPU usage is preferable since it improves the processing of requests. However, not all types of video cards are supported.
The Python Matcher service performs matching with lists. Matching requires CPU resources, however, you also should allocate as much RAM as possible for each Python Matcher instance. The RAM is used to store descriptors received from a database. Thus matcher does not require to request each descriptor from the database.
When distributing instances on several servers, you should consider the performance of each server. For example, if a large task is executed by several Python Matcher instances, and one of the instances is on the server with low performance, this can slow down the execution of the entire task.
For each instance of the service, you can set the number of workers. The greater the number of workers, the more resources and memory are consumed by the service instance. See the detailed information in the "Worker processes" section of the LUNA PLATFORM administrator manual.
Nginx#
Nginx is required when you use several instances of LUNA PLATFORM services.
Nginx configuration#
The Nginx configuration file includes parameters for balancing of requests to API, Faces, Image Store and Events.
Check the "/var/lib/luna/current/extras/conf/nginx.conf" configuration file before running NGINX.
Nginx listens requests on the specified ports and sends the requests to the available instances of services. The following servers and ports are set in Nginx configuration file.
Service name | Port set in Nginx | Ports for the launched services |
---|---|---|
API | 5000 | 5001-5004 |
Faces | 5030 | 5031-5034 |
Image Store | 5020 | 5021-5024 |
Events | 5040 | 5041-5044 |
Run Nginx container#
Note: You should configure the file for using with your launched services.
Use the docker run
command with these parameters to launch the Nginx container:
docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/luna/current/extras/conf/nginx.conf:/etc/nginx/nginx.conf \
--name=nginx \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/nginx:1.17.4-alpine
Here:
-v /var/lib/luna/current/extras/conf/nginx.conf:/etc/nginx/nginx.conf
— Nginx configuration file used for Nginx launching.
See the example of Nginx configuration file for more services instances launch here: "/var/lib/luna/current/extras/conf/nginx.conf"
Launching several containers#
There are two steps required for launching several instances of the same LP service
- Run several containers of the service.
You must launch the required number of service by using the corresponding command for the service.
For example, for the API service you must run the following command with updated parameters.
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=<port> \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/<folder_name>:/srv/logs \
--name=<name> \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-api:v.6.29.0
When running several similar containers the following parameters of the containers must differ:
-
--env=PORT=<port>
— Specified port for similar containers must differ. You must specify an available port for the instance. For example, "5001", "5002". The "5000" port will be specified for the Nginx balancer. -
/tmp/logs/<folder_name>:/srv/logs
— Specified folder name for logs must differ to distinguish logs for different service instances. -
--name=<container_name>
— Name of the launched container must differ as it is prohibited to launch two containers with the same name. For example, "api_1", "api_2". -
--gpus device=0
— CORE services usually utilize different GPU devices. Thus you should specify different device numbers. -
Configure your balancer (e.g., Nginx) for routing requests to the services.
For each scaled LP service, you must set a port where Nginx will listen to service requests and real ports of each service instance where Nginx will redirect the requests.
An example of Nginx configuration file can be found here:
“/var/lib/luna/current/extras/conf/nginx.conf”.
You can use another balancer, but its utilization is not described in this documentation.
Improve performance of services in Docker containers#
Follow these steps if you need to improve the performance of LUNA PLATFORM in Docker containers.
Docker security policies impose restrictions on the network interaction of LP services. If you perform the listed below actions and disable the policies, a new network connection of docker containers will be established approximately 20% faster.
Changing the settings may have a negative impact on security. Do not change the settings if are not an advanced Docker user.
Docker must be installed before performing these actions. See "Docker installation".
- Add
--network=host --security-opt seccomp=unconfined
parameters to the launch of each LP service container. The launch commands are described in section "Services launch".
The example below is given for the Faces service.
docker run --env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=5031 \
--env=WORKER_COUNT=1 \
--env=RELOAD_CONFIG=1 \
--env=RELOAD_CONFIG_INTERVAL=10 \
-v /etc/localtime:/etc/localtime:ro \
--name=luna-faces1 \
--restart=always \
--detach=true \
--network=host \
--security-opt seccomp=unconfined \
dockerhub.visionlabs.ru/luna/luna-faces:v.4.11.9
- Add
"userland-proxy": false
to the Docker configuration.
Create configuration file for Docker.
echo 'DOCKER_OPTS="--config-file=/etc/docker/daemon.json"' > /etc/default/docker
Create the file.
vi /etc/docker/daemon.json
Add the following option to the opened file.
{
"userland-proxy": false
}
Restart Docker.
systemctl restart docker
External PostgreSQL configuration#
This section describes commands required for configuring external PostgreSQL for working with LP. External means that you already have working DB and want to use it with LP.
You need to specify your external DB in the configurations of LP services.
Skip this section if you are using PostgreSQL from a Docker container supplied by VisionLabs.
You database user and other parameters may differ. Consider it during the commands execution.
PostgreSQL user creation#
Create a database user.
runuser -u postgres -- psql -c 'create role luna;'
Assign a password to the user.
runuser -u postgres -- psql -c "ALTER USER luna WITH PASSWORD 'luna';"
Configurator DB creation#
It is assumed that the DB user is already created.
Create the database for the Configurator service.
runuser -u postgres -- psql -c 'CREATE DATABASE luna_configurator;'
Grant privileges to the database user.
runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_configurator TO luna;'
Allow user to authorize in the DB.
runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'
Accounts DB creation#
It is assumed that the DB user is already created.
Create the database for the Accounts service.
runuser -u postgres -- psql -c 'CREATE DATABASE luna_accounts;'
Grant privileges to the database user.
runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_accounts TO luna;'
Allow user to authorize in the DB.
runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'
Handlers DB creation#
It is assumed that the DB user is already created.
Create the database for the Handlers service.
runuser -u postgres -- psql -c 'CREATE DATABASE luna_handlers;'
Grant privileges to the database user.
runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_handlers TO luna;'
Allow user to authorize in the DB.
runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'
Backport 3 DB creation#
It is assumed that the DB user is already created.
Create the database for the Backport 3 service.
runuser -u postgres -- psql -c 'CREATE DATABASE luna_backport3;'
Grant privileges to the database user.
runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_backport3 TO luna;'
Allow user to authorize in the DB.
runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'
Faces DB creation#
It is assumed that the DB user is already created.
Create the database for the Faces service.
runuser -u postgres -- psql -c 'CREATE DATABASE luna_faces;'
Grant privileges to the database user.
runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_faces TO luna;'
Allow user to authorize in the DB.
runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'
Events DB creation#
It is assumed that the DB user is already created.
Create the database for the Events service.
runuser -u postgres -- psql -c 'CREATE DATABASE luna_events;'
Grant privileges to the database user.
runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_events TO luna;'
Allow user to authorize in the DB.
runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'
Tasks DB creation#
It is assumed that the DB user is already created.
Create the database for the Tasks service.
runuser -u postgres -- psql -c 'CREATE DATABASE luna_tasks;'
Grant privileges to the database user.
runuser -u postgres -- psql -c 'GRANT ALL PRIVILEGES ON DATABASE luna_tasks TO luna;'
Allow user to authorize in the DB.
runuser -u postgres -- psql -c 'ALTER ROLE luna WITH LOGIN;'
Build VLMatch for PostgreSQL#
Note: The following instruction describes installation for PostgreSQL 16.
You can find all the required files for the VLMatch user-defined extension (UDx) compilation in the following directory:
/var/lib/luna/current/extras/VLMatch/postgres/
The following instruction describes installation for PostgreSQL 16.
For VLMatch UDx function compilation one needs to:
- Install RPM repository:
dnf install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-8-x86_64/pgdg-redhat-repo-latest.noarch.rpm
- Install PostgreSQL:
dnf install postgresql16-server
- Install the development environment:
dnf install postgresql16-devel
- Install the gcc package:
dnf install gcc-c++
-
Install CMAKE. The version 3.5 or higher is required.
-
Open the make.sh script using a text editor. It includes paths to the currently used PostgreSQL version. Change the following values (if necessary):
SDK_HOME
specifies the path to PostgreSQL home directory. Default value: /usr/include/postgresql/16/server
;
LIB_ROOT
specifies the path to PostgreSQL library root directory. Default value: /usr/lib/postgresql/16/lib
.
- Go to the make.sh script directory and run it:
cd /var/lib/luna/current/extras/VLMatch/postgres/
chmod +x make.sh
./make.sh
Add VLMatch function for Faces DB in PostgreSQL#
The Faces service requires an additional VLMatch function to be added to the database used. LUNA PLATFORM cannot perform descriptor matching calculations without this function.
The VLMatch library is compiled for a specific version of the database.
Do not use a library created for a different version of the database. For example, a library created for PostgreSQL version 12 cannot be used for PostgreSQL version 16.
This section describes how to create a function for PostgreSQL. The VLMatch library must be compiled and ported to PostgreSQL. See "Build VLMatch for PostgreSQL".
Add VLMatch function to Faces database#
The VLMatch function should be applied to the PostgreSQL DB.
- Define the function inside the Faces database.
sudo -u postgres -h 127.0.0.1 -- psql -d luna_faces -c "CREATE FUNCTION VLMatch(bytea, bytea, int) RETURNS float8 AS 'VLMatchSource.so', 'VLMatch' LANGUAGE C PARALLEL SAFE;"
- Test function by sending re following request to the service database.
sudo -u postgres -h 127.0.0.1 -- psql -d luna_faces -c "SELECT VLMatch('\x1234567890123456789012345678901234567890123456789012345678901234'::bytea, '\x0123456789012345678901234567890123456789012345678901234567890123'::bytea, 32);"
The result returned by the database must be "0.4765625".
Add VLMatch function for Events DB in PostgreSQL#
The Events service requires an additional VLMatch function to be added to the database used. LUNA PLATFORM cannot perform descriptor matching calculations without this function.
The VLMatch library is compiled for a specific version of the database.
Do not use a library created for a different version of the database. For example, a library created for PostgreSQL version 12 cannot be used for PostgreSQL version 16.
This section describes how to create a function for PostgreSQL. The VLMatch library must be compiled and ported to PostgreSQL. See "Build VLMatch for PostgreSQL".
Add VLMatch function to Events database#
The VLMatch function should be applied to the PostgreSQL DB.
Define the function inside the Events database.
sudo -u postgres -h 127.0.0.1 -- psql -d luna_events -c "CREATE FUNCTION VLMatch(bytea, bytea, int) RETURNS float8 AS 'VLMatchSource.so', 'VLMatch' LANGUAGE C PARALLEL SAFE;"
Test function within call.
sudo -u postgres -h 127.0.0.1 -- psql -d luna_events -c "SELECT VLMatch('\x1234567890123456789012345678901234567890123456789012345678901234'::bytea, '\x0123456789012345678901234567890123456789012345678901234567890123'::bytea, 32);"
The result returned by the database must be "0.4765625".
PostGIS installation for Events DB#
The Events service requires PostGIS for working with coordinates.
This instruction describes the PostGIS installation for PostgreSQL 16 database. The PostGIS version depends on the PostgreSQL version.
Install epel-release for access to extended package repository.
yum -y install epel-release
Install PostGIS.
yum -y install postgis34_16
Activate PostGIS in your database.
sudo -u postgres -h 127.0.0.1 -- psql -d luna_events -c "CREATE EXTENSION postgis;";
See additional information about PostGIS on its website: https://postgis.net/
VLMatch for Oracle#
Note: The following instruction describes installation for Oracle 21c.
You can find all the required files for the VLMatch user-defined extension (UDx) compilation in the following directory:
/var/lib/luna/current/extras/VLMatch/oracle
For VLMatch UDx function compilation one needs to:
- Install required environment, see requirements:
sudo yum install gcc g++
- Change
SDK_HOME
variable — oracle sdk root (default is$ORACLE_HOME/bin
, check$ORACLE_HOME
environment variable is set) in the makefile:
vi /var/lib/luna/current/extras/VLMatch/oracle/make.sh
- Open the directory and run the file "make.sh ".
cd /var/lib/luna/current/extras/VLMatch/oracle
chmod +x make.sh
./make.sh
- Define the library and the function inside the database (from database console):
CREATE OR REPLACE LIBRARY VLMatchSource AS '$ORACLE_HOME/bin/VLMatchSource.so';
CREATE OR REPLACE FUNCTION VLMatch(descriptorFst IN RAW, descriptorSnd IN RAW, length IN BINARY_INTEGER)
RETURN BINARY_FLOAT
AS
LANGUAGE C
LIBRARY VLMatchSource
NAME "VLMatch"
PARAMETERS (descriptorFst BY REFERENCE, descriptorSnd BY REFERENCE, length UNSIGNED SHORT, RETURN FLOAT);
- Test function within call (from database console):
SELECT VLMatch(HEXTORAW('1234567890123456789012345678901234567890123456789012345678901234'), HEXTORAW('0123456789012345678901234567890123456789012345678901234567890123'), 32) FROM DUAL;
The result returned by the database must be "0.4765625".