Skip to content

System scaling#

All LP services are linearly scalable and can be located on several services.

You can run additional containers with LP services to improve performance and fail-safety. The number of services and the characteristics of servers depend on your tasks.

To increase performance, you may either improve the performance of a single server or increase the number of servers used by distributing most resource-intensive components of the system.

Balancers are used for the distribution of requests among the launched service instances. This approach provides the necessary processing speed and the required fail-safety level for specific customer's tasks. In the case of a node failure, the system will not stop: requests will be redirected to another node.

The image below shows two instances of the Faces service balanced by Nginx. Nginx receives requests on port 5030 and routes them to Faces instances. The faces services are launched on ports 5031 and 5032.

Faces service balancing
Faces service balancing

Note. It is strongly recommended to regularly back up databases to a separate server regardless of the fail-safety level of the system. It allows you not to lose data in case of unforeseen circumstances.

MQs, databases, and balancers used by LUNA PLATFORM are products of third-party developers. You should configure them according to the recommendations of the corresponding vendors.

The Handlers service and the Python Matcher service perform the most resource-intensive operations.

The Handlers service performs mathematical image transformations and descriptors extraction. The operations require significant computational resources. Both CPU and GPU can be used for computations.

Note. GPU usage is preferable since it improves the processing of requests. However, not all types of video cards are supported.

The Python Matcher service performs matching with lists. Matching does not require significant CPU resources, however, you should allocate as much RAM as possible for each Matcher worker. The RAM is used to store descriptors received from a database. Thus matcher does not require to request each descriptor from the database.

Note. When distributing workers on several servers, you should consider the performance of each server. For example, if a large task is executed by several Python Matcher workers, and one of the workers is on the server with low performance, this can slow down the execution of the entire task.

Launching several containers#

There are two steps required for launching several instances of the same LP service

  1. Run several containers of the service

You must launch the required number of service by using the corresponding command for the service.

For example, for the API service you must run the following command with updated parameters.

docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=<port> \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/<folder_name>:/srv/logs \
--name=<name> \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-api:v.6.5.0

When running several similar containers the following parameters of the containers must differ:

--env=PORT=<port> - the specified port for similar containers must differ. You must specify an available port for the instance. For example, "5001", "5002". The "5000" port will be specified for the Nginx balancer.

/tmp/logs/<folder_name>:/srv/logs - the specified folder name for logs must differ to distinguish logs for different service instances.

--name=<container_name> - the name of the launched container must differ as it is prohibited to launch two containers with the same name. For example, "api_1", "api_2".

--gpus device=0 - CORE services usually utilize different GPU devices. Thus you should specify different device numbers.

  1. Configure your balancer (e.g., Nginx) for routing requests to the services.

For each scaled LP service, you must set a port where Nginx will listen to service requests and real ports of each service instance where Nginx will redirect the requests.

An example of Nginx configuration file can be found here:

“/var/lib/luna/current/extras/conf/nginx.conf”.

You can use another balancer, but its utilization is not described in this documentation.