System scaling#
All LP services are linearly scalable and can be located on several services.
You can run additional containers with LP services to improve performance and fail-safety. The number of services and the characteristics of servers depend on your tasks.
To increase performance, you may either improve the performance of a single server or increase the number of servers used by distributing most resource-intensive components of the system.
Balancers are used for the distribution of requests among the launched service instances. This approach provides the necessary processing speed and the required fail-safety level for specific customer's tasks. In the case of a node failure, the system will not stop: requests will be redirected to another node.
The image below shows two instances of the Faces service balanced by Nginx. Nginx receives requests on port 5030 and routes them to Faces instances. The faces services are launched on ports 5031 and 5032.
Note. It is strongly recommended to regularly back up databases to a separate server regardless of the fail-safety level of the system. It allows you not to lose data in case of unforeseen circumstances.
MQs, databases, and balancers used by LUNA PLATFORM are products of third-party developers. You should configure them according to the recommendations of the corresponding vendors.
The Handlers service and the Python Matcher service perform the most resource-intensive operations.
The Handlers service performs mathematical image transformations and descriptors extraction. The operations require significant computational resources. Both CPU and GPU can be used for computations.
Note. GPU usage is preferable since it improves the processing of requests. However, not all types of video cards are supported.
The Python Matcher service performs matching with lists. Matching requires CPU resources, however, you also should allocate as much RAM as possible for each Python Matcher instance. The RAM is used to store descriptors received from a database. Thus matcher does not require to request each descriptor from the database.
Note. When distributing instances on several servers, you should consider the performance of each server. For example, if a large task is executed by several Python Matcher instances, and one of the instances is on the server with low performance, this can slow down the execution of the entire task.
For each instance of the service, you can set the number of workers. The greater the number of workers, the more resources and memory are consumed by the service instance. See the detailed information in the "Worker processes" section of the LUNA PLATFORM administrator manual.
Nginx#
Nginx is required when you use several instances of LUNA PLATFORM services.
Nginx configuration#
The Nginx configuration file includes parameters for balancing of requests to API, Faces, Image Store and Events.
Check the "/var/lib/luna/current/extras/conf/nginx.conf" configuration file before running NGINX.
Nginx listens requests on the specified ports and sends the requests to the available instances of services. The following servers and ports are set in Nginx configuration file.
| Service name | Port set in Nginx | Ports for the launched services |
|---|---|---|
| API | 5000 | 5001-5004 |
| Faces | 5030 | 5031-5034 |
| Image Store | 5020 | 5021-5024 |
| Events | 5040 | 5041-5044 |
Run Nginx container#
Note! You should configure the file for using with your launched services.
Use the docker run command with these parameters to launch the Nginx container:
docker run \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/luna/current/extras/conf/nginx.conf:/etc/nginx/nginx.conf \
--name=nginx \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/nginx:1.17.4-alpine
-v /var/lib/luna/current/extras/conf/nginx.conf:/etc/nginx/nginx.conf - the Nginx configuration file used for Nginx launching.
See the example of Nginx configuration file for more services instances launch here:
/var/lib/luna/current/extras/conf/nginx.conf
Launching several containers#
There are two steps required for launching several instances of the same LP service
- Run several containers of the service
You must launch the required number of service by using the corresponding command for the service.
For example, for the API service you must run the following command with updated parameters.
docker run \
--env=CONFIGURATOR_HOST=127.0.0.1 \
--env=CONFIGURATOR_PORT=5070 \
--env=PORT=<port> \
-v /etc/localtime:/etc/localtime:ro \
-v /tmp/logs/<folder_name>:/srv/logs \
--name=<name> \
--restart=always \
--detach=true \
--network=host \
dockerhub.visionlabs.ru/luna/luna-api:v.6.10.0
When running several similar containers the following parameters of the containers must differ:
--env=PORT=<port> - the specified port for similar containers must differ. You must specify an available port for the instance. For example, "5001", "5002". The "5000" port will be specified for the Nginx balancer.
/tmp/logs/<folder_name>:/srv/logs - the specified folder name for logs must differ to distinguish logs for different service instances.
--name=<container_name> - the name of the launched container must differ as it is prohibited to launch two containers with the same name. For example, "api_1", "api_2".
--gpus device=0 - CORE services usually utilize different GPU devices. Thus you should specify different device numbers.
- Configure your balancer (e.g., Nginx) for routing requests to the services.
For each scaled LP service, you must set a port where Nginx will listen to service requests and real ports of each service instance where Nginx will redirect the requests.
An example of Nginx configuration file can be found here:
“/var/lib/luna/current/extras/conf/nginx.conf”.
You can use another balancer, but its utilization is not described in this documentation.