LUNA CARS Scaling

LUNA CARS system supports multiple options for its installation and configuration. Both scalability within a single server (vertical scaling) and on multiple servers (horizontal scaling and fault tolerance) are available.

It is possible to scale subsystems separately (for example, CARS Stream, CARS API) or the entire system at once.

This section provides information about alternative ways to install the system for scaling.

LUNA CARS installation on several servers using single database (DB)

LUNA CARS provides the ability to install multiple instances of the system on different servers but using a single database (DB) (Figure 7).

Figure 7. Several servers and single DB installation

In this case, it is necessary to perform a complete installation of the system on one server and installation without using the database on additional servers.

System configuration and installation on a server with DB

Before installing on the server where the database will be located, configure the file «docker-compose.yml» on this server:

1․ Go to the root of the installer:

cd cars-installer_v.*.*.*

2․ Open the file:

nano docker-compose.yml

3․ Uncomment following lines in the file:

   # ports:
   #   - ${POSTGRES_PORT}:${POSTGRES_PORT}

Also configure the file «analytics_instances.csv», which is responsible for configuring the transfer of images from other servers to a single database.

1․ Enter /common directory where the file is located:

cd cars-installer_v.*.*.*/extras/common

2․ Open the file any convenient way and fill in the data about each server where the LUNA CARS will be installed except for the server where the file is being filled in (Table 4):

Table 4. «analytics_instances.csv» fields description

Field	Description	Possible values
id	Server ID. Manually set by the user, does not affect the operation of the system	0, 1, 2,…,n
protocol	Data transfer protocol	HTTP/HTTPS
host	IP-address of the server where the system will be installed	IP-address
port	The port number of the connection to this server	Available ports

After that, perform a standard full installation of the LUNA CARS system on this server (see sections 2.3.1 – 2.3.5).

System configuration and installation on additional servers

To install the system on all additional servers, it is necessary to prepare the distribution kit by configuring the configuration files.

Configuration files are configured for each additional server on the server itself strictly before installing the system.

Configure «docker-compose.yml» file:

1․ Open the file and comment out or delete the following lines from the file related to calling DB container:

psql:
    image: ${DOCKER_REGISTRY}/postgres:12-vl-2
    hostname: ${POSTGRES_HOSTNAME}
    restart: unless-stopped
    environment:
      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_PORT: ${POSTGRES_PORT}
      POSTGRES_HOSTNAME: ${POSTGRES_HOSTNAME}
      POSTGRES_USER: ${POSTGRES_USER}
    # ports:
    #   - ${POSTGRES_PORT}:${POSTGRES_PORT}
    volumes:
      - "${ANALYTICS_PSQL_FOLDER}:/var/lib/postgresql/data/"
      - "./extras/postgres/:/tmp/postgresql/"

2․ Comment out or delete the line with - psql in back-db section that defines the dependency of DB and Redis. Example of commented line:

    depends_on:
      - redis
     # - psql

3․ Save changes and exit the file.

After that the DB container won’t be launched withing LUNA CARS installation on addition server.

In order to tell the system which database to save data to, configure the configuration file «.env»:

1․ Enter the root of installer:

cd cars-installer_v.*.*.*

2․ Open the file:

nano .env

3․ For parameter POSTGRES_HOSTNAME set IP-address of the server where DB is located as a value.

4․ For parameter POSTGRES_PORT set as a value the same port number as set in the «.env» file for the same parameter on the server where DB located.

5․ Save changes and exit the file.

Also configure the file «analytics_instances.csv». Open the file and fill in the data about each server where the LUNA CARS system will be installed, except for the server where the file is being filled in (for more information, see Table 4).

After all the settings, follow the standard algorithm for installing the system on the server.

Repeat all the steps in this paragraph for each additional server.

After all conditions are met and installed on each server, all data will be transferred to a single database from all servers used.

LUNA CARS installation on several servers and data replication settings

LUNA CARS supports the installation on several servers with a possibility of data replication. This can be used for the cases when there is a need to collect and read the data from several (remote) servers on one (main) server.

This type of installation is performed using Ansible.

As with the standard installation using Ansible, it is possible to install using an installation server (see section 2.8 for more details). So, the installation of the system will take place remotely from the installation server to the target and to remote servers (Figure 8).

Figure 8. Installation with replication when using the installation server

The use of the installation server is optional. The target server can act as an installation server.

Installation prerequisites

Before running the installation, make sure the following conditions are met:

• The installation server must have access to all servers – target and remote;
• The target server must have access to all remote servers;
• LUNA CARS distributive must be downloaded and unpacked on the installation server (see section 2.1);
• Ansible must be installed on the installation server (see section 2.8);
• SSH must be configured on the installation server (see section 2.8.2).

LUNA CARS licensing within installation with replication

When installing LUNA CARS on multiple servers with data replication, it is necessary to ensure the licensing of each server.

Activation of the license for each server is similar to the process described in section 2.8.4

Installation and configuration algorithm

Target server installation

After checking all prerequisites are applied, «hosts» file should be configured for the target server:

1․ Provide external IP-address of the server where the components will be installed for each component listed in «hosts» file. Also provide IP-addresses of all servers where CARS API and CARS Stream will be installed.

#CARS.Analytics
#Only 1 host
[frontend]
<IP_address>

#Only 1 host
[backend]
<IP_address>

#Only 1 host
[postgres]
<IP_address>

#Only 1 host
[redis]
<IP_address>

#CARS.API
#Multiple hosts allowed
[api]
<IP_address1>
    <IP_address2>
    <…>
    <IP_address#>

#Only 1 host
[nginx]
<IP_address>

#CARS.Stream
#Multiple hosts allowed
[stream]
<IP_address1>
    <IP_address2>
    <…>
    <IP_address#>

[Analytics_ALL:children]
frontend
backend
postgres
redis

[Api_ALL:children]
api
nginx

2․ After file configuration, run the installation command for the main server:

ansible-playbook -i hosts install_analytics.yml install_api.yml install_stream.yml

Installation on additional servers

1․ Create a copy of a «hosts» file that will be used for the system installation on the first remote server. Create a name for the copy. Provide this server IP-address inside the file for each component. 2․ After file configuration, run the installation command for additional server:

ansible-playbook -i <name of hosts copy file> install_analytics.yml install_api.yml install_stream.yml

3․ To install the system on several additional servers, repeat steps 1-3 for each server.

Replication settings

For data replication to work correctly, PSQL ports must be open on each server. The default port is 5432. If necessary, port changes can be made in the «.env» file.

1․ After system installation on each server, it is required to set the DB connection between the servers. Provide roles in «replica-hosts» file.

File «replica-hosts» is placed in /ansible directory.

Provide following data: Postgres IP-addresses of the servers that should send the data (masters) and the Postgres IP-address of the server that should receive the data (slave). There can be several master-servers and only one slave-server.

[postgres-masters]
IP_address1

[postgres-slave]
IP_address

2․ After file configuration, stay in the /ansible directory and run the following command to start the configuration:

ansible-playbook -i replica-hosts replica_conf.yml

For all additional master servers, repeat steps 1 and 2, specifying the IP address of the next additional master in «replica-hosts».

System installation and replica configuration are done.

To check the system performance, log in to the CARS Analytics interface. The data for logging into the web interface is provided in section 2.8.8.1.

For the System to work correctly, you should run each copy of the System (i.e., each separate port) in a separate browser.

When using LUNA CARS with replication setup, for the correct operation of the system, it is necessary to ensure the uniqueness of the Camera names on each server (for more information, see «CARS Analytics. User Manual»).

Running multiple instances of CARS Stream using Docker

Connecting multiple instances

CARS Installer implements the ability to run multiple instances of CARS Stream on the same server using Docker. The corresponding settings are provided in the file «docker-compose.yml» located in the root folder of the installer.

If you need to run multiple instances of CARS Stream on the same server, you need to run the following commands.

Go to the root of the distributive:

cd cars-installer_v.*.*.*

Open settings file:

nano docker-compose.yml

In the «Stream» section of this file, it is necessary to uncomment the lines for each additional instance (e.g. Stream 1, Stream 2, etc.). An example of commented lines is presented below:

# stream2:
# image: ${DOCKER_REGISTRY}/stream:${STREAM_VERSION}
# hostname: stream2
# #deploy:
# # resources:
# # reservations:
# # devices:
# # - driver: nvidia
# ## count: all
# # device_ids: ['${STREAM_GPU2}']
# # capabilities: [gpu]
# restart: unless-stopped
# ports:
# - ${STREAM_PORT2}:34569
# - ${STREAM_HASP_PORT2}:1947
# environment:
# # NVIDIA_DRIVER_CAPABILITIES: all
# # NVIDIA_VISIBLE_DEVICES: all
# API_PORT: ${API_PORT}
# API_HOSTNAME: ${API_HOSTNAME}
# ANALYTICS_HOSTNAME: ${ANALYTICS_HOSTNAME}
# ANALYTICS_FRONTEND_PORT: ${ANALYTICS_FRONTEND_PORT}
# ANALYTICS_BACKEND_PORT: ${ANALYTICS_BACKEND_PORT}
# HASP_WAIT_TIME: ${HASP_WAIT_TIME}
# HASP_LICENSE_SERVER: ${HASP_LICENSE_SERVER}
# HASP_LICENSE_FILE: ${HASP_LICENSE_FILE}
# DEBUG_STREAM: ${DEBUG_STREAM}
# HEALTH_CHECK_MAX_ERROR_COUNT: ${HEALTH_CHECK_MAX_ERROR_COUNT}
# ENABLE_BODY_DETECTOR: ${ENABLE_BODY_DETECTOR}
# ENABLE_ANIMAL_DETECTOR: ${ENABLE_ANIMAL_DETECTOR}
# STREAM_PLATFORM: ${STREAM_PLATFORM2}
# volumes:
# # Example external video files for carstream
# - "${STREAM_VIDEO_DIR}:${STREAM_VIDEO_DIR}"
# - "${ANALYTICS_MEDIA_ROOT_FOLDER}:/srv/cars/back/media/"
# - "${STREAM_LICENSE_FILE}:/srv/cars/stream/bin/data/license.conf"
# - "${HASP_LICENSE_FILE}:/tmp/test.v2c"

By default, CARS Installer can run up to 4 copies of CARS Stream. All the necessary additional settings have already been implemented for this number of instances. If the number of copies needs to be increased, an additional section «Stream #» (similar to the existing ones) should be added to the «docker-compose file.yml», and also configure the «.env» file.

In the «Stream» section, following parameters should be added:

Parameter	Value	Description
STREAM_PLATFORM#	- cpu;	# must match the number of the additional instance
	- gpu	- gpu – processing requests on the GPU;
		- cpu – processing requests on the CPU without support for AVX2 instructions
STREAM_PORT#	specify the port number for the additional instance	# must match the number of the additional instance
STREAM_GPU#	4…number of GPU instances	# must match the number of the additional instance
STREAM_HASP_PORT#	Available ports	# must match the number of the additional instance. After specifying the port number, it is necessary to check that this port is not used by other subsystems (e.g. CARS API)

Starting a video stream on a specific instance of CARS Stream

To start a video stream on one of several instances, you must manually register the address to a specific copy of CARS Stream when connecting the camera to CARS Analytics UI.

For more information, see CARS Analytics. User Manual.

Improving Image Processing Performance

CARS API is a system for processing the requests for processing images. It is the most resource-intensive subsystem. To process requests a mechanism for setting a task and obtaining a result in a synchronous mode is implemented. To ensure the scale of the CARS API system and its fault tolerance, an external http-request balancer (Nginx) should be used.

Vertical scaling is achieved by running multiple instances of CARS API and using a request balancer (Figure 9).

To connect multiple instances of the CARS API, you must specify in the configuration files the number of CARS APIs used by the copy:

• when installing via docker, you need to make changes to the API_PORT and API_COUNT variables in the «.env» file (see Appendix 2);
• when installing via ansible, it is necessary to make changes to the CARS_API_PORT and CARS_API_PORT_RANGE variables in the «all.yml» file (see Appendix 3).

Figure 9. Vertical scaling of CARS API

Horizontal scaling and fault tolerance is achieved by a similar configuration. To do this, you need to run multiple instances of CARS API on multiple servers.

Balancing and redistribution of requests in case of failure of one of the servers is provided by an external balancer of HTTP requests (Figure 10).

Figure 10. Horizontal scaling of CARS API