Scaling LUNA CARS#
Vertical scaling for CARS_Stream#
LUNA CARS allows you to run multiple instances of CARS_Stream on a single server using Docker. To scale and launch multiple instances, you need to configure the .env-vanilla file, where parameters such as the number of instances and the devices they will run on are specified.
To start multiple instances of CARS_Stream, follow these steps:
1․ Go to the root directory of the distribution:
cd cars-installer_v.*.*.*
2․ Open the .env-vanilla file in a text editor:
nano .env-vanilla
3․ In the STREAMS_LIST section, specify the number of instances and the devices where they will run. For example:
STREAMS_LIST=(cpu gpu0 gpu1) # Launches 3 instances: 1 on CPU, 1 on GPU with index 0, and 1 on GPU with index 1.
4․ Save the changes in .env-vanilla.
Notes: - In the
STREAMS_LISTsection, specify the devices on which CARS_Stream will run. For example,cpufor the processor orgpu0for the GPU with index 0. - To run more instances, simply add the required number of devices to this section.
Attention! GPU support
GPU support is available only for NVIDIA graphics cards with installed drivers and the NVIDIA Container Toolkit. You must install the NVIDIA drivers and the NVIDIA Container Toolkit. See the official NVIDIA installation guide for details.
NVIDIA GPU cards below the Turing architecture are not supported.
Parameters for configuring CARS_Stream#
By default, CARS_Installer can launch up to 4 instances of CARS_Stream. All the necessary additional settings have already been implemented for this number of instances.
| Parameter | Value | Description |
|---|---|---|
| STREAMS_LIST | cpu, gpu{N} | List of devices for running CARS_Stream instances. For example, cpu for processing on the central processor, gpu{N} for running the service on a GPU with index N (GPU numbering starts from 0) |
| STREAM_PORT_START | Port number | The starting port number for the CARS_Stream instance. This port will increase based on the number of instances. Each instance must have a unique port that does not overlap with the ports of other subsystems (e.g., CARS_API). As the number of instances increases, the port number will increment by 1, starting from the specified value |
| STREAM_HASP_PORT_START | HASP port number | The starting port number for connecting to the HASP server. This port must correspond to the additional CARS_Stream instance number and should not overlap with the ports of other subsystems (e.g., CARS_API). The port increases by 1 for each new instance |
Starting a video stream on a specific CARS_Stream instance#
To start a video stream on one of several CARS_Stream instances, specify the address of the desired instance when connecting the camera in the CARS_Analytics UI web interface.
Note!
Video frame width is limited to 4096 px. Exceeding this limit may cause GPU malfunctions.
For more details, see CARS_Analytics User Manual.
Vertical scaling for CARS_API#
CARS_API is a resource-intensive system for processing image requests. To improve performance and scalability, you can increase the number of API instances.
Vertical scaling for CARS_API is achieved by running multiple API instances on the same server. To do this, edit the parameters in the .env-vanilla file that specify the number of CARS_API instances and the devices they will run on.
To increase the number of CARS_API instances, follow these steps:
1․ Go to the root directory of the distribution:
cd cars-installer_v.*.*.*
2․ Open the .env-vanilla file in a text editor:
nano .env-vanilla
3․ In the APIS_LIST section, specify the number of instances and the devices where they will run. For example:
APIS_LIST=(cpu cpu gpu0 gpu0 gpu1) # Launches 2 API instances on CPU, 2 on GPU with index 0, and 1 on GPU with index 1
4․ Save the changes in .env-vanilla.
Note: -
APIS_LISTdefines how many CARS_API instances will be launched and on which devices (for example, CPU or GPU{N}). - Ensure that each instance has unique ports inAPI_PORTand that resources such as CPU and GPU{N} are distributed properly to avoid conflicts.
Attention! GPU support
GPU support is available only for NVIDIA graphics cards with installed drivers and the NVIDIA Container Toolkit. You must install the NVIDIA drivers and the NVIDIA Container Toolkit. See the official NVIDIA installation guide for details.
NVIDIA GPU cards below the Turing architecture are not supported.
Parameters for configuring CARS_API#
| Parameter | Value | Description |
|---|---|---|
| APIS_LIST | cpu, gpu{N} | List of devices for running CARS_API instances. The number of instances depends on the number of specified devices in the list. For example, cpu for processing on the central processor, gpu{N} for running the service on a GPU with index N (GPU numbering starts from 0) |
| API_PORT | Port number | Sets the port number for each additional CARS_API instance. Make sure that each instance has a unique port and corresponds to the number of the additional instance |
| API_HASP_PORT_START | HASP port number | The starting port number for connecting to the HASP server. This port must correspond to the additional CARS_API instance number and should not overlap with the ports of other subsystems (e.g., CARS_Stream). The port increases by 1 for each new instance |
| API_CPU_COUNT | Number of threads | Sets the number of threads processed by a single CARS_API instance. This defines how many threads are allocated to each instance |
| API_CPU_NT | Number of CPU cores | Sets the number of CPU cores used for neural networks (for example, for handling AI-based requests) |