Skip to content

CARS_API Performance#

This section presents the test results of CARS_API for different server hardware configurations.

The AVX2 instruction set is required for performing calculations on the processor's capabilities.

It is recommended to install a processor with the following instruction sets: AVX512, VNNI. When these instructions are available, the neural network inference time is significantly reduced, which increases the number of requests (RPS) that CARS API can handle.

Test Server Configuration#

The efficiency of recognition of vehicle or LP attributes depends on the parameters of the input image:

  • Image size;
  • Number of bits per color.

Input data:

  • Vehicle image resolution: 500х439 px;
  • LP image resolution: 117x33 px;
  • Format of the original image: jpeg.

Table 51. Test Server Options

№ Resource Values
1 CPU Model name: Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz, Thread(s) per core: 2, Core(s) per socket: 24
2 CPU frequency 2.40GHz
3 RAM 251Gb
4 Memory 240GB INTEL SSDSC2KB24
5 Video memory 15109MiB
6 Operation system CentOS 8

CARS_API performance measurements are provided for several types of accelerators.

Table 52. Description of accelerator parameters during testing

Name Description
AVX2 Running the CARS_API on a server with a CPU with support for AVX2 instructions
GPU Running the CARS_API on a server with GPU

Attention! GPU support

GPU support is available only for NVIDIA graphics cards with installed drivers and the NVIDIA Container Toolkit. You must install the NVIDIA drivers and the NVIDIA Container Toolkit. See the official NVIDIA installation guide for details.

GPU cards below Turing architecture and above Ada architecture are not supported.

The values presented in this manual are the average values obtained based on the results of at least 100 experiments. All available CPU cores were used in the experiments to determine the attributes of the TS and LPs listed in the table below.

Performance test results#

The testing was carried out using the following classifiers:

  • car_brand_model_v2;
  • vehicle_color;
  • vehicle_type;
  • vehicle_emergency_type;
  • public_transport_type;
  • special_transport_type;
  • grz_all_countries;
  • vehicle_axles;
  • vehicle_descriptor_v2.

/classify Request for classifiers#

Table 53. Test Results for /classify request

Parameter AVX2 GPU
CPU usage (%) 24.03 7.79
CPU memory (Mb) 4713 5528
GPU usage (%) - 94.75
GPU memory (Gb) - 2.26
RPS (transactions/s) 21.09 31.93
Response time (ms) 484 322

/detector request with cars and grz parameters#

Table 54. Test Results for /detector request with cars and grz parameters

Parameter AVX2 GPU
CPU usage (%) 24.2 9.37
CPU memory (Mb) 5430 6218
GPU usage (%) - 95.5
GPU memory (Gb) - 2.62
RPS (transactions/s) 14.27 23.79
CARS Response time (ms) 375 217
GRZ Response time (ms) 309.3 161