Detectors#
The detectors used in the CARS_API subsystem are designed to detect vehicles and license plates (LP) in images, and also highlight the corresponding bounding boxes (BBox).
The configuration of the vehicle and license plate detectors is set in the configuration file
/data/vehicleEngine.conf(for more information, see Appendix 1).The testing of the detectors is available through the CARS_API Tester interface (for more information, see «CARS_Analytics. Administrator's Guide»).
Vehicle detector#
The vehicle detector is designed to detect vehicles in images and also provides information about the vehicle's position in a video stream or video file.
The latest version of the vehicle detector includes a re-detection algorithm for improved accuracy, the ability to work with images captured by cameras installed at significant heights (birdview), and the configuration of additional parameters for the vehicle detector.
An example of an input image is shown in Figure 29.
Response example:
{
"detections": [
{
"x": 637.8615112304688,
"y": 161.5411834716797,
"width": 426.82147216796875,
"height": 274.2376708984375,
"score": 0.9445058107376099
},
{
"x": 115.89066314697266,
"y": 155.4100341796875,
"width": 476.1069030761719,
"height": 282.51666259765625,
"score": 0.9419910907745361
}
],
"detector": "car",
"execution_time": 53
}
Table 38. Description of the vehicle detection fields
| Field name | Type | Description | Possible values |
|---|---|---|---|
| detections | array | An array containing the coordinates and size of the detections of each vehicle in the image, as well as an estimate of the accuracy of vehicle detection | List of detected vehicles |
| x | int | The horizontal coordinate of the upper left corner of the BBox | 0…1920 |
| y | int | The vertical coordinate of the top left corner of the BBox | 0…1080 |
| width | int | Bbox width | 0…1920 |
| height | int | Bbox height | 0…1080 |
| score | float | Estimation of vehicle detection accuracy | 0.0000…1.0000 |
| detector | string | The name of the detector type | car |
| execution_time | int | Execution time in milliseconds | 0…1000 |
LP detector#
The license plate (LP) detector is designed to detect LPs in images and provides information about the position of the LP in a video stream or video file.
The latest version of the LP detector includes the ability to detect multiple plates on a single vehicle, as well as determine the relevant plate related to the correct vehicle. Additionally, the configuration of additional parameters for the LP detector is available.
An example of an input image is shown in Figure 30.
Response example:
{ "detections": [ { "x": 119.0, "y": 483.0, "width": 156.0, "height": 97.0, "score": 0.93145 }, { "x": 112.0, "y": 533.0, "width": 137.0, "height": 88.0, "score": 0.82027 } ], "detector": "grz", "execution_time": 53 }
Table 39. Description of the LP detection fields
| Field name | Type | Description | Possible values |
|---|---|---|---|
| detections | array | An array containing a list of all detected LPs on the image | List of detected LPs |
| x | int | The horizontal coordinate of the upper left corner of the BBox | 0…1920 |
| y | int | The vertical coordinate of the top left corner of the BBox | 0…1080 |
| width | int | Bbox width | 0…1920 |
| height | int | Bbox height | 0…1080 |
| score | float | Estimation of LPs detection accuracy | 0.0000…1.0000 |
| detector | string | The name of the detector type | grz |
| execution_time | int | Execution time in milliseconds | 0…1000 |
Vehicle redetector#
The vehicle redetector is designed to detect vehicles in a specified region of the frame. This method allows returning the coordinates of the BBox of the vehicle found in the specified area of the image.
When using the vehicle redetector, you can send photos and detections for performing re-detection (redetection) of vehicles. The process includes uploading the image, preprocessing, and performing the redetection to improve accuracy. The redetector allows detecting multiple vehicles in a single frame and accurately determining their location. In case of errors or failed redetection attempts, the system returns an error description with the relevant details.
An example of an input image is shown in Figure 31.
Response example:
{
"detections": [
{
"x": 162,
"y": 103,
"width": 1072,
"height": 749,
"score": 0.6372541785240173
}
],
"execution_time": 20,
"redetector": "car"
}
Table 40. Description of the Vehicle Detection fields
| Field name | Type | Description | Possible values |
|---|---|---|---|
| detections | array | An array containing the coordinates and size of each detected vehicle, as well as the detection accuracy score | List of detected vehicles |
| x | int | The horizontal coordinate of the upper left corner of the BBox | 0…1920 |
| y | int | The vertical coordinate of the top left corner of the BBox | 0…1080 |
| width | int | Bbox width | 0…1920 |
| height | int | Bbox height | 0…1080 |
| score | float | Accuracy score of the vehicle detection | 0.0000…1.0000 |
| redetector | string | The name of the redetector | car |
| execution_time | int | Execution time in milliseconds | 0…1000 |
Smoke and fire detector#
The smoke and fire detector is designed to detect fires in images, video streams, and video files.
An example of an input image is shown in Figure 32.
Response example:
{
"detections": [
{
"x": 0,
"y": 0,
"width": 960,
"height": 540,
"score": 0.8205,
"type": "fire"
}
],
"execution_time": 51,
"detector": "smoke_fire"
}
Table 41. Description of the Smoke and fire detector fields
| Field name | Type | Description | Possible values |
|---|---|---|---|
| detections | array | An array containing the coordinates, detection area size and the detections of fire or smoke in the image, as well as an estimate of the detection accuracy | List of detected zones |
| x | int | The horizontal coordinate of the upper left corner of the BBox | 0…1920 |
| y | int | The vertical coordinate of the top left corner of the BBox | 0…1080 |
| width | int | Width of detection zone | 0…1920 |
| height | int | Height of detection zone | 0…1080 |
| score | float | Estimation of smoke/fire detection accuracy | 0.0000…1.0000 |
| type | string | Type | - fire; |
| - black_smoke; | |||
| - white_smoke; | |||
| - none - no fire | |||
| execution_time | int | Execution time in milliseconds | 0…1000 |
| detector | string | The name of the detector type | smoke_fire |
Animal detector#
The animal detector is designed to detect animals in images, video streams, and video files.
An example of an input image is shown in Figure 33.
Response example:
{
"detections": [
{
"x": 323,
"y": 42,
"width": 137,
"height": 342,
"score": 0.9622,
"type": "big_animal"
}
],
"execution_time": 60,
"detector": "animal"
}
Table 42. Description of the Animal detector fields
| Field name | Type | Description | Possible values |
|---|---|---|---|
| detections | array | An array containing the coordinates and size of detections of each animal in the image, as well as an estimate of detection accuracy | List of detected animals |
| x | int | The horizontal coordinate of the upper left corner of the BBox | 0…1920 |
| y | int | The vertical coordinate of the top left corner of the BBox | 0…1080 |
| width | int | Bbox width | 0…1920 |
| height | int | Bbox height | 0…1080 |
| score | float | Estimation of animal detection accuracy | 0.0000…1.0000 |
| type | string | Type | - big_animal; |
| - small_animal; | |||
| - bird | |||
| execution_time | int | Execution time in milliseconds | 0…1000 |
| detector | string | The name of the detector type | animal |
Unified method#
The Unified method is designed for detecting and recognizing vehicles, license plates, animals, smoke and fire in images, video streams, and video files. This method allows processing multiple object types in a single request, which previously required separate requests. For example, you can simultaneously detect a vehicle and a license plate, as well as other objects such as animals or smoke. This reduces the number of requests and speeds up the process of obtaining results.
The method supports several image processing modes.
In the JSON response structure, the following levels of nesting are used: batches, batch, and photoProcessing.
photoProcessing— an array containing information about objects in the image, obtained through detectors or classifiers;batches— an array containing sets of images that are processed with different configurations of detectors or classifiers;batch— a group of images that are processed in one request. All images within a group are processed with the same detector or classifier settings. Each group can have its own settings.
Example: If there are two images, img1 and img2, which need to be processed with the same detector settings, they will be grouped into one batch. However, if these images need to be processed with different settings, they will be grouped into another batch.
Full Frame Processing#
In this mode, the method performs detection and recognition of all objects in the full-frame image, including vehicles, license plates, animals, smoke and fire.
Full Frame Processing for Vehicle Detection with Classifier#
Response example:
{
"batches": [
{
"batch": [
{
"photoProcessing": [
{
"detections": [
{
"bbox": {
"height": 296,
"width": 410,
"x": 15,
"y": 90
},
"detectionProcessing": [
{
"brand": "Porsche",
"classifier": "car_brand_model_v2",
"model": "Cayenne",
"score": 0.9407
}
],
"score": 0.9598
}
],
"detector": "car"
}
]
}
]
}
]
}
Table 43. Description of Full Frame Processing Fields for Vehicle Detection with Classifier
| Field name | Type | Description | Possible values |
|---|---|---|---|
| batches | array | An array containing groups of requests with processed images | List of objects |
| batch | array | An array containing image processing in a single request | List of objects |
| photoProcessing | array | An array containing information on detection and recognition of objects in the image | List of objects |
| detections | array | An array containing coordinates, size of detected objects, and their accuracy score | List of detections on image |
| x | int | Horizontal coordinate of the top-left corner of the BBox | 0…1920 |
| y | int | Vertical coordinate of the top-left corner of the BBox | 0…1080 |
| width | int | Width of the BBox | 0…1920 |
| height | int | Height of the BBox | 0…1080 |
| detectionProcessing | array | An array containing additional processing stages | List of objects |
| classifier | string | Name of the classifier | car_brand_model_v2 |
| brand | string | Vehicle brand | Porsche |
| model | string | Vehicle model | Cayenne |
| score | float | Accuracy score for recognizing the vehicle's brand and model | 0.0…1.0 |
| score | float | Accuracy score for detecting the object | 0.0…1.0 |
| detector | string | Type of detector | car |
Full Frame Processing for Vehicle and License Plate Detection with Classifier#
Response example:
{
"batches": [
{
"batch": [
{
"photoProcessing": [
{
"detections": [
{
"bbox": {
"height": 296,
"width": 410,
"x": 15,
"y": 90
},
"detectionProcessing": [
{
"detections": [
{
"bbox": {
"height": 21,
"width": 94,
"x": 121,
"y": 299
},
"class": 0,
"detectionProcessing": [
{
"classifier": "license_plate_ags_v1",
"score": 0.994
}
],
"score": 1.0
}
],
"detector": "grz"
}
],
"score": 0.9598
}
],
"detector": "car"
}
]
}
]
}
]
}
Table 44. Description of Full Frame Processing Fields for Vehicle and License Plate Detection with Classifier
| Field name | Type | Description | Possible values |
|---|---|---|---|
| batches | array | An array containing groups of requests with processed images | List of objects |
| batch | array | An array containing image processing in a single request | List of objects |
| photoProcessing | array | An array containing information about the detection and recognition of objects in the image | List of objects |
| detections | array | An array containing coordinates, size of detected objects, and their accuracy score | List of detections in the image |
| x | int | Horizontal coordinate of the top-left corner of the BBox | 0…1920 |
| y | int | Vertical coordinate of the top-left corner of the BBox | 0…1080 |
| width | int | Width of the BBox | 0…1920 |
| height | int | Height of the BBox | 0…1080 |
| detectionProcessing | array | An array containing additional processing stages | List of objects |
| detections | array | An array containing coordinates, size of detected objects, and their accuracy score | List of detections in the image |
| x | int | Horizontal coordinate of the top-left corner of the BBox | 0…1920 |
| y | int | Vertical coordinate of the top-left corner of the BBox | 0…1080 |
| width | int | Width of the BBox | 0…1920 |
| height | int | Height of the BBox | 0…1080 |
| class | int | Object class | 0 (license plate), 1 (vehicle) |
| detectionProcessing | array | An array containing additional processing stages | List of objects |
| classifier | string | Name of the classifier | license_plate_ags_v1 |
| score | float | Accuracy score for recognizing the license plate | 0.0…1.0 |
| score | float | Accuracy score for license plate detection | 0.0…1.0 |
| detector | string | Type of detector | grz |
| score | float | Accuracy score for vehicle detection | 0.0…1.0 |
| detector | string | Type of detector | car |
Full Frame Processing for Animal Detection#
Response example:
{
"batches": [
{
"batch": [
{
"photoProcessing": [
{
"detections": [
{
"bbox": {
"height": 342.9569,
"width": 137.2958,
"x": 323.9004,
"y": 42.0744
},
"score": 0.9622,
"type": "big_animal"
}
],
"detector": "animal"
}
]
}
]
}
]
}
Table 45. Description of Full Frame Processing Fields for Animal Detection
| Field name | Type | Description | Possible values |
|---|---|---|---|
| batches | array | An array containing groups of requests with processed images | List of objects |
| batch | array | An array containing image processing in a single request | List of objects |
| photoProcessing | array | An array containing information about the detection and recognition of objects in the image | List of objects |
| detections | array | An array containing the coordinates and size of detections of each animal in the image, as well as an estimate of detection accuracy | List of detected animals |
| x | int | The horizontal coordinate of the upper left corner of the BBox | 0…1920 |
| y | int | The vertical coordinate of the top left corner of the BBox | 0…1080 |
| width | int | Bbox width | 0…1920 |
| height | int | Bbox height | 0…1080 |
| score | float | Estimation of animal detection accuracy | 0.0000…1.0000 |
| type | string | Type | - big_animal; |
| - small_animal; | |||
| - bird | |||
| execution_time | int | Execution time in milliseconds | 0…1000 |
| detector | string | The name of the detector type | animal |
Full Frame Processing for Smoke and Fire Detection#
Response example:
{
"batches": [
{
"batch": [
{
"photoProcessing": [
{
"detections": [
{
"bbox": {
"height": 540,
"width": 960,
"x": 0,
"y": 0
},
"score": 0.8205,
"type": "fire"
}
],
"detector": "smoke_fire"
}
]
}
]
},
]
}
Table 46. Description of Full Frame Processing Fields for Smoke and Fire Detection
| Field name | Type | Description | Possible values |
|---|---|---|---|
| batches | array | An array containing groups of requests with processed images | List of objects |
| batch | array | An array containing image processing in a single request | List of objects |
| photoProcessing | array | An array containing information about the detection and recognition of objects in the image | List of objects |
| detections | array | An array containing the coordinates, detection area size and the detections of fire or smoke in the image, as well as an estimate of the detection accuracy | List of detected zones |
| x | int | The horizontal coordinate of the upper left corner of the BBox | 0…1920 |
| y | int | The vertical coordinate of the top left corner of the BBox | 0…1080 |
| width | int | Width of detection zone | 0…1920 |
| height | int | Height of detection zone | 0…1080 |
| score | float | Estimation of smoke/fire detection accuracy | 0.0000…1.0000 |
| type | string | Type | - fire; |
| - black_smoke; | |||
| - white_smoke; | |||
| - none - no fire | |||
| execution_time | int | Execution time in milliseconds | 0…1000 |
| detector | string | The name of the detector type | smoke_fire |
Detectors settings#
Detector settings allow you to retrieve the current parameters for vehicle and LP detectors, including detection accuracy thresholds and image sizes. The method returns the current settings for these detectors.
Response example:
{
"detectorsSettings": [
{
"detector": "car",
"scoreThreshold": 0.5,
"imageSize": 640
},
{
"detector": "grz",
"scoreThreshold": 0.5,
"imageSize": 220
}
]
}
Table 47. Description of the detectors settings fields
| Field name | Type | Description | Possible values |
|---|---|---|---|
| detectorsSettings | array | An array containing the current settings for vehicle and LP detectors | A list of objects with configuration parameters |
| detector | string | Type | car, grz |
| scoreThreshold | float | Detection accuracy threshold | 0.0000…1.0000 |
| imageSize | int | Image size for the detector | 100…1280 |