FaceStream version 5.1.18
Introduction
All URLs must be prefixed with API version like so: http://127.0.0.1:34569/api/1
. This document describes FaceStream backend service API. The service is capable of:
- getting current streams,
- providing streams preview.
Refer to streams
API for details on how to get stream params.
Refer to streams/preview
API for details on how to get stream preview.
Refer to metrics
API for details on how to get streams metrics.
API Versioning
Actual FaceStream version is available by URL like this: http://127.0.0.1:34569/version
.
/streams
Stream API allows clients to get information about processing streams.
It is also possible to request a video preview for the particular stream. Refer to streams/preview/
API for details.
Get list of current stream parameters.
get /streams
Get list of current stream parameters.
HTTP status code 200
Body
Media type: application/json
Type: array of object
Items: stream_info
- account_id: required(string)
new stream account ID for authentication in Luna Platform
- stream_id: required(string)
stream UUID
- name: required(string)
display name
- description: (string)
new stream description
- preview_url: required(string)
stream preview URL
- status: required(string)
stream status
- version: required(integer)
stream version
- error: required(string)
stream error
- data: required(object)
stream input parameters
- reference: required(string)
camera address, e.g.
- type: required(one of tcp, udp, videofile, images)
transport protocol for rtsp camera, by default "tcp"
- roi: (array of integer)
region of interest of camera frame (left, top, width, height), by default [0 0 0 0] - full frame
- droi: (array of integer)
region of interest of camera frame (left, top, width, height), by default [0 0 0 0] - full frame
- rotation: (integer)
angle of camera frame rotation, possible values are 0, 90, 180, 270, by default 0
- preferred_program_stream_frame_width: (integer)
frame width of the preferred program stream, by default 800
- reference: required(string)
- event_handler: required(object)
stream event handler data
- origin: required(string)
event handler service origin
- api_version: required(integer)
event handler service api version
- bestshot_handler: required(object)
event bestshot handler
- handler_id: required(string)
handler id in Luna Platform
- handler_id: required(string)
- detection_handler: (object)
event detection handler
- handler_id: required(string)
handler id in Luna Platform
- handler_id: required(string)
- frame_store: (string)
frame store URL
- origin: required(string)
- policies: required(object)
stream processing policies
- ffmpeg_threads_number: required(integer)
number of threads for video decoding, by default 0 - chosen by decoder library
- frame_processing_mode: required(one of auto, full, scale)
Determines whether full or scaled frame will be processed, by default "auto" - depends on fs3Config param 'convertFullFrame'
- real_time_mode_fps: required(integer)
- health_check: required(object)
stream health check parameters
- max_error_count: required(integer)
error count limit in period to consider stream to be alive, by default 10
- period: required(integer)
see description of max_error_count, measured in seconds, by default 3600
- retry_delay: required(integer)
time gap (in seconds) between attempts to reopen not alive stream, by default 5
- max_error_count: required(integer)
- filtering: required(object)
stream filtering parameters
- min_score: required(number)
score that defines detection quality, threshold for filtering detections sent to the server, by default 0.5187.
- detection_yaw_threshold: required(integer)
sets the maximum value of head yaw angle in relation to camera, by default 40.
- detection_pitch_threshold: required(integer)
sets the maximum value of head pitch angle in relation to camera, by default 40.
- detection_roll_threshold: required(integer)
sets the maximum value of head roll angle in relation to camera, by default 30.
- yaw_number: required(integer)
defines the number of frames for image filtration based on head tilt angle, by default 1.
- yaw_collection_mode: required(boolean)
sets the number of frames the system must collect to analyze head yaw angle, by default false.
- mouth_occlusion_threshold: required(number)
determines how much the mouth can be obscured in the frame, by default 0.0.
- min_score: required(number)
- sending: required(object)
stream sending parameters
- time_period_of_searching: required(integer)
interval in track after the end of which a best shot is sent to the server, by default -1.
- silent_period: required(integer)
interval between period. Once the analysis period is over, the system holds this silent_period before starting next period of frame analysis, by default -1.
- type: required(string)
sets the measurement metric for analysis periods and intervals between those (per frames or per seconds), by default "sec".
- number_of_bestshots_to_send: required(integer)
number of frames that the user sets to receive from the track or certain periods of this track, by default 1.
- send_only_full_set: required(boolean)
This parameter enables to send data only if the required amount of data is available.
- delete_track_after_sending: required(boolean)
This parameter enables to delete track after sending.
- time_period_of_searching: required(integer)
- primary_track_policy: required(object)
stream primary track policy parameters
- use_primary_track_policy: required(boolean)
is used in cases of Access Control Systems (turnstiles/gates at the office/bank entrances) for easier control and face recognition implementation in a secured area, by default false.
- best_shot_min_size: required(integer)
sets a minimal size for detections within Primary Track policy, by default 70.
- best_shot_proper_size: required(integer)
sets the size of detection for Primary Track policy. When a detection reaches the defined value, track immediately sends all its best shots to the server, by default 140.
- use_primary_track_policy: required(boolean)
- liveness: required(object)
stream liveness parameters
- use_shoulders_liveness_filtration: required(boolean)
enables checking the presence of a real person in the frame based on the head and shoulder areas, by default false.
- use_mask_liveness_filtration: required(boolean)
enables checking the presence of a real person in the frame based on backgrounds, by default false.
- use_flying_faces_liveness_filtration: required(boolean)
enables checking the presence of a real person in the frame based on the facial surrounding, by default false.
- liveness_mode: required(integer)
enables to specify which frames from a track will undergo Liveness check. There are three options for selecting a frame
- number_of_liveness_checks: required(integer)
enables to specify the number of frames to check for Liveness. The specified value is used in the liveness-mode parameter. By default 10.
- liveness_threshold: required(number)
is used to define the presence of a real person in a frame. The system confirms that it is a real person in the frame, and not a photo, only if Liveness returned a value higher than the one specified in the parameter. By default 0.8.
- livenesses_weights: required(array of any)
determines the involvement of each liveness check type (shoulders, mask, and flying_faces) in the resulting estimation of the presence of a human face in the frame. By default [0.0, 0.25, 0.75].
- mask_backgrounds_count: required(integer)
the number of background frames that are used for the corresponding checks, by default 300.
- use_shoulders_liveness_filtration: required(boolean)
- ffmpeg_threads_number: required(integer)
- location: (object)
stream source location parameters
- city: (string - maxLength: 36)
city where event is incidented
- area: (string - maxLength: 36)
area where event is incidented
- district: (string - maxLength: 36)
district where event is incidented
- street: (string - maxLength: 36)
street where event is incidented
- house_number: (string - maxLength: 36)
house number where event is incidented
- geo_position: (object)
geo position specified by geographic coordinates - longitude and latitude
- longitude: required(number - minimum: -180 - maximum: 180)
longitude in degrees
- latitude: required(number - minimum: -90 - maximum: 90)
latitude in degrees
- longitude: required(number - minimum: -180 - maximum: 180)
- city: (string - maxLength: 36)
- video_info: required(object)
stream source video info
- width: required(integer)
stream source video width
- height: required(integer)
stream source video height
- frame_rate: required(integer)
stream source video frame rate
- bit_rate: required(integer)
stream source video bit rate
- gop_size: required(integer)
stream source video gop size
- start_time: required(string)
stream start time (ISO 8601)
- width: required(integer)
Example:
[
{
"account_id": "7e91c63e-4d48-4664-b88b-763e1434213e",
"data": {
"droi": [
0,
0,
0,
0
],
"reference": "path",
"roi": [
0,
0,
0,
0
],
"rotation": 0,
"type": "tcp"
},
"description": "update",
"error": "ok",
"event_handler": {
"api_version": 6,
"bestshot_handler": {
"handler_id": "8ca87ad9-b649-4118-b538-163146165cf5"
},
"detection_handler": {
"handler_id": "d7112abb-2227-4386-95e4-496500816221"
},
"frame_store": "http://127.0.0.1:5020/1/buckets/frames/images",
"origin": "http://10.16.4.159:4000"
},
"location": {
"area": "Central",
"city": "Moscow",
"district": "Basmanny",
"geo_position": {
"latitude": 55.752,
"longitude": 36.616
},
"house_number": "23 bldg.3",
"street": "Podsosensky lane"
},
"name": "closed mouth",
"policies": {
"ffmpeg_threads_number": 0,
"filtering": {
"detection_pitch_threshold": 40,
"detection_roll_threshold": 30,
"detection_yaw_threshold": 40,
"min_score": 0.30000001192092896,
"mouth_occlusion_threshold": 0,
"yaw_collection_mode": false,
"yaw_number": 1
},
"frame_processing_mode": "auto",
"healthcheck": {
"max_error_count": 10,
"period": 3600,
"retry_delay": 5
},
"liveness": {
"liveness_mode": 1,
"liveness_threshold": 0.6000000238418579,
"livenesses_weights": [
0.05000000074505806,
0.44999998807907104,
0.5
],
"mask_backgrounds_count": 300,
"number_of_liveness_checks": 10,
"use_flying_faces_liveness_filtration": false,
"use_mask_liveness_filtration": false,
"use_shoulders_liveness_filtration": false
},
"primary_track_policy": {
"best_shot_min_size": 70,
"best_shot_proper_size": 140,
"use_primary_track_policy": false
},
"real_time_mode_fps": 0,
"sending": {
"delete_track_after_sending": true,
"number_of_bestshots_to_send": 8,
"send_only_full_set": true,
"silent_period": 0,
"time_period_of_searching": -1,
"type": "sec"
}
},
"preview_url": "/api/1/streams/preview/626907a7-c841-49f5-992c-03623a1d69e0",
"status": "in_progress",
"stream_id": "626907a7-c841-49f5-992c-03623a1d69e0",
"version": 1,
"video_info": {
"bit_rate": 0,
"frame_rate": 25,
"gop_size": 12,
"height": 1080,
"start_time": "2022-03-22T14:28:23.997Z",
"width": 1920
}
}
]
HTTP status code 400
Invalid query.
Body
Media type: application/json
Type: any
Example:
{
"error_code": 1,
"detail": "Any streams are not found"
}
Get stream info.
get /streams/{id}
Get stream info.
URI Parameters
- id: required(string)
Stream ID UUID.
HTTP status code 200
Ok.
Body
Media type: application/json
Type: object
Properties- account_id: required(string)
new stream account ID for authentication in Luna Platform
- stream_id: required(string)
stream UUID
- name: required(string)
display name
- description: (string)
new stream description
- preview_url: required(string)
stream preview URL
- status: required(string)
stream status
- version: required(integer)
stream version
- error: required(string)
stream error
- data: required(object)
stream input parameters
- reference: required(string)
camera address, e.g.
- type: required(one of tcp, udp, videofile, images)
transport protocol for rtsp camera, by default "tcp"
- roi: (array of integer)
region of interest of camera frame (left, top, width, height), by default [0 0 0 0] - full frame
- droi: (array of integer)
region of interest of camera frame (left, top, width, height), by default [0 0 0 0] - full frame
- rotation: (integer)
angle of camera frame rotation, possible values are 0, 90, 180, 270, by default 0
- preferred_program_stream_frame_width: (integer)
frame width of the preferred program stream, by default 800
- reference: required(string)
- event_handler: required(object)
stream event handler data
- origin: required(string)
event handler service origin
- api_version: required(integer)
event handler service api version
- bestshot_handler: required(object)
event bestshot handler
- handler_id: required(string)
handler id in Luna Platform
- handler_id: required(string)
- detection_handler: (object)
event detection handler
- handler_id: required(string)
handler id in Luna Platform
- handler_id: required(string)
- frame_store: (string)
frame store URL
- origin: required(string)
- policies: required(object)
stream processing policies
- ffmpeg_threads_number: required(integer)
number of threads for video decoding, by default 0 - chosen by decoder library
- frame_processing_mode: required(one of auto, full, scale)
Determines whether full or scaled frame will be processed, by default "auto" - depends on fs3Config param 'convertFullFrame'
- real_time_mode_fps: required(integer)
- health_check: required(object)
stream health check parameters
- max_error_count: required(integer)
error count limit in period to consider stream to be alive, by default 10
- period: required(integer)
see description of max_error_count, measured in seconds, by default 3600
- retry_delay: required(integer)
time gap (in seconds) between attempts to reopen not alive stream, by default 5
- max_error_count: required(integer)
- filtering: required(object)
stream filtering parameters
- min_score: required(number)
score that defines detection quality, threshold for filtering detections sent to the server, by default 0.5187.
- detection_yaw_threshold: required(integer)
sets the maximum value of head yaw angle in relation to camera, by default 40.
- detection_pitch_threshold: required(integer)
sets the maximum value of head pitch angle in relation to camera, by default 40.
- detection_roll_threshold: required(integer)
sets the maximum value of head roll angle in relation to camera, by default 30.
- yaw_number: required(integer)
defines the number of frames for image filtration based on head tilt angle, by default 1.
- yaw_collection_mode: required(boolean)
sets the number of frames the system must collect to analyze head yaw angle, by default false.
- mouth_occlusion_threshold: required(number)
determines how much the mouth can be obscured in the frame, by default 0.0.
- min_score: required(number)
- sending: required(object)
stream sending parameters
- time_period_of_searching: required(integer)
interval in track after the end of which a best shot is sent to the server, by default -1.
- silent_period: required(integer)
interval between period. Once the analysis period is over, the system holds this silent_period before starting next period of frame analysis, by default -1.
- type: required(string)
sets the measurement metric for analysis periods and intervals between those (per frames or per seconds), by default "sec".
- number_of_bestshots_to_send: required(integer)
number of frames that the user sets to receive from the track or certain periods of this track, by default 1.
- send_only_full_set: required(boolean)
This parameter enables to send data only if the required amount of data is available.
- delete_track_after_sending: required(boolean)
This parameter enables to delete track after sending.
- time_period_of_searching: required(integer)
- primary_track_policy: required(object)
stream primary track policy parameters
- use_primary_track_policy: required(boolean)
is used in cases of Access Control Systems (turnstiles/gates at the office/bank entrances) for easier control and face recognition implementation in a secured area, by default false.
- best_shot_min_size: required(integer)
sets a minimal size for detections within Primary Track policy, by default 70.
- best_shot_proper_size: required(integer)
sets the size of detection for Primary Track policy. When a detection reaches the defined value, track immediately sends all its best shots to the server, by default 140.
- use_primary_track_policy: required(boolean)
- liveness: required(object)
stream liveness parameters
- use_shoulders_liveness_filtration: required(boolean)
enables checking the presence of a real person in the frame based on the head and shoulder areas, by default false.
- use_mask_liveness_filtration: required(boolean)
enables checking the presence of a real person in the frame based on backgrounds, by default false.
- use_flying_faces_liveness_filtration: required(boolean)
enables checking the presence of a real person in the frame based on the facial surrounding, by default false.
- liveness_mode: required(integer)
enables to specify which frames from a track will undergo Liveness check. There are three options for selecting a frame
- number_of_liveness_checks: required(integer)
enables to specify the number of frames to check for Liveness. The specified value is used in the liveness-mode parameter. By default 10.
- liveness_threshold: required(number)
is used to define the presence of a real person in a frame. The system confirms that it is a real person in the frame, and not a photo, only if Liveness returned a value higher than the one specified in the parameter. By default 0.8.
- livenesses_weights: required(array of any)
determines the involvement of each liveness check type (shoulders, mask, and flying_faces) in the resulting estimation of the presence of a human face in the frame. By default [0.0, 0.25, 0.75].
- mask_backgrounds_count: required(integer)
the number of background frames that are used for the corresponding checks, by default 300.
- use_shoulders_liveness_filtration: required(boolean)
- ffmpeg_threads_number: required(integer)
- location: (object)
stream source location parameters
- city: (string - maxLength: 36)
city where event is incidented
- area: (string - maxLength: 36)
area where event is incidented
- district: (string - maxLength: 36)
district where event is incidented
- street: (string - maxLength: 36)
street where event is incidented
- house_number: (string - maxLength: 36)
house number where event is incidented
- geo_position: (object)
geo position specified by geographic coordinates - longitude and latitude
- longitude: required(number - minimum: -180 - maximum: 180)
longitude in degrees
- latitude: required(number - minimum: -90 - maximum: 90)
latitude in degrees
- longitude: required(number - minimum: -180 - maximum: 180)
- city: (string - maxLength: 36)
- video_info: required(object)
stream source video info
- width: required(integer)
stream source video width
- height: required(integer)
stream source video height
- frame_rate: required(integer)
stream source video frame rate
- bit_rate: required(integer)
stream source video bit rate
- gop_size: required(integer)
stream source video gop size
- start_time: required(string)
stream start time (ISO 8601)
- width: required(integer)
Allows clients to obtain stream preview as MotionJPEG: server sends JPEG stream of frames.
get /streams/preview/{id}
Allows clients to obtain stream preview as MotionJPEG: server sends JPEG stream of frames.
URI Parameters
- id: required(string)
Stream UUID.
Query Parameters
- quality: (integer)
JPEG compression level of stream preview image (is in [0, 100] range, values lower than 25 could produce bad visual quality), by default 75
- show_bbox: (integer)
Show tracks and some other info on stream preview (0 or 1), by default 1
- width: (integer)
Max preview width (scaled down if actual width is larger). Image is scaled with proportion saving. If it is equal 0, doesn't affect, by default 0
- height: (integer)
Max preview height (scaled down if actual height is larger). Image is scaled with proportion saving. If it is equal 0, doesn't affect, by default 0
- fps: (integer)
Max translation stream fps. If it is equal 0, doesn't affect, by default 0
HTTP status code 200
Ok.
Body
Media type: image/jpeg
Type: any
Example:
<jpeg image data>
HTTP status code 400
Invalid query or id.
Body
Media type: application/json
Type: any
Example:
{
"error_code": 1,
"detail": "Invalid uuid"
}
HTTP status code 404
Stream not found.
Body
Media type: application/json
Type: any
Example:
{
"error_code": 1,
"detail": "Stream with such uuid doesn't exist"
}
HTTP status code 500
Internal error.
Body
Media type: application/json
Type: any
Example:
{
"error_code": 2,
"detail": "Unhandled exception"
}
Gets stream last frame as image/jpeg.
get /streams/preview/frame/{id}
Gets stream last frame as image/jpeg.
URI Parameters
- id: required(string)
Stream UUID.
Query Parameters
- quality: (integer)
JPEG compression level of stream preview image (is in [0, 100] range, values lower than 25 could produce bad visual quality), by default 75
HTTP status code 200
Ok.
Body
Media type: image/jpeg
Type: any
Example:
<jpeg image data>
HTTP status code 400
Invalid query or id.
Body
Media type: application/json
Type: any
Example:
{
"error_code": 1,
"detail": "Invalid uuid"
}
HTTP status code 404
Stream not found.
Body
Media type: application/json
Type: any
Example:
{
"error_code": 1,
"detail": "Stream with such uuid doesn't exist"
}
HTTP status code 500
Internal error.
Body
Media type: application/json
Type: any
Example:
{
"error_code": 2,
"detail": "Failed to encode frame to jpeg"
}
/metrics
Stream API allows clients to get all metrics about Face Stream's current state, like streams or translations count, errors, etc.
Output format can be obtained from "https://prometheus.io/".
Metrics is subdivided into two sections: global metrics (all streams information) and each stream's metrics.
Global metrics section is started with comment "# Global counters".
Each stream metrics section is started with comment like "# Stream name = 'stream name' uuid = 'stream uuid'".
Global metrics counters:
Create_stream_errors - streams creating errors counter. Tags: "code" (text code of error) and "msg" (description of error)
Streams_count - current streams count. Tags: "all" (all streams count) and "alive" (alive streams count)
Translations_count - current translations count
Streams metrics counters:
Streams_errors - count of errors occurred in attempts to open stream or read frame from it. Tags: "action" (open or read frame), "error" (text code of error), "msg" (description of error)
Skips_count - total frames, that have been skipped (not processed) because of high load
Detections_count - current frame detections count
FPS - current FPS (of stream processing)
Allows clients to obtain metrics about streams.
/version
Receiving version information about FaceStream.