Now the “image_origin_policy” mechanism of the LUNA PLATFORM handler will be used to save the source frame if the resource “/images” is supposed to be used.
Previously, several internal requests were made to save the source frame.
The address to the “/images” resource is set in the “frame_store” parameter in the streams management settings (for example, “frame_store”: “http://127.0.0.1:5000/6/images”).
This feature does not apply to saving source images to the LUNA Image Store storage bucket.
Note that in order to send the source frame, the “send_source_frame” parameter must be enabled in the FaceStream settings, and the handler policy must have the saving of source images enabled (parameter “policies” > “storage_policy” > “image_origin_policy” > “store_image” = “1” ).
The ability to specify ROI or DROI coordinates as a percentage of the current frame size has been added.
To achieve this, a new field called “mode” has been added to the parameters “roi” and “droi” in the “create stream”, “get stream” and “get streams” requests of the LUNA Streams service.
This field enables you to set one of two modes:
If the “mode” field is not specified in the request body, then the value “abs” will be used. Thus, there is no need to recreate the old streams.
A new tag “estimator” has been added to the list of tags for logging (the “tags” parameter in the FaceStream settings), designed to collect and display statistics of completed estimates.
When the tag is enabled in the FaceStream logs, information is returned on the size of the butches with which the estimators were called. In future releases, the list of returned information may be expanded.
Example of log content with the “estimator” tag enabled:
[I0609 15:48:03.779697 65 EstimatorStatistic.cpp:85] [estimator] Batch statistic for estimator HeadPoseEstimator
Total calls: 1311 total time: 191 ms.
sz cnt
1 1311 (100.00%)
2 0 (0.00%)
3 0 (0.00%)
...
16 0 (0.00%)
In this case, the statistics show the following:
If the estimators were not called, then no information will be output in the FaceStream logs.
The ability to disable filtering by AGS and pitch, yaw, roll angles has been added.
To disable AGS filtering, you must set the value “0” for the “min_score” parameter in the stream management settings.
To disable angle filtering, you must set the values “180” for the parameters “detection_pitch_threshold”, “detection_roll_threshold” and “detection_yaw_threshold” in the stream management settings.
The GPU memory consumption of the FaceStream application has been reduced.
The default FaceStream monitoring settings are now similar to the default LUNA PLATFORM monitoring settings.
This means that if the default settings are used for monitoring the LUNA PLATFORM, then to enable FaceStream monitoring, it is enough to enable the “send_data” parameter in the FaceStream settings. In this case, the FaceStream monitoring data will be saved to the “luna_monitoring” bucket.
The section of activating the LUNA PLATFORM license for LUNA Streams has been moved to a separate license activation manual “FS_without_LP/FS_License_Activation_Manual_Standalone_Eng.pdf/html”.
This manual also includes information on activating the LUNA PLATFORM license using Guardant keys (see the LUNA PLATFORM v.5.45.1 release notes for more information).
Added support for relative ROI/DROI.
Reduced video memory consumption on the video decoder side.
Added a new tag ‘estimator’ designed to collect and display statistics of estimators.
User can switch off the AGS and/or angles filtrations. For this it is needed to set the stream parameters into special values. The parameter min_score must be set into 0 for switching off the AGS filtration. Also it is needed to set three parameters: detection_pitch_threshold, detection_roll_threshold, and detection_yaw_threshold into 180 for switching off angles filtration.
Send the full frame image together with portraits. For this it is needed to set the parameter ‘send_source_frame’ in FaceStream configuration file to ‘true’ and define the parameter ‘frame_store’ in stream event handler as ‘
It is needed to take in account the body and/or face detection handler must have the parameter storage_policy: image_origin_policy: store_image: 1.
Service dependencies were updated.
Added new roi and droi format for stream data. Old format is deprecated. See createStream, getStreams, getStream for more info.
Updated FSDK to version 5.13.0.
In this version of FSDK, the body detector has been updated to version 5.
Note that when updating the FSDK version, you need to pay attention to changes in the “TRACK_ENGINE_CONFIG” settings in the Configurator service. In this release, the “TRACK_ENGINE_CONFIG” settings have not changed.
Now, when changing the frame resolution during stream processing, processing will be completed with the error frame size is changed from <value> to <value>
.
If an automatic restart is set for stream (see the autorestart
section of the “create stream” query), it will be restarted. In this case, FaceStream will again accept the stream for processing with a new frame resolution.
If the user is viewing the preview, user needs to refresh the preview window.
Changed the order of applying estimators. Now the AGS estimator is applied first and then the HeadPose.
Removed filtering by Liveness Head&Shoulders.
In the LUNA Streams OpenAPI specification, the use_shoulders_liveness_filter
parameter is declared as Deprecated and will be ignored when trying to create a new stream.
The first value in the livenesses_weights
parameter, which is responsible for filtering by Liveness Head&Shoulders, will also be ignored. For example, if you set the value livenesses_weights = [0.1, 0.25, 0.75]
, the value 0.1
will be considered as 0.0
.
Fixed errors
Close the stream when frame resolution is changed. If the autorestart is set for stream then it will be restarted. The FaceStream takes the stream with new frame resolution in processing again.
If user views the preview then he is needed to update preview window. For example refresh the preview page in browser.
Remove the head and shoulders liveness filtration.
This filtration method was deprecated a long time. Now it is removed.
The parameter use_shoulders_liveness_filtration in stream configuration is ignored.
The order in which estimation filters are applied has been changed. Now the AGS filter is applied first.
Service dependencies were updated.
The use_shoulders_liveness_filtration (policies/liveness/use_shoulders_liveness_filtration in create stream request body) parameter is deprecated and will not affect any results.
Now, along with sending the source frame to LUNA PLATFORM, a header X-Luna-Meta-rescale
is sent containing the scaling factor of the source image in the form of a floating-point number.
Previously, when setting the width of the source frame for scaling (the size_source_frame
parameter), it was impossible to determine the coordinates of the bbox on the source image, since only the coordinates of the bbox of the scaled image were sent to the LUNA PLATFORM.
To determine the real coordinates of the bbox on the scaled source image, it is necessary to divide the size of the obtained image by the obtained scaling factor.
Note. The real size is calculated with an absolute error of less than 1 / (2 * scaling factor).
For example, if the obtained image has a size of 200x113, and the scaling factor is 0.104399, then the real size of the source image is 1916x1082 (200/0.104399=1916, 113/0.104399=1082).
The scaling factor can be found in LUNA PLATFORM:
with_meta = 1
query parameter;The X-Luna-Meta-rescale
header is only sent if sending the source frame is enabled (parameter send_source_frame = 1
). The LUNA PLATFORM handler should also have a policy of saving the source image.
If scaling is not set for the source frame (parameter size_source_frame = 0
), then the value of the X-Luna-Meta-rescale
response header will be equal to 1.
Section <LUNA_STREAMS_HTTP_SETTINGS
was added to the LUNA Streams configurations, containing settings responsible for processing HTTP connections.
The following settings are available:
request_timeout
- the duration of time between the instant when a new open TCP connection is passed to the server, and the instant when the whole HTTP request is received. Value (in seconds) is integer number, default 60.response_timeout
- the duration of time between the instant the server passes the HTTP request to the app, and the instant a HTTP response is sent to the client. Value (in seconds) is integer number, default 600.request_max_size
- how big a request may be (bytes). Value (in bytes) is integer number, default 1gb.keep_alive_timeout
- http keep alive timeout. Value (in seconds) is integer number, default 15.See the following link for details: https://sanic.dev/en/guide/deployment/configuration.html#builtin-values
New FORMAT
setting was added to all LIM services, which enables you to specify the logging format - default
(standard log output format) and json
(json log output).
Fixed errors
The error was fixed where enabling the send_source_frame
parameter could lead to a decrease in FaceStream performance.
The error was fixed in the Docker Compose script, due to which the uploading of LUNA Streams settings was performed through the script load_dump.py
(the old method used in versions 5.1.12 and below), and not through the migration mechanism (the new method used in versions 5.1.13 and above).
Note. If the previous version of FaceStream was launched via Docker Compose, then when updating, you must additionally execute the command to upload settings via the migration mechanism for the previous version of LUNA Streams v.0.5.17, and then execute the same command to upload settings for the new version of LUNA Streams v.0.6.2. See the section “Migrate settings” FaceStream update manual.
Add the rescale ratio in meta “X-Luna-Meta-rescale” for source image that is sent to Luna.
One may take this parameter with source image using the ‘get image’ request to Luna with query parameter ‘with_meta=1’. This value is a string in field ‘X-Luna-Meta-rescale’ of header. This string represents the floating point value of ratio. The ratio shows the scale coefficient that was used for scaling. To find real coordinates of source image it is needed to divide the size of received image on this rescale ratio.
Important to understand that the real size will be calculated with absolute error less than 1/(2*ratio).
For example, received image has size 200x113 and rescale ratio is equal to 0.104399 then the real size of source image is 1916x1082 (200/0.104399=1916, 113/0.104399=1082, |err| < 4.8).
Service dependencies were updated.
Setting format
was added for logging settings. Possible values are “default” - standard logs formatting or “json” - for json formatted logs.
A config section ’LUNA_STREAMS_HTTP_SETTINGS` was added. This section contains several settings which are responsible for process http connections. Available following settings now:
More detail see here.
Implemented functionality for collecting and sending FaceStream monitoring data to the Influx database. Monitoring for FaceStream is disabled by default. To enable FaceStream monitoring, you need to enable the “send_data” setting and set additional settings from the “monitoring” section of the FaceStream settings.
The following data is sent to InfluxDB:
measurement
element. It is equal to the value of fs-requests
.tag set:
fs_ip
- IP address where FaceStream is deployedsource
- the “name” field set when creating a stream in LUNA Streams (optional)stream_id
- stream IDfield set:
track_id
- track IDevent_id
- event ID received from LUNA PLATFORMrequest_id
- external ID for communication with monitoring of LUNA PLATFORM servicestrack_start_time
- track start timetrack_best_shot_time
- the time when the frame with the bestshot being sent appeared in the systemtrack_best_shot_min_size_time
(optional) - the time when the detection size reached the value specified in the “best_shot_min_size” parametertrack_best_shot_proper_size_time
(optional) - time when the detection size reached the value specified in the “best_shot_proper_size” parameterliveness_start_time
(optional) - Liveness start timeliveness_end_time
(optional) - Liveness end timebestshot_count
- the number of bestshots senttime_from_first_frame_to_send
- the time that passed from the appearance of the first frame in FS to sending to LPtrack_send_count
- the number of tracks sentTags containing time are sent as UTC with microsecond precision.
<timestamp>
element. Is the time the bestshot(s) was(were) sent in microseconds.
See the “Monitoring” section of the administrator manual for more information.
Added user interface for the LUNA Streams service.
The interface can be opened in a browser by specifying the address and port of the LUNA Streams service: <streams_server_address>:<streams_server_port>
The user interface of the service contains three tabs - “Streams”, “Groups” and “Queue”.
See the detailed information in the “LUNA Streams user interface” section of the administrator manual.
Implemented sending coordinates of bounding rectangles of faces (X, Y, width, height) when using an image without transformation (setting portrait_type=gost
). Sending is implemented by adding the face_bounding_boxes
parameter containing an array of bounding boxes corresponding to the detected faces.
The coordinates of the rectangle are defined in the portrait coordinate system.
The coordinates of the rectangle are saved in the face_bounding_boxes
field of the event generation request body. These coordinates are used when generating an event, which allows face redetection, which is performed faster than face detection.
The maximum size of batch with sets of images and detections sent for AGS and HeadPose estimation is limited to 16.
This reduces the memory consumption of the FaceStream.
Fixed errors
Fixed the error where the value of silent_period=-1
could cause the track to be sent several times.
This error caused multiple events with the same track_id
to be generated in LUNA PLATFORM.
Added facility for collecting and sending monitoring data.
The maximum batch size that is transferred to estimators is limited to 16.
Add sending the face_bounding_boxes parameter in generate event request to Luna. The parameter contents the array of elements corresponded to detected faces in GOST portrait mode. Each element is consisted from image file name and the rectangle { x, y, width, height} bounded the detected face. The coordinates of rectangle is defined in coordinate system of portrait.
Implemented sending coordinates of bounding rectangles of faces (X, Y, width, height) when using a sample as an image source (setting “portrait_type=warp”). Sending is implemented by adding the “face_bounding_boxes” parameter containing an array of bounding boxes corresponding to the detected faces.
The coordinates of the bounding box are defined in the coordinate system of the source image.
Support for saving coordinates to an event is implemented in LUNA PLATFORM v.5.36.5. Thus, starting from this version of the FaceStream build, it is required to use a LUNA PLATFORM version of at least v.5.36.5.
The coordinates of the bounding box are stored in the “face_detections” > “detection” > “rect” field of the LUNA PLATFORM event.
Manuals for installing and updating FaceStream were divided into two parts:
Fixed errors
The error was fixed where the “callback-mode” parameter of the TrackEngine settings was not read.
FaceStream only works with “callback-mode=0”. Attempts to set the value of “callback-mode=1” will result in a warning message in the FaceStream logs and the value of “callback-mode=0” will be used.
Reduced generation time of data in JSON format and implemented sequential processing of frames.
This resulted in a fix for a error where using the “json” value to set “sending > request_type” would spike memory usage.
Note. Long time to generate JSON data can lead to increased memory consumption.
The error was fixed in which the mode of sending images of detected faces without transformation (the “portrait_type=gost” parameter) didn’t work on the GPU.
LUNA Streams fixed errors
The error was fixed where “location” > “geo_position” > “latitude”/“longitude” fields could not be set to integers.
Now you can specify both integer and float numbers.
Fixed a problem when parameter callback-mode was not read from configurator. FaceStream read settings from file and configurator. FaceStream always works with callback-mode = 0. If the value 1 is set then the warning message appears and value 0 is still used.
Reduced json generation time for sending/request_type: “json” and made frame processing sequential. Long json generation time may lead to enormous memory consumption.
FSDK was updated to version 5.12.0.
In this version, the Liveness Flying Faces estimator has been updated.
The description of many parameters in the LUNA Streams specification has been expanded.
Fixed errors
The error has been fixed where FaceStream could send an incorrect preview URL in the “preview > live > url” field of the LUNA Streams stream, which caused the following error and FaceStream continued to work:
Failed to validate input json' to occur in the FaceStream logs. Path: 'streams.0.preview.live.url', message: 'URL host invalid', description: Bad/incomplete input data.
This error could occur due to the fact that LUNA Streams was started after FaceStream (for example, the server was restarted, where both FaceStream and LUNA Streams are running), and in order to determine its IP address, the FaceStream application needs to either connect to the LUNA Streams service, or read the predefined IP address in the system environment variable “VL_FACE_HOST” (see section “Nuances of working with stream preview” of the administrator manual).
Now if the IP address of FaceStream is determined by connecting to LUNA Streams and the service starts after FaceStream, then FaceStream will terminate with the error “Failed to get local IP address. Reason …”.
If the IP address of FaceStream is determined using the correctly set “VL_FACE_HOST” environment variable, then such an error will not occur.
The error has been fixed where the DROI bounding box flashed while viewing a stream in the browser if “show_bbox” was set to “0”.
The error has been fixed where some detections with human body coordinates for the best shots were not sent to LUNA PLATFORM.
Accordingly, these detections were absent in events generated using the dynamic handler specified in the “detection_handler” parameter.
A mechanism for migrating LUNA Streams settings set in the LUNA Configurator service has been added.
LUNA Streams update
Previously, LUNA Streams settings were automatically migrated when migrating the LUNA Configurator service. Now LUNA Streams settings are migrated using a separate migration mechanism. See the upgrade manual for more details.
Launch LUNA Streams from scratch
Now the LUNA Streams settings are loaded into the Configurator service using the migrations mechanism. The FaceStream settings are still loaded into the LUNA Configurator using the “load_dump.py” script. See installation manual for more details.
Fixed errors
FaceStream crash has been fixed that occurred when an incorrect address was specified in the “lunastreams” > “origin” parameter of the FaceStream settings.
Now the FaceStream operation is finished correctly, and the corresponding error is written to the logs.
The error has been fixed where the valid range (from 0 to 1024 pixels) was not checked for the “size_source_frame” parameter of the FaceStream settings, resulting in the possibility of setting any values for this parameter.
The error has been fixed where FaceStream would output a stack trace instead of an error if the port it was running on was busy.
Now the error Error occurred on initializing application. Description: bind: Address already in use
is displayed in the FaceStream logs.
The error has been fixed where changing the “enable_gpu_processing” setting of the FaceStream settings was not applied properly and caused a crash when the configuration was reloaded.
Fixed HTTP server crash on used port.
Fixed a bug where changing the ‘enable_gpu_processing’ option did not applied properly and lead to crush on config reloading.
Fixed a bug where invalid LunaStreams URL in lunastreams/origin lead to crash.
Fixed a bug on checking the resize source frame parameter in config. Add the checking of allowed range of parameter.
The settings migration mechanism was updated.
Service dependencies were updated.
Fixed errors
The error was fixed that caused the stream error counter to be processed incorrectly. If during the “period” the number of errors did not reach the value “max_error_count”, then the error counter was not reset, but continued to increase.
The error was fixed, due to which zero-length strings and “null” values from the “location” section of streams management settings were written to the LUNA PLATFORM event.
Fixed a bug where stream’s error counter was handled incorrectly.
Fixed a bug where empty strings and null fields of ‘location’ was treated equally.
Support for the new LUNA PLATFORM 5 authorization system was added.
New optional authorization field “event_handler > authorization” was supported in the stream creation settings, in which you need to specify “token” or “account_id”. The “event_handler” > “authorization” > “account_id” parameter must match the “account_id” parameter set when creating the stream. If the new authorization field is not filled in, the “account_id” set when creating the stream will be used.
When upgrading to new version of LUNA Streams, the database will be automatically migrated to support the new field.
Starting from this version of the FaceStream build, it is required to use the LUNA PLATFORM version at least v.5.32.0.
LUNA Streams and FaceStream service containers are now named the same when manually launched and when launched with Docker Compose.
The Python version was updated to 3.10 in the LUNA Streams and LUNA Configurator containers.
All commands related to using Python inside containers were updated in the documentation, namely, commands like “python3.9” were replaced with “python3”.
New documents “FS_Docker_Upgrade_Manual_Eng.pdf” and “FS_Docker_Upgrade_Manual_Eng.html” were added to the distribution package, describing the upgrade process from the previous build of LUNA Streams and FaceStream services, as well as LUNA PLATFORM services launched in accordance with the FaceStream documentation.
Fixed errors
The error was fixed where the “jpeg_quality_level” setting was not working correctly on the GPU.
With the “request_type” = “jpeg” value, only the first source frame was compressed. The rest of the source frames were not compressed.
With the “request_type” = “json” value, the source frames were not compressed, but all warps were compressed, except for the first one.
Service dependencies were updated.
Stream description length has been increased from 128 to 512 symbols, create stream for details.
Support for older python versions was dropped. Python 3.10 is required now.
FaceStream changes
DROI calculation mechanism was changed.
Previously, DROI represented coordinates relative to ROI, taking into account image rotation. DROI now represents coordinates relative to the original frame.
The FaceStream stream processing report regularly sent to LUNA Streams was expanded with new “last_frame” and “live” fields containing the URLs of the image’s last frame and the live stream.
These URLs can be received using specific requests to the LUNA Streams service (see below).
FaceStream operation was accelerated and GPU memory consumption was reduced by limiting the estimation of five landmarks in cases where such estimation is not required.
Displaying the IP address of the server running FaceStream was added to the FaceStream logs.
LUNA Streams service changes
Migration was added to LUNA Streams, in which all existing streams will be updated in accordance with the new DROI calculation mechanism (see FaceStream changes above).
When updating FaceStream, it is necessary to update LUNA Streams.
A “include_luna_services” parameter was added to “get health” request.
The “include_luna_services” parameter allows you to enable or disable healthcheck for the LUNA PLATFORM services that the LUNA Streams service depends on. If this option is enabled, then additional requests are sent to the “/healthcheck” resources of these services.
Two new resources were added - “get last frame preview” and “get live preview” that allow you to get the values of the “last_frame” and “live” fields from the FaceStream stream processing report (see FaceStream changes above).
New parameters “preview” > “last_frame” and “preview” > “live” were added to the “get streams logs” request.
The automatic restart of the stream now has its own statuses (the “status” parameter).
The automatic restart statuses are listed below:
The status of the automatic restart can be obtained using the “get stream” request.
The DROI calculation mechanism was changed.
Previously, DROI represented coordinates relative to the ROI, taking into account the rotation of the image.
From now on, DROI represents coordinates relative to the full frame.
All existing streams will be updated according to the new DROI calculation mechanism.
The live
and last_frame
stream preview were added.
See last frame preview and last frame preview for details.
The stream autorestart status was added. The stream autorestart mechanism was updated - the fatal error concept was added.
For more information see autorestart section of get stream response and streams automatic restart.
The include_luna_services parameter was added for healthcheck resource.
New field preview
for stream feedback was added. See stream feedback for details.
The script for uploading FaceStream and LUNA Streams dumps to the LUNA Configurator service can now be run on Python 2.x and Python 3.x versions. To run the script on the Python 2.x version, you must use the “python” command. To run the script on Python 3.x, you must use the “python3” command. Also, the script for loading dumps was added to Docker Compose. Now, to use the “start_facestream.sh” script, you only need to have running LUNA PLATFORM containers and an activated license.
Support for Liveness on the GPU was added.
FaceStream fixed errors
The error was fixed where the “Streams total” counter in the FaceStream logs counted streams with a “failure” status.
The error was fixed that was causing FaceStream crashes in a multi-threaded environment.
The error was fixed where image rotation, controlled by the “rotation” setting, was performed clockwise on the CPU and counterclockwise on the GPU. Rotation is now clockwise on both types of processors.
The error was fixed, in which FaceStream took more streams into processing than specified in the license key.
The FaceStream crash was fixed that occurred when setting a non-existent tag in the “logging” > “tags” field in the FaceStream settings.
The FaceStream crash was fixed that occurred when running on a GPU with “size_source_frame” from FaceStream settings set to non-zero.
LUNA Streams service changes
Significantly accelerated performing of requests to “get stream” and “get streams” resources with a large number of entries in the stream log tables of the LUNA Streams database.
New resource “delete streams logs” was added, which enables you to delete the streams logs older than the date specified in the request (the "log_time__lt" parameter), excluding the last entry in the log.
The getStream and getStreams requests have been accelerated in the presence of a large number of logs.
Added possibility to delete streams logs. See delete streams logs for more info.
New field authorization
for stream event handler was added.
Now the URL for sending the source frames of faces and bodies is specified in the “frame_store” parameter. Previously, the URL from the “origin” parameter was used to send the source frames of the bodies, and the URL to send the source frames of the faces was set in the “frame_store” parameter.
Now, for failed requests to LUNA PLATFORM services, the FaceStream logs will show the “Luna-Request-Id” value generated by FaceStream.
Based on the generated “Luna-Request-Id” value, you can find an error in the logs of LUNA PLATFORM 5 services.
A script “load_dump.py” was added to the “example-docker/luna_configurator/dumps/” directory of the distribution package, which enables you to upload FaceStream and LUNA Streams settings to LUNA Configurator.
Example of command to upload settings for LUNA Streams service: python -m load_dump --dump-file=streams_dump.json --luna-config=http://127.0.0.1:5070/1
This method is taken as the main one in the FaceStream installation manual (see the “Uploading settings to LUNA Configurator” section).
FaceStream fixed errors
The crash was fixed that occurred due to incorrect handling of server errors when sending the source frame.
The error was fixed, which led to a significant increase in RAM consumption during FaceStream operation.
LUNA Streams service changes
New parameters “duration” and “progress” were added to the “video_info” field of the response to a request to get information about stream (“get stream” request).
The “duration” parameter displays the total duration of the videofile in seconds, and the “progress” parameter displays the current videofile progress from 0 to 1, where 0 is the beginning of the videofile, 1 is the end of the videofile.
The “send_detection_path” parameter was removed from the FaceStream settings. Now the ability to send detections with the coordinates of the human body is controlled by the numerical value of the “minimal_body_track_length_to_send” parameter. If the “minimal_body_track_length_to_send” value is “0”, then no detections will be sent.
Now, when sending a source frame to the LUNA Image Store using the “frame_store” parameter, the account ID will also be sent. The <source_frame_id>.meta.json
file containing the “account_id” parameter will be saved in the folder with the source frame in the bucket.
LUNA Streams service changes
The error was fixed, in which after transferring the stream to the “pause” status, the status “in_progress” was displayed in the stream logs (see the “get streams logs” request).
The error was fixed that allowed setting very large values for parameters in “create stream” and “Post feedback on streams” requests.
Service dependencies were updated.
New fields duration
and progress
for stream feedback was added.
Fixed bug which allowed to set very large values to stream data, stream policies and stream feedback.
See createStream and postStreamsFeedback documentation for more info.
Fixed bug which caused extra entry to appear in stream logs after pausing the stream. See documentation for more info.
The ability to get the detection time (in seconds) of a face or body relative to the start of the video was added.
This time is stored in the “detect_ts” field of the LUNA PLATFORM event.
Getting face or body detection time is available only for the “videofile” source type.
LUNA Streams version was updated.
FaceStream fixed errors
The error was fixed where the stream would not go into “failure” status when invalid parameters were used.
Getting the “detections_count” metric on the GET “/metrics” request was fixed.
The “show_window” parameter, deprecated since version 5.0.6, was removed from the FaceStream settings.
The error was fixed that occurs on some graphics cards when previewing a stream using the “/streams/preview/{id}” resource while using FaceStream with GPU.
LUNA Streams service changes
New “endless” parameter was added to the “create stream” and “put stream” requests of the LUNA Streams service, which enables you to control the restart of the stream when a network error is received (the error is determined by the system as an eof
(end-of-file) marker).
The parameter is available only for the “udp” and “tcp” source types.
If the endless
parameter takes the value true
, then in case of receiving eof
and successful reconnection, the processing of the stream will continue. If all reconnection attempts failed (see the “healthcheck” section), then the stream will take the “failure” status. If the parameter takes the value false
, then the processing of the stream will not continue and the status of the stream will take the “done” status.
When using a video file as a “tcp” or “udp” source, it is assumed to use the value false
. This will avoid re-processing an already processed fragment of the video file when receiving eof
. If, when using a video file, the value of the endless
parameter is true
, then after the processing is completed, the video file will be processed from the beginning.
LUNA Streams service fixed errors
Two new launching keys for FaceStream were added - “CONFIG_RELOAD” and “PULLING_TIME”.
The “CONFIG_RELOAD” launching key enables checking for changes in the “FACE_STREAM_CONFIG” section of the LUNA Configurator service and takes the following values:
The “PULLING_TIME” launching key sets the period for receiving new parameters from the “FACE_STREAM_CONFIG” section of the LUNA Configurator service in the range [1…3600] sec. Used in conjunction with the “CONFIG-RELOAD” tag. The default value is “10”.
Launching keys are also available in FaceSteam mode with configuration files.
New parameter “min_body_size_threshold” was added to the “filtering” section in the streams management settings.
The parameter sets the body detection size, less than which it will not be sent for processing. It is calculated as the square root of the product of the body detection height (in pixels) by its width (in pixels).
Example: min_body_size_threshold = sqrt (64*128) = 90.5
If the value is “0”, then filtering of body detection by size will not be performed.
Fixed errors
The address of the source image is now recorded in the “detections” > “image_origin” field of the LUNA PLATFORM event. Previously, this address was recorded in the “face” > “user_data” field.
The error was fixed, in which the “limit” parameter (maximum number of the streams to return) was not processed in the request to get streams from the queue (“/1/streams/processing/queue” resource) when filtering by group.
SDK was updated to version 5.6.0.
The ability to group streams was added. Grouping is intended to combine streams with multiple cameras into logical groups. For example, you can group streams by territorial characteristic.
Any stream can belong to several groups, as well as none.
The group is created using the “create group” request. To create a group, you need to specify the required parameters “account_id” and “group_name”. If necessary, you can specify a description of the group.
Stream can be linked to a group in two ways:
using the “group_name” or “group_id” parameters during stream creation (“create stream” request).
using the “linker” request. In the request, you should specify the streams IDs and the group to which they need to be linked. You can also unlink streams from a group using this request.
If the stream was linked to a group, then the “get stream” or “get streams” requests will show the group in the “groups” field.
The ability to transfer local track timestamps to events created by LUNA PLATFORM 5 was added.
For an event created by a static handler, the “create_time” and “end_time” parameters are now recorded.
For an event created by a dynamic handler, the “end_time” parameter is now recorded.
For the bestshots (samples), the “detect_time” parameter is now recorded.
Now, when the “send_detection_path” parameter is enabled, the corresponding detection with body coordinates will be sent along with the bestshot by default. Previously, such a detection could not be sent.
The parameter “preferred_program_stream_frame_width” parameter was added to the streams management settings, intended to work with protocols that imply the presence of several channels with different bitrates and resolutions (for example, HLS).
If the stream has several such channels, then this parameter will enable you to select from all the channels of the whole stream the channel whose frame width is closer to the value specified in this parameter.
For example, there are 4 channels whose frame widths are 100, 500, 1000 and 1400. If the parameter “preferred_program_stream_frame_width” is equal to “800”, then a channel with a frame width of 1000 will be selected.
If the stream has only one channel, this parameter will be ignored.
This parameter is used only for “tcp” or “udp” types.
The following new parameters was added to the “lunastreams” section of FaceStream settings:
request_stream_period - parameter sets the time period between requests to receive new streams from LUNA Streams in the range from 0.1 to 3600 seconds.
send_feedback_period - parameter sets the time period between sending reports on processed streams to LUNA Streams in the range from 1.0 to 3600 seconds. The value of this parameter should not exceed the value of the “STREAM_STATUS_OBSOLETING_PERIOD” parameter, set in the LUNA Streams service settings.
max_feedback_delay - parameter sets the maximum report sending delay in the range from 1.0 to 3600 seconds. If the report has not been sent within the given time, then FaceStream will stop processing the current streams. The value of this parameter should not be less than the value of the parameter “send_feedback_period” and should not exceed the value of the parameter “STREAM_STATUS_OBSOLETING_PERIOD”, set in the LUNA Streams service settings.
docker-compose.yml
file that caused the error “host network_mode is incompatible with port_bindings” when running a Docker Compose script was fixed.The principle of creating streams was changed. Previously, settings for streams were specified in the “input.json” configuration file or in the “FS_CONFIG” section of the Configurator. Now the settings for streams are set in the body of the JSON format and sent using an HTTP request to the LUNA Streams service (see the “create stream” request in the “StreamsReferenceManual.html” document of the distribution).
After sending an HTTP request with the given parameters to the LUNA Streams service, the parameters are added to the LUNA Streams database under the unique identifier “stream_id”. Depending on the number of available streams in the license (this information comes from the LUNA Licenses service), the stream parameters enters the internal queue, where it is in the “pending” status until FaceStream takes them from the queue for further processing (see the detailed description of the service in the “Interaction of FaceStream with LUNA Streams” in the administrator manual).
Licensing changes:
The service can process a certain number of streams simultaneously. The number of such streams is set in the LUNA PLATFORM 5 license key parameter, which determines the streams number for LUNA Streams operation. Thus, starting from version 5.1.0, FaceStream is licensed using the LUNA PLATFORM 5 key.
To launch LUNA Streams and FaceStream, you need at least three additional LUNA PLATFORM components:
The InfluxDB OSS 2 container is also launched by default. Using the Influx database, LUNA PLATFORM services are monitored. If necessary, monitoring can be disabled.
If the LUNA PLATFORM components are not launched and the LUNA PLATFORM license is not activated, then the following steps should be performed before launching FaceStream:
If the LUNA PLATFORM components are launched and the LUNA PLATFORM license is activated, then the following steps should be performed before launching:
For a description of how to perform these steps, see the FaceStream installation manual.
The ability to deploy FaceStream using a Docker Compose script was added. The script enables you to automatically launch and configure LUNA Streams and FaceStream. Before running the script, the following steps should be performed:
Description of errors returned by the LUNA Streams and LUNA Licenses services, as well as general errors that may occur when interacting with the LUNA PLATFORM, was to the documentation.
The “lunastreams” section was added to the FaceStream settings (“FACE_STREAM_CONFIG” section of the LUNA Configurator service), enabling you to specify the address of the LUNA Streams service used (“origin” parameter), its API version (“api_version” parameter) and the upper bound on the number of FS streams (" max_number_streams"). If LUNA Streams service is launched on another server, then you should specify the current address of the service before launching FaceStream. See the FaceStream administrator manual for details.
The names and structure of streams management settings was reworked. All settings are now divided into logical blocks. The following settings was renamed:
The method of passing data to generate an event was reworked. Previously, to generate a LUNA PLATFORM event, the “output” > “url” parameter was used, in which, when working with faces, it was necessary to specify the address to the “/events” resource in the format http://<luna_adress>:<port>/6/handlers/<handler_id>/events
, where <handler_id>
is the LP handler identifier. When working with bodies, separate parameters “luna_human_handler_id” and “luna_dynamic_human_handler_id” were used, and the “url” field was filled in the format http://<luna_adress>:<port>/6/
. Now the handler ID is set in the “event_handler” > “bestshot_handler”/“detection_handler” > “handler_id” group, where “bestshot_handler” is a static handler for a face or body, “detection_handler” is a dynamic “handler_id” LUNA PLATFORM for working with bodies (ex. “luna_dynamic_human_handler_id”). The LUNA API service address and API version are now set separately. Below is an example of passing data to generate an event when static and dynamic handlers are set for the body.
"event_handler": {
"origin": "http://127.0.0.1:5000",
"api_version": 6,
"bestshot_handler": {
"handler_id": "ee4c42b6-23ae-410e-a2aa-a4220e64ba4b"
},
"detection_handler": {
"handler_id": "426542d6-5509-4e5b-8a01-e2abd5c0a8c6"
}
}
If it is necessary to process a face, then “handler_id” should be set with parameters for processing a face in the “bestshot_handler” group, and the “detection_handler” group is optional.
See the FaceStream administrator manual for details on stream management settings.
Support for normal FaceStream launch mode was dropped. Now you can launch FaceStream only in server mode.
Information on launching FaceStream in server mode with configuration files was moved to the administrator manual (see “Use FaceStream with configuration files” section).
The default logging parameters was changed in the FaceStream settings, in the “Logging” section:
SDK was updated to version 5.5.0.
The ability to get the current version of FaceStream using the GET “/version” request was added (see the detailed information in the “FaceStreamApi.html” document).
Fixed an error where when running FS in server mode with Configurator, the “save-debug-info” parameter from the trackengine configuration was not used.
Fixed an error where the source frame without detections with human body coordinates (detection_path) was not sent with the “send_source_frame” parameter set.
Fixed an error where sending “track_id” to faces did not work, which is why the value of “track_id” was not recorded in the LUNA PLATFORM event.
Fixed an error when using the mouth estimator. The error could occur when using two streams, when the first one used this estimator, and the second one did not.
FaceStream 5 is now distributed in Docker containers only. CentOS 8 is used for running FaceStream inside the container.
FaceStream distribution package was updated. Now it includes documentation and configuration files required for the FaceStream launch.
The “LP_Docker_Installation_Manual_Eng” document was added. It describes FaceStream launching in Docker container. The information is moved from the administrator manual. The document is provided in PDF and HTML formats.
SDK was updated to version 5.4.1.
Support for working with bodies was added. FaceStream searches for bodies in the stream and tracks them until they leave the frame or are overlapped.
There are additional settings for using FaceStream with bodies. Starting with FaceStream 5.0.6, all settings are divided into common and individual settings for faces and bodies in the documentation.
To enable body tracking, you need to change the “use-face-detector” parameter of the “trackengine.conf” configuration file to “use-body-detector” and also set individual settings for stream sources for bodies (see the relevant chapters).
The logic of creating events (sending bestshots and processing them by LUNA PLATFORM) for the body differs from the logic of creating an event for faces (see the administrator manual for details).
All parameter names in configuration files were unified. For example, the names “frame-processing-mode” and “real_time_mode_fps” were used. Now all the names look the same: “frame_processing_mode” and “real_time_mode_fps”.
Error messages were updated. Now they are more detailed and contain the path to the erroneous parameter.
Synchronous waiting for a response from LUNA PLATFORM (also affects the performance of the working with bodies) was removed.
The server time will now be sent with the bestshot. Previously, the device time was sent. This feature applies to all types of sources (video streams, video files and sets of images).
Support for LUNA PLATFORM API versions 4 and 5 was discontinued. FaceStream cannot send data to LUNA PLATFORM 3, LUNA PLATFORM 4, Backport 3, and Backport 4 starting with this version. Sending images is only possible in LUNA PLATFORM 5 (API version 6).
Using the “show_window” parameter and its associated parameters is no longer possible.