Algorithm

Bestshot selection

The interaction of components when selecting the bestshot is shown below (Figure 2).

Interaction diagram of System components when selecting the bestshot

An explanation for the figure is presented below (Table 3).

Table 3. The description of the interaction diagram of the System components when selecting the bestshot

Step	Description
(1)	The RSE Server receives a WebSocket connection request from a client. Example request: GET ws://127.0.0.1:4444/–establishing a WebSocket connection. 0–content of the message to start the session
(2)	RSE Server sends a request to RSEngine to launch the camera (to the camera library). Depending on which camera is connected, IntelRealSense, VLS LUNA CAMERA 3D or VLS LUNA CAMERA 2D, RSE Server passes the request to the appropriate RSEngine library: RealSense2 SDK, VLS LUNA CAMERA 3D SDK, VLS LUNA CAMERA 2D SDK.
(3)	Camera library launches the camera
(4)	The camera library receives RGB, IR, Depth video streams from the camera, splits them into frames and analyzes
(5)	The camera library passes a set of frames to the RSE Server. Depending on the cs_communication parameter, different data is transferred: - cs_communication = msg-pack: all three frames (RGB, IR, Depth) is transferred - cs_communication = json: only RGB frame is transferred
(6)	RSE Server sends a request to frame processing (performed on each frame) to the VisionLabs SDK
(7)	SDK VisionLabs performs: selection of every third frame from the incoming video stream (the number of frames is not adjustable in the settings); face detection on each frame; determination of attributes and parameters of detected faces; checking threshold values of the received parameters (head position, image quality, etc.). If all checks are passed, the process continues (go to step 8). If at least one check fails, SDK VisionLabs sends a request to the camera for new frames to perform a second check as long as there is detection (return to step 4)
(8)	VisionLabs SDK performs a Liveness check and compares the resulting value of the Liveness score to the threshold one. If the received Liveness value is higher than the threshold, then the current frame becomes the bestshot. If the received Liveness value is below the threshold, SDK VisionLabs sends a request to the camera to to receive new frames to perform a second check as long as there is detection (return to step 4)
(9)	If the Liveness check is successful, the received bestshot and facial attributes are sent to the RSE Server
(10)	RSE Server converts the selected bestshot and meta-information into MessagePack and sends it to the client in an external system

Monitor camera status

The interaction of components when monitoring the state of the camera is shown below (Figure 3).

Diagram of component interaction when monitoring camera status

An explanation for the figure is presented below (Table 4).

Monitoring is started by default once every 300 seconds, you can change the duration in the camera-monitoring and camera-monitoring-delay parameters in the rsengine.conf file.

Table 4. The description of the interaction diagram of the System components when monitoring the camera state

Step	Description
(1)	RSE Server sends a request to RSEngine to launch monitoring the camera
(2)	Camera Monitoring sends a request to the camera library for camera status—RealSense2 SDK, VLS LUNA CAMERA 3D SDK or VLS LUNA CAMERA 2D SDK depending on which camera is connected IntelRealSense, VLS LUNA CAMERA 3D or VLS LUNA CAMERA 2D SDK
(3)	The camera library transfers a request for camera status
(4)	The camera library receives the camera status
(5)	The camera library passes the camera status to the camera monitoring
(6)	RSEngine transfers camera status information to RSE Server
(7)	RSE Server entries camera status data to the registry (on Windows) or to the ./logs working folder (on Ubuntu 18.04 x64, Debian 10 x64 and Armbian 23)