LUNA PASS API#

The general principle of operation of LUNA PASS API is as follows:

LUNA PASS API receives a frame as input;
appropriate estimations and checks of the frame are performed using LUNA SDK;
three plugins (if enabled) are executed sequentially:
Bestshot plugin;
Luna3DescriptorQualityCheck plugin;
AggregateLiveness plugin;
ActiveLiveness plugin;
the result of the plugins work is analyzed.

Frame estimation process#

The frame estimation process includes the following checks performed sequentially one after another (Table 4):

Table 4. Checks in the frame estimation process

Check	Internal checks
Face check according to multiface policy	Includes the following checks: whether the face the only one in the frame; whether the face is the largest (main) in the frame; whether the face closest to the center
Image quality estimation	Includes the following checks: blur; exposure; darkening; glare on the face; contrast
Checking the size and position of the face in the frame	Includes the following checks: — checking the indentation of the face border from the frame border: top; at the bottom; to the right; on the left; — frame size checks: in width; in height; — checking the size of the face in the frame: in width; by height
Head position check	Includes the following checks: head tilt angle to the right/left (roll); head tilt angle up/down (pitch); head rotation angle right/left (yaw)
Checking for eyeglasses	Includes the following checks: eyes open; eyes closed
Mouth status check	Includes the following checks: whether the mouth is open; whether a smile is detected; whether the mouth is closed
Checking for glasses	Includes the following checks: *no glasses; glasses are present; sunglasses
AGS estimation (check the overall image quality metric)	None
Liveness estimation	Includes the following checks: liveness estimate; estimation of the quality of Liveness

If any of the checks is not passed, the current frame is cut off and the next frame is analyzed. In this case the user is displayed the corresponding prompts on the page with LUNA PASS UI.

Bestshot plugin#

The Bestshot plugin is enabled separately in the settings (in config.toml file) and allows you to select as bestshot the frame with the best quality and Liveness estimate by comparing the Liveness estimate with the values on the previous frames. A frame in which both eyes are closed cannot be selected as a bestshot, unless the use of sunglasses is permitted.

The working principle of the Bestshot plugin is shown in the figure (Figure 5).

Table 5. Description of the general working principle of the Bestshot plugin

Step

Description

(1)

The Liveness value of the current frame is checked: liveness_score >= LIVENESS_SCORE_THRESHOLD

If the check of the current frame for Liveness is not passed, the process is terminated and the current frame is not used further

If the check of the current frame for Liveness is passed, then proceed to the next step

(2)

The Liveness value of the current frame is compared with the Liveness value of the frame that was previously selected in this plugin as the bestshot

If the Liveness value of the current frame is lower than that of the existing bestshot, the process is terminated and the current frame is not used further

If the current frame is the first frame that passed the Liveness quality threshold, then it automatically becomes the bestshot

(3)

If the Liveness value of the current frame is higher than the existing bestshot selected by this plugin, then the current frame becomes the bestshot

Luna3DescriptorQualityCheck plugin#

The Luna3DescriptorQualityCheck plugin is enabled separately in the settings (in the config.toml file) and allows you to send a request to LUNA PLATFORM 3 to extract the descriptor and check its quality.

The Luna3DescriptorQualityCheck plugin only works with LUNA PLATFORM version 3.x.

The plugin allows you to reduce the number of filtering errors by descriptor quality on the LUNA PLATFORM 3 side.

The working principle of the Luna3DescriptorQualityCheck plugin is shown in the figure (Figure 6).

Table 6. Description of the general principle of operation of the Luna3DescriptorQualityCheck plugin

Step	Description
(1)	Checks whether the Bestshot plugin is enabled If the Bestshot plugin is enabled, proceed to step 6
	If the Bestshot plugin is disabled:
(2)	If the Bestshot plugin is disabled, a Liveness quality score of the current frame is performed: `liveness_quality_score >= LIVENESS_QUALITY_SCORE_THRESHOLD` If the check of the current frame for Liveness quality is not passed, the process is terminated and the current frame is not used further
(3)	If the Liveness quality score of the current frame is passed, the Liveness value check is performed: comparing the Liveness values of the current frame and the existing bestshot in this plugin If the Liveness of the current frame is less than the Liveness of the bestshot of this plugin, the process is terminated and the current frame is not used further If the current frame is the first frame to pass the Liveness quality score (in the current plugin or Bestshot plugin), it becomes the bestshot
(4)	If the current frame has a Liveness greater than the Liveness of the bestshot in this plugin, a request is made to LUNA PLATFORM 3 to extract the descriptor from the current frame and check the quality of its descriptor If validation in LUNA PLATFORM 3 fails (the descriptor quality is not enough), the process is terminated and the current frame is not used any further
(5)	If validation in LUNA PLATFORM 3 is passed (descriptor quality is normal), the current frame becomes the bestshot in this plugin
	If the Bestshot plugin is enabled:
(6)	A check is performed to see if the bestshot is selected in the Bestshot plugin If the bestshot in the Bestshot plugin is not selected, the current frame is passed on, no changes are made to the current plugin, and the process is terminated
(7)	If the bestshot in the Bestshot plugin is selected, its Liveness value is compared to the Liveness value of the bestshot of this plugin
(8)	If the Liveness of the bestshot from Bestshot is lower, the bestshot from this plugin becomes the bestshot in the Bestshot plugin
(9)	If the Liveness of the bestshot from Bestshot is higher, this frame is sent to LUNA PLATFORM 3 for descriptor extraction and checking the quality of its descriptor If the bestshot from Bestshot did not pass validation in LUNA PLATFORM 3 (descriptor quality was not enough), the bestshot from this plugin becomes the bestshot in the Bestshot plugin
(10)	If the bestshot from Bestshot passed validation in LUNA PLATFORM 3 (descriptor quality is normal), then the bestshot from Bestshot becomes the bestshot in this plugin

AggregateLiveness plugin#

The AggregateLiveness plugin is enabled separately in the settings (in the config.toml file) and allows you to calculate the aggregate Liveness value over multiple frames.

The working principle of the AggregateLiveness plugin is shown in the figure (Figure 7).

Table 7. Description of the general principle of operation of the AggregateLiveness plugin

Step

Description

(1)

Liveness quality score of the current frame is performed: liveness_quality_score >= LIVENESS_QUALITY_SCORE_THRESHOLD

If the check of the current frame for Liveness quality is not passed, the process is terminated and the current frame is not used further.

If the check of the current frame for Liveness quality is passed, then proceed to the next step

(2)

The plugin saves liveness_score* quality values by frames within a session, the number of such values is limited by AGGREGATE_LIVENESS_FRAMES_COUNT variable.

(3)

The plugin generates an aggregated Liveness value from the values stored in step 2

* liveness_quality_score — image quality estimation for Liveness recognition.

ActiveLiveness plugin works#

The ActiveLiveness plugin is enabled separately in the settings (in the config.toml file) and allows you to confirm wether a person on several frames is “real” or a fraudster through user interaction: blinking, turning the head left and right along the Y axis, tilting the head up and down the axis X.

Table 8. Description of the general principle of operation of the ActiveLiveness plugin

Step

Description

(1)

The administrator selects an arbitrary set of checks*. The number of checks is specified in the settings.

(2)

If the current frame fails Liveness check, the plugin generates errors. Errors tell what the user should do to pass the verification.

(3)

If the current frame passes Liveness check, then one or more frames are skipped so that photos with half-open eyes or in the process of turning the head are not counted. The cooldownFramesCount parameter is responsible for the number of skipped frames.

The resulting frame becomes a bestshot.

*Types of checks:

Blinking — the test begins when one eye/both eyes are open and the acceptOneEyed parameter is enabled. Next, two frames should come: with eyes closed (one eye closed) and again with open ones (one open);
Turns/tilts — to start the test, the yaw and pitch angle of the head must be less than *StartDeg. Next, two frames should come: with an angle greater than *ThresholdDeg and again less than *StartDeg.

Results processing#

General principle of operation of LUNA PASS API (Figure 8).

Table 9. Description of the general principle of operation of LUNA PASS API

Step	Description
(0)	LUNA PASS API gets the frame
(1)	LUNA PASS API performs frame estimation with the help of LUNA SDK and compares the obtained values with threshold values (more details in p. 4.1)
(2)	If the Bestshot plugin is enabled, processing is performed according to the process described in p. 4.2.
(3)	If the Luna3DescriptorQualityCheck plugin is enabled, processing is performed according to the process described in p. 4.3.
(4)	If the AggregateLiveness plugin is enabled, processing is performed according to the process described in p. 4.4.
(5)	If the ActiveLiveness plugin is enabled, processing is performed according to the process described in p. 4.5.
(6)	After performing the processing in the indicated plugins (performed in a strictly specified order) LUNA PASS API processes the obtained results LUNA PASS API checks: the number of frames that reached the Liveness estimation is greater than or equal to the value `LIVENESS_MAX_ATTEMPS` Depending on this check, further analysis is divided into two parts: Check if plugins are enabled and send the result. Checking if plugins are enabled and trying to get additional frames for processing
	If at step 6 the number of frames that reached the Liveness estimation is less than allowed, the following steps are performed:
(7)	LUNA PASS API checks if the AggregateLiveness plugin is enabled. If the plugin is enabled, it proceeds to the next step If the plugin is disabled, it proceeds to the step 15
(8)	LUNA PASS API checks if the aggregation window is full: the number of received Liveness values is greater than or equal to the value `AGGREGATE_LIVENESS_FRAMES_COUNT` If the aggregation window is not full, the next frame is processed (go to step 1). If the aggregation window is full, then the next step is executed
(9)	LUNA PASS API checks the value of the received aggregated Liveness: the aggregated Liveness is greater than or equal to the value of `LIVENESS_SCORE_THRESHOLD` If the value of the received aggregated Liveness is lower than the threshold, the next frame is processed (go to step 1) If the value of the obtained aggregated Liveness is higher or equal to the threshold, then the transition to the next step is performed
(10)	LUNA PASS API checks the result of the Bestshot plugin If the Bestshot plugin fails to select a successful frame, the next frame is processed (go to step 1)
(11)	If a successful frame was selected in the Bestshot plugin, the LUNA PASS API checks if the ActiveLiveness plugin is enabled
(12)	If the ActiveLiveness plugin is enabled, LUNA PASS API performs a check on the result of the ActiveLiveness plugin The session continues until the frame passes active Liveness
(13)	When the ActiveLiveness plugin results check is passed, the LUNA PASS API sends this frame to the LUNA PASS UI as a successful frame
(14)	If the ActiveLiveness plugin is enabled, the LUNA PASS API sends this frame to the LUNA PASS UI as a successful frame
(15)	LUNA PASS API checks if the Luna3DescriptorQualityCheck plugin is enabled If the plugin is enabled, then go to the next step If the plugin is disabled, then go to step 18
(16)	LUNA PASS API performs a check on the result of the Luna3DescriptorQualityCheck plugin If the Luna3DescriptorQualityCheck plugin fails to select a successful frame, the next frame is processed (go to step 1)
(17)	If the Luna3DescriptorQualityCheck plugin results in the selection of a successful frame that passed the descriptor quality check, the LUNA PASS API sends this frame to the LUNA PASS UI as a successful frame
(18)	LUNA PASS API checks if the Bestshot plugin is enabled. If the plugin is enabled, then proceed to step 10
(19)	If the plugin is disabled, LUNA PASS API sends to LUNA PASS UI the first frame that passed all checks
	If at step 6 the number of frames that reached the Liveness estimation is greater or equal to the allowable, the following steps are performed:
(20)	LUNA PASS API checks if the Luna3DescriptorQualityCheck plugin is enabled If the plugin is enabled, then go to the next step If the plugin is disabled, then go to step 24
(21)	A check for the bestshot in this plugin is performed
(22)	If the bestshot is selected in the Luna3DescriptorQualityCheck plugin, the LUNA PASS API sends that particular frame to the LUNA PASS UI as a successful frame
(23)	If the bestshot is not selected in the Luna3DescriptorQualityCheck plugin, the session is closed without a successful frame
(24)	A check for the enabled Bestshot plugin is performed If the plugin is enabled, then go to the next step If the plugin is disabled, then go to step 31
(25)	A a check for the bestshot in the plugin is performed
(26)	If the bestshot in the Bestshot plugin is not selected, the session is closed without a successful frame
(27)	If the bestshot in the Bestshot plugin is selected, the LUNA PASS API checks if the ActiveLiveness plugin is enabled
(28)	If the ActiveLiveness plugin is enabled, LUNA PASS API performs a check on the result of the ActiveLiveness plugin The session continues until the frame passes active Liveness
(29)	When the ActiveLiveness plugin results check is passed, the LUNA PASS API sends this frame to the LUNA PASS UI as a successful frame
(30)	If the ActiveLiveness plugin is enabled, the LUNA PASS API sends this frame to the LUNA PASS UI as a successful frame
(31)	If the Bestshot plugin is disabled, then the first frame that passed all checks is sent