Skip to content

LUNA PASS API#

The general principle of operation of LUNA PASS API is as follows:

  • a frame comes to the input;
  • LUNA SDK estimates and checks the frame;
  • plugins are executed sequentially if they are enabled:
  • FaceMatcher plugin;
  • VirtualCameraChecker plugin;
  • Bestshot plugin;
  • Luna3DescriptorQualityCheck plugin;
  • AggregateLiveness plugin;
  • ActiveLiveness plugin;
  • the result of the plugins work is analyzed.

Frame estimation process#

The frame estimation process is the sequential execution of checks. A description of the checks is presented below (Table 4).

Table 4. Checks for frame estimation process

Check

Internal checks

Multiface

— whether the face is:

  • the only one in the frame;

  • the largest (main) in the frame;

  • the face closest to the center

Image quality

  • blur;

  • exposure;

  • darkening;

  • glare on the face;

  • contrast

The size and position of the face in the frame

— indentation of the face border from the frame border:

  • top;

  • at the bottom;

  • to the right;

  • on the left;

— frame size:

  • in width;

  • in height;

— the size of the face in the frame:

  • in width;

  • by height

Head position

  • head tilt angle to the right/left (roll);

  • head tilt angle up/down (pitch);

  • head rotation angle right/left (yaw)

Eyes status

  • eyes open;

  • eyes closed

Mouth status

  • the mouth is open;

  • a smile is detected;

  • the mouth is closed

Glasses

  • no glasses;

  • glasses are present;

  • sunglasses

AGS (check the overall image quality metric)

\—

Liveness

  • Liveness estimation;

  • estimation of the quality of Liveness

Deepfake
  • Deepfake estimation with specified accuracy

If a frame does not pass any of the checks, then the next frame is analyzed. The user sees the corresponding prompts on the page with LUNA PASS UI.

FaceMatcher plugin#

The FaceMatcher plugin is enabled in the config.toml file. For this, set the parameter disallowMultipleFacesInSession = 1 in the LIVENESS configuration parameters. It allows you to terminate the session if several main faces are detected on two frames within one session.

Description of the general working principle of the FaceMatcher plugin:

If the first frame arrives, Facematcher remembers photo of face. If the second frame arrives, Facematcher compares photo of face from the received frame with the photo of the face from the previous frame:

  • if faces on the photos from the two frames are different, then no further checks are carried out;

  • if faces on the photos from the two frames are the same, then the frame is sent for further checks.

VirtualCameraChecker plugin#

VirtualCameraChecker plugin is enabled in the config.toml file. It allows you to detect when a user is using a virtual camera instead of a real device camera. Virtual camera detection data is recorded to the logs.

Bestshot plugin#

The Bestshot plugin is enabled in the config.toml file. It allows you to select as bestshot the frame with the highest Liveness estimate by comparing the Liveness estimate with the values on the previous frames. A frame in which both eyes are closed cannot be selected as a bestshot, unless the use of sunglasses is permitted.

The working principle of the Bestshot plugin is shown in the diagram (Figure 9). The description of the general working principle of the plugin is presented below (Table 5).

Bestshot plugin
Figure 9. Bestshot plugin

Table 5. Description of the general working principle of the Bestshot plugin

Step

Description

(1)

The Liveness value of the current frame is checked: liveness_score >= LIVENESS_SCORE_THRESHOLD

If the check of the current frame for Liveness is not passed, the process is terminated and the current frame is not used further

If the check of the current frame for Liveness is passed, then proceed to the next step

(2)

The Liveness value of the current frame is compared with the Liveness value of the frame that was previously selected in this plugin as the bestshot

If the Liveness value of the current frame is lower than that of the existing bestshot, the process is terminated and the current frame is not used further

If the current frame is the first frame that passed the Liveness quality threshold, then it automatically becomes the bestshot

(3)

If the Liveness value of the current frame is higher than the existing bestshot selected by this plugin, then the current frame becomes the bestshot

Luna3DescriptorQualityCheck plugin#

The Luna3DescriptorQualityCheck plugin is in the config.toml file. It allows you to send a request to LUNA PLATFORM 3 to extract the descriptor and check its quality.

The Luna3DescriptorQualityCheck plugin only works with LUNA PLATFORM version 3.x.

The plugin allows you to reduce the number of filtering errors by descriptor quality on the LUNA PLATFORM 3 side.

The working principle of the Luna3DescriptorQualityCheck plugin is shown in the diagram (Figure 10). The description of the general working principle of the plugin is presented below (Table 6).

Luna3DescriptorQualityCheck plugin
Figure 10. Luna3DescriptorQualityCheck plugin

Table 6. Description of the general principle of operation of the Luna3DescriptorQualityCheck plugin

Step

Description

(1)

Checks whether the Bestshot plugin is enabled

If the Bestshot plugin is enabled, proceed to step 6

If the Bestshot plugin is disabled:

(2)

If the Bestshot plugin is disabled, a Liveness quality score of the current frame is performed: liveness_quality_score >= LIVENESS_QUALITY_SCORE_THRESHOLD

If the check of the current frame for Liveness quality is not passed, the process is terminated and the current frame is not used further

(3)

If the Liveness quality score of the current frame is passed, the Liveness value check is performed: comparing the Liveness values of the current frame and the existing bestshot in this plugin

If the Liveness of the current frame is less than the Liveness of the bestshot of this plugin, the process is terminated and the current frame is not used further

If the current frame is the first frame to pass the Liveness quality score (in the current plugin or Bestshot plugin), it becomes the bestshot

(4)

If the current frame has a Liveness greater than the Liveness of the bestshot in this plugin, a request is made to LUNA PLATFORM 3 to extract the descriptor from the current frame and check the quality of its descriptor

If validation in LUNA PLATFORM 3 fails (the descriptor quality is not enough), the process is terminated and the current frame is not used any further

(5)

If validation in LUNA PLATFORM 3 is passed (descriptor quality is normal), the current frame becomes the bestshot in this plugin

If the Bestshot plugin is enabled:

(6)

A check is performed to see if the bestshot is selected in the Bestshot plugin

If the bestshot in the Bestshot plugin is not selected, the current frame is passed on, no changes are made to the current plugin, and the process is terminated

(7)

If the bestshot in the Bestshot plugin is selected, its Liveness value is compared to the Liveness value of the bestshot of this plugin

(8)

If the Liveness of the bestshot from Bestshot is lower, the bestshot from this plugin becomes the bestshot in the Bestshot plugin

(9)

If the Liveness of the bestshot from Bestshot is higher, this frame is sent to LUNA PLATFORM 3 for descriptor extraction and checking the quality of its descriptor

If the bestshot from Bestshot did not pass validation in LUNA PLATFORM 3 (descriptor quality was not enough), the bestshot from this plugin becomes the bestshot in the Bestshot plugin

(10)

If the bestshot from Bestshot passed validation in LUNA PLATFORM 3 (descriptor quality is normal), then the bestshot from Bestshot becomes the bestshot in this plugin

AggregateLiveness plugin#

The AggregateLiveness plugin is in the config.toml file. It allows you to calculate the aggregate Liveness value over multiple frames.

The working principle of the AggregateLiveness plugin is shown in the diagram (Figure 11). The description of the general working principle of the plugin is presented below (Table 7).

AggregateLiveness plugin
Figure 11. AggregateLiveness plugin

Table 7. Description of the general principle of operation of the AggregateLiveness plugin

Step

Description

(1)

Liveness quality score of the current frame is performed: liveness_quality_score >= LIVENESS_QUALITY_SCORE_THRESHOLD

If the check of the current frame for Liveness quality is not passed, the process is terminated and the current frame is not used further.

If the check of the current frame for Liveness quality is passed, then proceed to the next step

(2)

The plugin saves liveness_score1 quality values by frames within a session, the number of such values is limited by AGGREGATE_LIVENESS_FRAMES_COUNT variable.

(3)

The plugin generates an aggregated Liveness value from the values stored in step 2

ActiveLiveness plugin#

The ActiveLiveness plugin is in the config.toml file. It allows you to confirm vitality by several frames in which the user interacts with the Service: blinks, turns his head left and right along the Y axis, tilts his head up and down along the X axis. The description of the general working principle of the plugin is presented below (Table 8).

Table 8. Description of the general principle of operation of the ActiveLiveness plugin

Step

Description

(1)

The administrator selects an arbitrary set of checks*. The number of checks is specified in the settings.

(2)

If the current frame fails Liveness check, the plugin generates errors. Errors tell what the user should do to pass the verification.

(3)

If the current frame passes Liveness check, then one or more frames are skipped so that photos with half-open eyes or in the process of turning the head are not counted. The cooldownFramesCount parameter is responsible for the number of skipped frames.

The resulting frame becomes a bestshot.

ActiveLiveness checks#

Blinking

The check begins when both eyes are open, or one eye is open and the args.acceptOneEyed parameter is enabled. To pass the check, the two frames must then arrive: with eyes closed and again with open ones, or, if the args.acceptOneEyed parameter is enabled, a frame with one eye closed and a frame with one open.

Head turn

To start checking, the head turn angle must be less than the value of args.yawStartDeg. Next, the two frames must arrive: with a turn angle value greater than args.yawThresholdDeg and with a turn angle value less than args.yawStartDeg.

Head tilt

To start checking, the head tilt angle must be less than the value of args.pitchStartDeg. Next, the two frames must arrive: with a tilt angle value greater than args.pitchThresholdDeg and with a tilt angle value less than args.pitchStartDeg.

For more information about the values of the turn and tilt angles, see configuration parameters of the ActiveLiveness plugin

Results processing#

The general working principle of LUNA PASS API is shown in the diagram (Figure 12). The description of the general working principle is presented below (Table 9).

General principle of operation of LUNA PASS API
Figure 12. General principle of operation of LUNA PASS API

Table 9. Description of the general principle of operation of LUNA PASS API

Step

Description

(0)

The LUNA PASS API gets the frame

(1)

LUNA PASS API evaluates the frame using the LUNA SDK and compares the resulting values with thresholds. More details in the "Frame estimation process"

(2)

If the FaceMatcher plugin is enabled, processing is performed according to the process described above

(3)

If the VirtualCameraChecker plugin is enabled, then it checks whether the user is using a virtual camera instead of the real device camera

(4)

If the Bestshot plugin is enabled, processing is performed according to the process described above

(5)

If the Luna3DescriptorQualityCheck plugin is enabled, processing is performed according to the process described above

(6)

If the AggregateLiveness plugin is enabled, then processing is performed according to the process described above

(7)

If the ActiveLiveness plugin is enabled, processing is performed according to the process described above

(8)

After performing actions in the specified plugins, the LUNA PASS API processes the obtained results.

LUNA PASS API checks if the received frame is the first one. Depending on this check, the following steps are performed:

  • If the received frame is not the first one, then go to step 9

  • If only the first frame has arrived, then go to step 11

(9)

If the received frame is not the first, then the face from the current is compared with the face from the first frame:

  • If the faces do not match, then go to step 10

  • If the faces in the two frames match, then go to step 11

(10)

Multiple principals detected within the same session, the session is closed without a successful frame.

(11)

LUNA PASS API checks if the camera is real. Depending on the results of this check, the following steps are performed:

  • If the camera is virtual, then go to step 12

  • If the camera is real, then go to step 13

(12)

The user is detected using a virtual camera instead of a real device camera, the session is closed without a successful frame.

(13)

LUNA PASS API checks: the number of frames that reached the Liveness estimation is greater than or equal to the value LIVENESS_MAX_ATTEMPS

Depending on this check, further analysis is divided into two parts:

  1. Check if plugins are enabled and send the result.

  2. Checking if plugins are enabled and trying to get additional frames for processing

If at step 6 the number of frames that reached the Liveness estimation is less than allowed, the following steps are performed:

(14)

LUNA PASS API checks if the AggregateLiveness plugin is enabled.

If the plugin is enabled, it proceeds to the next step

If the plugin is disabled, it proceeds to the step 15

(15)

LUNA PASS API checks if the aggregation window is full: the number of received Liveness values is greater than or equal to the value AGGREGATE_LIVENESS_FRAMES_COUNT

If the aggregation window is not full, the next frame is processed (go to step 1). If the aggregation window is full, then the next step is executed

(16)

LUNA PASS API checks the value of the received aggregated Liveness: the aggregated Liveness is greater than or equal to the value of LIVENESS_SCORE_THRESHOLD

If the value of the received aggregated Liveness is lower than the threshold, the next frame is processed (go to step 1)

If the value of the obtained aggregated Liveness is higher or equal to the threshold, then the transition to the next step is performed

(17)

LUNA PASS API checks the result of the Bestshot plugin

If the Bestshot plugin fails to select a successful frame, the next frame is processed (go to step 1)

(18)

If a successful frame was selected in the Bestshot plugin, the LUNA PASS API checks if the ActiveLiveness plugin is enabled

(19)

If the ActiveLiveness plugin is enabled, LUNA PASS API performs a check on the result of the ActiveLiveness plugin The session continues until the frame passes active Liveness

(20)

When the ActiveLiveness plugin results check is passed, the LUNA PASS API sends this frame to the LUNA PASS UI as a successful frame

(21)

If the ActiveLiveness plugin is enabled, the LUNA PASS API sends this frame to the LUNA PASS UI as a successful frame

(22)

LUNA PASS API checks if the Luna3DescriptorQualityCheck plugin is enabled

If the plugin is enabled, then go to the next step

If the plugin is disabled, then go to step 18

(23)

LUNA PASS API performs a check on the result of the Luna3DescriptorQualityCheck plugin

If the Luna3DescriptorQualityCheck plugin fails to select a successful frame, the next frame is processed (go to step 1)

(24)

If the Luna3DescriptorQualityCheck plugin results in the selection of a successful frame that passed the descriptor quality check, the LUNA PASS API sends this frame to the LUNA PASS UI as a successful frame

(25)

LUNA PASS API checks if the Bestshot plugin is enabled. If the plugin is enabled, then proceed to step 10

(26)

If the plugin is disabled, LUNA PASS API sends to LUNA PASS UI the first frame that passed all checks

If at step 6 the number of frames that reached the Liveness estimation is greater or equal to the allowable, the following steps are performed:

(27)

LUNA PASS API checks if the Luna3DescriptorQualityCheck plugin is enabled

If the plugin is enabled, then go to the next step

If the plugin is disabled, then go to step 24

(28)

A check for the bestshot in this plugin is performed

(29)

If the bestshot is selected in the Luna3DescriptorQualityCheck plugin, the LUNA PASS API sends that particular frame to the LUNA PASS UI as a successful frame

(30)

If the bestshot is not selected in the Luna3DescriptorQualityCheck plugin, the session is closed without a successful frame

(31)

A check for the enabled Bestshot plugin is performed

If the plugin is enabled, then go to the next step

If the plugin is disabled, then go to step 31

(32)

A a check for the bestshot in the plugin is performed

(33)

If the bestshot in the Bestshot plugin is not selected, the session is closed without a successful frame

(34)

If the bestshot in the Bestshot plugin is selected, the LUNA PASS API checks if the ActiveLiveness plugin is enabled

(35)

If the ActiveLiveness plugin is enabled, LUNA PASS API performs a check on the result of the ActiveLiveness plugin The session continues until the frame passes active Liveness

(36)

When the ActiveLiveness plugin results check is passed, the LUNA PASS API sends this frame to the LUNA PASS UI as a successful frame

(37)

If the ActiveLiveness plugin is enabled, the LUNA PASS API sends this frame to the LUNA PASS UI as a successful frame

(38)

If the Bestshot plugin is disabled, then the first frame that passed all checks is sent

(39 )

  • If the frame, that passed the checks, is the last one, then go to step 40

  • If the frame, that passed the checks, is not the last one, then all checks begin again from step 1 for the new frame

(40)

The current session is closed without a successful frame

  1. liveness_quality_score—image quality estimation for Liveness recognition. ↩