Luna SDK models and dependencies
Feature name and description | Required algorithms and used structures | Best Shot Selection Functionality | Attributes Estimation Functionality | Emotion Estimation Functionality | Face Features Extraction Functionality | Liveness Check Functionality | Face Recognition Functionality | Human Detection Functionality
--- | --- | --- | --- | --- | --- | --- | --- | ---
Face detection NN versions 1 and 2 (<detector_type>) First, second and third NN should be launched consequently (for v1 or v2 respectively) | Structure(s): IDetector Plan(s): FaceDet_<detector_type>_first_<device>.plan FaceDet_<detector_type>_second_<device>.plan FaceDet_<detector_type>_third_<device>.plan |  |  |  |  |  |  |
Face detection NN version 3 | Structure(s): IDetector Plan(s): FaceDet_v3_<version>_<device>.plan |  |  |  |  |  |  |
Face redetection NN version 3 | Structure(s): IDetector Plan(s): FaceDet_v3_redetect_<version>_<device>.plan |  |  |  |  |  |  |
Image quality estimation (light, dark, blur, etc.) | Structure(s): IQualityEstimator Plan(s): model_subjective_quality_<version>_<device>.plan |  |  |  |  |  |  |
BestShotQuality estimation Performs head pose and AGS estimations | Structure(s): IBestShotQualityEstimator Plan(s): ags_angle_estimation_flwr_<device>.plan |  |  |  |  |  |  |
Face image warping | Structure(s): IWarperStruct Plan(s): No additional plans are required |  |  |  |  |  |  |
Head pose estimation (deprecated) This NN is still used for Atlas NPU only | Structure(s): IHeadPoseEstimator Plan(s): angle_estimation_flwr_<device>.plan |  |  |  |  |  |  |
AGS estimation (deprecated) This NN is still used for Atlas NPU only | Structure(s): IAGSEstimator Plan(s): ags_estimation_flwr_<device>.plan |  |  |  |  |  |  |
Extraction (NN version 54 backend) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn54b_<device>.plan |  |  |  |  |  |  |
Extraction (NN version 54 mobilenet) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn54m_<device>.plan |  |  |  |  |  |  |
Extraction (NN version 56 backend) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn56b_<device>.pland |  |  |  |  |  |  |
Extraction (NN version 56 mobilenet) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn56m_<device>.pland |  |  |  |  |  |  |
Extraction (NN version 57 backend) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn57b_<device>.plan |  |  |  |  |  |  |
Extraction (NN version 58 backend) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn58b_<device>.plan |  |  |  |  |  |  |
Matching | Structure(s): IDescriptorMatcher Plan(s): No additional plans are required |  |  |  |  |  |  |
Basic attributes estimation (age, gender) | Structure(s): IAttributeEstimator Plan(s): attributes_estimation_<version>_<device>.plan |  |  |  |  |  |  |
Child estimation | Structure(s): IChildEstimator Plan(s): childnet_estimation_flwr_<device>.plan |  |  |  |  |  |  |
Credibility check estimation | Structure(s): ICredibilityCheckEstimator Plan(s): credibility_Check_<device>.plan |  |  |  |  |  |  |
Color/Monochrome Estimation | Structure(s): IBlackWhiteEstimator Plan(s): No additional plans are required |  |  |  |  |  |  |
Emotions estimation | Structure(s): IEmotionsEstimator Plan(s): emotion_recognition_<version>_<device>.plan |  |  |  |  |  |  |
Mouth estimation | Structure(s): IMouthEstimator Plan(s): mouth_estimation_<device>.plan |  |  |  |  |  |  |
Glasses estimation | Structure(s): IGlassesEstimator Plan(s): glasses_estimation_flwr_<device>.plan |  |  |  |  |  |  |
Eyes state estimation (eyes status and keypoints for irises and eyelids) | Structure(s): IEyeEstimator Plan(s): eyes_estimation_flwr8_<device>.plan |  |  |  |  |  |  |
Eyes state estimation using IR cameras (eyes status and keypoints for irises and eyelids) | Structure(s): IEyeEstimator Plan(s): eyes_estimation_ir_<device>.plan |  |  |  |  |  |  |
Eyes state (lightened version, returns eyes state only) | Structure(s): IEyeEstimator Plan(s): eye_status_estimation_flwr_<device>.plan |  |  |  |  |  |  |
Gaze direction estimation | Structure(s): IGazeEstimator Plan(s): gaze_<version>_<device>.plan |  |  |  |  |  |  |
Gaze direction estimation using IR cameras | Structure(s): IGazeEstimator Plan(s): gaze_ir_<version>_<device>.plan |  |  |  |  |  |  |
Face overlap | Structure(s): IOverlapEstimator Plan(s): overlap_estimation_flwr_<device>.plan |  |  |  |  |  |  |
OrientationEstimator Orientation estimator for images rotation | Structure(s): IOrientationEstimator Plan(s): orientation_<device>.plan |  |  |  |  |  |  |
Precise 68 keypoints estimation (Precise version is available in Server and Jetson distributions only!) | Structure(s): Landmarks Plan(s): LNet_precise_<version>_<device>.plan |  |  |  |  |  |  |
Precise 68 keypoints estimation for IR cameras (Precise version is available in Server and Jetson distributions only!) | Structure(s): Landmarks Plan(s): LNet_ir_precise_<version&_<device>.plan |  |  |  |  |  |  |
Fast 68 keypoints estimation (fast version is available in Android Embedded edition only!) | Structure(s): Landmarks Plan(s): LNet_fast_<version>_<device>.plan |  |  |  |  |  |  |
Fast 68 keypoints estimation for IR cameras (fast version is available in Android Embedded edition only!) | Structure(s): Landmarks Plan(s): LNet_ir_fast_<version&_<device>.plan |  |  |  |  |  |  |
5 keypoints estimation | Structure(s): Landmarks Plan(s): slnet_<version&_<device>.plan |  |  |  |  |  |  |
Depth Liveness | Structure(s): ILivenessDepthEstimator Plan(s): depth_estimation_flwr8_<device>.plan |  |  |  |  |  |  |
Liveness flow (model_1 and model_2 are launched consequently) | Structure(s): ILivenessFlowEstimator Plan(s): faceflow_model_1_<device>.plan faceflow_model_2_<device>.plan |  |  |  |  |  |  |
IR Liveness universal | Structure(s): ILivenessIREstimator Plan(s): ir_liveness_universal_<device>.plan |  |  |  |  |  |  |
IR Liveness for Ambarella | Structure(s): ILivenessIREstimator Plan(s): ir_liveness_ambarella_<device>.plan |  |  |  |  |  |  |
Head and shoulders Liveness Head check and shoulders check models | Structure(s): IHeadPoseAndShouldersLivenessEstimator Plan(s): hs_shoulders_liveness_estimation_flwr_<device>.plan hs_head_liveness_estimation_flwr_<device>.plan |  |  |  |  |  |  |
Flying faces Liveness | Structure(s): ILivenessFlyingFacesEstimator Plan(s): flying_faces_liveness_v2_<device>.plan |  |  |  |  |  |  |
LivenessRGBM (models are launched consequently) | Structure(s): ILivenessRGBMEstimator Plan(s): rgbm_liveness_<device>.plan rgbm_liveness_pp_hand_frg_<device>.plan |  |  |  |  |  |  |
Human body keypoints | Structure(s): HumanLandmarks Plan(s): human_keypoints_<version>_<device>.plan |  |  |  |  |  |  |
Human body image warping | Structure(s): IHumanWarper Plan(s): no additonal plans required |  |  |  |  |  |  |
Human body detection NN | Structure(s): IHumanDetector, IHumanDetectionBatch Plan(s): human_<version>_<device>.plan |  |  |  |  |  |  |
Human body redetection | Structure(s): IHumanDetector Plan(s): human_redetect_<device>.plan |  |  |  |  |  |  |
Human body extraction | Structure(s): IDescriptor Plan(s): reid_<reid_type>_<version>_<device>.plan |  |  |  |  |  |  |

Example of the above table usage: if the user wants to find the best shots in his application, he needs to look at the "Best Shot Selection" column and see what network models and structures are required to run such functionality.The required models and structures are marked with green. Algorithms and structures marked with red are not used for the functionality.

The <version> placeholder shows that there can be several versions of the NN in distribution.

The <device> placeholder shows that the model can be used with different devices (e. g., CPU, GPU). The name of the model includes corresponding postfix (e. g., *_cpu.plan, *_gpu.plan). See "Hardware requirements" section in "FaceEngine Handbook" for information about devices supported by NN.

To decide what type of detector model to use in your specific use case (FaceDet_* networks) please refer to "FaceEngine Handbook" chapter "Face Detection Facility".

You should select one of the extraction CNNs listed in this table. See "FaceEngine Handbook" "Appendix A. Specification" for information about their performance and chapter "Descriptor Processing Facility" for information about CNNs.

You don't have to use both ordinary and mobilenet version for descriptor (cnn* networks), it's enough to choose one of these models. For details on difference see "FaceEngine Handbook" chapter "Descriptor Processing Facility".

"No additional plans are required" means that the feature is implemented without NN usage.

Luna SDK functionality
Functionality | Feature name and description | Required algorithms and used structures | Face detection NN versions 1 and 2 | Face detection NN version 3
--- | --- | --- | --- | ---
MedicalMaskDetection | Medical mask estimation | Structure(s): IMedicalMaskEstimator Plan(s): mask_clf_<version>_<device>.plan |  |
| PPEDetection | Personal Protection Equipment estimation | Structure(s): IPPEEstimator Plan(s): ppe_estimation_<version>_<device>.plan |  |
MobileLiveness | LivenessOneShotRGBE | Structure(s): ILivenessOneShotRGBEstimator Plan(s): liveness_model_<version>_<device>.plan |  |
TrackEngine | TrackEngine | Structure(s): ITrackEngine Plan(s): No additional plans are required |  |
LivenessEngine | LivenessEngine | Structure(s): ILivenessEngine Plan(s): No additional plans are required |  |
DescriptorIndex | Index creation | Structure(s): IIndexBuilder, IDenseIndex, IDynamicIndex Plan(s): No additional plans are required |  |

The table above includes separately licensed functionality and their general features.

Additional features required for the functionality utilization are shown in the columns to the right.

Luna SDK general structures
SDK CORE facilities | Required structures
--- | ---
General FaceEngine interface | IFaceEngine
Serialization interface | ISerializableObject
Archive storage interface | IArchive
Reference counted interface | IRefCounted
Settings provider | ISettingsProvider