Luna SDK models and dependencies

Feature name and
description

Required algorithms and used structures

Best Shot
Selection
Functionality

Attributes
Estimation
Functionality

Emotion
Estimation
Functionality

Face Features
Extraction
Functionality

Liveness
Check
Functionality

Face
Recognition
Functionality

Human
Detection
Functionality

ISO
Check
Functionality
Face detection NN versions 1 and 2
(<detector_type>)
First, second and third NN
should be launched consequently
(for v1 or v2 respectively) 		Structure(s):
IDetector

Plan(s):
FaceDet_<detector_type>_first_<device>.plan
FaceDet_<detector_type>_second_<device>.plan
FaceDet_<detector_type>_third_<device>.plan
Face detection NN version 3 Structure(s):
IDetector

Plan(s):
FaceDet_v3_<version>_<device>.plan
Face redetection NN version 3 Structure(s):
IDetector

Plan(s):
FaceDet_v3_redetect_<version>_<device>.plan
Image quality estimation
(light, dark, blur, etc.)		 Structure(s):
IQualityEstimator

Plan(s):
model_subjective_quality_<version>_<device>.plan
BestShotQuality estimation
Performs head pose and AGS estimations		 Structure(s):
IBestShotQualityEstimator

Plan(s):
ags_angle_estimation_flwr_<device>.plan
Background estimation
Performs background estimation		 Structure(s):
IBackgroundEstimator

Plan(s):
background_v2_<device>.plan
EyeBrow estimation
Performs eyebrow estimation		 Structure(s):
IEyeBrowEstimator

Plan(s):
eyebrow_estimation_v2_<device>.plan
FacialHair estimation
Performs facial hair estimation		 Structure(s):
IFacialHairEstimator

Plan(s):
face_hair_v2_<device>.plan
FishEye estimation
Detects a fish eye effect on the source face image		 Structure(s):
IFishEyeEstimator

Plan(s):
fisheye_v2_<device>.plan
HeadWear estimation
Detects a headwear status and headwear type		 Structure(s):
IHeadWearEstimator

Plan(s):
head_wear_v2_<device>.plan
HumanAttributes estimation
Performs human attributes estimation		 Structure(s):
IHumanAttributeEstimator, HumanAttributeResult

Plan(s):
human_attributes_v2_<device>.plan
NaturalLight estimation
Performs natural light estimation		 Structure(s):
INaturalLightEstimator

Plan(s):
natural_light_<device>.plan
PortraitStyle estimation
Performs portrait style estimation		 Structure(s):
IPortraitStyleEstimator

Plan(s):
portrait_style_v3_<device>.plan
RedEye estimation
Performs red eye estimation		 Structure(s):
IRedEyeEstimator

Plan(s):
red_eye_v1_<device>.plan
DynamicRange estimation
Performs dynamic range estimation		 Structure(s):
IDynamicRangeEstimator

Plan(s):
No additional plans are required
Personal Protection Equipment (PPE) estimation
Performs personal protection equipment estimation		 Structure(s):
IPPEEstimator

Plan(s):
ppe_estimation_v1_<device>.plan
Face image warping Structure(s):
IWarperStruct

Plan(s):
No additional plans are required
Head pose estimation (deprecated)
This NN is still used for Atlas NPU only		 Structure(s):
IHeadPoseEstimator

Plan(s):
angle_estimation_flwr_<device>.plan
AGS estimation (deprecated)
This NN is still used for Atlas NPU only		 Structure(s):
IAGSEstimator

Plan(s):
ags_estimation_flwr_<device>.plan
Extraction (NN version 54 backend) Structure(s):
IDescriptor, IDescriptorBatch, IDescriptorExtractor

Plan(s):
cnn54b_<device>.plan
Extraction (NN version 54 mobilenet) Structure(s):
IDescriptor, IDescriptorBatch, IDescriptorExtractor

Plan(s):
cnn54m_<device>.plan
Extraction (NN version 56 backend) Structure(s):
IDescriptor, IDescriptorBatch, IDescriptorExtractor

Plan(s):
cnn56b_<device>.plan
Extraction (NN version 56 mobilenet) Structure(s):
IDescriptor, IDescriptorBatch, IDescriptorExtractor

Plan(s):
cnn56m_<device>.plan
Extraction (NN version 57 backend) Structure(s):
IDescriptor, IDescriptorBatch, IDescriptorExtractor

Plan(s):
cnn57b_<device>.plan
Extraction (NN version 58 backend) Structure(s):
IDescriptor, IDescriptorBatch, IDescriptorExtractor

Plan(s):
cnn58b_<device>.plan
Extraction (NN version 59 backend) Structure(s):
IDescriptor, IDescriptorBatch, IDescriptorExtractor

Plan(s):
cnn59b_<device>.plan
Extraction (NN version 59 mobilenet) Structure(s):
IDescriptor, IDescriptorBatch, IDescriptorExtractor

Plan(s):
cnn59m_<device>.plan
Extraction (NN version 60 backend) Structure(s):
IDescriptor, IDescriptorBatch, IDescriptorExtractor

Plan(s):
cnn60b_<device>.plan
Matching Structure(s):
IDescriptorMatcher

Plan(s):
No additional plans are required
Basic attributes estimation
(age, gender) 		Structure(s):
IAttributeEstimator

Plan(s):
attributes_estimation_<version>_<device>.plan
Child estimation Structure(s):
IChildEstimator

Plan(s):
childnet_estimation_flwr_<device>.plan
Credibility check estimation Structure(s):
ICredibilityCheckEstimator

Plan(s):
credibility_Check_<device>.plan
Color/Monochrome Estimation Structure(s):
IBlackWhiteEstimator

Plan(s):
black_white_and_ir_v1_<device>.plan
Emotions estimation Structure(s):
IEmotionsEstimator

Plan(s):
emotion_recognition_<version>_<device>.plan
Mouth estimation Structure(s):
IMouthEstimator

Plan(s):
mouth_estimation_v4_<device>.plan
Glasses estimation Structure(s):
IGlassesEstimator

Plan(s):
glasses_estimation_flwr_<device>.plan
Eyes state estimation (eyes status and
keypoints for irises and eyelids)		 Structure(s):
IEyeEstimator

Plan(s):
eyes_estimation_flwr8_<device>.plan
Eyes state estimation using IR cameras
(eyes status and
keypoints for irises and eyelids) 		Structure(s):
IEyeEstimator

Plan(s):
eyes_estimation_ir_<device>.plan
Eyes state
(lightened version, returns eyes state only)		 Structure(s):
IEyeEstimator

Plan(s):
eye_status_estimation_flwr_<device>.plan
Gaze direction estimation Structure(s):
IGazeEstimator

Plan(s):
gaze_<version>_<device>.plan
Gaze direction estimation
using IR cameras		 Structure(s):
IGazeEstimator

Plan(s):
gaze_ir_<version>_<device>.plan
Face overlap Structure(s):
IOverlapEstimator

Plan(s):
overlap_estimation_flwr_<device>.plan
OrientationEstimator
Orientation estimator for images rotation		 Structure(s):
IOrientationEstimator

Plan(s):
orientation_v2_<device>.plan
Precise 68 keypoints estimation
(Precise version is available in Server
and Jetson distributions only!)		 Structure(s):
Landmarks

Plan(s):
LNet_precise_<version>_<device>.plan
Precise 68 keypoints estimation for IR cameras
(Precise version is available in Server
and Jetson distributions only!)		 Structure(s):
Landmarks

Plan(s):
LNet_ir_precise_<version>_<device>.plan
Fast 68 keypoints estimation
(fast version is available in
Android Embedded edition only!)		 Structure(s):
Landmarks

Plan(s):
LNet_fast_<version>_<device>.plan
Fast 68 keypoints estimation for IR cameras
(fast version is available in
Android Embedded edition only!)		 Structure(s):
Landmarks

Plan(s):
LNet_ir_fast_<version>_<device>.plan
5 keypoints estimation Structure(s):
Landmarks

Plan(s):
slnet_<version>_<device>.plan
Depth Liveness Structure(s):
ILivenessDepthEstimator

Plan(s):
depth_estimation_v2_1_<device>.plan
Liveness flow (model_1 and model_2
are launched consequently)		 Structure(s):
ILivenessFlowEstimator

Plan(s):
faceflow_model_1_<device>.plan
faceflow_model_2_<device>.plan
IR Liveness universal Structure(s):
ILivenessIREstimator

Plan(s):
ir_liveness_universal_<device>.plan
IR Liveness for Ambarella Structure(s):
ILivenessIREstimator

Plan(s):
ir_liveness_ambarella_<device>.plan
Head and shoulders Liveness
Head check and shoulders check models		 Structure(s):
IHeadPoseAndShouldersLivenessEstimator

Plan(s):
hs_shoulders_liveness_estimation_flwr_<device>.plan
hs_head_liveness_estimation_flwr_<device>.plan
Flying faces Liveness Structure(s):
ILivenessFlyingFacesEstimator

Plan(s):
flying_faces_liveness_v4_<device>.plan
LivenessRGBM
(models are launched consequently)		 Structure(s):
ILivenessRGBMEstimator

Plan(s):
rgbm_liveness_<device>.plan
rgbm_liveness_pp_hand_frg_<device>.plan
Liveness OneShot
(models are launched consequently)		 Structure(s):
ILivenessOneShotRGBEstimator

Plan(s):
oneshot_rgb_liveness_<version>_model_<number>_<device>.plan
Human body keypoints Structure(s):
HumanLandmarks

Plan(s):
human_keypoints_<version>_<device>.plan
Human body image warping Structure(s):
IHumanWarper

Plan(s):
no additonal plans required
Human body detection NN Structure(s):
IHumanDetector, IHumanDetectionBatch

Plan(s):
human_<version>_<device>.plan
Human body redetection Structure(s):
IHumanDetector

Plan(s):
human_redetect_<device>.plan
Human's head detection Structure(s):
IHeadDetector, IDetectionBatch

Plan(s):
HeadDet_<version>_<device>.plan
Human body extraction Structure(s):
IDescriptor

Plan(s):
reid_<reid_type>_<version>_<device>.plan
Crowd estimation Structure(s):
ICrowdEstimator, ICrowdEstimation

Plan(s):
crowd_<version>_<device>.plan
HumanFace detection Structure(s):
IHumanFaceDetector, IHumanFaceBatch

Plan(s):
human_face_<version>_<device>.plan

Example of the above table usage: if the user wants to find the best shots in his application, he needs to look at the "Best Shot Selection" column and see what network models and structures are required to run such functionality.The required models and structures are marked with green. Algorithms and structures marked with red are not used for the functionality.

The <version> placeholder shows that there can be several versions of the NN in distribution.

The <device> placeholder shows that the model can be used with different devices (e. g., CPU, GPU). The name of the model includes corresponding postfix (e. g., *_cpu.plan, *_gpu.plan). See "Hardware requirements" section in "FaceEngine Handbook" for information about devices supported by NN.

To decide what type of detector model to use in your specific use case (FaceDet_* networks) please refer to "FaceEngine Handbook" chapter "Face Detection Facility".

You should select one of the extraction CNNs listed in this table. See "FaceEngine Handbook" "Appendix A. Specification" for information about their performance and chapter "Descriptor Processing Facility" for information about CNNs.

You don't have to use both ordinary and mobilenet version for descriptor (cnn* networks), it's enough to choose one of these models. For details on difference see "FaceEngine Handbook" chapter "Descriptor Processing Facility".

"No additional plans are required" means that the feature is implemented without NN usage.


Luna SDK functionality

Functionality

Feature name and
description

Required algorithms and used structures

Face detection NN
versions 1 and 2

Face detection NN
version 3
MedicalMaskDetection Medical mask estimation Structure(s):
IMedicalMaskEstimator

Plan(s):
mask_clf_<version>_<device>.plan
PPEDetection Personal Protection Equipment estimation Structure(s):
IPPEEstimator

Plan(s):
ppe_estimation_<version>_<device>.plan
MobileLiveness LivenessOneShotRGBE Structure(s):
ILivenessOneShotRGBEstimator

Plan(s):
liveness_model_<version>_<device>.plan
TrackEngine TrackEngine Structure(s):
ITrackEngine

Plan(s):
No additional plans are required
LivenessEngine LivenessEngine Structure(s):
ILivenessEngine

Plan(s):
No additional plans are required
DescriptorIndex Index creation Structure(s):
IIndexBuilder, IDenseIndex, IDynamicIndex

Plan(s):
No additional plans are required
ISOCheck ISO check estimators Structure(s):
IBackgroundEstimator, IBlackWhiteEstimator, IEyeBrowEstimator, IFishEyeEstimator, INaturalLightEstimator, IPortraitStyleEstimator, IDynamicRangeEstimator, IRedEyeEstimator

Plan(s):
background_v2_<device>.plan, black_white_and_ir_v1_<device>.plan, eyebrow_estimation_v2_<device>.plan, fisheye_v2_<device>.plan, natural_light_<device>.plan, portrait_style_v3_<device>.plan, red_eye_v1_<device>.plan
ReIdDescriptor Work with human descriptors (ReID) Structure(s):
IDescriptor, IDescriptorBatch, IDescriptorExtractor, IDescriptorMatcher, ICrowdEstimator, IHumanFaceDetector, IHumanAttributeEstimator

Plan(s):
crowd_<version>_<device>.plan, human_attributes_v2_<device>.plan
VehicleEngine VehicleEngine Structure(s):
IVehicleEngine

Plan(s):
For detailed information about plans see VehicleEngine documentation

The table above includes separately licensed functionality and their general features.

Additional features required for the functionality utilization are shown in the columns to the right.


Luna SDK general structures

SDK CORE facilities

Required structures
General FaceEngine interface IFaceEngine
Serialization interface ISerializableObject
Archive storage interface IArchive
Reference counted interface IRefCounted
Settings provider ISettingsProvider