Feature name and |
Required algorithms and used structures |
Best Shot |
Attributes |
Emotion |
Face Features |
Liveness |
Face |
Human |
ISO |
DeepFake |
|---|---|---|---|---|---|---|---|---|---|---|
| Face detection NN version 2 First, second and third NN should be launched consequently |
Structure(s): IDetector Plan(s): FaceDet_v2_first_<device>.plan FaceDet_v2_second_<device>.plan FaceDet_v2_third_<device>.plan |
|||||||||
| Face detection NN version 3 | Structure(s): IDetector Plan(s): FaceDet_v3_<version>_<device>.plan |
|||||||||
| Face redetection NN version 3 | Structure(s): IDetector Plan(s): FaceDet_v3_redetect_<version>_<device>.plan |
|||||||||
| Image quality estimation (light, dark, blur, etc.) |
Structure(s): IQualityEstimator Plan(s): model_subjective_quality_<version>_<device>.plan |
|||||||||
| BestShotQuality estimation Performs head pose and AGS estimations |
Structure(s): IBestShotQualityEstimator Plan(s): ags_v3_<device>.plan headpose_v3_<device>.plan |
|||||||||
| Background estimation Performs background estimation |
Structure(s): IBackgroundEstimator Plan(s): background_v2_<device>.plan |
|||||||||
| EyeBrow estimation Performs eyebrow estimation |
Structure(s): IEyeBrowEstimator Plan(s): eyebrow_estimation_v2_<device>.plan |
|||||||||
| FacialHair estimation Performs facial hair estimation |
Structure(s): IFacialHairEstimator Plan(s): face_hair_v2_<device>.plan |
|||||||||
| FishEye estimation Detects a fish eye effect on the source face image |
Structure(s): IFishEyeEstimator Plan(s): fisheye_v2_<device>.plan |
|||||||||
| HeadWear estimation Detects a headwear status and headwear type |
Structure(s): IHeadWearEstimator Plan(s): head_wear_v2_<device>.plan |
|||||||||
| HumanAttributes estimation Performs human attributes estimation |
Structure(s): IHumanAttributeEstimator, HumanAttributeResult Plan(s): human_attributes_v2_<device>.plan |
|||||||||
| NaturalLight estimation Performs natural light estimation |
Structure(s): INaturalLightEstimator Plan(s): natural_light_<device>.plan |
|||||||||
| PortraitStyle estimation Performs portrait style estimation |
Structure(s): IPortraitStyleEstimator Plan(s): portrait_style_v3_<device>.plan |
|||||||||
| RedEye estimation Performs red eye estimation |
Structure(s): IRedEyeEstimator Plan(s): red_eye_v1_<device>.plan |
|||||||||
| DynamicRange estimation Performs dynamic range estimation |
Structure(s): IDynamicRangeEstimator Plan(s): No additional plans are required |
|||||||||
| Personal Protection Equipment (PPE) estimation Performs personal protection equipment estimation |
Structure(s): IPPEEstimator Plan(s): ppe_estimation_v1_<device>.plan |
|||||||||
| Face image warping | Structure(s): IWarperStruct Plan(s): No additional plans are required |
|||||||||
| Head pose estimation This NN is still used for Atlas NPU only |
Structure(s): IHeadPoseEstimator Plan(s): headpose_v3_<device>.plan |
|||||||||
| AGS estimation This NN is still used for Atlas NPU only |
Structure(s): IAGSEstimator Plan(s): ags_v3_<device>.plan |
|||||||||
| Extraction (NN version 56 backend) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn56b_<device>.plan |
|||||||||
| Extraction (NN version 56 mobilenet) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn56m_<device>.plan |
|||||||||
| Extraction (NN version 57 backend) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn57b_<device>.plan |
|||||||||
| Extraction (NN version 58 backend) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn58b_<device>.plan |
|||||||||
| Extraction (NN version 59 backend) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn62_<device>.plan |
|||||||||
| Extraction (NN version 59 mobilenet) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn59m_<device>.plan |
|||||||||
| Extraction (NN version 60 backend) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor Plan(s): cnn60b_<device>.plan |
|||||||||
| Matching | Structure(s): IDescriptorMatcher Plan(s): No additional plans are required |
|||||||||
| Basic attributes estimation (age, gender) |
Structure(s): IAttributeEstimator Plan(s): attributes_estimation_<version>_<device>.plan |
|||||||||
| Credibility check estimation | Structure(s): ICredibilityCheckEstimator Plan(s): credibility_Check_<device>.plan |
|||||||||
| Color/Monochrome Estimation | Structure(s): IBlackWhiteEstimator Plan(s): black_white_and_ir_v1_<device>.plan |
|||||||||
| Emotions estimation | Structure(s): IEmotionsEstimator Plan(s): emotion_recognition_<version>_<device>.plan |
|||||||||
| Mouth estimation | Structure(s): IMouthEstimator Plan(s): mouth_estimation_v4_<device>.plan |
|||||||||
| FaceOcclusion estimation | Structure(s): IFaceOcclusionEstimator Plan(s): face_occlusion_v1_<device>.plan |
|||||||||
| Glasses estimation | Structure(s): IGlassesEstimator Plan(s): glasses_estimation_v2_<device>.plan |
|||||||||
| Eyes state estimation (eyes status and keypoints for irises and eyelids) |
Structure(s): IEyeEstimator Plan(s): eyes_estimation_flwr8_<device>.plan |
|||||||||
| Eyes state estimation using IR cameras (eyes status and keypoints for irises and eyelids) |
Structure(s): IEyeEstimator Plan(s): eyes_estimation_ir_<device>.plan |
|||||||||
| Eyes state (lightened version, returns eyes state only) |
Structure(s): IEyeEstimator Plan(s): eye_status_estimation_<device>.plan |
|||||||||
| Gaze direction estimation | Structure(s): IGazeEstimator Plan(s): gaze_<version>_<device>.plan |
|||||||||
| Gaze direction estimation using IR cameras |
Structure(s): IGazeEstimator Plan(s): gaze_ir_<version>_<device>.plan |
|||||||||
| Face overlap | Structure(s): IOverlapEstimator Plan(s): overlap_estimation_v1_<device>.plan |
|||||||||
| OrientationEstimator Orientation estimator for images rotation |
Structure(s): IOrientationEstimator Plan(s): orientation_v2_<device>.plan |
|||||||||
| Precise 68 keypoints estimation (Precise version is available in Server distributions only!) |
Structure(s): Landmarks Plan(s): LNet_precise_<version>_<device>.plan |
|||||||||
| Precise 68 keypoints estimation for IR cameras (Precise version is available in Server distributions only!) |
Structure(s): Landmarks Plan(s): LNet_ir_precise_<version>_<device>.plan |
|||||||||
| Fast 68 keypoints estimation (fast version is available in Android Embedded edition only!) |
Structure(s): Landmarks Plan(s): LNet_fast_<version>_<device>.plan |
|||||||||
| Fast 68 keypoints estimation for IR cameras (fast version is available in Android Embedded edition only!) |
Structure(s): Landmarks Plan(s): LNet_ir_fast_<version>_<device>.plan |
|||||||||
| 5 keypoints estimation | Structure(s): Landmarks Plan(s): slnet_<version>_<device>.plan |
|||||||||
| Depth Liveness | Structure(s): ILivenessDepthEstimator Plan(s): depth_estimation_v2_1_<device>.plan |
|||||||||
| IR Liveness universal | Structure(s): ILivenessIREstimator Plan(s): ir_liveness_universal_<device>.plan |
|||||||||
| Flying faces Liveness | Structure(s): ILivenessFlyingFacesEstimator Plan(s): flying_faces_liveness_v4_<device>.plan |
|||||||||
| LivenessRGBM (models are launched consequently) |
Structure(s): ILivenessRGBMEstimator Plan(s): rgbm_liveness_<device>.plan rgbm_liveness_pp_hand_frg_<device>.plan |
|||||||||
| LivenessDepth | Structure(s): IDepthLivenessEstimator Plan(s): depth_liveness_v2_<device>.plan |
|||||||||
| Liveness OneShot Predicts whether the person's face is real or fake (photo, printed image) |
Structure(s): ILivenessOneShotRGBEstimator Plan(s): oneshot_rgb_liveness_<version>_model_<number>_<device>.plan v9 plans are bundled by default, while v8 plans are bundled in a separate archive |
|||||||||
| NIRLiveness | Structure(s): INIRLivenessEstimator Plan(s): nir_liveness_v2_model_<number>_<device>.plan |
|||||||||
| Human body image warping | Structure(s): IHumanWarper Plan(s): no additional plans required |
|||||||||
| Human body detection NN | Structure(s): IHumanDetector, IHumanDetectionBatch Plan(s): human_<version>_<device>.plan |
|||||||||
| Human body redetection | Structure(s): IHumanDetector Plan(s): human_redetect_<device>.plan |
|||||||||
| Human's head detection | Structure(s): IHeadDetector, IDetectionBatch Plan(s): HeadDet_<version>_<device>.plan |
|||||||||
| Human body extraction | Structure(s): IDescriptor Plan(s): reid_<reid_type>_<version>_<device>.plan |
|||||||||
| Crowd estimation | Structure(s): ICrowdEstimator, ICrowdEstimation Plan(s): crowd_<version>_<device>.plan |
|||||||||
| HumanFace detection | Structure(s): IHumanFaceDetector, IHumanFaceBatch Plan(s): human_face_<version>_<device>.plan |
|||||||||
| DeepFakeEstimator | Structure(s): IDeepFakeEstimator Plan(s): deepfake_estimation_<version>_model_1_<device>.plan deepfake_estimation_<version>_model_2_<device>.plan |
Example of the above table usage: if you want to find the best shots in your application, look at the "Best Shot Selection" column and see what network models and structures are required for running such functionality. The required models and structures are marked with green. Algorithms and structures marked with red are not used for the functionality.
The <version> placeholder shows that there can be several versions of the NN in a distribution.
The <device> placeholder shows that the model can be used with different devices (e. g., CPU, GPU). The name of the model includes postfix (e. g., *_cpu.plan, *_gpu.plan). See the "Hardware requirements" section in "FaceEngine Handbook" for information about devices supported by a NN.
To decide what type of detector model to use in your specific use case (FaceDet_* networks), please refer to "FaceEngine Handbook" chapter "Face Detection Facility".
You should select one of the extraction CNNs listed in this table. See "FaceEngine Handbook" "Appendix A. Specification" for information about their performance and chapter "Descriptor Processing Facility" for information about CNNs.
You don't have to use both ordinary and mobilenet version for descriptor (cnn* networks), it's enough to choose one of these models. For details on difference, see "FaceEngine Handbook" chapter "Descriptor Processing Facility".
"No additional plans are required" means that the feature is implemented without a NN usage.
Functionality |
Feature name and |
Required algorithms and used structures |
Face detection NN |
Face detection NN |
|---|---|---|---|---|
| MedicalMaskDetection | Medical mask estimation | Structure(s): IMedicalMaskEstimator Plan(s): mask_clf_<version>_<device>.plan |
||
| PPEDetection | Personal Protection Equipment estimation | Structure(s): IPPEEstimator Plan(s): ppe_estimation_<version>_<device>.plan |
||
| MobileLiveness | LivenessOneShotRGBE | Structure(s): ILivenessOneShotRGBEstimator Plan(s): liveness_model_<version>_<device>.plan |
||
| TrackEngine | TrackEngine | Structure(s): ITrackEngine Plan(s): vlTracker_detection_<device>.plan vlTracker_template_<device>.plan vlTracker_update_<device>.plan |
||
| LivenessEngine | LivenessEngine | Structure(s): ILivenessEngine Plan(s): No additional plans are required |
||
| DescriptorIndex | Index creation | Structure(s): IIndexBuilder, IDenseIndex, IDynamicIndex Plan(s): No additional plans are required |
||
| ImageModification | Image modification estimation | Structure(s): IImageModificationEstimator Plan(s): image_modification_<version>_<device>.plan |
||
| ISOCheck | ISO check estimators | Structure(s): IBackgroundEstimator, IBlackWhiteEstimator, IEyeBrowEstimator, IFishEyeEstimator, INaturalLightEstimator, IPortraitStyleEstimator, IDynamicRangeEstimator, IRedEyeEstimator Plan(s): background_v2_<device>.plan, black_white_and_ir_v1_<device>.plan, eyebrow_estimation_v2_<device>.plan, fisheye_v2_<device>.plan, natural_light_<device>.plan, portrait_style_v3_<device>.plan, red_eye_v1_<device>.plan |
||
| ReIdDescriptor | Work with human descriptors (ReID) | Structure(s): IDescriptor, IDescriptorBatch, IDescriptorExtractor, IDescriptorMatcher, ICrowdEstimator, IHumanFaceDetector, IHumanAttributeEstimator Plan(s): crowd_<version>_<device>.plan, human_attributes_v2_<device>.plan |
||
| FightsEstimation | Fights estimation | Structure(s): IFightsEstimator Plan(s): fights_<version>_<device>.plan |
||
| DepthRGBLiveness | Work with LivenessDepthRGBEstimator | Structure(s): ILivenessDepthRGBEstimator Plan(s): depth_rgb_<version>_<device>.plan |
||
| DepthLiveness | Work with DepthLivenessEstimator | Structure(s): IDepthLivenessEstimator Plan(s): depth_liveness_v2_<device>.plan |
||
| ImageModification | Work with ImageModificationEstimator | Structure(s): IImageModificationEstimator Plan(s): image_modification_v1_<device>.plan |
||
| VehicleEngine | VehicleEngine | Structure(s): IVehicleEngine Plan(s): For detailed information about plans see VehicleEngine documentation |
The table above includes separately licensed functionality and their general features.
Additional features required for the functionality utilization are shown in the columns to the right.
SDK CORE facilities |
Required structures |
|---|---|
| General FaceEngine interface | IFaceEngine |
| Serialization interface | ISerializableObject |
| Archive storage interface | IArchive |
| Reference counted interface | IRefCounted |
| Settings provider | ISettingsProvider |