Skip to content

Working with TrackEngine#

TrackEngine is based on face detection and analysis methods provided by FaceEngine library. This document does not cover FaceEngine usage in detail, for more information please see FaceEngine_Handbook.pdf.

To create a TrackEngine instance use the following global factory functions

  • ITrackEngine tsdk::createTrackEngine(fsdk::IFaceEngine engine, const char* configPath)

    • engine - pointer to FaceEngine instance (should be already initialized)
    • configPath - path to TrackEngine config file
    • return value - pointer to ITrackEngine

  • ITrackEngine tsdk::createTrackEngine(fsdk::IFaceEngine engine, const fsdk::ISettingsProviderPtr& provider)

    • engine - pointer to FaceEngine instance (should be already initialized)
    • provider - settings provider with TrackEngine configuration
    • return value - pointer to ITrackEngine

It is not recommended to create multiple TrackEngine instances in one application.

The main interface to TrackEngine is Stream - an entity to which you submit video frames. To create a stream use the following TrackEngine method

  • IStream* ITrackEngine::createStream()

    • return value - pointer to IStream

You can create multiple streams at once if required (in cases when you would like to track faces from multiple cameras). In each stream the engine detects faces and builds their tracks. Each face track has its own unique identifier. It is therefore possible to group face images belonging to the same person with their track ids. Please note, tracks may break from time to time either due to people leaving the visible area or due to the challenging detection conditions (poor image quality, occlusions, extreme head poses, etc).

The frames are submitted on a one by one basis and each frame has its own unique id.

TrackEngine emits various events to inform you what is happening. The events occur on a per-stream basis.

You can set up an observer to receive and react to events.

Stream observer interfaces:

  • BestShotObserver

  • VisualObserver

  • DebugObserver

By implementing one or several observer interfaces it is possible to define custom processing logic in your application.

BestShotPredicate type defines recognition suitability criteria for face detections. By implementing a custom predicate one may alter the best shot selection logic and, therefore, specify which images will make it to the recognition phase.

BestShotObserver#

  • void bestShot(const tsdk::DetectionDescr& descr) called for each emerged best shot. It provides information on a best shot, including frame number, detection coordinates, cropped still image, and other data (see `DetectionDescr structure definition below for details.) Default implementation does nothing.

    • descr - best shot detection description
struct TRACK_ENGINE_API DetectionDescr {
    //! Source image
    fsdk::Image image;

    //! Face landmarks
    fsdk::Landmarks5 landmarks;

    //! Detection
    fsdk::Detection detection;

    //! Index of the frame
    tsdk::FrameId frameIndex;

    //! Index of the track
    tsdk::TrackId trackId;
};

  • void trackEnd(const tsdk::TrackId& trackId) tells that the track with trackId has ended and no more best shots should be expected from it. Default implementation does nothing.

    • trackId - id of the track

VisualObserver#

  • void visual(const tsdk::FrameId &frameId, const fsdk::Image &image, const tsdk::TrackInfo * trackInfo, const int nTrack) allows to visualize current stream state. It is intended mainly for debugging purposes. The function must be overloaded.

    • frameId - current frame id
    • image - frame image
    • trackInfo - array of currently active tracks
struct TRACK_ENGINE_API TrackInfo {
    //! Face landmarks
    fsdk::Landmarks5 landmarks;

    //! Last detection for track
    fsdk::Rect rect;

    //! Id of track
    TrackId trackId;

    //! Score for last detection in track
    float lastDetectionScore;

    //! Is it detected or tracked bounding box
    bool isDetector;
};
  • nTrack - number of tracks

DebugObserver#

DebugObserver#

  • void debugDetection(const tsdk::DebugDetectionCallbackData& descr) detector debug callback. Default implementation does nothing.
    • descr - detection debugging description
struct TRACK_ENGINE_API DetectionDebugInfo {
    //! Detection description
    DetectionDescr descr;

    struct TRACK_ENGINE_API DebugDetectionCallbackData {
        //! Detection description
        DetectionDescr descr;

        //! Is it detected or tracked bounding box
        bool isDetector;

    //! Filtered by user bestShotPredicate or not.
    bool isFiltered;

    //! Best detection for current moment or not
    //! value that checkBestShot returned
    bool isBestDetection;
};

  • void debugForegroundSubtraction(const tsdk::FrameId& frameId, const fsdk::Image& firstMask, const fsdk::Image& secondMask, fsdk::Rect * regions, int nRegions) background subtraction debug callback. Default implementation does nothing.

    • frameId - frame id of foreground
    • firstMask - result of background subtraction operation
    • secondMask - result of background subtraction operation after procedures of erosion and dilation
    • regions - regions obtained after background subtraction operation
    • nRegions - number of returned regions

BestShotPredicate#

  • bool checkBestShot(const tsdk::DetectionDescr& descr) Predicate for best shot detection. This is the place to perform any required quality checks (by means of, e.g. FaceEngines Estimators). This function must be overloaded.

    • descr - detection description
    • return value - true, if descr has passed the check, false otherwise

Threading#

TrackEngine is multi-threaded. The number of threads is configurable and depends on the currently bound FaceEngine settings.

TrackEngine calls Observer functions in separate threads. The checkBestShot function is called in the main frame processing thread. It is recommended to avoid expensive computations in checkBestShot.

Back to top